Scripting News

Photos of downtown Oakland

Fri, 30 May 2008 01:48:40 GMT

Here's a set of photos I took yesterday in Oakland.

Overlooked in the McClellan coverage

Thu, 29 May 2008 13:26:06 GMT

The coverage of the McClellan tell-all book has focused on the White House spin, which amid all the bluster about surprise and how this isn't the Scott they all knew (come on, why should voters care that you're surprised), they aren't really contesting the assertions, or if they are, they're doing it weakly.

Probably some of them want to have jobs in the future, and lying right now wouldn't help them in the careers. Further I think almost everyone who has been paying attention knows that what McClellan says is true. Why didn't he speak out earlier? Why didn't a lot of people? Also consider the possibility that other people in the White House got scooped, the ones trashing McClellan and are jealous that his tell-all book got out before theirs, and others are likely to be tried and perhaps go to jail for their actions. In other words, they all have axes to grind here.

The other point being overlooked, and this is a real problem, is that he says that the press was complicit. This is the more important allegation, and unsurprisingly, it's being swept aside by the press. Had they done their job, and pressed for the truth, it would have been easier for insiders to tell them the truth. But corporate-owned media isn't interested in helping us make decisions as a country, they're only interested in ad revenue. That's why it's so important that we're creating new media that isn't so conflicted, and why the question of whether bloggers run ads or not is far from a trivial issue.

In court, if you have a conflict of interest, you're supposed to disclose it, and if it's serious enough, it disqualifies you. I've recommended many times that professional news media should have relationships with less conflicted bloggers for circumstances like this, so when they become the story, the public can have a discussion about them using the channels they own. They don't have much of a choice here, because the channels are going to develop with them or without them. We could all save a bunch of time if they didn't fight it, and welcomed amateurs into their midst.

Jury Duty, day 2

Thu, 29 May 2008 12:53:32 GMT

I have another day of jury duty. I haven't been selected, but the voir dire is going into a second day because we started so late yesterday. I am not allowed to talk about the specifics of the case as long as I might potentially be a juror, however I can say that I think it's not likely that I will be selected. And I can say is that it is, again, an inspiring process. A few comments follow.

Until I moved to the East Bay in 2006, I had never ridden on BART. It doesn't go down the peninsula to where I used to live, but it's a fixture of life on this side of the bay and in San Francisco. I like riding BART not only because it's usually faster than driving, but also because I get to see my fellow citizens without their cars. It's fascinating to see who my neighbors are. In NY and Boston, where public transit is much more a part of daily life, you get that experience all the time. Not so much on the west coast.

Well, jury duty is like riding BART, only more so. It's as if you were riding on BART, but each passenger, in turn, tells you what they do for a living, who they live with, where they came from, in some cases why they can't serve (always dramatic). What the judge says about jury duty is true, it's what makes America work. There are no professional jurors, just BART people. You just have to be registered to vote or have a driver's license for them to find you.

Which leads to a curious inexplicable fact. First, judging from the jury, the Bay Area is a remarkably diverse place. But if you went by the jury alone, you'd conclude that there are no blacks in Alameda County. The only black person of the 100 or so potential jurors was an African immigrant. However, if you go outside the courthouse in downtown Oakland, or ride BART, you'd see that there are lots of blacks. What happened? Maybe they don't vote or drive? I honestly don't know. It seems very improbable that a random drawing would be so skewed.

I was riveted listening to the stories people told. It was fascinating. I also had opinions of the lawyers, and the parties. But I can't talk about that yet. What I can say and will is that I was struck by the nobility of my fellow citizens. When asked to serve, they all rise to the occasion. People who doubt that there is cause for hope should sign up for jury duty and go through the process. It is something to behold.

From the Mind of Hugh

Thu, 29 May 2008 01:26:49 GMT

An open source Twitter client?

Wed, 28 May 2008 19:19:49 GMT

There have been a lot of back-channel conversations about the future of Twitter, and one subject that keeps coming up (I keep bringing it up actually) is the need for a Twitter client that isn't aligned with any commercial interest. One that, by design, is meant to serve users and users only. I think it's time to start a public discussion about this, not sure if there's interest among developers -- this isn't something I'd write myself. The only way to find out is to ask.

Jury duty

Wed, 28 May 2008 16:34:47 GMT

I got called for jury duty in Oakland.

They have wifi here but it's $5.95 per day.

Can you imagine, here we are doing our civic duty, taxpayers all of us, and they want us to pay to get online. It's a crime I tell you! Put me on a jury to decide this and I'll vote to convict.

I served on a jury in 1996 and wrote up the experience.

Update at 10AM: They called the first group, I was not in it. Is this good news or bad? (I'd rather not serve this time.)

Update at 10:30AM: I'm in the second group. Here we go. But we don't have to be here until 1PM. So now what? Oy.

Update at 10:45AM: All the group 3 people were excused. Envy.

Spewage update

Tue, 27 May 2008 19:19:55 GMT

Update: JR suggests adding a column to the Twitter spew report that shows the percent of total spew each person is responsible for. Good idea! Done.

Should Twitter charge high-spew users?

Mon, 26 May 2008 20:05:24 GMT

Om Malik asks if Twitter should charge users like Scoble who have huge numbers of followers.

It's a fair question because these users are super-expensive for Twitter, much more so than users with modest numbers of followers.

To get an idea, I have a little agent script that counts and ranks people I have followed in the recent past to get a rough idea of how much work they generate for Twitter's system software.

http://twitter.scripting.com/spewage.html

You can see that Scoble tops the list with a "spew factor" of 308,359,436. I'm #5 with a spew of 77,174,172.

Imho, they shouldn't charge these people because they're feeding the growth of Twitter. If you charge them a competitor will come along and might actually pay them to use their system because it will attract so many other users.

Right now Twitter doesn't need more money. They need a design that works, and an implementation of that design. They have lots of money and can get lots more.

How to do data portability

Mon, 26 May 2008 15:03:50 GMT

I've heard a lot about data portability conferences and workshops, I've even been criticized for not going to one which happened on the west coast while I was in the east earlier this month. I don't plan to go to any of them, I don't see what's accomplished by having public meetings about this stuff. People who control users' data can accomplish a lot more by finding ways to give them the power to use it more effectively. Talking about principles of data portability only achieves talk. It gives people a sense of propriety over talking, not data, and people giving up propriety over talking are just yielding the floor, not yielding any power over users.

The best way to achieve data portability is to just do it.

I know that sounds silly, or obvious, but there is so much pretending that there's more to it, that it has to be said.

If you want to accomplish something by talking, call up a friend who works at Netflix or Yahoo and ask them if they'll let users move around their movie rating data. I've been asking about this for years. No one's email addresses are involved. All I want is the power to give Netflix permission to read an XML file on yahoo.com that contains my movie rating data (assuming Yahoo goes first). Anyone can see how much power this would give Yahoo. Why don't they do it? I honestly don't know. If I were them, I would.

Another example -- if Twitter wanted to buy itself some time and growth, and give developers something exciting to do, they would store as much user profile data as they can off twitter.com servers and on Amazon. Simple XML formats, use some of their ability to raise investment capital (which they have proven) to grow the human network while they patch up or rewrite their system software. The more data they can move off their outage-prone systems, the more the network can grow around them, but not dependent on them. Amazon has proven they can keep their servers running. Leverage that.

The discussion about data portability so far has fixed on the hardest most vexing technical, privacy and economic issues, the ones that probably don't have a resolution. My advice is to instead pick a few relatively easy data portability problems and solve them. Flying around the world to go to conferences to talk about the hardest problems won't actually achieve any data portability.

Update: Brad Feld argues for APIs. A few months ago I would have agreed, but today I don't think an API is enough. As we've seen with Twitter, when the service goes down, there is no API and there is 100 percent lock-in. We need more. The most vital data must be stored off-site, so it doesn't go away when the service goes down.

The 16-year rewrite

Mon, 26 May 2008 14:57:46 GMT

In February 1992, I started work a piece of Frontier called the scheduler. It's the equivalent of what they call "cron" in Unix-Land. You can put scripts in four different places: 1. everyMinute scripts, 2. hourly scripts, 3. overnight scripts and 4. threads. It was a simple bit of code that's been running now for 16 years, on every copy of Frontier, Radio, or the OPML Editor.

It was built on the foundation for background processes that existed in 1992. A few years later a better foundation was built, but the scheduler was never adapted to run on that.

It's always had a certain flakiness, and I never had the patience to track it down. It's old code, written before I learned a lot of things about the Frontier environment, what works and what doesn't. I just lived with the flakiness.

Yesterday I got tired of it, and I did what programmers like to do, I rewrote it. It took a few hours, but the new version is *much* cleaner, and already runs much more reliably.

Proving the point that sometimes code rewrites are the way to go.

I've released the new part to OPML Editor users. There's no code that uses it yet, but there will be soon.

Rowan & Martin

Sun, 25 May 2008 05:52:06 GMT

Her political career is over

Sat, 24 May 2008 00:01:07 GMT

This time Hillary Clinton has gone too far.

Strike 1. Positioning Barack Obama as the black candidate after the South Carolina primary.

Strike 2. Refusing to say Obama is Christian, not Muslim.

Strike 3. Saying white voters go with her, in desperation, after the Indiana and North Carolina primaries.

Strike 4. Suggesting she should stay in the race because Bobby Kennedy was assassinated in June in 1968.

Too much benefit of the doubt. Bringing assassination into political discourse as an "issue" is too much.

Hillary Clinton is a sophisticated, calculating politician. She doesn't make accidental statements, and has in no way earned the benefit of the doubt. Her explanation is a non-explanation. Her retraction was not a retraction, and she was nowhere in the neighborhood of an apology

Screwing around with life and death is over the top.

Earlier today I posted on Twitter that it was inevitable that she would be on the ticket with Obama. Once again I forgot how awful this person is. It won't happen again.

Update: MP3 of Keith Olbermann's special comment on Countdown tonight.

Monitoring Twitter's down-ness

Sat, 24 May 2008 01:06:24 GMT

I've been thinking about writing an app that would check Twitter every ten minutes to see if it's down or up, and track it over time. Today, being a particularly bad day for Twitter, I had a bit of time to actually write the app.

Screen shot.

When it's been running for a few days I'll put up a report page. Hopefully it'll be up more than it's down.

I twitted it on FriendFeed.

Update: A site that monitored Twitter uptime in 2007.

FriendFeed gets rooms, so does Scripting News

Fri, 23 May 2008 00:57:46 GMT

An interesting but mysterious new feature appeared today in FriendFeed, rooms, and of course we have to try it out.

I created a room for Scripting News. What shall we use it for? I have no clue. Help me figure it out.

Something just jumped out at me. Is this the Digg-in-a-Box that I was asking for many months ago?

These guys always confuse me, in a fun way.

Twitter begins to communicate with their users

Thu, 22 May 2008 23:59:39 GMT

Today Twitter began to communicate with their users, which they are to be commended and congratulated for. A journey of a thousand miles begins with a single step. There's a long road ahead, and everyone's going to learn a lot as we travel down it, and no doubt it's awkard at first. Now we can look back at the first step and think how much we accomplished, and look forward to the second.

There were two posts, one from Jack Dorsey, the CEO and another from Alex Payne, an engineer at "Twitter HQ." Payne, by pointing to a piece that dismisses decentralization and says we don't understand Twitter, thereby shows (sorry to say) that he doesn't understand it. It's like the elephant being described by a million blind men. Each of us sees something different and thinks, incorrectly, that we have the whole picture. In fact, none of us do.

It's easy to prove that Twitter is different for everyone. I'll start with myself. I have 9644 followers on my main account, and I follow 663 people and have updated 7870 times.

I can't keep up all the people I follow. Back when I only followed 100 people, Twitter made it possible for me to learn about the people I followed. It was remarkable. Today I never make it through all 10 pages of history, the rope slips through my hands. This wonderful feature of Twitter, that I reveled in last year this time, is now lost to me. (And some people follow over 20,000. Can you imagine they have any idea what all those people are doing?)

So even with just that one account I already am three or four of the blind men who think they grok Twitter, and each of them sees something different. I could write a 5000 word article for each of them, the experiences would be very different.

Dennis Metzcher lives in New Jersey and describes himself as an "iPhone-toting super-hero." He's updated 216 times, has 36 followers and follows 48. His posts are thoughtful, and he seems to communicate with lots of A-twisters but also quite a few people with a relatively small number of followers. When he goes to Twitter it probably doesn't look like a firehose. I'm sure he sees it very differently from me.

Alex Payne -- the Twitter engineer -- follows 319 people, has 3181 followers and has updated 3430 times.

Cliff Gerrish is a website builder in SF, follows 350 people, is followed by 309 people and has updated 1650 times.

I can't explain how each of these people sees Twitter, but I'm absolutely sure we all see it differently.

To me, Twitter is a publishing medium. I wish it weren't so, I wish I had the bandwidth to really follow 663 people. I have no idea how to thin out the ranks of the people I'm following, and I don't plan to. But to the extent that the company is having trouble scaling it for me, I think they should stop worrying about it. Further, and this is important, most of the things I post are not especially time-sensitive. If it took 1/2 hour to deliver each one to each of my 9644 followers it wouldn't be the end of the world. However, a week ago I had a news scoop, the first report of an earthquake in Falls Church, VA. That was an example of a high priority message.

I have a requirement that many other Twitter users might not have -- I have to stay on the air when Twitter goes down, and this, if you think about it, makes decentralization not an option but an absolute necessity, no matter what the architecture guys may think. I must have a Plan B, because I intend to build a business that depends on this service, or something like it. I want to start that business in the next few weeks (actually I started it quite some time ago, but it's about to evolve).

It won't do any good for any of us blind men to dismiss anyone else's point of view. That's what I would like to get through to the people who run Twitter, who work there, their investors. To me, nothing is so frustrating as people who think they understand better than others when they don't. Please, don't fall into that trap. Remember we're all blind men, you too -- keep in touch with your humility.

I look forward to a long, interesting conversation about this stuff!

PS: A discussion has started on FriendFeed under a post by Fred Wilson.

HRC's historic opportunity

Wed, 21 May 2008 17:49:42 GMT

Hat's off to David Gergen, who until now I thought was just another empty suit talking head. Turns out he's got some guts.

On CNN on Tuesday night he called on Hillary Clinton to reject racist votes. It would cost her nothing now that the racists have finished voting. She complains of sexism at the same time she's been openly encouraging people to vote for her because she's white.

Gergen says, righteously, that Hillary could say: "If you want to vote against him because he's black, I don't want your vote." Amen brother! (And let's go Hillary.)

Thanks to Culturekitchen for the clue.

Podcasting and RSS at Berkman

Wed, 21 May 2008 16:36:48 GMT

I mentioned here my surprise and pleasure that, at the Berkman\@10 conference, Harvard Law prof Terry Fisher claimed, in his opening remarks, that Berkman played a pioneering role in podcasting. It's very true. And it happened in a number of ways.

1. The first meeting of people interested in the technology took place at the Oct 2003 BloggerCon, hosted at Harvard. You'll see this meeting mentioned prominently in every history of podcasting. A lot of the people working in this area were there and freely exchanged ideas, techniques and enthusiasm. (The Day 2 grid for the first BloggerCon.)

2. It was at Berkman, with the help of Bob Doyle and the talent of my fellow Berkmanite Chris Lydon and the support of John Palfrey and the rest of the Berkman team that we did the first podcast program, a series of interviews of early bloggers, technology leaders and people making news. We distributed these through Chris's blog, and also through, for the first time, an RSS 2.0 feed with enclosures.

3. It was also at Berkman, in June 2004 and through the Democratic Convention in Boston that summer that I started Morning Coffee Notes, my own podcast, that broke new ground. It seems that my amateurish but very enthusiastic (and imho creative) efforts served to inspire many others. Where people heard my rough podcasts many thought "Hey I could do that too."

Nothing like podcasting ever takes off like people say things like that do, it's never a big bang, or the "build a better mousetrap" myth. It's always iterative, trial and error. You needed Chuck Berry and many others before the British Invasion could happen and then the Beatles. We're probably still in the early days of the art of podcasting, but there's no doubt that Berkman played a big role in incubating and nuturing the initial seed.

John Palfrey gave an interview to Harvard Law Today where he summed up the story with remarkable economy. If you're interested in the area, the PDF scan is worth a read.

John also talks about the invaluable role Berkman played in stabilizing the RSS 2.0 standard.

PS: A list of topics discussed at all four BloggerCons. I keep looking for this list.

Furious digging happening

Wed, 21 May 2008 14:46:53 GMT

Sorry for the lack of updates here the last few days.

I'm working on a new project I find very interesting. Not sure if it's going to pan out, but I'm having a blast working on it.

Stay tuned.

nytimesriver.com back

Mon, 19 May 2008 23:47:37 GMT

While I was traveling the machine that was generating nytimesriver.com overheated and stopped running the app.

This morning the first thing I did was get it running again, and made a few tweaks and small performance improvements.

http://nytimesriver.com/

Thanks everyone for being so patient.

Nothing from nothing leaves nothing

Mon, 19 May 2008 16:44:27 GMT

Mark Evans asks a provocative question -- are pro bloggers going to be extinct soon?

If you were to ask a pro blogger this question, they would say of course not.

Now if you ask me -- there never was such a thing as a pro blogger. It's a contradiction in terms. It's like calling someone a professional amateur. It's like salty orange juice, a drink whose taste is derived from its acidity. Blogging is an amateur activity. It's users writing about what they do, not professionals writing about what users do.

That pros have tried to hijack the term doesn't somehow evade the old question of what happens to pros in an age where users go direct to each other. They thought they could pull a fast one "Oh we'll just steal their name" and somehow their economic model will start making sense when it didn't before.

Oh lord, you can't buy a Mercedes Benz with that.

Remember this old Doc Searlsism. We make money because we blog not from our blog. We earn because we learn from sharing our experiences with others, not because we let advertisers hitch a ride on our writing for a fee. No one pays attention to the ads, so it doesn't matter if you include them or not.

As Billy Preston once sang, nothing from nothing leaves nothing.

Hopefully that answers the question Mr. Evans asked.