Scripting News

What a 140-char message looks like in RSS

Sun, 19 Jul 2009 17:37:57 GMT

Here's a screen shot to contemplate.

There are two 140-character messages. Each illustrates features of the new shipwreck I hope to sink, to create a new coral reef for Twitter-like systems to grow on and around.

The first three items in each message are fairly obvious:

<description> holds the 140-character text.

<pubDate> is the timestamp, when the message was created.

<guid> is the identifier for the message, so a reader can tell if they've seen it before.

This makes it possible for the messages to be edited after publication, a common feature requests from writers using Twitter.

After that come optional elements.

<category> works like tags in apps like Flickr or YouTube. You specify them in a dialog, blanks separate them, you can create tags with blanks by putting them inside quotes.

<link> is used to point to web pages. No need to shorten the URLs because they don't take up space in the 140 characters.

<enclosure> is how you attach media objects to messages. Again, no need to shorten the URLs. And since the clients know the media type, they can show a preview, or embed a player.

These all use well-understood elements of RSS 2.0. Nothing new needed to be invented.

Craigslist is progress

Sun, 19 Jul 2009 20:41:10 GMT

I don't think I've ever written about Craigslist here.

Probably because I don't spend much time thinking about it, or worrying about it. But I know that some people do, for example Terry Gross, the host of NPR's Fresh Air. It comes up when people talk about the Internet destroying things that matter, like the classified ads in newspapers. At one point in an interview with Wired editor Chris Anderson she asks, in a bewildered way, what happened. She was saying it was a shame that Craigslist comes along and does what the newspapers were doing, for a fraction of the cost, employing a small fraction of the people who used to support the classified ads in newspapers.

I'm not surprised, and if you think about it, it's very predictable. It's called productivity, and it's what new technology is supposed to do. We used to employ 20 percent of the workforce in agriculture, now it's just 2 percent. That's because of technology. You may say it's bad, but there's also less hunger in the US now than there was then. And there probably are far more classified ads today, now that they're mostly free, than there were when they cost money.

It's productivity. It basically a good thing. And as long as we invest in progress it's inevitable.

Here's an MP3 of the segment quoted above.

Another test post

Sun, 19 Jul 2009 17:09:32 GMT

It's amazing how much discussion these test posts get over on FriendFeed.

I have to do them because my app needs something new to ping about to see if the apps that are subscribed to this get the updates, in real-time of course, via RSS.

A test case for RSS Clouds

Sun, 19 Jul 2009 00:23:54 GMT

I'm preparing a test of a Twitter-like service based on rssCloud, but then realized that I'm within striking distance of something simpler -- re-adding a <cloud> element to Scripting News. It had one for quite a while, we had a server that supported it when Radio 8 shipped in 2002.

So I added it in, and before I could construct a test, some aggregators started registering handlers! In other words, the old network started rebooting. If you don't understand, don't worry about it, but if you do -- man that's spooky.

Anyway, the RSS feed for scripting.com does indeed now have a <cloud> element, and if you happen to have an aggregator that knows what to do with it, please feel free to register a handler. I'll have a lot more to say about this in the coming week, knock wood, Murphy-willing.

BTW, the errors in the log are the result of apps registering receivers who are unable to receive notification because they're behind a firewall or NAT. I obviously have to unsub them when I get that error.

And that's the way it is...

Sat, 18 Jul 2009 00:29:13 GMT

NPR: Walter Cronkite, The Nation's Narrator.

NY Times: Walter Cronkite, Iconic Anchorman, Dies.

This is not an earth-shaking announcement

Fri, 17 Jul 2009 22:40:33 GMT

I'm just one guy programming away, but I have met Valentino Rossi, and when I'm very productive, as I have been this week, I feel like I'm programming the way he rides a motorcycle.

When things are clicking, programming-wise, it activates other parts of my creativity. I cook more imaginatively, and I consider moving to Italy, so I can enjoy the good life while reorganizing the world. They have Internet in Italy so it's hard to imagine how Berlusconi could interfere.

Anyway, this is not an earth-shaking announcement. I just got something working today that I imagined for quite some time, and it's nice.

The goal is to have a Small Pieces Loosely Joined equivalent of Twitter.

I know now that there are people at Google who share this vision. They have the resources to do centralization. What we have to make sure is that the Rest Of Us have the ability to route around the centralization. I hope they don't take it personally at Google, but enough with letting one company control the flow of the real-time web. There are always pundits who are willing to sell us out to the BigCo's, but I am not one of them. Never have. I remember when Google was One Of Us. Hopefully that thread still runs strong inside them now that they are a BigCo.

While everyone was debating the morality of Arrington releasing the Twitter info, I was thinking "Geez these people are focusing on the wrong stuff." The real question is how big TwitterCorp plans to get while holding the control tightly within the confines of their Corp. That can't work. It never has. I'm amazed that smart people like the people who run Twitter are willing to bet on that, still, so far into it. That they think a single company can run the Pulse of the Planet is a sign that they are drinking too much of their own Kool Aid. This can't work. Can't.

Anyway, here's what I have working.

A Twitter-like RSS feed with a single subscriber who gets notified by the cloud when the feed has updated. It then reads the feed and displays the new stuff. This all happens before I can refresh the page. It's the same speed as the connection between Twitter and FriendFeed. Now there will be people who say it can't scale. 1. They don't know. 2. They might be right. And even if it's slower than Twitter, it's worth the tradeoff. Because Twitter is going to break. Be sure of it. Nothing in the history of the Internet has ever done what they're trying to do. I don't know for sure, but I suspect it can't be done. And even if it can, it's bad.

Anyway, time for me to enjoy what's left of the day. It's gorgeous in California. I'm going to get some exercise then have my Italian dinner and then more work tomorrow! Buongiorno and arrivaderci!

BadHair at 7PM

Fri, 17 Jul 2009 00:37:37 GMT

It's been an interesting couple of days with TechCrunch first teasing and then releasing internal notes from Twitter Corp meetings.

At first the debate was over the propriety of TechCrunch releasing this information. But now that at least some of the information is out there, the discussion is turning to the information and attitudes they reflect.

Were having a Bad Hair Day podcast at the normal time, 7PM Pacific this evening and we'll be talking about the release. You can catch the show live on BlogTalkRadio or as a podcast shortly after the completion of the show at 7:45PM.

Our guest for this evening's podcast is Chris Saad, who we had originally scheduled to talk about commenting and blogging and everything related, which is just about everything in community software.

There was a fair amount of TechCrunch agenda in there. I didn't get that they're at war with RSS from reading the TwitterCorp notes, just that they don't feel all RSS content should flow through Twitter. I concur. Twitter and RSS are used for different purposes, and there's far too much new stuff in RSS for Twitter's system to handle.

After reading the TC piece about Twitter I thought: "How wrong that a company owns this medium." Always felt that, reinforced it.

Wonder what Doc Searls thinks about this. Seems we're derailed from the Cluetrain.

If you have comments, please post them here.

Rebooting the RSS cloud

Tue, 14 Jul 2009 23:23:01 GMT

In the early days of RSS, we had the idea that instantaneous updates would be the next step. That was 2001. It took a little longer than we thought, but now with "realtime" as the Next Big Thing, it's time to reboot all that lovely stuff.

http://rsscloud.org/

If it works, it'll be a bootstrap. That means at first the results will be a few sites pinging each other and updating in realtime. If it gains traction, it'll get support from a lot of tech and media companies. It could happen very quickly.

One thing's for sure -- we'll need implementations in every language and runtime. I'm doing mine in the OPML Editor, of course. But just think of that as a reference implementation. I think the really scalable versions will be in Python, PHP or C.

Also thinking about using Amazon's SimpleDB to store the graph. We'll see..

Mulberry ice cream

Tue, 14 Jul 2009 07:00:00 GMT

The editorial team of InBerkeley.com had lunch today at Saul's in North Berkeley. I had a bowl of cold borscht with sour cream, and a toasted bagel with lox and cream cheese.

After lunch we all went down the block to Chez Panisse and had a Bastille Day special mulberry ice cream cone for $2.

What, if anything, did Microsoft announce?

Tue, 14 Jul 2009 03:10:20 GMT

This morning, before we recorded the RTN podcast, there was evidence that Microsoft had announced something. Apparently they had briefed the press at some undisclosed location on some date we don't know about some products we don't understand.

Last week I gave Google a ton of grief for announcing an operating system that has been shipping for 17 years and a web browser that had been shipping for about a year as a new product with the same name as the browser that had been shipping for a year.

In the podcast we talked about a river of realtime news. The analogy fits these pseudo-events in the following way. Sometime in the past weeks Microsoft held a private event, trying to build a dam on the river, hoping to blow the dam at a predetermined time earlier today, thereby creating a rush of news that would impress everyone. It didn't work because apparently the dam developed a leak in the middle of the night and the water rushed down the river of news while everyone was sleeping. No one was impressed. Sad Microsoft.

The moral of the story: Companies probably should announce products when they are new, and when ordinary people and the savvy insiders can try it all out and share their opinions. That way if the product is any good it will generate interest. If it's not good, no one need bother get excited.

ReadWriteWeb: Microsoft Office Web Access Not Here Yet.

Marc Canter leaves California

Mon, 13 Jul 2009 23:19:45 GMT

I remember when Marc came to California in 1988. He got here just in time for the Loma Prieta quake. Two people died outside his office on Townsend. They had just been to visit Marc.

There would be no tech industry South Of Market if Marc hadn't moved his small company here from Chicago in the late 80s. He was young then, he had a purpose, he was going to turn desktop computers into movie machines. He did that.

He was hugely influential, although time has a way of paving that over. Remember that, young people, you may be important now, but there will come a day when no one knows your name. Prepare for that day.

He got rich, very rich -- but he blew through the money like a drunk rock star with an entourage, which he had.

Marc is a wild man. California has captured his wildness for a long time. Now that wildness belongs to the rest of the USA.

Good luck man -- keep blogging so we know what's up with you.

Rebooting The News #16

Mon, 13 Jul 2009 23:16:43 GMT

Recorded this morning at 9AM Pacific.

Show page here, with notes written by Jay.

More Pubsubhubbub feedback

Sat, 11 Jul 2009 16:25:09 GMT

When I travel to Europe, I wonder why they couldn't just do electric plugs the same way we do in the US. That way I wouldn't have to carry an adapter and I'd be able to plug in more than one device at a time. I wish their cell phones worked the same way ours do (I gather they do now, somewhat) and that billing worked the same (I'll let you know when the bill from my June trip arrives). When I travel to London I wish they had the good sense to drive on the correct side of the road.

Each of these inconveniences were caused by engineers thinking they didn't "have to" worry about the way things were done before. They were right, they didn't have to, and all future users paid for their insistence. Think how much better it would all have worked if they cared.

And some things are, thankfully, the same. For example -- a wifi router is the same in Europe and the US. The Euro is a way of rolling up currency incompatibilities, although some countries in Europe, Denmark, the UK and Switzerland, aren't on board. But think about all the trouble they've gone to get that compatibility. What if they had been compatible from the start?

Anyway, how does this apply to notification?

Googler DeWitt Clinton asked for Feedback on Friendfeed's proposal for notification, which is different from Google's. I'm already confused! Both of them are different from the weblogs.com method which is now almost ten years old (and deployed in every blogging app and CMS out there).

I make the same suggestion to them that I made to the IETF when they were embarking on Atom. I offered that they should start with RSS 2.0 and change whatever they felt they can't live with, and document their rationales. They didn't take my advice, so now we're in this silly situation where there are two names for everything. What RSS calls an <item>, Atom calls a <froofraw> (or whatever, I can never remember).

2003: Prior art as a design method.

So, if you're working on notification, I suggest starting with weblogs.com pinging with changes.xml as your output, and then change whatever you feel you can't live with, and document your rationales. That way what you end up with will be minimally different from what's already out there, and future implementers won't curse us for not having the sense to have one way to do things. (That's right, they'll curse all of us, they won't know or care who went first.)

Now, if forced to make a choice, I'd probably go with Pubsubhubbub for three reasons: 1. It's at least XML, even if it's not RSS. 2. They say they'll support RSS, giving a sense of being in touch with the world they live in. 3. It's Google, so they have a certain amount of sway with users and developers. However, neither of them adopts the prior art method of format design outlined above. If either of them did, I wouldn't even have to make a choice.

A more low-tech approach to ping hubs

Sat, 11 Jul 2009 03:15:09 GMT

When talking with the Google guys earlier today I told them that there was an even more low-tech approach than the <cloud> element for the kind of notification they were doing. As I was reading their spec, I decided to look into it to refresh my memory. I'm writing it up here, so everyone can compare.

1. Unlike <cloud> this protocol was very widely implemented. Support for this protocol is already baked into almost all blogging software, and (likely) many CMSes.

2. The feed indicates which hub it belongs to using a <category> element. You can see an example looking in the feed for my Radio weblog. I've made a copy of that feed in case the link goes bad (I hear that Radio weblog hosting may end in December.)

This means that if you want to find out if this feed changed, you should monitor the indicated changes.xml file.

3. When the feed updates, it pings the server that maintains that changes.xml file. The coupling here is much looser than the coupling that Google is using. But the changes.xml file can be read once a minute. If your application can handle up-to-the-minute updates instead of up-to-the-second, then this approach works fine.

A neat conference hack?

Fri, 10 Jul 2009 23:00:17 GMT

Hey I just had an idea for a conference hack you can do at a traditional audience-oriented conference.

I have a theory that you could grab any random person from the audience and put them on stage and they'd give a better talk than the usual conference speaker because they wouldn't have had time to prepare slides or get nervous and plan a speech in their head.

So, why not do exactly that!

Have a "surprise panel" mid-afternoon on the first day, around 3PM. The conference moderator takes the podium and says: "Would the following people please come up on stage." And then he'd name four people chosen at random from the audience. Then they'd have a discussion about the previous panels and speeches, the topics of the day in whatever industry or profession the conference is about.

The only problem with this idea is that by 3PM most of the people would already be out in the hallway schmoozing because the speeches and panels were so boring. Not exactly sure what to do about that.

Google's Pubsubhubbub

Fri, 10 Jul 2009 22:26:55 GMT

It's got a weird name, and I found the spec somewhat hard to understand.

But thanks to Brad Fitzpatrick and Brett Slatkin from the team at Google that implemented it, I now understand what Pubsubhubbub does.

It allows you to receive updates of RSS feeds without polling.

It makes it possible to build a distributed Twitter-like system with components that are not made by a single company, and with servers not run by a single company.

It makes instant updates possible for RSS.

It makes it possible to build a Twitter without the limitations of Twitter. (For example, no 140-character limit, the ability to handle enclosures, categories without #hashtags.)

The protocol it defines seems reasonable (I'll have to implement one side of it to be sure) and because it has the backing of Google, one of a very small number of companies with the resources to make something like this work, it has a chance of gaining traction and when it does, scaling.

In fact, it's part of one of the components I asked Google to implement in a blog post here on May 28, as Brett pointed out in our phone conversation earlier today. It's nice to see that at least a few people at Google see the possibility of assembling a Twitter-like notification system with the Small Pieces, Loosely Joined approach.

Drilling in one more level, here's how it flows.

1. Any feed that wants to participate in this network must add a bit to the feed that indicates which ping server is handling notifications on its behalf. There can be more than one.

2. When a subscribing application initially parses the feed and notices this bit, it sends a notification to each server saying "I want to be notified when this feed updates."

3. When the feed updates, it pings each of the servers it has registered with saying "I have updated."

4. The server then pings each of the subscribers saying "He updated."

The subscriber must have a known address, therefore must not be behind a firewall or NAT. For client apps, they need some kind of proxy that has a known address. This limit is signficant, but certainly not insurmountable.

I would like to see them understand RSS syntax in addition to Atom syntax, and I understand from the spec that that is forthcoming.

Update: http://superfeedr.com/ has also implemented this protocol.

Imho, the OPML Editor is not hard

Fri, 10 Jul 2009 18:04:00 GMT

I've heard people say, here and there, that the OPML Editor is too hard to use, or overkill for certain projects, but honestly -- I don't think it is. I think there may be other problems, and confusion about what it does, because it surely does a lot. But for a specific task, it's not really that hard to set up and use. If it is, I want to work on making it easier.

Let's start with an application that a fair number of people have, that the OPML Editor has a solution for -- backing up your tweets, and those of the people you follow. As Apple likes to say about the iPhone, "There's an app for that."

http://editor.opml.org/twitterCalendarTool.html

The docs on that page explain how to install the tool.

Installing it is much like installing an Adobe Air application. First you have to install the runtime, I don't think there's anything complicated about that, then install the tool, which requires a little setup, but again it's not complicated. If you were to install an app in .Net or Java it would work the same way.

Then, once you have the runtime installed, installing new tools is even easier. There's a list of tools that are available, it's called the Tool Catalog. There's a menu item in the OPML Editor app that opens the catalog. Next to each tool there's an Install link. If you click on it, guess what, it installs the app. A confirmation dialog appears, making sure that's what you want to do. And if the app requires you to set some prefs, a page where you enter those prefs appears.

Now I'm working to make it easier. And if people hit walls installing this stuff, I want to fix them. There are still some things I can't do. I can't build the OPML Editor kernel on Macintosh, and this may present a problem down the road, but for right now everything is cool. And if more people use it, it'll be easier to get the build process streamlined. It's all chicken and egg.

So when you use the OPML Editor you help make it better and make it easier for me to develop more tools, which I am working on, all the time.

Bad Hair Day #3

Fri, 10 Jul 2009 16:32:31 GMT

Marshall is buying a new house, so I recruited two guests for this podcast, and they were excellent.

They had really bad hair!

Michael Gartenberg is a Interpret analyst, an expert on mobile devices.

Andrew Baron is a video producer and entrepreneur, founder of Rocketboom and the brand new video aggregator, Mag.ma. At the end of the show he gives out beta access codes for the new service.

We talk about Google's Chrome OS, iPhones, video, realtime stuff and of course Andrew's Mag.ma service.

The feed: http://badhair.us/rss.xml

Find Good Enemies

Thu, 09 Jul 2009 19:17:56 GMT

Mark Bernstein in a rambling quote-filled piece has one of the nicest descriptions of yours truly. "Dave Winer could be rough if you got athwart his hawse, but he was generally a nice guy who always seemed to want to get a lot of bright people around a big table with plenty of food."

Maybe I'm not the most hated person on the Internet after all?

Re Barger's list playing a central role in the origins of blogging, not sure I accept that the most important thing was a list of blogs, or even a network of them. People who think the task of blogging is to pull people together miss, imho, the important thing about blogging -- that it separates people and gives each individual a place to express themselves, not subject to veto. In that way it is different from a mail list. Blogs emphasize the individual over the group.

The argument continues to this day. People who say Twitter is a conversational medium would agree with those who say Barger was the founder. I see Twitter as a publishing environment, a place to push links, a notification system. Oddly, I think Barger with his linkblog approach (which was the same as the early Scripting News or the News Page of the 24 Hours project) would agree.

Bernstein says many wise things in his post. I thought this one stood out.

"If you wish to shed light on a debate, reply to a weblog post on your own weblog."

Amen.

Hyperlocal bloggers at Cal J-School

Thu, 09 Jul 2009 01:18:53 GMT

I'm at a meeting of hyperlocal bloggers at Cal J-school.

People are going around the room introducing themselves. The stories are all interesting, the people are very good public speakers. Almost everyone here is doing, not talking about doing.

Really interesting story from Lydia Chavez, managing editor of misionlocal.org.