Scripting News

Implementor's guide to rssCloud

Thu, 23 Jul 2009 15:02:40 GMT

I have a cloud server running now.

I have feeds that are connected with the cloud and more on the way. I have an aggregator that is wired into the cloud.

Now, if you're a developer of a Twitter authoring tool or want to start a small community of your own (emphasis on small), it's time to start at least thinking about your role in the bootstrap.

Here's the new implementor's guide, first cut:

http://rsscloud.org/walkthrough.html

We need implementations in all environments. My code runs in the OPML Editor. We need everything else.

Dumb XML Question answered

Thu, 23 Jul 2009 18:07:41 GMT

I just stumbled on an answer to Tuesday's Dumb XML Question.

If you recall, I wanted to provide a way for users to view the XML of a feed that contains a <cloud> element. I didn't want them to have to do anything like View Source, for a simple reason. I wanted to make it one-click to refresh, so you could quickly see the effect of a change on the XML. I really missed this from the days before the browser vendors hacked up the viewing of RSS in the browser.

Here's what I did...

1. Add a button to the LifeLiner editing window called View.

2. When the user clicks it, I read the feed XML from the server.

3. Write it out to the local file system with the name preview.xml. Turns out the extension is significant.

4. Send the OS an openDocument message.

5. Voila -- you see the XML as God intended.

Let's hope they don't "fix" this. :-)

What worked for HBO won't work for news

Thu, 23 Jul 2009 13:53:50 GMT

First a couple of upfronts.

1. I love The Wire. It's the best TV series ever. I've paid for it twice, once on HBO and once on DVD. So I not only believe in paying for content I love, I practice it. Redundantly!

2. I'm drinking coffee. There's no spittle in the corner of my mouth. Writing about something I've spent my whole career working on and thinking about. And I'm no kid. I'm five years older than David Simon, former Baltimore Sun journo and co-writer of The Wire.

Simon wrote a remarkable piece in the Columbia Journalism Review, saying that the NY Times and Washington Post must charge for their work the way HBO charges for shows like The Wire.

With all due respect, putting up a "pay wall" is exactly what these organizations don't need. They need to decentralize, get further out into the world, not hole-up behind a wall and try to tough it out.

What worked for HBO won't work for the news because HBO is ficition, and news is not. You can take years writing and developing a story on HBO, polish it, cut out parts that don't support the plot you've devised, even drop the series in the middle if you lose interest. That doesn't happen with the news. News is happening all the time, on its own schedule, all over the place, including many places you don't have reporters. (Think about the arrest of Prof Gates in his own house in Cambridge, this week. Sounds like something that would happen on The Wire.)

And often, the stories are far more complex than reporters can comprehend. This is something I know many people in the news business disagree with. I just don't think the reporter model is working. All it does is inflate the self-importance of these people, turn them into gatekeepers, and often bullies. People who behave like the power brokers they're supposedly covering, when they're forced into playing footsie with them if they want access. Usually under the table but sometimes in plain sight.

As a user of news, I'm sure the future is in shortening the distance between the sources and the readers. Yes, there was a time when, if you wanted to get a story on the wire you had to call a reporter. But that's less true every year, as new channels of news have developed, channels that the NYT (not so much the Post) are just starting to participate in. Watch out as that develops, because it's a potent combination. News people immersed in a sea of news makers. I don't know what news will look like coming out the other end, but it won't look like the system Simon and I grew up in.

This must run its course. The idea of putting up a paywall will just force more reporters outside of it if they want to do their jobs and shrink the publications further. It's no solution.

PS: I don't believe in citizen journalism, which has amateurs playing the role of reporters. I think news is being refactored, unbundled -- broken into components, much the same as other aggregators like travel agents and stock brokers. Music A&R people. I believe in the Sources Go Direct model, the disintermediation of journalism. I also think there's a need for aggregation, but it's a practice people like Simon often mock. In fact reporters base their work on generous people who contribute their knowledge for free -- sources.

One of our heroes dies

Wed, 22 Jul 2009 19:03:25 GMT

MSNBC: "Gidget, the Chihuahua best known for her Taco Bell ad campaign, died from a stroke on Tuesday night at age 15."

5/6/98: "The dog is cool, and Taco Bell owns [her], for a while. Then some ad guy at some agency realizes that he could get a dog too and that dog could eat dog food and like all dogs that we love, the dog farts. Yay!"

Snappy retarded answers

Wed, 22 Jul 2009 18:44:46 GMT

Joseph Smarr asked if I had a snappy answer to why rssCloud is better than The Leading Brand.

I said I do have snappy answers, but like all such answers, they are retarded.

But I gave him a list anyway.

1. Google sux.

Or

2. Feedburner sux.

Or

3. I love RSS, they haven't heard of it.

4. Simple is better.

5. Trade one Big centralized server owned by the tech industry for... another one? You must be kidding.

6. Let's have fun again!

I don't mean any of these things. It's the tech industry way of explaining why the BigCo won't crush your or eat your lunch, or worse, crush you and eat your lunch. In all my years in the tech biz, the only times I've seen the Bigs ever actually crush anyone was when the crushee bought into the crushing.

I don't think Google will crush RSS, any more than TechCrunch will. If they try they're:

1. Assholes.

2. Idiots.

3. Losers.

So they won't try.

Evan Williams vs the Internet

Tue, 21 Jul 2009 16:28:28 GMT

In October 1994, at the dawn of blogging, I wrote a piece that actually shook the software world. At the time, the idea of a mere software developer expressing an opinion in public, unedited, in his own words, without the help of a major publication, was unheard of. It had never happened.

The piece was called Bill Gates vs the Internet. The thought was pretty simple. The tech industry was mired and exhausted. Too many BigCo's struggling to be the one who controls the future. As if a company could control the future. But the headlines in the business press encouraged them to think this way. Much as the leading tech blogs encourage Schmidt, Zuckerberg and Williams today to think of themselves as masters of the universe. They aren't and it's a losing strategy today as it was 15 years ago.

The problem for Bill Gates in 1994, the newly crowned King of Tech, was the Platform Without a Platform Vendor, the Internet. The difference between the Internet platform and the Microsoft platform was this: No Microsoft. No one to hold on to the family jewels. No one to put a developer out of business if they personally offended Bill. No one to keep the personalities of developers under control. No one to cut off their air supply.

In 1994, there was a revolution brewing. Bill didn't believe. But it happened anyway, even though he struggled mightily against it.

Blogging is one of the things that came out of this revolution, and along with it archives. So I can point to a piece I wrote in 1998 and it's still there. It was systematized, in software. This idea didn't come from a BigCo, and it didn't get killed by one. The free Internet solves problems pretty well. BigCo's don't solve problems.

So now instead of Bill Gates it's Evan Williams.

I read the piece on TechCrunch and thought it sounds like the transcripts of conversations from Microsoft in the mid-90s. Both were trying to compete with the Internet. Ev's problem is how is he going to keep his key engineers from defecting to the competition. How are they going to let developers use the "firehose" without using it to kill TwitterCorp. These are problems the Internet doesn't have. It doesn't employ any engineers, and when they leave one company to work for another they still work for the Internet. On the Internet no company owns all the data, so no one can control it. If you don't like the way a service works, use another.

The tech industry keeps having this argument with the Internet. It keeps thinking "this time we gotcha" but nahh, the Internet keeps right on going.

Moral of the story: If you find yourself in competition with the Internet, you should find a way out. Imho.

Frontier bug

Tue, 21 Jul 2009 19:25:47 GMT

Wanted to record this to be sure I get back to it some point.

Problem is with all HTTP requests emanating from tcp.httpClient.

Setting the timeout has no effect when its not possible to open a connection on the server because tcp.openStream doesn't take a timeout parameter. It's always 20 seconds, near as I can tell.

Once I get back into the C source again I'll have to check this out.

You can see the problem on the log page for rssCloud -- when testing the link back from a remote app registering a handler, the timeout is never less than 20 seconds, even when it's unreachable. I have the timeout param in tcp.httpClient set at 180 ticks (3 seconds), which is plenty to find out if there's anyone at the other end.

Dumb XML question

Tue, 21 Jul 2009 18:33:08 GMT

0. I know about View Source.

1. I use Firefox.

2. When I view my site's RSS feed I want to see the XML, not a stylesheet rendering of the XML.

3. To be clear, I want to see the actual XML.

4. Is there some way to force the browser to do this?

5. Please no lectures on how this isn't the way it's supposed to work. TIA.

6. I know about View Source.

7. I know about View Source.

Update: This post seems to explain what's going on. Most browsers do funny stuff with RSS.

Speedbumps and a city's carbon footprint?

Mon, 20 Jul 2009 19:54:22 GMT

We have a mail list for the back channel at InBerkeley.com, and from time to time a question comes up that requires research. If the question is interesting, my first impulse is: Write It Up!

Now, this is the result of 12-plus years as a blogger. I know my community loves interesting questions, and we have an informal approach on Scripting News that I'd like to port to InBerkeley.com.

So, in that spirit -- here's a question posed by my colleague Mark Haas.

Do Berkeley's infamous speed bumps, traffic diverters and other traffic-related policies, like politically-motivated, too-low speed limits raise the city's carbon footprint?

We just need a qualified author. Anyone know any traffic engineers, or perhaps someone at the UC Berkeley Institute for Transportation Studies? Other experts?

Walter Cronkite's 'Cosmic Disaster' editorial

Mon, 20 Jul 2009 14:54:54 GMT

In this week's Rebooting The News podcast, I chose Walter Cronkite as our inspiration of the week.

What a 140-char message looks like in RSS

Sun, 19 Jul 2009 17:37:57 GMT

Here's a screen shot to contemplate.

There are two 140-character messages. Each illustrates features of the new shipwreck I hope to sink, to create a new coral reef for Twitter-like systems to grow on and around.

The first three items in each message are fairly obvious:

<description> holds the 140-character text.

<pubDate> is the timestamp, when the message was created.

<guid> is the identifier for the message, so a reader can tell if they've seen it before.

This makes it possible for the messages to be edited after publication, a common feature requests from writers using Twitter.

After that come optional elements.

<category> works like tags in apps like Flickr or YouTube. You specify them in a dialog, blanks separate them, you can create tags with blanks by putting them inside quotes.

<link> is used to point to web pages. No need to shorten the URLs because they don't take up space in the 140 characters.

<enclosure> is how you attach media objects to messages. Again, no need to shorten the URLs. And since the clients know the media type, they can show a preview, or embed a player.

These all use well-understood elements of RSS 2.0. Nothing new needed to be invented.

Craigslist is progress

Sun, 19 Jul 2009 20:41:10 GMT

I don't think I've ever written about Craigslist here.

Probably because I don't spend much time thinking about it, or worrying about it. But I know that some people do, for example Terry Gross, the host of NPR's Fresh Air. It comes up when people talk about the Internet destroying things that matter, like the classified ads in newspapers. At one point in an interview with Wired editor Chris Anderson she asks, in a bewildered way, what happened. She was saying it was a shame that Craigslist comes along and does what the newspapers were doing, for a fraction of the cost, employing a small fraction of the people who used to support the classified ads in newspapers.

I'm not surprised, and if you think about it, it's very predictable. It's called productivity, and it's what new technology is supposed to do. We used to employ 20 percent of the workforce in agriculture, now it's just 2 percent. That's because of technology. You may say it's bad, but there's also less hunger in the US now than there was then. And there probably are far more classified ads today, now that they're mostly free, than there were when they cost money.

It's productivity. It basically a good thing. And as long as we invest in progress it's inevitable.

Here's an MP3 of the segment quoted above.

Another test post

Sun, 19 Jul 2009 17:09:32 GMT

It's amazing how much discussion these test posts get over on FriendFeed.

I have to do them because my app needs something new to ping about to see if the apps that are subscribed to this get the updates, in real-time of course, via RSS.

A test case for RSS Clouds

Sun, 19 Jul 2009 00:23:54 GMT

I'm preparing a test of a Twitter-like service based on rssCloud, but then realized that I'm within striking distance of something simpler -- re-adding a <cloud> element to Scripting News. It had one for quite a while, we had a server that supported it when Radio 8 shipped in 2002.

So I added it in, and before I could construct a test, some aggregators started registering handlers! In other words, the old network started rebooting. If you don't understand, don't worry about it, but if you do -- man that's spooky.

Anyway, the RSS feed for scripting.com does indeed now have a <cloud> element, and if you happen to have an aggregator that knows what to do with it, please feel free to register a handler. I'll have a lot more to say about this in the coming week, knock wood, Murphy-willing.

BTW, the errors in the log are the result of apps registering receivers who are unable to receive notification because they're behind a firewall or NAT. I obviously have to unsub them when I get that error.

And that's the way it is...

Sat, 18 Jul 2009 00:29:13 GMT

NPR: Walter Cronkite, The Nation's Narrator.

NY Times: Walter Cronkite, Iconic Anchorman, Dies.

This is not an earth-shaking announcement

Fri, 17 Jul 2009 22:40:33 GMT

I'm just one guy programming away, but I have met Valentino Rossi, and when I'm very productive, as I have been this week, I feel like I'm programming the way he rides a motorcycle.

When things are clicking, programming-wise, it activates other parts of my creativity. I cook more imaginatively, and I consider moving to Italy, so I can enjoy the good life while reorganizing the world. They have Internet in Italy so it's hard to imagine how Berlusconi could interfere.

Anyway, this is not an earth-shaking announcement. I just got something working today that I imagined for quite some time, and it's nice.

The goal is to have a Small Pieces Loosely Joined equivalent of Twitter.

I know now that there are people at Google who share this vision. They have the resources to do centralization. What we have to make sure is that the Rest Of Us have the ability to route around the centralization. I hope they don't take it personally at Google, but enough with letting one company control the flow of the real-time web. There are always pundits who are willing to sell us out to the BigCo's, but I am not one of them. Never have. I remember when Google was One Of Us. Hopefully that thread still runs strong inside them now that they are a BigCo.

While everyone was debating the morality of Arrington releasing the Twitter info, I was thinking "Geez these people are focusing on the wrong stuff." The real question is how big TwitterCorp plans to get while holding the control tightly within the confines of their Corp. That can't work. It never has. I'm amazed that smart people like the people who run Twitter are willing to bet on that, still, so far into it. That they think a single company can run the Pulse of the Planet is a sign that they are drinking too much of their own Kool Aid. This can't work. Can't.

Anyway, here's what I have working.

A Twitter-like RSS feed with a single subscriber who gets notified by the cloud when the feed has updated. It then reads the feed and displays the new stuff. This all happens before I can refresh the page. It's the same speed as the connection between Twitter and FriendFeed. Now there will be people who say it can't scale. 1. They don't know. 2. They might be right. And even if it's slower than Twitter, it's worth the tradeoff. Because Twitter is going to break. Be sure of it. Nothing in the history of the Internet has ever done what they're trying to do. I don't know for sure, but I suspect it can't be done. And even if it can, it's bad.

Anyway, time for me to enjoy what's left of the day. It's gorgeous in California. I'm going to get some exercise then have my Italian dinner and then more work tomorrow! Buongiorno and arrivaderci!

BadHair at 7PM

Fri, 17 Jul 2009 00:37:37 GMT

It's been an interesting couple of days with TechCrunch first teasing and then releasing internal notes from Twitter Corp meetings.

At first the debate was over the propriety of TechCrunch releasing this information. But now that at least some of the information is out there, the discussion is turning to the information and attitudes they reflect.

Were having a Bad Hair Day podcast at the normal time, 7PM Pacific this evening and we'll be talking about the release. You can catch the show live on BlogTalkRadio or as a podcast shortly after the completion of the show at 7:45PM.

Our guest for this evening's podcast is Chris Saad, who we had originally scheduled to talk about commenting and blogging and everything related, which is just about everything in community software.

There was a fair amount of TechCrunch agenda in there. I didn't get that they're at war with RSS from reading the TwitterCorp notes, just that they don't feel all RSS content should flow through Twitter. I concur. Twitter and RSS are used for different purposes, and there's far too much new stuff in RSS for Twitter's system to handle.

After reading the TC piece about Twitter I thought: "How wrong that a company owns this medium." Always felt that, reinforced it.

Wonder what Doc Searls thinks about this. Seems we're derailed from the Cluetrain.

If you have comments, please post them here.

Rebooting the RSS cloud

Tue, 14 Jul 2009 23:23:01 GMT

In the early days of RSS, we had the idea that instantaneous updates would be the next step. That was 2001. It took a little longer than we thought, but now with "realtime" as the Next Big Thing, it's time to reboot all that lovely stuff.

http://rsscloud.org/

If it works, it'll be a bootstrap. That means at first the results will be a few sites pinging each other and updating in realtime. If it gains traction, it'll get support from a lot of tech and media companies. It could happen very quickly.

One thing's for sure -- we'll need implementations in every language and runtime. I'm doing mine in the OPML Editor, of course. But just think of that as a reference implementation. I think the really scalable versions will be in Python, PHP or C.

Also thinking about using Amazon's SimpleDB to store the graph. We'll see..

Mulberry ice cream

Tue, 14 Jul 2009 07:00:00 GMT

The editorial team of InBerkeley.com had lunch today at Saul's in North Berkeley. I had a bowl of cold borscht with sour cream, and a toasted bagel with lox and cream cheese.

After lunch we all went down the block to Chez Panisse and had a Bastille Day special mulberry ice cream cone for $2.

What, if anything, did Microsoft announce?

Tue, 14 Jul 2009 03:10:20 GMT

This morning, before we recorded the RTN podcast, there was evidence that Microsoft had announced something. Apparently they had briefed the press at some undisclosed location on some date we don't know about some products we don't understand.

Last week I gave Google a ton of grief for announcing an operating system that has been shipping for 17 years and a web browser that had been shipping for about a year as a new product with the same name as the browser that had been shipping for a year.

In the podcast we talked about a river of realtime news. The analogy fits these pseudo-events in the following way. Sometime in the past weeks Microsoft held a private event, trying to build a dam on the river, hoping to blow the dam at a predetermined time earlier today, thereby creating a rush of news that would impress everyone. It didn't work because apparently the dam developed a leak in the middle of the night and the water rushed down the river of news while everyone was sleeping. No one was impressed. Sad Microsoft.

The moral of the story: Companies probably should announce products when they are new, and when ordinary people and the savvy insiders can try it all out and share their opinions. That way if the product is any good it will generate interest. If it's not good, no one need bother get excited.

ReadWriteWeb: Microsoft Office Web Access Not Here Yet.