Scripting News

OAuth update

Fri, 13 Feb 2009 23:40:55 GMT

I'm still stuck where I was last night with getting OAuth to work with Twitter. I've been able to get all the way up to making a call that requires authentication, and I've constructed the query with what I think is correct OAuth credentials, but Twitter says its an "Invalid OAuth Request."

Matt at Twitter posted a note saying that it might be because my app hasn't been turned on, but later he told me it had been turned on but the query still isn't working.

Now I'm going to look for other OAuth services to test against. If you have any suggestions please post a comment. Thanks!

Connecting with Twitter using OAuth

Fri, 13 Feb 2009 01:22:40 GMT

Okay, so the easy part of OAuth is done, I have it connecting to the demo server. Now comes the part where I try to use it to control Twitter. That part, no surprise, isn't working -- yet. I will get it to work. I'm determined.

This much does work.

1. I am able to get a request token.

2. I am able to direct the user to the page where they give the OPML Editor permission to use my account, and I am able to give the OPML Editor that permission.

3. The OPML Editor, waiting in the background, polling Twitter, determines that it has been given access, and saves the access key and the access secret in the database.

All that happened in 5 minutes, so I was fairly confident that the next step would work, but blam I hit a brick wall and stopped right there. It's insisting that it cannot authenticate me.

4. Here's what I want to do, I want to ask Twitter for my friends' timeline. Before OAuth, you'd make a request of this URL, with nonsecure HTTP authentication, and it would return an XML structure that contains information about recent status updates from the people the user is following. I make the request using the same code I used to make the request of the Irish server, the request that worked, but I get back the following result from Twitter: "Invalid OAuth Request."

I'm doing something wrong but damn if I know what it is!

PS: Here's the current OAuth app table, for the gutsy Frontier programmers.

OAuth hail mary quick code clinic and plea for help

Thu, 12 Feb 2009 16:22:00 GMT

Hi everybody!

As you probably know Twitter is getting ready to support OAuth, and this is a good thing, cause it'll make it easier to trust websites with access to your account cause you won't have to give up your password. But OAuth is hard to implement, it's complicated, and because I'm basically programming the OPML Editor on my own, if I want to support it, I have to write the code. Which is okay cause it's interesting, and it'll mean I'll have a very deep background in OAuth when it's done.

I've been through one of these before. Flickr has a similar authentication system, although it's simpler than OAuth (probably fewer cooks and less compromise in the design). So last night I got coding finally and made a lot of progress, thanks to some help from a tutorial at Hueniverse. But as I was finishing it up I was pretty sure it wouldn't work when I tested it against a server running in Ireland, and sure enough it didn't.

At this point what you do is put up a source listing ahd ask other programmers to have a look. I bet there are a dozen things I'm not doing that I should be. Based on Leah Culver's code, I think I may have to set some headers, but I'm not doing any of that. What else?

Anyway, here's the listing.

http://scripting.com/misc/programming/oauthlisting.txt

Gratitude for any help will be psychically and demonstrably expressed!

Update at 11:50AM: I got signatures working. Here's the updated code listing. How I did it was to fill in the values in the Hueniverse tutorial and step through my code and check my values against theirs. There were differences. Where they disagreed, I made mine match theirs. Once I got them producing the same signature, I tested it against the server in Ireland and it worked. Anyone who's trying to get theirs to work, I recommend doing the same. It takes all the guesswork out of it. Now I have to step through the rest of the dance and see how it goes.

Update at 12:45PM: I'm done with my OAuth library, I've worked through all the levels with the test server in Ireland, and have made arbitrary authenticated calls. I even see roughly how this will plug into Twitter. It means rewriting all my glue code, but should not effect any of the higher-level code. After a break I'll get started testing against twitter.com.

How I made over $2 million with this blog

Wed, 11 Feb 2009 17:19:33 GMT

On Twitter early this morning I said something provocative. "I've made over $2 million from my blog and Dan thinks blogs can't make money. He needs to get out of the box more often."

I was referring to Dan Lyons, who had written a piece in Newsweek that said among other things: "While blogs can do many wonderful things, making huge amounts of money isn't one of them."

I agree. Blogs don't make money. But people with blogs can.

Dan, this is not a fairy tale, I got the check and it had seven figures. But this isn't one of those pitches to get you to buy a book or a video or to come hear me give a lecture. I've made a lot of money with this blog, and may make a bunch more, but I'm not going to show you how to do it. But I will try to get you to change the way you think about blogs and other social media.

Let me start by asking a question.

Assume you run a business and you advertise. How much money did you make from your advertisements?

Presumably your ads make money, otherwise why are you running them?

Now in figuring out how much money you made from the ads, did you look for ads on your ads? That makes no sense of course. Why would anyone try to make money by putting an ad on an ad?

So when I told you I made over $2 million with this blog, why did you immediately look for ads? I can save you the trouble -- there aren't any. And in the 12 years this blog has been here there has never been an ad on this blog. With a caveat, unless you count me talking about my products. Because I do talk about my products here. I try to stay as balanced as I can, but of course I tilt toward the positive. I have a bias -- I wouldn't have made the products if I didn't think they were good. But like all people with real products I know they're not perfect, sometimes they're imperfect, and I try to be honest about that.

Now -- how did I make more than $2 million with this blog?

Well, I ran a commercial company for quite a few years, UserLand Software, and I used this blog to talk about what the company was doing. We had reasonable sales -- probably over $1 million while I had the blog. We never took out an ad, or hired a PR firm. All the promotion ever done for the company was done right here. So let's count half of that $1 million toward the total.

Then there are the consulting gigs I've gotten over the years I've been running the blog. None of them directly resulted from pitches I made here, I never said "Hire me to tell you how to build your product, or work with your community, or serve your users," but the posts I've written here have served as a calling card, a way of keeping my name and ideas on people's minds. Over the years, that's a few hundred thousand.

But the real whopper was the deal where I sold weblogs.com to Verisign for $2.3 million. Again, a product that never had an ad, never had a PR campaign, the only way anyone heard of it was through this blog.

So we're already over $3 million -- and all I did was what any blogger does -- talk about what I'm doing. And that's the role of a blog, it's a way of communicating what you're doing. Companies, consultants and authors need to do a lot of communicating, and blogs allow you to go direct, and be more efficient, less diluted. People get a real feel for who you are and how you think and what you're like as a person.

Why would I ever let someone else hitch their "message" on this -- it would get in the way of me making money!

If I had any advice to offer it's this -- get in the habit of communicating directly with the people you want to influence. Don't charge them to read it and don't let others interfere with your communication. Talk through your blog as you would talk face to face. You'd never stop mid-sentence and say "But first a word from my sponsor!" -- so don't do that on your blog either. I can't promise you'll make any money from your blog, and I think the more you try the less chance you have. Make a good product and listen to your customers to make it better, and use the tools to communicate, and you may well make money from the whole thing. To expect the blog alone to pay your bills is to misunderstand what a blog can do. You'll only be disappointed like Dan Lyons was.

How we look vs Who we are

Tue, 10 Feb 2009 22:07:05 GMT

I was talking with a friend this morning, part of an ongoing conversation about how people are judged by their outside image and how this may be at odds with how they see themselves. I think we spend our whole lives studying this.

A few stories...

1. About 15 years ago I took a picture of my father and showed it to him. I thought he looked good, happy and fit -- I thought he would be pleased, but instead he winced. I said "Dad that's what you look like." And he said one of the sweetest things I remember him saying -- that inside he still feels like he's 19 and this picture reminded him that he was not (he was about 65 at the time).

2. A friend got rich, quickly, and then just as quickly got poor. Never saw anything like it before or since. I formed a theory that inside he felt poor and the riches were at odds with that. It was easier to get rid of the money than to get rid of the feeling of worthlessness. (None of this is conscious by the way, it's all about the subconscious, which is much more powerful.)

3. I was dating a very attractive woman. I got invited to see the world from her point of view, and man, what a difference. She thought everyone was always trying to get in each others' pants. I told her this was not true! But I don't think she ever believed me.

4. A friend tells a story about very gentle person, a man, who has very dark eyebrows. He explained that everyone always thinks he's angry even when he's happy or wistful or curious or sad. Everyone reacts to him all the time as if he were angry. To him the world looks like a very defensive place.

5. Me, I was always a tall boy growing up. But there came a time when I shot up and went from 70 or 80 pounds to 150 or 170, probably in two or three years. From 60 inches tall to 72 inches. Inside I was still a child, but outside I was scaring everybody. I know that now but I didn't understand it when I was a kid, because I was inside the body looking out. I remained more or less constant, but the effect I had on people changed dramatically. This left me confused for years! A lot of people must go through this, we all grow up and the people around us remember us as a cute little kid and all of a sudden the cute kid is gone, replaced by a strapping young man or a shapely young woman. Maybe they never forgive you for stealing the cute little kid?

Anyway, back to my friend who started this thread. I had asked her to tune into my Twitter feed and she did, and we talk about it when we get together. Today she wanted to know why my icon is King Kong. I said it's a joke -- that's what people say about me, but it's not really me. Inside I'm nothing like that, but on the outside, often it seems to me that the world is relating to me as if I were. She asked why not change the image. I laughed and asked -- what should I change it too? Ballet slippers? A pink tutu? No, she said, how about a teddy bear or Gentle Ben? Hmmm. Well, I'm not quite ready for that, after giving it some thought. I don't see myself that way, and it's important that the iconography not only reflect how you'd like people to see you, but also reflect how you feel inside. So I looked through my archive and settled, for now, on the accordion player. He's a very frequent guest here, and in many ways I identify with him. Playing a tune, giving people a song to sing, but folksy -- even schmaltzy -- that's how I'd like to be seen. Not too heavy, but not all cuddles either. Maybe someday I'll feel okay with Gentle Ben, but not yet.

So I'm archiving my Twitter imagery today and replacing it with new stuff. Here's an overall screen shot of what it used to look like, and here's the old avatar, and the background image. I'm replacing it with the accordion guy and a picture of the SF skyline taken from my perch in the Berkeley Hills, roughly what I see when I look out over the world while I'm twittering or blogging.

Of course while I was trying to make the change I had a visit from the whale. I'll have to wait till he let's me do my thing. :-(

Twitter kills Google in real-time search

Tue, 10 Feb 2009 09:44:49 GMT

Matt Cutts: "The real-time web is not the threat. Google can index data in seconds.'

He's the head of webspam team at Google, and a man who, obviously, knows a lot about search.

That was 17 hours ago as I write this at almost 2AM, Twitter's real-time search has had a story, a few thousand night owls were up, in LA and around the world, and news was breaking, the kind of sensational prurient goop that Twitter loves, a LA car chase, covered by helicopter. As I write this, Twitter is about 1 hour ahead of Google, that is, Google's latest news is 1 hour behind Twitter's.

Here's a screen shot of a Google News search done at 1:50AM Pacific.

Granted, this is not (likely) an earth-shaking story. But if it were, the same technology would apply. It's a good test case, a good dry run.

Also granted, the Twitter result is scattered and disorganized. If you weren't watching the event unfold in realtime you would not be able to piece together the story. However I was watching realtime while it was happening, hitting Twitter's realtime search and watching my incoming twitstream. I follow over 800 people, and a lot of them were fascinated with the story of the slow-speed chase through LA of the police by someone driving a white Bentley. Who is it? Why is it taking so long?

We watched as a helicopter hovered over the scene where the driver was apprehended. Ominously one Twitterer says the driver killed himself on camera on ABC. Now there are some published mainstream reports that say this too.

At 1:57AM, Google has a link to an AP story with an update about the driver being taken away in an ambulance. The story has a post time of 1:34AM, about 25 minutes ago as I write this at 2AM.

Now all this is likely to get washed out in a few hours when the reports are filed and there's been a press conference. My point is that Twitter is doing something new and with all due respect to Google, something that Google isn't. However, there's a lot of room for improvement, and connections between the various parts of the news ecosystem.

Every serious news outlet should have someone monitoring Twitter 24 hours a day, 7 days a week. They should be following at least a few hundred people, feel free to clone my follow list, if you like. It got me in the loop on this within minutes. When a story breaks, a reporter should be dispatched to cover the news that the Twitter community uncovers. The first news organization that does this well is going to get a ton of flow and attention and be well-positioned for the new 24-minute news cycle. Google clearly needs to get their hooks into the Twitter flow, but it's not clear that Twitter wants them in there, hence the last part of @mattcutts's quote, which is ominous: "The larger issue is when search engines can't see data." As we often say "Bing!" -- that's the crux of the biscuit.

As happens so often in tech, we're coming up to a repeat of the sitdown scene in The Godfather where the heads of the other families tell Don Corleone that he must share his Senators, which he keeps in his pocket, like so many coins. (I'll look for the exact quote.) Twitter is holding some valuable coins in its pocket. But Google need not sit on the sideline, they're a big force, and they're moving into social networkings like an aircraft carrier, slowly and deliberately, one step at a time.

Another note -- this is a thread we've been following here on scripting.com since 1996. Today we call it "real-time search," back then I called for JIT-SEs or Just-in-time search engines.

One more observation: Sometimes having insomnia pays off.

Update: At 2:25AM, the LA Times has a lot more detail.

Update: The full Godfather quote from the screenplay by Mario Puzo. "Don Corleone is too modest. He had the judges and politicians in his pocket and he refused to share them. His refusal is not the act of a friend. He takes the bread out of the mouths of our families. Times have changed, it's not like the old days where everyone can go his own way. If Don Corleone had all the judges and politicians in New York, then he must share them or let others use them. Certainly he can present a bill for such services, we're not Communists, after all. But he has to let us draw water from the well. It's that simple."

David Recordon

Sun, 08 Feb 2009 20:01:31 GMT

Just had a 1/2 hour phone conversation with David Recordon who's involved with OpenID and OAuth and works at SixApart. There were no deals made, we talked in generalities, I wanted him to understand where I was at with these two technologies. In the spirit of being open and transparent, here's where I'm at:

1. I'm a user of OpenID. If you look at the HTML source for the scripting.com home page you'll see all the OpenID information. Pretty sure I understand how it works at a technical level, and that I could if needed do a server implementation.

2. I'm confused by OAuth, but I know I have to implement it, at the lowest level, because there's no community to do it for the OPML Editor, and that's where I do my development. I've done the Flickr pre-cursor to OAuth, and hated it, but I have it working, and it works without flaw or problem, so that's behind me. The docs that exist for OAuth are daunting. But it's high on my to-do list.

3. Because of the inevitability of OAuth, I'm concerned that the big companies are going to start monkeying with it, and that at some point there will be many flavors of OAuth and I'll have to implement them all. This is just a fear, only based on experience with other formats and protocols.

4. In the struggle to achieve simplicity for OpenID, I came to the conclusion that it has to be built into each web browser. Interestingly David came to the same conclusion and wrote an article about it in December last year. Imho, the browsers that need to do this are: Firefox, MSIE, Safari and Chrome. If they do it the rest will follow. I think if even one does it, all the rest will follow because it is such a compelling idea, and users will like it and insist on it from their browser vendor.

5. He agrees that we need the ability to attach arbitrary metadata to a user ID.

6. He and I see eye to eye on many of the same things. He's younger, but seems to be going down a similar path as I did. He's in for a lot of pain, but he's doing good work, and I told him so, and I wanted to get that out here.

7. I still think it's trouble to support Facebook having an invite-only meeting for design around open technologies. However he says they have removed the "invite-only" phrase from the meeting description, and they may webcast it. My main concern is that the OpenID that enters the room is the same one that exits when the meeting is over.

Kettle Chips

Sun, 08 Feb 2009 23:51:46 GMT

Yummmm!

Facebook and Twitter, OpenID

Sat, 07 Feb 2009 23:43:55 GMT

There was some really interesting news yesterday about Facebook opening up their model so that status updates can flow in and out of their sandbox through their API. A lot of people who are familiar with programming Facebook (I'm not) say this will open Facebook to the same kinds of apps that have been developed around Twitter, which has always had this kind of API.

Immediately I felt drawn to know more about how Facebook works, for the first time. I've been happy writting Twitter apps and FriendFeed apps. But a lot more people use Facebook, and as we know and have said many times, the important thing is where the people are.

The best post I've read so far on the politics between Twitter and Facebook was written by Marc Canter. I laughed out loud a few times while reading it, it's that good. I don't think they're laughing about it at Twitter headquarters. But I also don't think it's in any way over, as some do. Techno-political chess matches are rarely finished after the first few moves, it usually takes a number of rounds, backs and forths, before you can develop a theory about the outcome, and even then it's hard to knock momentum. One of my investors once said, when Apple was just a $1 billion company in 1983, that "Billion dollar companies don't just disappear."

Okay, Twitter is a $0 billion company, but that's not the point, they've got a lot of momentum, a lot -- and it's not going to disipate because of a technical innovation at Facebook. It's going to take more than that. And in the meantime, it certainly means that Twitter will be more attentive to its partners and developers, and perhaps be a bit more aggressive with feature rollouts (It's been sitting on a lot of new features for a long time, imho).

I called Marc to congratulate him on his piece, and asked about the meeting at Facebook next week about OpenID, and he said something I was afraid of -- it's invite-only. Which means they're screwing around, both Facebook and especially the OpenID people. You can't have invite-only meetings about open standards. It's not at all a small thing. I strongly recommend that the meeting be held outside of Facebook, that they be made totally welcome to participate, and that everyone else who's interested be welcome as well.

OpenID has come a long way and as far as I know has always been a level playing field. I've seen people trying to have private meetings about SOAP and RSS, and I always put my foot down and said no, and blew the whistle publicly if I needed to. It's time for the leaders of OpenID to defend the open-ness of their work. You can't have invite-only meetings about open protocols. And there can't be any exceptions to that.

Update: "Because we want to keep it small and focused, the event is invite-only."

Don't boycott Kellogg

Fri, 06 Feb 2009 20:12:54 GMT

First, a disclaimer -- I used to smoke pot, but I haven't in many years. I don't smoke anything, or actually take any drugs that get me high. However, I strongly believe that marijuana should be decriminalized, even made legal. I think it's the ultimate in hypocrisy to argue that former members of the excecutive branch of the US govt should not be prosecuted for war crimes because it would be looking backward, while our jails are full of people whose only "crime" is that they smoke pot. It's like we have two legal systems, one for the powerful and one for the rest of us. It's so un-American, I don't know why people can't see that.

Anyway, I remember what it was like to smoke pot, and I definitely remember what it was like after smoking pot. It gives you the munchies, an appetite for anything that's rich and sweet. I think sometimes American industry worked at creating products just for people with munchies, like Krispy Kreme donuts. Kellogg's makes many such products.

Now Kellogg's has dropped Michael Phelps as a spokesmodel because, shudder, he got caught smoking pot. Geez Louise, that's like getting caught having a beer. Yeah it's illegal, so is torture, what are you going to do? Fact is, after winning all those gold medals in Beijing and bringing glory to our country, he's entitled to kick back, smoke some dope, eat some poptarts and rice krispies, and hang out for a bit while he basks in the glow of our admiration. He should get laid too. A lot. (But use protection, you don't want to get a STD.) What's the point of being young and successful if you can't enjoy it!

What a bunch of stinkers they are at Kellogg's. They could score so many points by saying something like this: "We don't encourage pot smoking, but we understand that some people do it. We have so many bigger problems to tackle in this country, and Michael Phelps is such an incredible young man and hero, we decided to be heroic ourselves, and cut him some slack, and keep him on the corn flakes box."

So I don't think we should boycott Kellogg's, let's buy them. And put up ads on our websites explaining why we're buying them. Because we want them to do the right thing, and get the big fat stick out of their butt, and lighten up a bit. Maybe even step out back of the office in Battle Creek and smoke a doobie.

Instant Outiner bootstrap

Fri, 06 Feb 2009 19:47:20 GMT

I've been quietly working with a small number of users on the next interation on the OPML Editor's instantOutline tool. I wrote about it in January, after Hutch Carpenter discovered its previous incarnations. I was getting a new version ready for a project I was working on in December, so it was pretty easy for me to get it ready for wider use, which I have now done.

The new tool has several prerequesites: 1. You must have the OPML Editor installed on your desktop computer, it's available either for Windows or the Macintosh. 2. You must be a member of FriendFeed and 3. You must be a member of the instant-outline-beta group on FriendFeed. I've built most of the back-end on the excellent realtime updating API in FriendFeed, that's why you have to be a member to use the I/O tool.

Documentation is here: instantOutline.html.

I'm monitoring the workgroup. Hopefully this is a next step in something fun and excellent. Look forward to seeing you there.

One more time -- open the news industry!

Fri, 06 Feb 2009 16:55:51 GMT

As I said in a podcast a few days ago, since the beginning of my career in the early 80s, I've been meeting with people in the news industry to try to play a role in its transition to an electronic medium. But that's only half of it, the easy half. The hard half: I want to be a reporter, but a new kind of reporter. Instead of one of the few, I want to be one of the millions. And I want technology to find a way to do what reporters of the 20th century used to do, to organize all the information from what they used to call "sources" into reports that people like you and me can read and think about and discuss.

Reporting is a connecting art, like a real estate broker, travel agent, stock trader. The writing part of reporting is mundane, you want the reporter to stay out of your way as much as possible, and the good ones do. It's like the other arts -- who wants a real estate broker who sells you on how great it would be to live in a house while you're looking at it. They don't know how you live, their chatter interferes with your dreaming, and it's the dream that buys the house. I once had a travel agent who loved to golf, so I ended up staying at hotels near golf courses. I don't golf. So now I do my own travel agenting. It takes more time, but I stay in places that are a better fit.

The news people talk about paying for news, but the suppliers of news, the sources, are never paid. So if we can find a way to do what reporters do, without paying reporters, then voila, we can have our news for free. Before you rattle off some tired rationale, think about it. What are reporters doing that amateurs and/or software can't do?

Jay Rosen explained this to me once -- the word for what reporters do that machines don't is "authority." Humans convey authority. But -- only until humans teach us how to do it for them.

Here's the idea I would program into the heads of people who run the news corporations if I could turn them upside down and hang them by the feet until all the old wrong ideas ran out of their heads, forming a fetid puddle on the ground beneath them. News people are all around you, anxious to get in there and work, for free, on the news. At first thousands of them, and then once the glitches are worked out, tens of thousands. There's no shortage of people who want to inform others. The challenge is to figure out which ones want to do it for love. And that might not be such a challenge. I can show you a few dozen, and I bet they could show you a few more and so on. In the end you might not be able to make money at news, but you're not making money now, so what else is new?

The manufacturing process for news has radically shifted. The question is, as with the economy, whether we can transition the existing process to become the new one (imho preferable) or does the old system have to collapse before the new one can rise to take its place.

The key is to look at all those empty newsrooms, and to envision, before they completely shut down, filling them with volunteers -- who we can teach to write the news.

One more thought -- as with all post-apolcalyptic thinking, post-Katrina New Orleans provided the testbed, the dry run. Look at what the Times-Picayune did in the days after the hurricane. In my humble opinion a great newspaper rose overnight where a mediocre one had been the day before. The printing presses weren't running, and the normal management structure was heavily disrupted. But they had a story, a great one -- and if you go back to the roots of news, that's when it really happens, not when someone pays you well, but when you have a great story. (Same thing happens in software, when you're shipping a winner, somehow everyone on the team knows, and they put it in an even better performance.)

That's what we all want to be part of -- something great. I think that expresses the best of the human spirit. As young people we want to be the greatness, but as we grow we want to be part of greatness. That's much more exciting.

Update: Here's an example of the kind of reporting I find riveting, Pulitzer-worthy, written by an amateur, with passion.

Katrina, USA

Fri, 06 Feb 2009 01:25:32 GMT

I had lunch with an old friend from college, Sandy Wilbourn. We both went to Tulane in the 70s and majored in math. Then a funny thing happened, shortly after I started grad school in computer science at UW-Madison, I ran into Sandy on campus. He was getting a degree in math there. Then a lot of years later I was shopping at Roberts Market in Woodside and saw Sandy. He lived in the Valley too. Earlier this year his uncle left him a house in Berkeley, a few blocks from where I live. Hey it's a good thing we like each other, we seem to be in the same karass.

Anyway, Sandy is the first person I know from New Orleans who I've talked with about my visit after Katrina in 2005. It stirred some memories cause we both know all the same landmarks, where the river bends and where the levees are. But Sands hadn't been back to New Orleans so I told him about places that had been wrecked that, last time he saw them, were fine.

I remembered a lesson from Katrina, the human side of something Krugman keeps saying: once deflation starts it's very hard to pull out.

New Orleans went out of business. One day every business in the city shut down. Some were destroyed, could never return (for example those in neighborhoods that were under water for weeks). But every other business had to restart from a dead stop. It's as if from an economic standpoint the city didn't exist.

The city's recovery will be slow, if it ever fully recovers. Some parts seem likely to come back. The richest parts, the oldest parts, the business parts. But it was all damaged and virtually all the people had gone, and many haven't come back.

Now, huge parts of the world economy have shut down. Some have been shut down for months. Economies don't just start back up once they shut down. They can start back up from a slowdown much more easily. But once a business is gone, it's gone.

Time for heads to roll at Meet The Press

Thu, 05 Feb 2009 22:51:25 GMT

First some disclaimers, disclosures, etc...

1. I stopped watching the Sunday morning news shows after the election. Now I listen to the podcasts, when I have a chance. I'm fed up with the gotcha crap. Gotcha, gotcha gotcha, that's all they know.

2. I don't like "gotcha" interviewing. I want to hear what the people have to say. I'm very circumspect. I'll form my own opinion on what they say. I know they're all lying and spinning. No one ever gets anyone with a gotcha.

3. Gotchas only interest the reporter and his or her competitors. It's their way of keeping score. No one else cares. And because they all do it all the time, it breeds politicians who are good at saying nothing because the prime gotcha is "I caught you saying something. Gotcha!"

4. Like every other element of the political system this needs reforming. If you believe in the primacy of the network filter (for me it's fading realllly fast) then nothing can get done until they evolve beyond gotcha.

5. Make a list of the reporters who can interview someone and just let them tell their story, help them along as needed, represent the audience, and stop playing the insiders' game. Bill Moyers. Some of the PBS people like Gwen Ifil. Terry Gross except when she's interviewing the terrorist who Obama palled around with. And the guy I like best: Aaron Brown.

Which brings me to the point.

Tom Brokaw was pretty good. He's old enough and senior enough not to really care what the other reporters think of him. Even so, he did play gotcha while he was filling in. But he was the best of the three. Now that he's gone, Stephanopoulos is the best, Gregory is a toad. He's a tiny mind. In way over his head. He says the stupidest gotcha stuff. Here's my favorite -- he pressed Rahm Emanuel to explain how hiring more teachers was going to create jobs. Yeah he actually said that. Several times. Give Rahm some points for not calling him a fucking idiot to his face.

Given a few years maybe Gregory will grow into the job. But we don't have a few years.

If you have to go for another interim host, get Moyers if he'll do it. But if you really want to make news on Sunday morning work, and raise the bar for everyone else, get Aaron Brown to do Meet The Press. Pay him $20 million.

I beg you.

A metadata race?

Wed, 04 Feb 2009 17:35:10 GMT

Steve Gillmor perceives a race between Twitter and FriendFeed, which I find interesting, even if I don't think it's as much a race as he does. At this point there is so much distance between the services they provide, it's hard to see them as competitors.

However, there is one area where all the current providers of social networks are in competition, and I'm not sure they're all aware of it, and so far no one has entered the arena, but I get a sense that at least a couple are poised to -- Twitter and FriendFeed.

There's an interesting story that came up that's very much in line with the Where's Your Data? and What Happened To NakedJen? threads here on scripting.com. Last week, a bookmarking service, ma.gnolia, lost all its data. All of it. It was bound to happen sooner or later. Everyone who operates a public service back-end knows how fragile these things are, I sure do. At UserLand we came close to melting down a few times, and every one of those times we urged our users, as best we could, to keep local copies of their data.

Anyway, ma.gnolia crashes, and someone there realizes that FriendFeed has been archiving their data for users that established the connection between the two services. And FriendFeed has an API. So voila -- write a web app to pull the data out of FF and re-populate their database, at least for some of their users.

This suggest a buddy system for web service providers. Not every service can rely on the same back-end for safety, what happens when that service goes down? And who knows all the dependencies we're creating -- what if FF is using Amazon's back-end and Amazon gets attacked by an asteroid or a crazy terrorist, or whatever.

All of this leads me to a wish that one of these companies would allow, through their API, for us to store arbitrary data on a per-user basis. I'm working with a small group of users on a new build of the Instant Outliner, and am using FriendFeed's "room" structure to great advantage. It's almost at the point where I don't have to write a back-end at all, I could almost completely depend on theirs. If only. If only they allowed me to store a relatively small amount of XML-based data with each user. Less than a megabyte per user. Probably way less.

Something like Amazon's SimpleDB only even simpler, would do the job. The equivalent of a Perl hash or a Python dictionary. The same data we pass around in XML-RPC would be very good.

I know all this sounds super-technical, but the architects at Twitter and FriendFeed know exactly what I'm asking for, and it wouldn't surprise me at all if one or both had this facility almost ready to go.

Archiving Twitter in OPML

Wed, 04 Feb 2009 16:23:21 GMT

In mid-January I started a project to archive the Twitter posts of the people I follow. At first I experimented with rendering the archives in an XML-compatible form of HTML, but decided the point would largely be lost, so I decided to go with OPML.

You can find the folder of archives here:

http://twitter.opml.org/calendar/

At the top level of each sub-directory is calendar.opml and today.opml. The former links to every OPML file for that user, and the latter contains all the twits from today, or the last day that person posted something to Twitter.

There's also a folder called 2009, and under that two sub-folders: 01 and 02 for January and February. And under each of those is a file for each day. In March there will be a folder called 03, and so on.

I do not archive posts for people whose Twitter accounts are private.

The archive follows the whims of my follows and unfollows. If I started following someone on January 29, their archive would start on that day. If I unfollowed and then followed, there will be a gap in their archive.

The archive is updated once a minute.

The server is in Amazon's EC2 cloud.

No guarantees are made that this service will remain up, I'm doing it entirely as an experiment, to learn what the issues and perhaps what the opportunities are.

What happened to NakedJen on Facebook?

Wed, 04 Feb 2009 03:19:09 GMT

NakedJen was a very early Facebook user because she worked for a "sister company" -- one that was funded by the same venture capital firm as Facebook, Inc.

Her account, which was recently deleted by the company for unknown reasons, was non-commercial, it represented simply a person, wasn't excessively large, didn't contain any nudity or other objectionable material. When she asked for an explanation, they told her to read the terms and conditions. When Scoble asked Zuckerberg about it he gave an annoying explanation about how the company would rather have some "false positives" instead of have their system abused.

As a friend of NakedJen's (whose birth name is Jennifer Neal, a name I have never used for her) I don't think of her as anything like a "false positive" -- she's much more of a "true positive" -- and a really cool human being. I named her my Blogger of the Year for 2007. That says it all as far as I'm concerned.

Now I'd like Zuckerberg to get in touch with his true positive -- and get a clue that his users are people who use his system in the most personal way imaginable. If you're going to kill someone's presence on Facebook, please -- give them some idea why you're doing it. And if you screw up, as you certainly did this time, please have the guts to say so and give the user the satisfaction of knowing that you care, just a little, what they think of you.

Twitter API for the social graph

Wed, 04 Feb 2009 01:06:43 GMT

Twitter just announced two new APIs that should make a host of new applications possible. The two APIs allow applications to navigate the "social graph" defined by Twitter under program control. I'm going to write a few little apps to test it out and report back here.

Kevin Marks: "the api returns numeric IDs, not twitter handles? that seems lame."

I won't go as far as Kevin, but I do wonder how we're supposed to use this data. For example, I follow 828 people. How am I supposed to get the handles for each of those people? Should I make 828 calls? I guess they're assuming I'm storing them in a database using the numeric id as a key. I don't. My databases always use the mnemonic as the key, for example, in my calendar application, I access Kevin's data through this address: config.twitterCalendar.users.kevinmarks which corresponds to a folder on a server.

Where's your data?

Sun, 01 Feb 2009 18:26:45 GMT

Craig Burton knows what that question means.

He was the CTO at Novell, the leading network company of the PC world in the 80s and 80s. Follow the link from his name and read what you should be thinking about your data and whether and how much you should trust companies to keep it safe.

However, pause when you get to this part: "I am still reeling from the transition to Wordpress. I lost years of data, links, discussions. No thanks to Dave Winer. Dave, I love you, but I think you left a ton of us locked into your silo with no way out."

First, I love you too Craig, but you're totally wrong about that. At UserLand it was our religion not to lock users in. You could download your entire website from our servers, even the ones we hosted for free. We pleaded with people to do it, but most users either didn't understand, or didn't care enough to do it. We also provided a tool to convert those websites to folders of XML files, to make it easy to port it to other blogging tools. The only way we could have made it easier was to write the import routines for our competitors.

Since we were one of the earliest vendors of blogging tools, we hoped to set a high bar for all to come. Unfortunately this didn't turn out, except in RSS aggregators where portable subscription lists are now the norm because Radio 8 taught users the value of being able to switch.

Pay attention to what Craig says, and don't store anything on anyone else's server unless you know how you're going to get it off when you need to. Even better, don't store the original on someone else's server, keep that in your space and share a pointer to the data.

We're headed into tough economic times and a lot of the for-free companies are going to go under. I think that's a virtual certainty. Further, a lot of them are cutting back, so their technical staffs are going to be thinner and more likely to make a mistake that costs you your data. Read the Terms of Service, they're usually not obligated to do anything to help you in times of trouble.

Hugs to Forbes

Fri, 30 Jan 2009 03:32:14 GMT

Big hugs to Forbes for saying I'm one of the 25 nicest people on the web. You guys are the greatest. Love ya all, Dave