Scripting News

How investigative research happens in the blogosphere

Sun, 11 Jan 2009 17:49:28 GMT

One of the common complaints from people in journalism about bloggers is that we just comment on reports in the news, we don't do original reporting. It's so often repeated it's become a cliche, but it's simply not true and I can prove it.

Draw a continuum of different kinds of news, at the left edge put a dot and label it "Event." At the extreme right edge put "Commentary." On the line between those two extremes, somewhere put a circle and label it "Reporting."

No one questions that bloggers produce writing that belongs at the right edge of the continuum, but then so do pros. That's editorial and op-ed pages. People like Frank Rich, Maureen Dowd, David Broder, Krugman -- in days gone by James Reston, Russell Baker (my childhood role model). Think of it as shared territory between pros and amateurs.

At the left edge, bloggers win hands-down. We have far more original sources blogging than the pros have writing in their pages. Over time, as the thirst for news goes up and the volume provided by the pros goes down, expect to see more. The thirst to be heard has always been high, and this is fueling the explosion of blogging at this edge.

Now, if you're a regular reader of Scripting News, you see us do investigative reporting all the time. And I'm not using the plural in a regal sense, I very much do mean "us" -- as in me and you and a whole lot of other people. For example, in December, after seeing Tweetree, and having put a lot of work into systematizing thumbnail generation (with the ongoing help of a reader, who lives in Turkey), I wanted to connect the two. So I proposed a way to do that. Immediately a flood of comments probing my decision, and a number of suggestions that I look at work done by others that I hadn't found when I looked. Now this process is not trivial, it's the equivalent of a reporter making dozens of phone calls to experts. Sure, I'm not probing their ethics, looking for malfeasance, whether or not they're bugging Watergate, or stealing funds from taxpayers or widows, instead -- they're doing that to me. Which I've learned to live with, and see as a good thing. I know my heart is pure, but they don't, at least not at first. They are right to be suspicious of a fellow technologist. Many are corrupt. I've writen about that here. (Part of the reason there's so much corruption, btw, is that the professional press including some of the pubs we all revere, have been playing footsy with the industry for a long time. But that's another story, one often told here, btw, which has gotten me a rep with the press for being an "irascible gadfly.")

Anyway, a comment left here by a reader in December bothered me, so I looked into it and found he had something, so I revised my approach, and published the revision here, on Friday. Then another reader suggested I use a format that Digg had proposed, and I agreed, so -- another revision. Now here's a key point, in all the reporting I had done up till this point, I had never stumbled across this. It hadn't been reported on by pros as far as I could tell, and up till that point, none of my readers knew about it either. Is this an investigative process? Absolutely. Is it journalism? I don't see how it's any different from the best journalism done by the pros, except 1. it's done out in the open, that's an essential element, and 2. it's being done by people who do it for reasons other than being paid, directly, for it. To trivialize those reasons is to ignore why people want knowledge, to ignore the motivations of the sources and the readers, both of whom are essential to the news processs (though the journos are often loathe to acknowledge this), and neither of whom are paid for their efforts.

But even then the story is not over. Another reader says there's a mistake, the format wasn't authored by Digg, it was actually authored by Facebook! I did a bit of investigation and couldn't find a claim that they authored it, or a news report, but I did determine that they supported it, by doing a test, that worked. We saw a citation on the Digg site that said that Facebook was the source, so that was good enough for me, and I revised my piece accordingly. (I allow myself that until the day closes, then if I want to revise I have to write another piece, like the one I'm doing now, that's why serious stories on Scripting News tend to come in a series.)

Finally I published a "cap" to the series, and reported on the significance of all this work. A full week of work, with lots of human contact, research, sourcing, fact checking. I'm sure there will be some who say this doesn't belong in the circle in the middle of the continuum above, and I'm sure we'll get ample opportunity to discuss that, and learn from each other.

Elmer Fudd goes on a Westful We Tweet

Mon, 12 Jan 2009 05:04:37 GMT

I am not an economist

Sun, 11 Jan 2009 23:52:56 GMT

Even though I'm not an economist, I'm pretty sure that the US government can print money.

So if we have a $1 trillion deficit this year, that does not imply that someone has to lend us $1 trillion and it does not imply therefore that someone will have to pay someone back that money at some date in the future. If there's an economist listening who thinks this is not true, please say so and everyone else ignore what I'm about to say.

So then why does the discussion on Meet The Press and This Week revolve around how much better the private sector could spend the money or how we're passing on the problem to future generations. That seems like hooey. They have these discussions based on a make-believe premise that there are only two ways for the US government to pay for something -- taxation or borrowing. There is a third way, the one they're taking -- they just invent the money.

Now, in normal times you pay for this invention with inflation, but inflation isn't a problem -- quite the opposite, the world economy is having a massive going out of business sale. Prices are going down. Think they're low this week? Wait till next week, they'll be even lower. And wait is what people do, and that drives the prices lower. It's like inflation in reverse. Worrying about inflation now is like a starving man worrying about getting fat if he eats too much. You'll take your calories any way you can get them. Anything that makes people want to buy things now is a good thing for the economy. One way to do that is to give them money by hiring them.

So forgive me for thinking the analysts on TV are either corrupt or idiots, or whatever.

And of course please forgive me if I'm wrong and the government isn't going to print money to get out of this hole.

Back at the dawn of time

Sun, 11 Jan 2009 11:19:15 GMT

I happened to trip over this Youtube video of John Lennon, Paul Simon and Andy Williams at the Grammies in (I think) 1975. They're announcing the winner of the Best Song, I think -- as usual Lennon is very funny and wry, and Simon is funny too.

Some observations...

1. There's a surprise guest toward the end, I won't spoil the fun by saying who it is, but we might spill the beans in the comments, so if you don't like spoilers watch the video before reading the comments.

2. Either John Lennon was short or Paul Simon is standing on a box. It's weird how a lot of famous people are really short, but you never find out until they stand next to someone who you know is short.

3. I don't get the joke about "dawn" -- Lennon says: "So this is what Dawn does." Dawn of Tony Orlando and Dawn? Hmm.

4. Weren't they both disapponted that Olivia Newton-John won? The competition was so good, Elton John, Roberta Flack, Joni Mitchell, Maria Muldaur.

5. Muldaur's song, her only hit, Midnight At The Oasis, also played a big role in the fabulous movie, Lost In Translation.

6. Unrelated, on NPR yesterday I heard that Enya has sold 70 million albums and lives in a castle in Ireland and is not lonely.

Two old friends: Twitter status and spewage

Sun, 11 Jan 2009 11:06:21 GMT

It's amazing to me, the Twitter Status Report and Twitter Spewage Report are both still running. You leave those background projects around and they just don't go away.

I wonder sometimes how long after I die will some "Dave Winer code" be running somewhere on this planet? It could be over very quickly, or you never know, maybe some of my code will run a long time. I, of course, won't be around to find out.

Anyhoo, here are the links...

http://twitter.scripting.com/status.html

http://twitter.scripting.com/spewage.html

And here's the blog post in April last year announcing the spew report. It was shortly after this that Twitter started becoming unreliable. Wonder if there's a correlation? Heh. That's supposed to be a joke. Okay sorry.

How to include a thumbnail in the HTML of a page

Sat, 10 Jan 2009 19:12:27 GMT

If we had a registry for emerging web standards, we would have added one to it yesterday. The emerging format has been proposed by Facebook, supported by Digg and ratified by FriendFeed, and yesterday -- Scripting News. The format allows a photo-oriented web site or individual web page to include a thumbnail, so that when it's referred to by a site such as Digg or FriendFeed the representation may include that thumbnail.

I support this because: 1. I have such a website. 2. And I believe that sites like Digg, FriendFeed, Twitter are more attractive and useful when they include little bits of color graphics. It's the same principle that guided Apple to create the Macintosh in 1984, almost 25 years ago, when the current form of user interface was character-based.

To support this format, include a <link> element like this:

<link rel="image_src" href="http://static.flickrfan.org/afp/thumbnails/2009/01/10/trwas2108818.jpg" type="image/jpeg" data-width="150" data-height="122" />

And some other metadata, as explained on the Digg site.

View source on this page to see a real-world example.

I tested it with both Digg and FriendFeed and found that it works, although there at least one place where FriendFeed could support it, but they don't, yet.

Update: James Holderness says this came from Facebook, and Digg says so on their site, but the page they link to doesn't seem to say anything about thumbnails. I searched and found a citation here, but he doesn't link to any Facebook spec. I think it's fair to say Facebook supports it. I'll test now. Result: Yes! Here's a screen shot of how they appear in my Facebook profile. So they must be added to the list, which I will do now.

Harry Truman probably would have liked Twitter

Fri, 09 Jan 2009 18:06:58 GMT

Harry Truman: "I never did give them hell. I just told the truth, and they thought it was hell."

80 characters, including punctuation.

It occurs to me this morning this is one of the most American ideas out there.

"If you can't stand the heat, get out of the kitchen," another Trumanism.

52 characters.

At dinner last night in a discussion about the future of the University of California Journalism School, I figured something out. Journalism has always been about the sources, but somehow we lost our way and focused on the reporters, the conduits, the pipes. Okay, so the way we move news from sources to their destinations is changing, but when it's all done, the news process yields the same result.

So this explains why a 140-character limit in Twitter is so in tune with our times. It teaches us how to summarize, to condense, and it rewards those of us who are good at it by making our ideas go further.

It also levels the playing field. Last night there was a 5.0 earthquake in southern California. Not a big one, but Twitter had the full story within a minute. Was there more to say, for cameras and analysts to pore over? Not this time. 140-characters was plenty.

Update: The buck stops here. 20 characters, including punctuation.

Thumbnails, rev #2

Fri, 09 Jan 2009 21:56:48 GMT

12/28/05: "One way to do something, no matter how flawed that way is, is better than two, no matter how much better the second way is."

Earlier today I revised a proposal from December for including thumbnail images in HTML. I am now revising that proposal again to be compatible with Digg's (undated) proposal, which I just found out about.

I'm not going to add another format when one exists that differs from mine only by the choice of a name (they call it an image_src, I called it a thumbnail).

Their format doesn't call for a type, or height and width -- I'm going to include that info and hope that Digg ignores it.

You can see an example by viewing source on this page.

To test, I've submitted the example to Digg. They ask if it's an news article, video or image, I chose image. A few steps later and it's submitted, and apparently they picked up the thumbnail. Looks good!

Seems like a done deal!

Update: FriendFeed does support this format in their bookmarklet. I tested it, and it works. Happy! That's already a lot of compatibility for less than an hour's worth of work. My photo site plugs into Digg and FriendFeed.

Bootstrapping thumbnails, revisited

Fri, 09 Jan 2009 20:11:28 GMT

Update: After posting this, I learned of a Digg-authored format that does the same thing, and I've adopted it. I'm leaving the rest of this here to form a record, but the format it describes is not implemented.

At the very end of last month I started a thread here about adding thumbnails to photo pages in such a way that web apps such as Tweetree, and others that can display graphics inline, can grab info about the thumbnail directly from the HTML of the page that the full image is displayed on. If we're going to have a future of graphics-capable Twitter-like services, or if Twitter itself is going to grok images, then thumbnails are not just nice-to-have but must-have. And a machine-readable way to link to them from the original HTML is needed.

My first impulse was to create a <link> variant, but to do so would have meant adding width and height attributes. I could have omitted them, but that's ridiculous -- all image processing apps have that information at-hand, and if you don't put it in the HTML, then the client has to do an HTTP reference to get the data, and that's a waste of bandwidth. Better to put the info in one place rather than have N clients fetch it for themselves.

However, Zach Beane objected that you can't just invent elements for <link>, so I took a different route and defined an XML namespace for a new thumbnail element. Later, Sam Ruby suggested that HTML allows for arbitrary attributes to be added where they make sense as long as their names begin with "data-". I didn't know about this. So I'm going back to the initial simpler approach and use a <link> element.

So here's the new template...

<link rel="thumbnail" type="image/jpeg" href="http://static.flickrfan.org/afp/thumbnails/2008/12/28/trpar2329681.jpg" data-width="150" data-height="87">

You can see an example by viewing source on this page.

Update: Over on FriendFeed, Paul Buchheit, one of FF's founders, says that Digg has a format that does most of what I'm doing here. There are some weird things about it, like why specify the title in a <meta> element when HTML already has a <title> but I think I have to go with the Digg format. The philosophy: "One way to do something, no matter how flawed that way is, is better than two, no matter how much better the second way is." It's a corollary to Postel's Law, being conservative in what you send. I asked Paul if FF is supporting the Digg proposal.

A better choice for Surgeon General

Fri, 09 Jan 2009 16:05:58 GMT

Our future President (11 days!) is said to be considering a famous TV doctor for Sur-Gen.

Paul Krugman explains why he's not a good choice, and I concur, but something has been bothering me about this, and an email from a friend helped nail it.

If you really want to turn things upside-down for the better, instead of a healthy young doctor, how about an older person who is not a doctor, who has health problems and has been treated by the system, someone who has actual experience being a user of American health care.

Then let the doctors and insurance companies and HMOs listen a bit. There's no doubt the other users would hear what this person says. (There's a scene in the latest Clint Eastwood film that illustrates this principle beautifully.)

I am not suggesting an average or ordinary person, not a Joe The Patient, not a knucklehead or idiot, rather someone with a life of accomplishment, a passion for living, but someone who hasn't lived the perfect life and paid a price, and maybe someone like the future President who saw a relative die sooner or suffer more because of deficiency in the system.

That would signal a very pragmatic change -- from health care defining an ideal most of us won't achieve, to improving or just sustaining the reality we make the best of.

Thanks to Ann Greenberg, a longtime friend and Berkeley neighbor, for the perspective-shifting email.

Palm Pre a possibility?

Fri, 09 Jan 2009 01:58:02 GMT

I left a comment on jkOnTheRun about the new Palm Pre that was announced today at CES.

First question: Can it tether?

That is, can it play the role of the Cradlepoint router I just got, and the Sprint EVDO modem plugged into it?

Are they going to be as locked down as Apple is with the iPhone?

When and how can I get one to play with?

This morning I couldn't imagine why anyone would even go to a Palm press conference, and now I'm on the edge of wanting one of these to try. I'm ready to get off my iPhone, I'm sick of the locked up mentality. If this thing pairs nicely with a netbook, I might just switch to it for a year or so.

The next step in the evolving netbook is the cellphone that pairs with it. Whatever it is it must be reasonably debugged both in software and philosophy. Apple has the software but their philosophy is totally up a creek.

Now I'm looking for some Palm Pre clipart.

Conclusion of the Feedburner latency test

Thu, 08 Jan 2009 23:33:57 GMT

It appears that it takes FB some amount of time to recognize a feed once its been registered, but that once it does, it's pretty close to perfect at caching a feed for 30 minutes before refreshing its copy from the original.

A table that reports on the test.

Notes on the test when it started are here.

Here's the original feed and here's the Feedburner version.

I looked for docs on how to ping Feedburner, but came up with confusing and contradictory instructions, none of which worked. They all got Java errors from the server. I tried pinging using their form and through pingomatic, neither of which had any effect on the latency.

I tried adding a <ttl> element to the feed, set it to 1 minute to see if that had any effect. I'll let you know.

Update: Apparently Feedburner ignores <ttl>.

Update: I turned the test off for now.

Measuring Feedburner's latency

Thu, 08 Jan 2009 16:56:36 GMT

Yesterday I listened to a Gillmor Gang podcast that focused on one issue -- how much time does it take Feedburner to reflect the changes in a feed they're hosting. Steve had some evidence that it was taking as much as three hours for it to reflect changes in his feed at techcrunchit.com.

Being an engineer, I wondered what was going on, so I constructed a test with a feed to see what Feedburner would do with it.

Here's the original feed and here's the Feedburner version.

Here's what my test does. Every minute it reads the Feedburner version and compares it against the original. If they don't match, it does nothing. When they do match, it notes the time in a log, generates a new version of the test feed and repeats the process.

I'm going to let the test run for a few hours and then make one change -- I'll ping their server when I create the new version.

And of course I'll report the results here when they are available.

A note: I ran the test overnight and got what to me are astonishing results. Feedburner never noticed the change in the original feed. Anyone who was subscribed to it would not have known there had been news. I couldn't believe this, I felt there had to be a bug somewhere in my test, and it could be that there is. That's why I'm re-running it this morning while I'm working and can keep an eye on it.

Update at 11:10AM Pacific: First results after running the experiment for almost 2 hours: It took the following amount of time for Feedburner to reflect a change in the original feed: 24 minutes, 31 minutes and it's still returning old results after 61 minutes. This is without pinging.

Update at 2:20PM Pacific: Here's a table that summarizes the results.

Friends Of Dave

Wed, 07 Jan 2009 18:31:55 GMT

A Twitter feed with the blog updates of 15 of my friends.

http://twitter.com/friendsOfDave

A dynamic list of the feeds.

.

This post is basically a test of the dynamic list. Let's see if the sucka works!

Update #1: It works. I have to add that having jkOnTheRun at CES makes it mostly unnecessary for me to be there. I've already learned about a new Netgear 3G router. They pretty much precisely care about the things I care about. Keep up the great work. Speaking of which, I had the opportunity to really use the new Cradlepoint router last night at a dinner party, and it works fantastically. Very fast. Super nice to just put the hotspot in the knapsack, turned on, nothing extraneous hanging off my netbook.

Update #2: I added Betsy Devine. I must add some more people I used to hang out with in Boston. The whole point of this exercise is to keep in touch with people I lose touch with. It's possible to do this if we have blogs, and some of them, like Betsy, do.

Update #3: http://identi.ca/friendsofdave

Update #4: http://friendfeed.com/friendsofdave

Friends Of Dave for Identi.ca

Wed, 07 Jan 2009 21:22:29 GMT

Just added a way for identi.ca users to follow the Friends-Of-Dave feed.

There's no other way to test it -- I have to push an item through and see if it makes it over to identi.ca in addition to the Twitter place. Let's see if it works...

http://identi.ca/friendsofdave

It does! Cooool.

Friends Of Dave for FriendFeed

Wed, 07 Jan 2009 21:59:43 GMT

Just one more service, I promise. A test to see if it works...

http://friendfeed.com/friendsofdave

It works!

Blog clip art

Wed, 07 Jan 2009 19:37:34 GMT

This is a post I've been meaning to write for some time, but this blog post, reviewing the interesting practices of bloggers finally got me off my butt.

If you've been reading Scripting News for a while surely you've noticed the graphics that often appear in the right margin of stories here. Sometimes they are directly related to the story, and other times only artistically. They are meant to invoke your own thoughts and feelings, to show you something about yourself. Whatever they are you can be sure that I found them interesting. Beyond that, what they mean is up to you. That's what art is about, always -- don't let anyone tell you otherwise. That's why artists cringe when people ask them what the art is about or say that a piece does nothing for them. They'll always come back and ask what it means to you, or say that nothing is something. They're not just saying it to be difficult (although people always think artists are difficult).

Anyway -- the point...

Many times the art I use is commercial, pictures of products. My suggestion is that the companies behind the products should make the clip art easy to find and re-use. Think of it as free brand advertising. Often it's amazingly difficult to find a clippable picture of a product. Examples. Every airline should have an iconic picture of an airplane with their trade dress, on a pure white background. TV sets or laptop computers should come with blank screens, making it easy to superimpose a picture of a dead relative or someone you want to make more interesting by putting them on TV.

The SEO and PR people are all over the place, so guys and gals -- get to work. Every brand should have great clip art for bloggers to use and re-use. It's free advertising. And you guys like free, don't you!

Some babies are destined for greatness

Wed, 07 Jan 2009 16:49:40 GMT

I met Nicco (center) on my first day at Dean For America in Burlington, VT at the end of the campaign for Iowa. Since then we've been friends, across generations -- and I've become friends with his lovely Morra, and his puppy Rascal (pictured at the left). I've always expected great things from Nicco, but that's nothing compared to the feeling I get about his newborn son, Asa Archibald Mele (who will be known as Archie, I hear). I've only seen him in pictures, and it's probably only through knowing his family that I sense the greatness in this young man.

Born on January 3 of the New Year, in Boston, a warm welcome to Master Archie!

Turning Twitter into my friend-feed

Wed, 07 Jan 2009 04:37:12 GMT

I was doing a little work on a tool I wrote in April 2007 that pushed RSS content to Twitter, and made a simple enhancement: instead of having a Twitter account reflect the content of a single feed, I made it reflect the content of an arbitrary number of feeds.

This let me do something I've been wanting to do for a while, but never thought of using Twitter for -- I set it up to reflect the content of my blogging friends, people like Doc Searls, Scott Rosenberg, Scoble, Sylvia Paull, Andrew Baron, NakedJen, Nicco Mele, Michael Gartenberg, Marc Canter and a few others.

As usual with experiments, I'm not sure if this is going to amount to anything, but I thought it was worth noting. The tool is twitterRiver.root, and the feed it's associated with is friendsofdave:

http://twitter.com/friendsofdave

You may of course choose to follow this feed if you find it interesting, and I will probably release the tool at some point in the future.

PS: Arrington and Calacanis will find it gratifying that this is an aggregation of blog posts not Twitter fire hoses. That's why it's possible to include Scoble alongside Andrew Baron and Scott Rosenberg, without drowning them out.

Julie and Julia

Tue, 06 Jan 2009 22:06:06 GMT

Just got an email from Andrew Grumet with an amazing story.

He writes: "Julie Powell, who blogged her way through a Julia Child book on blogs.salon.com. Then the blog got her a book deal and some minor celebrity. Now they're making a movie out of it... with Meryl Streep!!! (in the role of Julia Child).

http://us.imdb.com/title/tt1135503/

Chris Lydon did a podcast with Julie Powell in his pioneering 2004 series where he interviewed many of the early bloggers.