Scripting News

Checking out VPSes

Sun, 26 Oct 2008 09:40:10 GMT

I've been waiting for Amazon or some other large tech company to provide stability for hosting services. So I tried out EC2 this week, and I more or less understand what it does and how it works, and I'm confident that if I decided to go that way, I could make my public web presence work in their environment. But I'm not sure if I should do it.

If it were anyone but Amazon I wouldn't go for it. Buying a service like this isn't like buying a laptop or groceries. You're wholly dependent on the company you're contracting with. If they go out of business at the wrong time it could cost you a lot. Or how they deal with outages could matter a lot. I had an ISP flake out on me in 2000 in the middle of a big onsite meeting followed by a user conference. We lost a few months of forward motion, at least, in the week that our Internet access and hosting (all in the same basket) was down. A couple of years later, Exodus went out of business, and that's where we moved to after the Y2K outage. I always seek reliability and stability, but given the state of the economy you gotta wonder if any of these service providers are going to be around much longer.

If a company like Amazon did VPSes, Virtual Private Servers, I'd go for that right away. It's much more like what I'm using now, two co-located servers, but I hadn't been watching the prices, they're much much cheaper. I'm wasting a fair amount of money going the colo route. But I don't know any of the companies. This is where I could use some help from readers of this blog. If you use a VPS, which one, is there a consensus, one that's considered a no-brainer, that some larger entities depend on? No one wants to be the largest customer of an ISP.

PS: I need Windows VPS, not Linux.

Windows on Amazon, Day 3

Sat, 25 Oct 2008 17:27:38 GMT

I might be ~~Now I'm~~ getting somewhere.

I mostly paused yesterday, read docs, thought, tried to understand how the various pieces fit together, and I think I've got some of the basics down.

1. You don't do a lot of customization of your AMIs, you might tweak up some of the settings for the OS, but don't install too much software in there, because all changes have to be bundled and saved and that's a slow process, and you don't want to do that too often.

2. The work is done in Amazon EBS Volumes. Install your software there, it can be attached to any instance. They're analogous to a hard disk drive in cloud space.

3. Now I'm wondering about costs. I've got the minimal system. It costs 12.5 cents per hour. That means in a 31 day month it will cost: $93 and that's without any storage costs. I'm having trouble estimating how much storage this instance will use. I have a bundle saved in my S3 space, and it's huge. Obviously I'm paying for that. I'll pay something for at least one IP address. Another question for oldbies -- how much should I expect to pay on a monthly basis for the most modest possible server?

4. All the docs say you can't depend on an instance staying up, but how does it relaunch if it goes down? I can't believe that's done manually. How can you build a reliable web service if it goes down unpredictably? (Not that my servers currently don't crash from time time, they do.)

5. I'm guessing you need to access the SOAP interfaces from inside your running instance. I'm going to have to find a concise overview of the interface. I find WSDL to be really hard to parse, would much prefer the equiv of Unix man pages. Have to go looking for that.

6. Wondering if all this is worth it. It must be, it seems, cause so many developers are deploying systems in EC2. How will it compare to what Microsoft announces next week? That'll be interesting to see. I can't get away to go to their conference in LA, but I'll be watching the news as closely as I can.

Do you know what to do?

Sat, 25 Oct 2008 22:03:13 GMT

When you go to vote...

Do you know what to do if someone tells you you're not on the list?

1. Don't just walk away.

2. Only as a last resort should you accept a provisional ballot

3. Call 866- OUR-VOTE or go to 866OURVOTE.org to get information on where to vote and the facts on your right to vote. A trained team of advisors is available to help you resolve your problem.

Over 10,000 lawyers are dispatched to the polling places on Election Day.

That's all you need to know.

Is Twitter the next Netscape?

Sat, 25 Oct 2008 07:00:00 GMT

Fred Wilson in his famous answer compared Twitter to Google when it was a pre-revenue startup. A nice problem to have, for sure, but what if Twitter is more like Netscape than Google?

I was a web developer when Microsoft passed Netscape. They did it in a classic style, perfectly executed to take advantage of every door Netscape left open.

1. Netscape had left their Mac browser to languish while they focused on Windows. Microsoft, realizing that most web developers used Macs, produced an excellent Mac browser first, and worked closely with Mac developers to make sure their browser worked with the Mac software web developers used.

2. Netscape let anyone download their browser for free, but charged corporate users for the software. Microsoft's browser was totally free for everyone.

3. Microsoft fixed bugs, enhanced performance, listened to market and responded, did all the things a mature company that remembered its entrepreneurial roots could do. Netscape, being a disorganized, chaotic Valley wunderkind, did none of this.

Now, Netscape could have anticipated that Microsoft was going to do all this, could have kept up the investment on the Mac, made their software fully free, and become the first startup in history to be deeply rooted in everyone else's ego instead of their own. But all that would have been very very hard to pull off. In retrospect, you'd have to say that Netscape tried to own too much, became spread too thin too early. They probably should have narrowed their focus on something very valuable and defensible.

I thought of Netscape when I read this well-intentioned post by Alex Payne, who is single-handedly grappling with the most vexing of strategic problems on behalf of Twitter, without a clear model of the landscape of the market that's ahead of them.

The problem is this, how much of the flow of Twitter should they let outside their cloud and under what terms. You can see the promise to grapple with this in the last section of his piece, The Proverbial "Firehose."

So many things to say about this, but for now -- this reminds me of IBM's attempt to put the genie back in the bottle in the transition from PC-DOS to OS/2 in the late 80s. They wanted to shut down the clonemakers, Compaq, Dell, HP, etc, without losing their base of software. This opened the door for Microsoft to welcome all the clonemakers to their platform, Windows, and now OS/2 is only of historical note.

When one of the big guys competes with Twitter, they will do everything Twitter does, compatibly, and they will also offer a firehose without restrictions, licenses or approval. Twitter will have to follow suit, but then it will be too late, they will be following in the market they created.

Much better to get out ahead of it, narrow the focus, welcome the competitors, and reserve for itself the position of the naming authority. It will be impossible to unseat them from this position if they play it right. They can of course continue to operate twitter.com, and with a fully open firehose a bigger competitor might not even find a way into their market. Either way, Twitter must find a defensible posture, they've definitely staked out too much territory, they're spread too thin.

John McCain in October 2000

Fri, 24 Oct 2008 22:57:01 GMT

Full text in RSS?

Fri, 24 Oct 2008 23:28:44 GMT

Guardian: "Our feeds now contain the full content of each article so that you can take guardian.co.uk with you wherever you prefer to get your news."

First, I'm not aware of any other publication of the Guardian's size and stature that has gone this way, and I know some people will be excited about it and welcome it.

However I don't think full text makes sense in all circumstances.

I prefer the River of News approach, as exemplified by nytimesriver.com and bbcriver.com, and full text feeds wouldn't work very well there. If you want to skim the news quickly, from a large number of sources, the style favored by the Times and the BBC works better. I wouldn't want to see all news sources feel pressure to go the same way as the Guardian.

I think giving readers choice is the best way to go.

BTW the Scripting News feed has always been full text.

Windows on Amazon, Day 2

Fri, 24 Oct 2008 18:12:25 GMT

Yesterday's initial exploration of EC2 was a success. I have a server running in Amazon's cloud. It was up overnight. Still there the next day, exactly as I left it.

I was able to get Firefox installed and run a few performance tests of the net connection. Nothing that would blow you away, but enough for a modest server. There must be a way to up the bandwidth, but that's not a concern right now.

The next thing to learn how to do is: 1. Save my work so in case the server crashes I can restart it, and... 2. Reserve an IP address for the server, so I can map a domain name to it, and so that others can talk to the server over the Internet as you would any server. Eventually I will run scripting.com in the Amazon cloud, at least that's the plan right now.

As before, I'm just going to narrate my work here, and ask any questions I have here, and if they get resolved, provide the answers.

First question is whether the AMI that I create from my instance is private. I haven't entered any passwords into the Firefox install, so I'm not really risking anything. I guess I'm wondering whether I should install my server software in the AMI or in a disk image that gets mounted by the instance when its launched. My guess is the latter. Though I really would like to create a public AMI that has my server software built into it. This is one of the things I find really exciting, that I can create turnkey servers that other people can operate on their own, without me having to nurse the servers.

I put together a cheat sheet that summarizes the steps to creating an AMI from an instance.

I'm now bundling an AMI from my instance. I feel like I'm beginning to master the jargon!

Also bundling seems to be really slow. It's been stuck at 3% for about five minutes now.

Next frontier: Voting

Thu, 23 Oct 2008 07:18:26 GMT

With our guy way up in the polls, and less than two weeks to go, it's starting to feel pretty real right about now. Most Americans seem to understand how important this election is.

As usual there's 40 percent who are sure to vote one way and 40 percent who are sure to vote the other, and the decision will be made by the 20 percent who could go either way.

Me, I'm in the 20 percent. Sometimes I vote Republican, sometimes Democratic. About half the time I vote for the winner and half the time I vote for the loser. I'm no Missouri or Ohio, I'm far from a bellweather, more like a coin flip. So if you support Obama, don't take any comfort in the fact that I do too.

Now the campaigns start to focus on making sure their supporters actually go to the polls. If they do, it seems our guy is a shoe-in. But there's a nagging doubt in everyone who's voting for Obama, remembering the Gore-Bush election and Florida, where thousands of votes were "supressed" which is a fancy word for "thrown in the trash."

Sad fact: If the Republicans hadn't been so good at throwing out Democratic votes, Gore would have won Florida and would have won the election.

In 2000 our democracy walked up to the abyss and barely came back. I can't watch this election unravel over corruption, not this time. And not while there's still time to do something about it.

I'm just beginning to understand what's being done to prevent voter suppression. Apparently there's a huge effort here in California to organize lawyers to make them available to people across the country. If you've been asked to accept a provisional ballot (someone is challenging your right to vote), these people will help. I barely understand what's being done. But I'm going to find out what we can do if we feel we are being disenfranchised.

Salon: Where the GOP could get dirty.

I will have a hard time accepting McCain as President, should he win. But if he does, there must not be the slightest chance that the election was won fraudulently. That would not put us in a very good place here in the US and around the world.

The Hill: Police prepare for unrest.

Windows on Amazon

Thu, 23 Oct 2008 15:21:09 GMT

This is my next drop-everything project.

http://aws.amazon.com/windows/

I'll be running software there soon, Murphy-willing.

Mary Jo says Amazon is releasing this stuff today in anticipation of something similar from Microsoft next week.

I just took a look at the docs for setting up Windows in EC2 and geez, it's got a lot of weird hoops to jump through. I don't see why getting started has to be any more complicated than setting up a new Windows machine, which believe me, I have plenty of experience with. Then if I want to do more fancy stuff later, I can learn how to do it later. It seems like all the arcane stuff has to be addressed up front.

Command-line tools? Really? Oh please help. I haven't used a command line since I left MS-DOS in the early 90s.

Maybe Elasticfox circumvents all the command line stuff? I'll give it a look.

I've had mucho success getting Elasticfox set up. Not really sure what it's doing, but it sounds like it's doing the same thing as the command line tools but with a browser-based GUI which is what I'm kind of used to (I use the S3 Organizer Firefox tool to do interactive management of my S3 storage). Elasticfox is similar in approach to that.

Feedback to anyone at Amazon who's listening -- your docs tell you too much if all you want to do is get started. You should get the user to a Remote Desktop Connection window of a functioning server asap. I'm working through the Elasticfox docs, and they're telling me so much more than I need to know (even though this time I understand it cause it's in terms that I understand). Also I think you should steer people to this tool up front, not make it a maze they have to climb around to figure out what to use. This could be a lot easier.

At 11:54AM -- I've got my Windows instance running in Remote Desktop from a Mac. That was really cool. Elasticfox gets you all the way there. Very nice!

First thing to do -- install Firefox. Looking around, I've got 160GB to play with. Should be plenty. Two drives, C and D. C has 10GB and D has 150GB. Install Firefox on C.

Next I want to do a speed test, to see how fast the Internet connection is. That requires that I install Flash. When I try to download it I get a security error. I didn't know Firefox had anything like that. Not sure how to change that. Later... Turns out Firefox 3 respects security settings of Windows. Instructions are here.

Okay so I'm running the speed test now. It says my server is in Seattle. Hah. Makes sense.

Okay, I'm ready to take a break now. I consider this a success and I think I'll be using this service, probably starting right away.

Next thing I have to figure out is if the IP address of this server is persistent. I saw something about that in the EC2 FAQ. I'm sure I'll have a lot of suggestions for them. I'm quite excited about the possibilities of providing turnkey servers for end-users. I don't think there's any problem with this being an end-user service, their docs just need better organization, but there's nothing to say I can't write some new docs. Lots of possibilities here.

New Business Models for News

Thu, 23 Oct 2008 13:05:59 GMT

There's a "summit" in NYC for new business models for news today.

Through the miracle of the Internet, I can participate too!

Here's what I think...

Where ever you see a barrier that says that only a professional can do this job, don't just get rid of the rule, proactively break it. Make the pros compete on the same playing field as the amateurs. The time is running short to do this, soon the professional news organizations won't have anything of value to the amateurs nor will there be many pros left to compete.

Here are some examples.

1. Get amateur bloggers through the hurdles necessary to cover political events. Give them credentials that the campaigns recognize. Help amateurs get on the press lists of of the campaigns. (My experience in the 2008 election -- McCain and Obama ignored all requests, Clinton hit me up for money.)

2. Put an amateur on your op-ed page along with the pros. A regular columnist, not a guest column, so they can build up some sway, and learn how it works.

3. Take news reports from amateurs and run them through the same editorial process. Then you will have amateurs participating in the editorial process.

4. Open your newsroom! After all your layoffs, you've got plenty of space. Have a budget of 20 local bloggers who have press room passes. This means of course that you have to get to know some of the local bloggers.

Each of these steps will create context for interaction, places where assumptions are challenged, where the arguments can happen, so people on both sides can find out what comes next. The whole discussion, as clearly shown by the participant list (they call them attendees, that's another mistake) for the NY conference, has been between people on one side. No wonder they're not figuring it out!

Now, I'm afraid this advice would have worked a lot better ten years ago, but believe me, I was pushing for the barriers to come down ten years ago too.

Anyway, hope y'all have a great conference.

PS: There's live video!

Why I like netbooks

Wed, 22 Oct 2008 13:49:24 GMT

John Markoff quotes Steve Jobs. "We don't know how to build a sub-$500 computer that is not a piece of junk."

As with all Jobsisms, it's beautifully elegant, true -- and misleading. You have to read it very carefully.

He isn't saying no one knows how to build one, just that "we" don't know how to. Fine. And the last part is almost Republican it's so clever and nasty. He's not actually slamming Asus, Dell, HP, Lenovo, Acer and MSI, but if you don't read it carefully you might think he's saying they're pieces of junk. I think he's been studying Sean Hannity.

Now here's what Dave Winer, Mac user, says: They are not pieces of junk. Quite the opposite, they are elegant Mac-like products, and you can be absolutely sure behind the scenes Steve is throwing tantrums at his engineers day and night extolling their virtues and telling them to hurry up cause their lunch is being eaten. This is the same guy who said people don't want video on their iPods until he had an iPod with video.

Jobs then said that the iPhone could be seen as Apple's netbook. Hmmm. Maybe Jobs doesn't understand what's so appealing about netbooks. I suppose it's possible.

Look, iPhones are not and never will be netbooks. Just like writing for the NYT is not and never will be blogging (Markoff once said the NYT was his blog).

iPhones are too locked to be netbooks.

OK, I suppose it's time to say what a netbook is...

1. Small size.

2. Low price.

3. Battery life of 4+ hours. Battery can be replaced by user. Atom processor seems to be a requirement, those that aren't Atom aren't selling (and are apparently being discontinued).

4. Rugged.

5. Built-in wifi, 3 USB ports, SD card reader. It seems it must have 802.11n to be taken seriously.

6. Runs my software.

7. Runs any software I want (no platform vendor to decide what's appropriate).

8. Competition (users have choice and can switch vendors at any time).

As a Mac user I would very much like to see a Mac netbook. Yes, I know if I'm willing to hack, I can get Mac OS to run on one, but I have a hard enough time keeping supported hardware working.

On the other hand, Windows XP/Home is not so bad as long as it doesn't get infected with malware. So far I'm happy.

What I am using (the most frequent question potential netbook owners ask): Asus Eee PC 901, purchased in July for $600, now sells for $440. I took it with me to the DNC and it was the only computer i used. Now when I travel, I leave the MacBook Pro at home. Too heavy, too much computer to carry.

I've suggested elsewhere that it might be time to have a Netbook conference. I'd be happy to participate as a host, organizer, or speaker. There's an active community of bloggers following netbooks, and it's a happy cooperative place. It feels like the early days of the Apple II or IBM PC.

If Jobs is missing the excitement that would be a shame because it would be nice to have an Apple netbook, and no the iPhone is not a netbook. Not even close.

Today the MSI Wind really went back

Wed, 22 Oct 2008 23:55:19 GMT

I got the DLink router, and btw, it has a fantastic browser-based interface, best I've ever used -- anyway -- I carefully restarted all the machines after carefully setting up the router -- and then the moment of truth, boot up the MSI Wind. And the exact same thing happened. It took the router down! I don't know how it's even possible, but I repeated the drill and it did it again, and that was the end for me. I am not a professional hardware debugger, I'm just going to say I got a bad unit. I boxed it up and sent it back to Amazon in Lexington, KY. I still had more unwinding to do, because the Slingbox doesn't like the DLink, so finally now I'm back to my Airport Extreme, the router I forgot I had. It worked so well with everything but you-know-what. Kevin Tofel, who I respect enormously has nothing but praise for the MSI, but I paid my dues now, I'm going to take a deep breath and move on to other work for the rest of the week. Five days of futzing with hardware is enough!

Subscribe via Google?

Wed, 22 Oct 2008 15:59:38 GMT

I just tripped over a heretofore unknown feature of Google.

Theoretically, unless I misunderstand, and I probably do, if you "subscribe" to Scripting News, then you'll be more likely to see results from this site in your Google searches?

Oy it seems I have to add keywords to my RSS feed.

Google doesn't need keywords. That's the whole point of Google. That's what makes it great, why it works. It can find stuff without keywords.

This is 1/2 a great idea (I've been lobbying for something like it for years) but I'm not jumping through all these hoops.

This is wrong in so many ways.

Of course I could still be completely missing the point.

PS: There was no need to invent a new namespace for the feature they added to RSS, it already has a category element that does what their new element does. They made the same mistake Apple made with iTunes. If they had looked before they lept they would both be using the same element and their feeds would be interchangeable.

What does a recession look like?

Wed, 22 Oct 2008 16:19:29 GMT

Yesterday I went to lunch at one of my favorite local lunch places. 12:30PM on a weekday, the place usually is buzzing, but not today. I was the only lunch customer. While I waited another person came in. The place had an awful dead feel to it. I thought to myself: Okay this is what a recession looks like.

FriendFeed's new realtime API

Wed, 22 Oct 2008 13:20:01 GMT

Yesterday I got an email from Bret Taylor at FriendFeed saying that their new "realtime" API had been deployed publicly, so I quickly took a look, and found that, as with all other elements of their API, it's simple and easy to support. I immediately wrote glue to connect it to the OPML Editor.

FriendFeed: Real-time API (beta).

To understand how it works, consider a hypothetical web app. I live about 1/2 block from a bus that goes to the BART station. Suppose there's a web page that updates when the bus passes a stop about a mile from my house. I'd point my web browser to the page, but it wouldn't refresh right away, but when the bus approached, the page would upload, and flash some kind of message saying "Get out there Dave!"

That's how FriendFeed's realtime API works. Ask a question and wait for a response. You might wait a long time, minutes, even hours, and that's a good thing. When the event you asked about happens, you can act on it instantaneously. And instead of making thousands of calls asking "Is it done yet?" you make just one.

Now it may sound like a silver bullet, but like all things in computer design, there's a tradeoff. You have to keep a process running waiting for the answer, and over on the FriendFeed side, they have to keep a process running too. But it's probably a good tradeoff. And the performance is stunning. I tested it with my own FriendFeed account and the script running on my machine detects updates the instant they happen; unlike polling apps I have running against Twitter that sometimes take 2 or 3 minutes to detect a change.

I have a project that's been waiting for just this functionality, I hope to get to it after the election.

One more comment -- FriendFeed does a good job with the APIs. When asked, I recommend that other developers just do it the way FriendFeed does. I was able to get something running within a couple of minutes. Once I got that far, I'm pretty well hooked, but only had to put in another hour to complete the project. Considering that the goal of an API is to get developers to hook into your service, this feature, and the performance of the service which is also excellent, is all that it takes to get a chance at uptake. That's why I'll generally put aside other work when there's a new feature in the FriendFeed API.

Evolution

Wed, 22 Oct 2008 00:42:36 GMT

A second look at MSI Wind

Tue, 21 Oct 2008 19:52:47 GMT

Since today is a day of second looks, I've decided to hold on to the MSI Wind for a few more days. I have 30 days to return it, after all, so no hurry -- I guess.

The incompatibility is definitely with the Airport Extreme. After reporting that, I've gotten lots of clues from people here and on Twitter that indicate that there are serious compatibility issues with that router, I've even reported on one before with my Nokia N800 handheld computer (which I never managed to get working with the Apple router).

So, I'm willing to try another 802.11N router, even though I gotta wonder why people who make WLAN adapters don't debug their issues with Apple's (imho very popular ~~and well-designed~~) router.

I really like DLink USB hubs, do they make good routers too?

BTW, I did install the new drivers from the MSI site. I have the "RT2700E" hardware.

A second look at W.

Tue, 21 Oct 2008 16:09:32 GMT

Something Oliver Stone said about W. got me thinking.

"I really, in my heart, think that this guy's policies are going to be around for a long time. And my grandchildren are going to be talking about this guy Bush like the way they talk about Teddy Roosevelt, the way they talk about Lincoln, the way they talk about Washington. I think he's had monumental impact on the future of this country."

Weird. I wonder what he means. I don't think he's right, but he did get me thinking, as I said.

Even if W wasn't a great President, he did become President and that's not easy to do as so many can testify. What he had down, what he knew how to do, was how to get elected President. Because:

1. He has good genes, no matter how dumb you may think he is, he isn't.

2. He was deeply involved with his father's losing 1992 campaign (not just according to Stone).

3. He loved his father and sought his approval, as all sons do, but didn't feel the love coming back.

So he did what he knew how to do, arranged it so he would run for President. After winning, he had no clue how to be President or even what it meant. I bet his father didn't take him into his confidence when he was in office. And while his father wasn't a great campaigner, he did really know to be President, he had been around the office for 20 years before he occupied it, as did many of the people he brought in with him.

Another observation, some based on intuition, and other parts based on observation of the 2008 campaign. While Bush II probably couldn't have done it without Karl Rove, it seems equally true that Rove needed Bush to win. But Rove may have found his new protege, who's getting broken in on the campaign trial, Sarah Palin, as Bush II was broken in by his father. That's the nightmare waiting for us should Obama win, which of course is nothing compared to the nightmare that awaits us if he loses.

If the Republicans lose

Tue, 21 Oct 2008 16:33:16 GMT

Upfront caveat: I'm not willing to say "When the Republicans lose" but I am hoping I get to say "When the Republicans lost."

So if the Republicans lose on November 4, they will have to make a much more serious transition than they think they'll have to.

They've been winning elections by saying nonsense things to the electorate, things that are precisely true, even though they know damned well that when the people hear them on TV or via email or word of mouth, they're hearing something different from the precisely true things they say. And lately they haven't even been worrying about the line between true and false. They say things like Palin was exonerated in Troopergate, when the exact opposite is true, she was indicted.

Now the "elite" -- people who know how to read and bother to, know this is what's going on. The Repubs didn't care because they could blow by our opinion of them, and as long as they were winning, what could we do about it?

Now it seems the Republicans may actually lose, and if they do, here's the change: If they want to get heard, they're going to have to stick to arguments that are based on what's actually going on.

I know -- it's tough.

MSI Wind goes back

Tue, 21 Oct 2008 01:04:53 GMT

Apparently the problems I've had this weekend are connected to an incompatibility between the MSI Wind netbook and my Airport Extreme router. At least there's a correlation, when the Wind is on, the router goes down, quickly, within a minute or two. If I turn off the wifi adapter on the Wind, I can leave it on indefinitely and the router works without a hitch. Or if I turn the Wind off, everything is fine. Or if I return the Wind to Amazon. :-(

Another piece of data, my networking problems started at the exact moment that I first turned on the Wind.

Conclusion: The Wind goes back to the factory and I travel with the Eee 901, for now at least. It has none of these problems, the Airport Extreme likes it, and it likes the Airport Extreme.