John Sequeira: blog

'This looks good in practice. But does it work in theory?'

Fri, 09 May 2008 11:42:05 GMT

google-jstemplate is open sourced. (The link points to a slideshow-overview containing the quote above).

I've been pushing much more typically-server-side functionality down to the client in my web apps lately, but am finding a wall at some point when things like performance etc come into play. When this happens, I move the functionality back to the server (5 lines of jquery becomes 30 lines of perl or 100 lines of C#) where I have more control and predictability with things like caching/dom parsing etc.

I am very interested in learning from the google-maps folks who normally blow through this wall on a regular basis. [via Gregor]

see also Jemplate for client side templating coolness.

Abbreviated Parody of Paul Graham's 'Boss'

Mon, 24 Mar 2008 20:10:23 GMT

People aren't meant to have a spouse. I saw some married people in the cafe the other day and something seemed wrong. They weren't passionate. I have a unique perspective on human relations, having been an eligible bachelor and active user of various dating websites since the category's inception over 10 years ago. Long term monogamous relationships are especially hard on people. Relationships are passionate early on, so why bother having one for longer than that? Structurally, long term relationships often lead to kids. Kids are complex. And difficult to fund. Funding them with credit cards is stupid. For both reasons long term relationships should be avoided. Now, it's not your partner's fault. I saw lions and monkeys in Africa one time. Monkeys don't get married, and neither do lions. They're both happy, so people shouldn't either. Also, nonmonogamy is definitely something that can be learned. At AshleyMadison.com, we help married people come out of their shells and find other people willing to overcome the straight-jacket of the married/long term monogamous life. Having seen that happen so many times is one of the things that convinces me that being non-monogamous, at least in a small group, is the natural way for people to live. Watching previously faithful, married, and, frankly dull people get transformed into passion-filled 'players' makes it clear that the difference between the two is due mostly to environment — and in particular that the environment in long-term relationships is toxic to people. In the first couple weeks of working on their first affair they seem to come to life, because finally they're living hedonistically the way people are meant to. THE END Sorry, I couldn't resist.

Debugging My Long Distance Bill

Fri, 08 Feb 2008 12:21:56 GMT

I noticed my long distance telephone bills growing at a rapid clip, since I've been doing a lot more remote-tech-lead work than normal.

It occurred to me that there may actually a solution to the problem of excessive long distance bills. I set up GrandCentral's Address Book view as one of my IE home page tabs, and whenever I have to call the team I use that page's click-2-dial function.

When you click a phone number in GC, their telco provider first dials you, then dials the other party. Since there's no outbound call, so there's no cost. The quality so far has been great. There are occasional issues, but they're probably fewer than my cell phone for comparison.

This seems like a big deal ... maybe not because it's not scalable to the country (yet), but the recent announcement that the non ILEC's were implementing peering agreements for no-cost long distance brings us oh-so-tantalizingly-close to free long-distance for the masses.

Elastra delivers scalable OLTP. For Real?

Thu, 07 Feb 2008 12:26:00 GMT

Elastra now offers scalable OLTP via Amazon's compute cloud. Supported Db's include:

MySQL

Postgresql

EnterpriseDB When Elastra first decloaked late 2007, they were all about scaling read-only BI databases, which is the easy problem to solve and I wasn't very excited. This Enterprise DB press release, however, implies that they've got a point-and-clicky active-active-active-etc cluster technology, with automatic-versioning for ~ $400/mo (per server). Quote the release:

EnterpriseDB, the Oracle-compatible database company, today announced EnterpriseDB Advanced Server Cloud Edition, a version of the company’s flagship RDBMS that is built on the Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3) Web services. EnterpriseDB has selected Elastra, the world’s first provider of elastic relational databases on demand, as its premier cloud computing software partner. Elastra’s Elastic Database Technology enables EnterpriseDB Advanced Server to run in a virtual, highly scalable, cloud-computing environment

Wow.

Mono for VM's?

Fri, 16 Nov 2007 15:59:32 GMT

BEA recently announced a version of their app server optimized for deployment via virtualization. They figured that since the Java platform virtualizes/abstracts the OS anyway, why not abstract to virtual hardware and just skip the guest OS completely? Less attack surface, less s/w to manage/update. Wins all around, as it represents a collapse of the distinction between container-based virtualization and OS-level virtualization, combining the often-competing benefits of performance and isolation.

Of course, the argument in favor of collapsing OS and language runtime virtualization applies equally well to the .NET platform. It occurred to me that, while I have every expectation that Microsoft will eventually abandon their unsustainable license-every-vm-instance strategy to both app and OS licensing, it could take a long time. Novell, just as they've jumped into decoupling Netware services from a host OS, has an opportunity to reinvent something that looks a lot like Netware+NLM for the virtualization crowd: a mono app server that runs on a hypervisor. This too seems inevitable, and much more compelling than the current duct-tape-required approach to mono-flavored ASP.NET deployment. Although mono is addressing their web deployment woes by going FastCGI (an absolute must for ISP deployment, and long overdue considering FastCGI amazing cost/benefit), Miguel : consider going virtual as well. I think there's much more chance of mono becoming a viable enterprise deployment option if you leapfrog Microsoft with this, and honestly it's a far worthier dev milestone than playing silverlight catchup with moonlight. As I've said before, I love the ambition+audacity that mono represents. At the same time, I'm often pessimistic that it will ever appear on my radar, (is it just cygwin++?). I think getting ahead of Microsoft is the key, and until now I haven't seen how that is really possible. Here's hoping that it is.

jQuery -- Love it

Fri, 21 Sep 2007 14:20:14 GMT

I've been meaning to write about jQuery, a fantastic javascript library that Dossy turned me on to.

jQuery is a new-school javascript library that allows you to do fancy ajax-y special effects in a microscopic amount of code. What sets jQuery apart from other frameworks is it's heavy use of CSS and XPath selectors to determine which elements you want to work on (add events to, hide/show, apply styles etc). Since every project I work on nowadays has an XHTML or XML component, I've invested a lot of time getting good with both scheme's, and as such I had a pretty good running start getting productive with jQuery. jQuery has a perl feel to it, in that you can pack so much expressive power into either a CSS/XPath selector -- they're the regex's of the structured text world. Also perl-ly is that when you're done, you really do have just a handful of unobtrusive javascript code to maintain. Admittedly, not everyone likes tiny code but it suits my scripting-language-addled brain. The base API is concise, easy to keep in your head, and even better: so ridiculously small in KB (14?) the page tax is ~ nil.

If the concise API doesn't do what you want easily, or you're solving a more sophisticated UI problem that should have already been solved (tabs, dialogs, etc), then you'll appreciate that jQuery is even more perl-y (CPAN-y?) in that there is a thriving community of plug-ins you can cherry pick that solve common UI issues.

The recently released jquery UI project offers you a quality assured set of UI extensions that follow certain code guidelines, for those looking for the don't-make-me-cherry-pick, batteries-included approach offered by dojo/scriptaculous/yui.

There have been two occasions recently where I had to implement small tweaks to different web based enterprise application where it wasn't clear to me how to do it with using the vendor's supported API's, or if it was even possible. In both cases, I implemented the customizations by including the jQuery library in a global template, adding an onDOMReady event handler, and writing in some new DOM elements using jQuery in ~ 5 lines of code. Problem solved - fast. Greasemonkey-type short-cuts like this risk not being upgrade friendly, but when implementing something not supported by vendor API's you're usually off the seamless upgrade path anyway. In addition, doing it the vendor way at the Java/C#/php level would almost definitely result in more complex code, and much more of it, that would be harder to maintain/merge anyway. And finally, if the need arises, it's much much simpler to turn off a jQuery page hack by commenting out a single line unobtrusive event-handler-add than it is to undo a compiled or scripting language library modification. It's downgrade friendly.

I now reach for jQuery every chance I get. Take a look.

Lies, Damn Lies, and Benchmarks

Thu, 13 Sep 2007 15:16:47 GMT

From Curt Monash:

"2. Vertica’s software is 50X faster than anything non-columnar and 10X faster than anything columnar. Now, some of these stats surely come from the syndrome of comparing the future release of your product, as tuned by world’s greatest experts on it who also hope to get rich on their stock options in your company, vs. some well-established production release of your competitors’ products, tuned to an unknown level of excellence,* with the whole thing running test queries that you, in your impartial wisdom, deem representative of user needs. Or something like that …

lol. Well said, Curt.

Elastra -- To Infinite Database and Beyond

Thu, 13 Sep 2007 12:03:26 GMT

EC2-ISV Elastra decloaked recently. The website is so-so, but here's a quote blog comment from an employee:

Though a $360/month list price may seem comparable to a hosted solution, you are getting a lot more. Besides not having to buy hardware, software, bandwidth, or storage (we give 1 Terabyte of S3 storage free), you get the ability to really run your solution on demand.

Imagine allocating hardware to set up a 10 node cluster, setting up databases on it, having users or analytical programs hook up to it for a month of heavy traffic or heavy reporting. Imagine having to expand those clusters to 20 nodes or contract them to 3 because you are doing so well, or you no longer need the processing power. Now, think about being able to do these things at a minute's notice.

Imagine having 3 versions of your complete Data Warehouse for testing, running all three at the same time for, say, three days, picking out the best design, tools, analytical engines and taking the other two down by clicking an icon while archiving the other two in case you want to try them later.

The point with ELASTRA is not just that it's the first solution that actually uses S3 as a disk drive and not a tape drive, but that is also helps the application vendor create, scale, manage, and deploy their solution as their enterprise (hopefully) grows.

I know hosting people will scream that they can also add servers easily, but for the screamers: your competition is Amazon, not ELASTRA. We don't have anything against you if you can offer our platform the same capabilities and the same prices as Amazon can and so effectively does.

Sounds pretty cool if you want to run your analytics db apps on EC2. I haven't personally experienced the situation where there were enough BI users to warrant this. Usually BI apps target a handful of decision-makers, and versioning isn't so important with star schemas that capture historical information. Additionally, I don't think it's a stretch to say that BI implementations using open source db's will probably be located on the budget-constrained part of the spectrum, and have even less overlap with customers needing high-end availability requirements. Hmmm. It will be interesting to see if they find a market. This reminds me a bit of another startup I'd been meaning to mention: SaaS lab management provider Replicate Technologies founded by long-time blog-reader Ken Novak. Ken walked me through some of the scenarios that his team had been enabling, and they sound somewhat familiar to the elastra scenarios, except with a lab management focus on whole system- and network topology- versioning as opposed to database versioning and clustering. With replicate's technology, you of course create and version all the VMs and groups of VMs you like, but you also can simulate and capture network topologies that would otherwise be a big pain e.g. lossy WAN links if you want to test an Exchange remote failover scenario, or a huge number of low bandwidth connections for testing a sensor-web data collection application ( two actual use cases ). Replicate is not deployed on EC2 like elastra and 3terra etc - they've rolled their own virtualization platform. But if Amazon's plans to own the grid enabler space (continue to?) pan out I'm sure EC2 deployment will be in the cards.

Square Swapping

Fri, 29 Jun 2007 12:22:07 GMT

My office sublease expires tomorrow at the hawk's nest on the 18th floor of the old Thinking Machines building in Kendall Square. Last night I moved my officestuff to my new digs above Dewey, Cheetham and Howe overlooking Harvard Square (aka One CarTalk Plaza)

I liked Kendall quite a bit ( great office, river views, already miss it a little.. sniff), but Harvard is much hipper and studenty/lively compared to the officepark/biotech everyone's-gone-home-by-8pm canyons of Kendall. (Note to self: repeat this over and over and you won't miss the river views).

And as annoying and bittersweet as moves can be, there was one thing I looked forward to: repointing my GrandCentral number to the new office space for a seamless, ILEC-free transition. Location, you are so dead to me.

I too think Opera is a pretty good browser

Wed, 20 Jun 2007 20:19:25 GMT

I read a post recently by Jonathan Ellis that I've been meaning to write myself for a few months:

I've been trying Opera 9.2 for a week, and I'm pleased with it enough that it's going to continue to be my main browser. The main selling points for me are
* MDI weirdness is mostly hidden now, I hated earlier Opera UIs
* 20-30% less memory use; even after poking about in the guts of about:config to force FF's memory cache to the same 10MB that I gave Opera (which exposes this option right in the UI), Opera consistently uses less memory for the same workload. (Without adding this option to FF, it would max out around 400MB instead of 150MB.)
* feels snappier; opera seems quicker to start rendering something useful on slow-loading sites like 1up.com, although total render time is about the same. It's also instantaneous to open a new tab, which consistently takes around 1s on FF after I've been using it a while. I open and close tabs frequently...

My firefox2 memory issues have also moved me over to Opera. The only things I use FF for are google docs, and whenever I need to use the FireBug, WebDeveloper, or Selenium IDE extensions ( oh - DocuFarm is awesome too) or whenever else I hit an opera-unfriendly site. Okay, so I'm not exactly firefox free ... but I try to keep the tab count down to about 5 or so. When I have to do research and open up 10 pages of doc's / google results it's definitely Opera that I use.

I suspect it's not entirely FF2's fault, but the very extensions that I'm infatuated with are part (the cause?) of the problem. I installed the latest FF3 Alpha using Altiris (ditto Safari/Windows - I love Altiris) and while extension-free, it's pretty darn fast ( faster than ff2). Over the course of a day, the memory footprint is much better too. I'm hopeful the release reflects this, and I can winnow down my browsing choices, but until then I feel like it's not the worst thing in the world to better acquaint myself with cross-browser rendering behavior.

Google Gears Roadmap

Fri, 15 Jun 2007 12:55:47 GMT

MS's Dare Obasanjo harshes on the Google Gears beta because it doesn't solve the big problem of offline/online synchronization. That seemed like a reasonable criticism, because that is a hard problem, but I think it misses a couple of points.

Read-mostly applications -- ones you use for reference and just need a local copy of -- are pretty important. Having a local copy of your organization's CRM database is a valuable thing, whether or not you can write back to it. If Gears was used predominantly for this, I think it would still have a big impact on web architecture.
The mobile folks have gotten pretty good mileage out of SyncML for solving this general problem. In fact in a lot of ways online/offline browser apps are simply a recasting of connected/disconnected mobile apps. Right? I would be surprised not to see a Gears+SyncML hookup in the near future.

The conflict resolution issues (who over-writes who, when, and how) Dare points out are very application specific, and I don't see any generic technology-based solution addressing them. Those are much more likely to be addressed by convention ... and we won't establish those conventions until folks start bushwhacking down the online/offline web path.

Making dumb phones smart

Thu, 07 Jun 2007 10:56:59 GMT

I also have become a fanatical Jott user. Jott is a cool piece of software that allows you to call in from your cell phone (or any phone you set it up for) and record a message. That message is then transcribed and sent to you as an email or an SMS. It changes your voice to text. Jott has just rolled out some incredible updates where you can send a note, not only to yourself, but to other people who's email addresses or SMS #'s you have loaded into your contacts. Send a message to one person, or a group of people. It's super easy. And did I mention that both Jott and Gmail are FREE?!?

I'm not a fan of smart phones. I've been accused of being a phone-luddite, and that's probably accurate. There are a few services that might justify me going smart, but I've always held out hope that voice-enabled tech will make my old clunker do the fancy stuff.

The first service that gave me this idea was a dotbomb called something like qixo (google+memory is failing me. And no, it's not the travel site). After signing up for quixo[?], you uploaded your addressbook to them, and gave them your cell #. When you called them, you'd speak the name of the person you wanted to talk to, and a human being would connect you to them or ask for clarification. Say goodbye to remembering phone numbers - a neat idea.

It was free, and provided a couple years of Kozmo-like novelty while they lasted. But importantly they enlightened me to the idea that you could make 'dumb phones' smart. Jott is a service along these lines that I've enjoyed using. As mentioned above, they are a free voicemail dictation service that actually use humans to do their voice recognition. Your voice files go to India where they're transcribed and emailed back. I've found it to be a great way to capture things-to-do and keep faith with my favorite GTD tenet: ( either record the task or spend brain cycles reminding yourself to remember it).

It's such an endearing and useful service that it doesn't bother me that the voice recognition is absolutely horrendous. I don't think I've ever gotten a message transcribed without errors, normally horrible errors. Either it's my diction, my cell phone quality, or (as I suspect) the hiring situation in outsource-mad India is quite dire. Overall it's *much* worse than my experience with Dragon Dictate, circa 2001. But since I use it only once or so a day, I always remember what I meant to say and have no trouble decrypting the unintentionally steganographic transcriptions. And there's a weird b-movie kind of entertainment value to interacting with such a low quality service. Anyway, if you set your expectations really low and don't have serious privacy issues then I recommend it -- use it now before they run out of VC/angel money.

Aside: profanity is not transcribed by Jott - a friend who didn't know about the mturk architecture found this out. It wasn't me- honest :-) Update: 1-800-Goog-411 is an essential speed-dial entry for dumb phones. It's awesome.

Upcoming: Stonebreaker on 'Which of my databases you should buy'

Wed, 09 May 2007 18:23:44 GMT

Michael Stonebreaker's actual talk tomorrow (7-9pm) at MIT is entitled:

"One Size Fits All in DataBase Management: An Idea Whose Time Has Come and Gone"

And I hope to be there. I'm anxious to learn more about Streambase and Vertica, his event-oriented and column-oriented db progenies.

If you google for Twitter and scalability, you'll find a lot of articles on how Twitter's Rails setup has buckled under their obscene growth curve. People have pointed fingers at Rails, and Ruby, and various other things to explain these scalability issues. But I think that fundamentally they're using the wrong technology for what they're trying to accomplish.

I think their implementation of a messaging architecture on top of an RDBMS was not a great choice for their current volume, and I strongly suspect that some event processing platform like Streambase would be a better choice. I'm a complete event processing novice and I know less about twitter so they might already do this ( and their scalability issues are completely about rendering event streams to the web). But regardless I'm hoping this talk will give me a better idea of when to use messaging-oriented technology as opposed to the tried-and-true rdbms approach.

Cubulus: Open Source OLAP ++

Wed, 25 Apr 2007 02:28:07 GMT

Cubulus is a new kid on the open source OLAP block. Written in ~ 3K lines of python, it implements mySQL-based ROLAP w/calculated aggregations stored in memcached, and includes rudimentary MDX and distributed parallel query support. It sounds like an ambitious project ... and one I'll definitely be keeping an eye on.

also on John's Open Source OLAP radar:

Pentaho's Mondrian -- the original java-based ROLAP engine
OpenI - kitchen sink platform similar to pentaho
OpenOLAP - Big in Japan ( needs help on translation )
Palo - MOLAP store for excel et al.
LucidDB - trendy column oriented architecture
MonetDB - also column oriented.

Note that I have only very limited experience with the above items ... my clients always seem to have a licensed copy of SQL Server around, which gives them free (zero-marginal-cost) analytics.

Blikistan?

Wed, 25 Apr 2007 02:12:32 GMT

Blikistan is a blogging tool named after the small ex-Soviet country Blikistan. Blikistan (the software) features a magic engine which pulls all the blog configuration and postings from a Socialtext Wiki. Blikistan (the country) has a population of 32,768 people, and the population is fully literate. Blikistan's Government never meets in person, all communication and planning is done on a wiki. Blikistan (the software) is the only blogging software that allows the blogger to choose which language their blog will be powered by. The Magic Engine can be implemented in any language using Perl's Inline modules. In Blikistan (the country) taxes are filed online using SocialCalc. Blikistan (both the country and software) is completely paperless.

I did not know that.

Stateless Linux for the masses

Thu, 19 Apr 2007 14:33:27 GMT

Babeldisc looks like a consumer version of the virtual desktop/thin client/LTSP/etc concept. They use a Linux-based LiveCD (and soon USB fobs) to boot the OS, paired with networked storage for user data hosted at Rackspace (what... no s3?) It looks pretty cool ... but I wonder if they'll get killed on support costs for device driver troubleshooting. Photoprinter/wifi/etc headaches have consistently killed all my desktop linux livecd explorations.

Marathon Man

Thu, 19 Apr 2007 13:02:14 GMT

A friend from high school, Matt Simms, placed 186th in the recent Boston marathon. Congrats Matt! I haven't been in touch with Matt since he went off to Annapolis, but our extended families still live in the same small town and see each other pretty regularly. I learned that last year he won the Boston Marathon in Iraq while on tour there. His Baghdad time was 2:53 which was only 17 minutes more than his recent Boston time. The Iraq version was the normal 26.2mi run ... with the additional challenge of 115 degree heat. A-mazing.

Two Horizontal Scaling Tools for LAMP/LAPP

Tue, 03 Apr 2007 16:41:26 GMT

Here are two recently released resources for horizontal scaling your db tier :

Skype has developed and open-sourced PL/Proxy for Postgresql

Google has released Shards for Hibernate, to add horizontal partitioning support to the Java ORM library.

YAMB: Yet Another Mysql Backend

And I just ran across a reference to ... MyISAM++ at MySQL Camp Google Notes

MySQL guy: They’re releasing to MyISAM++ will do everything InnoDB does, but way better. Readers don’t block writes, etc etc. Different from Falcon. Both will be out next year. I guess technically it's not another backend, just a new rev of the old MyISAM mainstay. Still, I saw an April Fool's announcement of a postgresql backend for mysql (?link?) and it just doesn't seem so unlikely to me.

JeOS: Fractional Operating Systems?

Tue, 03 Apr 2007 16:32:23 GMT

JeOS is the most interesting part of the virtual appliances discussion because it provides a small system footprint and an automatic update mechanism, simplifying work for ISVs wanting to provide a secure and reliable solution.

I was wondering when this would arrive. JeOS seems like it might remove the distinction between openVZ-like os partitioning and full-system virtualization (VMWare). In addition, it's also an important first step in shrinking both the attack surface and the management load of the fractional machines to match their fractional service delivery.

Big Time LAMP

Tue, 27 Mar 2007 14:06:10 GMT

Excellent talk on state-of-the-art LAMP scaling techniques by Ask Bjørn Hansen. This comes at a good time for me, since I'm working on deploying several LAMP clusters.

Making Phone work like Email. For Free

Fri, 02 Mar 2007 13:40:13 GMT

I've been waiting for a couple of services that help make my phone work more like my email. Thanks to Lifehacker, I found out that two of these are now available for free.

The first is GotVoice, the phone equivalent of fetchmail. The service dials into your cell phone voice mail account, navigates the phone tree, plays and rerecords your messages, and then sends you an email notice of this fact. I sometimes leave my phone turned off at work, and don't always get the voice mail notification in a timely fashion (ditto when traveling and/or in areas with poor reception). At the same time, no matter where I am, I do manage to check email. So GotVoice let's my compulsive-email-checking make up for my non-compulsive voice mail checking. Nicely done.

If you pay GotVoice they'll email you the voice mail, but otherwise you go to their site and it plays in a flash widget. I'll take the ads to get the unified inbox - thanks GotVoice.

The second is GrandCentral -- which is the phone equivalent of pobox.com. They provide you with a number which you then simply connect to multiple other numbers (home/office/cell) that you want to ring when someone calls your GrandCentral number. They too implement voice mail, and will email you a notice about it. You can define times when not to ring your home number, and customize by times (don't call home during the day) and person (send ceiling insulation salesperson to /dev/null ). Lots to play with.

What's the biggest appeal for me? Almost all of my billing-related-calls to my phone or cell provider leave me gnashing my teeth, and wishing my switching costs were much closer to zero than they are. Grand Central lowers the switching cost by being the one number I can hand out to people, decoupled from where the phone actually rings. In addition, I get further inbox unification and find-me-follow-me -- thanks Grand Central.

GotVoice Aside: GV is a very high-touch application at the moment, sending you an ad-laden email a few times a day that no voice mail was found, so writing a quick mail filter to strip the out negative results is time well spent.

GotVoice Aside #2: Stephen O'Grady recommends CallVantage -- ad-free GotVoice-like functionality, so you can skip the mail filter. They do not support SprintPCS at the moment, so they are dead to me. Google for CallVantage etc. on Lifehacker for a decent competitor comparison, if you're in the market for voicemail++.

DownloadableOnDemand Data Center Vendor: QLayer ?

Mon, 12 Feb 2007 13:42:23 GMT

In an email response to my post on Amazon's EC2 offering, where I suggested it wouldn't get interesting until the next layer of cluster management/deployment was rolled out, an observant blog reader pointed me to QLayer - a vendor of a data center on demand service, that sounds suspiciously similar to 'something that would make Amazon EC2 extremely compelling'. QLayer manages

OS images (QImage),

storage (QSan),

network topology (QNetwork) and

replication for deployment/failover (QReplicate)

so that you can instantiate a VM-based data center topology by clicking some buttons on a web page ( I think ). Viewing their flash intro, they even have a control panel slide with checkboxes for "Click here for Mail Server, Click here for Database". Awesome. Lazy sysadmins (and developer's who are occasionally press-ganged into system administration like myself ) can rejoice. The Belgium-based firm raised $9M last week in a second round of funding -- they are definitely someone to keep an eye on. Related Thought #1:: the missing piece in all this is some sort of meta-distribution or vendor that can certify/support clusters that work in harmony - making sure all the ports/network address/authentication/file paths etc line up. Clustered apt-get? MetaYum? I haven't even read about anything like this, so please let me know if you have. I know that there are grid management tools, but I don't think they cope with the heterogeneity of a data center (vs the clone-army regularity of a compute farm). It's possible that you have to go to someone like Opsware and sign away your first child to get this, but I'm hopeful for something simpler/developer friendly. Related Thought #2: It would be nice if Windows were invited to this party ... but consider the licensing complexity of using non-OSS software in the above scenarios: Instead of 'Check here for Mail Server' it would probably take you several hours to figure out what combination of Exchange+AD+w2k3 Server CPU-based licenses, per -seat or device access licenses, a-la-cart vs. silver level support contract vs gold level, etc ... ARG! That's not where you wanted to go today.

You want to run what?, on what? on what? (Amazon style)

Fri, 26 Jan 2007 21:15:18 GMT

I found instructions for running Windows images in QEMU on Xen on (presumably) Linux. Phew. This reminded me that I've been meaning to post about Amazon's new grid offerings s3 and ec2. I get Amazon's s3 storage-on-demand -- I think it's great and I'm actually using it to store older database backup files for a production server. IIRC, Amazon is charging a tenth what our hosting provider charges for local SCSI/RAID disk space. The s3 tools are way cruder than I'd like (even jungledisk's webdav->disk mapping is a bit hacky), but I see the potential. I'm having trouble wrapping my head around EC2, though. I'm spoiled by desktop virtualization, and my clients are starting to offer quick virtual hosting of servers on their intranets. So, when/why would having a server that runs for a fraction of a day/month/whatever make sense? I know it's exciting because it's new and all, but I just don't get it and I'm not sure who the target audience is. I suspect I'll have to wait until folks start building apps on top of EC2 before it gets on my radar. Update: Ten minutes after I posted this, it did occur to me when/why you'd want to choose EC2. Amazon is in the process of rolling out something similar to VMWare's Virtual Appliance directory with the difference of offering a rapid deployment/hosting option. These appliances are somewhat interesting to imagine running on a per/minute basis, but considering how easy it is to download to your desktop it's not too great. It begins to become really compelling once you move beyond individual machines and into the concept of what's now termed "Lab Management" - when you have a testing environment that consists of many machines acting in tandem, and you need to build up the cluster, test it, and tear it down and restart it , many times and in many different configurations. Covering your test matrix for distributed applications/SOA is hard, and Lab Management is ridiculously easier than the alternative. Lab Management will remain inside the enterprise, but the value and direction of EC2 is in applying the testing-focused Lab Management concept to production user-facing applications: what I like to call the Downloadable Data Center. Why is this cool? Well, consider the difference between your typical startup and a mature web enterprise: to really run a web hosted application according to best practices, you should have

staging setup
production setup,
hot standby, DR plan
version control repository/bug tracker
integrated authentication
distributed file system
load balancer
firewall/intrusion detection
etc.

And no one does initially because it takes a lot of time, money and expertise to put all these pieces in place. But what if you could have it all initially and it didn't cost an arm and a leg? The idea of a vendor (like, say Novell or RH) pre-provisioning all the machines required to pull the above off, and offering them via the Amazon EC2 Control Panel is quite compelling. Imagine the options:

Statefull Firewall with mod_security? Check.
Dedicated Image Server pre-configured with optional Akamai CDN support? Check.
Web analytics reporting server? Check
Offline bi/olap database with real-time replication? You get the idea.

Each check on that control panel is the equivalent of days or weeks of work on your hand-rolled data center. Accordindly, enabling composable application clusters like this would be worth much more than $60-70/mo/machine. EC2 ISV's like Novell would clearly be due some royalty or rent on top of the EC2 grid-dial-tone costs, in addition to their traditional subscription/support revenue. In the near term, it's hard to imagine shifting from a hand-rolled mini data center to a commodity one because of all the weird hacks that evolve. You could call this "The ActiveGrid Challenge" since that product, which sniffs around the same space, has essentially identical adoption barriers. Over time, as developers become aware of the prefab/click-to-deploy options and start coding to them, however, it's hard to imagine it not taking a good chunk of the hosting marketplace. To give credit where credit is due, Marc Andreeson had this idea with his LoudCloud startup. For the small fee of $100K/month, they would professionally provision and maintain your data center. I liked the idea, but thought that although it sounded like a nifty escape valve for harried CTO's, they'd probably run of of customers pretty quickly at that price, and they did. I'm betting the same fate does not await EC2 much more mass-market effort.

Qumranet ?

Fri, 26 Jan 2007 19:03:36 GMT

For some reason, I went looking for Moshe Bar's weblog. I've been a Moshe fan since the Byte days, and it just occurred to me to find out what he's doing now. I found this on Wikipedia

Moshe Bar is the founder, main developer and project manager of openMosix. Moshe is founder of the company behind the Xen software, XenSource, Inc. Moshe also co-founded Qlusters Inc, an open source systems management software company with headquarters in Palo Alta, California and offices in New York City and Tel Aviv, Israel (also see openQRM).[1] Moshe is also founder and CTO of the company Qumranet which is behind the development of the KVM virtualization technology in the Linux kernel.

Moshe did a stint at XenSource as CTO ... From what I had heard about KVM, I assumed it was just a lone hacker project like QEMU. I didn't realize that the project looks to be a Xen competitor with it's corporate sponsorship and XenSource-like venture backing. I wonder what they're up to (beyond a compelling-sounding, legacy-free VT-based hypervisor)?

H2 Database v 1.0

Fri, 19 Jan 2007 13:46:15 GMT

The H2 Java Database went version 1.0 this week. I like the provocative benchmarking chart - every db project should have one of these ( and they all probably could for the right workload ): Now I don't really have much need for an embedded Java db but I thought this history quote was quite inspiring:

The development of H2 was started in May 2004, but it was first published on December 14th 2005. The author of H2, Thomas Mueller, is also the original developer of Hypersonic SQL. In 2001, he joined PointBase Inc. where he created PointBase Micro. At that point, he had to discontinue Hypersonic SQL, but then the HSQLDB Group was formed to continued to work on the Hypersonic SQL codebase. The name H2 stands for Hypersonic 2; however H2 does not share any code with Hypersonic SQL or HSQLDB. H2 is built from scratch.

The engineer who couldn't stop making databases - whether as an employee or on his own time. Kudos Thomas.