John Sequeira: blog

The Black Swan: Living in a power-law world

Mon, 06 Oct 2008 13:09:00 GMT

Taleb's "The Black Swan" is a recently read book that describes in great detail one of the underlying phenomenom of the current credit crisis (via Gregor):

Our brains are wired for narrative, not statistical uncertainty. And so we tell ourselves simple stories to explain complex thing we don't--and, most importantly, can't--know. The truth is that we have no idea why stock markets go up or down on any given day, and whatever reason we give is sure to be grossly simplified, if not flat out wrong.
Nassim Nicholas Taleb first made this argument in Fooled by Randomness, an engaging look at the history and reasons for our predilection for self-deception when it comes to statistics. Now, in The Black Swan: the Impact of the Highly Improbable, he focuses on that most dismal of sciences, predicting the future. Forecasting is not just at the heart of Wall Street, but it’s something each of us does every time we make an insurance payment or strap on a seat belt.
The problem, Nassim explains, is that we place too much weight on the odds that past events will repeat (diligently trying to follow the path of the "millionaire next door," when unrepeatable chance is a better explanation). Instead, the really important events are rare and unpredictable. He calls them Black Swans, which is a reference to a 17th century philosophical thought experiment. In Europe all anyone had ever seen were white swans; indeed, "all swans are white" had long been used as the standard example of a scientific truth. So what was the chance of seeing a black one? Impossible to calculate, or at least they were until 1697, when explorers found Cygnus atratus in Australia.

(from the amazon page) I believe we learned everything we needed to prepare for the current crisis with the Long Term Capital Mgmt failure (risk model failure during market-wide sell-off; fund too big to fail thanks to positions in complicated unregulated derivates, failing; govt bailout) which the book discusses at length. And we have addressed none of the risks behind network failures in the intervening years, as interconnections have grown.

Here's hoping that the fix (and I don't mean the bailout) addresses systemic issues this time around and we don't just bury our head in the sand like we did last time, pretending to deal/regulate 1st order relationships while ignoring 2nd order+ effects of the financial web.

Update 1:

a better quote from the actual book...

"Globalization creates interlocking fragility, while reducing volatility and giving the appearance of stability. In other words it creates devastating Black Swans. We have never lived before under the threat of a global collapse. Financial Institutions have been merging into a smaller number of very large banks. Almost all banks are interrelated. So the financial ecology is swelling into gigantic, incestuous, bureaucratic banks – when one fails, they all fall. The increased concentration among banks seems to have the effect of making financial crisis less likely, but when they happen they are more global in scale and hit us very hard. We have moved from a diversified ecology of small banks, with varied lending policies, to a more homogeneous framework of firms that all resemble one another. True, we now have fewer failures, but when they occur ….I shiver at the thought."
— Nassim Taleb, The Black Swan (2006)

A pretty geeky article summarizing the book by the author

A funny-in-hindsight Forbes article from 7/2007 questioning the premise of the book:

Q:Is A Black Swan In The Way Of The 14,000 Dow?
A: Yes. Yes it is.

Loving Google Desktop

Mon, 06 Oct 2008 12:49:50 GMT

I really love the quicksearch function in the latest google desktop:

"The fastest way to search the web, and your own computer, is to use the Quick Search Box in the center of your desktop (simply press the "Ctrl" button twice to call up the search box, and press "Ctrl" twice again to hide it). Just type a few letters or words into the search box and your top results pop up instantly. You can also use it to launch applications without having to surf the Start menu; for example, you can launch Microsoft Word by typing "wor" into the Quick Search Box and selecting "Microsoft Word" in the list of results that appears. Or, simply type in your search term and press "Enter" to search the web."

I realize app-launchers is not a new thing, but I stay away from utilities like that because I switch desktops too often : it's annoying to rely too much on any customizations or desktop tweaks and few stand the test of time. Google desktop's search-web+gmail+desktop at the same time is too damn useful, but letting my launch apps speeds things up a lot.

[WindowsKey][WindowsKey]"CPAN" -- install new perl module using Strawberry perl cpan shell

[WindowsKey][WindowsKey]"Quicken" -- enter check from client etc.

Jonathan Ellis on AppEngine: Meh

Fri, 05 Sep 2008 11:27:53 GMT

Jonathan Ellis writes something suspiciously similar to what I think I would write about App Engine (but he actually went through the trouble of developing with it):

Having been eyeball deep in App Engine for a while, I've reluctantly concluded that I don't like it. I want to like it, since it's a great poster child for Python. And there are some bright spots, like the dirt-simple integration with google accounts. But it's so very very primitive in so many ways. Not just the missing features, or the "you can use any web framework you like, as long as it's django" attitude, but primarily a lot of the existing API is just so very primitive.

I can't get past the fact that there's no SQL. I know why there's no SQL, (Brewer's Conjecture etc) and I have some experience with it. I've been doing years of mostly-simple mashup stuff with quickbase, which is a similar PaaS-backing non-relational data store. It has it's place, but it's hard to imagine really cranking out something complicated with it. When I do code reviews for clients, the biggest improvements come from transforming procedural cursor logic back into set-oriented db-logic, and AppEngine wants you to go the opposite direction. Sigh.

Looking at the bigger picture, I'm trying to figure out what the drivers for PaaS adoption will be, especially in the case where the flavor of PaaS requires retiring your db muscles. Is giving the ability for developers (or non-developers) to clone and enhance code without sweating enough? Maybe there's a non-trivial demand for this (suggested by IBM's Project Zero, or stretching it a bit, MS-Sharepoint), but it seems like a long-term bet.

Aside: a cool App Engine project is Ben Adida's open source, cryptographically secure, online voting project Helios. You can conduct auditable elections and make sure they election-holders counted your vote the way you wanted. The auditing can be done by you personally, or a group that you trust. This all happens while hiding the actual candidate or choice you voted for, so you can't be coerced or sell it. Go Ben! The world needs Helios (but I'd be happy to start with just Florida).

Facebook Scaling

Fri, 22 Aug 2008 13:31:48 GMT

From Kellan:

Interesting to me was noting that:
they just got around to this 8 months ago, and they're fscking Facebook (which means you can wait)
they're still doing all writes to a single datacenter
they're hacking an object-level mark/sweep into the MySQL replication stream suggesting a certain parable of a hammer and nails.

Lol. Takeaway #3 is my favorite. Regarding the other two takeaways, I think Moore's Law will ensure the simplicity of single-datacenter and/or onsite deployments will be a compelling alternative to the clouds for quite some time. In other words, use Google App Engine et al if you like the cost and API's. For the distributed scalability, YNGNI.

The Paradox of Choice

Tue, 24 Jun 2008 10:38:45 GMT

The Paradox of Choice is a book with a thesis that's stuck in my head since I read about it a few years back. Simply put, more choice doesn't necessarily make us happier. The video and book gives some hard-to-dispute examples and data to back this up.

The thought that chases me is how much of technological progress is done in pursuit of choice/options/etc, without, as the book makes clear, accounting for the aggregate tragedy-of-the-commons cost of these options. The paradox is that someone somewhere will benefit (immensely?) from the choices technology produces, and there's no easy way to say when enough is enough.

Should we go back to 4-5 channels of TV? Laughable, right? Is it possible we'd be happier if we did? Hmmmm. That's just one way the book starts messing with your head.

The book repeats the central idea many times ... it's a simple enough that you get it well before the end, but I still found it worthwhile to ponder for the duration of the book. I would suggest that you pick up a copy, but that would only be adding to your reading choices (see how evil it is?)

Quick introduction on 'paradox' by author:

[via Stefano]

Roll Your Own Netflix

Mon, 23 Jun 2008 20:46:58 GMT

more praise for 'discontinuous fractional ownership'

In response to Props to Bookmooch, Replicate Technologies founder Ken Novak writes:

Hey John, this sounds good. For dvds, cds and paperbacks, zunafish.com works well (albeit not quite so karmic). For $1 plus $2 postage, the site sets up swaps. I own about 20 dvds, and I gradually swap them after I watch them, so that about 3/4 are ones I haven't seen. It's like netflix, with a bit less choice online but more DVDs at home and no fixed fees. Now if I could only acquire the time to watch everything I'd like to watch... #

I like the idea ... after getting over the initial Netflix rush during which the near-infinite film choice collided with finite movie watching time, I opted out and tried dvdswap.com. The experiment didn't last long, mainly because I had a big problem with liquidity. At the time, there just weren't enough swaps to be had. Now I just buy on half.com and resell on ebay. The buy/sell spread is usually about $4/movie (includes postage), which is the price I pay at my local video store, except no late fees, costs you nothing if you can't get to it, and no monthly commitment. I use Half.com because bidding on ebay is too much work and the selection of used inventory is pretty good.

Anyway, Bookmooch is incredibly liquid - I think the inventory is > 2 Million books and growing very rapidly. I think 'bigger than amazon' is definitely a possibility.

Props to Bookmooch

Fri, 20 Jun 2008 20:24:43 GMT

I'm enjoying the bookmooch service quite a bit. It's kind of an honor system based inter-library loan, except you aren't loaning books, you own them free and clear. And it's not between libraries, but individuals. But other than that, it's exactly like inter-library loan. :-)

Here's how it works: You put in a list of books gathering dust on your shelf, and a list of books you'd like to read. When there's a match between have's and want's, bookmooch shoots you an email to send or receive the book. You pay postage on outgoing, but the incoming books are free.

It's a treat to have a free book show up in your mailbox ... netflix has that same psychological edge over the local video store, except in the case of netflix they only want you to think it's free.

They keep an upload/download ratio just like the BBS's of yore, but I haven't quite figured out the reputation mgmt system yet. I do know, however, that you can donate your accrued book-donating karma to hospitals so they can get more book donations for patients. How cool is that?

So far I've sent 8 and received three, and I recommend the service to everyone I get within earshot (apologies to friends). Give it a shot.

People Powered Transit

Tue, 17 Jun 2008 13:52:04 GMT

My friend Andy launched a startup about 6 months ago, and has been getting lots of press in the last week. He's a bike nut, so that's always what we talk about when I see him : the crazy cost of the big dig and the missed opportunity for public transit, the cities without cars exhibit at the MIT Museum, Boston's legendary bike-unfriendliness, wouldn't it be great to work as a bike messenger instead of behind a desk, yada yada. He decided to make a business from his passion for bikes, and launched the New Amsterdam Project, a bicycle delivery service.

I confess that I was skeptical about the business plan when he described it to me a year ago or so. Somehow I thought he would be doing food delivery with bike messenger overhead ( the bastard son of Kozmo and Webvan perhaps? ). But I'm glad to see that's completely wrong.

It clicked for me when Andy mentioned, "We're really a trucking company" and described some of his first customers: delivering local produce to restaurants and farm-share subscribers. It's all scheduled recurring delivers, mostly business-to-business. He replaces trucks with bikes - as simple as that.

Anyway, I get it now. The zero-carbon-footprint approach to deliveries, making local produce workable, taking cars/trucks off the streets -- it's all incredibly timely with gasoline breaking the $4/gal limit. The MSM has picked up on it : he was on NPR Morning Edition last week and one or more 24 hour cable news networks sometime in the next week or so. And with every media mention, more businesses realize they can somewhat painlessly redirect necessary and frequent spending in an environmentally friendly way.

I believe we can make cities look very different than they do now, and make them work better for the cars and people whose coexistence is quite strained. Some of the change will be driven by technology, but a lot can happen with old-school determination and guts. Andy has the latter, and I wish him luck.

Coverage

Good blog post from MatterNetwork

Somewhat goofy Dutch TV station interview on YouTube:

http://www.youtube.com/watch?v=Am3CeGutnaA

p.s. and feel compelled to note that yes, Paul Graham would be proud.

3 Cool things from Microsoft now in my toolbox

Mon, 09 Jun 2008 19:27:23 GMT

LINQ - I wrote my first LINQ query last week. My normal web app works with heterogeneous data sources ( xml/object/sql ), and I *know* I'm going to love moving up the LINQ learning curve. This is a no-brainer, but it's going to take a long time before clients are ready to move from .NET 2.0, so I'm grateful for any chance I get to obtain LINQ-fu
LogParser - a command line tool with knowledge of various log formats, supporting a sql dialect. The size and scope of the tool might tempt you to pass for something simpler: don't. Think of it as LINQ for logs, packaged into a neat executable.
Powershell - I love reducing data integration challenges to a batch script. It makes it so much easier to troubleshoot when moving scripts around between environments ( big tools and compiled code have their place, but they tend to be the enemy of getting done on time.) I've done a ton of vbscript/jscript and plain old DOS batch scripting, and occasionally I've had to resort to activeperl. But now it's time to retire all of these approaches. Powershell's pipeline has thoroughly baked into it my favorite perl idioms : map and grep, but these are now object-aware, shell,file system, Win32 and CLR integrated. It is fantastic.

There's a fourth cool thing that kind of belongs on the list above, but it's only CTP right now: Code Name Velocity - Microsoft's version of memcached. I tried out the Windows version of the original memcached for a client recently, and it's one of the only server-based pieces of software I can recall where deployment of the software took less time than deployment of the virtual machine:

Unzip, double click ... deployed.

Copy/paste, have ASP.NET code writing to a pool of 2 memcached instances. After Velocity gets through paying the Microsoft tax, supporting integrated authentication/ad, Live Mesh/Sync Framework, MS Cluster, dcom/XA transactions, etc will it still be MS's answer to memcached? Or just another reason to use memcached? I'm wondering

'This looks good in practice. But does it work in theory?'

Fri, 09 May 2008 11:42:05 GMT

google-jstemplate has been open sourced. (The link points to a slideshow-overview containing the quote above).

I've been pushing much more typically-server-side functionality down to the client, but am finding a wall at some point when things like performance etc come into play. When this happens, I move the functionality back to the server (5 lines of jquery becomes 30 lines of perl or 100 lines of C#) where I have more control and predictability with things like caching/dom parsing/same-origin-policy etc.

I am very interested in learning from the google folks who normally blow through this wall on a regular basis. [via Gregor]

see also Jemplate for client side templating coolness. From the most recent Jemplate changelog:

---
version: 0.21
date:    Mon Apr 28 12:44:13 CST 2008
changes:
- Robert Krimen provided massive improvements including:
  - jQuery support
  - YUI support
  - Many new command line options for fine grained control
  - Doc changes
  - Much refactoring
- Ingy added:
  - A standalone Jemplate compiler (bin/jemplate)
  - Some doc: (bin/README)

Abbreviated Parody of Paul Graham's 'Boss'

Mon, 24 Mar 2008 20:10:23 GMT

People aren't meant to have a spouse. I saw some married people in the cafe the other day and something seemed wrong. They weren't passionate. I have a unique perspective on human relations, having been an eligible bachelor and active user of various dating websites since the category's inception over 10 years ago. Long term monogamous relationships are especially hard on people. Relationships are passionate early on, so why bother having one for longer than that? Structurally, long term relationships often lead to kids. Kids are complex. And difficult to fund. Funding them with credit cards is stupid. For both reasons long term relationships should be avoided. Now, it's not your partner's fault. I saw lions and monkeys in Africa one time. Monkeys don't get married, and neither do lions. They're both happy, so people shouldn't either. Also, nonmonogamy is definitely something that can be learned. At AshleyMadison.com, we help married people come out of their shells and find other people willing to overcome the straight-jacket of the married/long term monogamous life. Having seen that happen so many times is one of the things that convinces me that being non-monogamous, at least in a small group, is the natural way for people to live. Watching previously faithful, married, and, frankly dull people get transformed into passion-filled 'players' makes it clear that the difference between the two is due mostly to environment — and in particular that the environment in long-term relationships is toxic to people. In the first couple weeks of working on their first affair they seem to come to life, because finally they're living hedonistically the way people are meant to. THE END Sorry, I couldn't resist.

Debugging My Long Distance Bill

Fri, 08 Feb 2008 12:21:56 GMT

I noticed my long distance telephone bills growing at a rapid clip, since I've been doing a lot more remote-tech-lead work than normal.

It occurred to me that there may actually a solution to the problem of excessive long distance bills. I set up GrandCentral's Address Book view as one of my IE home page tabs, and whenever I have to call the team I use that page's click-2-dial function.

When you click a phone number in GC, their telco provider first dials you, then dials the other party. Since there's no outbound call, so there's no cost. The quality so far has been great. There are occasional issues, but they're probably fewer than my cell phone for comparison.

This seems like a big deal ... maybe not because it's not scalable to the country (yet), but the recent announcement that the non ILEC's were implementing peering agreements for no-cost long distance brings us oh-so-tantalizingly-close to free long-distance for the masses.

Elastra delivers scalable OLTP. For Real?

Thu, 07 Feb 2008 12:26:00 GMT

Elastra now offers scalable OLTP via Amazon's compute cloud. Supported Db's include:

MySQL

Postgresql

EnterpriseDB When Elastra first decloaked late 2007, they were all about scaling read-only BI databases, which is the easy problem to solve and I wasn't very excited. This Enterprise DB press release, however, implies that they've got a point-and-clicky active-active-active-etc cluster technology, with automatic-versioning for ~ $400/mo (per server). Quote the release:

EnterpriseDB, the Oracle-compatible database company, today announced EnterpriseDB Advanced Server Cloud Edition, a version of the company’s flagship RDBMS that is built on the Amazon Elastic Compute Cloud (Amazon EC2) and Amazon Simple Storage Service (Amazon S3) Web services. EnterpriseDB has selected Elastra, the world’s first provider of elastic relational databases on demand, as its premier cloud computing software partner. Elastra’s Elastic Database Technology enables EnterpriseDB Advanced Server to run in a virtual, highly scalable, cloud-computing environment

Wow.

Mono for VM's?

Fri, 16 Nov 2007 15:59:32 GMT

BEA recently announced a version of their app server optimized for deployment via virtualization. They figured that since the Java platform virtualizes/abstracts the OS anyway, why not abstract to virtual hardware and just skip the guest OS completely? Less attack surface, less s/w to manage/update. Wins all around, as it represents a collapse of the distinction between container-based virtualization and OS-level virtualization, combining the often-competing benefits of performance and isolation.

Of course, the argument in favor of collapsing OS and language runtime virtualization applies equally well to the .NET platform. It occurred to me that, while I have every expectation that Microsoft will eventually abandon their unsustainable license-every-vm-instance strategy to both app and OS licensing, it could take a long time. Novell, just as they've jumped into decoupling Netware services from a host OS, has an opportunity to reinvent something that looks a lot like Netware+NLM for the virtualization crowd: a mono app server that runs on a hypervisor. This too seems inevitable, and much more compelling than the current duct-tape-required approach to mono-flavored ASP.NET deployment. Although mono is addressing their web deployment woes by going FastCGI (an absolute must for ISP deployment, and long overdue considering FastCGI amazing cost/benefit), Miguel : consider going virtual as well. I think there's much more chance of mono becoming a viable enterprise deployment option if you leapfrog Microsoft with this, and honestly it's a far worthier dev milestone than playing silverlight catchup with moonlight. As I've said before, I love the ambition+audacity that mono represents. At the same time, I'm often pessimistic that it will ever appear on my radar, (is it just cygwin++?). I think getting ahead of Microsoft is the key, and until now I haven't seen how that is really possible. Here's hoping that it is.

jQuery -- Love it

Fri, 21 Sep 2007 14:20:14 GMT

I've been meaning to write about jQuery, a fantastic javascript library that Dossy turned me on to.

jQuery is a new-school javascript library that allows you to do fancy ajax-y special effects in a microscopic amount of code. What sets jQuery apart from other frameworks is it's heavy use of CSS and XPath selectors to determine which elements you want to work on (add events to, hide/show, apply styles etc). Since every project I work on nowadays has an XHTML or XML component, I've invested a lot of time getting good with both scheme's, and as such I had a pretty good running start getting productive with jQuery. jQuery has a perl feel to it, in that you can pack so much expressive power into either a CSS/XPath selector -- they're the regex's of the structured text world. Also perl-ly is that when you're done, you really do have just a handful of unobtrusive javascript code to maintain. Admittedly, not everyone likes tiny code but it suits my scripting-language-addled brain. The base API is concise, easy to keep in your head, and even better: so ridiculously small in KB (14?) the page tax is ~ nil.

If the concise API doesn't do what you want easily, or you're solving a more sophisticated UI problem that should have already been solved (tabs, dialogs, etc), then you'll appreciate that jQuery is even more perl-y (CPAN-y?) in that there is a thriving community of plug-ins you can cherry pick that solve common UI issues.

The recently released jquery UI project offers you a quality assured set of UI extensions that follow certain code guidelines, for those looking for the don't-make-me-cherry-pick, batteries-included approach offered by dojo/scriptaculous/yui.

There have been two occasions recently where I had to implement small tweaks to different web based enterprise application where it wasn't clear to me how to do it with using the vendor's supported API's, or if it was even possible. In both cases, I implemented the customizations by including the jQuery library in a global template, adding an onDOMReady event handler, and writing in some new DOM elements using jQuery in ~ 5 lines of code. Problem solved - fast. Greasemonkey-type short-cuts like this risk not being upgrade friendly, but when implementing something not supported by vendor API's you're usually off the seamless upgrade path anyway. In addition, doing it the vendor way at the Java/C#/php level would almost definitely result in more complex code, and much more of it, that would be harder to maintain/merge anyway. And finally, if the need arises, it's much much simpler to turn off a jQuery page hack by commenting out a single line unobtrusive event-handler-add than it is to undo a compiled or scripting language library modification. It's downgrade friendly.

I now reach for jQuery every chance I get. Take a look.

Lies, Damn Lies, and Benchmarks

Thu, 13 Sep 2007 15:16:47 GMT

From Curt Monash:

"2. Vertica’s software is 50X faster than anything non-columnar and 10X faster than anything columnar. Now, some of these stats surely come from the syndrome of comparing the future release of your product, as tuned by world’s greatest experts on it who also hope to get rich on their stock options in your company, vs. some well-established production release of your competitors’ products, tuned to an unknown level of excellence,* with the whole thing running test queries that you, in your impartial wisdom, deem representative of user needs. Or something like that …

lol. Well said, Curt.

Elastra -- To Infinite Database and Beyond

Thu, 13 Sep 2007 12:03:26 GMT

EC2-ISV Elastra decloaked recently. The website is so-so, but here's a quote blog comment from an employee:

Though a $360/month list price may seem comparable to a hosted solution, you are getting a lot more. Besides not having to buy hardware, software, bandwidth, or storage (we give 1 Terabyte of S3 storage free), you get the ability to really run your solution on demand.

Imagine allocating hardware to set up a 10 node cluster, setting up databases on it, having users or analytical programs hook up to it for a month of heavy traffic or heavy reporting. Imagine having to expand those clusters to 20 nodes or contract them to 3 because you are doing so well, or you no longer need the processing power. Now, think about being able to do these things at a minute's notice.

Imagine having 3 versions of your complete Data Warehouse for testing, running all three at the same time for, say, three days, picking out the best design, tools, analytical engines and taking the other two down by clicking an icon while archiving the other two in case you want to try them later.

The point with ELASTRA is not just that it's the first solution that actually uses S3 as a disk drive and not a tape drive, but that is also helps the application vendor create, scale, manage, and deploy their solution as their enterprise (hopefully) grows.

I know hosting people will scream that they can also add servers easily, but for the screamers: your competition is Amazon, not ELASTRA. We don't have anything against you if you can offer our platform the same capabilities and the same prices as Amazon can and so effectively does.

Sounds pretty cool if you want to run your analytics db apps on EC2. I haven't personally experienced the situation where there were enough BI users to warrant this. Usually BI apps target a handful of decision-makers, and versioning isn't so important with star schemas that capture historical information. Additionally, I don't think it's a stretch to say that BI implementations using open source db's will probably be located on the budget-constrained part of the spectrum, and have even less overlap with customers needing high-end availability requirements. Hmmm. It will be interesting to see if they find a market. This reminds me a bit of another startup I'd been meaning to mention: SaaS lab management provider Replicate Technologies founded by long-time blog-reader Ken Novak. Ken walked me through some of the scenarios that his team had been enabling, and they sound somewhat familiar to the elastra scenarios, except with a lab management focus on whole system- and network topology- versioning as opposed to database versioning and clustering. With replicate's technology, you of course create and version all the VMs and groups of VMs you like, but you also can simulate and capture network topologies that would otherwise be a big pain e.g. lossy WAN links if you want to test an Exchange remote failover scenario, or a huge number of low bandwidth connections for testing a sensor-web data collection application ( two actual use cases ). Replicate is not deployed on EC2 like elastra and 3terra etc - they've rolled their own virtualization platform. But if Amazon's plans to own the grid enabler space (continue to?) pan out I'm sure EC2 deployment will be in the cards.

Square Swapping

Fri, 29 Jun 2007 12:22:07 GMT

My office sublease expires tomorrow at the hawk's nest on the 18th floor of the old Thinking Machines building in Kendall Square. Last night I moved my officestuff to my new digs above Dewey, Cheetham and Howe overlooking Harvard Square (aka One CarTalk Plaza)

I liked Kendall quite a bit ( great office, river views, already miss it a little.. sniff), but Harvard is much hipper and studenty/lively compared to the officepark/biotech everyone's-gone-home-by-8pm canyons of Kendall. (Note to self: repeat this over and over and you won't miss the river views).

And as annoying and bittersweet as moves can be, there was one thing I looked forward to: repointing my GrandCentral number to the new office space for a seamless, ILEC-free transition. Location, you are so dead to me.

I too think Opera is a pretty good browser

Wed, 20 Jun 2007 20:19:25 GMT

I read a post recently by Jonathan Ellis that I've been meaning to write myself for a few months:

I've been trying Opera 9.2 for a week, and I'm pleased with it enough that it's going to continue to be my main browser. The main selling points for me are
* MDI weirdness is mostly hidden now, I hated earlier Opera UIs
* 20-30% less memory use; even after poking about in the guts of about:config to force FF's memory cache to the same 10MB that I gave Opera (which exposes this option right in the UI), Opera consistently uses less memory for the same workload. (Without adding this option to FF, it would max out around 400MB instead of 150MB.)
* feels snappier; opera seems quicker to start rendering something useful on slow-loading sites like 1up.com, although total render time is about the same. It's also instantaneous to open a new tab, which consistently takes around 1s on FF after I've been using it a while. I open and close tabs frequently...

My firefox2 memory issues have also moved me over to Opera. The only things I use FF for are google docs, and whenever I need to use the FireBug, WebDeveloper, or Selenium IDE extensions ( oh - DocuFarm is awesome too) or whenever else I hit an opera-unfriendly site. Okay, so I'm not exactly firefox free ... but I try to keep the tab count down to about 5 or so. When I have to do research and open up 10 pages of doc's / google results it's definitely Opera that I use.

I suspect it's not entirely FF2's fault, but the very extensions that I'm infatuated with are part (the cause?) of the problem. I installed the latest FF3 Alpha using Altiris (ditto Safari/Windows - I love Altiris) and while extension-free, it's pretty darn fast ( faster than ff2). Over the course of a day, the memory footprint is much better too. I'm hopeful the release reflects this, and I can winnow down my browsing choices, but until then I feel like it's not the worst thing in the world to better acquaint myself with cross-browser rendering behavior.

Google Gears Roadmap

Fri, 15 Jun 2007 12:55:47 GMT

MS's Dare Obasanjo harshes on the Google Gears beta because it doesn't solve the big problem of offline/online synchronization. That seemed like a reasonable criticism, because that is a hard problem, but I think it misses a couple of points.

Read-mostly applications -- ones you use for reference and just need a local copy of -- are pretty important. Having a local copy of your organization's CRM database is a valuable thing, whether or not you can write back to it. If Gears was used predominantly for this, I think it would still have a big impact on web architecture.
The mobile folks have gotten pretty good mileage out of SyncML for solving this general problem. In fact in a lot of ways online/offline browser apps are simply a recasting of connected/disconnected mobile apps. Right? I would be surprised not to see a Gears+SyncML hookup in the near future.

The conflict resolution issues (who over-writes who, when, and how) Dare points out are very application specific, and I don't see any generic technology-based solution addressing them. Those are much more likely to be addressed by convention ... and we won't establish those conventions until folks start bushwhacking down the online/offline web path.

Making dumb phones smart

Thu, 07 Jun 2007 10:56:59 GMT

I also have become a fanatical Jott user. Jott is a cool piece of software that allows you to call in from your cell phone (or any phone you set it up for) and record a message. That message is then transcribed and sent to you as an email or an SMS. It changes your voice to text. Jott has just rolled out some incredible updates where you can send a note, not only to yourself, but to other people who's email addresses or SMS #'s you have loaded into your contacts. Send a message to one person, or a group of people. It's super easy. And did I mention that both Jott and Gmail are FREE?!?

I'm not a fan of smart phones. I've been accused of being a phone-luddite, and that's probably accurate. There are a few services that might justify me going smart, but I've always held out hope that voice-enabled tech will make my old clunker do the fancy stuff.

The first service that gave me this idea was a dotbomb called something like qixo (google+memory is failing me. And no, it's not the travel site). After signing up for quixo[?], you uploaded your addressbook to them, and gave them your cell #. When you called them, you'd speak the name of the person you wanted to talk to, and a human being would connect you to them or ask for clarification. Say goodbye to remembering phone numbers - a neat idea.

It was free, and provided a couple years of Kozmo-like novelty while they lasted. But importantly they enlightened me to the idea that you could make 'dumb phones' smart. Jott is a service along these lines that I've enjoyed using. As mentioned above, they are a free voicemail dictation service that actually use humans to do their voice recognition. Your voice files go to India where they're transcribed and emailed back. I've found it to be a great way to capture things-to-do and keep faith with my favorite GTD tenet: ( either record the task or spend brain cycles reminding yourself to remember it).

It's such an endearing and useful service that it doesn't bother me that the voice recognition is absolutely horrendous. I don't think I've ever gotten a message transcribed without errors, normally horrible errors. Either it's my diction, my cell phone quality, or (as I suspect) the hiring situation in outsource-mad India is quite dire. Overall it's *much* worse than my experience with Dragon Dictate, circa 2001. But since I use it only once or so a day, I always remember what I meant to say and have no trouble decrypting the unintentionally steganographic transcriptions. And there's a weird b-movie kind of entertainment value to interacting with such a low quality service. Anyway, if you set your expectations really low and don't have serious privacy issues then I recommend it -- use it now before they run out of VC/angel money.

Aside: profanity is not transcribed by Jott - a friend who didn't know about the mturk architecture found this out. It wasn't me- honest :-) Update: 1-800-Goog-411 is an essential speed-dial entry for dumb phones. It's awesome.

Upcoming: Stonebreaker on 'Which of my databases you should buy'

Wed, 09 May 2007 18:23:44 GMT

Michael Stonebreaker's actual talk tomorrow (7-9pm) at MIT is entitled:

"One Size Fits All in DataBase Management: An Idea Whose Time Has Come and Gone"

And I hope to be there. I'm anxious to learn more about Streambase and Vertica, his event-oriented and column-oriented db progenies.

If you google for Twitter and scalability, you'll find a lot of articles on how Twitter's Rails setup has buckled under their obscene growth curve. People have pointed fingers at Rails, and Ruby, and various other things to explain these scalability issues. But I think that fundamentally they're using the wrong technology for what they're trying to accomplish.

I think their implementation of a messaging architecture on top of an RDBMS was not a great choice for their current volume, and I strongly suspect that some event processing platform like Streambase would be a better choice. I'm a complete event processing novice and I know less about twitter so they might already do this ( and their scalability issues are completely about rendering event streams to the web). But regardless I'm hoping this talk will give me a better idea of when to use messaging-oriented technology as opposed to the tried-and-true rdbms approach.

Cubulus: Open Source OLAP ++

Wed, 25 Apr 2007 02:28:07 GMT

Cubulus is a new kid on the open source OLAP block. Written in ~ 3K lines of python, it implements mySQL-based ROLAP w/calculated aggregations stored in memcached, and includes rudimentary MDX and distributed parallel query support. It sounds like an ambitious project ... and one I'll definitely be keeping an eye on.

also on John's Open Source OLAP radar:

Pentaho's Mondrian -- the original java-based ROLAP engine
OpenI - kitchen sink platform similar to pentaho
OpenOLAP - Big in Japan ( needs help on translation )
Palo - MOLAP store for excel et al.
LucidDB - trendy column oriented architecture
MonetDB - also column oriented.

Note that I have only very limited experience with the above items ... my clients always seem to have a licensed copy of SQL Server around, which gives them free (zero-marginal-cost) analytics.

Blikistan?

Wed, 25 Apr 2007 02:12:32 GMT

Blikistan is a blogging tool named after the small ex-Soviet country Blikistan. Blikistan (the software) features a magic engine which pulls all the blog configuration and postings from a Socialtext Wiki. Blikistan (the country) has a population of 32,768 people, and the population is fully literate. Blikistan's Government never meets in person, all communication and planning is done on a wiki. Blikistan (the software) is the only blogging software that allows the blogger to choose which language their blog will be powered by. The Magic Engine can be implemented in any language using Perl's Inline modules. In Blikistan (the country) taxes are filed online using SocialCalc. Blikistan (both the country and software) is completely paperless.

I did not know that.

Stateless Linux for the masses

Thu, 19 Apr 2007 14:33:27 GMT

Babeldisc looks like a consumer version of the virtual desktop/thin client/LTSP/etc concept. They use a Linux-based LiveCD (and soon USB fobs) to boot the OS, paired with networked storage for user data hosted at Rackspace (what... no s3?) It looks pretty cool ... but I wonder if they'll get killed on support costs for device driver troubleshooting. Photoprinter/wifi/etc headaches have consistently killed all my desktop linux livecd explorations.