A computing revolution is underway -- in broad daylight – yet most are unaware. Perhaps because there are only a few hushed and furtive conversations and no Paine-like polemics, we do not feel the winds of change. They howl and rage nevertheless, stirred by three intertwined, powerful technical forces that are reshaping entertainment, discourse, commerce and culture: big data centers, cloud computing and the mobile Memex.
Diverse computing services, backed by massive data centers, are now being delivered not just to businesses but to individuals and their mobile devices. Life in the clouds is here, primitive but evolving rapidly in response to powerful social and economic forces. Paradoxically, we are witnessing one of the greatest consolidations of computing power since IBM ruled the computing world, yet the diversification of services and market narrowcasting has never been greater. It’s The Long Tail, powered by big iron.
Data Centers: They’re Big, Really Big
Google, Microsoft, Yahoo and Amazon are building massive (and I mean massive in the most literal, visible from Earth orbit, sense) data centers to support a new generation of ubiquitous, on-demand services. Given the intensity of the competition, all are justifiably secretive about just how big and costly the centers are, yet news stories and public filings suggest their scope and scale. Just a few examples illustrate this.
In western North Carolina, Google is building a $600M data center in Lenoir, drawn by an array of tax incentives (estimated to be at least $100M). Another Goggle megacenter is rising in The Dalles, Oregon, and Microsoft and Yahoo are building comparable facilities in Wenatchee and Quincy, Washington. Why these sites? Among many reasons, inexpensive electrical power is among the most critical. With a 20-50 MW power requirement for each center, power costs and power reliability really matter. Load shedding during a brownout is a quality of service disaster.
Interestingly and perhaps ironically, this same geographic attribute – cheap power – attracted a previous generation of companies to the same sites. One need only think about textile and furniture manufacturing in North Carolina or bauxite refining in the Pacific Northwest to realize that, as in real estate, it is location, location, location for massive, lights out data centers.
It’s not about the big iron, nor is it just about web searching, and it hasn’t been for quite a long time. It’s about hosting the services that support B2B, electronic commerce, entertainment-on-demand and social networks, among others. It’s about business and consumer mind share. Standard web service APIs and now ubiquitous mashups are the most obvious examples of this cloud computing revolution. Mashup development has evolved from an ad hoc process to one supported by powerful tools, with Yahoo’s Pipes, Google’s eponymous mashup editor and Microsoft’s Popfly on the vanguard.
I’m convinced, though, that these are just the tip of the proverbial iceberg. In that spirit, I’ve found Amazon’s Elastic Compute Cloud (EC2) play enormously interesting. To quote, EC2 is “…a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers. … Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to requisition machines for use, load them with your custom application environment, manage your network's access permissions, and run your image using as many or few systems as you desire.”
With EC2, one can configure a custom software stack (atop a Linux platform) and launch it within minutes. Already, some startup companies are using EC2 to outsource their entire computing infrastructure. This is the software analog of a fabless semiconductor firm. As other capabilities emerge, I expect an even richer set of service options.
This and other web service capabilities raise interesting questions about the nature of academic research and education, including training students to compute at very large scale. That makes the recent Google-IBM academic announcement timely.
Mobile Memex: It’s Everywhere
Vannevar Bush envisioned the Memex as an intellectual aide and amplifier, making information accessible when needed. Powerful mobile devices with GPS and broadband (cellular and WiFi) communication, inexpensive sensors and rich cloud services suggest Bush’s dream is nearing realization.
This ubiquitous infosphere will rely on sensors to identify and record social and emotional context, manage personal history and environment, monitor and coordinate health care and support business processes. We are not there yet. There are thorny privacy and security issues, our broadband network coverage is incomplete and inadequate, we face a software Tower of Babel, and our ability to manage complex systems is woefully inadequate.
I remain, however, a technological optimist. Revolutions are always exciting. Bring your best ideas and join the fray.