Reed's Ruminations: A Blog by Dan Reed

Goldilocks, Porridge and Science: Getting It Right

We all remember some variant of the fairy tale of Goldilocks and the Three Bears. It is a tale of the search for a good meal and a nap, though it also involves breaking and entering, culinary theft and accidental vandalism. If, perchance, you feel a worrisome gap in your erudition and erstwhile encyclopedic knowledge of this element of the Western cultural lexicon, fear not, for I will summarize the key elements of the tale. And please bear with me, this does relate to science! (Pun intended.)

Porridge and Bears

The young girl, Goldilocks, visits the home of the bears and samples the porridge of Papa, Mama and Baby Bear, pronouncing each, in turn, too hot, too cold and just right. Sated with porridge, she then seeks a place to sit, finding the chairs of Papa and Mama Bear too large. As she settles comfortably into Baby Bear's chair, it collapses. She then explores each of the beds before falling asleep in Baby Bear's bed, where she is found by the returning bear family. On discovery, she flees.

This tale, of course, raises worrisome questions. Do bears really like porridge? Could they sit in chairs and would they sleep in beds? More tellingly, are they underwater on their cottage mortage? Though all worthy of whimsical rumination, let us defer literal scrutiny of the fairy tale's purported ursine facts and the anthromophrized bears and focus on the porridge bowls and their metaphorical implications for science.

Science: Matching Need and Availability

Like porridge bowls, science comes in many sizes, from research conducted by a single investigator (e.g., a theoretical computer scientist studying computational complexity) to experimental projects that involve thousands of people (e.g., the ATLAS and CMS detectors at CERN). More to the point, there is no "right size" for science, other than matching the resources to the nature of the problem and the approach, albeit with the usual assessment of investment versus potential payoff. Nor is there, a priori, differential value between large and small science. Groundbreaking discoveries with broad and transformative effects have occurred across the entire spectrum of project sizes.

Scientific culture does vary widely with discipline and scale, however. Just as the ethos and metrics of biology differ markedly from those in physics, so too do the approaches and politics of small and large science. Large-scale experimental science typically involves long-term planning, geo-political negotiations, and infrastructure construction that may span multiple years before data can be captured and analyzed.

Big Infrastructure for Small Science

Historically, these two worlds – small-scale science (both theoretical and experimental) and large-scale science – have largely depended on separate and distinct infrastructure. That historical separation is now disappearing, mediated by the tsunami of data now being produced by new generations of scientific instruments and computational models. (See The Zeros Matter: Bigger Is Different and Language Shapes Behavior: Our Poor Cousin, Data.)

Historically, the researcher who had unique experimental infrastructure also maintained a competitive advantage, for he or she could conduct experiments and capture unique data. With the rise of large-scale, shared instrumentation in a host of disciplines, most notably in biology, astronomy and physics, and open access to the resulting research data, advantage instead accrues to the researcher who can ask and answer more interesting questions. This is a profound shift in scientific culture with deep implications.

It is now incumbent upon us to rethink how we facilitate discovery and innovation in this brave new world of large data, for practitioners of both small and large science. Simply put, we must reconsider how we fund, construct, manage and operate scientific data repositories.

As I have learned at Microsoft, there are lessons that can be drawn from the construction of cloud data centers, web search engines and tools for analyzing ill-structured data that could both accelerate and simplify scientific discovery. These tools and practices are not only applicable to individual scientific domains, they are also especially relevant to the problems that lie at the intersection of multiple disciplines, where scientific cultural divides and divergent terminologies often inhibit collaboration and exploration. We will also need new models of public/private partnership to realize this vision, something we are pursuing with Microsoft's worldwide engagements on client plus cloud infrastructure.

Remember the porridge, the bears and Goldilocks. We can match the needs of all scientists, working together. Whether you like it hot or cold, large or small, it can be "just right" for everyone.

It’s the Sum, Not the Parts

If you have ever designed a house or been part of the design process for a building, you know that successful designs are the gestalt of many ideas and often conflicting objectives. Collaboration spaces, meeting rooms, window offices, LEED certification, storage and loading dock access, utility routing, computer spaces, telecom closets, and environmental safety – the list is seemingly endless and desires always exceed budgets. However, when the conflicting objectives are harmonized, the result is both functional and elegant, satisfying both esthetic and practical expectations.

In each design, a senior architect is ultimately responsible for eliciting harmony from the cacophony of competing objectives. Although he or she inevitably works with a cadre of designers and subsystem experts, each senior architect brings a certain panache and philosophy to each project. The great ones and their work are instantly recognizable. Look no further than I. M. Pei and the Louvre pyramid, Frank Gehry and MIT's Stata Center, or Mies van der Rohe and the Seagram building to see this elemental truth embodied in steel, masonry and glass.

Computer Architect

It is worth pausing to consider the exegesis and etymology of the phrase "computer architect" and the phrase's broader relationship to architecture in the grand sense of style and design. In this context, a computer architect is far more than a system integrator, but someone who combines disparate elements to create something compelling and new that elegantly and efficiently solves an important problem.

Simply put, computer architects create something of functional beauty, just as do their cousins who work in the more tangible media of rock and stone. Consider John Cocke and the IBM 801, Ken Batcher and the Goodyear MPP, Tadashi Watanabe and the NEC SX. Similarly, Seymour Cray's designs balanced many aspects of power engineering, packaging and cooling, memory bank design and functional unit architecture, and his success arguably rested as much on his prowess as a packaging and cooling designer as it did on his acumen as a processor and memory system designer.

The facts differ, but the story is the same. The sum has always been more than the parts.

The Gestalt Matters

As we debate possible exascale designs, evaluate eco-friendly mega data center approaches, ponder the implications of heterogeneous multicore processors, struggle to overcome the memory wall and explore alternatives to rotating magnetic storage, we should remember the broad meaning of architect and its implication for multidisciplinary design. In an era of hyper-specialization and local optimization, we often forget the importance of global optimization. Sometimes it really is better to know something about everything than everything about almost nothing.

Whether building or machine, effective design incorporates and balances diverse elements and capabilities. The sum has always been more than the parts.

Egg Baskets, Lambdas and Geo-resilience

If you have ever been sent to the henhouse to fetch eggs, you know the practical truth of the old aphorism, "Don't put all your eggs in one basket." With just one stumble – wait for it – you can have egg on your face. (Sorry, I just could not avoid the pun.) All of which is to say, distributing one's assets is normally a successful risk mitigation strategy. At least that is what we all thought about our retirement mutual funds until the world financial markets collectively declined.

I hear rustling among my faithful readers, collectively wondering how these colorful, though meandering, introductory paragraphs will segue to more salient topics, such as clouds or high-performance computing. Fear not, the answer is here, manifest as a few observations on massive data centers, wide area networking, and techniques for geo-distribution and resilience.

Protecting the Big Asset

Every operator of massive cloud data centers chooses the locations of those facilities after careful, qualitative and quantitative analysis of several factors, including the tax climate, availability and cost of electricity, the natural stability of the site (i.e., probability of earthquakes, tsunamis, hurricanes and other natural disasters), the region's political stability and physical security, access to optical fiber and broadband networks and a host of other issues. Given an objective scoring of these criteria, it is not surprising that multiple operators and competitors often site their facilities in the same region.

Regardless of location, a substantial fraction of the infrastructure associated with a data center has historically been devoted to electrical power and resilience. This has typically included multiple diesel generators, collectively capable of generating many megawatts of electricity, and large, lead acid battery banks to ensure an uninterruptable power supply (UPS). Not only are these generators and battery banks expensive, their deployment and acceptance are time consuming and often difficult. Indeed, before the economic downturn, demand for generators was so high that sites competed to secure a place in line for delivery, further delaying data center construction.

The reason we strive to protect data centers is obvious, their failure often has catastrophic business implications. Without doubt, though, some workloads are less affected by failure than others. If your web search for basket weaving supplies times out, you are probably content to retry the query a few seconds later. If your attempt to file a tax return electronically fails, that is more problematic.

This focus on resilience raises one rather obvious question, is there a better way? Returning to my initial metaphor, rather than protecting the one big basket of eggs, might one distribute the eggs in several, smaller baskets?

Geo-dispersion: The Other Alternative

If it were possible to replicate data and computation across multiple, geographically distributed data centers, one could reduce or eliminate UPS costs, and the failure of a single data center would not disrupt the cloud service or unduly affect its customers. Rather, requests to the service would simply be handled by one of the service replicas at another data center, perhaps with slightly greater latency due to time of flight delays. This is, of course, more easily imagined than implemented, but its viability is assessable on both economic and technical grounds.

In this spirit, let me begin by suggesting that we may need to rethink our definition of broadband WANs. Today, we happily talk of deploying 10 Gb/s lambdas, and some of our fastest transcontinental and international networks provision a small number of lambdas (i.e., 10, 40 or 100 Gb/s). However, a single mode optical fiber has much higher total capacity with current dense wave division multiplexing (DWDM) technology, and typical multistrand cables contain many fibers. Thus, the cable has an aggregate bandwidth of many terabits, even with current DWDM.

Despite the aggregate potential bandwidth of the cables, we are really provisioning many narrowband WANs across a single fiber. Rarely, if ever, do we consider bonding all of those lambdas to provision a single logical network. What might one do with terabits of bandwidth between data centers? If one has the indefeasible right to use (IRU) or owns the dark fiber, one need only provision the equipment to exploit multiple fibers for a single purpose.

Of course, exploiting this WAN bandwidth would necessitate dramatic change in the bipartite separation of local area networks (LANs) and WANs in cloud data centers. Melding these would also expose the full bisection bandwidth of the cloud data center to the WAN and its interfaces, simplifying data and workload replication and moving us closer to true geo-dispersion and geo-resilience. There are deep technical issues related to on-chip photonics, VCSELs and ROADMs, among others, to make this a reality.

In the end, these technical questions devolve to risk assessment and economics. First, the cost of replicated, smaller data centers without UPS must be less than that of a larger, non-replicated data center with UPS. Second, the wide area network (WAN) bandwidth, its fusion with data center LANs and their cost must be included in the economic assessment.

These are interesting technical and economic questions, and I invite economic analyses and risk assessments. I suspect, though, that it is time we embraced the true meeting of high-speed networking and put our eggs in multiple baskets.

Matryoshka Dolls and Data Centers

Over the past year, there has been much debate on the energy efficiency of large data centers and how best to measure that efficiency. Power Usage Effectiveness (PUE) has emerged as the de facto standard, where PUE is defined as the ratio of power entering the facility to power drawn by the computing equipment. Intuitively, the ideal PUE is unity (i.e., all incoming power is used by the computing equipment, with no overhead for cooling or energy distribution). For details on PUE and data center efficiency, I encourage you to read the documents from the The Green Grid, which has promulgated and evangelized these measurement standards. (Truth in advertizing: Microsoft is a leading member of the The Green Grid.)

Legacy and Modern

Many legacy data centers -- those built more than a few years ago – have PUEs in excess of two, or even three. This is largely due to inefficient computer room air-conditioning (CRAC) units, lack of hot and cold aisles, energy losses due to multiple (unnecessary) voltage conversions and aging or inappropriate building designs. If your "data center" is located in the nearest available space to your laboratory -- a retrofitted janitor's closet – and cooled by two box fans from Walmart, odds are you are not a paragon of PUE virtue, even if your aggregate computing power is small.

Today, state of the art data centers have PUEs below 1.5, and there are new designs that could approach a PUE of one by reducing UPS support where appropriate, operating at substantially higher temperatures and exploiting ambient cooling. Many people do not realize that computing hardware is much more resilient to high temperature than history and practice would suggest. It need not be chilled to temperatures suitable for polar bears.

Last year, Microsoft's Christian Belady illustrated this point by operating a group of servers in a tent for over six months, with zero failures. Although not a statistically valid sample, the experiment did illustrate why ASHRAE and hardware vendors have broadened the temperature and relative humidity envelope of acceptable operation.

More generally, efficient data center design and analysis are much like Matryoshka dolls, because the PUE one obtains depends on where and what one measures. Ideally, the bounding box for measurement should be the entire data center ecosystem, from power distribution and cooling for servers to data center networks to operations and support. Equally importantly, a PUE near one does not mean the computing system itself is efficient and well-matched to the offered load or that the computations are either necessary are useful. Choose wisely and measure thoughtfully and carefully.

Appearance and Utility

Finally, I would be remiss if I did not opine on the most obvious, visual difference between cloud data centers and high-performance computing (HPC) facilities. The former are designed for function, not appearance. They are usually nondescript facilities optimized for efficient hardware operation at large scale, not for human accessibility or for comfort. Indeed, container-based data centers look more like a warehouse and distribution center with parking and utility connections than Hollywood's idea of a computing center. Conversely, HPC facilities are usually showpieces with signs, elegant packaging and lighted spaces suitable for tours by visiting dignitaries.

At large scale, efficient trumps pretty. It's all about what one measures.

Language Shapes Behavior: Our Poor Cousin, Data

The Sapir-Whorf Hypothesis holds that language and culture shape behavior and thought. The idea has a long, rich and sometimes controversial history, though recent cognitive linguistic research, together with brain imaging, has lent it new credence. Lately, I have been reflecting on the nature of linguistic relativity in scientific research, particularly for HPC and cloud computing. In particular, how has our focus on floating point operations per second (FLOPS) shaped discourse, science politics and outcomes?

Look, It's Terascale!

If I were to remark, "That is a terascale system," you would likely interpret my comment as an assessment of the system's computing performance, more specifically its floating point capabilities. For the cognoscenti, the sentence likely engenders visions of multicore systems with out of order instruction issue and deep pipelining, small clusters with fast interconnects and (perhaps) some accelerators such as GPUs. You might even reflect on the historical evolution of high-performance computing, when a terascale system was ranked first on the Top500 list about a decade ago.

Given my remark, you would be unlikely to eye your desktop computer with its terascale (terabyte) commodity disk drive. Yet, based on its storage capacity, that desktop is just as surely a terascale platform as is the compute cluster. Ah, you say, I have mixed capacity (bytes) with capability (operations). Terabyte per second storage systems are rare, you say. Absolutely true, yet as this bit of third rate linguistic legerdemain illustrates, language and, perhaps more importantly, connotation really does trump denotation in our discourse.

Whether Petascale and Exascale?

Today, the world's fastest computing platforms have broken the one petaflop mark on the high-performance Linpack (HPL) benchmark, and planning groups across the United States, Europe and Asia are debating political and technical approaches for the construction of exascale computing systems. I cannot help but wonder, though, why we in technical computing talk so little about petascale and exascale data platforms? Why is there no international initiative to build a low latency, high bandwidth (many terabytes/second to even a petabyte/second) data analysis engine with multiple exabytes of storage capacity?

I suspect the reason lies in our background and linguistic heritage. Most of us in high-performance computing came from mathematical backgrounds, where success was defined by a proof and an equation and their embodiment in a computational model, rather than by insight gleaned from large-scale data analysis. I believe it is time we found a lingua franca that bridges FLOPS and bytes and reclaim the full connotation of exascale.

I Want My Exabytes

It is worth remembering that the explosive growth of observational data is itself largely a product of inexpensive CMOS sensors, based on the same semiconductor technologies that begot microprocessors. The examples of high resolution instruments are legion, from the Large Hadron Collider (LHC) and its international hierarchy of data archives to the proposed Square Kilometer Array (SKA) radio telescope, which may produce as much as an exabyte of data every few days. This data tsunami is not limited to the physical sciences; the biological and social sciences are being inundated as well.

One of the major lessons from web search and cloud data centers is the power of truly massive scale, near real-time data analysis. When anyone with a cheap cell phone and a web browser can extract data and insights from a non-trivial fraction of the human knowledge base, behavior and culture are transformed. I would like to believe that we can bring the same data-driven analytics to scientific research as are routinely exploited to find a good lasagna recipe. We need a balanced exascale initiative, in every sense of the word, lest we be a technical confirmation of the Sapir-Whorf hypothesis.

Microsoft Extreme Computing Group (XCG)

Here is a small item that may be of possible interest, creation of the Extreme Computing Group (XCG) at Microsoft. XCG was formed in June 2009 with the goal of developing radical new approaches to ultrascale and high-performance computing hardware and software. The group's research activities include work in computer security, cryptography, operating system design, parallel programming models, cloud software, data center architectures, specialty hardware accelerators and quantum computing.

Nothing Left But the Smile …

Long ago, we in HPC recognized the inevitability of the Attack of the Killer Micros, and with few exceptions, all of today's HPC systems are based on some variant of commodity microprocessors and their commodity cousins, GPUs. A similar revolution swept the secondary storage market. After all, the I in RAID standards for inexpensive, a synonym for commodity.

Like the Cheshire Cat in Alice's Adventures In Wonderland, it seems increasingly clear that in the high-performance, low latency interconnection network space, we will be left with nothing but the smile. Simply put, the last remaining non-commodity component of HPC clusters – the high-performance interconnect – is in grave danger, due to the global economic downturn and the price-performance pressures of commodity Ethernet.

Remember Coaxial Cables and BNC?

The old geezers among us, including yours truly, remember when Ethernet meant coaxial cables and CSMA/CD at 10 megabits/second. (Here's a shout out to my colleague Chuck Thacker, one of the co-inventors of Ethernet.) Since then, we have seen many generations of advances in transport fabrics, switches and routers and speeds.

One gigabit Ethernet is now a commodity item, contained on almost all PC motherboards, 10 gigabit Ethernet is the high bandwidth standard, and 40 gigabit and 100 gigabit Ethernet standards are under development by the IEEE. Infiniband is one of the last major competitors to 10 gigabit Ethernet, and even the Infiniband vendors are increasingly offering Ethernet compatibility (i.e., so-called converged fabrics).

Cloud Data Centers and HPC

I invite you to ponder your response to the following question. What is the difference between a megascale data center and a petascale computing system? Increasingly, the answer is "not much, but a few elements of the software stack."

Of course, this need not be the answer, but it likely will be unless we change our research and development strategies and also our procurement models. Ironically, megascale data centers could also benefit from lower latency, higher bandwidth communication fabrics with scalable bisection bandwidth. It is time we bring these two together, for the HPC community could leverage the economics of megascale data centers and their needs.

Scientific Clouds: Blowin’ in the Wind

N.B. I recently responded to some questions from John West (HPCWire) regarding the Microsoft Cloud Computing Futures (CCF) research project. In that Q&A, I also commented on the relevance of cloud computing to computational science. What follows is an augmented subset of the Q&A, but focused on just the relevance of clouds to technical computing.

Cirrus, stratus, altostratus, cumulus: they are the scientific names of the common clouds. They drift across the sky, reflecting the changing wind and weather. A new front is blowing into computational science, and cloud computing will soon advance scientific and engineering discovery.

That is one of the reasons I am excited about cloud services. I believe we are at a technological transition point, just as profound as that engendered by the "attack of the killer micros." This is true whether you are enamored of Microsoft's Azure, Amazon's AWS or Google's Apps.

Learning from History

Let's step back and gain some perspective, starting with the "Branscomb pyramid" ("From Desktop to TeraFlop: Exploiting the U.S. Lead in High Performance Computing, Lewis Branscomb et al) and the diverse types of technical computing that now exist. We tend to focus on the apex of the computing pyramid, now exemplified by petascale systems intended to support only a handful of applications and users. However, most science is conducted at lower levels of the pyramid, using desktop computers, laboratory clusters and university-scale computing infrastructure. By analogy, it's exciting to talk about international hypersonic transport, but most people care more about efficiently and painlessly commuting to work each day.

Over the past decade, we successfully leveraged commodity hardware to create large clusters. What was nearly heretical when we first deployed clusters at NCSA is now commonplace. However, this scaling has not been without cost. Cluster programming remains difficult at scale, we have turned a generation of researchers into parallel programmers and system administrators, institutions are struggling with rising demands for machine space, power and cooling, and duplicated facilities make sharing expertise and data difficult. We are heavily focused on computing at a time when data analysis now dominates much of science and engineering. Like many of you, I contributed to this state of affairs, and I feel some responsibility to help us find a new path.

Hype and Reality

Let's separate the hype from the reality. Clouds won't magically restore your 401(k) retirement fund, cure halitosis or even help you drop twenty pounds before your upcoming high school reunion. Like all new technologies, however, they challenge some conventional computing wisdom and change some of our operating assumptions.

Personal computing was a non sequitur when computers filled rooms. Internet search was nonsensical when there were only a handful of research web sites. Social networking services depend on inexpensive, ubiquitous broadband access and mobile devices. Hosted cloud services and software are now possible given the confluence of inexpensive but powerful multicore processors, high-capacity storage, broadband networks and the economies of scale that consolidation in cloud data centers make possible.

Five Reasons Clouds Matter

First, the economies of scale from mega-data center provisioning mean capital and operating costs can be lower. When buying servers in 500,000 unit lots and designing facilities at scale, the provider does have some financial and technical leverage. This would allow universities, laboratories and federal agencies to devote a larger fraction of precious funding to research rather than infrastructure. Remember Dan's computational science corollary; it's the science, not the infrastructure, which matters.

Second, truly large-scale data analysis, particularly multidisciplinary data fusion, can become routine. In the scientific community, we have worked hard to build workflows for access to distributed data. Consolidation and co-location enable new approaches, and we tend to forget that cloud data centers have many, many petabytes of disk storage. It really is possible to query multiple petabytes of data using intuitive, easy-to-use desktop tools – the business community does it all the time. Jim Gray proved the power of database tools on several scientific data analysis projects, including the SkyServer.

Third, clouds facilitate time-space tradeoffs. It is just as cost-effective to run 100,000 individual jobs simultaneously as sequentially (e.g., for a parameter study), something that our batch queuing strategies strongly discourage on high-performance computing systems. In geek terms, the area is the same, whether one uses tall, skinny rectangles (lots of resources for a small interval) or short, long rectangles (a few resources for a long interval). The elasticity of clouds, a consequence of multiplexing many users and workloads, means that the resources are always available without waiting.

Fourth, I also believe that the cloud will offer HPC services at increasing scale, beginning with that typified by today's laboratory clusters. This is already happening, and as I/O device virtualization continues to improve, communication latencies will decrease and tightly coupled computations will be attractive at ever larger scale.

Finally, clouds can provide seamless extension of familiar desktop tools and interfaces, allowing computing and analysis to scale within the same environment that researchers use every day. We can leverage consumer software, just as we have leveraged consumer hardware. There is no reason our computational science tools and our "every day" tools need be different.

Shameless Microsoft Plug

To this point, I've written about clouds in a vendor-neutral sense. With a nod to the company name now on my paycheck, if you haven't already, I encourage you to take a look at Windows Azure and its cloud computing and storage services. In addition to rich web services, there is both open source and Visual Studio programming support. In a future post, I will describe an Azure example application for computational science. Here endeth the marketing pitch.

Insight, Not Infrastructure

As Richard Hamming famously noted, "The purpose of computing is insight, not numbers." Dan's computational science corollary is simple, "The purpose of computational science infrastructure is scientific discovery, not big iron bragging rights." It's time to focus on what matters and embrace the future. Our graduate students and post-docs will thank us.

Beyond The Azure Blue

From the first day I arrived at Microsoft, my academic colleagues have been asking me about Microsoft's strategy for cloud computing and when (or if) there would be public announcements. Those questions rose to a crescendo as academic groups prepared responses to the NSF eXtreme Digital (XD) TeraGrid solicitation. All I could say was that we were working on a plan, and it would become clear soon.

I don't normally pitch Microsoft products in the blog, preferring to discuss science policy, technology research and development and global competitiveness. However, something big just happened at Microsoft, something I think will affect all of us. Moreover, as I write this, the Pacific Northwest sky is clear and azure blue, and that doesn't happen often this time of year. An omen, perhaps?

Microsoft Azure Cloud Services

At our Professional Developers Conference (PDC), Microsoft announced Azure, our cloud computing platform, with on-demand compute and storage to host, scale and manage Internet or cloud applications. The press release has additional business perspective and a link to the presentation. Azure is one element of the vision Ray Ozzie (See "Mind to Mind: Building Innovation") described in his 2005 Internet Services Disruption memorandum.

The simplest description of Azure is that the initial release allows you to develop hosted Windows applications using .NET Services, though future releases will support unmanaged code and open source tools as well (Eclipse, Ruby, PHP, and Python). Within Azure, a fabric controller manages application instances and access to storage via SQL Data Services (SDS), and it hosts applications atop virtualized multicore hardware. Finally, Microsoft's Live Services offerings will be layered atop the Azure framework.

You can read the white paper for details on the Azure design and usage approach. In addition, the software development kit (SDK) is available for download. In addition to the Azure SDK itself, there are SDKs for Visual Studio, .NET and SDS Services. Finally, there are Java and Ruby SDKs for .NET Services as well. This is a Community Technology Preview (CTP), meaning Microsoft welcomes feedback on these early capabilities and will continue to expand the capabilities of Azure over the coming months.

Science and Technology Implications

Earlier in the year, I wrote on both my blog and in HPCWire ("Dan's Cloudy Crystal Ball") about the possibility of outsourcing research computing services and infrastructure to the cloud. I noted then that the explosive growth of computing as an enabler of scientific discovery had strained university capabilities and Federal research budgets. Given our current economic crisis, university operating budgets and Federal research expenditures will be under even greater strain and there will be increased scrutiny on the need for each investment.

In a world of (at best) modest research budget increases, we must ask hard questions about the best use of limited funds. Cloud computing offers a potential mechanism to increase the efficiency of current research, ensure continuity of critical data and enable new kinds of research not now feasible.

In this model, researchers focus on the higher levels of the software stack -- applications and innovation, not low-level infrastructure. University and Federal research agency administrators, in turn, procure services from the providers based on capabilities and pricing. Finally, the cloud service providers deliver economies of scale and capabilities driven by a large market base and energy efficient infrastructure. Remember, computing infrastructure exists to enable discovery, not as monuments to technological prowess.

In addition to efficiency, the scalability of cloud services and infrastructure opens new research possibilities. Not only is it possible federate multidisciplinary research data at far larger scales than possible in a university environment (think tens to hundreds of petabytes of low latency storage), we can escape the pernicious cycle of transitory research infrastructure.

How often have we created data repositories as part of research projects, only to find few mechanisms to ensure their long-term sustainability and access by the broader research community? How often have we faced a miasma of distributed data sources with unknown provenance and non-compatible metadata, each supported pro bono on a best effort basis? (See my recent comments on digital document preservation.) Instead, imagine multidisciplinary data fusion and mining, where students can pose queries against integrated but diverse data sources using robust tools?

Finally, by leveraging "pay as you go" models, we can trade time and scale on a continuous basis. Imagine applying 50,000 processors for one hour at the same cost as 50 processors for one thousand hours. In the cloud, the integral under the curve is the same and the costs are comparable, but the research effects are qualitatively different.

The Standard Questions

The standard questions always arise about new approaches to computing. Cloud services and data storage inevitably raise the standard ones.

• Is it reliable and will my data persist?
• Is it safe, private and secure?
• Will I be captured and become captive?
• What does it cost and what if I can't continue paying?

We tend to forget that there are complementary issues about local infrastructure because we have already internalized and accepted the implications and risks. Moreover, local failures are rarely publicized.

• What happens if my disks crash?
• What if I can't pay for backups or maintenance or physical plant or …?
• What if my network is penetrated?

These are the standard cost/benefit/risk tradeoffs. One must make them based on statistics, economics and practical constraints. Remember that we debated the same issues when we shifted research computing from vendor-backed HPC designs to predominantly commodity components.

Let's Reason Together

I welcome discussion of how we can exploit cloud services and infrastructure effectively – all cloud infrastructure, not just Microsoft's Azure. To do this, the cloud service providers, hardware vendors, universities and Federal government must work together to outline an agenda, conduct experiments at scale and speak with a united voice on the opportunities.

It's a sunny day, but my head is in the clouds.

Preserving the Past: Educating the Future

A recent front page article in the New York Times, entitled In the Digital Age, Federal Files Slip into Oblivion, really caught my attention. The article described a problem with which I am painfully and intimately familiar, namely the struggle to preserve the electronic record of government processes and deliberations. Quoting from the article,

Many federal officials admit to a haphazard approach to preserving e-mail and other electronic records of their work. Indeed, many say they are unsure what materials they are supposed to preserve.

This confusion is causing alarm among historians, archivists, librarians, Congressional investigators and watchdog groups that want to trace the decision-making process and hold federal officials accountable. With the imminent change in administrations, the concern about lost records has become more acute.

Even with an army of government clerks, there is a limit to how many pieces of paper the federal government could produce. However, the explosive growth of digital communications and document preparation has far outstripped the processes and technology available to the Library of Congress and the National Archives and Records Administration (NARA). However, it is not just the volume of digital data, it is the diversity of electronic formats and the myriad of physical devices on which the data is stored.

Imagine receiving a truck filled with PC disk drives and being expected to identify, curate and manage the data contained on them. Sound daunting and farfetched? It isn't. This is precisely what the Clinton White House delivered to the National Archives for preservation; though it included a mere 32 million e-mail messages. (Remember that the White House did not have Internet access until DARPA and Randy Katz wired it in the 1990s.)

Given the growth of electronic communication since the early 1990s, the Bush administration will undoubtedly have generated hundreds of millions of e-mail messages that must be preserved, along with a plethora of electronic documents in a dizzying array of file formats. In addition to the standard challenges of document identification, extraction and preservation, the Archives of course must deal with national security and classification issues, further exacerbating the challenge.

I have seen this struggle first hand, as a member of the Advisory Committee for the Electronic Records Archive (ACERA), the digital document preservation project of the National Archives. The National Archives are building a web accessible, indexed repository that will eventually host at least a portion of the torrent of digital data pouring from the federal government. It is an arduous and difficult journey, with more work ahead.