Reed's Ruminations: A Blog by Dan Reed

Microsoft Extreme Computing Group (XCG)

Here is a small item that may be of possible interest, creation of the Extreme Computing Group (XCG) at Microsoft. XCG was formed in June 2009 with the goal of developing radical new approaches to ultrascale and high-performance computing hardware and software. The group's research activities include work in computer security, cryptography, operating system design, parallel programming models, cloud software, data center architectures, specialty hardware accelerators and quantum computing.

HPC: Making a Small Fortune

N.B. I also write for the Communications of the ACM (CACM). The following essay recently appeared on the CACM blog.

There is an old joke in the high-performance computing community that begins with a question, "How do you make a small fortune in high-performance computing?" There are several variations on the joke, but they all end with the same punch line, "Start with a large fortune and ship at least one generation of product. You will be left with a small fortune." Forty years of experience, with companies large and small, has confirmed the sad truth of this statement.

As we all know, the computing industry is extremely competitive, and new trends and technologies have repeatedly had transformative effect. One need look no further than the regular inductees to the Dead Supercomputing Society to see the devastating effects of the ongoing attack of the killer micros on the market for custom high-performance computing system designs. The microprocessor performance increases over the past thirty years due to decreasing feature sizes, higher clock rates and greater architectural complexity have repeatedly dashed the hopes of many high-performance computing entrepreneurs.

The market lesson is that one false step inevitably leads to failure, particularly for startup companies struggling to establish a new niche in the face of commodity economics. It has never been truer than in today's economy, where potential buyers are retrenching and evaluating each purchase with a discriminating and sometimes jaundiced eye. Recently, the high-performance computing industry lost several established companies to merger and acquisition, due to weak market positions. We have also seen startup companies fail due to missteps and financial pressures.

This reminds me of another old analogy, which compares building computer hardware and software to playing pinball – one's reward for playing well is the opportunity to keep playing via free games. The punishment for not playing well is equally clear; one must continue to insert quarters into the machine. Venture capitalists know this well, as they evaluate the pinball skills of those pitching business plans.

Without doubt, we need a new generation of high-performance computing systems, from consumer devices to exascale platforms, to drive innovation, improve health care, manage critical infrastructure and ensure safety and defense. The question is whether the rise of multicore and manycore chips and explicit parallelism in the commodity microprocessor and GPU markets will finally change a few of the rules of the pinball game, via a combination of consumer economics pressures and technological need, the latter due to clock frequency and power limitations.

I believe we are at an inflection point, where new approaches must both survive and flourish if we are to continue to deliver higher performance in effective and reasonable ways. It is worth remembering that Andy Grove's famous comment, "Only the paranoid survive," is but the trailing phrase in a larger, more perspicuous comment, "Success breeds complacency. Complacency breeds failure. Only the paranoid survive."

We cannot be complacent about the future, especially now. We must continue to innovate, even if – especially if – that means adding quarters to the innovation machine.

Escaping from Flatland

In 1884 (no, that's not a typographical error), Edwin Abbott wrote a satire about Victorian England and its social hierarchy, in the guise of a mathematical story about life in a two-dimensional world, whence the whimsical title, Flatland: A Romance of Many Dimensions. If one can look beyond Abbott's misogyny to the crux of the mathematical story, it is an illuminating introduction to geometry in one, two and three dimensions, with some generalizing hints about higher dimensional geometries. (You can read a copy at the Internet Archive, or purchase a hardcopy).

The story is told from the perspective of a square, a resident of Flatland, who receives a visit from an inhabitant of a three-dimensional world – Spaceland. In Flatland, of course, the sphere is perceived only as a circle whose diameter varies based on the sphere's orientation to the plane. The sphere preaches the existence of higher dimensions, but the Flatlander leaders attempt to suppress all such information.

Denizens of Waferland

In fine recursive fashion, the Flatland story is itself a metaphor for our tenacious embrace of our two-dimensional world of semiconductors, particularly as it relates to memory technologies. We are deeply entangled in the angst, ennui, despair and perhaps even the clinical depression related to our encounter with the limits of instruction level parallelism (ILP), sequential execution semantics and the microprocessor power wall.

As a community, we have grudgingly and guardedly recognized the need for multicore processors (See Three Views on Multicore and Manycore: Able Was I Ere I Saw Elba.) However, we are still clinging tenaciously to our dual in-line memory module (DIMM), two-dimensional packaging and double data rate (DDR) memory designs. We need a visitor from the third dimension, preaching the gospel of chip stacking to the denizens of chip waferland.

Chip Stacking: Beyond DDR

We are approaching scaling limits for our pin-based interfaces. Each generation, we have dropped voltages, increased clock rates and doubled the number of words transferred. DDR memory first operated at 2.5V and up to 400 Mb/s, dropped to 1.8V and increased to 800 Mb/s for DDR2, and is at 1.5V and 1600 Mb/s for DDR3. We can see DDR4 on the horizon, perhaps in 2012 at ~1V.

It is time – long past time – for us to move to the third dimension and stack our chips. With chip (die) stacking, need not be constrained by connections to the perimeters of our chips, but can exploit connectivity across a larger fraction of their area. IBM, Intel, Samsung and others are exploring variations of this idea, as this smattering of press releases and articles illustrates.

With lower power, multicore designs, through silicon vias (TSVs), and wafer thinning for heat dissipation, we can crack the memory wall that has plagued us for so long. More to the point, those of us in the big iron/fast iron camp could learn a few things from our compatriots in the embedded systems world, where innovative packaging is a fundamental market driver.

Make no mistake; this will not be easy, as it requires new approaches to via fabrication, as well changing our ecosystem of chipsets and interface standardization processes. We may not have tachyon-based hyperspatial communication, but surely we can escape from Flatland.

When Petascale Is Just Too Slow

N.B. I also write for the Communications of the ACM (CACM). The following essay recently appeared on the CACM blog.

It seems as if it were just yesterday when I was at NCSA and we deployed a one teraflop Linux cluster as a national resource. We were as excited as proud parents by the configuration: 512 dual processor nodes (1 GHz Intel Pentium III processors), a Myrinet interconnect and (gasp) a stunning 5 terabytes of RAID storage. It achieved a then astonishing 594 gigaflops on the High-Performance Linpack (HPL) benchmark, and it was ranked 41^st on the Top500 list.

The world has changed since then. We hit the microprocessor power (and clock rate) wall, birthing the multicore era; vector processing returned incognito, renamed as graphical processing units (GPUs) ; terabyte disks are available for a pittance at your favorite consumer electronics store; and the top-ranked system on the Top500 list broke the petaflop barrier last year, built from a combination of multicore processors and gaming engines. The last is interesting for several reasons, both sociological and technological.

Petascale Retrospective

On the sociological front, I remember participating in the first petascale workshop at Caltech in the 1990s. Seymour Cray, Burton Smith and others were debating future petascale hardware and architectures, a second group debated device technologies, a third discussed application futures, and a final group of us were down the hall debating future software architectures. (I distinctly remember talking to Seymour about his "parity is for farmers" comment regarding memory ECC.) All this was prelude to an extended series of architecture, system software, programming models, algorithms and applications workshops that spanned several years and multiple retreats.

By the way, you can read the original report here; it is fascinating to look back. Paul Messina, Thomas Sterling and others deserve our thanks for launching the seminal activity.

At the time, most of us were convinced that achieving petascale performance within a decade would require some new architectural approaches and custom designs, along with radically new system software and programming tools. We were wrong, or at least so it superficially seems. We broke the petascale barrier in 2008 using commodity x86 microprocessors and GPUs, Infiniband interconnects, minimally modified Linux and the same message-based programming model we have been using for the past twenty years.

However, as peak system performance has risen, the number of users has declined. Programming massively parallel systems is not easy, and even terascale computing is not routine. Horst Simon explained this with an interesting analogy, which I have taken the liberty of elaborating slightly. The ascent of Mt. Everest by Edmund Hillary and Tenzing Norgay in 1953 was heroic. Today, amateurs still die each year attempting to replicate the feat. We may have scaled Mt. Petascale, but we are far from making it pleasant or even routine weekend hike.

This raises the real question, were we wrong in believing different hardware and software approaches were needed to make petascale computing a reality? I think we were absolutely right that new approaches were needed. However, our recommendations for a new research and development agenda were not realized. At least in part, I believe this is because we have been loathe to mount the integrated research and development needed to change our current hardware/software ecosystem and procurement models.

Exascale Futures

I recently participated in the International Exascale Software Project Workshop (IESP), the first in a series of meetings designed to explore organizational and technical approaches to exascale system design and construction. The workshop built on several earlier meetings and studies, including the DARPA exascale hardware study and the forthcoming exascale software study (in which I participated), as well as the DOE exascale applications study. Complementary analyses are underway in the European Union and in Asia.

Evolution or revolution, it's the persistent question. Can we build reliable exascale systems from extrapolations of current technology or will new approaches be required? There is no definitive answer, as almost any approach might be made to work at some level with enough heroic effort. The bigger question is what design would enable the most breakthrough scientific research in a reliable and cost effective way?

My personal opinion is that we need to rethink some of our dearly held beliefs and take a different approach. The degree of parallelism required at exascale, even with future manycore designs, will challenge even our most heroic application developers, and the number of components will raise new reliability and resilience challenges. Then there are interesting questions about manycore memory bandwidth, achievable system bisection bandwidth and I/O capability and capacity. There are just a few programmability issues as well!

I believe it is time for us to move from our deus ex machina model of explicitly managed resources to a fully distributed, asynchronous model that embraces component failure as a standard occurrence. To draw a biological analogy, we must reason about systemic, organism health and behavior rather than cellular signaling and death, and not allow cell death (component failure) to trigger organism death (system failure). Such a shift in world view has profound implications for how we structure the future of international high-performance computing research, academic-government-industrial collaborations and system procurements.

Three Views on Multicore

 

Andrew Chien, Dave Patterson and I have each written articles on the challenges and opportunities inherent in multicore hardware and software for the Community Computing Consortium (CCC) blog. My recent article, on the challenge of software, is now posted. In the article, I argued that we must re-envision parallel computing and a new generation of applications that explicitly exploit the scale and heterogeneity of multicore. You can read the article on the CCC blog or below.

Multicore: It's The Software

For over thirty years, we have watched the great cycle of innovation defined by the commodity hardware/software ecosystem – faster processors enable software with new features and capabilities that in turn require faster processors, which beget new software. The great wheel has turned, but it no more, as power constraints and device physics now limit the performance achievable with single microprocessors.

Multicore chips – those with multiple, lower power processors per chip – are now the norm. Moreover, current multicore chips (those with 4-8 cores/chip) are but the beginning. We can expect hundreds of cores per chip in the future, with diverse functionality (graphics, packet protocol processing, DSP, cryptography and other features).

The software research challenge is clear – developing effective programming abstractions and tools that hide the diversity of multicore chips and features while exploiting their performance for important applications. Hence, we need a vibrant community of researchers exploring diverse approaches to parallel programming – languages, libraries, compilers, tools – and their applicability to multiple application domains.

Microsoft researchers are investigating all of these approaches, from coordination languages for robots and distributed systems to mobile phones to desktops and data center clouds. To engage the academic community, Microsoft funds multicore research projects and many sites, and we have partnered with Intel to fund the Universal Parallel Computing Research Centers (UPCRCs) at the University of California at Berkeley and the University of Illinois at Urbana-Champaign.

As Richard Hamming famously noted, "The purpose of computing is insight, not numbers." In that spirit, I believe our research challenge is to break free from the limitations of the desktop metaphor and exploit the ever greater performance of multicore chips to create new human-computer interaction metaphors that are more natural and intuitive. This will require new approaches to parallel computing education and increased collaboration with researchers in application domains.

As an example, consider one possible future – "spatial computing" – where real-time vision and speech processing, coupled with knowledge bases, distributed sensors and responsive objects, enhance human activities in contextually relevant ways while remaining otherwise unobtrusive. Such an infosphere would adapt to its user's needs and behavior and move seamlessly across home, work and play.

Multicore brings enormously interesting intellectual challenges and the opportunity to rethink much of how we approach computing. Let's embrace the opportunity!

ManyCore: Able Was I Ere I Saw Elba

As I write this, I am attending the ETH
LASER summer school on concurrency, which is being held on the island of Elba. The island sits off the coast of Tuscany, a few miles from Pisa. It is perhaps best known as the place where Napoleon was exiled after his forced abdication and where he spent the interregnum before his final defeat at Waterloo. (Let me express my thanks to Bertrand Meyer for the invitation to speak at the summer school.)

As I prepare to deliver six lectures on multicore and cloud computing here on Elba, the geographic irony of grand ambition, hubris and ignominious defeat is not lost on me. We have been struggling for the past forty years to find elegant and efficient parallel and distributed programming paradigms, with modest success. To continue my 19^th century metaphor, we remain, as Matthew Arnold sadly put it, "Swept with confused alarms of struggle and flight, where ignorant armies clash by night."

The Virtuous Cycle

Metaphors aside, our struggle is real and extraordinarily important. The virtuous cycle that has long driven the computing industry is in flux, and if it is broken, we will struggle restart it – for deep economic reasons. The desire for new functionality leads to richer, more complex software, which imposes greater demands on extant hardware, with concomitant performance constraints. In turn, this stimulates demand for faster processors, and the cycle of innovation turns.

One interesting corollary of this cycle is that we demand new, faster processors at the same price, rather than the same performance at a lower price. This consumer demand generates the revenue needed to fuel commercial software development, power new chip designs and fund semiconductor fabrication line construction. These are multibillion dollar (U.S.) investments, ones only repaid if tens to hundreds of millions of units are sold. In turn, this creates deep partnerships among companies such as Microsoft, Intel,
AMD and the PC vendors. A similar virtuous cycle exists in the mobile telephone market.

ManyCore Directions

This ecosystem of software and hardware innovation is challenged by consumer parallelism, in the form of large-scale multicore (manycore) chips. No longer can we expect dramatic increases in single core performance, due to power and heat dissipation constraints on consumer devices. Perhaps more tellingly, all of us wonder what the next "killer app" will be that excites and incents consumers to buy new, manycore systems. Personally, I believe it will be some combination of graphics-intensive massively multiplayer games (MPGs) and contextually-adaptive, situationally-aware information spheres. (One can think of the latter as Vannevar Bush's
Memex reborn.)

Of course, as James Thornton (CDC) said many years ago, "Anyone who says he knows how computers should be built should have his head examined! The man who says it is either inexperienced or really mad." I'm too old to be inexperienced, so perhaps I am really mad after all!

The fundamental question is how large multicore (manycore) chips and development software will evolve. I see at least four architectural directions, at least two of which are already commercially prevalent. The first is the "cookie cutter" homogeneous multicore design, exemplified by today's Intel and AMD flagship x86 offerings, along with similar homogeneous multicore designs from SUN (Niagara) and IBM (Power5-7). Tilera's
TILE64 and Intel's Larrabee are other examples of this approach, combining standard cores (x86 and MIPS-derived, respectively) with a regular interconnect (mesh for Tilera and ring for Larrabee).

The second is ISA-compatible homogeneous, but performance heterogeneous multicore. In this case, one combines, for example, a smaller number of complex, out-of-order cores with a larger number of simpler, in-order cores. The motivations for this approach are simple – the implications of Amdahl's Law and the need to execute legacy code efficiently while still delivering some of the performance and power advantages of low-power multicore. In this same spirit, Mark Hill has recently written a great paper about the importance of performance heterogeneity in multicore design.

The third is functional heterogeneity, currently exemplified by chips such as the IBM Cell, AMD's announced Fusion chip and a host of embedded and domain-specific chips. I believe there are many opportunities for architectural innovation in this space, combining graphics, DSP, packet processing, cryptographic functions, SDR and a host of other functions with novel interconnects and memory sharing approaches.

The fourth is what I call non-traditional architectures that embody more radical alternatives. One great example of this class is Doug Burger's
TRIPS system, based on data graph execution. Doug recently joined Microsoft Research from the University of Texas at Austin, and I am excited about the collaboration possibilities.

Back to the War

I believe we are at an inflection point in parallel computing, with the economic impetus of consumer parallelism now driving us. Let us hope that we fare better than the warriors in another 19^th century war, the Battle of Balaclava during the Crimean War. As Tennyson wrote so well, the Light Brigade charged into the mouth of hell. With cannons to the right (multicore architecture), cannons to the left (programming models) and cannons in front (next-generation applications), we ride into an unknown future, one fraught with peril but also with opportunity.

Showing Up and Two Corollaries

"Eighty to ninety percent of life is showing up." The line has been variously attributed to Yogi Berra, Woody Allen or even an anonymous wag. It's wise, though obvious advice – showing up and doing the expected generally allows one to avoid a host of problems. Appearing for jury duty avoids one being held in contempt of court, and you can't fly if you don't show up at the airport on time.  I was reflecting on the implications of "showing up" while at a recent meeting in Italy.

Show Up and See What Happens

My friend, Dave Turek, IBM's Vice President for Deep Computing, once explained IBM's open source and Linux strategy by saying that IBM had a deeply considered, two phase strategy for Linux and clusters for HPC, "Show up and see what happens." As he once remarked at an NCSA Private Sector Partners (PSP) meeting, "We've showed up. Now, we are waiting to see what happens."

At NCSA, we partnered with IBM in 2001 to deploy two of the first large-scale commodity clusters for open scientific use: two 1 teraflop systems based on Intel Pentium III and Itanium processors. At the time, this was a radical, almost heretical idea – deploying commodity PC clusters as production HPC platforms. Of course, such commodity clusters now dominate the Top500 list.

In a reprise of this experience, Microsoft and NCSA recently partnered to deploy Windows HPC Cluster 2008 on the latest incarnation of commodity cluster hardware. (The customer story has the technical details). I don't generally evangelize for Microsoft products in this blog, but I was very impressed that Windows HPC Cluster achieved substantially higher performance on the same hardware than did Linux. Microsoft, in the form of Kyril Faenov's HPC team, has definitely "showed up" in this space in a big way, and I think there are great opportunities to offer not only Windows compute clusters but also backend acceleration for desktop applications. Of course, all of this is ultimately connected to the ferment in cloud computing.

Avoid the Obviously Wrong

At the recent Cetraro meeting on High-Performance Computing and Grids, Miron Livny extended the "show up and see what happens" maxim by offering a corollary, "Show up and avoid doing something stupid." His observation was that evolutionarily, human success was defined by avoiding being trampled by a woolly mammoth, eaten by a hungry Bengal tiger or falling into a crevasse.

The computing implication of Livny's corollary is that one should do reasonable things when presented with opportunities. In terms of research infrastructure, this means avoiding our academic tendency to delight in second system syndrome – building complex systems that embody all of our personally favorite features without determining if they are either needed or useful.

At Cetraro, we debated the impact of the multicore revolution, the similarities and differences between Grids and clouds, and the commonalities between future exascale systems and the architecture of megascale data centers. (By the way, if you have not read the Department of Energy's exascale computing study, I highly recommend it.)

There are deep technical challenges in all of these areas. However, we must avoid being trampled by the woolly mammoths; this domain is fraught with academic, government and industrial politics. I believe we need a wider dynamic range (time horizon, risk/reward and fiscal scale) of research and development projects if we are to solve these problems.

I have made this point many times, most recently as part of the PCAST report on the U.S. NITRD program. I am scheduled to testify about this again to the House Science and Technology Committee on July 31. I will report on the hearing in August.

Do Simple Things Quickly

At the same Cetraro meeting, I opined that there was a second corollary, "Do the obvious, simple things quickly." I think this is the key lesson to be drawn from web2.0 mashups, and the rapid evolution of commercial clouds. The simplicity of the APIs and hosted infrastructure encourages external groups to innovate rapidly. We have seen the clear evidence of this in the explosive growth in social networking sites and in the hosted services that have appeared.

By contrast, I think this is one of the places we have struggled with academic Grids. The software has often been too complex, and this complexity has been engendered by the distributed nature of the participating organizations, requiring "glue code" to integrate disparate policies and infrastructure across virtual organizations. In contrast, mashups and cloud services can be deployed quickly (by academic standards) using very simple APIs and service level agreements (SLAs). It will be interesting to see how the Grid/Cloud mashup evolves.

HPC and Climate Change: Senate Hearing

On Thursday, May 8, I testified to the U.S. Senate Committee on Commerce, Science and Technology. The full committee hearing was on improving the "Capacity of U.S. Climate Modeling for Decision Makers and End-Users." The other members of the hearing panel were

• Alexander (Sandy) MacDonald, Director, Earth System Research Laboratory, Deputy Assistant Administrator for Laboratories and Cooperative Institutes, Office of Oceanic and Atmospheric Research, National Oceanic and Atmospheric Administration
• James (Jim) Hack, Director, National Center for Computational Sciences, Oak Ridge National Laboratory 
• Edward Sarachik, Co-Director, Center for Science in the Earth System, Professor Emeritus of Atmospheric Sciences, Joint Institute for the Study of the Atmosphere and Ocean, University of Washington
• Bruce Carlisle, Assistant Director, Massachusetts Office of Coastal Zone Management
• John Walsh, President's Professor of Climate Change, Chief Scientist, International Arctic Research Center, University of Alaska

Jim Hack and I represented the computing and computational science issues, and the other four focused on the climate aspects. Within a few weeks, our written testimony will be posted on the Committee's hearing page, and in due time (many months), our oral testimony will appear in the Congressional Record.

Random Musings

Like many of you, I give lots of public (and not so public) presentations, on a variety of topics. A couple of those were recently captured and placed on the web. The first was one of the opening talks at the Big Data Symposium, recently held at Yahoo!, as part of the Computing Community Consortium (CCC) big ideas series. If you are struggling to sleep, you can catch the video here.

The second was an open-ended discussion on Microsoft's Channel9. I rambled on about multicore processors, big data center design challenges and a bit of web history. If the first video didn't put you to sleep, you can get the double feature effect here

Salishan, Exascale and Heterogeneous Multicore

Update: The proccedings from the meeting are now online.

Every year, the three U.S. Department of Energy (DOE) weapons laboratories (LANL, LLNL and SNL) organize a workshop on the state of high-performance computing and computational science. By long tradition, the meeting is held at the Salishan lodge on the Oregon coast. The attendees are drawn from the three weapons laboratories, the DOE Office of Science laboratories, other computing-intensive elements of the U.S. government (e.g., the NSA and DoD HPC Modernization Program), the NSF supercomputing centers, key academic researchers and industry high-performance computing leaders. I've been attending for many years, both as the former NCSA director and as a federal science policy wonk.