Reed's Ruminations: A Blog by Dan Reed

HPC In The Clouds: ISC Reflections

I just spent an all too brief two days in Hamburg, attending the International Supercomputing Conference (ISC). It was a chance to catch up with some old friends, discuss the ever changing nature of computing technology and reflect on the Weltanschauung that is the HPC community. In addition to the buzz around the latest Top500 list and the new Japanese machine atop the list, there were many debates about the intersection of HPC and cloud computing.

My friend and former collaborator in North Carolina, Wolfgang Gentzsch, organized a session at ISC on HPC and the Cloud, where I was delighted to participate. In my talk, I made four points, several of which were echoed by Josh Simons from VMWare in a great talk that followed mine.

First, HPC has experienced several punctuated equlibria, as we transitioned from one parallel computing model to another. From instruction level parallelism in the mainframe era through vector systems, MPPs, SMPs, clusters and clusters plus accelerators, each transition has brought technical and cultural challenges. The technical challenges have been well documented; the cultural challenges are less often discussed, for they affected the technology providers (vendors), the technology operators (HPC centers and facilities) and the technology users (scientists, engineers and other researchers.) Clouds bring another set of technical and cultural challenges.

When any new technology appears, there is a great temptation to see it through the lens of the old, either in nomenclature or behavior (see "horseless carriage"), or attempt to converge the new into a variant of the old (see assimilation by the Borg). Many corporate acquisitions fail for just this reason, failing to recognize one is acquiring something precisely because it is not like the acquirer. Successful adoption and exploitation occur when all parties are willing to work together to help create the ecosystem of capabilities needed. I believe we are moving down the path of successful adoption with cloud-based HPC.

Second, I talked about the need to continue to democratize access to HPC, targeted the "excluded middle," those potential and actual technical computing users who need more computing than available on desktops but who find traditional HPC too difficult to use. They are the majority of scientists and engineers. Client plus cloud acceleration is one possible solution to this problem, and could empower a new set of HPC users. See HPC and the Excluded Middle.

Third, I reminded the audience that there is another burgeoning challenge and opportunity – big data. We have big problems with big data, and the scientific and engineering communities are drowning in the data produced by a new generation of sensors and scientific instruments. We need simple, easy to use tools that allow researchers to mine and extract insight from that data. In that spirit, I discussed our continuing work on Excel DataScope, which allows one to use the familiar Excel interface while applying rich analytics algorithms in the cloud. See The Zeros Matter: Bigger Is Different.

Finally, I highlighted the worldwide progress of Microsoft's research cloud partnerships. We now have over 75 active projects, launched in partnership with research agencies in the U.S., the EU, Japan, Australia and Taiwan. (See Cloud Seeding: Stimulating Discovery and Innovation.) The partnerships are an opportunity for us to work together to understand how clouds can best support technical computing and discovery, driving change in the cloud infrastructure space and service providers and also educating researchers and HPC providers about this technical and cultural transition.

It's always exciting to be a part of these technical collaborations.

Nothing Lasts Forever

Those of us in computing and especially those of us in high-performance computing (HPC) have been received Pavlovian conditioning to believe that computers will always be faster if we just wait eighteen to twenty-four months. (Excuse me while I wipe the saliva off my chin, triggered by thinking about this.) It seems this trend has ever been so and ipso facto, so shall it ever be.

Unfortunately, no exponential continues forever, at least outside the mathematics textbooks. All technology-based advances are ultimately limited by something, usually some physical or economic limit. Innovation then shifts to other metrics, against which advances can be measured and valued. Computing is no exception.

Leavin' On A Jet Plane

In the heady days of aviation, when world speed records were set every year, it must have seemed that the sky was literally the limit on achievable airspeeds. And it was, for supersonic flight proved impractical given the societal response to sonic booms, the fuel inefficiency of high-speed transports and the limiting characteristics of materials at high temperatures. The SR-71 may well have been the apogee of hypersonic flight, Aurora rumors not withstanding.

Once those practical limits on airspeed were reached, aerospace innovation did not stop, it simply changed direction. The Boeing 747 and Airbus A380 fly at subsonic speeds, just as the Boeing 707 and 727, but they carry far more passengers and have longer range. Similarly, improvements in jet engines dramatically increased fuel efficiency and reduced maintainence costs. In turn, glass cockpits and electronics improved safety.

Perhaps most of all, changing expectations and airline deregulation democratized air travel, making it economically accessible to almost all, though with all the ambience one associates with riding a prison bus on a hot summer day. (Parenthetically, there are days I regret the loss of amenities as board yet another airplane. I still remember hot meals on flights, and I mourn the loss of legroom.)

The Need for Speed

There is no reason to believe that computing is exempt from the cruel realities of bounded exponentials. The difficulties of continued Dennard scaling and the rise of dark silicon are both challenging our ability to deliver higher performance single and multicore chips. (See Battling Evil: Dark Silicon) Instead, there is an increasing emphasis on function-specific accelerators and devices, hardware-software co-design and lower power designs for embedded use.

In the HPC world, the performance of the systems on the Top500 list has been rising faster than single processor performance, in large measure due to increasing system sizes. This is not, in the long term, a sustainable trend. Similarly, design studies for exascale systems suggest the need for dramatic improvements in energy efficiency (operations/joule), system reliability and programmability.

These may – or may not – be achievable at economically viable price points. The risk of brute force solutions and "research by acquisition" is that we build a Concorde SST, an economic dead end. Instead, we need to conduct the basic research and explore the new ideas in high-performance computing that might solve some of our fundamental problems. After all, studies of hypersonic scramjets continue.

Lest I be called a naysayer or predictor of doom, let me note that there are plenty of other opportunities for innovation in computing in general and in high-performance computing in particular. In addition to pursuing scramjet designs, we need the HPC equivalent of the Boeing 737 – an everyday workhorse that is reliable, efficient, easy to operate and ubiquitous. We must address the infamous excluded middle, making HPC accessible for everyday use by businesses, researchers and governments. There's plenty of room for innovation in algorithms, software tools, programming models, resource management, and energy efficient design.

Remember, no exponential is forever.

On Licking the Cookies

All organizations and cultures develop their own idiosyncratic acronyms, abbreviations, lingo and metaphors. (Yes, there are probably a few similes and alliterations as well.) One need look no further than the social networks and Twitter to find a veriable cornucopia of such phrases and acronyms (e.g., LOL or OMG). The hacker community (the original one) also gifted us with the gems RTFM and UTSL.

I was once offered my first full time job in the software industry because I knew what JCL was needed to apply a PTF to BTAM. That's IBM speak for applying a software patch to telecommunications software. In this sense, Microsoft is not unique, for its worldwide employees have developed their own lexicon of metaphors and abbreviations, as well as appropriating others were useful. As a relative newcomer to the corporate world after twenty-five years in academia, several of these abbreviations, words and phrases immediately caught my ear.

Licking the Cookies

Not to be confused with the joy of double stuff Oreos or even licking the spoon, licking the cookies refers to claiming action or skill in an area but in a way that prevents anything from happening. It is intended to conjure an image of a cookie plate, where one licks the cookies then returns them to the plate, rendering them inedible by all. As the phrase suggests, you have not eaten the cookie (i.e., taken action) but you have prevented others from doing so either.

Cookie licking is endemic to any large organization or culture where multiple teams compete for credit and accomplishment. We have all seen this behavior in our organizations and occasionally been guilty of it ourselves. This is one of the more pernicious forms of non-collaborative, non-team play.

After understanding the phrase, I realized that subtle and not so subtle forms of cookie licking occur in academia as well. I have seen it in debates over academic directions, discussions of research priorities and in paper and grant proposal reviewing. Of course, academia has its own set of arcane phrases.

Eating Your Own Dog Food (aka Dogfooding)

Dogfooding is a hoary classic, well used and abused across the computing milieu. It refers to using early (alpha or beta) versions of a system you and others have developed before it is released, often as an informal tester. It's not haute cuisine or even nouveau cuisine; it's dog food, edible and nutritious, but neither elegant nor tasty.

Learnings

Finally, there are learnings, as in "Our learnings on this suggest optimization is our first priority." The first time I heard this linguistic oddity, I was nonplussed and puzzled, confused and uncertain, perhaps even mortified, but I was not chagrined. After hearing it several times, I finally asked if the speaker really meant insights, understanding, observation, or even experience. The answer, of course, was yes. Nobody I have yet asked, however, knows the origin of the usage, though I am sure the exegesis would be fascinating.

BFN, I am OOF.

Battling Evil: Dark Silicon

As a boy, I remember reading all the superhero comic books, from the standard Superman and Batman to the Green Lantern, Thor, The Flash and Spider-Man. Some superhero stories and backstories were better than others, but all were escapism from small town life. For me, Thor was always a favorite, though I am not quite sure why. Maybe I just liked the head gear! The summer "action movie" season is upon us, and several of these superheros will grace the screen in new Hollywood fantasies, battling the forces of evil.

All of this made me think about the resident evil now facing us in computing – the rise of dark silicon. The very phrase sounds ominious and it is, for I believe it will profoundly reshape how we think about computing in the next decade. But before we talk about the grand denouement where our Justice League of silicon process experts, computer architects and software gurus do battle with dark silicon, perhaps we should begin with a bit of backstory.

Moore's Law and Silicon Scaling

We are all familiar with Moore's Law, namely that the number of transistors in a given area doubles roughly every two years. It is worth noting that Gordon Moore never really claimed this was a law, merely that it was a statistical truth based on observational data and technology trends. It is also interesting how a rather informal 1965 Electronics paper, entitled "Cramming More Components onto Integrated Circuits," had such impact. Over the past thirty years, the "law" has held true due to the hard work and creativity of a great many people, as well as many billions of dollars of investment in process technology and silicon foundries.

For a detailed understanding of why reducing transistor feature sizes really has been possible, I highly recommend reading the original, seminal paper by Robert Dennard, Fritz Gaensslen, Hwa-Nien Yu, V. Leo Rideout, Ernest Bassous and Andre LeBlanc, entitled "Design of ion-implanted MOSFET's with Very Small Physical Dimensions." It appeared in the October 1974 issue of the IEEE Journal of Solid-State Circuits, and it defined a scaling process that has driven the semiconductor industry ever since.

Simply put, Dennard scaling says that reducing the length, width and gate oxide thickness of transistor features by a constant k results in transistors that are k² times smaller, k times faster, and dissipate k² less power. This was Moore's Law made manifest and has let us move from feature sizes measured in micrometers to nanometers. Since the elucidation of Dennard scaling, all has been well in Metropolis, at least until very recently.

Power and Performance: Evil Lurks

As transistor sizes continued to shrink and clock rates continued to increase, we have seen the rise of high-k dielectrics and silicon on insulator (SOI) to address static current leakage. In that spirit, one of my colleagues recently remarked that transistors are now less like solid doors that keep out the cold (current flow) unless open and more like screen doors that always have some air flow (current leakage) even when closed.

Transistor density, clock rates and chip sizes have also made heat dissipation an ever bigger problem. While at Intel, Pat Gelsinger once remarked that unless things changed, chip temperatures would approach that of the sun's surface, an obvious absurdity only possible in the comic books. All of this has led to multicore designs, bounds on clock rates for most microprocessors and decreases in operating voltages.

We have also seen new semiconductor processes and transistor design options to reduce leakage current. (Intel's recent FinFET announcement is one such example.) Although I expect to see transistor feature sizes continue to shrink, with sub-11 nanometer feature sizes achievable, it will not be easy, nor will it be cheap. Lest there be any doubt, the cheap part really matters, for it is the economics that will determine if scaling continues.

These semiconductor challenges have been mirrored by an increasingly empty bag of general purpose architectural tricks. Almost all of the techniques once found in supercomputer processor designs – superpipelining, scoreboarding, and vectorization – are now in mainstream microprocessors. It is not obvious that we can extract more parallelism while maintaining the illusion of sequential execution. In fact, the evidence is substantial that we cannot.

Battling Evil: Dark Silicon

That's the backstory. Now let's turn to the true evil lurking among us, dark silicon. If one considers process scaling trends and limits on heat dissipation, the conclusion is rather obvious. We soon will have (and in many cases already do have) chips with more transistors than can be concurrently activated. The practical implication is that most of the chip silicon will be "dark" (unpowered) most of the time.

If it's dark most of the time, is it of any value? Yes, but only if we embrace our fears. This is the true end of the hegemony of von Neumann computing and the birth of a new era, one based on chips with a diverse set of specialized functional units, accelerators and tailored architectures that each target specific tasks. The mobile world has already learned this lesson, for functional specialization increases energy efficiency (more operations/joule) and allows designers to mix and match the set of functional units to target domains. Arguably, heterogeneous multicore is one, very tentative step in this direction for client and server computing.

This transition to diverse functional units, accelerators and tailored architectures will also challenge many of our assumptions about programming models and system software, as well as the economics of chip design. In this new world, hardware/software co-design becomes de rigeur, with devices and software systems deeply interdependent.

It is a different ecosystem, with different cultures and different processes. It means far less general-purpose performance increases, much more diversity in the hardware, elevation of multivariate optimization – power, performance, reliability – in programming models, new system software resource management challenges and a plethora of new applications.

More generally, with lots of silicon real estate (dark silicon), chip designers and architects have the space to place functional blocks that are used only part of the time. However, at some point (and we are arguably close to that point now), we will have deployed those functional units that cover the common tasks and adding more will have possibly deleterious effects due to larger chip size. That potentially drives the economics to smaller chip sizes, which, given foundry capacity, leads to lots of small chips. That is the collision of the big chip server world with the small chip mobile and Internet of Things world, a battle already raging.

To address these challenges and battle dark silicon, we need new ideas in computer architecture, system software, programming models and end-to-end user experiences. It's an epic struggle for the future of computing. Giants are battling, and not all will survive, nor will the survivors emerge unscathed. Heroes will also emerge, sometimes from unexpected places.

I think I left my hammer, Mjölnir, around here somewhere. Do you feel your spider sense tingling?

Microsoft/NSF Cloud Research Partnership

Today, Microsoft and the U.S. National Science Foundation (NSF) announced the latest awardees for the joint Microsoft-NSF research partnership in cloud computing. (See the NSF announcement: The Sky Is No Limit: 13 Research Teams Compute in the Clouds.) This is a continuation of the program that then NSF Assistant Director for Computer and Information Science and Engineering (CISE), Jeannette Wing, and I announced in February 2010 (See Microsoft and the National Science Foundation Enable Research in the Cloud and Democratizing Research: How "Client Plus Cloud" Computing Can Amplify What's Possible for Scientists).

As part of this agreement with the NSF, Microsoft is making available access to Windows Azure, Microsoft's cloud computing platform. In addition, a support team of Microsoft researchers and developers is working with grant recipients to equip them with a set of common tools, applications and data collections that can be shared with the broad academic community, and also providing expertise in research, science and cloud computing. Broader details on the program and its research tools and software for Windows Azure can be found here.

Diverse Projects

Increasingly, the important scientific questions lie at the intersections of traditional disciplines, and insights from multidisciplinary collaborations drive innovation, economic development and our response to complex problems and natural disasters. This is one of many reasons I am so pleased with the technical diversity among the list of NSF-Microsoft award recipients.

All of the award recipients were selected via NSF's rigorous peer review process, which emphasized the scientific merit of the proposed work. Projects range from exploring scientific applications of cloud computing as well as extensions of cloud computing with new tools and techniques.

• Cornell University (Kenneth Birman) - Building Scalable Trust in Cloud Computing
• J. Craig Venter Institute (Audrey Tovchigrechko) - Bettering Interactive Protein-Protein Docking
• SUNY at Buffalo (Tevfik Kosar) - Enhancing Stork Data Scheduler for Azure
• University of California, San Diego (Kenneth Yocum) - Utilizing Continuous Bulk Processing
• University of Colorado, Boulder (Richard Han) - Enabling Mobile Cloud Computing
• University of Michigan, Ann Arbor (Qiaozhu Mei) - Refining Language Models using Web-scale Language Networks
• University of North Carolina at Charlotte (Zhengchang Su) - Predicting Transcription Factor Binding Sites for Genes
• University of South Carolina (Jonathan Goodall) and the University of Virginia (Marty A. Humphrey) - Managing Large Watershed Systems
• University of Southern California (Viktor Prasanna) - Tackling Large Scale Graph Problems
• University of Texas at Austin (Michael Walfish) - Storing Data with Minimal Trust
• University of Washington (Magdalena Balazinska) - Understanding Relational Data Markets
• Virginia Tech (Wuchun Feng) - Conducting Intensive Biocomputing
• Virginia Tech (Kwa-Sur Tam) - Effectively and Widely Using Renewable Energy Sources

  Worldwide Reach

The Microsoft/NSF partnership is but one part of a broader international program that the eXtreme Computing Group (XCG) and Microsoft Research are building with the scientific research agencies worldwide. We believe cloud computing, coupled with powerful client tools, can transform the nature of research. This worldwide program currently targets collaborations with several national and international research partnerships, including

• National Institute of Informatics of Japan Info-plosion project
• European Commission FP7 funding program, Venus-C project
• U.K. Engineering and Physical Sciences Research Council (EPSRC) Horizon project
• French INRIA-Microsoft collaboration on cloud computing
• Australian engagements with NICTA, ANU and CSIRO

Power via Simplicity

This Microsoft/government partnership is about so much more than access to cloud computing resources. The deep partnership between academic and Microsoft researchers, creation and release of easy-to-use client tools and an exploration of the new world of massive data are all key elements of our shared journey to a new model of data rich analysis enabled by powerful client tools and cloud services.

Any successful technology ultimately becomes invisible, enabling and empowering without requiring the user to focus on the idiosyncrasies of the technology itself. Technical computing can and should be an invisible intellectual amplifier, as easy to use as any other successful consumer technology. As I wrote last year when we first launched this program, it is really about transforming how we conduct research by broadening researcher capabilities, fostering collaborative research communities, and accelerating scientific discovery by shifting the focus from infrastructure to empowerment.