Reed's Ruminations: A Blog by Dan Reed

White Spaces and Adaptive Communication

I have been spending quite a bit of time on approaches to address the spectrum crisis – the seemingly insatiable demand for more wireless communication as the number of Internet-connected devices continues to grow. We are working with a consortium in the UK to host a major white spaces trial. Details can be found on the Microsoft blog, along with a video explaining the challenges and opportunities.

HPC In The Clouds: ISC Reflections

I just spent an all too brief two days in Hamburg, attending the International Supercomputing Conference (ISC). It was a chance to catch up with some old friends, discuss the ever changing nature of computing technology and reflect on the Weltanschauung that is the HPC community. In addition to the buzz around the latest Top500 list and the new Japanese machine atop the list, there were many debates about the intersection of HPC and cloud computing.

My friend and former collaborator in North Carolina, Wolfgang Gentzsch, organized a session at ISC on HPC and the Cloud, where I was delighted to participate. In my talk, I made four points, several of which were echoed by Josh Simons from VMWare in a great talk that followed mine.

First, HPC has experienced several punctuated equlibria, as we transitioned from one parallel computing model to another. From instruction level parallelism in the mainframe era through vector systems, MPPs, SMPs, clusters and clusters plus accelerators, each transition has brought technical and cultural challenges. The technical challenges have been well documented; the cultural challenges are less often discussed, for they affected the technology providers (vendors), the technology operators (HPC centers and facilities) and the technology users (scientists, engineers and other researchers.) Clouds bring another set of technical and cultural challenges.

When any new technology appears, there is a great temptation to see it through the lens of the old, either in nomenclature or behavior (see "horseless carriage"), or attempt to converge the new into a variant of the old (see assimilation by the Borg). Many corporate acquisitions fail for just this reason, failing to recognize one is acquiring something precisely because it is not like the acquirer. Successful adoption and exploitation occur when all parties are willing to work together to help create the ecosystem of capabilities needed. I believe we are moving down the path of successful adoption with cloud-based HPC.

Second, I talked about the need to continue to democratize access to HPC, targeted the "excluded middle," those potential and actual technical computing users who need more computing than available on desktops but who find traditional HPC too difficult to use. They are the majority of scientists and engineers. Client plus cloud acceleration is one possible solution to this problem, and could empower a new set of HPC users. See HPC and the Excluded Middle.

Third, I reminded the audience that there is another burgeoning challenge and opportunity – big data. We have big problems with big data, and the scientific and engineering communities are drowning in the data produced by a new generation of sensors and scientific instruments. We need simple, easy to use tools that allow researchers to mine and extract insight from that data. In that spirit, I discussed our continuing work on Excel DataScope, which allows one to use the familiar Excel interface while applying rich analytics algorithms in the cloud. See The Zeros Matter: Bigger Is Different.

Finally, I highlighted the worldwide progress of Microsoft's research cloud partnerships. We now have over 75 active projects, launched in partnership with research agencies in the U.S., the EU, Japan, Australia and Taiwan. (See Cloud Seeding: Stimulating Discovery and Innovation.) The partnerships are an opportunity for us to work together to understand how clouds can best support technical computing and discovery, driving change in the cloud infrastructure space and service providers and also educating researchers and HPC providers about this technical and cultural transition.

It's always exciting to be a part of these technical collaborations.

Nothing Lasts Forever

Those of us in computing and especially those of us in high-performance computing (HPC) have been received Pavlovian conditioning to believe that computers will always be faster if we just wait eighteen to twenty-four months. (Excuse me while I wipe the saliva off my chin, triggered by thinking about this.) It seems this trend has ever been so and ipso facto, so shall it ever be.

Unfortunately, no exponential continues forever, at least outside the mathematics textbooks. All technology-based advances are ultimately limited by something, usually some physical or economic limit. Innovation then shifts to other metrics, against which advances can be measured and valued. Computing is no exception.

Leavin' On A Jet Plane

In the heady days of aviation, when world speed records were set every year, it must have seemed that the sky was literally the limit on achievable airspeeds. And it was, for supersonic flight proved impractical given the societal response to sonic booms, the fuel inefficiency of high-speed transports and the limiting characteristics of materials at high temperatures. The SR-71 may well have been the apogee of hypersonic flight, Aurora rumors not withstanding.

Once those practical limits on airspeed were reached, aerospace innovation did not stop, it simply changed direction. The Boeing 747 and Airbus A380 fly at subsonic speeds, just as the Boeing 707 and 727, but they carry far more passengers and have longer range. Similarly, improvements in jet engines dramatically increased fuel efficiency and reduced maintainence costs. In turn, glass cockpits and electronics improved safety.

Perhaps most of all, changing expectations and airline deregulation democratized air travel, making it economically accessible to almost all, though with all the ambience one associates with riding a prison bus on a hot summer day. (Parenthetically, there are days I regret the loss of amenities as board yet another airplane. I still remember hot meals on flights, and I mourn the loss of legroom.)

The Need for Speed

There is no reason to believe that computing is exempt from the cruel realities of bounded exponentials. The difficulties of continued Dennard scaling and the rise of dark silicon are both challenging our ability to deliver higher performance single and multicore chips. (See Battling Evil: Dark Silicon) Instead, there is an increasing emphasis on function-specific accelerators and devices, hardware-software co-design and lower power designs for embedded use.

In the HPC world, the performance of the systems on the Top500 list has been rising faster than single processor performance, in large measure due to increasing system sizes. This is not, in the long term, a sustainable trend. Similarly, design studies for exascale systems suggest the need for dramatic improvements in energy efficiency (operations/joule), system reliability and programmability.

These may – or may not – be achievable at economically viable price points. The risk of brute force solutions and "research by acquisition" is that we build a Concorde SST, an economic dead end. Instead, we need to conduct the basic research and explore the new ideas in high-performance computing that might solve some of our fundamental problems. After all, studies of hypersonic scramjets continue.

Lest I be called a naysayer or predictor of doom, let me note that there are plenty of other opportunities for innovation in computing in general and in high-performance computing in particular. In addition to pursuing scramjet designs, we need the HPC equivalent of the Boeing 737 – an everyday workhorse that is reliable, efficient, easy to operate and ubiquitous. We must address the infamous excluded middle, making HPC accessible for everyday use by businesses, researchers and governments. There's plenty of room for innovation in algorithms, software tools, programming models, resource management, and energy efficient design.

Remember, no exponential is forever.