Reed's Ruminations: A Blog by Dan Reed

Bridging Cultures for Collaboration

N.B. I also write for the Communications of the ACM (CACM). The following essay recently appeared on the CACM blog.

Several years ago, I was invited to give the welcome address to the extended community participants in a new computational science project. The meeting was being held at a facility a few blocks from the nearest hotel, which happened to be very near the ocean. On the first day of the meeting, I made my way downstairs from my hotel room, only to encounter an unexpected sight.

It seemed there were two major groups staying at the hotel, in addition to those I expected to attend the computational science meeting. One group was dressed in t-shirts, shorts and sandals and would have blended nicely with the nearby beachgoers. The second group was dressed in conservative business suits that would not have drawn a second glance in the corporate world.

Both groups streamed from the hotel, seemingly bound for recreation and business, respectively. Imagine my surprise when I saw members of both groups entering the building where I was scheduled to speak! One was a group of engineers; the other, not surprisingly, was a group of computer scientists. (Full disclosure: I am proud to say that all of my degrees are from engineering schools, and I spent twenty years in a computer science department located in a college of engineering.)

As I stepped to the podium to begin my remarks, I glanced down at my own attire: dress shoes, pants, button down shirt, and a blazer, but no tie. Recognizing the humor of the situation, I began by saying that I was a community bridge builder, the statistical mean of the audience's sartorial distribution. I continued by recommending that those wearing ties get better acquainted with those wearing sandals, as that was precisely the cross-cultural engagement necessary to ensure the success of a computational science or engineering project.

The point of this anecdote is the importance of cultural flexibility and willingness to understand the diversity of reward metrics and expectations across disciplines. Only in so doing can we realize the true power of computational science and engineering, or more broadly, of computing plus X, where X is any of the possible domains of computing applicability, from the humanities, arts and social sciences, through government, business and consumer products, to science and engineering.

All too often, our technical curricula fail to focus on the human aspect of cross-domain collaboration. Technical skills are necessary, but not sufficient. One must also understand and meld the disparate motivations of the collaborative team in a positive and productive way to achieve success.

Language Shapes Behavior: Our Poor Cousin, Data

The Sapir-Whorf Hypothesis holds that language and culture shape behavior and thought. The idea has a long, rich and sometimes controversial history, though recent cognitive linguistic research, together with brain imaging, has lent it new credence. Lately, I have been reflecting on the nature of linguistic relativity in scientific research, particularly for HPC and cloud computing. In particular, how has our focus on floating point operations per second (FLOPS) shaped discourse, science politics and outcomes?

Look, It's Terascale!

If I were to remark, "That is a terascale system," you would likely interpret my comment as an assessment of the system's computing performance, more specifically its floating point capabilities. For the cognoscenti, the sentence likely engenders visions of multicore systems with out of order instruction issue and deep pipelining, small clusters with fast interconnects and (perhaps) some accelerators such as GPUs. You might even reflect on the historical evolution of high-performance computing, when a terascale system was ranked first on the Top500 list about a decade ago.

Given my remark, you would be unlikely to eye your desktop computer with its terascale (terabyte) commodity disk drive. Yet, based on its storage capacity, that desktop is just as surely a terascale platform as is the compute cluster. Ah, you say, I have mixed capacity (bytes) with capability (operations). Terabyte per second storage systems are rare, you say. Absolutely true, yet as this bit of third rate linguistic legerdemain illustrates, language and, perhaps more importantly, connotation really does trump denotation in our discourse.

Whether Petascale and Exascale?

Today, the world's fastest computing platforms have broken the one petaflop mark on the high-performance Linpack (HPL) benchmark, and planning groups across the United States, Europe and Asia are debating political and technical approaches for the construction of exascale computing systems. I cannot help but wonder, though, why we in technical computing talk so little about petascale and exascale data platforms? Why is there no international initiative to build a low latency, high bandwidth (many terabytes/second to even a petabyte/second) data analysis engine with multiple exabytes of storage capacity?

I suspect the reason lies in our background and linguistic heritage. Most of us in high-performance computing came from mathematical backgrounds, where success was defined by a proof and an equation and their embodiment in a computational model, rather than by insight gleaned from large-scale data analysis. I believe it is time we found a lingua franca that bridges FLOPS and bytes and reclaim the full connotation of exascale.

I Want My Exabytes

It is worth remembering that the explosive growth of observational data is itself largely a product of inexpensive CMOS sensors, based on the same semiconductor technologies that begot microprocessors. The examples of high resolution instruments are legion, from the Large Hadron Collider (LHC) and its international hierarchy of data archives to the proposed Square Kilometer Array (SKA) radio telescope, which may produce as much as an exabyte of data every few days. This data tsunami is not limited to the physical sciences; the biological and social sciences are being inundated as well.

One of the major lessons from web search and cloud data centers is the power of truly massive scale, near real-time data analysis. When anyone with a cheap cell phone and a web browser can extract data and insights from a non-trivial fraction of the human knowledge base, behavior and culture are transformed. I would like to believe that we can bring the same data-driven analytics to scientific research as are routinely exploited to find a good lasagna recipe. We need a balanced exascale initiative, in every sense of the word, lest we be a technical confirmation of the Sapir-Whorf hypothesis.

A test of ping.fm

That’s One Small Step for {a} Man …

N.B. I have posted additional reflections on the lessons of Apollo for sustaining innovation on the CACM blog.

On Sunday afternoon, July 20, 1969, I rode my AMF Roadmaster bicycle to the local Gulf gas station and asked for their cardboard model of the Apollo Lunar Module. (Some of you will recall that it was originally called the Lunar Excursion Module, whence the common name LEM.) I pedaled home in time to watch the Apollo 11 lunar landing later that afternoon, holding the cardboard model of the LEM in my hands.

It was only years later that I came to understand how perilous that landing had been, due to inadequate lunar images for landing site selection, a critical fuel shortage and an overloaded guidance computer with less power and storage than one of today's engineering calculators. As Apollo 13 later showed, the risks were real. Nonetheless, that day remains burned in my memory and it made no difference to me that the images transmitted from the moon were in black and white, because all we had at home was a black and white television.

I was amazed then, as I am now, that NASA had planned for the astronauts to sleep before venturing to the lunar surface. Fortunately for me, the decided to forgo the sleep period and stepped onto the lunar surface while it was still before my bedtime, as a twelve year old.

It Only Seems Miraculous the First Time

As a budding geek and a voracious reader of science fiction, I was convinced then that the lunar landings were the beginning of humanity's long-term commitment to space exploration. Forty years have proven me wrong. As I noted in Innovation: A Plane of Excellence, my friend Thomas Sterling once posed a question that still haunts me, "In which year of birth (1930 or 1970) would one have had the higher probability of walking on the moon?" We know the answer. What really happened?

We all know the "space race" was a creature of the Cold War. After the Apollo 11 landing, public interest waned quickly, the Vietnam War consumed increasing amounts of the U.S. federal budget, and science had always taken back seat to the race itself. (Harrison Schmitt was the only geologist among the astronauts who walked on the moon.) Many people do not realize that the NASA budget was already in decline at the time of the first landing.

Exploration Touches Something Primal

At the request of the White House Office of Science and Technology Policy (OSTP), a panel chaired by Norm Augustine is currently reviewing NASA's human space flight program, with a six month timeline to produce recommendations. We can all debate the cost/benefit ratios of human and robotic space exploration, and I will be the first to argue for the power and value of automated exploration. However, there's something primal about being there. It touches something deep in our psyches, to explore and see for ourselves. More to the point, we spend more money on far less important things.

In a Sunday New York Times editorial, Tom Wolfe captured this conundrum, noting that NASA lacked – in his words – a philosopher, someone who could capture the passion and thrill of exploration and relate that to the public. It's simple really; it's a hunger older than history, a passion never fully satisfied – the eternal desire to discover and to know. Let's go back, and then go beyond – Mars beckons.

IT’s Not Easy Being Green (Actually, It Is Easy If You Try)

I can hear the groan's already at the bad pun embodied in the title of this essay. (Here's the cross-cultural deconstruction. First, there's the capitalized "IT" for information technology. Then, there's the reference to energy efficient computing ("green"). However, the cultural reference is to the Muppet Show and Kermit the Frog, whose refrain often was, "It's not easy being green.")

Contrary to the pun, actually, it is easy to be green, if one wants to do so. This is a point I tried to make in an interview that is part of a series InsideHPC has begun on green computing. In the interview, I discussed the challenges and opportunities associated with energy efficient computing at scale, whether operating large-scale data centers or petascale high-performance computing systems.

In the interview, I pointed out that the most obvious way to reduce computing-related energy consumption is simply to power down and turn off those systems not being used -- QED. However, that is insufficient alone. After all, one presumably wants to do some computing. Thus, systems and infrastructure must be designed for energy and operational efficiency and must be managed appropriately during operation.

As a practical matter, one really wants to maximize a ratio

(Effective operations)/(Cost times Watts)

Simply put, the goal is to maximize the number of effective operations relative to cost and energy consumption. This convolves many ideas, including the match of the application to the system (application execution efficiency), the system design and architecture, energy and power supply efficiency, packaging and cooling overhead, market costs for power and hardware and the costs of people and money.

Microsoft is absolutely committed to green computing, across its entire range of products and infrastructure, and a big portion of my team's research is related to developing more energy efficient computing systems at scale.