Reed's Ruminations: A Blog by Dan Reed

March Madness, The Zone and The Magic

It is March Madness time, with the Final Four in sight. Fear not. Despite the ominous connotations, the United States is not in the throes of an epidemic of mad cow disease (bovine spongiform encephalopathy) or another prion-based dementia, nor do we tremble in the apocalyptic shadow of Four Horsemen. Rather, we are collectively enthralled by the unscripted athletic performance art that defines the NCAA basketball tournaments.

They're On Fire!

As I have watched bits of the basketball games and felt the faint spark of yearning for my long faded and oh so limited athletic prowess, I cannot help but think about what athletes of all levels in diverse sports call The Zone. It's that all too ephemeral interval when time seems to slow down, when the basket looks as big as a lake each time the ball touches your hands, when the baseball drifts slowly up to the plate like a candy-filled piñata, when a magnet seems to draw the golf ball toward the cup.

Yes, it's an endorphin high, but it is also a psychological state. Truth be told, this is why we practice and why we play the games. Yes, we are competitive and we want to win, but our deeper motivation is the love of the game – and if we are really lucky – the hope of entering The Zone.

An entire sports vernacular has evolved to describe this experience: "Putting on a clinic." "He/she is on fire." "He/she is scoring at will." "They're playing a different game." Every sport has phrases that capture some element of our vicarious joy, and the most iconic performances assume near mythic status in our collective sports memories.

Discovery Magic

In scientific circles, we speak in more antiseptic and officious terms. The rhetoric concerns the joy of discovery, a passion for research, or a boundless curiosity. It is more personal, though, than we often are willing to admit. Doing so would suggest unseemly emotion in domains where we pretend to be dispassionate Diogenesians, seeking only truth. Our intellectual subterfuge belies reality – The Zone has an analog in research.

Over the past thirty years, I have asked scientists of varying distinction and age and across cultures and disciplines to explain the rationale for their intellectual passions. After some prodding and embarrassment, most tell a variant of the same story. It's the shared tale of The Magic. I suspect you know it too.

The Magic is an experience that comes unbidden and without warning, touching each in different ways. It is the wide eyed wonder in a child when insight burns bright, inspiring a lifetime yearning for another discovery fix. It is the theorist who creates a proof to illuminate connections among the heretofore unrelated. It is the experimentalist astonished at seeing for the first time what nature has hidden in plain view. It is the computer scientist who incarnates logic as software, with results that surprise and amaze.

The Magic first touched me when I was eight, as I spent hours in the garage conducting simple optics and electricity and magnetism experiments. As a college and graduate student, and later as a young professor, The Magic would come back to me in the quiet of the night. It was when jumbled facts suddenly fused and I knew – truly knew – something for the first time. The lemmas and theorems glistened with beauty; the code took life and danced; and the words flowed like water.

For me, it was a Zen state, achievable only after long hours of focused thought. I hoped it would last forever, but I knew it would fade, as it always did. At best, I could hold it for a few hours, sleepless and enthralled.

The Magic, scientists yearn for it with the lingering hunger of memory and thee dreams of future insights.

The Human Condition

Perhaps in another brane where the quantum foam bubbles in different ways, an announcer is exclaiming, "She's on fire! That's the fifth longstanding conjecture proved this year!" Or in another, we hear the color commentator say, "He's putting on a clinic! Look at the elegance of that API specification. That's a Hall of Fame move!"

Though we can dream, scientists are unlikely to experience the adrenalin rush that comes when thousands of screaming fans chant their name. Yet The Zone and The Magic, though different, are fundamentally the same. Both are essential elements of the human condition. Each is the progeny of long hours of human endeavor, with the power to inspire and amaze.

As March Madness unfolds, watch the players in The Zone, think of The Magic, and share your own story.

Innovation Via Client Plus Cloud: Microsoft-NSF Partnership

Update

Here is a bit more background and news coverage:

• New York Times article
• NSF-Microsoft press conference
• Inside HPC article
• Microsoft's own PR article

Today, February 4, Microsoft and the U.S. National Science Foundation (NSF) announced a collaborative project where Microsoft will offer individual researchers and research groups (selected through NSF's merit review process) free access to advanced client-plus-cloud computing. Our focus is on empowering researchers via intuitive and familiar client tools whose capabilities extend seamlessly in power and scope via the cloud.

I am very excited about this, as it is the fruit of nearly two years of planning and collaboration across Microsoft product and research teams, as well as many discussions with researchers, university leaders and government agencies. As part of this project, a technical computing engagement team, led by Dennis Gannon and Roger Barga, will work directly with NSF-funded researchers to port, extend and enhance client tools for data analysis and modeling. We also appreciate the support of the Microsoft Dreamspark, Technical Computing, Windows Azure, Azure Dallas, Public Sector, education and evangelism (DPE) teams, among others, to build and deliver this capability.

By extending the capabilities of powerful, easy-to-use PC applications via Microsoft cloud services, our objective is to broaden researcher capabilities, foster collaborative research communities, and accelerate scientific discovery by shifting the focus from infrastructure to empowerment. This is a potentially profound shift in how we innovate in the 21^st century.

21^st Century Innovation

The brief history of computing is replete with social and technological inflection points, when a set of quantitative and qualitative technology changes led to new computing modalities. I believe we are now at such an inflection point in computing-mediated discovery and innovation, enabled by four social and technical trends:

• Massive, highly efficient cloud infrastructures, driven by search and social networking demands
• Explosive data growth, enabled by inexpensive sensors and high-capacity storage
• Research at the interstices of multiple disciplines, conducted by distributed, virtual teams
• Powerful, popular and easy-to-use client tools that facilitate data analysis and modeling

The first two of these are well documented, and I have written about them before. (See Beyond the Azure Blue and Language Shapes Behavior: Our Poor Cousin Data.) The late Jim Gray also lectured and wrote perspicuously about data-intensive scientific discovery, which he called The Fourth Paradigm. As a logical complement to theory, experiment and computation, the fourth paradigm is based on extracting insight from the prodigious amounts of social, business and scientific data now stored in facilities whose scale now dwarfs all previous computing capabilities.

Climate change and its environmental, economic and social implications; genetics, proteomics, lifestyle, environment, health care and personalized medicine; economics, global trade, social dynamics and security – these are all complex, multidisciplinary problems whose exploration and understanding depend critically on expertise from diverse disciplines with differing cultures and reward metrics.

As our research problems rise in complexity, the enabling tools must rise commensurately in power while retaining simplicity. I have seen far too many multidisciplinary projects founder on the rocks of infrastructure optimization and complexity, when they should have focused on simplicity, familiarity and ease of use.

As Fred Brooks once remarked, "We must build tools so powerful that full professors want to use them, and so simple that they can." Simplicity really, really matters. It is for this reason that Excel spreadsheets, high-level scripting languages and domain-specific client toolkits are now the lingua franca of multidisciplinary innovation, the harbingers of invisibility.

Invisible Simplicity

Any successful technology ultimately becomes invisible, enabling and empowering while user knowledge of the technology's underlying theory and operation declines. This is success, for it allows all but the technology experts to focus on the raison d'être for computing or any other technology: enabling and expanding the human experience. By that metric, technical computing and its subset – high-performance computing— have experienced a long adolescence.

Sadly, our technical computing experiences have been dominated and shaped by a focus on technology and infrastructure, rather than empowerment and simplicity. We talk routinely of data and software repositories, toolkits and packages; of cyberinfrastructure and technology roadmaps. In our technological fascination, it is all too easy to lose sight of the true objective. Infrastructure exists to enable. If it is excessively complex, cumbersome or difficult to use, its value is limited. The mathematician Richard Hamming's admonition remains apt: "The purpose of computing is insight, not numbers."

Empowering the Majority

To address 21^st century challenges, we must democratize access to data and computational models, recognizing that the computing cognoscenti are, by definition, the minority of those who can and should benefit from computing-mediated innovation and discovery. Instead, we must enfranchise the majority, those who do not and will not use the low-level technologies – clusters, networks, file systems and databases – but wish to ask and answer complex questions. Remember, most researchers do not write low-level code; nor should they need to.

I believe we must focus on human productivity, not cyberinfrastructure, and leverage popular and intuitive client tools that hide infrastructural idiosyncrasies. This means deploying tools like Excel that can manipulate data of arbitrary size, reaching into the cloud to access petabytes of data and executing massive computations for epidemiological analysis as easily as one might balance a checkbook. It means coupling multiple, domain-specific toolkits via a script of a dozen lines, and launching a parametric microclimate study as easily as one searches the web. Perhaps more importantly, it means rethinking public and private sector partnerships for innovation, identifying and leveraging core competencies.

It all comes back to simplicity and invisibility. Technical computing can and should be an invisible intellectual amplifier, as easy to use as any other successful consumer technology. Now is the time; and Microsoft is committed to making it a reality. We look forward to working with the community.

Bridging Cultures for Collaboration

N.B. I also write for the Communications of the ACM (CACM). The following essay recently appeared on the CACM blog.

Several years ago, I was invited to give the welcome address to the extended community participants in a new computational science project. The meeting was being held at a facility a few blocks from the nearest hotel, which happened to be very near the ocean. On the first day of the meeting, I made my way downstairs from my hotel room, only to encounter an unexpected sight.

It seemed there were two major groups staying at the hotel, in addition to those I expected to attend the computational science meeting. One group was dressed in t-shirts, shorts and sandals and would have blended nicely with the nearby beachgoers. The second group was dressed in conservative business suits that would not have drawn a second glance in the corporate world.

Both groups streamed from the hotel, seemingly bound for recreation and business, respectively. Imagine my surprise when I saw members of both groups entering the building where I was scheduled to speak! One was a group of engineers; the other, not surprisingly, was a group of computer scientists. (Full disclosure: I am proud to say that all of my degrees are from engineering schools, and I spent twenty years in a computer science department located in a college of engineering.)

As I stepped to the podium to begin my remarks, I glanced down at my own attire: dress shoes, pants, button down shirt, and a blazer, but no tie. Recognizing the humor of the situation, I began by saying that I was a community bridge builder, the statistical mean of the audience's sartorial distribution. I continued by recommending that those wearing ties get better acquainted with those wearing sandals, as that was precisely the cross-cultural engagement necessary to ensure the success of a computational science or engineering project.

The point of this anecdote is the importance of cultural flexibility and willingness to understand the diversity of reward metrics and expectations across disciplines. Only in so doing can we realize the true power of computational science and engineering, or more broadly, of computing plus X, where X is any of the possible domains of computing applicability, from the humanities, arts and social sciences, through government, business and consumer products, to science and engineering.

All too often, our technical curricula fail to focus on the human aspect of cross-domain collaboration. Technical skills are necessary, but not sufficient. One must also understand and meld the disparate motivations of the collaborative team in a positive and productive way to achieve success.

On Getting The “Right” Answer

Update:  Jim Pool reminds me of the following book, which touches on this issue.

The parallel application contains millions of lines of code, combining multiple models of physical, engineering, biological, social and/or economic processes, operating over temporal and spatial scales that span ten orders of magnitude. It was written by tens or even hundreds of graduate students, post-doctoral associates, software developers and yes, even a few professors, over a decade. It involves numerical libraries and functions from diverse research groups and companies, and a single execution requires thousands of hours on tens of thousands of processor cores. In short, it's a typical example of an extreme scale high-performance computing code.

And The Answer Is …

What is the probability that any execution of this code produces the "right" answer? Does a "right" answer even exist? If so, how might we know? Or, is this a nonsensical question?

These are not simply Gedanken experiments in software engineering or hardware reliability, nor are they just epistemological questions about the philosophy of knowledge. Rather, they are very practical and real questions about the nature of extreme scale computational science. They are the essence of verification and validation (V&V) processes, and we should be much more rigorous and systematic about their application.

The central lesson of software engineering is that regardless of the rigor of design processes, testing methodologies and boundary condition specifications, large applications, regardless of type, will contain multiple errors. We have all seen and experienced examples, from the blue screen of death on personal computers to the loss of the Mars Climate Orbiter due to mixed unit calculations. Computational science applications have no special dispensation to escape this destiny.

Second, because the raison d'être for multidisciplinary applications is enabling researchers to gain insight into complex and often poorly understood phenomena, testing them can be problematic, as the answers are often known only for simplified, model problems or boundary conditions. What constitutes a rigorous test suite when little or no experimental data is available for independent comparison?

Third, there's the small matter of numerical stability. The IEEE floating point standard balances range (exponent) and precision (mantissa) in a fixed number of bits, and necessarily approximates many real numbers. After large numbers of floating point operations, even a stable algorithm and a well-conditioned problem will contain some degree of error. Rarely do we bound the possible error via interval analysis.

Finally, there's the small matter of hardware errors, an increasingly common phenomenon for large memories constructed from high-capacity DRAMs and for microprocessor datapaths. We have been taught that error correcting codes correct DRAM bit errors. In truth, the standard SECDED code only corrects single bit errors and detects double bit errors. Burst DRAM errors (triple bit or more) are not corrected and are increasingly common. Some iterative algorithms can recover from bit errors, converging to the "correct" answer; others cannot.

Computation As Experiment

Are you afraid? We all should be. It is time to embrace the scientific process for computational science. We must view the execution of a large, multidisciplinary code as what it is – an experiment, with all the possible error sources attendant with any physical experiment. This includes repeating the experiment (computation) to determine confidence intervals on the answer, conducting perturbation studies to determine the sensitivity of the answer to environmental (hardware and software) conditions, identifying sources of experimental bias and defining the experiment rigorously for independent verification.

In the days of analog scientific computing via slide rule, we all understood that the computation was an approximation. It's time to relearn that lesson.

Publication Quarks

I write a quarterly column for the Computing Research Association (CRA)'s newsletter, Computing Research News (CRN). The following is a preview of my upcoming column, which will appear in the March 2009 issue.

Over the past thirty years, I have accumulated the common artifact of an academic research career – bookshelves overflowing with research journals and conference proceedings. Each time I pull an old and yellowing volume from my shelves, it is simultaneously nostalgic and thought provoking to read a few randomly selected articles. Not only does this stroll down memory lane illuminate how far we have come, both technologically and theoretically, it shows how profoundly the publication culture of our field has changed.

Conference Hegemony

Not that many years ago, CRA published a "best practices" memorandum entitled, "Evaluating Computer Scientists and Engineers for Promotion and Tenure." At a time when many departments were struggling to make the case to their science and engineering colleagues that conference publications mattered, this memorandum demonstrated that computing conference publications were of a quality comparable to those in archival journals.

The perception battle won, is all right with our publication world? Perhaps, but I suspect not. Our prestigious conferences have become the moral equivalent of highly selective journals in other fields. The computing conference review process is rigorous and highly selective, and polished results are required for publication. In many of our sub-disciplines, the conference paper is the final result. There is no expectation that the preliminary results will be expanded, augmented and published in a journal. Consequently, many – arguably most – of our journals have receded in significance. I believe this is a regrettable and worrisome development.

First, it has truncated the continuum of publication options. In most disciplines, conferences are the venue where late breaking results, thought providing theories and controversial ideas are aired and debated. Many of these later are proven incorrect or validated and expanded with additional data, but the free exchange of ideas stimulates research and innovation. At the risk of sounding like an "old geezer," I encourage you to read some old conference proceedings. It is illuminating to see how our many of our conferences have evolved from idea exchanges to publication venues.

<BEGIN Old Geezer Story>

Recently, I told a group about one of my undergraduate experiences – being caught in an unexpected thunderstorm with my compiler under my arm. That would be the box of punched cards containing my compiler. I spent most of the evening with an iron and ironing board, flattening my cards very carefully, so the card reader could process it and then punch a new, undamaged deck. My audience looked at me as if I were a walking dinosaur. Now back to our regularly scheduled programming.

<END Old Geezer Story>

Our emphasis on the conference cycle has also encouraged and rewarded production of publishing "quarks" – units of intellectual endeavor that can be generated, summarized and reviewed in a calendar year. We now see new faculty and research staff candidates with more publications than were once common in promotion and tenure dossiers.

Do not misunderstand. I am not suggesting that our current conference-centric culture is all bad, merely that we should be more thoughtful regarding the timescales and range of our publication options. I would also humbly suggest that we consider how this approach shapes the types and kinds of research conducted. We all know that quality trumps quantity, and that research results have a wide range of natural sizes and time scales.

Journal Revival

What then becomes of our often languishing journals? Are they a hidebound and archaic notion, doomed to irrelevance by ubiquitous electronic access? To be sure, the nature of publication is in flux in both popular and professional culture, with the physical artifacts likely to disappear. However, the notions of scholarly review and archival recording of research are independent of these artifacts.

I believe we need to restore journals to their rightful place as the lasting archives of scientific knowledge. This will require a cultural shift, making our conferences the harbingers of extended, rigorous publication in journals. Equally importantly, it will require us to review those journal submissions thoughtfully and with alacrity.

As anyone who has ever been the editor of computing journal knows, obtaining timely reviews is challenging. Even with gentle (and sometimes not so gentle) nagging, the weeks can stretch to months; the months sometimes turn to years. Contrast this with other technical disciplines where submissions can be reviewed and published in weeks or months. Is it any wonder that paper authors in our field eschew journals for conferences with known publication dates?

As a discipline, we benefit from the entire continuum of venues for communicating research ideas and results, from informal workshops and conferences to research surveys and expanded publication in archival journals. Let's recognize and embrace the distinct and important roles that each plays in the free and fruitful exchange of research ideas.