Reed's Ruminations: A Blog by Dan Reed

One Life to Live

A few years ago, a foundation ran a billboard campaign with an image of Kermit the Frog and the caption, "Eats Flies, Dates a Pig, Hollywood Star. Live Your Dreams." Beyond being a huge fan of the Muppet Show and its sardonic humor, especially as offered by Statler and Waldorf, I have always loved this billboard because it captures an essential element of human life, the desire to make a difference, to live life with the passion of commitment. It is a feeling Thoreau captured in Walden, to the enduring delight of secondary school English teachers across North America:

I went to the woods because I wished to live deliberately, to front only the essential facts of life, and see if I could not learn what it had to teach, and not, when I came to die, discover that I had not lived.

I was reflecting on this recently as the world lost two innovators in computing. About the first, Steve Jobs, much has been written, but far less about another pioneer of computing, Dennis Ritchie.

Remembering Ritchie

UNIX and C transformed our notions of operating systems and programming, seeding a cornucopia of software tools and technologies. I still have my dog-eared copy of the 1978 Bell Systems Technical Journal (BSTJ), which was devoted to the "UNIX Timesharing System."

I was a graduate student then, and it brought my first exposure to UNIX on a VAX 11/780 at Purdue. We ran VMS part of the time and UNIX the rest, largely because VMS had a decent FORTRAN compiler. The numerical analysts among us used FORTRAN, but for the rest, it was UNIX and C. It was a heady time, when new tools and approaches were transforming academic computing research from a largely theoretical discipline to an experimental science. I created a compiler for John Backus' FP language using lex and yacc on that VAX, and I wrote a Ph.D. dissertation using troff.

As da Vinci supposedly remarked, "Simplicity is the ultimate sophistication." UNIX was elegant in its simplicity and functionality. It and the C language inspired multiple generations of hackers (and I use that word in its original, pure sense) to create so many things. It is not exaggeration to say that one can date operating systems as pre-UNIX and post-UNIX.

Richie's passing was not noted with great fanfare, but he made a difference. All of us in computing honor that.

Meditations on Dreams

In between the endless work deadlines, the email deluge and the frenzied travel, take a deep breath and ask yourself what matters and what you are doing. The answers may surprise you, particularly as mortality seems more than a tale told by the wise and aged to the young and uncertain. Those are the essential facts Thoreau described.

Throughout my life, I have been able to live many of my dreams. Far too many people are not so fortunate. (For the record, eating flies, dating a pig and being a Hollywood star have not been among them.) I have a few dreams left, both in computing and in life, and I suspect you do too. In my case, there is a book or two trying to escape via my fingers. (See Eudora, You Got the Love?)

Seize the opportunity. Live your dreams. Make a difference. It is all that any of us can hope to do.

Analog Computing: Time for a Comeback?

N.B. I also write for the Communications of the ACM (CACM). The following essay recently appeared on the CACM blog.

In the early dates of the automobile, there was a lively competition among disparate technologies for hegemony as the motive power source. Steam engines were common, given their history in manufacturing and locomotives, and electric vehicles trundled through the streets of many cities. The supremacy of the internal combustion engine as the de facto power source was by no means an early certainty. Yet it triumphed due to a combination of range, reliability, cost and safety, relegating other technologies to historical curiosities.

Thus, it is ironic that we are now assiduously re-exploring several of these same alternative power sources to reduce carbon emissions and dependence on dwindling global petroleum reserves. Today's hybrid and electric vehicles embody 21^st century versions of some very old ideas.

There are certain parallels to the phylogenic recapitulation of the automobile now occurring in computing. Perhaps it is time to revisit some old ideas.

Analog History

Use of the word "computer" conjures certain images and brings certain assumptions. One of them, so deeply ingrained that we rarely question it, is that computing is digital and electronic. Yet there was a time not so long ago when those adjectives were neither readily assumed nor implied when discussing computing, just as the internal combustion engine was not de rigueur in automobile design.

The alternative to digital computing – analog computing – has a long and illustrious history. Its antecedents lie in every mechanical device built to solve some problem in a repeatable way, from the sundial to the astrolabe. Without doubt, analog computing found its apotheosis in the slide rule, which dominated science and engineering calculations for multiple centuries, coexisting and thriving alongside the late comer, digital computing.

The attraction of analog computing has always been its ability to accommodate uncertainty and continuity. As Cantor showed, the real numbers are non-countably infinite, and their discretization in a floating-point representation is fraught with difficulty. Because of this, the IEEE floating-point standard is a delicate and ingenious balance between range and precision.

All experimental measurements have uncertainty, and quantifying that uncertainty and its propagation in digital computing models is part of the rich history of numerical analysis. Forward error propagation models, condition numbers, and stiffness are all attributes of this uncertainty and continuity.

Hybrid Futures

I raise the issue of analog computing because we face some deep and substantive challenges in wringing more performance from sequential execution and the von Neumann architecture model of digital computing. Multicore architectures, limits on chip power, near threshold voltage computation, functional heterogeneity and the rise of dark silicon are forcing us to confront fundamental design questions. Might analog computing and sub-threshold computing bring some new design flexibility and optimization opportunities?

We face an equally daunting set of challenges in scientific and technical computing at very large scale. For exascale computing, reliability, resilience, numerical stability and confidence can be problematic when input uncertainties can propagate, and single and multiple bit upsets can disturb numerical representations. How can we best assess the stability and error ranges on exascale computations? Could analog computing play a role?

Please note that I am not advocating a return to slide rules or pneumatic computing systems. Rather, I am suggesting that we step back and remember that the evolution of technologies brings new opportunities to revisit old assumptions. Hybrid computing may be one possible way to address the challenges we face on the intersecting frontiers of device physics, computer architecture and software.

A brave new world is aborning. Might there be a hybrid computer in your hybrid vehicle?

Surrounded By Opportunities

Perhaps the closest I have approached the feeling of intellectual omnipotence (and it has not been often) was when I was a graduate student, teaching undergraduate FORTRAN programming for science and engineering students. Those were the days of punched cards, batch processing and line printer output. It was a time when dinosaurs roamed the earth, my slide rule was still a fond memory, and we were computing on massive machines with less performance than my smartphone has today. (Ignore that part about the dinosaurs, as it was a wee bit earlier.)

During my office hours, students would troop through for help with their programming assignments, bringing program printouts. These printouts began with a "day file" showing the timeline of compilation and execution steps (JCL steps on IBM S/360), followed by the program and then any output. After having seen a few students, you realize that elementary programming errors have just a few equivalence classes, often identifiable from the day file alone. I took great pride in listening to the student description of the problem, predicting the root cause and then immediately locating it on the printout. The student was inevitably amazed. Yes, it was a cheap parlor trick, but as a graduate student, you take gratification where you can find it.

All of this reminds me that sequential programming remains difficult and parallel programming even more difficult, despite all our efforts to tame the unruly beast with syntactic sugar, specification and scripting languages, libraries and tools. I am no expert in parallel programming language design, nor am I a cognitive psychologist, but like many of my readers, I have pulled a few all-nighters writing code. Perhaps that at least qualifies me to have an opinion.

In a world of parallel programming, multicore architectures, dark silicon, and rising software complexity, we face some difficult choices. It's time to learn from the past.

Fading Abstractions

We have been fortunate to date. Although sequential programming at scale is intellectually challenging, we have been riding the wave of ever-increasing processor performance, made possible by semiconductor advances. In turn, these were based on an abstraction – the Mead-Conway design rules –that largely separated device physics from microprocessor architecture. Finally, the von Neumann abstraction has also served us well, allowing decades of software development atop an easily understood model of sequential instruction execution.

Our run of good luck is over. Put another way, we are surrounded by opportunities! Parallelism is now the norm for almost all software developers, not just those of us in high-performance computing. More perniciously, heterogeneous parallelism is here, with disparate cores, accelerators and functional units. To manage dark silicon and exploit continuing transistor density increases, this trend will only accelerate, with a dizzying array of heterogeneous chips and devices challenging code portability and performance tuning. Finally and most worrisome, our device physics abstractions are breaking down, with device physics affecting architecture. We face real silicon reliability, process variation and energy challenges, and near threshold voltage computations emerging.

Holistic Collaboration

The abstractions of device physics, processor architecture and programming models naturally led to a corresponding intellectual specialization. Consequently, those working in materials science and device physics have little in common with those designing software tools. I believe that must change if we are to address our challenges.

We need new, integrated design methodologies and rich collaborations that consider end-to-end design and capability. In many ways, this is a return to the past, necessitated by the desire for continuing increases in performance, mediated by the desire for reliability, scalability, programmability and economic feasibility. It will not be easy, but it is a way forward.