Reed's Ruminations: A Blog by Dan Reed

Computing Futures: Technical, Economic, and Geopolitical Challenges Ahead

In February 2022, I wrote an essay entitled American Competitiveness: IT and HPC Futures – Follow the Money and the Talent, where I outlined a series of challenges facing the state of the global information technology ecosystem. These spanned the shifting locus of innovation and economic power, from those companies we have traditionally called computing vendors (Intel/AMD, HPE, and IBM) to the hyperscalers and social media giants (Amazon, Facebook, Google, and Microsoft, along with their Asian siblings), the influence of smartphone vendors (Apple and Samsung), and the artificial intelligence juggernaut driving them all.

I also discussed the end of Dennard scaling and the slowing/end of Moore’s Law, both of which have ended the “free lunch” cycle of faster and cheaper microprocessors every two years. Then, within this framework, I outlined the technical, economic, and cultural challenges bedeviling leading edge high-performance computing, where ever larger machines deliver increasingly small fractions of the peak hardware performance. (It is often the case that complex, multidisciplinary applications achieve single digit percentages of the peak hardware performance.) I ended by discussing the sad state of U.S. STEM education and the implications of these trends for U.S. global competitiveness and national security.

As I had hoped, the essay generated a great deal of federal policy and vendor introspection, as well as considerable Internet chatter. 

Based on many discussions and helpful feedback from several readers – including Doug Burger, Jim Larus, Tony Hey, and John Shalf, I asked my friends and colleagues, Dennis Gannon and Jack Dongarra (this year’s Turing Award winner), to help me formalize my blog observations in a March 2022 arXiv paper, entitled Reinventing High Performance Computing: Challenges and Opportunities. There, we expanded the high-performance computing history lesson and expounded on the rise of cloud computing and deep learning, something Dennis and I were privileged to see first-hand at Microsoft.  Later in March, I participated in an insideHPC podcast, where I discussed several of our observations.

We followed this with a standing room only discussion in November at SC22, with additional panel members Dorian Arnold (Emory), Jack Dongarra, Torsten Hoefler (ETH Zurich), and Kathy Yelick (UC-Berkeley). (For those of you who attended SC22 or are ACM/IEEE members, you can view the panel discussion here.)  Then in January 2023, John Shalf organized a panel discussion with Kathy, Neil Thompson, and me at the Exascale Computing Project (ECP) annual meeting, where we discussed the topic as well.

Finally, the Communications of the ACM has published an abridged version of our arXiv paper in the February 2023 issue.  The latest version of the article is entitled HPC Forecast: Cloudy and Uncertain, and you can listen to Dennis, Jack, and me discuss the maxims and their implications at this link.

A Few Maxims

In both our arXiv paper and our Communications of the ACM paper,  we outlined some maxims that we believe should guide any consideration of advanced computing futures.

• Maxim One: Semiconductor constraints dictate new approaches. The “free lunch” of lower cost, higher performance transistors via Dennard scaling and faster processors via Moore’s Law is at an end. (Most do not realize that the inflection point for transistor costs occurred near the 20 nanometer feature size.  Since then, chip feature sizes have continued to decline, with more transistors per unit area, but the per transistor costs began to rise due to yield challenges and rising fabrication facility costs.) This minimax problem of maximizing chip yields, minimizing costs, and maximizing performance has driven the adoption of the chiplet model, mixing and matching multiple chips via interconnects such as UCIe, which opens new opportunities for scientific computing optimization and domain-specific accelerators.

• Maxim Two: End-to-end hardware/software co-design is essential. Given current semiconductor constraints, substantially increased system performance will require more intentional end-to-end co-design, from device physics to applications. (See Advanced Computing: Integrative Thinking for the Future, for a discussion of end-to-end design.) There is also an important corollary about changes to procurement models at scale.
• Maxim Three: Prototyping at scale is needed to test new ideas. Across multiple axes – semiconductors, chiplets, AI hardware, cloud innovations – the computing system is now in great flux.
• Maxim Four: The space of leading edge HPC applications is far broader now than in the past. Domains dominated by complex optimization problems and time-dependent partial differential equations will always matter, but other areas of advanced computing are of high and growing importance, enabled by advances in deep learning. The AI revolution is also driving the hardware ecosystem in new ways.
• Maxim Five. Cloud economics and deep learning have changed the supply chain ecosystem. The largest HPC systems are now dwarfed by the scale of commercial cloud infrastructure deployments.
• Maxim Six. The societal implications of technical issues really matter, and a diverse and inclusive domestic and global workforce is critical. This speaks to our challenges around workforce development and attracting and retaining talent individuals.

Chips Shortages and Globalization

While we were sounding the alarm about the technical and economic challenges in advanced computing, the public debate on what became the U.S. Chips and Science Act continued apace. Ultimately, the act appropriated money to help “on shore” a larger fraction of U.S. semiconductor manufacturing and authorized, but did not fully fund, major increases in funding for the National Science Foundation (NSF) and the Department of Energy’s (DOE) Office of Science. (See Working the Hill for some perspectives on where my research interests and public policy roles – as chair of the National Science Board, which oversees NSF, and chair DOE’s Advanced Scientific Computing Advisory Committee (ASCAC) – intersect.)

How did we find ourselves in this situation, you might ask?  Simply put, via a series of locally rational and arguably, even short-term optimal decisions, companies focused on quarterly profits and shareholder value. They outsourced chip fabrication and device manufacturing, benefiting from lower offshore costs, and focused on design and integration of the chips into high margin products.  Meanwhile, Intel, the dominant U.S. semiconductor fabricator, struggled with the transition to extreme ultraviolet lithography (EUV) and ceded global semiconductor leadership to Taiwan Semiconductor Manufacturing Company (TSMC). Nevertheless, all seemed well until two things upset the global supply chain – COVID-19 with disruptions in global supply chains and the rising economic tensions and national security concerns involving China. Now, economic uncertainties, declining tech sector profits, and corporate layoffs have further exacerbated the challenges.

The pressure to maximize short-term gains obscured the longer term need to control the means of production, exposing the risks of what my friend Tren Griffin calls the power of wholesale transfer pricing (i.e., the bargaining power of company A that supplies a unique product XYZ to Company B which may enable company A to take the profits of company B by increasing the wholesale price of XYZ.) Put another way, it is very attractive to be a fabless semiconductor company, but only when your semiconductor fabrication partner is both inexpensive and highly reliable.

As Warren Buffet famously said about the hidden risks some companies take, “Only when the tide goes out do you discover who’s been swimming naked.” The U.S. has been swimming naked for quite a while, dependent on international sources for key semiconductor components. If you have tried to buy anything containing embedded electronics – and almost everything these days does, from automobiles to greeting cards – then you know there has been a global chip shortage, manifest in rising product costs and even scarcity.  Nor has high-performance computing been immune from these shortages. To complete construction of the Frontier supercomputer at Oak Ridge National Laboratory, the U.S. Department of Energy had to involve national security priorities to secure access to scarce chips.

How big is the U.S. dependence on Asian semiconductor foundries? As the chart shows, Taiwan has about 65 percent of the global market, with TSMC holding the largest fraction of this, followed by UMC, PSMC, and VIS.  Korea (Samsung) has about 18 percent, China (SMIC and HHGrace) has about 5 percent, and the United States is in the “other” category – the final 12 percent – along with Global Foundries.

Why don’t we just build more fabrication plants, you say? We are, stimulated by Chips and Science Act subsidies, but building new facilities takes time and serious money.  To put this in perspective, a state-of-the-art fabrication line for high-end microprocessors and graphics accelerators (GPUs) now costs at least $10B and is rising. Moreover, much of the chip shortage is not at the bleeding edge, but for older chip technologies (e.g., for car electronics) that are fabricated on fully depreciated semiconductor fabrication lines (e.g., 45-28 nanometer feature sizes).  Vendors rarely build new lines for older technologies, they simply extract cash from existing fabrication lines, and until the recent economic downturn, there was unprecedented demand for devices normally fabricated on these lines.  (The New York Times has a reasonable summary of the global semiconductor dynamics.)

National Security

Semiconductors have become more than just an economic engine; they are now critical to any country’s national security and global aspirations.  Indeed, semiconductor manufacturing and supply chain control have become proxies in the ongoing battle for global hegemony.  In a bit – but only a bit – of hyperbole, at the World Economic Forum event in Davos, Intel’s Pat Gelsinger recently called chips the new oil of geopolitics. The U.S. Department of Commerce recently tightened export controls around China, both to purchase and to manufacture certain high-end semiconductor chips used in military applications. (See the October 2022 Department of Commerce declaration.)

Meanwhile, the new U.S. National Security Strategy focuses on protecting semiconductor supply chains and continued advances in semiconductors, advanced computing, and next-generation communications, among others. The U.S. National Security Advisor, Jake Sullivan, has highlighted four pillars of U.S. strategy:

• Investing in the science and technology ecosystem,
• Nurturing top STEM talent,
• Protecting technology advantages, and
• Deepening and integrating alliances and partnerships.

All these goals emphasize three families of technologies, including (a) biotechnologies and biomanufacturing, (b) clean energy, and (c) computing-related technologies, including microelectronics, quantum information systems, and artificial intelligence.

Nor are these issues of concern only to the U.S.  Korean leaders have raised similar concerns. Whatever one might wish to be true, it is increasingly clear the world is moving rapidly into a post-globalization era, with the increasingly likelihood of a 21^st century non-kinetic Cold War, albeit one with very different dynamics than the one of the 1950s to 1980s.  The global economy – and especially that of the U.S. and China – are deeply intertwined in ways unthinkable during the days of the standoff between the Warsaw Pact and NATO. Meanwhile, the Russian invasion of Ukraine has highlighted the uncertain future of Taiwan and western dependence on TSMC.

Coda

How might the U.S. respond in a positive way?  The short answer is we must out-compete – invest more and continue to expand an ecosystem that attracts and retains the best and brightest talent the planet can produce, both domestic (The Missing Millions) and international. I have sought to summarize some of these objectives in another series of essays.  As I wrote in Not Alone: Leaders in Focus, let’s dream big and bright about a larger, more diverse and inclusive science and engineering ecosystem that continues to be a magnet for global talent. (See For Science and Society, the Future Begins with Better Dreams.)

Cyberinfrastructure: Lots of Room at the Edge

Twenty years ago, while director of the National Center for Supercomputing Applications (NCSA) at the University of Illinois, I gave a series of presentations on the future of cyberinfrastructure.  Looking through my presentation archives, I found this diagram, which emphasized building up (i.e., larger and more capable computing systems and scientific instruments) and building out (i.e., ubiquitous access to computing systems and instruments of all sizes, including distributed sensor/actuator networks).

At the time, we did not understand that fungal infections from chytridiomycosis were in part responsible for widespread death and extinction among amphibian species. I suggested with some seriousness that we instrument selected biological environments with MEMS smart dust and track flora and fauna movements in situ. Or, as I put it at the time, “Why not give every frog an IP address?” Of course, understanding ecosystem evolution eventually became the raison d'etre for the for the National Ecological Observatory Network (NEON) – more on that later. I also had several conversations with then National Science Foundation (NSF) Director, Rita Colwell, about the prospects for HPC-enabled in silico computational modeling of biological processes, something only now becoming possible. (See In Silico Whole Cell Modeling.)

With the perspective afforded by twenty years, how have we fared in realizing this integrated vision?  Building up, we have done well in most contexts, but we now face the twin realities of escalating costs and technology challenges at very large scales, both for computing platforms and for scientific instruments. It is unclear how many multibillion-dollar instruments or computing platforms we have either the political or economic will to construct. As for building out, it is a mixed story of broader access to midrange commodity Linux clusters and new instruments, but we are far from having created a ubiquitous scientific infosphere.

Building Up

In 2002, the first terascale computing systems had just become operational, including the NSF TeraGrid, which connected NCSA, SDSC, Argonne National Laboratory, and Caltech via a then extraordinarily fast 40 Gb/s transcontinental network. (See NCSA@30: The Revolution Continues.) At the time, we were still dreaming of petascale data storage systems, and research discussions about the path to petascale computing were well underway. The 1996 and 1999 Caltech Petaflops workshops at Bodega Bay and Santa Barbara were seminal in shaping the future that because petascale computing.  As this photograph shows, we have all aged a bit in the intervening twenty five years, and sadly, a few of our colleagues are no longer with us.

Today, terascale and petascale computing systems via GPU accelerators are widely available, spanning desktops to laboratory and university cluster environments. In May 2022, the latest incarnation of the TOP500 list of (some of) the world’s fastest computers was revealed at the ISC High Performance conference in Hamburg, Germany. There was much hype and chest thumping as the Oak Ridge National Laboratory (ORNL) Frontier system broke the exascale barrier based on the high-performance LINPACK (HPL) benchmark for dense matrix factorization.

I say “some of” the world’s fastest computers because submission to the TOP500 list is entirely voluntary, and some organizations chose not to submit entries, for a variety of reasons, some political and economic, some technical, and some related to national security. In particular, China has at least two exascale systems (OceanLight and Tianhe-3), but has not submitted either of them to the TOP500 list, likely to avoid further inflaming tensions with the United States.

Meanwhile, the very definition of exascale is itself now subject to debate, both because the dense linear algebra of the HPL benchmark is widely acknowledged as no longer a good predictor of multidisciplinary application performance and because mixed precision arithmetic is now common in many applications, particularly those with AI-accelerated components. When convolved with system and chip energy constraints, semiconductor fabrication challenges, and ecosystem shifts, a sense of quo vadis now pervades the field. (In a recent essay and associated article, my colleagues Jack Dongarra, Dennis Gannon, and I offered a few thoughts on strategic directions.)

In the world of computing, the locus of innovation has focused on the small and many (think smartphones and the Internet of Things) and the few and large (think commercial cloud infrastructure and leading edge HPC systems); scientific instruments are dominated by a similar dichotomy, though the large and few receives most of the attention and visibility.  As for large-scale scientific instruments, one need look no further than the aging Hubble Space Telescope, the recently upgraded Large Hadron Collider (LHC), the International Thermonuclear Experimental Reactor (ITER) being built by a global consortium, the soon to be operational Vera Rubin Observatory (nee Large Synoptic Survey Telescope) and James Webb Space Telescope (JWST), or the three proposed and under construction Extremely Large Telescopes. Biology has joined the physical sciences in large-scale instrumentation, spanning the Human Genome Project to the National Ecological Observatory Network (NEON).

Like leading edge supercomputers, these big instruments are the shiny objects featured in press releases and popular science stories, and well they should be, for they produce scientific results achievable via no other mechanism. Each involves hundreds, thousands, and sometimes tens of thousands of scientists and engineers; each requires billions of dollars, euros, or yuan to design, build, and operate; and produces torrents of data that demand petascale data archives and global content distribution networks. Thanks to these new instruments, the associated explosion of scientific data has been hugely democratizing, allowing many researchers to focus on the key intellectual questions rather than the mechanics of data acquisition. 

However, this brave new world and the shift from data paucity to plethora have also raised new questions about data preservation and triage, shared and individual responsibilities, FAIR access, and the costs of data retention, as mandated by government sponsors.  (See Research Data Sustainability and Access and My Big Scientific Data Are Lonely.) As the late Nobel Laureate Herbert Simon once noted,

What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.         

In my humble opinion, we have not yet fully internalized the social, political, economic, and technical implications of the shift from data scarcity to data ubiquity.  What data we keep and for how long, how we manage access, and how we distribute the very real costs of data management and preservation have not yet been fully resolved.  Serious science policy issues remain unresolved.

On Being the Right Size

Although our intellectual curiosity is rightfully unbounded, our financial resources are not. What are the right sizes and relative numbers for our computing platforms and our scientific instruments?  Is it all about faster and bigger, which in almost any realistic budget scenario, also means fewer?  In the limiting case, as illustrated by ITER and the LHC, fewer may well mean only one per planet, financially sustained only via a global scientific partnership.

Nor does the fewer and bigger choice only affect the high end, it squeezes investment in the long tail of campus and laboratory infrastructure as well. These are important science policy questions, especially when one realizes that the day-to-day, “bread and butter” work of science is conducted with many instruments of much smaller size, educating students in scientific processes.  (See On Being the Right Size: Science Scaling and Power Laws.)

Make no mistake; form and scale absolutely follow function, as the British geneticist J. B. S. Haldane first wrote in a now famous essay entitled On Being the Right Size. (You can read the original essay as originally published in Harper’s Magazine, skipping the political pontification at the end.)  In it, Haldane offered cogent arguments about surface area to volume ratios, structures, respiration and energy, noting:

The most obvious differences between different animals are differences of size … it is easy to show that a hare could not be as large as a hippopotamus, or a whale as small as a herring. For every type of animal, there is a most convenient size, and a large change in size inevitably carries with it a change of form.

Computing systems and scientific instruments are no different. A simple refractor telescope design cannot be expanded unchanged to create a 30-meter optical observatory; the materials properties do not scale linearly, and the control systems differ markedly. A child’s horseshoe magnet cannot be scaled to create a 100-tesla pulsed research magnet. Nor is an exascale computer just a big deskside cluster, despite some comparable elements. The lack of self-similarity at scale means new design approaches are needed in different operating regimes, each with associated design costs.

Building Out

Returning to my twenty-year-old diagram, we have made progress in building out, with midrange computing clusters now more common and new classes of midrange instruments (e.g., cryo-EM) changing the face of science. Cyberinfrastructure training materials are now widespread, and we even host student cluster construction competitions; building a Linux cluster is no longer an arcane art. Indeed, one can now build an inexpensive Raspberry Pi cluster with almost all the same software tools found on leading edge supercomputers.

Despite this progress, our intellectual fascination with a small number of large instruments and computing platforms often blinds us to the power of distributed discovery and innovation with large numbers of instruments at smaller scales. Despite this, we have made enormous progress in sensor miniaturization, perhaps even more than Feynman envisioned in his seminal, “There’s Plenty of Room at the Bottom” nanotechnology lecture. (See Come to the Supercomputing Big Data Edge.) In this sense, we have a long way to go to realize the vision of a ubiquitous infosphere. Let me illustrate with a specific example.

SAGE: A Software Defined Sensor Network

Over the past two years, I have greatly enjoyed playing a small part in the NSF-funded SAGE project, which is seeking to build a flexible and extensible hardware and software cyberinfrastructure for AI at the edge.  Led by Pete Beckman, SAGE builds on many ideas from Charlie Catlett’s urban science and environmental science Array of Things (AoT) project, which deployed sensors on utility poles across the City of Chicago and surrounding areas.

AoT engaged community residents in defining acceptable data use policies, explored data driven social science by direct measurement, and stimulated citizen science and K-12 hands-on education. For a review of the AoT project and the motivating lessons for SAGE, I recommend this IEEE Xplore article, which several of us wrote recently.

SAGE is based on the notion of software-defined, configurable sensors that include edge computing, standard sensor interfaces, a set of baseline sensors, and extension interfaces for environment-specific sensors, along with backend data storage and analysis facilities.

A standard “wild SAGE” node (named for its hardened packaging and fault tolerant design to run unattended in harsh conditions, or “in the wild”)  includes the following:

• One or two NVIDIA Xavier NX GPUs with Wi-Fi, Ethernet, and a full Linux stack
• Power over Ethernet (POE) sky and ground facing cameras
• An optical rain sensor and a microphone
• Relative humidity, barometric pressure, and temperature sensors in a Stevenson shield (that’s the box on the left in the photograph, which also contains a Raspberry Pi to manage the Bosch BME 680 sensor)
• POE and USB connectors for additional, domain-specific sensors

Why deploy AI at the sensor edge?  First, the volume of streaming data may preclude storage, either due to bandwidth limitations or storage capacity.  For these same reasons, the LHC’s detectors rely on real-time triggers detect and record only a small subset of particle collisions. Second, urban and environmental monitoring bring strong security and privacy concerns, particularly involving video. By extracting events and features entirely at the edge, the raw video need never leave the sensor, obviating many privacy concerns (e.g., the timing and frequency of pedestrian interactions may suffice, without saving any identifying video). Third, AI at the edge means the sensor can adapt dynamically to changing conditions (e.g., increasing measurement frequency based on a detected event or even triggering alternative measurements at other locations). Fourth, latency constraints may demand a response time possible only when decisions are made at or near the edge.

In SAGE, the key to this flexibility is an end-to-end architecture – Waggle AI@Edge – that allows domain scientists to focus on their science, rather than building generic infrastructure.  One need only connect new sensors, develop domain-specific AI plugins using widely available AI tools (e.g., OpenCV, PyTorch, and TensorFlowLite) to process sensor data in situ on SAGE nodes, and use tools such as Jupyter notebooks to analyze and correlate data.

To demonstrate this integration capability, the SAGE project has been working with NEON to integrate inexpensive SAGE nodes for augmented environmental monitoring and with California and Oregon wildfire researchers to detect and respond to the growing prevalence of wildfires. There are now SAGE plugins for vehicular traffic detection, bird song classification, cloud cover, smoke, and other flora and fauna events. 

At Utah, we have deployed one SAGE node at the Natural History Museum of Utah (NHMU), which allows us to track environmental conditions and local flora and fauna.  Additional SAGE nodes are being deployed in downtown Salt Lake City to monitor PM 2.5 particulate air pollution and to study advanced wireless technologies in collaboration with the NSF POWDER project.

Much More Room at the Bottom

At ~$15,000 each, SAGE sensors are inexpensive compared to multibillion dollar mega-instruments.  Yet even this price is prohibitive for many research contexts, limiting their deployment in science and community education projects where resources are even more bounded and precious.  Fortunately, there is much more room at the bottom.  In this spirit, members of the SAGE education team are working on lower cost SAGE versions based on the Jetson Nano and the Raspberry Pi.

However, imagine a cyberinfrastructure world containing even lower cost sensors -- tens of thousands of them at ~$50 each! Leveraging a wealth of inexpensive and open source hardware and software, it is now possible to create and deploy a powerful sensor infrastructure for just a few hundred dollars.

Let’s start with server infrastructure. A Raspberry Pi with a complete Linux software stack, including Docker and Kubernetes virtualization, a high-capacity SD card for system software and data storage, Wi-Fi, Ethernet, and LoRaWAN long-distance communication transceiver can be assembled for approximately $100.

Such a system can readily serve as a base station for hundreds of microcontroller nodes, each supporting a wide variety of ~$5 sensors. As an example, one of my former colleagues at Iowa fabricated the sensor node shown at right based on my specifications. This node includes

• Powerful ESP32 Tensilica microcontroller with Wi-Fi and Bluetooth LE
• Bosch BME 280 pressure, temperature, and humidity sensor
• LoRaWAN low bandwidth, long distance communication
• I²C, analog, and digital Grove, solderless sensor connectors
• Battery and/or solar power interface for long-term episodic operation

Such sensor nodes can be readily designed using open source hardware tools such as KiCad and fabricated in quantity for less than $100 each, or similar ones can be purchased commercially (e.g., Heltec Wi-Fi LoRa32 V2 or Adifruit Huzzah32 Feather with Feather Wing addon sensors).  Equally importantly, they and the Raspberry Pi base stations can be integrated directly with environments such as SAGE. Just as commodity clusters opened computing to a wide range of practicing scientists, inexpensive sensors can be the missing and catalyzing element to realize the vision of an immersive sensor/actuator infrastructure.

Over the past two years, as a research proof-of-concept, I have developed a generic software infrastructure for Raspberry Pis and ESP32 sensors, while Charlie Catlett has been exploring complementary hardware and software models based on inexpensive home automation tools.  We are now drafting a “how to” book, targeting K-12 STEM education and citizen science.

Cyberinfrastructure Futures: The Missing Elements

We have come far in building out a 21^st century cyberinfrastructure, but there is more to do if we are to fully exploit the opportunities now afforded by scale – both small numbers of large computing systems and instruments and large numbers of small computing systems and instruments. As we imagine a future world of software defined sensors, I believe we must address the following research challenges, and I encourage all of us think about how we might mount national research programs to explore each of them.

An Extensible Cyberinfrastructure Baseline. At present, we lack a generic sensor toolkit that is broadly applicable to diverse measurement environments.  The SAGE project is an early attempt at designing and deploying such a generic infrastructure, drawing on lessons from the earlier Array of Things (AoT) project, but much more experimentation and targeted development is required to separate systemic needs from domain-specific science.  Even more importantly, the lifetime of these longitudinal sensor data surveys may last for years or even decades, placing an even greater premium on organizational and infrastructure stability.

Intentional Programming at Scale.   Task or node level specification is central to most of our programming models, yet so many of the questions that matter with distributed sensors are systemic ones involving identification of either data outliers (e.g., vibrations on this bridge exceed norms) or broad trends (e.g., rainfall along the severe storm front is approaching critical levels).  Equally importantly, such data should trigger certain global actions (e.g., if viral loads in sewage rise, increase global sampling rates). These global, “if this happens, then do this” intentional specifications, with some tolerance for sampling errors and uncertainty, are not typical of our programming models, but are ideal for specifying system behaviors and responses. We need new research on systemic, intentional programming models and specification systems (e.g., based on AI qualitative reasoning and zero shot planners).

Run Forever Heterogeneity. Those of us in computing are intimately acquainted with forklift hardware replacement, when old hardware gives way to new systems.  Depending on facility capacity, there may be some period of system overlap, but inevitably the forklifts remove the old hardware, leaving only a fully integrated, new system in their wake. Alas, the real world of distributed sensors bears much more resemblance to that of embedded real-time systems, where replacing sensors can be both costly and impractical.  Sometimes this is because sensors are deployed in critical infrastructure with hard real-time constraints and uptime quality of service expectations. In other cases, the cost of replacement may simply be prohibitive. 

For example, replacing the Chicago AoT nodes required access to City of Chicago utility bucket trucks, something possible only rarely and as a low priority relative to operational utility needs.  Similarly, replacing sensors in remote field sites may require extensive and difficult travel.  One consequence of such difficulties is extensive sensor heterogeneity, with new sensors and technology operating in concert with older, less capable sensors.  This has profound implications for both programming models and software configuration and deployment, as well as the types and resolution of data being captured and kinds of in situ analysis possible. 

Bags of Interdisciplinary Data. In addition to disciplinary access, much of the value of longitudinal sensor data accrues from cross-disciplinary data fusion, asking questions never envisioned by any of the original collectors of the data. The latter necessarily requires integration across diverse data schemas, none of which were designed with cross-disciplinary questions in mind.  We need to embrace data schema heterogeneity as an inevitability rather than as an impediment, encouraging development of new techniques that support “come as you are” data analysis.  None of the search firm giants have a master schema for web pages; rather, they index data and build search engines capable of responding to a wide range of queries.  We need to draw insights and lessons from there experiences.

A Data Economy. Finally, government mandates for data preservation and access are creating new challenges for researchers and research institutions alike.  What should be saved and for how long?  Can such answers be determined solely by the constituent disciplines or should there be a combination of discipline-specific and university specifications?  When can and should data be discarded?  Are those choices based solely on usage frequency, uniqueness, or cost of replication relative to cost of retention?  Research librarians long ago adopted deaccession policies, triaging holdings and negotiating interlibrary loan policies for rare or unique artifacts?  We need a coherent data economy that balances data utility against costs, driven by scientific needs.

Coda

We have come far in developing a coherent national and international cyberinfrastructure for research and innovation.  However, there are still unmet needs and unresolved infrastructure and research questions surrounding distributed instruments and large numbers of inexpensive sensors.  It is time to reason together and build the missing pieces.

A {Racing} Bicycle for the Mind: Helmet Recommended

The March 1973 issue of Scientific American contained an article entitled simply “Bicycle Technology,” written by S. S. Wilson.  As the title suggests, the article recounted the technological evolution of the bicycle and how those insights contributed to the development of early automobiles and airplanes.  After all, many of the mechanical techniques used by the Wright brothers in the Wright Flyer, the first heavier-than-air aircraft, were developed in their Dayton, Ohio bicycle shop and tested at windswept Kitty Hawk, North Carolina.

This Scientific American article would now be of only passing historical interest were it not for a single diagram (shown below), which plots the cost of transport (in calories per gram per kilometer) against body weight (in kilograms) for several organisms and devices, all on a log-log scale. As one would expect, flying organisms tend to be small in weight but expend energy at high levels to remain airborne. Land-based mammals, though much larger, consume less energy relative to their weight. Among the latter, bipedal humans are unremarkable, more efficient than dogs but less efficient than horses.  Similarly, helicopters are less efficient than airplanes or automobiles.  All of this matches our intuition and experience.

Ah, but here’s the story.  A human on a bicycle ranks first in energy efficiency among mobile animals and machines in energy consumed to move a certain distance as a function of body weight, and not by a little, but a lot! The chart shows that the rate of a bicyclist’s energy consumption is about one fifth that of a walking human. Anyone who has bicycled knows this is true.  A reasonably fit bicyclist can cover more than one hundred miles in a day, the so-called century ride.

By Geeks for Geeks

The fact that a human on a bicycle is quantitatively and qualitatively more efficient than anything nature has engineered via evolution caught the eye of an eager young reader, Steve Jobs.  More on that in just a minute.  First, let’s talk about the magazine, Scientific American.

Founded in 1845, like many magazines, Scientific American had its ups and downs until three partners reinvented it during the 1960s, a few years before I first encountered it.  For at least the next thirty years, it embodied everything the well-read – wait for it – scientific American might hope to know about cutting edge science and technology.   Unlike other popular science magazines, the articles were written by the scientists responsible for the discoveries they described, which gave them gravitas often lacking elsewhere. The Amateur Scientist column by Jearl Walker described experiments that could be conducted by savvy amateurs, and the Mathematical Games column by Martin Gardner enchanted many a budding mathematician and computer scientist.  

Scientific American was the kind of magazine geeks awaited eagerly each month, then stood by the mailbox reading cover-to-cover with undisguised joy, only to rush inside afterward to regale unsuspecting family members with their latest insights. I knew this firsthand, having had my own transformative experiences with the magazine, beginning with a stash of old copies I was given as a young teenager. There, I read about stellar nucleosynthesis of the elements, progress toward the Standard Model integrating three of the four fundamental forces, and new insights into DNA and gene actions, among many others. I have since been a loyal subscriber for over forty years, though I hope the magazine has been as successful in recruiting younger readers as it was with me.

I read many fascinating things in Scientific American over the years, but two things stand out most vividly. The first was Christopher Zeeman’s delightful exegesis on catastrophe theory, a mathematical model of non-linearity in complex systems, something that I wrote about recently in the context of COVID-19. (See On Catastrophes and Rebooting the Planet.) The second was Martin Gardner’s description of John Conway’s Game of Life, a beguilingly simple but powerful cellular automaton.  Sadly, John Conway, who spent many years at the Institute for Advanced Study, passed away during the COVID-19 pandemic.

The Game of Life became a staple in computer science circles, engaging both experimenters and theorists who derived self-replicating entities and ultimately proved the Game of Life is Turing complete. That rabbit hole is very deep, for via its three simple rules, the Game of Life is capable of computing all that is mechanically computable, bounded only by Gödel incompleteness and the Entscheidungsproblem. From there, it is just a hop-skip-and jump to the Chomsky hierarchy and Turing’s musings on what it means to think.

As an undergraduate, I burned through prodigious amounts of IBM S/370 computer time myself, running Game of Life simulations, enough so that I got called on the carpet by the computing center leadership.  Being fellow geeks themselves, after I showed them the code and explained what I was doing, I was given a friendly blessing and sent on my way. (See A Feeling for the Code.) Since then, I have used the Game of Life as an example in more than one parallel computing class, illustrating parallelism, irregular workloads, and synchronization constraints.

Amplifying Human Talent

Given the impact of Scientific American on the science and technology crowd, it is no surprise that a young Steve Jobs saw the article on bicycle technology and how it augmented human capabilities. It was an idea that became a passion, finding the intellectual analog of the bicycle, something that could augment the human mind in the same dramatic way a bicycle amplified locomotion. You can watch Jobs talk about it here and here, at two different periods in his life. His memory about the condor as the most efficient was incorrect; it was not on the list. However, he captured the essence of the story, when he talked about humans as tool builders and a computer being a bicycle for the mind, able to take us far beyond our human abilities. As Jobs put it,

… somebody at Scientific American had the insight to test the efficiency of locomotion for a man on a bicycle. And, a man on a bicycle, a human on a bicycle, blew the condor away, completely off the top of the charts.

And that’s what a computer is to me. What a computer is to me is it’s the most remarkable tool that we’ve ever come up with, and it’s the equivalent of a bicycle for our minds.

Jobs tried to name the then internally code-named Macintosh project the “Bicycle,” but it didn’t stick. Apple even used the bicycle for the mind analogy in its marketing, as this 1980 Wall Street Journal advertisement shows. (Alas, no copyeditors verified the condor allusion.)

A Universal Intellectual Amplifier

Over the years, I have crafted my own version of this analogy, emphasizing that computing is distinct from all other human tools in its universality. (See Intellectual Amplification via Computing.) Physical tools, from the simple inclined plane through a pulley to a screwdriver, rely on mechanical advantage, each specialized for a particular task. Put another way, a screwdriver will not help you lift a weight, nor will a pulley help you fasten two objects together. Each is useful, but each is specialized.

That specialization extends to scientific and intellectual endeavors as well.  Telescopes, such as the Vera Rubin Observatory or James Webb Space Telescope, expand our vision, reaching far beyond the electromagnetic range of visible light and the limits of the human retina. They let us see the faint, large, and distant, but do nothing to illuminate the spatiotemporal dynamics of a biochemical reaction. Nor can a cryo-EM’s amazing efficacy in understanding macromolecular structures measure the cosmic microwave background (CMB) or plumb the mysteries of dark matter and dark energy.

In contrast, computing is universally applicable to diverse human intellectual efforts. That bicycle for the mind can help write sonnets (or via AI, write them itself), explore the spread of viral disease, understand galactic evolution, or a thousand other things, limited only by human imagination.

In that spirit, my friend and former colleague, Fred Brooks, one of the fathers of IBM’s System/360 family of computers, once described himself as being in the “intelligence amplification” business. After leaving IBM and moving to the University of North Carolina at Chapel Hill, he met with a University of North Carolina at Chapel administrator, whom he then wryly asked who on campus most needed their intelligence amplified.  Beyond the twinkle in Fred’s eyes as he told the story, I’ve always loved the anecdote, because it speaks to the human nature of computing as a collaborative enabler for creativity.

The Supercomputing Racing Bike

If the personal computer – Mac or PC – is a bicycle for the mind, then a supercomputer is a carbon fiber composite racing bicycle with electronic shifters, hydraulic disc brakes, and tires engineered to minimize rolling resistance.  Though the principles are the same, a supercomputing bicycle is far different than the bike you rode as a child.

Back in May 2004, I testified to the U.S. Congress about the needs of racing bicycles for the mind, something I have done several times. At the time, I was serving as the Founding Director of the Renaissance Computing Institute (RENCI), itself based on the idea that computing was universally applicable to enriching the human experience.  It was an idea I learned from my NCSA colleague and friend, Donna Cox, an artist who has enriched the lives of so many via her Renaissance teams and scientific visualizations.

The other witnesses that day were Jack Marburger, then Director of the White House Office of Science and Technology Policy (OSTP); Irving Wladawsky-Berger, a Senior Vice President from IBM; and Rick Stevens from Argonne National Laboratory.  You can read the entire transcript of the hearing here. At that hearing of the House Science Committee, I offered these words about the universality of high-performance computing and its power to expand the frontiers of human knowledge, noting

The breadth of these examples highlights a unique aspect of high-performance computing that distinguishes it from other scientific instruments – its universality as an intellectual amplifier.  Powerful new telescopes advance astronomy, but not materials science.  Powerful new particle accelerators advance high energy physics, but not genetics.  In contrast, high-performance computing advances all of science and engineering, because all disciplines benefit from high-resolution model predictions, theoretical validations and experimental data analysis. As new scientific discoveries increasingly lie at the interstices of traditional disciplines, high-performance computing is the research integration enabler.

I went on to say something that I believed then but seems even more true now

Although this universality is the intellectual cornerstone of high-performance computing, it is also its political weakness.  Because all research domains benefit from high-performance computing, but none is solely defined by it, high-performance computing lacks the cohesive, well-organized scientific community of advocates found in other disciplines. In turn, this has led to over-dependence on market forces to shape the design and development of high-performance computing systems, to our current detriment.

In almost twenty years since, we have seen phenomenal computing advances.  We have broken the petascale and the exascale performance barriers.  Advanced AI delivers modern day miracles, capable of playing chess and Go at levels that humble even the world’s best human players, able to translate human languages with high and rapidly improving accuracy, adept at predicting how proteins fold, and able to compose prose with near human facility.

We have come far on the racing bicycle for the mind.  We need to keep pedaling, for we still have far to go to reach our destination, given both the manifest opportunities and the equally daunting challenges. 

I have written about the challenges of building ever faster computers, given economic and technical constraints, and I have called for new approaches to designing and building advanced computers that lessen our dependence on traditional market forces, recognizing that advanced computers are increasingly bespoke research instruments rather than commercial commodities. (See American Competitiveness: IT and HPC Futures – Follow the Money and the Talent and a related article.)

We also have deep technical and social issues surrounding AI ethics, digital equality, and digital communications. We are very far from the general AI (strong AI) so omnipresent in the movies, yet as our machines take on more tasks once viewed only as the province of humankind, we need to think about roles and responsibilities, and how we best imbue our machines with the best of humanity’s values. (See Reluctant Revolutionaries, the Trolley Paradox, and Ender’s Game.)

In addition to the handlebar streamers and the thrill of the wind in our hair, a helmet might be advised.

Coda

Make no mistake, Steve Jobs was absolutely right; the computer is arguably one of the most remarkable tools we humans have ever invented.

I have been proud and humbled to be a racing bicycle for the mind designer for these past forty years. Pedal on.

Memos, Janitors, Teams, and Innovation

Memoranda (memos) are the detritus of organizational processes, frequently written only so the writer can point to the message as prima facia evidence of purported communication, concern, action, or principles: mistakes were made, committees have been appointed, or actions will be taken. Some are internal, revealing the jaded reality behind polished exteriors.  Some are leaked, some are belatedly discovered in the archives of history, and still others are published and promulgated widely.

Not surprisingly, whether in academia, industry, or government, cynicism often runs high among any memo’s target audiences, for such memos can be turgid and meandering, written and edited by committees. Consequently, most are little read, and even fewer are remembered, much in the spirit of the old Communist adage, “They pretend to pay us, and we pretend to work.”  The exceptions prove the rule, exposing existential concerns and broader truths, part of an authentic dialog.

Memo Filtering

Long ago, I was privileged to serve as head of the computer science department at the University of Illinois, one of the country’s great academic teams, famous for its work in high-performance computing.  In those pre-Internet days, a veritable blizzard of paper memoranda came to me from the offices of the college dean and the university administration.  After scanning them, I discarded the vast majority, forwarding only a few of the most relevant to the paper mailboxes of my colleagues, who still good-naturedly complained about unnecessary deforestation.  Reminding them that I filtered the chaff and that they were only seeing the “good stuff,” I bided my time.

Eventually yet another memo from the campus reached my desk regarding campus pouring rights and the new campus contract for either Coca-Cola or Pepsi products. Though practically important to the campus for its financial contributions to scholarship, it was largely irrelevant to most.  Seizing my opportunity for friendly parody, I wrote an extended, tongue-in-cheek cover memo encouraging my colleagues to engage in shared governance and participatory democracy by attending the upcoming campus town hall and expressing their views on one of the great and important issues of our day – the relative merits of Coca-Cola and Pepsi.  I’d made my point, which elicited some laughter and friendly discussion at our next faculty meeting.

Including the Janitor

One of the most famous memos in all of computing, perhaps rivaled only by Bill Gates’ 1995 Internet Tidal Wave memo, was written by IBM President Thomas Watson Jr. on August 28, 1963.  The entire memo, directed to the company’s senior leadership prior to an upcoming retreat at Jenny Lake, New York, is reproduced below:

August 28, 1963

Last week CDC had a press conference during which they officially announced their 6600 system. I understand that in the laboratory developing this system there are only 34 people, “including the janitor.” Of these, 14 are engineers and 4 are programmers, and only one has a Ph. D., a relatively junior programmer. To the outsider, the laboratory appeared to be cost conscious, hard working and highly motivated.

Contrasting this modest effort with our own vast development activities, I fail to understand why we have lost our industry leadership position by letting someone else offer the world’s most powerful computer. At Jenny Lake, I think top priority should be given to a discussion as to what we are doing wrong and how we should go about changing it immediately.

T. J. Watson, Jr.

Condensed to its essence, Watson asks how a pipsqueak little company could develop a world-leading technological marvel that eclipsed the efforts of a Fortune 500 corporation. After all, IBM’s slogan was and is simply THINK.  I pondered similar things myself while at Microsoft, as we in the senior leadership looked at the Apple iPad and iPhone with similar dismay during our executive retreats. 

At the time of Watson’s memo, IBM was the world’s largest computing company, deep in the development of the IBM System/360, a multibillion dollar, “bet the company” plan to develop an entire product line of instruction set compatible computers.  The successful result reshaped the computing industry and ensured IBM’s dominance for the next twenty years.  In the years prior, IBM had developed the IBM 7030 Stretch, the fastest computer in the world until the CDC 6600 appeared, a leadership role IBM would not reclaim until almost forty years later.

On being told of Watson’s memo, CDC’s Seymour Cray, the architect of the CDC 6600 and widely known in computing for eschewing bureaucracy, reportedly quipped, “It seems like Mr. Watson has answered his own question.”  Cray, universally lauded as the father of supercomputing, later designed the eponymously named Cray-1, once again the world’s fastest computer. However, this is not a story about supercomputers, but about culture, shared responsibility, and respect.

As Watson noted, IBM had vast development activities and a large cadre of visionary and talented people, luminaries like Fran Allen, John Backus, Fred Brooks, John Cocke, and Gene Amdahl, several of whom, along with Seymour Cray, I have known over the years, and others I was blessed to count as colleagues. Where IBM struggled, by its own admission, was not talent; it was bureaucracy and culture.  Some of its best minds left the company to pursue their own ideas, and others were relieved of responsibility when they challenged the orthodoxy. 

As an example, John Cocke’s own, legendary work on the IBM 801, which long predated the reduced instruction set computer (RISC) revolution, took years to see the light of day. In a story that has been told and retold, and embellished with retelling until it has become the stuff of legend, John returned from a meeting with management, frustrated and disgruntled by their lack of vision.  He gathered his team and reputedly said, “<EXPLETIVE> them!  We are going to build the best <EXPLETIVE> computer the world has ever seen.”

With IBM’s tolerance and unencumbered by market niches and product lines, they did just that, though the reality was more nuanced and complicated than this story might suggest. It was a powerful combination -- John’s vision and the quality of his team, each of whom knew this was the intellectual opportunity of a lifetime. Because the project lacked the blessing and imprimatur of any product line,  they named it the 801, after the offices where they worked. (The New York Times has a great story about John Cocke and his maverick ways.) 

Team Cultures

Mr. Watson’s deprecating allusion to the janitor is telling.  As Tolstoy remarked in Anna Karenina, “Happy families are all alike; every unhappy family is unhappy in its own way.”  Dysfunctional teams can and do fail in a variety of ways. Conversely, great teams, regardless of the mission and objective, share several common attributes. 

First, the team members believe passionately in a greater goal and mission, as something worthy of shared commitment and mutual sacrifice. (“We are going to build the fastest computer the world has ever seen.”) Individually and collectively, they are united in their desire to do something that will be remembered, something others may believe improbable or impossible; they want to make a difference. Whether that vision originates from the team’s leader or from one of its members matters not.

Second, a successful team is greater than its parts.  Each team member recognizes and understands they each have a critical role, and if they fail individually, the team will fail collectively.  Hence, each depends on the others to not only meet their commitments but to exceed them. They will disappoint or fail their partners.  Everything and everyone matter, and everyone knows it, including the janitor.

It is also why being “lean and mean” is often so critical to success.  With no distractions, no tangential activities, leadership shielding them from bureaucratic intrusions, and just enough resources to complete the task – but no more – everyone is highly motivated and keenly focused on the goal and the competition. The fierce urgency of now motivates one and all. Venture capitalists know this well; overfunding a startup is as deadly as critically underfunding.

Third, leaders know the buck stops with them.  They bear the ultimate responsibility for the team’s overall performance, and they will accept blame if necessary to protect the team.  Steve Jobs often talked about the difference between the janitor’s responsibility and that of a leader. As he put it, if the janitor cannot empty the trash because the locks have been changed and the janitor lacks a key, that is understandable. Conversely, a leader must anticipate such problems and clear the path for the team. Locked doors cannot be a barrier.

The Spider-Man rule applies to leaders: With great power comes great responsibility. Sometimes, that responsibility means providing the vision and strategy that animates the team, other times it means holding the bureaucracy at bay.  Or, it can mean getting new keys made over the weekend, so the janitor can empty the trash without interruption. In other cases, it can mean buying and personally delivering pizza to fuel the inevitable late nights, or sweeping the floors if the janitor is ill.  Good leaders do what needs to be done, instinctively and unhesitatingly, without drama.

Finally, and most importantly, in a great team, everyone is respected for their differential skills and responsibilities.  The janitor matters, just as does the Ph.D., the engineer, and the software developer, albeit each with different roles and expectations.  As M. L. King, Jr. once said, “Whatever your life’s work is, do it well. A man should do his job so well that the living, the dead, and the unborn could do it no better.”  There is virtue and honor in being a great janitor, just as there is in being a computer architect.

Never forget, the way leaders treat each individual sets the organization’s tone. In the cultures of great teams, the members feel empowered to generate ideas, offer suggestions, question authority, and take the initiative. If the leader respects and values the janitor, treating him or her as an equal and valued partner, the other team members will know, intuitively and immediately, that they too are valued, and they will respond accordingly. This is the working definition of servant leadership.  

Feed the Mind, Thank the Janitor

Inertia is endemic to large organizations, where bureaucratic processes can hamper or even strangle innovation.  In response, wise leaders often create alternative cultures, research institutes, and skunkworks that let radical innovation flourish unimpeded by memoranda, processes, and bureaucracy.  It is precisely this dichotomy – those who work within established paradigms and those who are compelled to create new ones – that Thomas Kuhn explored in his book, The Structure of Scientific Revolutions. Both are needed, and effective cultures support and nourish both.

Unfettered exploration by passionate individuals and committed teams is the fundamental premise of academic freedom and exploratory research, whether in universities or in industry.  As Vannevar Bush perspicuously noted in his famous memo to President Roosevelt at the end of World War II, “Scientific progress on a broad front results from the free play of free intellects, working on subjects of their own choice, in the manner dictated by their curiosity for exploration of the unknown.” Such individuals and teams rarely read memos; trust me on this.

And the next time you see a janitor, or anyone else engaged in a seemingly thankless but important task, stop, smile, and say thank you; they too make a difference. The word will spread, far faster and far more effectively than via any memo.

American Competitiveness: IT and HPC Futures – Follow the Money and the Talent

N.B.  A more formal version of these arguments is posted on arXiv (https://arxiv.org/abs/2203.02544), co-authored with Dennis Gannon and Jack Dongarra.

This is a long post, containing perspectives that may be controversial to some.  That's okay; it is why we have lively intellectual debates.

Charles Dickens’ A Tale of Two Cities opens with this line:

IT was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of light, it was the season of darkness, it was the spring of hope, it was the winter of despair.

It’s an apt metaphor for the state of information technology and the state of U.S. competitiveness in computing and science, technology, engineering, and mathematics (STEM).  In some ways, the hegemony of U.S. IT has never been greater, with Apple and the cloud companies (Google, Microsoft, Amazon, and Facebook) dominating the Nasdaq with trillion dollar (and more) market capitalizations and reshaping the very definitions of computing with global services, deep learning, and advanced AI.

Conversely, the semiconductor shortage has highlighted the economic and national security risks from over-dependence on off-shore fabrication, and the relative influence of traditional computing companies on the IT ecosystem has waned. Meanwhile, the future of U.S. leadership in bleeding edge scientific computing (aka supercomputing) is at a critical crossroads.

 Finally, while the global competition for STEM talent in general and IT talent in particular has never been more intense, the U.S. is substantially underperforming relative to other countries in STEM education.  Let’s take a look at how all of this has happened.  Remember, follow the money and the talent.

Remembering the Past

Let’s start by playing a parlor game. Name the first computer company that comes to mind.  If you are an old timer, you might be inclined to mention IBM, Amdahl, and the BUNCH (Burroughs, UNIVAC, NCR, CDC, and Honeywell).  Although IBM once bestrode the computing world like a colossus, today, even its relevance has faded. Indeed, few now remember that IBM was once locked in a bitter struggle with the U.S. Department of Justice for monopolistic practices under anti-trust laws.  As for the BUNCH, they are either gone from the computer business or niche players at best. Later, others – Compaq, Digital Equipment Corporation (DEC), and SGI – came and went.

As an aside, I learned to program on an IBM System 360/50, wrote lots of code on a DEC PDP-11/45 for my first paid software development job, ported code from a GE 600 timesharing system, and used a CDC 6500 and 6600 during graduate school. I programmed in FORTRAN, COBOL, PASCAL, LISP, and PI/I (in addition to many more recent languages), and I’ve toggled absolute binary into the front panel. I’m also old enough to remember when one could do Mead and Conway VLSI design with colored pencils.  I also used a slide rule, while walking barefoot in the snow to school, uphill (both ways), but I digress. (See You Might Be a Computing Old Timer If …)

There’s a sad, parallel tale (pun intended) in the land of high-performance computing, where the cemetery is filled with departed denizens, some large, some small, but all now permanent members of the Dead Computer Society. Whether by misjudging markets or technologies, each in in its own way proved the truth of the old adage that the best way to make a small fortune in supercomputing is by starting with a large fortune, then shipping one or two generations of product. Sic transit gloria mundi SGI, Cray, and a host of others. (See HPC: Making a Small Fortune.) 

More to the point, the list of high-end supercomputing players keeps getting shorter.  Cray is gone, now part of HPE; and both IBM and Intel are struggling to right themselves, each for different reasons.  In the United States, this leaves HPE as the lone high-end HPC integrator in the United States.  Time will tell if Nvidia, with its recent acquisitions, steps into this space, though its proposed acquisition of ARM seems unlikely.

Meanwhile, the information technology ecosystem and the locus of innovation and global leadership have shifted dramatically. There are deep and important implications for the U.S.

Contemplating the Present

If you are Gen Z or Gen Alpha, the world begins and ends with Apple, Google, and Samsung smartphones and the cloud services behind them.  All of which brings us to the FAANG companies (Meta née Facebook, Amazon, Apple, Netflix, and Alphabet née Google) plus Microsoft and their Asian BAT (Baidu, Alibaba, and Tencent) counterparts.  These companies are economically exothermic in ways perhaps unprecedented since the days of the Standard Oil monopoly.  Doubt me?  Look at their market capitalizations  and free cash flow relative to traditional computer companies. (As I write this, Microsoft just made a $69B all-cash offer for Activision and reported over $50B in quarterly revenue.)

I first watched this power dynamic begin to shift while at Microsoft.  For years, the Wintel duopoly locked Microsoft and Intel in a collaborative/competitive partnership as rough peers.  I sat in more than one “frank and spirited” exchange between leaders of the two companies, where each pushed the other to accommodate the other’s market needs. As server chips, clouds, and infrastructure grew in importance relative to desktops and laptops, the power dynamic began to shift.  Today, Intel is the far weaker player, in many cases forced to respond to cloud vendor demands.  In hindsight, it is all obvious.

Technological and Economic Shifts

Today, Apple and the other cloud service companies dominate the computing hardware and software ecosystem, both in scale and in technical approaches. Initially, they purchased standard hardware for deployment in traditional colocation centers (colos). Then they began designing purpose-built data centers, optimized for power usage effectiveness (PUE), deployed at sites selected via multifactor optimization – inexpensive energy availability, tax breaks and political subsidies, political and geological stability, network access, and customer demand. 

As scale, complexity, and operational experience grew, new optimization and leverage opportunities emerged, including software defined networking, protocol offloads, and custom network architectures (greatly reducing dependence on traditional network hardware vendors), quantitative analysis of processor, memory, and disk failure modes, with consequent redesign for reliability and lower cost (dictating specifications to vendors via consortia like Open Compute), custom processor SKUs, custom accelerators (FPGAs and ASICs), and finally, complete processor design (e.g., Apple silicon, Google TPUs and AWS Gravitons).  In between, they deployed their own global fiber networks.

This virtuous cycle of insatiable consumer demand for rich services, business outsourcing to the cloud, expanding data center capacity, and infrastructure cost optimization has had several effects.  First, it has dramatically lessened – and in many cases totally eliminated -- FAANG/Microsoft/BAT dependence on traditional computing vendors and ameliorated the risks of wholesale transfer pricing. Put another way, Amazon AWS, when necessary, can now negotiate pricing terms with Intel and AMD in the same way Walmart does with Procter and Gamble (i.e., from a position of strength as a dominant customer).

Second, the absolute scale of infrastructure investment, denominated in billions of dollars per year for each cloud service vendor, means these companies are shaping computing technology research and development priorities in ways traditional computing vendors rarely can, driving design and fabrication priorities in every element of the ecosystem.

Bigger is not simply bigger; bigger is different, fueling investment in new technologies at a phenomenal rate. (See The Zeros Matter: Bigger Is Different.) Relatedly, I cannot overemphasize the massive scale of these cloud data center deployments, dwarfing anything seen in academia or national laboratories, with the already large gap widening each year.  The CAGR is simply astounding.

Third, custom infrastructure is about more than economic leverage; it’s about market differentiation and competitive advantage.  Anything not differentiating is standardized and commoditized, driving unnecessary cost out of the system, and concomitantly reducing profits for commodity purveyors.  Conversely, anything differentiating is the focus of intense internal optimization and aggressive deployment.

Equally importantly, client plus cloud services vendors predominately make their money from the services offered atop their infrastructure, much less from the infrastructure itself. Unlike Intel, AMD, and Nvidia, which sell silicon, Apple is not selling A15 SoCs; it is selling iPhones, just as Amazon and Google are selling services atop TPUs and Gravitons.

Fourth, all of this is occurring against a backdrop of big data and machine learning. Large-scale AI itself is fueling a computing revolution, with an insatiable hardware demand to train ever larger machine learning models.  As an example, modern speech recognition models now have hundreds of billions of parameters, and multiple groups have announced models with over one trillion parameters. Similarly, work on generative adversarial networks (GANs) has led to impressive advances in game playing and other tasks. Nor is deep learning limited to “AI workloads;” there are impressive results on solving differential equations as well.  (See Dennis Gannon’s excellent summary of Karniadakis’ work here.) Similarly, look at developments in generative adversarial networks (GANs),

Finally, all of these trends are convolved with the end of Dennard scaling, dark silicon, functional specialization, chiplets, semiconductor fabrication challenges, rising fabrication facility costs, and an uncertain political future for Taiwan and TSMC.  Moreover, the actual cost of transistors (dollars/transistor) is increasing as we move closer to one nanometer scaling.  In fact, the transistor cost minimum occurred at roughly the time (~28 nanometers) we switched from planar CMOS to FinFETs. It will likely only get worse as gate-all-around FETs predominate, given increased fabrication complexity.

In the midst of all this, Intel stumbled, losing its lead in process technology just as EUV finally and belatedly became mature, and TSMC staked a lead. Time will tell if Pat Gelsinger can turn things around.

The Future

Okay, let’s put all the technology and economics puzzle pieces together:

• Advanced computing requires non-recurring engineering (NRE) investment to develop new technologies and systems.
• The FAANG/Microsoft/BAT companies are cash rich (i.e., exothermic), and they are designing, building, and deploying their own hardware and software infrastructure at unprecedented scale.
• The traditional computing vendors are now small – in some cases, tiny – economic players in the computing ecosystem, and many are dependent on federal investment (i.e., endothermic) for NRE to advance the bleeding edge.
• AI is fueling a revolution in how we think about problems and their numerical solution.
• Dennard scaling is over and performance advances increasingly depend on functional specialization via custom ASICs and chiplet-integrated packages.
• Transistor costs are likely to continue to increase as we push to smaller feature sizes.
• Nimble hardware startups are exploring new ideas, driven by the AI frenzy.
• Talent is following the money and the opportunities, which are increasingly in industry.

With this backdrop, what is the future? I believe some of it is relatively obvious and secure, given the current dominance of U.S. cloud service providers. However, broader U.S. leadership in semiconductors, information technology, and bleeding edge HPC is less certain. I believe any viable solutions will require rethinking our approaches and assumptions, including how, where, and when we spend money and what will both grow and retain talent.  Let’s start with the broad issues around the U.S. information technology ecosystem.

Information Technology Futures

The big changes outlined above make it obvious that a computing revolution is underway, one that is reshaping global leadership in critical areas.  All revolutions create opportunity, but only for those prepared to accept the risks and act accordingly. In Far Side cartoon parlance, we must be tall enough to attack the city.

First, we need to be honest about the social, political, economic, and national security risks of losing U.S. leadership and control of the silicon fabrication ecosystem.  Fabless semiconductor firms are important, but we also need onshore, state of the art fabrication facilities, and we need to invest and act accordingly. It is why the U.S. CHIPS Act and its successors are so important, and why the recent announcements by Intel, TSMC, and GlobalFoundries that they are building new chip fabrication faculties in the U.S. (each for different reasons), is so important.

Second, the seismic business and technology shifts have also led to equally dramatic talent shifts. People gravitate to opportunities, and that giant sucking sound is people leaving academia, national laboratories, and traditional computing companies to pursue those opportunities.  Seymour Cray once said, “I wanted to make a contribution,” when asked why he had used so many high risk technologies in the Cray-1. The desire to make a difference is no less powerful today.

The academic brain drain among artificial intelligence researchers is well documented.  After all, in the commercial world one can develop and test ideas at a scale simply not possible in academia, using truly big data and uniquely scaled hardware and software infrastructure.  The same is true for chip designers and system software developers.

Making matters worse, the U.S. talent pipeline is uncertain.  As the recent 2022 Science and Engineering Indicators shows, the U.S. is losing ground compared to the rest of the world, trailing China in both patents and publications, as summarized in this Science story about the recent Science and Engineering Indicators report. 

Our K-12 STEM education performance lags most of Asia and Europe – as shown in the chart to the right, we rank between Hungary and Turkey in mathematics performance by 15-year-olds.  We also have deep and systemic inequalities in STEM education, and we have sadly created an environment that is less attractive to international graduate students and scholars than it once was.  The latter is especially worrisome given that over 60 percent of computer science and mathematics Ph.Ds. working in the United States were born overseas.

Practically, this means we need to recognize the need for determined actions:

• Raise the levels of federal investment in ongoing information technology R&D. This means billions, not millions of dollars.
• Invest in the talent pipelines needed to make academic and laboratory R&D attractive.
• Partner deeply with the U.S. private sector in new and creative ways.

Now, what about advanced scientific computing futures?

Leading Edge HPC Futures

It now seems self-evident that supercomputing, at least at the highest levels, is endothermic, requiring regular infusions of non-revenue capital to fund the NRE costs to develop and deploy new technologies and successive generations of integrated systems. In turn, that capital can come from either other, more profitable divisions of the business or from external sources (e.g., government investment). 

While at Microsoft, I once pitched a technical computing idea to CEO Steve Ballmer.  He listened patiently to my spiel, then simply said, “I’m not interested.”  Being nothing if not persistent, I asked why.  To which he replied, “It’s not that I think you are wrong.  I actually think you are right – we could drive a few hundred million dollars in revenue, but the opportunity cost is too high, and the return on investment (ROI) is too low.  Come back when you have an idea that starts with B (billions), not M (millions).”

I was chastened, but it was a valuable lesson that I fear too many in high-end scientific and engineering computing have not learned.  A business leader must always look at the opportunity cost (time and cost of money) for any NRE investments.  The core business question is always how to make the most money with the money you have, absent some other marketing or cultural reason to spend money on loss leaders, bragging rights, or political positioning.   The key phrase here is “the most money;” simply being profitable is not enough. 

Therein lies the historical “small fortune” problem for HPC.  The NRE costs for bleeding edge supercomputing are now large relative to the revenues and market capitalization of those entities we call “computer companies,” and they are increasingly out of reach for U.S. government agencies, at least under current funding envelopes. The days when a few million dollars could buy a Cray-1/X-MP/Y-MP/2 or a commodity cluster and land in the top ten of the Top500 list are long gone. Today’s game requires hundreds of millions of dollars to deploy a machine at the high end and at least similar, if not larger investments in NRE.

What does this brave new world mean for those of us in the big iron, bleeding edge HPC crowd?  First, as I mentioned over a decade ago, we are really the fast iron crowd, as our biggest systems are now dwarfed by the overall scale of commercial cloud infrastructure deployments. Second, we may still be invited to dinner, but economically, we are increasingly relegated to eating SpaghettiOs and drinking Kool-Aid at the kids table, while the adults are dining on lobster and drinking limited release, estate-bottled wine. 

Put another way, the golden rule applies – buying a $500M supercomputer every five years is not much financial leverage against a multibillion dollar spend each year. Equally importantly, most of the traditional supercomputing vendors are sitting at the kids table with us.  In turn, federal investment (e.g., DOE’s exascale PathForward program) is quite small compared to the scale of commercial cloud investments and their leverage with the same vendors.

First, it means acknowledging that we in bleeding edge HPC have limited leverage; we cannot influence vendors in the same ways we did in the past. The delays and cost overruns for Argonne National Laboratory’s Aurora system and the (resolving) deployment pains for Oak Ridge National Laboratory’s Frontier system speak to these challenges. Meanwhile, China has already deployed two exascale systems, both built using Chinese components.

We also need to recognize that bleeding edge HPC is about more than solving time-dependent partial differential equations on complex meshes.  They will always matter, but other areas of advanced computing are also of high and growing importance.  After all, the Science 2021 Breakthrough of the Year was for AI-enabled protein structure prediction, with transformative implications for biology and biomedicine.  This is a cultural shift that will require us to lessen our hold on some cherished beliefs and acknowledge the world has changed.

Second, it’s time we got serious – really serious – about end-to-end co-design, from device physics to applications, following the lead of the cloud ecosystem operators, and where possible and appropriate, partnering with them. This means more than encouraging tweaks of existing products or product plans.  Rather, it means looking holistically at the problem space, then envisioning, designing, testing, and fabricating appropriate solutions. (See Advanced Computing: Integrative Thinking for the Future.)  As a contact sport, it means substantially expanded investment in basic research and development, something that unfortunately suffered as the U.S. Exascale Computing Project (ECP) diverted limited funds to timeline-driven deployments

Third, we will need to embrace the chiplet ecosystem and work collaboratively with the commercial players to establish interoperability standards (e.g., the Open Domain-Specific Architecture (OSDA) project within the Open Compute Project). Chiplets are more than a way to mix and match capabilities from multiple sources, they are an economic and engineering reaction to the interplay of chip defect rates, the cadence of feature size reductions, and semiconductor manufacturing costs (CAPEX and OPEX). Concomitantly, we must both work with the new chiplet design and integration companies while also designing chiplets, both driven by the specific needs of quantified scientific workloads.

There ARE interesting things happening in the hardware space, but many of them are in the startup world.  One need look no further than companies like Graphcore, Groq, and Cerebras for inspiration, and at the ferment in edge devices and AI. I will not name other companies, as several are still in stealth mode. Part of this requires some flexibility and imagination.  (Full disclosure, I started NCSA on the GPU accelerator push by building a cluster of Sony Playstations and formed the first partnership with NSF on cloud computing.)

Fourth, we will need to build real hardware and software prototypes at scale, not just incremental ones, but ones that truly test new ideas using custom silicon and associated software. In turn, that means accepting the risk of failure at substantial scale (e.g., that $50M prototype didn’t work, but we learned some important things, and we are going to build another one).  (See Luck Is a Fickle Friend.)

To do these things, we will need to invest in recruiting and sustaining research teams – chip designers, system software developers, and packaging engineers – in an integrated end-to-end way AND create opportunities than make it intellectually attractive to work on science and engineering problems. Simply put, we will need to create integrated teams and build integrated prototypes at scale, both at national laboratories and in academia.

Finally, and I realize this may be heretical to some, we need to ask when and where we would be best served by embracing commercial cloud services for our HPC needs.  The performance gaps between cloud services and HPC have lessened substantially over the past decade.  See, for example, a recent comparative analysis by the UC-Berkeley group. HPC as a service is now both real and effective.

I realize some in academia and national laboratory community will immediately say, “But, we can do it cheaper, and our systems are bigger!”  Perhaps., though proving such claims means being dispassionately honest about technological innovation, NRE costs, opportunity costs, staff career goals, and the true total cost of ownership (TCO). In turn, this requires a mix of economic and cultural realism in making build versus use decisions and asking what cloud service vendors would be willing to do for academic and government consortia at large scale (e.g., a billion dollar annual partnership). 

In summary, it is increasingly unlikely that future high-end HPC systems will be procured and assembled solely by commercial integrators from commodity components.  Rather, they are likely to be (a) collaborative partnerships with cloud providers, (b) increasingly bespoke, designed and built collaboratively to support key scientific and engineering workload needs, or (c) a combination of these two.

Put another way, bleeding edge, exascale class computing systems are no different than the Vera Rubin Observatory, the LIGO gravity wave detector, or the Advanced Photon Source.  Each contains commercially designed and constructed technology, but they also contain large numbers of custom elements for which there is no sustainable business model.  Instead, we build these instruments because we want them to explore open scientific questions, and we recognize that their design and construction requires government investment.

Coda

As our U.S. political divisions illustrate, investing in the future will not be easy, but it is critical if the U.S. is to remain globally competitive. Let’s be clear.  The price of innovation keeps rising, the talent is following the money, and many of the traditional players – companies and countries – are struggling to keep up.  Andy Grove was only partially right.  It is true that only the paranoid survive, but they’d best have deep pockets and enticing intellectual opportunities as well.  That needs to be us.

N.B. My thanks to Jack Dongarra, Doug Burger, Dennis Gannon, Tren Griffin, Tony Hey, and Jim Larus for thoughtful feedback on drafts of this post.  Any errors of fact and all opinions are my own. In full disclosure, I am a former Microsoft corporate vice president and a current stockholder.

In Silico Whole Cell Modeling

First principles, ab initio computational modeling of biological processes is one of those grand visions, like artificial general intelligence, that both inspires the imagination and conjures images of mad scientists exclaiming, “It’s alive!”  At present, we are a long way from either, but real progress is being made. At times generative adversarial networks and systems biology may seem like magic, but they are not, Arthur C. Clarke’s observation (“Any sufficiently advanced technology is indistinguishable from magic.”) notwithstanding.  There is no magic, simply a plethora of scientific and technological advances by global teams of researchers.

The dream of predictive biological modeling is an old one, with many proponents, dating to some of the earliest days of digital computing.  Since then, progress has been slow, but steady, dependent on both greater understanding of biological processes and increased computational power. To be clear, ab initio biological modeling is distinct from artificial life, which examines systems and emergent behavior similar to, though different from biology, though there are important connections.

Why do we want to build such models?  The first and foremost reason is to validate our understanding of biological processes, comparing computational predictions with experimental results.  As Richard Feynman once remarked, “What I cannot create, I do not understand.” If we cannot build models that accurately duplicate experiment, we do not fully understand the biology. The second, of course, is to allow us to understand how changes to genes and gene expression can affect biological processes, with profound implications for the environment, and for health care and treatments.  A corollary of this is the ability to predict the effects of gene editing and other modifications, all within critical ethical boundaries.

Twenty years ago, when I was director of NCSA, the NSF Director, Rita Colwell, and I used to discuss the possibility of predictive whole cell modeling, the limitations in our biological knowledge and the computing infrastructure that might be required to make such a thing possible.  A brilliant biologist, she knew for more about the biological aspects than I ever have, though I’d like to believe I brought a bit of computing game to the discussion. Mind you, at that time, the Human Genome Project was still underway, and we were really excited to have just deployed terascale Linux clusters and the NSF TeraGrid as national resources.  Since then, we have come a long way in both biology and computing.

A Minimal Genome

Experimentalists have long targeted certain, simple lifeforms (e.g., Caenorhabditis elegans, a transparent nematode; Arabidopsis thaliana, a small flowering plant with a short lifecycle; Drosophila melanogaster, the fruit fly; and Saccharomyces cerevisiae, a yeast) as model organisms. The simplicity or specific morphology of those organisms simplify study. Similarly, those building computational biological models have long chosen to start with the simplest living cells. 

This raises the obvious question: what is the minimal genome required for life (i.e., the cell cycle of birth and death), and concomitantly, can we either find or construct such a minimal genome.  Minimal is a subtle thing, as the set of genes needed for a cell to grow and reproduce in a perfect medium is different from the larger and interrelated set that allows it to thrive in adverse or changing conductions.  Finally, the interrelation is key; the sum (the gene network) is more than the parts (the genes themselves).

In 2010, researchers at the J. Craig Venter Institute (JCVI) first created a chemically synthesized genome (JCVI-syn1.0), then in 2016 succeeded in creating a synthetic bacterial cell with a minimal number of genes, but capable of reproducing in a controlled environment.  JCVI-syn3A consists of fewer than 500 genes on a single circular chromosome. (See the Science articles in 2010 and 2016 for the details.)  Not surprisingly, JCVI-syn3A has since been the target of whole cell modeling, of increasing sophistication.

I wrote about the excitement surrounding the 2010 synthetic genome work in the Communications of the ACM, in a piece entitled In Vivo, In Vitro, In Silico. (I also posted on this blog.) As I noted then, the work was a fascinating example of the interplay of biology and computing, but it also highlighted that we did not fully understand the function of all the genetic material in even this simple genome. Indeed, when the 2016 article was published, the precise biological functions of roughly 30 percent of the JCVI-syn3.0 genes were unknown; the function of more, though not all, is now understood.

In Silico Modeling

In 2019, a University of Illinois team (Luthey-Schulten and colleagues) constructed a computational genome scale metabolic model (GSMM), a flux balance model, of the metabolic network (the metabolism) of JCVI-syn3A, showing how DNA specifications drive molecular processes.  That work, published in eLife, also raised questions about the minimal genome, offered some suggestions for further gene removal, and raised questions about some essential genes whose function was unclear.

I was beyond thrilled to read the latest whole cell modeling results from this team, which has continued expanding in silico models of JCVI-syn3A. Remember, a detailed whole cell model (WCM) includes the cell’s size, shape, components, reactions within the cell, and environmental interactions, all along the cell lifecycle. Published in Cell earlier this year, these simulations of a group of roughly 200 cells are a fully dynamical kinetic model capable of demonstrating emergent behavior.  (The University of Illinois press release can be read here.)

As the authors note, metabolism, genetic information processes, and growth can, at present, only be modeled via hybrid stochastic and deterministic simulations.  Specifically, they model the metabolic network via deterministic ordinary differential equations (ODEs) and the kinetics of genetic processes via stochastic models on a cubic lattice of the cell interior. Given their computational requirements, the latter are dependent on GPUs for acceleration. The additional details on the mathematics of the models are found in the paper, and the code is available on github (here and here), a substantial fraction of which was written in Python.

Looking Forward

Multiple groups are continuing to build ever more detailed cellular models, leveraging both growing biological understanding and greater computing capability.  Modeling the cellular behavior of a cell with a minimal genome is another step into an exciting future.

Stored Program Computing: Ideas and Individuals Do Change the World

Johnny von Neumann is a name often spoken with considerable reverence in scientific circles. The epitomy of prodigy and polymath – and even those words fail to do him justice – von Neumann had the kind of mind that made even the world’s most brilliant stare in slack jawed awe.  In idle musings on new topics, he often offered insights that would have been the crowning achievement and life’s work of other, lesser talents.

Indeed, based on his seminal contributions to mathematics, physics, economics, statistics, and yes, computing, one can make a rather compelling case that Johnny may well have been the smartest human being on the planet during his lifetime.  (Well, assuming he wasn’t a Martian, as Leo Szilard sometimes joked about the group of Hungarian expats.) As Edward Teller once put it, “von Neumann would carry on a conversation with my 3-year-old son, and the two of them would talk as equals, and I sometimes wondered if he used the same principle when he talked to the rest of us.” 

The “von Neumann” Design

Given von Neumann’s involvement in the Manhattan Project and its associated computing needs,  as well his deep familiarity with the work of Gödel and Turing, it is no surprise that he turned his attention to digital computing, writing the 101-page, “First Draft of a Report on the EDVAC,” while commuting by train from Princeton’s Institute for Advanced Study (IAS) to Los Alamos.  Released on June 30^th, 1945, the draft contained the first published description of the stored-program concept, now often called a von Neumann architecture, where both instructions and data were contained in the same memory, central arithmetic and control units coordinated instruction execution, and input/output devices serviced the computing infrastructure. It is no exaggeration to say the draft report transformed and codified modern digital computing. 

The report originated from von Neumann’s role as a consultant to the EDVAC (Electronic Discrete Variable Automatic Computer) design, and on its publication, the report generated some controversy. The EDVAC team, notably Mauchly and Eckert, felt von Neumann had simply summarized their design without listing them as authors, and via publication, prevented patents on the stored program ideas.  (In March 1946, both Eckert and Mauchly left the University of Pennsylvania over intellectual property disputes and formed the company that developed UNIVAC.)

Given his prodigious talent and contributions in other areas, one cannot help but believe that von Neumann generalized and codified several ideas he heard during the EDVAC conversations, but we are unlikely to ever know for sure. Regardless of the controversy, von Neumann’s report had a major influence on computer design and construction for the next decade, when each machine was a bespoke design.

Illinois Innovations

My first intellectual home, the University of Illinois, built two machines based on von Neumann’s IAS design, driven by Illinois faculty who were previously associated with the IAS. The first, ORDVAC, was delivered to the U.S Army Aberdeen Proving Grounds in 1951.  As part of that contract, Illinois was given funds to build a second copy in 1952, which became ILLIAC I (Illinois Automatic Computer), the first computer built and owned entirely by a U.S. educational institution.

It was the beginning of a long, illustrious, and continuing history of advanced computing innovation at Illinois, with the ILLIAC II (ECL, core memory, and pipelined operations), ILLIAC III (fine-grained SIMD pattern matching), ILLIAC IV (parallel processing), and Cedar (hierarchical shared memory) computing systems, as well as PLATO (Programmed Logic for Automatic Teaching Operations) computer-aided instruction, the Eudora e-mail client, and the Mosaic web browser, itself the progeny of the National Center for Supercomputing Applications (NCSA) ecosystem.

As an aside, just a few years before I arrived, the four color theorem, unresolved for nearly one hundred years, had been proven by Illinois mathematicians Appel and Haken, using a theoretical framework to test a provably finite number of cases by computer.  For many years, mail from the University of Illinois was embossed with the simple phrase “Four colors suffice.”  I loved that quiet and understated academic trash talking, a way of communicating, as Dizzy Dean once did, “It ain’t braggin’ if you can do it.”

Personal Reflections

I have written about several of these Illinois experiences over the years and how they have shaped my professional life, including the formation of the Renaissance Computing Institute (RENCI), application of high-performance computing ideas to energy efficient cloud data center design while at Microsoft, and my work as an academic leader:

• Luck Is a Fickle Friend (lessons from ILLIAC IV)
• Eudora, You Got the Love? (why passion matters)
• Mind to Mind: Building Innovation (reflections on PLATO and human empowerment)
• NCSA@30: Reflections on the Revolution (musings on rapid technological change)
• Renaissance Teams: Reifying the School at Athens (interdisciplinary collaboration insights)
• The Magic Behind the Curtain: Hardware and Software (the joy of teaching computing)

I have said elsewhere, and I will say again here, that I went to Illinois as a young assistant professor because it was as close to heaven as a young high-performance computing researcher could find here on Earth.  I was not disappointed!  I loved my time at Illinois because of the incredible people (Bardeen, Bitzer, Cox, Holonyak, Kuck, Lawrie, Smarr, Robertson, Slotnick, and so many others) and their repeated proof-by-demonstration that one should not just dream about the future, but that one can and must build it! (See Predicting Potentialities, Reifying Futures.)

I am proud to have had a role in adding a few small pieces to that rich history via my own work on distributed memory parallel computing, leadership of the Illinois’ Department of Computer Science, recruitment of new faculty, construction of Siebel Center, and the conception, design, and construction of the NSF TeraGrid (later XSEDE) while NCSA director. In between, we also had some fun, playing with gigabit testbeds and I-WIRE (thanks, Charlie Catlett), building parallel computing clusters from Sony PlayStations (See the New York Times article), and visualizing web traffic in the CAVE, when NCSA’s webserver was the busiest in the world.

Coda

At a time when some question our continuing will and ability to mount big, audacious initiatives, it is worth pausing to draw a few intellectual lessons from the pioneers of digital computing, and remember the transformative power of dedicated individuals and compelling ideas.  (See On the Etiology of Heroes and That’s One Small Step for {a} Man.)  It has always been important to imagine the future, not as a dream, but as reality to be realized.

As the legendary Chicago architect, Daniel Burnham remarked,

Make no little plans; they have no magic to stir mens’ blood and probably themselves will not be realized. Make big plans; aim high in hope and work, remembering that a noble, logical diagram once recorded will never die, but long after we are gone will be a living thing, asserting itself with ever-growing insistency.

The logical diagram of the ”von Neumann” architecture has lived on and continued to transform our world. Today’s supercomputers, smartphones, and the Internet of Things (IoT) all rely on variants of the von Neumann idea.

Yet, like all great ideas, the “von Neumann” architecture has its limitations. (See Nothing Lasts Forever.) The bandwidth and latency bottlenecks of the separated execution and storage units continue to drive architectural and technological innovation, in the pursuit of higher performance via end-to-end co-design.  (See Advanced Computing: Integrative Thinking for the Future.) The end of Dennard scaling has led to renewed focus on parallelism, architectural specialization, and energy efficiency.  If he were alive today, I am confident Johnny would still be excited about computing and would be deeply involved in work on quantum computing and approximate computations.

Ask the deepest questions.  Run with the big dogs.  The company, the journey, and the destinations all matter.  It has always been and always will be where the action is. On that, Johnny would wholeheartedly agree!

Advanced Computing: Integrative Thinking for the Future

SC19, the annual gathering of the advanced computing cognoscenti, many sporting shirts festooned with corporate and institutional logos and geek jokes, is now underway in Denver, Colorado. After arriving from the outskirts of western Nebraska and Kansas (i.e., Denver International Airport), high-performance computing fans are descending on the Colorado Convention Center in downtown Denver, eager to discuss new server design, interconnect advances, and software, while surfing the artificial intelligence tsunami that now consumes all things computing, whether exascale or edge, bare metal or containerized.

 I will soon be at SC19 myself, visiting with all my geeky friends, after being at the National Science Board and having dinner with the new PCAST members, but this essay is about none of that. Rather, I want to highlight two recent reports, both of which outline the need for a rethinking of our approaches.  Put another way, Feynman’s 1959 “go small” clarion call in There’s Plenty of Room at the Bottom has been answered, and there’s not much more room at the inn, for CMOS FinFETs are down to small numbers of atoms.

Concurrently, we are encountering fabrication and economic challenges below 7 nanometers, though there is still progress to be made. As I told Nature in 2016, I think we will run out of money before we run out of physics. (See The Chips Are Down for Moore’s Law.)  Regardless, innovation will continue, albeit along other axes.  (See Nothing Lasts Forever for musings on how aircraft innovation has morphed from a focus on speed.)

Last week, the U.S. Department of Energy released the Basic Research Needs for Microelectronics report, which many of us have been editing and writing for the past year.  I was pleased to co-chair the effort with Cherry Murray (Harvard/Arizona) and Supratik Guha (Argonne National Laboratory).  The report, a collaboration among three Department of Energy offices – Advanced Scientific Computing Research (ASCR), Basic Energy Sciences (BES), and High Energy Physics (HEP) – highlights five critical needs:

1. Flip the current paradigm by defining innovative material, device, and architecture requirements, driven by applications, algorithms, and software. We must break down the hierarchical abstractions that have served us so well and engage in holistic, end-to-end design, where, for example, material properties can shape design of an algorithm.  This requires, of course, a new model of education and collaborative design, of which the following four items are specific instances.
2. Revolutionize memory and data storage. Our current memory hierarchies are too energy inefficient and too slow, and their capacities are too long to meet future needs.  We need some new materials science and physics insights, coupled with innovative design.
3. Reimagine information flow unconstrained by interconnects. Current intrachip and interchip energy costs and data transfer rates, coupled with von Neumann architectures limit design options.
4. Redefine computing by leveraging unexploited physical phenomena. Seek new materials, beyond just CMOS evolution.
5. Reinvent the electricity grid via new materials, devices, and architectures. This is but one of many application challenges, driven by the rise of renewable energy and the need to build a next-generation, highly resilient grid.

Last week, the Fast Track Action Committee on Strategic Computing, part of the U.S. National Science and Technology Council (NSTC), released the National Strategic Computing Initiative Update: Pioneering the Future of Computing. The report’s opening sentence echoes many of the points in the Department of Energy study, noting,

The national computing landscape is undergoing rapid evolution along multiple dimensions due to the introduction of new and potentially disruptive technologies and the demands of new classes of data intensive applications. Computer architectures and systems are more heterogeneous and complex, and the challenges associated with the complexity and sustainability of software are significant.

The NSTC strategic computing report recommendations include

1. Embrace a diversity of hardware and software approaches for the future of computing and leverage the innovation ecosystem
2. Encourage novel solutions that leverage in-network and edge-processing capabilities to process data close to the source as part of end-to-end application workflows
3. Support the rapid translation to practice of basic R&D and technology to address scientific challenges requiring the effective integration of advanced software and hardware

There’s trouble in River City. The free ride from Dennard scaling is over; the von Neumann architecture is showing its age; and the hierarchical abstractions we have so cherished are now an intellectual box from which we must escape. 

It’s time for ab initio, integrative, transdisciplinary thinking, as I discussed in Renaissance Teams: Reifying the School at Athens.

Technical Potpourri

Edge Computing

SC17, the annual gathering of the supercomputing community, was held in mid-November, the week before U.S. Thanksgiving. As I mentioned in a previous post (Come to the Supercomputing Big Data Edge), we presented the results of the Big Data and Extreme-scale Computing (BDEC) workshops on the technical and social convergence of big data, machine learning, high-performance computing, and computing hardware and software. You can find the report here. At SC17, we had a lively discussion about edge computing, global competitiveness, and the nature of scientific revolutions.

HPC Data Center Energy Efficiency

In addition to the BDEC discussion, I also chaired an SC17 panel on the energy efficiency of HPC data centers. The panel was organized with the help of the Energy Efficient High Performance Computing Working Group (EEHPCWG). The panel participants were Sadaf Alam (CSCS/ETH), Bill Gropp (NCSA/Illinois), Satoshi Matsuoka (GSIC/Tokyo Tech), and John Shalf (NERSC/LBL). You can find the slides and a video of the panel here, courtesy of InsideHPC.

Autonomous Vehicles

Finally, I was recently a guest on Iowa Public Radio's lunchtime conversation, River to River. We had a lively and thoughtful conversation about the potential social consequences of autonomous vehicles. You can listen to that audio here.

Related articles

Exascale Computing and Big Data: Time to Reunite

Come to the Supercomputing Big Data Edge

SC17, the annual gathering of the supercomputing community is nigh. As always, there will be talk of next-generation technology – hardware, software, algorithms – and new applications. All of this will take place against the backdrop of the race to exascale computing – bigger and bigger, faster and faster. We will all geek out, and we will love it.

While we discuss operations/joule, quantum and neuromorphic computing, and new and exotic memory technologies to enable ever faster systems, there is another computing revolution underway, at the other extreme. As Richard Feynman famously said about nanoscale devices, "There's plenty of room at the bottom." Folks, I'm talking edge devices, the world of streaming data sensors that is changing the nature of computing and increasingly, of science as well. This is the subject of the forthcoming Big Data and Extreme-scale Computing (BDEC) report.

Computing History 101

The history of computing is one of punctuated equilibrium, with each pendulum swing, from centralized to distributed, bringing new and unexpected changes. It has also been a story of disruption from below, as successive generations brought order of magnitude shifts in price-performance ratios and sales volumes. Mainframes beget minicomputers, which begat workstations, which begat personal computers, which begat smartphones. The same has been true in scientific computing. Monolithic supercomputers (think the original Cray series) gave way to symmetric multiprocessors (SMPs) and distributed memory supercomputers. These gave way to clusters, now augmented with accelerators. Mind you, I still miss the lights on the Thinking Machines CM-5.

In both technical and "mainstream" computing, our largest systems are of unprecedented scale, whether globe spanning networks of massive cloud data centers or petascale and soon-to-be exascale HPC systems. As the Internet of Things sweeps across the computing landscape like a tsunami, one should ask about its analog in technical computing – the burgeoning networks of sensors that now capture science, engineering, environment, health and safety, and social data. (See Mobility, Transduction, and Garages)

The Sensor Edge

Today's smartphone-cloud ecosystem places most of the intelligence and data in the cloud. Indeed, all of the intelligent assistants – Siri, Cortina, and others – run in the cloud, dependent on precious wireless network bandwidth and relying on both behavioral history and deep learning algorithms. Limited by smartphone processing speeds, memory capacity, power budgets, and battery lifetime, and human expectations and physiology, there are good economic and technical reasons for this design.

There are also disadvantages, particularly in science. First, transmitting raw, time sensitive data to central sites places major demands on national and international networks. When the Large Hadron Collider (LHC) was under construction, this bandwidth and data processing demand was a major concern, triggering an international collaboration to build a distributed data storage hierarchy; those concerns remain as luminosity upgrades to the HLC are being planned.

The planned data pipeline for the Large Synoptic Survey Telescope (LSST) will send the raw data from the telescope in Chile to a dedicated supercomputer at NCSA, with a sixty second (maximum) delay before global alerts of transient phenomena must be distributed. Although this centralized approach can and will work when the daily data volume is measured in terabytes, the data deluge of the Square Kilometer Array (SKA) will require in situ reduction.

My friends in Chicago's Array of Things project grappled with another aspect of this local/remote tradeoff when designing urban sensors to capture and analyze human interactions. Communities were happy to support social dynamics analysis, but they did not want Big Brother video to leave the sensors. The privacy and social constraint requires low power, high-performance machine learning solutions on the sensors themselves.

Scale Challenges

Centralization also brings operational and security risks. My former Microsoft colleague, James Hamilton (now at Amazon), when asked why cloud data centers are not even bigger, said, "… the cost savings from scaling a single facility without bound are logarithmic, whereas the negative impact of blast radius is linear." We have all experienced the blast radius effect, at least metaphorically; when a cloud data center goes down, it affects both social networks and business, and it is international news. There is, perhaps, a salutatory lesson here for the ultimate, maximum size of supercomputers, but that is a discussion for another day.

In addition, the stateless design of the venerable TCP/IP stack includes no notion of data geo-location or intermediate data processing. Everyone who has debugged "last mile" network performance problems knows this all too well. To remediate this, research groups and companies have designed a variety of domain-specific network overlays, data caches, publisher-subscriber systems, and content delivery networks (CDNs).

Dramatic declines in computing costs mean it is now possible to deploy truly large numbers of inexpensive sensors. How dramatic? I have posed the following question on my campus. What would you do with 10,000, disposable ~$10 sensors? I was met with surprise at the scale of my question. For many of us, even those who came of age in the smartphone era, this scaling is a difficult question to answer because it defies our worldview of the possible. It is not a science fiction fantasy; it is a present reality.

A traditional hub and spoke model of data storage and sensor data analysis is increasingly neither scalable nor reliable. Amazon's Greengrass architecture is an early example of a generic, distributed approach, while NVidia's Metropolis system targets video in smart cities. There is an even more general model, where generic services support distributed data caching and processing throughout the network. Micah Beck (Tennessee) has made a thoughtful case for the design of such an approach, which he calls an Exposed Buffer Processing Architecture. Similarly, the Array of Things leaders have argued that edge processing is essential when

• The data volume overwhelms the network capability
• Privacy and security encourage short-lived data
• Resilient infrastructure requires local actuation/response
• Latency constraints demand the response times of dedicated resources

Simply put, these requirements imply a much more complex, distributed data storage and processing architecture for data-driven scientific computing than we have developed to date.

Supercomputing Edge Manifesto

This brave new world of streaming data has profound implications for the future of supercomputing. Here are just a few:

• Streaming data is continuous and requires a reconceptualization of workflow and scheduling systems. Periodic, batch processing is necessary but not sufficient.
• Sensor scale and complexity span many orders of magnitude, from inexpensive, disposable devices to nation-scale instruments. New orders of magnitude always expose new bottlenecks and necessitate architectural change to identify and decompose domain-specific and domain-independent elements.
• Systemic resilience takes on new urgency, extending beyond central HPC to include networks and distributed devices.
• Machine learning, modeling, and data assimilation apply concurrently at many sites and multiple levels, subject to energy and packaging constraints at the edge.
• We in HPC are not the only players in this space. Much as we learned and adapted much from the open source Linux community, we must learn from and embrace distributed data analytics from the sensor community.

As the name implies, the Big Data and Extreme-scale Computing (BDEC) project has been examining this nexus and the changing nature of scientific discovery enabled by the fusion of big data – from sensors large and small – machine learning, and computational modeling. It is a rapidly shifting environment, with new hardware and software ecosystems evolving rapidly.

Much of it remains largely unknown to the traditional HPC community, just as software containerization and machine learning were just a few years ago. That will change very soon. If you are attending SC17, come to the Birds of a Feather (BOF) discussion, late Wednesday, November 15.

A Literary Coda

In his book, Hell's Angels, the inimitable gonzo journalist Hunter S. Thompson once wrote about living on the edge – riding flat out – in the dark – on a motorcycle:

The Edge…There is no honest way to explain it because the only people who really know where it is are ones who have gone over. The others–the living–are those who pushed their control as far as they felt they could handle it, and then pulled back, or slowed down, or did whatever they had to when it came time to choose between Now and Later.

In supercomputing, we have always felt the need for speed, exploiting every new technology in the endless search for higher performance and greater capacity. It's time for the supercomputing community to come to the edge, where sensors and streaming data meet HPC, computational models, and machine learning.

There's lots of room at the edge for innovation and discovery. Let's not crash in the dark.

Related articles

Exascale Computing and Big Data: Time to Reunite