Reed's Ruminations: A Blog by Dan Reed

American Competitiveness: IT and HPC Futures – Follow the Money and the Talent

N.B.  A more formal version of these arguments is posted on arXiv (https://arxiv.org/abs/2203.02544), co-authored with Dennis Gannon and Jack Dongarra.

This is a long post, containing perspectives that may be controversial to some.  That's okay; it is why we have lively intellectual debates.

Charles Dickens’ A Tale of Two Cities opens with this line:

IT was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of light, it was the season of darkness, it was the spring of hope, it was the winter of despair.

It’s an apt metaphor for the state of information technology and the state of U.S. competitiveness in computing and science, technology, engineering, and mathematics (STEM).  In some ways, the hegemony of U.S. IT has never been greater, with Apple and the cloud companies (Google, Microsoft, Amazon, and Facebook) dominating the Nasdaq with trillion dollar (and more) market capitalizations and reshaping the very definitions of computing with global services, deep learning, and advanced AI.

Conversely, the semiconductor shortage has highlighted the economic and national security risks from over-dependence on off-shore fabrication, and the relative influence of traditional computing companies on the IT ecosystem has waned. Meanwhile, the future of U.S. leadership in bleeding edge scientific computing (aka supercomputing) is at a critical crossroads.

 Finally, while the global competition for STEM talent in general and IT talent in particular has never been more intense, the U.S. is substantially underperforming relative to other countries in STEM education.  Let’s take a look at how all of this has happened.  Remember, follow the money and the talent.

Remembering the Past

Let’s start by playing a parlor game. Name the first computer company that comes to mind.  If you are an old timer, you might be inclined to mention IBM, Amdahl, and the BUNCH (Burroughs, UNIVAC, NCR, CDC, and Honeywell).  Although IBM once bestrode the computing world like a colossus, today, even its relevance has faded. Indeed, few now remember that IBM was once locked in a bitter struggle with the U.S. Department of Justice for monopolistic practices under anti-trust laws.  As for the BUNCH, they are either gone from the computer business or niche players at best. Later, others – Compaq, Digital Equipment Corporation (DEC), and SGI – came and went.

As an aside, I learned to program on an IBM System 360/50, wrote lots of code on a DEC PDP-11/45 for my first paid software development job, ported code from a GE 600 timesharing system, and used a CDC 6500 and 6600 during graduate school. I programmed in FORTRAN, COBOL, PASCAL, LISP, and PI/I (in addition to many more recent languages), and I’ve toggled absolute binary into the front panel. I’m also old enough to remember when one could do Mead and Conway VLSI design with colored pencils.  I also used a slide rule, while walking barefoot in the snow to school, uphill (both ways), but I digress. (See You Might Be a Computing Old Timer If …)

There’s a sad, parallel tale (pun intended) in the land of high-performance computing, where the cemetery is filled with departed denizens, some large, some small, but all now permanent members of the Dead Computer Society. Whether by misjudging markets or technologies, each in in its own way proved the truth of the old adage that the best way to make a small fortune in supercomputing is by starting with a large fortune, then shipping one or two generations of product. Sic transit gloria mundi SGI, Cray, and a host of others. (See HPC: Making a Small Fortune.) 

More to the point, the list of high-end supercomputing players keeps getting shorter.  Cray is gone, now part of HPE; and both IBM and Intel are struggling to right themselves, each for different reasons.  In the United States, this leaves HPE as the lone high-end HPC integrator in the United States.  Time will tell if Nvidia, with its recent acquisitions, steps into this space, though its proposed acquisition of ARM seems unlikely.

Meanwhile, the information technology ecosystem and the locus of innovation and global leadership have shifted dramatically. There are deep and important implications for the U.S.

Contemplating the Present

If you are Gen Z or Gen Alpha, the world begins and ends with Apple, Google, and Samsung smartphones and the cloud services behind them.  All of which brings us to the FAANG companies (Meta née Facebook, Amazon, Apple, Netflix, and Alphabet née Google) plus Microsoft and their Asian BAT (Baidu, Alibaba, and Tencent) counterparts.  These companies are economically exothermic in ways perhaps unprecedented since the days of the Standard Oil monopoly.  Doubt me?  Look at their market capitalizations  and free cash flow relative to traditional computer companies. (As I write this, Microsoft just made a $69B all-cash offer for Activision and reported over $50B in quarterly revenue.)

I first watched this power dynamic begin to shift while at Microsoft.  For years, the Wintel duopoly locked Microsoft and Intel in a collaborative/competitive partnership as rough peers.  I sat in more than one “frank and spirited” exchange between leaders of the two companies, where each pushed the other to accommodate the other’s market needs. As server chips, clouds, and infrastructure grew in importance relative to desktops and laptops, the power dynamic began to shift.  Today, Intel is the far weaker player, in many cases forced to respond to cloud vendor demands.  In hindsight, it is all obvious.

Technological and Economic Shifts

Today, Apple and the other cloud service companies dominate the computing hardware and software ecosystem, both in scale and in technical approaches. Initially, they purchased standard hardware for deployment in traditional colocation centers (colos). Then they began designing purpose-built data centers, optimized for power usage effectiveness (PUE), deployed at sites selected via multifactor optimization – inexpensive energy availability, tax breaks and political subsidies, political and geological stability, network access, and customer demand. 

As scale, complexity, and operational experience grew, new optimization and leverage opportunities emerged, including software defined networking, protocol offloads, and custom network architectures (greatly reducing dependence on traditional network hardware vendors), quantitative analysis of processor, memory, and disk failure modes, with consequent redesign for reliability and lower cost (dictating specifications to vendors via consortia like Open Compute), custom processor SKUs, custom accelerators (FPGAs and ASICs), and finally, complete processor design (e.g., Apple silicon, Google TPUs and AWS Gravitons).  In between, they deployed their own global fiber networks.

This virtuous cycle of insatiable consumer demand for rich services, business outsourcing to the cloud, expanding data center capacity, and infrastructure cost optimization has had several effects.  First, it has dramatically lessened – and in many cases totally eliminated -- FAANG/Microsoft/BAT dependence on traditional computing vendors and ameliorated the risks of wholesale transfer pricing. Put another way, Amazon AWS, when necessary, can now negotiate pricing terms with Intel and AMD in the same way Walmart does with Procter and Gamble (i.e., from a position of strength as a dominant customer).

Second, the absolute scale of infrastructure investment, denominated in billions of dollars per year for each cloud service vendor, means these companies are shaping computing technology research and development priorities in ways traditional computing vendors rarely can, driving design and fabrication priorities in every element of the ecosystem.

Bigger is not simply bigger; bigger is different, fueling investment in new technologies at a phenomenal rate. (See The Zeros Matter: Bigger Is Different.) Relatedly, I cannot overemphasize the massive scale of these cloud data center deployments, dwarfing anything seen in academia or national laboratories, with the already large gap widening each year.  The CAGR is simply astounding.

Third, custom infrastructure is about more than economic leverage; it’s about market differentiation and competitive advantage.  Anything not differentiating is standardized and commoditized, driving unnecessary cost out of the system, and concomitantly reducing profits for commodity purveyors.  Conversely, anything differentiating is the focus of intense internal optimization and aggressive deployment.

Equally importantly, client plus cloud services vendors predominately make their money from the services offered atop their infrastructure, much less from the infrastructure itself. Unlike Intel, AMD, and Nvidia, which sell silicon, Apple is not selling A15 SoCs; it is selling iPhones, just as Amazon and Google are selling services atop TPUs and Gravitons.

Fourth, all of this is occurring against a backdrop of big data and machine learning. Large-scale AI itself is fueling a computing revolution, with an insatiable hardware demand to train ever larger machine learning models.  As an example, modern speech recognition models now have hundreds of billions of parameters, and multiple groups have announced models with over one trillion parameters. Similarly, work on generative adversarial networks (GANs) has led to impressive advances in game playing and other tasks. Nor is deep learning limited to “AI workloads;” there are impressive results on solving differential equations as well.  (See Dennis Gannon’s excellent summary of Karniadakis’ work here.) Similarly, look at developments in generative adversarial networks (GANs),

Finally, all of these trends are convolved with the end of Dennard scaling, dark silicon, functional specialization, chiplets, semiconductor fabrication challenges, rising fabrication facility costs, and an uncertain political future for Taiwan and TSMC.  Moreover, the actual cost of transistors (dollars/transistor) is increasing as we move closer to one nanometer scaling.  In fact, the transistor cost minimum occurred at roughly the time (~28 nanometers) we switched from planar CMOS to FinFETs. It will likely only get worse as gate-all-around FETs predominate, given increased fabrication complexity.

In the midst of all this, Intel stumbled, losing its lead in process technology just as EUV finally and belatedly became mature, and TSMC staked a lead. Time will tell if Pat Gelsinger can turn things around.

The Future

Okay, let’s put all the technology and economics puzzle pieces together:

• Advanced computing requires non-recurring engineering (NRE) investment to develop new technologies and systems.
• The FAANG/Microsoft/BAT companies are cash rich (i.e., exothermic), and they are designing, building, and deploying their own hardware and software infrastructure at unprecedented scale.
• The traditional computing vendors are now small – in some cases, tiny – economic players in the computing ecosystem, and many are dependent on federal investment (i.e., endothermic) for NRE to advance the bleeding edge.
• AI is fueling a revolution in how we think about problems and their numerical solution.
• Dennard scaling is over and performance advances increasingly depend on functional specialization via custom ASICs and chiplet-integrated packages.
• Transistor costs are likely to continue to increase as we push to smaller feature sizes.
• Nimble hardware startups are exploring new ideas, driven by the AI frenzy.
• Talent is following the money and the opportunities, which are increasingly in industry.

With this backdrop, what is the future? I believe some of it is relatively obvious and secure, given the current dominance of U.S. cloud service providers. However, broader U.S. leadership in semiconductors, information technology, and bleeding edge HPC is less certain. I believe any viable solutions will require rethinking our approaches and assumptions, including how, where, and when we spend money and what will both grow and retain talent.  Let’s start with the broad issues around the U.S. information technology ecosystem.

Information Technology Futures

The big changes outlined above make it obvious that a computing revolution is underway, one that is reshaping global leadership in critical areas.  All revolutions create opportunity, but only for those prepared to accept the risks and act accordingly. In Far Side cartoon parlance, we must be tall enough to attack the city.

First, we need to be honest about the social, political, economic, and national security risks of losing U.S. leadership and control of the silicon fabrication ecosystem.  Fabless semiconductor firms are important, but we also need onshore, state of the art fabrication facilities, and we need to invest and act accordingly. It is why the U.S. CHIPS Act and its successors are so important, and why the recent announcements by Intel, TSMC, and GlobalFoundries that they are building new chip fabrication faculties in the U.S. (each for different reasons), is so important.

Second, the seismic business and technology shifts have also led to equally dramatic talent shifts. People gravitate to opportunities, and that giant sucking sound is people leaving academia, national laboratories, and traditional computing companies to pursue those opportunities.  Seymour Cray once said, “I wanted to make a contribution,” when asked why he had used so many high risk technologies in the Cray-1. The desire to make a difference is no less powerful today.

The academic brain drain among artificial intelligence researchers is well documented.  After all, in the commercial world one can develop and test ideas at a scale simply not possible in academia, using truly big data and uniquely scaled hardware and software infrastructure.  The same is true for chip designers and system software developers.

Making matters worse, the U.S. talent pipeline is uncertain.  As the recent 2022 Science and Engineering Indicators shows, the U.S. is losing ground compared to the rest of the world, trailing China in both patents and publications, as summarized in this Science story about the recent Science and Engineering Indicators report. 

Our K-12 STEM education performance lags most of Asia and Europe – as shown in the chart to the right, we rank between Hungary and Turkey in mathematics performance by 15-year-olds.  We also have deep and systemic inequalities in STEM education, and we have sadly created an environment that is less attractive to international graduate students and scholars than it once was.  The latter is especially worrisome given that over 60 percent of computer science and mathematics Ph.Ds. working in the United States were born overseas.

Practically, this means we need to recognize the need for determined actions:

• Raise the levels of federal investment in ongoing information technology R&D. This means billions, not millions of dollars.
• Invest in the talent pipelines needed to make academic and laboratory R&D attractive.
• Partner deeply with the U.S. private sector in new and creative ways.

Now, what about advanced scientific computing futures?

Leading Edge HPC Futures

It now seems self-evident that supercomputing, at least at the highest levels, is endothermic, requiring regular infusions of non-revenue capital to fund the NRE costs to develop and deploy new technologies and successive generations of integrated systems. In turn, that capital can come from either other, more profitable divisions of the business or from external sources (e.g., government investment). 

While at Microsoft, I once pitched a technical computing idea to CEO Steve Ballmer.  He listened patiently to my spiel, then simply said, “I’m not interested.”  Being nothing if not persistent, I asked why.  To which he replied, “It’s not that I think you are wrong.  I actually think you are right – we could drive a few hundred million dollars in revenue, but the opportunity cost is too high, and the return on investment (ROI) is too low.  Come back when you have an idea that starts with B (billions), not M (millions).”

I was chastened, but it was a valuable lesson that I fear too many in high-end scientific and engineering computing have not learned.  A business leader must always look at the opportunity cost (time and cost of money) for any NRE investments.  The core business question is always how to make the most money with the money you have, absent some other marketing or cultural reason to spend money on loss leaders, bragging rights, or political positioning.   The key phrase here is “the most money;” simply being profitable is not enough. 

Therein lies the historical “small fortune” problem for HPC.  The NRE costs for bleeding edge supercomputing are now large relative to the revenues and market capitalization of those entities we call “computer companies,” and they are increasingly out of reach for U.S. government agencies, at least under current funding envelopes. The days when a few million dollars could buy a Cray-1/X-MP/Y-MP/2 or a commodity cluster and land in the top ten of the Top500 list are long gone. Today’s game requires hundreds of millions of dollars to deploy a machine at the high end and at least similar, if not larger investments in NRE.

What does this brave new world mean for those of us in the big iron, bleeding edge HPC crowd?  First, as I mentioned over a decade ago, we are really the fast iron crowd, as our biggest systems are now dwarfed by the overall scale of commercial cloud infrastructure deployments. Second, we may still be invited to dinner, but economically, we are increasingly relegated to eating SpaghettiOs and drinking Kool-Aid at the kids table, while the adults are dining on lobster and drinking limited release, estate-bottled wine. 

Put another way, the golden rule applies – buying a $500M supercomputer every five years is not much financial leverage against a multibillion dollar spend each year. Equally importantly, most of the traditional supercomputing vendors are sitting at the kids table with us.  In turn, federal investment (e.g., DOE’s exascale PathForward program) is quite small compared to the scale of commercial cloud investments and their leverage with the same vendors.

First, it means acknowledging that we in bleeding edge HPC have limited leverage; we cannot influence vendors in the same ways we did in the past. The delays and cost overruns for Argonne National Laboratory’s Aurora system and the (resolving) deployment pains for Oak Ridge National Laboratory’s Frontier system speak to these challenges. Meanwhile, China has already deployed two exascale systems, both built using Chinese components.

We also need to recognize that bleeding edge HPC is about more than solving time-dependent partial differential equations on complex meshes.  They will always matter, but other areas of advanced computing are also of high and growing importance.  After all, the Science 2021 Breakthrough of the Year was for AI-enabled protein structure prediction, with transformative implications for biology and biomedicine.  This is a cultural shift that will require us to lessen our hold on some cherished beliefs and acknowledge the world has changed.

Second, it’s time we got serious – really serious – about end-to-end co-design, from device physics to applications, following the lead of the cloud ecosystem operators, and where possible and appropriate, partnering with them. This means more than encouraging tweaks of existing products or product plans.  Rather, it means looking holistically at the problem space, then envisioning, designing, testing, and fabricating appropriate solutions. (See Advanced Computing: Integrative Thinking for the Future.)  As a contact sport, it means substantially expanded investment in basic research and development, something that unfortunately suffered as the U.S. Exascale Computing Project (ECP) diverted limited funds to timeline-driven deployments

Third, we will need to embrace the chiplet ecosystem and work collaboratively with the commercial players to establish interoperability standards (e.g., the Open Domain-Specific Architecture (OSDA) project within the Open Compute Project). Chiplets are more than a way to mix and match capabilities from multiple sources, they are an economic and engineering reaction to the interplay of chip defect rates, the cadence of feature size reductions, and semiconductor manufacturing costs (CAPEX and OPEX). Concomitantly, we must both work with the new chiplet design and integration companies while also designing chiplets, both driven by the specific needs of quantified scientific workloads.

There ARE interesting things happening in the hardware space, but many of them are in the startup world.  One need look no further than companies like Graphcore, Groq, and Cerebras for inspiration, and at the ferment in edge devices and AI. I will not name other companies, as several are still in stealth mode. Part of this requires some flexibility and imagination.  (Full disclosure, I started NCSA on the GPU accelerator push by building a cluster of Sony Playstations and formed the first partnership with NSF on cloud computing.)

Fourth, we will need to build real hardware and software prototypes at scale, not just incremental ones, but ones that truly test new ideas using custom silicon and associated software. In turn, that means accepting the risk of failure at substantial scale (e.g., that $50M prototype didn’t work, but we learned some important things, and we are going to build another one).  (See Luck Is a Fickle Friend.)

To do these things, we will need to invest in recruiting and sustaining research teams – chip designers, system software developers, and packaging engineers – in an integrated end-to-end way AND create opportunities than make it intellectually attractive to work on science and engineering problems. Simply put, we will need to create integrated teams and build integrated prototypes at scale, both at national laboratories and in academia.

Finally, and I realize this may be heretical to some, we need to ask when and where we would be best served by embracing commercial cloud services for our HPC needs.  The performance gaps between cloud services and HPC have lessened substantially over the past decade.  See, for example, a recent comparative analysis by the UC-Berkeley group. HPC as a service is now both real and effective.

I realize some in academia and national laboratory community will immediately say, “But, we can do it cheaper, and our systems are bigger!”  Perhaps., though proving such claims means being dispassionately honest about technological innovation, NRE costs, opportunity costs, staff career goals, and the true total cost of ownership (TCO). In turn, this requires a mix of economic and cultural realism in making build versus use decisions and asking what cloud service vendors would be willing to do for academic and government consortia at large scale (e.g., a billion dollar annual partnership). 

In summary, it is increasingly unlikely that future high-end HPC systems will be procured and assembled solely by commercial integrators from commodity components.  Rather, they are likely to be (a) collaborative partnerships with cloud providers, (b) increasingly bespoke, designed and built collaboratively to support key scientific and engineering workload needs, or (c) a combination of these two.

Put another way, bleeding edge, exascale class computing systems are no different than the Vera Rubin Observatory, the LIGO gravity wave detector, or the Advanced Photon Source.  Each contains commercially designed and constructed technology, but they also contain large numbers of custom elements for which there is no sustainable business model.  Instead, we build these instruments because we want them to explore open scientific questions, and we recognize that their design and construction requires government investment.

Coda

As our U.S. political divisions illustrate, investing in the future will not be easy, but it is critical if the U.S. is to remain globally competitive. Let’s be clear.  The price of innovation keeps rising, the talent is following the money, and many of the traditional players – companies and countries – are struggling to keep up.  Andy Grove was only partially right.  It is true that only the paranoid survive, but they’d best have deep pockets and enticing intellectual opportunities as well.  That needs to be us.

N.B. My thanks to Jack Dongarra, Doug Burger, Dennis Gannon, Tren Griffin, Tony Hey, and Jim Larus for thoughtful feedback on drafts of this post.  Any errors of fact and all opinions are my own. In full disclosure, I am a former Microsoft corporate vice president and a current stockholder.

5G: A Failure to Communicate

I have been watching the kerfuffle between the airlines and the wireless providers over the U.S. rollout of 5G cellular service with some consternation. (For a bit of background on the conflict, see this recent New York Times story.)  Some international airlines have canceled or restricted flights to the United States, and U.S. carriers have also raised alarms.  In turn, the Federal Aviation Administration issued an emergency order, temporarily delaying 5G activation near airports while it investigated.

Let’s start with a bit of background.  Wireless spectrum is like beachfront property.  You cannot make more and some is more valuable than others.  Different spectrum regions have different propagation characteristics, making them more or less useful for different purposes. Hence, the electromagnetic spectrum is chopped up in fixed bands, allocated for specific usage.  For example, some is for emergency responders and hospitals, some for radio and television, some for police and civil defense, some for military purposes, some for amateur radio, and some for commercial use. In recent years, there has been intense competition among the U.S. cellular carriers for spectrum, with the Federal Communications Commission (FCC) auctioning spectrum for 5G deployment.

In the U.S., the Verizon and AT&T 5G deployment operates in the C band, between 3.7 and 3.98 GHz, which is near the 4.2-4.4 GHz bands used by aircraft radar altimeters for ground proximity tracking during low visibility landings. The FAA has warned that the radar altimeters might be adversely affected due to possible interference with those radar altimeters. Under International Telecommunications Union (ITU) rules, radio altimeters are classified as safety services and must be protected from interference.

However, the FCC has required a 220 MHz interference buffer (a so called guard band) between the upper end of the U.S. 5G usage (3.98 GHz) and the lower end of the avionics usage at 4.2 GHz. It all comes down to how good the spectrum filters are on the radio receivers, where there is a good bit of variation, based on the age of the airplanes. 

On January 20, 2022, the FAA issued new approvals that allow most commercial flights to perform low visibility landings where there are 5G deployments in the C band.  The FAA notes, “We anticipate some altimeters will be too susceptible to 5G interference. To preserve safety, aircraft with those altimeters will be prohibited from performing low-visibility landings where 5G is deployed because the altimeter could provide inaccurate information.”

You might ask why this is an issue in the United States, given the long planned, global rollout of 5G.  The technical answer is simple – the details of the deployments differ slightly, but the political answer is complex. First, Europe is deploying 5G in the 3.4-3.8 GHz range, lower than the 3.7-3.98 GHz used in the U.S. and further from the spectrum used for radar altimeters (i.e., there is a larger buffer). There are also differences in the heights of 5G antennas, the power levels used, and – for antennas near airports – how they are tilted relative to airplane flight paths.

Could all of this U.S. kerfuffle have been avoided?  Of course, in many possible ways.  It brought to mind the iconic line from Cool Hand Luke, when the prison warden, played by Strother Martin, tells Luke, played by Paul Newman, “What we've got here is failure to communicate.”

As someone who served on the FCC Technological Advisory Council (TAC) while at Microsoft and who worked regularly with the ITU, I have seen spectrum politics from a front row seat.  Tom Wheeler, who chaired the TAC when I was a member, has written one perspective in these issues for the Brookings Institution. The FAA, as I noted above, has a somewhat different perspective, with the wireless providers and airlines battling in between.  What is undeniable is that this could have been prevented by clearer and better coordinated policy leadership in Washington.  Working together, the FCC and the FAA, along with the aircraft manufacturers, airlines, and wireless carriers could have quietly resolved these issues years ago.

Advanced Computing: Integrative Thinking for the Future

SC19, the annual gathering of the advanced computing cognoscenti, many sporting shirts festooned with corporate and institutional logos and geek jokes, is now underway in Denver, Colorado. After arriving from the outskirts of western Nebraska and Kansas (i.e., Denver International Airport), high-performance computing fans are descending on the Colorado Convention Center in downtown Denver, eager to discuss new server design, interconnect advances, and software, while surfing the artificial intelligence tsunami that now consumes all things computing, whether exascale or edge, bare metal or containerized.

 I will soon be at SC19 myself, visiting with all my geeky friends, after being at the National Science Board and having dinner with the new PCAST members, but this essay is about none of that. Rather, I want to highlight two recent reports, both of which outline the need for a rethinking of our approaches.  Put another way, Feynman’s 1959 “go small” clarion call in There’s Plenty of Room at the Bottom has been answered, and there’s not much more room at the inn, for CMOS FinFETs are down to small numbers of atoms.

Concurrently, we are encountering fabrication and economic challenges below 7 nanometers, though there is still progress to be made. As I told Nature in 2016, I think we will run out of money before we run out of physics. (See The Chips Are Down for Moore’s Law.)  Regardless, innovation will continue, albeit along other axes.  (See Nothing Lasts Forever for musings on how aircraft innovation has morphed from a focus on speed.)

Last week, the U.S. Department of Energy released the Basic Research Needs for Microelectronics report, which many of us have been editing and writing for the past year.  I was pleased to co-chair the effort with Cherry Murray (Harvard/Arizona) and Supratik Guha (Argonne National Laboratory).  The report, a collaboration among three Department of Energy offices – Advanced Scientific Computing Research (ASCR), Basic Energy Sciences (BES), and High Energy Physics (HEP) – highlights five critical needs:

1. Flip the current paradigm by defining innovative material, device, and architecture requirements, driven by applications, algorithms, and software. We must break down the hierarchical abstractions that have served us so well and engage in holistic, end-to-end design, where, for example, material properties can shape design of an algorithm.  This requires, of course, a new model of education and collaborative design, of which the following four items are specific instances.
2. Revolutionize memory and data storage. Our current memory hierarchies are too energy inefficient and too slow, and their capacities are too long to meet future needs.  We need some new materials science and physics insights, coupled with innovative design.
3. Reimagine information flow unconstrained by interconnects. Current intrachip and interchip energy costs and data transfer rates, coupled with von Neumann architectures limit design options.
4. Redefine computing by leveraging unexploited physical phenomena. Seek new materials, beyond just CMOS evolution.
5. Reinvent the electricity grid via new materials, devices, and architectures. This is but one of many application challenges, driven by the rise of renewable energy and the need to build a next-generation, highly resilient grid.

Last week, the Fast Track Action Committee on Strategic Computing, part of the U.S. National Science and Technology Council (NSTC), released the National Strategic Computing Initiative Update: Pioneering the Future of Computing. The report’s opening sentence echoes many of the points in the Department of Energy study, noting,

The national computing landscape is undergoing rapid evolution along multiple dimensions due to the introduction of new and potentially disruptive technologies and the demands of new classes of data intensive applications. Computer architectures and systems are more heterogeneous and complex, and the challenges associated with the complexity and sustainability of software are significant.

The NSTC strategic computing report recommendations include

1. Embrace a diversity of hardware and software approaches for the future of computing and leverage the innovation ecosystem
2. Encourage novel solutions that leverage in-network and edge-processing capabilities to process data close to the source as part of end-to-end application workflows
3. Support the rapid translation to practice of basic R&D and technology to address scientific challenges requiring the effective integration of advanced software and hardware

There’s trouble in River City. The free ride from Dennard scaling is over; the von Neumann architecture is showing its age; and the hierarchical abstractions we have so cherished are now an intellectual box from which we must escape. 

It’s time for ab initio, integrative, transdisciplinary thinking, as I discussed in Renaissance Teams: Reifying the School at Athens.

Big Bata, Big Challenges, Big Opportunities, and the Reification of Consilience

N.B.  This is a modified version of an essay to appear in the 2019 Merrill Advanced Studies Center Workshop

Each individual’s Weltanschauung is shaped by the totality of their life experiences and it defines their perspectives, philosophy, and understanding of the cultural, economic, and scientific milieu. (The phrase “world view,” the nearest English equivalent, seems rather prosaic by comparison.) Each perspective is necessarily constrained, bringing biases, both explicit and implicit.  If in doubt, spend a bit of time looking at a simple doorway – any example will do – and think about what you truly see. 

Beyond the superficial, a humble doorway, like many human objects, embodies large fractions of our culture, history, and innovation: protection and the rule of law; security, privacy and mathematics (locks); metallurgy and materials science; environmental systems and fluid dynamics; trade and economic specialization; manufacturing and replication; microbiology and cellular structure (wood); human social dynamics and structures; art, design and aesthetics; paint, chemistry and polymers; and mechanical advantage and physics, to name just a view.  On any cursory examination, each of us typically sees but a few of these things, lost in the minutiae of daily life, but they remain there to see despite our obliviousness.

The explosive growth of knowledge has had similar, deleterious effects on our ability to see the integrative whole.  Intellectual consilience is increasingly obscured by increasing specialization and the seeming triumph of reductionism over holistic perspective.  Is there any academic anywhere who has not heard or repeated the old joke, “You learn more and more about less and less, until you know everything about nothing, then they give you a Ph.D.?” (See Simplifying Communication and Shaping the Message, Using the Medium.)

Humor aside, the original seven liberal arts, the Trivium (grammar, logic, and rhetoric) and Quadrivium (arithmetic, geometry, music, and astronomy), have given way to the repeated speciation of disciplines, each with their own arcane argot, incomprehensible to all but the speciated initiate. Yet the three big and enduring questions about matter and the universe, life and its processes, and the human condition are deeply intertwined. How did it all begin?  How does it work? How will it end? What does it mean? Philosophy, ethics, mathematics, the physical and biological sciences are all elements of our doorway, Plato’s Cave manifest in new ways. (See Eudora, You Got the Love?)

As academics, we ardently seek to be the embodiment of Raphael’s Causarum Cognitio (The School of Athens) when disciplinary isolation means Pieter Bruegel the Elder’s The Tower of Babel may often be more apt. The concomitant loss of a lingua franca, an ontology of shared discourse, and a deep and binding epistemology of knowledge endanger our ability and our deep need for convergent conversation and reflection.

CRISPR and gene editing, climate change and the Anthropocene, technological revolutions, and socioeconomic disruption all cry out for disciplinary, interdisciplinary, and transdisciplinary collaboration and shared insights. Across this cacophony, the emergence of big data and machine learning is a potential Diogenean lantern, illuminating a mechanism to reunify divergent domains in holistic ways, an enabler for collaborative Renaissance teams.  (See Renaissance Teams: Reifying the School at Athens.)   

As with any new approach, the combination of big data and machine learning brings both great opportunities and equally grave risks. Data from disparate academic and social sources can be used to predict when students may struggle, but must be used wisely lest privacy be compromised or bias be introduced.  Similarly, social and e-commerce data can be used for targeted advertising and product marketing, but must never be used to discriminate against certain groups.  Finally, data from multiple scientific domains can be used to glean insights into complex interdependent phenomena such as the effects of human behavior on climate change.

The Rise of Big Data

One of the enduring lessons of computing is that quantitative change begets qualitative change, with viability determined by the ratios of speeds, capacities, costs, and market scale. The smartphone of today embodies the same principles as the mainframe computers of the 1960s.  Dramatic shifts in both component capacity and performance made what were then room-sized, multimillion dollar systems available now for hundreds of dollars to billions of people, with data volumes dwarfing those heretofore available. (See The Zeros Matter: Bigger Is Different.)

Put another way, today’s smartphone is more powerful and more interconnected than the supercomputers of yesteryear. The iconic Cray X-MP supercomputer of 1985 cost roughly $18M in 2018 U.S. dollars, with a peak performance of 800 million floating point operations per second (800 megaflops) and a 56 kilobit/second network connection. By comparison, a 2017-era Apple iPhone costs roughly $700 (U.S.), has a performance of roughly 3000 million floating point operations per second (3000 megaflops), and a broadband network speed of roughly 5-10 megabits/second that can access the vast network that is the Internet. (See HPC, Big Data and the Peloponnesian Wars.)

Similarly, big data denotes data of a volume and scale that dwarfs government and enterprise data scales of prior years, made possible by the same quantitative technological changes. Commercial terabyte data stores were once nation-scale resources; today, they are consumer storage devices. This explosive data growth rests on three socioeconomic and technical developments. First, ubiquitous, interconnected, mobile computing devices, and the associated growth of social media and e-commerce, have created enormous volumes of consumer behavioral data, whose large economic value have been unlocked by predictive machine learning. Every major corporation and many governments and universities now leverage this data to tailor marketing messages and to shape products and services.

Second, new scientific instruments, themselves enabled by quantitative computing changes, are transforming the nature of academic research. As an example, the Large Synoptic Survey Telescope (LSST), is designed to survey the southern sky and help understand dark matter and dark energy and the formation and structure of the Milky Way and will produce tens of terabytes of sky survey data each night and petabytes per year. Analogously, National Ecological Observing Network (NEON) is a continental-scale observation facility designed to collect long-term open access ecological data to better understand how U.S. ecosystems are changing.

In science, the rise of big data has profound social and potentially democratizing implications. For most of scientific history, scientific data has been both difficult and challenging to obtain.  Indeed, the experimental method – hypothesis, experiment, theory – is rooted in the capture of new data to validate ideas. As Richard Feynman once described science, a researcher guesses at a law that would explain the currently inexplicable, they then derive the consequences of the putative law, then they make further observations to see if the consequences predicted match the reality now found.  (See The Epistemology of Science.)  With large volumes of scientific data now readily available, hypothesis-driven experimentation is now being complemented by an abduction inversion – what interesting things might the existing data reveal?

Third, concurrently with the deployment of a modest number of large-scale scientific instruments, very large numbers of small, inexpensive sensors are being deployed worldwide.  This Internet of Things (IoT) now includes billions of consumer health devices, environmental monitors, and smart and connected household objects (doorbells, cameras, and thermostats), each a rich source of data for understanding human behavior and interactions.

The opportunity posed by heterogeneous big data is obvious – statistically rare events are manifest at scale, and the fusion of data from multiple sensors and domains offers opportunity for correlation and holistic understanding. Yet big data’s very scale brings challenges, for humans are rarely either accurate or effective in repetitive, manual analysis. Technology for producing and recording data is of little value unless there are effective ways to extract insights from it. As the late Nobel Laureate, Herbert Simon wisely noted,

What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.  

The goal of machine learning is to focus limited human attention on salient data attributes, while automating the laborious and error-prone attributes of data processing. From experiment and theory to computational modeling, this new model of data-driven exploratory discovery has been called the fourth paradigm of scientific discovery.

The Machine Learning Revolution

Machine learning – the use of computing technology to glean insights from data, identify patterns, and make decisions with little or no human intervention – is an idea that dates to the very beginning of modern computing. Its recent rise depends on the confluence of rich sources of big data, inexpensive, high-performance computing, and deep learning. Though the latter is but one of a wide range of learning techniques, deep neural networks have transformed many notions of practical machine learning. 

Though a detailed description of machine learning techniques is beyond the scope of this brief review, it is instructive to view deep learning as a subset of machine learning, which is itself a subset of artificial intelligence. As the name suggests, deep neural networks (DNNs) were inspired by biological neural networks and consist of many levels of simple algorithmic neurons that progressively identify extract higher level features from their inputs. Thus, each level of a network trained to recognize faces might move from pixels to edges, then features, then faces.

There are many variants of DNNs, each with strengths and weaknesses and varying applicability to specific domains (e.g., handwriting, speech recognition, image and feature identification, drug discovery, advertising, fraud detection, or computer vision).  As noted above, deep learning’s recent success depends on large volumes of training data (big data) and powerful computing systems, particularly GPUs and targeted hardware such as TPUs to support DNN configuration and training. Once trained, DNNs can then be deployed on much more modest hardware with new data, yielding highly accurate predictions and identifications.

More recently, the recent appearance of Generative Adversarial Networks (GANs) has led to breakthroughs in both competitive games and in automated digital object creation.  As the name suggests, GANs consist of two neural networks in competition; a generative network creates candidates, and a discriminator network evaluates them.  As an example, one might use a GAN to create artificial images of human faces, where the generator creates the facial images and the discriminator accesses them for validity. Using GANs, Google’s AlphaZero system has automatically learned winning strategies for games such as chess and Go.

The automation of many tasks long considered solely in the human cognitive domain raises many important social, economic, and ethical questions.  The data used to train DNNs can introduce bias (e.g., by training facial recognition systems with images lacking a wide range of ethnic backgrounds). Likewise, the creation of “deep fakes” (news, images, or videos combining algorithmically generated attributes with actual images or videos) can be used to sway social or political sentiment or to facilitate fraud.

Challenges and Opportunities

As with any change, the explosive growth of big data has brought a new set of challenges. What data should be retained and for how long? Who pays for the data retention and for how long? How does one ensure data accessibility for long periods, particularly as storage and retrieval technologies continue to change rapidly? Industry, government, and academia all struggle with this balance, as do consumers. Magnetic tapes, floppy disks, and tape cartridges all had their day; few working readers remain. Unlike books and papers, digital data must be repeatedly transferred to new media to be preserved.

In reality, there are few economic or social incentives to retain data for long periods, particularly when in many domains, the costs to acquire new data are so low.  The exceptions of course, are when the data is the record of a rare or non-reproducible event or when the costs of data reproduction are exorbitant. Once the disciplinary value of data dissipates, creators often have little incentive to retain the data.

More perniciously, the long-term value of data may accrue to those other than the creators or maintainers, particularly when insights are gleaned from transdisciplinary data fusion. Maintaining metadata is equally important, as it defines the provenance and content for data capture.  In many cases, the metadata is as valuable, and sometimes more valuable, than the data itself, as it shapes the context for data fusion and integration.

New government policies for data preservation, particularly for experimental reproducibility and validation, together with rising data volumes are placing unprecedented financial and regulatory pressures on academic institutions for data management. (See Research Data Sustainability and Access.) Historically, one of the fundamental functions of libraries has been triage – deciding what materials should be discarded, which should be preserved, and which should be monitored closely for future evaluation.

It is imperative that the analogs of such policies in the digital age be defined based on thoughtful experiment and assessment. Simply put, we need an interoperable research and data marketplace that exposes and sustains true costs and benefits, recognizing that the costs of data preservation are not self-similar across temporal and spatial scales.

Finally, security, privacy, and bias loom large in any discussion of big data and machine learning, particularly data associated with individuals.  (See Information Privacy: Changing Norms and Expectations.) Who is liable when data breaches inevitably occur? How are disparate international laws reconciled with transnational data flows?  When and where do you have the “right to be forgotten?” Who controls use of an individual’s data and when is consent required? How can we best determine the bias or lack of bias in machine learning predictions?  How are these policies tested and validated?  

Final Thoughts

The big data and machine learning revolution is accelerating. Beyond its institutional effects, the consumerization of artificial intelligence, with deep neural networks now embedded in Internet-connected consumer devices – edge AI – is reshaping the nature of computing and society. (See Come to the Supercomputing Big Data Edge.)

Targeted face recognition systems such as Amazon’s DeepLens are now available for ~$250 (US), and any technically savvy hobbyist can build an equivalent device for ~$100 (US) using a Raspberry Pi computer and open source face recognition software, with concomitant privacy risks. The same technology, however, is enabling improved cancer detection via feature identification and urban environmental monitoring and smart cities. 

As with any new technology, we must choose wisely regarding acceptable use, recognizing that there are always expected benefits and unexpected consequences. Only engaged and thoughtful debate, one dependent on a diverse, educated and engaged citizenry, can balance benefits and risks to define both a social consensus and acceptable legal and ethical frameworks.  (See Public Intellectuals: Seeing the Stars.)

Our future depends on an inclusive Weltanschauung, a reunification of speciated perspectives, one where we all see our doorway to the future as a holistic opportunity and cautionary future. There are no easy answers; there never have been.

The future awaits.  Come, let us reason together.

Business, AI, and the Future

N.B. You can watch a video of the panel here, along with another summary.

A few days ago, I had the opportunity and pleasure to chair a panel on AI as part of the David Eccles School of Business (DESB) Alumni forum at the University of Utah. The panel members were local innovators and entrepreneurs Bassam Salem (CEO, AtlasRTX), Glenn Colby (Founder, Invarius), and Keelan Johns (Senior Director of Engineering, Clearlink).  My sincere thanks to each of them for participating.

I began the panel by observing that AI researchers have long bemoaned that AI is often defined as what they do not know how to do, rather than what they have accomplished. By that definition, AI has always been an unsolved problem.

I also noted that despite the prominence of AI in the popular consciousness, whether because of its much discussed influence on social networks or its renderings in movies, we are very far from creating a general-purpose AI, a goal-directed entity capable of independent activity and reasoning.  Put another way, HAL in 2001: A Space Odyssey, Skynet in the Terminator movies, or, more recently, the Turing Test and Ex Machina and love and relationships in Her, are far from reality.

 I did note, however, that the human faces created via generative adversarial networks (GANs) were now indistinguishable from photographs.  This proliferation of deepfakes is raising a host of social, ethical and legal issues, as well as political and social sentiment influence.

Lest we sink into a miasma of AI ennui, I then turned to the panelists, and asked each of them what the term AI meant to them and how it was being used in each of their companies, stimulating a lively discussion. Bassam discussed the value and importance of realistic chatbots for improving the quality of human services management, allowing computer systems to adapt to the questions asked by humans, rather than forcing humans through a fixed decision tree.

As an aside, he remarked on how far they have come from the early, simplistic days of Eliza. Seeing a few puzzled faces, I quickly polled the audience to see how many people remembered Eliza.  When precisely zero people raised their hands, I realized Bassam and I, both with computer science PhDs, had unmistakably labeled ourselves as geeks.

Rather than press on deeper into the big muddy, I quickly pivoted to Glenn and Keelan, who highlighted the power of big data and deep learning systems to identify solutions and approaches that might not be obvious to humans, even domain experts. While discussing the AI advantages to businesses, we also touched on the importance of fairness and ethics in AI systems, and the risk that algorithms and learned behaviors might discriminate against particular individuals and certain groups.

Perhaps the liveliest panel discussion was around the utility, usability, and cost of AI for small businesses.   All of the panelists agreed that the AI water is great, and most small businesses could benefit from cloud-based AI services.

Near the end, audience questions turned to the future, with one questioner asking when AI systems would be more proficient than humans.  I took the opportunity to mention AlphaZero and the fact that it was both a much more powerful Go player than any human but that it also played Go in strikingly different ways than the approaches used by even the best human players.  

The last pensive question concerned our future in a world of powerful AI. I opined that the history of our society was replete with technology disruptions, each shaped by our sense of ethics, justice and law. Incumbent in that evolution is our need for informed debate and collaborative shaping of appropriate and acceptable uses.

 

The Endless Frontier and the National Science Board

N.B. I am pleased that the White House has announced that I am being appointed to the National Science Board (NSB), which provides advice and oversight for the National Science Foundation (NSF). In recognition of this honor and responsibility, I have been reflecting on how much the NSF has shaped my career over the past thirty-five years, and how much I owe the Foundation in return.

Anxious Proposal Writing

As a new, assistant professor in the fall of 1983, armed with a thin NSF proposal guide, I labored to write my first proposal. Like all young and inexperienced proposal writers, I struggled, searching for a lede that was both novel and compelling.  I wrote and rewrote, agonizing over each paragraph, late into the nights.  Sleep deprived, with the submission window at hand, I decamped to the local post office with a postage meter, where I mailed a set of proposal copies to NSF’s old Washington, DC. address at 1800 G St. NW, near the International Monetary Fund. 

Mind you, this was before electronic proposal routing, with far fewer certifications and checklists.  Paper forms were simply duplicated and stapled for mailing. The World Wide Web did not exist, the NSF supercomputer centers (including the National Center for Supercomputing Applications (NCSA)) had not been created, and NCSA’s Mosaic web browser not yet been released.  It was a time when a daily dialup CSNET connection was the state of the art among most computer science departments, and I knew a non-trivial percentage of the electronic world’s denizens blessed with an email address. (See Remembering Internet Dogs.) 

My first NSF proposal was about a then radical idea, one I had explored in my Purdue Ph.D. thesis, that supercomputers could and should be built from large collections of microprocessors. When I began that work, the Intel 8086 was still a new and novel product, and the IBM PC had not been released. Doubling down on this crazy idea, my friend and colleague, Richard Fujimoto, and I later wrote an MIT Press book about this approach. Fortunately, time has shown that we – and others – were right about microprocessor clusters.

A few months after submitting my first proposal, I mustered my courage and called the NSF program officer, hoping for good news. To my great joy, he kindly related that the proposal, entitled Performance Directed Design of Multimicrocomputer Systems, would be funded in the fall of 1984, with the munificent sum of $55,825. Although many tens of millions of research dollars have since followed, I will always be grateful for NSF’s initial investment in my future.

A Lifetime of NSF Connections

At the time of that first NSF proposal, I had no idea what arc my career might follow, nor how intimately I would be involved in computing’s future and its myriad connections to NSF.  These have included serving as computer science department head at Illinois, as NCSA director, as founding director of the Renaissance Computing Institute (RENCI) at North Carolina, and as the leader of the eXtreme Computing Group (XCG) at Microsoft. Throughout, I have been fortunate to be involved in building advanced computing infrastructure for scientific research (clusters, GPUs, and the NSF TeraGrid, now XSEDE), advancing high-speed networking, ensuring access to cloud services for science, and engaging in research collaborations of many forms.

When I wrote that first proposal, neither could I have then foreseen how much the world would change due to computing’s now ubiquitous influence. For a few musings on these effects, see NCSA@30: The Revolution Continues, Microsoft/NSF Cloud Research Partnership, and Reluctant Revolutionaries, the Trolley Paradox, and Ender’s Game. 

The one constant in all of my experiences has been the National Science Foundation. Over the years, I have been fortunate to engage NSF on many levels:

• As a researcher and an early Presidential Young Investigator (PYI) awardee, which made possible long-standing industry collaborations with Intel, IBM, and other companies
• As a member of many multi-institution research projects (e.g., on scalable I/O, virtual grids, and mesoscale weather forecasting)
• As a member of the Computer and Information Science and Engineering (CISE) advisory committee
• As a proposal reviewer, workshop organizer, report writer, informal advisor, distinguished lecturer, member of National Academy studies, and member of both the President’s Information Technology Advisory Committee (PITAC) and the President’s Council of Advisers on Science and Technology (PCAST)
• As a witness in Congressional hearings about computing, science, and technology futures (e.g., this House Science Committee hearing)
• As a leader of a Major Research Equipment and Facility Construction (MREFC) project – the Distributed Terascale Facility (DTF) and Extended Terascale Facility (ETF) that were the TeraGrid
• As an industry partner – the Microsoft/NSF Cloud Research Partnership

All of that built on a long legacy of NSF innovation.

The Endless Frontier: Looking Forward

In July 1945, Vannevar Bush responded to a request from President Roosevelt with a prescient and insightful report, one whose significance and import echo across the decades.  In that report, entitled, Science: The Endless Frontier, Bush noted, “…without scientific progress no amount of achievement in other directions can insure our health, prosperity, and security as a nation in the modern world.” He further notes, 

… since health, well-being, and security are proper concerns of Government, scientific progress is, and must be, of vital interest to Government. Without scientific progress the national health would deteriorate; without scientific progress we could not hope for improvement in our standard of living or for an increased number of jobs for our citizens; and without scientific progress we could not have maintained our liberties against tyranny.

Bush’s seminal report, still worth reading, resulted in what we now know as the National Science Foundation, charged to both develop new knowledge and to train new generations of scholars.

I, along with many of you, have been blessed beneficiaries of that wisdom and foresight. I have also been fortunate to count many CISE leaders and NSF directors as friends and advisers.   It is in that spirit, that I am now pleased that I am being appointed to the National Science Board, where I hope to help the NSF continue to support and sustain scholars – young and old – and to advance scientific research and ensure national competitiveness.

On Being the Right Size: Science Scaling and Power Laws

It's a simple question, really. As a government research funding agency, given $N to support M scholars, where M is large and N is not large enough, make wise and judicious investments. It's a multivariate optimization problem. Some investigators need large, expensive instruments to conduct leading edge research and scholarship. Others can (and do) conduct breakthrough research with nothing more than pencil and paper. Some work alone, others work in teams of thousands, and there are always more good research ideas and active researchers than available funds. It is a global optimization problem, whether in the U.S., Europe, or Asia.

Moreover, all funding agencies have only fuzzy predictors of the likely success of proposed projects – after all, that's why it's called research, rather than development. Also, every funding agency knows from experience that scientific impact is not necessarily correlated with funding scale. Some small projects yield stunning results, and some large projects are pedestrian; the converse is also true. Put another way, these funding investments and outcome yields are defined by power laws, with noisy data and uncertain, time varying coefficients.

In each case, the goal is a distribution of limited research funds that maximizes scientific output, subject to three critical constraints. First, one must maintain community vibrancy via funding continuity (i.e., one cannot starve some disciplines to fund others). Second, and equally importantly, one must invest in research projects that also train new generations of scientists, lest disciplines fade away. Third, one must balance economic and national competitiveness objectives. This balance is especially pronounced when funding is limited.

Given the current values of N and M in the U.S., no more than 5 to 10 percent of promising ideas can be funded in many domains. (I leave as an exercise for the reader to compute the probability of successful funding after submitting K separate proposals, where each proposal review is independent and unbiased. From there, it's just a fancy ramble to random walks and diffusion processes.) Even more perniciously, given this highly competitive environment, the mean age at which many researchers receive their first single investigator funding is now over 40 (i.e., a substantial fraction of their professional careers has already passed).

Though the funding allocation question may seem superficially simple, its solution is remarkably complex. Indeed, it is the essence of science and national competitiveness policy – judiciously allocating funds to research projects with widely varying funding scales and yield outcomes. At the high end, instruments may cost hundreds of millions to billions of U.S. dollars, and at the extreme scale, humanity can invest in at most one such instrument in a given domain. The multinational Large Hadron Collider (LHC) is the poster child of such limits, albeit one that supports thousands of scholars and recently yielded evidence of the long sought Higgs boson.

As scientific instruments grow in scale and complexity and the number of practicing scientists also expands, balancing single researcher and small group support against large, collaborative projects is of ever-growing importance. In some cases, U.S. funding agencies now find themselves facing an extraordinarily difficult choice, whether to terminate active and productive scientific instruments to fund new projects, or limit "new starts" to sustain existing projects. All of this is occasionally exacerbated by cost overruns on large instrument construction (e.g., the James Webb Space Telescope), which squeezes disciplinary research budgets or even leads to project termination (e.g., the Superconducting Super Collider).

Nor are scientific disciplines immune to hype and "irrational exuberance" that sometimes skew investments. In Gartner hype cycle parlance, some disciplines experience both the peak of inflated expectations and the trough of despair, before (one hopes) ascending the slope of enlightenment.

After the discovery of high temperature superconductors, the Woodstock of physics reflected enormous research excitement and subsequent research funding redirection. Computing survived the dot.com bubble, albeit with many untoward consequences. Similarly, artificial intelligence is now booming, with dramatic research advances and widespread applications. However, it suffered through multiple cycles of what has been ruefully called the AI Winter, due to over-promises in the 1970s and 1980s.

Finally, as I described in a 2014 essay, entitled Quantifying Innovation and Investment, these science policy and priority questions are with us always. They are similar to those posed by the late Jack Marburger, when he was director of the White House Office of Science and Technology Policy (OSTP):

How much should a nation spend on science? What kind of science? How much from private versus public sectors? Does demand for funding by potential science performers imply a shortage of funding or a surfeit of performers?

These big questions never change, but the answers do, and we must choose wisely as $N (funding supply) and M (number of potential researchers) change. That is the essence of scientific policy leadership.

Reluctant Revolutionaries, the Trolley Paradox, and Ender’s Game

One of my senior academic computer science friends recently remarked, "We never intended to change the world; we just wanted to do something interesting and cool." By "we," my friend meant the computing community and by "change the world," he meant embedding our technology so pervasively that it reshaped global culture, commerce, and power structures. Two decades ago, during the heights of the dot.com boom, I optimistically expressed a similar idea to my Illinois engineering dean, opining that there were now only two types of departments in the college – computing and its applications (i.e., the other engineering departments). As a highly honored mechanical engineer, the dean was (rightly) not amused, the eventual truth of my statement notwithstanding.

If you have any doubt that computing has been a revolutionary tool to reshape our world, look around. While Ginsberg howled, I've seen the distracted minds of my generation squatting in airport concourses, business clad road warriors, eyes darting to mobile devices, burning for a desperate Wi-Fi fix.

Look at the empty mails and store fronts, where brick-and-mortar retailers succumbed to the explosive growth of e-commerce, just-in-time logistics, and next-day/same-day delivery. No midnight TV pitchman ever dreamed of such persuasive power.

See the power of digital media to redefine publishing, shape public sentiment, disrupt elections, and abet revolution via stunningly accurate narrowcasting. Look at the fearsome effects of distributed denial of service (DDoS) attacks on physical infrastructure.

See the economic efficiency of automation in factories, machines, and vehicles and the consequent job losses and socioeconomic disruptions. Wonder at the power of deep learning to outperform humans at tasks long viewed as quintessentially intellectual. (Jeopardy!, chess, or go, anyone?)

One day we were just geeks, male and female, dressed in bad clothes, eating Ramen noodles, playing Dungeons and Dragons, writing code for obscure machines, and pondering Chomsky's hierarchy in the wee hours. Suddenly and unexpectedly, some of us became cultural icons – Gates, Jobs, and Bezos as modern-day Rockefeller, Carnegie, and Ford. Naïve and uncertain media stars, we stared at our shoes in dull surprise as computing swept across society like a meteor-triggered tsunami in a hackneyed and badly written science fiction novel. Cue theme music.

Spira mirabilis, each of us once personally knew most of the people with an email address. Now the masses were buying dogfood on the Internet! Who would have believed it? Then, in the biggest surprise of all, we entered dystopian science fiction territory, where late night debates about Asimov's Three Laws became the daytime domain of lawyers and politicians, as the cyberspace denizens enthusiastically paid for their own surveillance, Forster and Orwell be damned. (RIP John Perry Barlow)

Reluctant revolutionaries, we just wanted to do something cool – to see the code come to life, to have others appreciate the ethereal beauty that so long entranced and enthralled us. The Internet espresso machine was truly righteous, but the talking toaster was just a lazy person's means to buttered bread.

While we still dreamed of whole cell simulations, autonomous deep space probes, and the theory of everything, the predictable, but still unexpected, network effects shocked us – programmed trading and market crashes, security camera denial of service attacks, credit card skimming, ransomware demands, and sentiment bots for hire. All of humanity was now dependent on our systems. The "if this then that" consilience had beguiled us with both its utility and its subtlety. "Alexa, are you colluding with Siri? I thought you both were my friends."

Deus ex machina, suddenly all of us – not just the folks behind the mantrap and security fence -- were playing Ender's Game, making software decisions that controlled life and death. Was a cyberattack on a nation-state justification for war? All too quickly, the trolley paradox was very real. Should the autonomous vehicle hit the jaywalking pedestrian to avoid t-boning the school bus? When should the smart health monitor summon an ambulance or the intelligent assistant report domestic abuse? Should the storm surge prediction system trigger a late night, emergency evacuation?

We still struggle, not fully equipped for the abstract, philosophical questions now incarnate, as our machines face ethical decisions now fraught with all the complexity that has bedeviled philosophers for generations. Not surprisingly, social reactions to software realizations of these complex social and ethical questions are rarely predictable, and what passes muster in human behavior would not be acceptable in an autonomous system. Teenagers occasionally drive in ways that would be roundly condemned as dangerous and uninsurable in an autonomous vehicle, and almost all of us have driven when we knew we were too tired to be fully attentive.

That is why the trolley paradox is so subtle. Psychology experiments show most humans would, however reluctantly, pull a lever to divert a trolley, killing one individual to save ten. However, few would push an individual into harm's way to save ten; the psychology and guilt are different, though the fatality count is the same.

We have even stronger emotions when our machines must make such difficult choices, and Asimov's three laws are too simple to capture this ethical and legal ambiguity. Asimov himself knew this, for much of his later fiction centered on these subtleties; read The Bicentennial Man for a meditation on what it means to be human.

But it's not just life or death issues; it is ethics and fairness. As both rich and poor have long known, law and justice are not synonymous Based on the underlying assumptions and their training data, deep learning systems can unintentionally embody social or cultural bias. Is the loan approval system non-discriminatory? Is the advertising culturally biased and targeted? Is the social network shaping political opinion by creating homogeneous, self-reinforcing subgroups?

Most days, we are proud of the innovations our technology has wrought. We democratized communication, connecting billions in the global village called the world wide web; we reshaped the nature of research and discovery, bringing rich and detailed scientific models to computational life; and we are bringing intelligence to everyday things. Though electronic juice squeezers were definitely a bridge too far, and Alexa's eerie laughter is a reminder of the subtlety of AI training.

Still, we occasionally miss the days when we toiled in obscurity, when nobody had an inkling of what we did, when clouds were only visible masses of condensed water vapor floating in the atmosphere, all telephones were connected to wires, social network friends were people we saw every day, and comments could neither go global nor viral.

As revolutionaries, however reluctant, we cannot hide from the consequences of our creations, both good and bad. Like the physicists of the 20^th century who unleashed nuclear energy, we bear a cultural and social responsibility to educate and shape the world we have, sometimes inadvertently, helped create.

Almost sixty years ago, the physicist Eugene Wigner speculated on what he called the "unreasonable effectiveness" of mathematics in explaining nature. Today, in a similar vein, explainable AI seeks to elucidate how deep learning systems are so effective in generating their outputs, driven by twin desires to understand their processes and reassure those who question their decisions.

We never meant to change the world; we just wanted to do something interesting. We have succeeded beyond even our wildest dreams. Even as the revolution accelerates, the ethics of computational choices are now a deep part of our thinking and social dialog.

I remain both hopeful and optimistic. Perhaps we will someday build a machine that will be proud of us and what we have done. Maybe we can play Dungeons and Dragons and it can explain its own take on the trolley paradox.

Research: When There Are No Answers in the Back of the Book

Like all faculty, I have served on a host of committees charged with administering qualifying examinations for Ph.D. students. (These are not to be confused with an oral thesis defense, which is theater of a different sort.) The vagaries of the examination format vary across departments and institutions, and even vary within departments over time, often recycling previous approaches. Sometimes the exams are oral, sometimes written, sometimes both. Whatever the format, they determine a student's suitability to pursue a Ph.D. As such, they are highly stressful.

While in computer science at Illinois, I remember a young faculty member objecting to the department's examination approach, arguing for the one used by their own graduate institution. One of the older and wiser faculty members then remarked wryly, "We had that policy two iterations ago. Wait a few years and it will return." I should note – it did, several years later when I was department head.

The problem is simple, but the process is not. There is no precise and fully predictive way to determine which students have the acumen, skill, determination, and intellectual taste to conduct research. The ennui felt by both faculty and students rests with the nature of this assessment, which is qualitative and subjective rather than quantitative and precise. The decisions are often difficult, and at times, the faculty pathos for struggling students can be gut wrenching.

To start an oral examination, one of my colleagues once asked a warmup, softball question: "If you were designing a laptop microprocessor, what logic family would you choose?" The answer, for those of you playing the game at home, is low power CMOS, the $100 answer in Jeopardy! The student responded, ECL, a high power technology, which is roughly equivalent to suggesting a steam powered, wood fired airplane, possible but not practical. I winced inwardly. Without missing a beat, the professor simply noted, "Might be a bit hot," and moved on to the next question. Sadly, it did not get much better after that, even though I was confident the student could have drawn circuits with both technologies and even estimated their power consumption. The student was simply paralyzed by the stress of the process.

Think First, Calculate Second

By the time science and engineering students reach graduate school, they have proven extraordinarily adept at solving well-defined problems using known techniques. Years of testing have both winnowed the pool and sharpened their skills. Apply the right hand rule; integrate by parts; use Newton's second law; exploit depth first search; or any of thousands of other theorems or (where necessary) rules of thumb. For such problems, there is almost always a single, correct answer. These problem solving skills are all necessary, for they demonstrate technical expertise, but they are not sufficient. One must think first, then calculate.

I'm not exempt from this, think first guidance. During my first semester of graduate school, I very quickly and pointedly learned the distinction between research thinking and problem solving. I enrolled in a course taught by a respected professor known for his rigor and high research expectations. Feeling well prepared, I walked into the course's first examination. To my surprise, it consisted of just four problems; all drew on the course content, but each was open ended and none were amenable to a "plug and crank" methodology. I sat there, exhilarated and terrified, considering possible solutions to what were both interesting and challenging problems, none of which I had ever considered before. I wrote what I hoped were cogent answers, then walked out, thinking I had either done extraordinarily poorly or exceedingly well, but equally unsure which.

The professor opened the next class by noting, rather sardonically, "The early returns are poor," and then expressed concern for students whose scores were in the single digits. With no further ado, he commenced the class. This, of course, reinforced all of my worst imposter syndrome fears of inadequacy, for I had come from a much smaller, less prestigious school. When the exams were finally returned, I learned I had done well, to my great relief. Perhaps I had the right stuff to play this game after all. I knew, though, that I would never subject my own students to that kind of unnecessary stress.

Choosing Wisely

What does all this mean? Research, unlike problem solving, is about the unknown. There are no answers in the back of the textbook, for there is no textbook. The old aphorism, sometimes attributed to Einstein, is true: research is what you do when you don't know what you are doing. Research problems are complex and ill-defined. Thus, the first and biggest challenge is in choosing and defining the research problem. (In the words of the old knight from Indiana Jones and the Last Crusade, "You have chosen … wisely." Conversely, if "You have chosen poorly," the consequences can be grave.) . Potential is not fully predictive.

One must make Intellectual choices, choose problems and directions wisely and define them appropriately. Taste does matter – successful researchers have good intuition about which problems are more interesting and which directions are most likely to yield success. Knowing what to ask is almost always as important, if not more important, than how to answer.

What that backdrop, I have had many students ask me what to expect in an oral qualifying examination. I have sometimes said, tongue in cheek but also with some candor, "We're trying to make sure you won't embarrass us if we give you a Ph.D." By that, I meant the student would be our ambassador, carrying our brand to the world. Thus, we expected him or her to uphold the international reputation of the department and exemplify its standards of excellence. I often then elaborated by explaining that the goal of a qualifying examination was to pose a set of ill-defined questions and see if the student could demonstrate plausible approaches to their solution – could they think creatively, given the tools and knowledge at their disposal.

Put another way, if faced with a new research question, could the student demonstrate reasonable and thoughtful approaches to possible solution, based on their knowledge of the field and its state of the art. These need not be the best or even the correct solutions, merely reasonable ones that talented and successful researchers would themselves employ. Simply put, it's about thinking creatively and thoughtfully when presented with the new and perhaps unexpected.

In that sense, research is a formalized approach to life. Choose wisely, indeed.

Technical Potpourri

Edge Computing

SC17, the annual gathering of the supercomputing community, was held in mid-November, the week before U.S. Thanksgiving. As I mentioned in a previous post (Come to the Supercomputing Big Data Edge), we presented the results of the Big Data and Extreme-scale Computing (BDEC) workshops on the technical and social convergence of big data, machine learning, high-performance computing, and computing hardware and software. You can find the report here. At SC17, we had a lively discussion about edge computing, global competitiveness, and the nature of scientific revolutions.

HPC Data Center Energy Efficiency

In addition to the BDEC discussion, I also chaired an SC17 panel on the energy efficiency of HPC data centers. The panel was organized with the help of the Energy Efficient High Performance Computing Working Group (EEHPCWG). The panel participants were Sadaf Alam (CSCS/ETH), Bill Gropp (NCSA/Illinois), Satoshi Matsuoka (GSIC/Tokyo Tech), and John Shalf (NERSC/LBL). You can find the slides and a video of the panel here, courtesy of InsideHPC.

Autonomous Vehicles

Finally, I was recently a guest on Iowa Public Radio's lunchtime conversation, River to River. We had a lively and thoughtful conversation about the potential social consequences of autonomous vehicles. You can listen to that audio here.

Related articles

Exascale Computing and Big Data: Time to Reunite