In February 2022, I wrote an essay entitled American Competitiveness: IT and HPC Futures – Follow the Money and the Talent, where I outlined a series of challenges facing the state of the global information technology ecosystem. These spanned the shifting locus of innovation and economic power, from those companies we have traditionally called computing vendors (Intel/AMD, HPE, and IBM) to the hyperscalers and social media giants (Amazon, Facebook, Google, and Microsoft, along with their Asian siblings), the influence of smartphone vendors (Apple and Samsung), and the artificial intelligence juggernaut driving them all.
I also discussed the end of Dennard scaling and the slowing/end of Moore’s Law, both of which have ended the “free lunch” cycle of faster and cheaper microprocessors every two years. Then, within this framework, I outlined the technical, economic, and cultural challenges bedeviling leading edge high-performance computing, where ever larger machines deliver increasingly small fractions of the peak hardware performance. (It is often the case that complex, multidisciplinary applications achieve single digit percentages of the peak hardware performance.) I ended by discussing the sad state of U.S. STEM education and the implications of these trends for U.S. global competitiveness and national security.
As I had hoped, the essay generated a great deal of federal policy and vendor introspection, as well as considerable Internet chatter.
Based on many discussions and helpful feedback from several readers – including Doug Burger, Jim Larus, Tony Hey, and John Shalf, I asked my friends and colleagues, Dennis Gannon and Jack Dongarra (this year’s Turing Award winner), to help me formalize my blog observations in a March 2022 arXiv paper, entitled Reinventing High Performance Computing: Challenges and Opportunities. There, we expanded the high-performance computing history lesson and expounded on the rise of cloud computing and deep learning, something Dennis and I were privileged to see first-hand at Microsoft. Later in March, I participated in an insideHPC podcast, where I discussed several of our observations.
We followed this with a standing room only discussion in November at SC22, with additional panel members Dorian Arnold (Emory), Jack Dongarra, Torsten Hoefler (ETH Zurich), and Kathy Yelick (UC-Berkeley). (For those of you who attended SC22 or are ACM/IEEE members, you can view the panel discussion here.) Then in January 2023, John Shalf organized a panel discussion with Kathy, Neil Thompson, and me at the Exascale Computing Project (ECP) annual meeting, where we discussed the topic as well.
Finally, the Communications of the ACM has published an abridged version of our arXiv paper in the February 2023 issue. The latest version of the article is entitled HPC Forecast: Cloudy and Uncertain, and you can listen to Dennis, Jack, and me discuss the maxims and their implications at this link.
A Few Maxims
In both our arXiv paper and our Communications of the ACM paper, we outlined some maxims that we believe should guide any consideration of advanced computing futures.
- Maxim One: Semiconductor constraints dictate new approaches. The “free lunch” of lower cost, higher performance transistors via Dennard scaling and faster processors via Moore’s Law is at an end. (Most do not realize that the inflection point for transistor costs occurred near the 20 nanometer feature size. Since then, chip feature sizes have continued to decline, with more transistors per unit area, but the per transistor costs began to rise due to yield challenges and rising fabrication facility costs.) This minimax problem of maximizing chip yields, minimizing costs, and maximizing performance has driven the adoption of the chiplet model, mixing and matching multiple chips via interconnects such as UCIe, which opens new opportunities for scientific computing optimization and domain-specific accelerators.
- Maxim Two: End-to-end hardware/software co-design is essential. Given current semiconductor constraints, substantially increased system performance will require more intentional end-to-end co-design, from device physics to applications. (See Advanced Computing: Integrative Thinking for the Future, for a discussion of end-to-end design.) There is also an important corollary about changes to procurement models at scale.
- Maxim Three: Prototyping at scale is needed to test new ideas. Across multiple axes – semiconductors, chiplets, AI hardware, cloud innovations – the computing system is now in great flux.
- Maxim Four: The space of leading edge HPC applications is far broader now than in the past. Domains dominated by complex optimization problems and time-dependent partial differential equations will always matter, but other areas of advanced computing are of high and growing importance, enabled by advances in deep learning. The AI revolution is also driving the hardware ecosystem in new ways.
- Maxim Five. Cloud economics and deep learning have changed the supply chain ecosystem. The largest HPC systems are now dwarfed by the scale of commercial cloud infrastructure deployments.
- Maxim Six. The societal implications of technical issues really matter, and a diverse and inclusive domestic and global workforce is critical. This speaks to our challenges around workforce development and attracting and retaining talent individuals.
Chips Shortages and Globalization
While we were sounding the alarm about the technical and economic challenges in advanced computing, the public debate on what became the U.S. Chips and Science Act continued apace. Ultimately, the act appropriated money to help “on shore” a larger fraction of U.S. semiconductor manufacturing and authorized, but did not fully fund, major increases in funding for the National Science Foundation (NSF) and the Department of Energy’s (DOE) Office of Science. (See Working the Hill for some perspectives on where my research interests and public policy roles – as chair of the National Science Board, which oversees NSF, and chair DOE’s Advanced Scientific Computing Advisory Committee (ASCAC) – intersect.)
How did we find ourselves in this situation, you might ask? Simply put, via a series of locally rational and arguably, even short-term optimal decisions, companies focused on quarterly profits and shareholder value. They outsourced chip fabrication and device manufacturing, benefiting from lower offshore costs, and focused on design and integration of the chips into high margin products. Meanwhile, Intel, the dominant U.S. semiconductor fabricator, struggled with the transition to extreme ultraviolet lithography (EUV) and ceded global semiconductor leadership to Taiwan Semiconductor Manufacturing Company (TSMC). Nevertheless, all seemed well until two things upset the global supply chain – COVID-19 with disruptions in global supply chains and the rising economic tensions and national security concerns involving China. Now, economic uncertainties, declining tech sector profits, and corporate layoffs have further exacerbated the challenges.
The pressure to maximize short-term gains obscured the longer term need to control the means of production, exposing the risks of what my friend Tren Griffin calls the power of wholesale transfer pricing (i.e., the bargaining power of company A that supplies a unique product XYZ to Company B which may enable company A to take the profits of company B by increasing the wholesale price of XYZ.) Put another way, it is very attractive to be a fabless semiconductor company, but only when your semiconductor fabrication partner is both inexpensive and highly reliable.
As Warren Buffet famously said about the hidden risks some companies take, “Only when the tide goes out do you discover who’s been swimming naked.” The U.S. has been swimming naked for quite a while, dependent on international sources for key semiconductor components. If you have tried to buy anything containing embedded electronics – and almost everything these days does, from automobiles to greeting cards – then you know there has been a global chip shortage, manifest in rising product costs and even scarcity. Nor has high-performance computing been immune from these shortages. To complete construction of the Frontier supercomputer at Oak Ridge National Laboratory, the U.S. Department of Energy had to involve national security priorities to secure access to scarce chips.
How big is the U.S. dependence on Asian semiconductor foundries? As the chart shows, Taiwan has about 65 percent of the global market, with TSMC holding the largest fraction of this, followed by UMC, PSMC, and VIS. Korea (Samsung) has about 18 percent, China (SMIC and HHGrace) has about 5 percent, and the United States is in the “other” category – the final 12 percent – along with Global Foundries.
Why don’t we just build more fabrication plants, you say? We are, stimulated by Chips and Science Act subsidies, but building new facilities takes time and serious money. To put this in perspective, a state-of-the-art fabrication line for high-end microprocessors and graphics accelerators (GPUs) now costs at least $10B and is rising. Moreover, much of the chip shortage is not at the bleeding edge, but for older chip technologies (e.g., for car electronics) that are fabricated on fully depreciated semiconductor fabrication lines (e.g., 45-28 nanometer feature sizes). Vendors rarely build new lines for older technologies, they simply extract cash from existing fabrication lines, and until the recent economic downturn, there was unprecedented demand for devices normally fabricated on these lines. (The New York Times has a reasonable summary of the global semiconductor dynamics.)
National Security
Semiconductors have become more than just an economic engine; they are now critical to any country’s national security and global aspirations. Indeed, semiconductor manufacturing and supply chain control have become proxies in the ongoing battle for global hegemony. In a bit – but only a bit – of hyperbole, at the World Economic Forum event in Davos, Intel’s Pat Gelsinger recently called chips the new oil of geopolitics. The U.S. Department of Commerce recently tightened export controls around China, both to purchase and to manufacture certain high-end semiconductor chips used in military applications. (See the October 2022 Department of Commerce declaration.)
Meanwhile, the new U.S. National Security Strategy focuses on protecting semiconductor supply chains and continued advances in semiconductors, advanced computing, and next-generation communications, among others. The U.S. National Security Advisor, Jake Sullivan, has highlighted four pillars of U.S. strategy:
- Investing in the science and technology ecosystem,
- Nurturing top STEM talent,
- Protecting technology advantages, and
- Deepening and integrating alliances and partnerships.
All these goals emphasize three families of technologies, including (a) biotechnologies and biomanufacturing, (b) clean energy, and (c) computing-related technologies, including microelectronics, quantum information systems, and artificial intelligence.
Nor are these issues of concern only to the U.S. Korean leaders have raised similar concerns. Whatever one might wish to be true, it is increasingly clear the world is moving rapidly into a post-globalization era, with the increasingly likelihood of a 21st century non-kinetic Cold War, albeit one with very different dynamics than the one of the 1950s to 1980s. The global economy – and especially that of the U.S. and China – are deeply intertwined in ways unthinkable during the days of the standoff between the Warsaw Pact and NATO. Meanwhile, the Russian invasion of Ukraine has highlighted the uncertain future of Taiwan and western dependence on TSMC.
Coda
How might the U.S. respond in a positive way? The short answer is we must out-compete – invest more and continue to expand an ecosystem that attracts and retains the best and brightest talent the planet can produce, both domestic (The Missing Millions) and international. I have sought to summarize some of these objectives in another series of essays. As I wrote in Not Alone: Leaders in Focus, let’s dream big and bright about a larger, more diverse and inclusive science and engineering ecosystem that continues to be a magnet for global talent. (See For Science and Society, the Future Begins with Better Dreams.)
Comments