An AI method has accelerated and optimized chip design, and its superhuman chip layouts are used in hardware around the world.
In 2020, a preprint introduced a novel reinforcement learning method for designing chip layouts. This method was later published in Nature and open-sourced.
A Nature addendum is being published, detailing the method and its impact on chip design. A pre-trained checkpoint is also being released, sharing the model weights and announcing its name: AlphaChip.
Computer chips have fueled remarkable progress in artificial intelligence (AI), and AlphaChip returns the favor by using AI to accelerate and optimize chip design. The method has been used to design superhuman chip layouts in the last three generations of Google’s custom AI accelerator, the Tensor Processing Unit (TPU).
AlphaChip was one of the first reinforcement learning approaches used to solve a real-world engineering problem. It generates superhuman or comparable chip layouts in hours, rather than taking weeks or months of human effort, and its layouts are used in chips all over the world, from data centers to mobile phones.
AlphaChip’s groundbreaking AI approach revolutionizes a key phase of chip design.
How AlphaChip works
Designing a chip layout is not a simple task. Computer chips consist of many interconnected blocks, with layers of circuit components, all connected by incredibly thin wires. There are also many complex and intertwined design constraints that all have to be met simultaneously. Due to its sheer complexity, chip designers have struggled to automate the chip floorplanning process for over sixty years.
Similar to AlphaGo and AlphaZero, which learned to master games like Go, chess, and shogi, AlphaChip was developed to approach chip floorplanning as a type of game.
Starting from a blank grid, AlphaChip places one circuit component at a time until all components are positioned. It is then rewarded based on the quality of the final layout. A novel “edge-based” graph neural network enables AlphaChip to learn relationships between interconnected chip components and generalize across different chips, allowing it to improve with each layout designed.
Left: Animation showing AlphaChip placing the open-source, Ariane RISC-V CPU, with no prior experience. Right: Animation showing AlphaChip placing the same block after having practiced on 20 TPU-related designs.
Using AI to design Google’s AI accelerator chips
AlphaChip has generated superhuman chip layouts used in every generation of Google’s TPU since its publication in 2020. These chips make it possible to massively scale-up AI models based on Google’s Transformer architecture.
TPUs are central to powerful generative AI systems, including large language models like Gemini, and image and video generators such as Imagen and Veo. These AI accelerators are also fundamental to Google’s AI services and are available to external users through Google Cloud.
A row of Cloud TPU v5p AI accelerator supercomputers in a Google data center.
To design TPU layouts, AlphaChip first practices on a diverse range of chip blocks from previous generations, such as on-chip and inter-chip network blocks, memory controllers, and data transport buffers. This process is called pre-training. AlphaChip then operates on current TPU blocks to generate high-quality layouts. Unlike prior approaches, AlphaChip improves in speed and quality as it solves more instances of the chip placement task, similar to human experts.
With each new generation of TPU, including the latest Trillium (6th generation), AlphaChip has designed improved chip layouts and contributed more to the overall floorplan, accelerating the design cycle and resulting in higher-performance chips.
Bar graph showing the number of AlphaChip designed chip blocks across three generations of Google’s Tensor Processing Units (TPU), including v5e, v5p and Trillium.
Bar graph showing AlphaChip’s average wirelength reduction across three generations of Google’s Tensor Processing Units (TPUs), compared to placements generated by the TPU physical design team.
AlphaChip’s broader impact
AlphaChip’s impact is evident through its applications across Alphabet, the research community, and the chip design industry. Beyond designing specialized AI accelerators like TPUs, AlphaChip has generated layouts for other chips across Alphabet, such as Google Axion Processors, the company’s first Arm-based general-purpose data center CPUs.
External organizations are also adopting and building on AlphaChip. For example, MediaTek, one of the top chip design companies in the world, extended AlphaChip to accelerate development of their most advanced chips while improving power, performance, and chip area.
AlphaChip has triggered an explosion of work on AI for chip design and has been extended to other critical stages of chip design, such as logic synthesis and macro selection.
AlphaChip has inspired an entirely new line of research on reinforcement learning for chip design, cutting across the design flow from logic synthesis to floorplanning, timing optimization and beyond.
Creating the chips of the future
AlphaChip is believed to have the potential to optimize every stage of the chip design cycle, from computer architecture to manufacturing, and to transform chip design for custom hardware found in everyday devices such as smartphones, medical equipment, agricultural sensors, and more.
Future versions of AlphaChip are currently in development, with efforts focused on collaborating with the community to further revolutionize this area and achieve a future where chips are even faster, cheaper, and more power-efficient.
