Google Expands Eighth-Generation TPU Portfolio With 8tx and 8i Max Chips for Secure AI Inference, High Compute Density

Google introduced the 8tx and 8i Max chips on Wednesday as its two new eighth-generation Tensor Processing Unit (TPU) systems. With the unveiling of the two new TPUs, the Mountain View-based tech giant now has four eighth-generation chips, including TPUs 8t and 8i. The company first introduced its eighth-generation range of TPUs at Google Cloud Next in April, initially including two chips. Google further highlighted that the launch of the two TPUs marks the first time it has introduced four systems in a single year. The new TPU 8tx and TPU 8i Max are designed for confidential computing and large context and density computing, respectively.

Google Introduces TPU 8tx and TPU 8i Max for Confidential AI Inference

In a blog post, Google introduced its two new specialised TPU systems, namely TPU 8tx and TPU 8i Max. The TPU 8tx is designed and optimised for secure and confidential AI inference. The company claims that the chip can also be used for small-scale training of AI models. Google's TPU 8tx is built for regulated enterprises and governments, who want to run frontier or proprietary AI models and autonomous AI agents, while ensuring the security of their sensitive data.

The TPU 8tx has been paired with the fifth-generation AMD EPYC CPU hosts. It is built on the same silicon as the 8t model, launched earlier this year. Google claims that TPU 8tx builds security directly into the hardware to secure and isolate sensitive data and model weights during active use. Further, it features the low-latency Boardfly network for real-time reasoning, delivering up to 2x lower latency than TPU 8t “with a near-zero performance penalty”. The TPU also allows customers to fine-tune AI models using private data, distill large AI models into smaller models, and train AI agents through reinforcement learning.

On the other hand, the TPU 8i Max system is built for “massive” context windows and the “highest compute” density per rack. It is claimed to scale compute and memory in lockstep, which enables 2x compute, SRAM bandwidth, and HBM bandwidth. The TPU system is claimed to fit 2x the silicon within existing racks, allowing data centres to “deploy significantly more computing power per square foot and per watt”.

The tech giant co-designed the new 8i Max and 8tx TPUs, like the previous systems, with Google DeepMind. However, the company claims that feedback and inputs from external organisations, such as “some highly” regulated and cloud customers, were also taken into consideration.