Meta unveils its newest custom AI chip as it races to catch up

The next-gen Meta Training and Inference Accelerator (MTIA) is Meta's latest custom AI chip designed to run and train AI models, specifically for tasks like ranking and recommending display ads on Meta's properties such as Facebook. This chip is the successor to the previous MTIA v1 and features improvements such as a 5nm process, a larger physical design with more processing cores, increased power consumption at 90W, more internal memory (128MB), and a higher average clock speed (1.35GHz).

The next-gen MTIA has several advancements over the MTIA v1:

• It is built on a 5nm process compared to the 7nm process of MTIA v1.
• The chip has a larger physical design with more processing cores.
• It consumes more power at 90W versus the 25W of MTIA v1.
• It has double the internal memory, with 128MB compared to 64MB.
• The average clock speed has been increased to 1.35GHz from 800MHz.

These improvements contribute to the next-gen MTIA delivering up to 3x overall better performance compared to its predecessor.

Meta is developing its own AI chips such as the next-gen MTIA for several reasons:

• To catch up to rivals in the generative AI space.
• To achieve greater efficiency in running and training AI models by controlling the whole stack, as opposed to using commercially available GPUs.
• To reduce costs associated with using third-party GPUs, as Meta is set to spend an estimated $18 billion by the end of 2024 on GPUs for training and running generative AI models.
• To explore the potential for in-house hardware as an alternative to expensive training costs for cutting-edge generative models, which can range in the tens of millions of dollars.

No, the next-gen MTIA will not replace GPUs for running or training AI models. Instead, it is intended to complement them. Meta acknowledges that while the next-gen MTIA is not currently being used for generative AI training workloads, there are several programs underway exploring this possibility. The aim is to achieve a measure of independence from third-party GPUs, but for now, GPUs will still play a role alongside Meta's custom AI chips.