Meta unveils new generation of AI chip in challenge to Nvidia

Meta Platforms has lifted the curtain on its latest breakthrough: the next iteration of its in-house artificial intelligence accelerator chip. Meta unveiled details of the new generation of AI chips on Wednesday two months after the company announced it was building new custom AI chips to drive its AI initiatives.

According to Meta, the upcoming generation of its expansive infrastructure is meticulously engineered with AI at its core. This strategic move aims to bolster support for novel generative AI products, sophisticated recommendation systems, and cutting-edge AI research. It’s a calculated investment poised to scale in tandem with the escalating computational demands of increasingly intricate AI models.

In a recent blog post, Meta elaborated, stating, “This chip’s architecture is fundamentally focused on providing the right balance of compute, memory bandwidth, and memory capacity for serving ranking and recommendation models.”

Notably, Meta had previously unveiled its Meta Training and Inference Accelerator (MTIA) v1 last year—a pioneering AI inference accelerator engineered in-house, meticulously tailored to Meta’s unique AI workloads. Specifically honed for deep learning recommendation models, MTIA v1 has been instrumental in enhancing a myriad of user experiences across Meta’s suite of apps and technologies.

Meta added that the latest iteration of MTIA is a vital part of its comprehensive development plan for tailored silicon, specifically designed to tackle our unique tasks and systems.

“This new version of MTIA more than doubles the compute and memory bandwidth of our previous solution while maintaining our close tie-in to our workloads. It is designed to efficiently serve the ranking and recommendation models that provide high-quality recommendations to users,” Meta said in the blog post.

The chip’s architecture prioritizes striking the right balance between compute power, memory bandwidth, and capacity, specifically tailored for ranking and recommendation models.

The social giant said it had already implemented MTIA in its data centers, where it’s actively running models in real-world scenarios. Early feedback indicates promising outcomes, allowing us to allocate more computational resources to our demanding AI tasks. Initial testing demonstrates that this MTIA chip effectively handles both simple and complex ranking and recommendation models, crucial components of Meta’s products. Thanks to our end-to-end control over the entire system, we achieve higher efficiency compared to off-the-shelf GPUs.

Meanwhile, earlier reports from Reuters this year hinted at Meta’s plans to roll out a fresh iteration of its custom data center chip to cope with the surging computational demands requisite for running AI-driven products across Facebook, Instagram, and WhatsApp. Internally dubbed “Artemis,” this chip is positioned to diminish Meta’s dependence on Nvidia’s AI chips while concurrently slashing overall energy expenditure.

However, the launch has not stopped Meta from buying Nvidia chips. The company is also investing heavily in purchasing Nvidia and other AI chips. CEO Mark Zuckerberg announced this year that Meta intends to procure approximately 350,000 flagship H100 chips from Nvidia. Zuckerberg further mentioned that, when combined with chips from other suppliers, Meta aims to amass the equivalent of 600,000 H100 chips by the end of the year.