Tech Gadget Central Latest Tech News and Reviews
Meta revealed its in-house infrastructure for AI workloads, including generative AI like that used in its recently unveiled ad design and creation tools, at a virtual event this morning.
Meta tried to show its superiority by adopting AI-friendly hardware systems, which has slowed its progress compared to Google and Microsoft.
“Building our own [hardware] capabilities gives us control at every layer of the stack, from datacenter design to training frameworks,” Meta VP of Infrastructure Alexis Bjorlin told . Vertical integration is needed to grow AI research.
Meta has spent billions attracting top data scientists and inventing new AI, including AI that drives its discovery engines, moderation filters, and ad recommenders. The company has struggled to commercialize its more ambitious AI research advances, particularly in generative AI.
Meta will perform its AI workloads on CPUs and a proprietary processor until 2022, which are less efficient than GPUs. Meta cancelled its 2022 bespoke chip launch and ordered billions of dollars’ worth of Nvidia GPUs, which needed substantial data center redesigns.
Meta began creating a more ambitious in-house chip in 2025 that could train and run AI algorithms to turn things around. The presentation focused on that.
Meta calls the new chip the Meta Training and Inference Accelerator (MTIA), part of a “family” of processors that accelerate AI training and inference workloads. “Inferencing” means running a trained model. An ASIC, the MTIA, can be designed to perform many functions simultaneously.
“To improve efficiency and performance across our important workloads, we needed a tailored solution that’s co-designed with the model, software stack, and system hardware,” Bjorlin said. “This improves user experience across multiple services.”
Big Tech companies increasingly favor custom AI CPUs. The Google TPU (tensor processing unit) trains big generative AI systems like PaLM-2 and Imagen. Amazon’s Trainium and Inferentia chips are available to AWS customers. Microsoft is apparently developing Athena, an in-house AI chip, with AMD.
Meta produced MTIA v1 in 2020 on a 7-nanometer technology. In a Meta-designed benchmark test, the MTIA handled “low-complexity” and “medium-complexity” AI models more efficiently than a GPU. It can expand from 128 MB to 128 GB.
Meta believes the chip’s memory and networking still need work as AI models evolve, necessitating workloads to be spread over numerous processors. Meta purchased Graphcore’s Oslo-based AI networking team. The MTIA now only infers “recommendation workloads” across Meta’s app family.
Meta said that the MTIA, which it continues to modify, “greatly” improves performance per watt when executing recommendation tasks, allowing it to execute “more enhanced” and “cutting-edge” AI workloads.
AI supercomputer
Meta may eventually outsource its AI workloads to MTIA banks. The Research SuperCluster (RSC) GPUs are currently powering the social network.
The Penguin Computing, Nvidia, and Pure Storage-built RSC, revealed in January 2022, has finished its second phase. Meta reports 2,000 Nvidia DGX A100 systems with 16,000 GPUs.
Why develop an in-house supercomputer? First, peer pressure. Microsoft announced a cooperation with Nvidia to construct a new AI supercomputer in the Azure cloud several years after announcing its OpenAI-built AI supercomputer. Google’s AI supercomputer, with 26,000 Nvidia H100 GPUs, outperforms Meta’s.
Meta claims the RSC lets its researchers train models using real-world instances from its production systems. The company’s old AI infrastructure used only open-source and public datasets.
A Meta spokeswoman stated the RSC AI supercomputer pushes AI research in various disciplines, including generative AI. “AI research productivity matters. We intended to give AI researchers state-of-the-art technology to construct models and a training environment to progress AI.”
The RSC can reach approximately 5 exaflops of computational power, according to the manufacturer, making it one of the fastest. (Some experts doubt the exaflops performance metric, and the RSC is much outclassed by many of the world’s fastest supercomputers.)
Meta utilized the RSC to train LLaMA, a twisted acronym for “Large Language Model Meta AI,” a huge language model that the business released as a “gated release” to researchers earlier this year (and which leaked in various online communities). Meta says 2,048 A100 GPUs trained the largest LLaMA model in 21 days.
“Building our own supercomputing capabilities gives us control at every layer of the stack—from datacenter design to training frameworks,” the representative said. “RSC will help Meta’s AI researchers build new and better AI models that can learn from trillions of examples, work across hundreds of languages, seamlessly analyze text, images, and video, develop new augmented reality tools, and much more.”
Video encoder
Meta announced today that it is creating another chip to handle specific computing workloads in addition to MTIA. Meta’s first in-house ASIC solution for video on demand and live streaming is the Meta Scalable Video Processor (MSVP).
Readers may recall that Meta announced a video transcoding and inferencing ASIC in 2019. Some of those efforts and a fresh push for live video’s competitive advantage resulted in this.
In a blog post published this morning, Meta technical lead managers Harikrishna Reddy and Yunqing Chen stated, “On Facebook alone, people spend 50% of their time on the app watching video.” To serve the wide variety of devices worldwide (mobile devices, laptops, TVs, etc.), videos uploaded to Facebook or Instagram, for example, are transcoded into multiple bitstreams with different encoding formats, resolutions, and quality. MSVP is programmable and scalable, so it can efficiently support both high-quality transcoding for VOD and low latency and faster processing times for live streaming.”
Meta plans to offload most of its “stable and mature” video processing tasks to the MSVP and use software video encoding exclusively for applications that demand customization and “significantly” greater quality. Meta states that MSVP video quality is being improved through clever denoising, image improvement, artifact removal, and super-resolution.
Reddy and Chen said MSVP will enable efficient distribution of generative AI, AR/VR, and other metaverse content.
AI focus
Meta is desperately trying to accelerate generative AI with today’s hardware announcements.
Previously telegraphed. Mark Zuckerberg, who has prioritized increasing Meta’s AI compute power, launched a new top-level generative AI team in February to “turbocharge” R&D. CTO Andrew Bosworth recently stated that Zuckerberg and he were focusing on generative AI. Meta chief scientist Yann LeCun aims to use generative AI technologies to make virtual reality goods.
“We’re exploring chat experiences in WhatsApp and Messenger, visual creation tools for posts in Facebook and Instagram and ads, and over time video and multi-modal experiences as well,” Zuckerberg said during Meta’s Q1 earnings call in April. I expect these technologies to benefit everyone, from average people to creators to businesses. Once we master that experience, AI bots for business messaging and customer assistance will be popular. Over time, this will extend to our metaverse efforts, making avatars, items, worlds, and programming to connect them easier to develop.
Investors are concerned that Meta isn’t moving fast enough to capture the (possibly enormous) generative AI industry. It can’t match Bard, Bing Chat, or ChatGPT yet. Image generation, another fast-growing area, hasn’t improved either.
The generative AI software market might exceed $150 billion if estimates are correct. Goldman Sachs anticipates 7% GDP growth.
Meta wasted billions investing in “metaverse” technology, including augmented reality glasses, meeting software, and VR playgrounds like Horizon Worlds. Reality Labs, Meta’s augmented reality branch, lost $4 billion last quarter and expects “operating losses to increase year over year in 2023,” according to its Q1 call.
