Zyphra trains first large-scale MoE model on AMD hardware

SANTA CLARA, Calif. - Zyphra has developed ZAYA1, the first large-scale Mixture-of-Experts (MoE) foundation model trained entirely on AMD hardware, according to a press release from AMD (NASDAQ:AMD), a prominent player in the Semiconductors industry with a market capitalization of $331.76 billion.

The model was trained using AMD Instinct MI300X GPUs, AMD Pensando networking, and the AMD ROCm open software stack, as detailed in a technical report published by Zyphra on Monday.

According to benchmark results, ZAYA1-Base, which has 8.3 billion total parameters with only 760 million active at once, matches or exceeds the performance of several prominent models across reasoning, mathematics, and coding tasks. These include Qwen3-4B from Alibaba, Gemma3-12B from Google, Llama-3-8B from Meta, and OLMoE.While AMD stock has taken a significant hit of 15.28% over the past week, InvestingPro data shows impressive six-month returns of 84.73%, reflecting investor confidence in AMD’s AI strategy. Subscribers can access 20+ additional ProTips and comprehensive analysis in AMD’s Pro Research Report.

The 192 GB high-bandwidth memory capacity of the AMD Instinct MI300X GPUs allowed Zyphra to avoid expert or tensor sharding during training, which the company claims reduced complexity and improved throughput. Zyphra also reported achieving more than 10 times faster model save times using AMD optimized distributed I/O.

"Efficiency has always been a core guiding principle at Zyphra. It shapes how we design model architectures, develop algorithms for training and inference, and choose the hardware with the best price-performance," said Krithik Puthalath, CEO of Zyphra.

For the training infrastructure, Zyphra collaborated with AMD and IBM to deploy a large-scale cluster combining AMD Instinct MI300X GPUs with IBM Cloud’s fabric and storage architecture.

The development demonstrates the capabilities of AMD hardware for production-scale AI workloads, according to Emad Barsoum, corporate vice president of AI and engineering at AMD’s Artificial Intelligence Group.AMD is well-positioned to capitalize on AI growth with analysts forecasting 32% revenue growth this year. The company operates with moderate debt levels and maintains strong liquidity with a current ratio of 2.31. According to InvestingPro’s Fair Value assessment, AMD appears to be trading near its fair value despite its high P/E ratio of 108. Discover AMD’s complete financial health profile and growth prospects in the comprehensive Pro Research Report, available exclusively to subscribers.

In other recent news, Advanced Micro Devices (AMD) has been actively involved in several significant developments. AMD and Eviden, part of the Atos Group, have been chosen to construct Alice Recoque, a next-generation supercomputer, which will be France’s first and Europe’s second exascale system. This €544 million project is a collaborative effort funded by EuroHPC JU, with contributions from several European entities. On the financial side, TD Cowen has reiterated a Buy rating for AMD, maintaining a price target of $290, citing the company’s potential in the growing artificial intelligence market. Similarly, Truist Securities has also reaffirmed its Buy rating with a $279 price target, emphasizing AMD’s potential earnings growth and its role as a trusted partner in the data center and AI sectors. Raymond James has assumed coverage of AMD with an Outperform rating and a price target of $337, acknowledging the company’s leadership in CPUs, APUs, and GPUs. Additionally, Super Micro Computer Inc has introduced a new AI-accelerated server powered by AMD Instinct MI355X GPUs, promising significant improvements in AI computing and inferencing performance. These developments reflect AMD’s strategic advancements in technology and partnerships, which are drawing attention from investment firms.

This article was generated with the support of AI and reviewed by an editor. For more information see our T&C.