TLDR:
- Amazon Web Services unveiled new AI servers with Trainium2 chips, directly competing with Nvidia’s dominance in the AI chip market
- Apple announced it’s using AWS’s custom AI chips for services like Siri and search, with plans to evaluate Trainium2 for AI model pre-training
- AWS claims their new servers offer 40% lower costs compared to Nvidia chips for some AI models
- AWS will launch a massive supercomputer using hundreds of thousands of Trainium2 chips, with AI startup Anthropic as the first customer
- Amazon plans to release their next-generation Trainium3 chip in 2024
Amazon Web Services (AWS) has unveiled a new generation of data center servers equipped with its custom-designed Trainium2 AI chips, marking a direct challenge to Nvidia’s dominance in the artificial intelligence chip market.
The announcement gained additional weight with Apple’s public confirmation that it will use these chips for various services.
The new server system, called Trn2 UltraServers, contains 64 Trainium2 chips per unit. AWS plans to connect these servers into a massive supercomputer containing hundreds of thousands of chips. AI startup Anthropic has been named as the first customer to utilize this extensive computing system.
During the AWS Reinvent conference in Las Vegas, Apple’s senior director of machine learning and AI, Benoit Dupin, revealed that Apple currently uses AWS’s custom AI chips for services including Siri, Apple Maps, and Apple Music. The company has already implemented Amazon’s Inferentia and Graviton chips for search services, resulting in a 40% improvement in efficiency.
AWS executives assert that their new offerings can deliver more computing power than what’s available from Nvidia today. According to Gadi Hutt, who leads business development for AI chips at AWS, customers can train some AI models at 40% lower cost compared to using Nvidia chips.
The partnership between Apple and AWS spans more than a decade, with Dupin noting that AWS infrastructure has proven both reliable and capable of serving Apple’s worldwide customer base. This public endorsement from Apple represents a major win for AWS as it competes with Microsoft Azure and Google Cloud for AI-related spending.
Apple has expressed interest in evaluating the Trainium2 chip for pre-training its proprietary models, with early testing suggesting up to 50% improvement in efficiency during the pre-training phase. This indicates that Amazon’s chips could be valuable not just for running AI models but also for developing new ones.
Matt Garman, AWS Chief Executive, announced that the company’s next-generation chip, Trainium3, will debut next year. The current generation of servers and the planned supercomputer are scheduled to come online in 2024, though specific dates haven’t been provided.
Both AWS and Nvidia rely on Taiwan Semiconductor Manufacturing for chip production. However, AWS reports being well-positioned from a supply chain perspective, with Hutt noting that the Trainium chips are the only component they cannot source from multiple suppliers.
The competition in the AI chip market has intensified as major tech companies develop their own custom solutions. While Nvidia maintains over 70% market share, companies like Meta Platforms, Microsoft, and Google have created their own AI chips. Meta uses its chips internally, while Amazon and Google offer their chips to paying customers.
AWS’s new servers will compete directly with Nvidia’s flagship server, which contains 72 of its latest “Blackwell” chips. Both companies offer proprietary technology for connecting these chips, though AWS claims it can connect more chips together than Nvidia’s system allows.
Apple’s AI strategy differs from many competitors, as it prioritizes on-device processing using iPhone, iPad, or Mac chips for many tasks. The company recently launched Apple Intelligence, a collection of services that can perform tasks like summarizing notifications and rewriting emails.
This fall marked Apple’s entry into generative AI products with Apple Intelligence, which will soon integrate with OpenAI’s ChatGPT. The company plans to enhance Siri’s capabilities next year, enabling more natural speech and improved app control.
The collaboration demonstrates AWS’s growing influence in the AI infrastructure space, as it attracts major tech companies while developing increasingly powerful chip solutions. AWS’s hardware innovations and partnerships indicate a shift in the AI chip market, traditionally dominated by specialized graphics processors.
Throughout 2024, the tech industry will closely watch the performance and adoption of AWS’s new AI servers and chips, as well as the development of the massive supercomputer project with Anthropic.
