A promising new chip technology aimed at reducing energy usage is not yet reliable enough for inclusion in Nvidia's flagship graphics processing units (GPUs), according to Nvidia CEO Jensen Huang.
Co-packaged optics, the emerging technology, utilizes beams of laser light to transmit information via fiber optic cables between chips, offering faster connections and superior energy efficiency compared to traditional copper cables.
During a keynote address at a crowded hockey stadium, Huang announced that Nvidia would incorporate co-packaged optical technology in two new networking chips for switches atop its servers, claiming the new chips would be three and a half times more energy efficient than their predecessors. These switch chips are expected to debut later this year and into 2026, marking a significant step forward for the technology.
However, Huang informed a group of journalists afterward that while Nvidia has considered a broader application of the technology in its flagship GPU chips, there are currently no plans to do so. He explained that traditional copper connections are "orders of magnitude" more reliable than the current co-packaged optical connections. "That's not worth it," Huang stated regarding the direct use of optical connections between GPUs, emphasizing that copper remains superior.
Huang is focused on establishing a reliable product roadmap for Nvidia's customers, including Oracle. He highlighted the upcoming investment in AI infrastructure, noting, "In a couple of years, several hundred billion dollars of AI infrastructure is going to get laid down."
Silicon Valley entrepreneurs and investors are optimistic about the optics technology, believing it will play a critical role in developing larger computers for AI systems. Huang emphasized that, despite advancements by companies like DeepSeek, AI systems will require even more computing power to process complex tasks.
Startups such as Lightmatter and Celestial AI have raised substantial venture capital, some backed by Nvidia, to integrate co-packaged optical connections directly onto AI chips. Both are pursuing public offerings.
While copper connections are inexpensive and fast, they have limited data transmission distances, impacting Nvidia's product lineup over the past several years. The current flagship product incorporates 72 chips in a single server, consuming 120 kilowatts of electricity and generating significant heat, necessitating a car engine-like liquid cooling system. The flagship server unveiled for release in 2027 will accommodate hundreds of Vera Rubin Ultra Chips in a single rack and consume 600 kilowatts of power.
To fit more than double the number of chips into the same space within a two-year timeframe will require substantial engineering efforts from Nvidia and its partners. This is crucial as AI workloads involve extensive data movement between chips, and Nvidia aims to maximize the number of chips within the limited range of copper connections.
Mark Wade, CEO of Ayar Labs, which has secured venture support from Nvidia, remarked that the chip industry is still figuring out how to manufacture co-packaged optics at lower costs and with greater reliability. He indicated that the broader transition to optics might not occur until 2028 or later, yet the chip industry must eventually move away from copper to accommodate the demand for larger servers.
"Just look at the power consumption increasing on racks with electrical connections," Wade stated. "Optics is the only technology that can lead us away from that trajectory."