Nvidia struck a $20 billion licensing deal with AI chip startup Groq, gaining access to Groq’s inference technology while hiring its CEO Jonathan Ross and key engineering talent. This isn’t a full acquisition. Groq continues operating independently under new CEO Simon Edwards.
In this post, I intended to provide an explainer about this deal for people who do not regularly follow the AI hardware industry. So if you are a regular follower of this industry, you might find this post a bit simplistic.
What Groq Does
Groq builds specialized AI chips called Language Processing Units (LPUs) designed exclusively for AI inference i.e. the stage where a trained AI model actually responds to user queries in real-time, such as whenever you chat with an AI chatbot.
To use an analogy, training an AI model is like teaching a student (computationally expensive, done once), while inference is like the student answering questions on an exam (needs to be fast and cheap, done millions of times).
Groq’s LPUs excel at delivering ultra-fast responses with lower costs by using on-chip SRAM memory that’s 10x faster than the memory in traditional GPUs. This architecture makes them ideal for applications requiring instant AI responses, like chatbots, autonomous vehicles, and real-time analytics
Does Groq compete directly with Nvidia?
There’s partial overlap but different focus areas. Nvidia’s GPUs dominate AI training, the computationally intensive process of building AI models from scratch. However, Nvidia also competes in inference, where Groq specializes exclusively.
The key difference is that Nvidia offers general-purpose GPUs that handle both training and inference, making them flexible but less optimized for inference speed. Groq’s LPUs are custom-built ASICs (application-specific chips) that sacrifice flexibility for dramatically faster inference performance at lower cost per query.
As AI shifts from “training new models” to “deploying existing models efficiently,” Groq’s inference-first approach directly challenges Nvidia’s inference business.
What does Nvidia get from this deal with Groq?
I’ll answer this in 2 parts. From a technology standpoint and from a Business standpoint.
1. Technology Benefits
Nvidia gains Groq’s revolutionary LPU architecture that uses on-chip SRAM memory instead of external HBM (High-Bandwidth Memory), delivering up to 100× faster data access and 80 TB/s memory bandwidth. This eliminates latency bottlenecks that plague traditional GPUs, achieving sub-millisecond response times for AI inference. Groq’s deterministic, single-core design avoids “wasted cycles” common in GPU scheduling, providing 2-10× faster inference performance.
Nvidia also acquires Jonathan Ross (Google’s former TPU inventor) and elite engineers who bring years of specialized low-latency chip design expertise.
This technology compresses what would have been years of internal R&D into immediate capability, filling critical gaps in Nvidia’s inference offering
2. Business Strategy (Eliminates a Rising Competitor)
Nvidia essentially neutralizes Groq before it becomes a serious threat. Groq’s LPU chips were gaining traction as a faster, cheaper alternative for AI inference, the exact market segment where Nvidia’s GPUs are being challenged. By absorbing Groq’s technology and talent, Nvidia removes a competitor that could have captured market share in the rapidly growing inference business.
Regulatory issues about the Nvidia-Groq deal
By structuring this as a licensing deal rather than a full acquisition, Nvidia avoids traditional antitrust reviews despite its already dominant 92% market share in data center GPUs. Groq remains nominally independent, continuing its GroqCloud service under new CEO Simon Edwards, which creates the appearance of ongoing competition.