DeepSeek’s DeepInfer Unlocks High-Margin Inference Secrets
DeepSeek open-sources its inference framework, revealing how it cuts compute costs—plus NVIDIA’s new open models for Europe.
The story of the day
DeepSeek open-sourced DeepInfer, the framework behind its high-margin AI inference. The move exposes the technical underpinnings of its cost efficiency—previously a competitive edge—letting builders replicate its low-compute serving strategy. For developers, this means actionable blueprints for squeezing margins out of frontier models. For investors, it’s a rare glimpse into how Chinese AI labs offset chip constraints with software. The trade-off: widespread adoption may erode DeepSeek’s advantage, but the PR win and ecosystem growth could outweigh it.
New tools & launches
DeepInfer — Open-source framework for high-efficiency model serving, optimized for low compute costs. [The Information]
NVIDIA Nemotron 3.5 Super VL & Bolt — Open-weight multimodal and text models tailored for European languages/industries. [NVIDIA AI Blog]
NVIDIA Kumo — Open foundation models and libraries for structured business data. [NVIDIA AI Blog]
Models & research
PolyGraph (arXiv) introduces plan-buffered CUDA graphs to adapt speculative decoding dynamically, avoiding the need to pre-capture multiple shapes. On an H800, the prototype reduces overhead to a 28-byte memory update per step—critical for serving systems juggling variable batch sizes.
Action-Conditioned World Models for EHR (arXiv) separates patient states from clinical actions in latent trajectories, enabling time-resolved predictions and multi-step rollouts. The approach preserves predictive signal and temporal structure in synthetic patient data.
Industry pulse
Europe’s AI ambitions hit a thermal wall: a German project to repurpose data center heat for homes stalled, highlighting the tension between climate goals and compute demand. [Wired] Policymakers now face a choice—subsidize green AI or accept slower growth.
Stat of the day
Southeast Asia’s digital economy is projected to reach $368B in 2026, with AI adoption as the primary driver. [Google AI Blog]
Get the full analysis — subscribe to Gradient Pro.
- DeepSeek Open-Sources Framework Behind Its High-Margin AI Inference — The Information AI
- Europe Wants Data Centers. Its Climate Goals Are Getting in The Way — Wired AI
- Southeast Asia’s digital economy to hit $368 billion — Google AI Blog
