NVIDIA and Princeton Unveil ReplaySSM to Accelerate Hybrid-Model Inference
ReplaySSM, a joint project by NVIDIA and Princeton's Dao AI Lab, reduces state memory traffic in hybrid language models by caching recent inputs rather than full state matrices, delivering up to 1.80× kernel speedups at serving batch sizes.
NVIDIA and Princeton Introduce ReplaySSM to Accelerate Hybrid AI Model Decoding
ReplaySSM addresses state-space model bottlenecks by caching inputs instead of state, delivering up to 1.8× faster decoding and 2.1× speculative serving throughput on production models.
TileRT Achieves 469 tok/s on AMD Instinct MI355X GPUs
TileRT sets a new benchmark on AgentX, delivering 469 tok/s with GLM-5.3 on AMD Instinct MI355X GPUs, surpassing NVIDIA by 100 tok/s.
Nvidia's SWE-Serve Highlights AI Inference Testing Gaps
Nvidia's SWE-Serve benchmark reveals critical gaps between local AI tests and live inference serving, targeting real-world deployment challenges.
NVIDIA Brings Confidential AI Inference to Blackwell GPUs
NVIDIA's confidential computing delivers secure AI inference on Blackwell GPUs, retaining 96-98% performance while safeguarding sensitive data.
NVIDIA Dynamo EPD Boosts Multimodal AI Model Speed by 7x
NVIDIA’s EPD disaggregation in Dynamo accelerates multimodal AI inference by up to 7x, optimizing vision encoding, prefill, and decode stages.
NVIDIA Details GPU Sizing for AI Inference and TCO Optimization
NVIDIA explains how to size GPUs for AI inference workloads, balancing performance and TCO. Key for enterprises scaling generative AI.
OpenAI's Jalapeño Chip Outpaces Rivals in AI Inference Performance
OpenAI's Jalapeño custom AI chip achieves up to 1.9x better throughput per kilowatt and 3.6x lower latency than competing systems, redefining efficiency in AI inference.
OpenAI Unveils Jalapeño Chip with Industry-Leading AI Efficiency
OpenAI's Jalapeño chip outperforms commercial systems in AI inference efficiency, signaling a new era in custom silicon for advanced AI workloads.
NVIDIA Groq 3 LPX Achieves Full Production for Agentic AI
NVIDIA's Groq 3 LPX and Vera Rubin NVL72 enhance AI inference with faster token generation and lower costs, reshaping agentic AI infrastructure.