NVIDIA details TensorRT Global Performance Tuner build-route design
NVIDIA explains how TensorRT 11.1’s Global Performance Tuner searches build routes, measures accuracy and selects the fastest eligible engine.
AMD says EAGLE-4 runs on Instinct MI355X GPUs from day one
AMD says EAGLE-4 speculative decoding runs on Instinct MI355X GPUs in SGLang and ATOM, with benchmark gains over plain decoding and built-in MTP.
NVIDIA and Princeton Introduce ReplaySSM to Accelerate Hybrid AI Model Decoding
ReplaySSM addresses state-space model bottlenecks by caching inputs instead of state, delivering up to 1.8× faster decoding and 2.1× speculative serving throughput on production models.
TileRT Achieves 469 tok/s on AMD Instinct MI355X GPUs
TileRT sets a new benchmark on AgentX, delivering 469 tok/s with GLM-5.3 on AMD Instinct MI355X GPUs, surpassing NVIDIA by 100 tok/s.
Nvidia's SWE-Serve Highlights AI Inference Testing Gaps
Nvidia's SWE-Serve benchmark reveals critical gaps between local AI tests and live inference serving, targeting real-world deployment challenges.
NVIDIA Brings Confidential AI Inference to Blackwell GPUs
NVIDIA's confidential computing delivers secure AI inference on Blackwell GPUs, retaining 96-98% performance while safeguarding sensitive data.
NVIDIA Dynamo EPD Boosts Multimodal AI Model Speed by 7x
NVIDIA’s EPD disaggregation in Dynamo accelerates multimodal AI inference by up to 7x, optimizing vision encoding, prefill, and decode stages.
NVIDIA Details GPU Sizing for AI Inference and TCO Optimization
NVIDIA explains how to size GPUs for AI inference workloads, balancing performance and TCO. Key for enterprises scaling generative AI.
OpenAI's Jalapeño Chip Outpaces Rivals in AI Inference Performance
OpenAI's Jalapeño custom AI chip achieves up to 1.9x better throughput per kilowatt and 3.6x lower latency than competing systems, redefining efficiency in AI inference.
OpenAI Unveils Jalapeño Chip with Industry-Leading AI Efficiency
OpenAI's Jalapeño chip outperforms commercial systems in AI inference efficiency, signaling a new era in custom silicon for advanced AI workloads.