AI INFERENCE
NVIDIA TensorRT Brings FP8 Quantization to AI Deployment
NVIDIA TensorRT optimizes AI inference with FP8 quantization, offering faster performance and smaller models for scalable deployment.
MiniMax-M3 Launches 1M-Token Model With Sparse Attention
MiniMax-M3 debuts with 1M-token context and multimodality, leveraging Together AI's optimizations for efficient large-scale inference.
NVIDIA Dynamo Snapshot Tackles Kubernetes AI Cold-Start Problem
NVIDIA's Dynamo Snapshot reduces Kubernetes AI inference cold-start times, leveraging CRIU and GPU Memory Service for sub-5-second deployment speed.
Together AI Joins Pearl Labs to Cut AI Inference Costs With Blockchain
Together AI partners with Pearl Research Labs to slash AI inference costs using Proof of Useful Work, generating crypto rewards for GPU workloads.
DeepSeek-V4 Tackles Million-Token Context on NVIDIA HGX B200
DeepSeek-V4 introduces a 1M-token context window with a hybrid attention architecture, shifting the challenge to inference systems on NVIDIA hardware.
Mamba-3 SSM Drops With Inference-First Design Beating Transformers at Decode
Together.ai releases Mamba-3, an open-source state space model built for inference that outperforms Mamba-2 and matches Transformer decode speeds at 16K sequences.
NVIDIA Unveils Groq 3 LPX Rack System for Ultra-Low Latency AI Inference
NVIDIA's new Groq 3 LPX delivers 315 PFLOPS and 35x better inference throughput per megawatt, targeting agentic AI workloads on the Vera Rubin platform.
NVIDIA Blackwell Smashes Finance AI Benchmark With 3.2x Speed Gains
NVIDIA's GB200 NVL72 sets new STAC-AI record for LLM inference in financial trading, delivering up to 3.2x performance over Hopper architecture.
NVIDIA Blackwell Delivers 4x Inference Boost for India's Sarvam AI Models
NVIDIA's hardware-software co-design achieves 4x inference speedup for Sarvam AI's 30B parameter sovereign models, showcasing Blackwell's NVFP4 capabilities.
NVIDIA TensorRT for RTX Brings Self-Optimizing AI to Consumer GPUs
NVIDIA's TensorRT for RTX introduces adaptive inference that automatically optimizes AI workloads at runtime, delivering 1.32x performance gains on RTX 5090.
NVIDIA Achieves 10x AI Image Generation Speedup on Blackwell Data Center GPUs
NVIDIA's new NVFP4 optimizations deliver 10.2x faster FLUX.2 inference on Blackwell B200 GPUs versus H200, with near-linear multi-GPU scaling.
NVIDIA Grove Simplifies AI Inference on Kubernetes
NVIDIA introduces Grove, a Kubernetes API that streamlines complex AI inference workloads, enhancing scalability and orchestration of multi-component systems.
NVIDIA Enhances AI Inference with Dynamo and Kubernetes Integration
NVIDIA's Dynamo platform now integrates with Kubernetes to streamline AI inference management, offering improved performance and reduced costs for data centers, according to NVIDIA's latest updates.
NVIDIA Dynamo Tackles KV Cache Bottlenecks in AI Inference
NVIDIA Dynamo introduces KV Cache offloading to address memory bottlenecks in AI inference, enhancing efficiency and reducing costs for large language models.
Reducing AI Inference Latency with Speculative Decoding
Explore how speculative decoding techniques, including EAGLE-3, reduce latency and enhance efficiency in AI inference, optimizing large language model performance on NVIDIA GPUs.