Search Results for "tensorrt"
NVIDIA's Breakthrough: 4x Faster Inference in Math Problem Solving with Advanced Techniques
NVIDIA achieves a 4x faster inference in solving complex math problems using NeMo-Skills, TensorRT-LLM, and ReDrafter, optimizing large language models for efficient scaling.
NVIDIA Introduces Skip Softmax for Enhanced LLM Inference Efficiency
NVIDIA's Skip Softmax in TensorRT-LLM offers up to 1.4x faster inference for LLMs by optimizing attention computation, enhancing performance on Hopper and Blackwell architectures.
StreamingLLM Breakthrough: Handling Over 4 Million Tokens with 22.2x Inference Speedup
SwiftInfer, leveraging StreamingLLM's groundbreaking technology, significantly enhances large language model inference, enabling efficient handling of over 4 million tokens in multi-round conversations with a 22.2x speedup.
