Search results for
tensorrt-llm
NVIDIA H100 GPUs and TensorRT-LLM Achieve Breakthrough Performance for Mixtral 8x7B
NVIDIA's H100 Tensor Core GPUs and TensorRT-LLM software demonstrate record-breaking performance for the Mixtral 8x7B model, leveraging FP8 precision.
NVIDIA TensorRT-LLM Boosts Hebrew LLM Performance
NVIDIA's TensorRT-LLM and Triton Inference Server optimize performance for Hebrew large language models, overcoming unique linguistic challenges.
Enhancing Large Language Models with NVIDIA Triton and TensorRT-LLM on Kubernetes
Explore NVIDIA's methodology for optimizing large language models using Triton and TensorRT-LLM, while deploying and scaling these models efficiently in a Kubernetes environment.
NVIDIA's TensorRT-LLM Enhances AI Efficiency with KV Cache Early Reuse
NVIDIA introduces KV cache early reuse in TensorRT-LLM, significantly speeding up inference times and optimizing memory usage for AI models.