Together AI Launches Cost-Efficient Batch API for LLM Requests
Together AI introduces a Batch API that reduces costs by 50% for processing large language model requests. The service offers scalable, asynchronous processing for non-urgent workloads.
NVIDIA Introduces High-Performance FlashInfer for Efficient LLM Inference
NVIDIA's FlashInfer enhances LLM inference speed and developer velocity with optimized compute kernels, offering a customizable library for efficient LLM serving engines.
Optimizing LLM Inference Costs: A Comprehensive Guide
Explore strategies for benchmarking large language model (LLM) inference costs, enabling smarter scaling and deployment in the AI landscape, as detailed by NVIDIA's latest insights.
Understanding the Emergence of Context Engineering in AI Systems
Discover the rise of context engineering, a crucial component in AI systems that ensures effective communication and functionality for large language models (LLMs).
Enhancing LLM Workflows with NVIDIA NeMo-Skills
NVIDIA's NeMo-Skills library offers seamless integration for improving LLM workflows, addressing challenges in synthetic data generation, model training, and evaluation.
Optimizing LLM Inference with TensorRT: A Comprehensive Guide
Explore how TensorRT-LLM enhances large language model inference by optimizing performance through benchmarking and tuning, offering developers a robust toolset for efficient deployment.
Together AI Introduces Flexible Benchmarking for LLMs
Together AI unveils Together Evaluations, a framework for benchmarking large language models using open-source models as judges, offering customizable insights into model performance.
NVIDIA's NeMo Framework Enables Weekend Training of Reasoning-Capable LLMs
NVIDIA introduces an efficient method to train reasoning-capable language models over a weekend using the NeMo framework, leveraging the Llama Nemotron dataset and LoRA adapters.
NVIDIA's ProRL v2 Advances LLM Reinforcement Learning with Extended Training
NVIDIA unveils ProRL v2, a significant leap in reinforcement learning for large language models (LLMs), enhancing performance through extended training and innovative algorithms.
Enhancing LLM Inference with CPU-GPU Memory Sharing
NVIDIA introduces a unified memory architecture to optimize large language model inference, addressing memory constraints and improving performance.
NVIDIA's Run:ai Model Streamer Enhances LLM Inference Speed
NVIDIA introduces the Run:ai Model Streamer, significantly reducing cold start latency for large language models in GPU environments, enhancing user experience and scalability.
Enhancing LLM Inference with NVIDIA Run:ai and Dynamo Integration
NVIDIA's Run:ai v2.23 integrates with Dynamo to address large language model inference challenges, offering gang scheduling and topology-aware placement for efficient, scalable deployments.