Search results for "llm"

Ray Serve LLM Enhances Distributed Inference with 24x Boost

Ray Serve LLM Enhances Distributed Inference with 24x Boost

Ray Serve LLM achieves 24x higher throughput with new direct streaming, HAProxy integration, and vLLM backend upgrades, pushing LLM inference forward.

ParallelKernelBench Exposes LLM Weakness in Multi-GPU Kernels

ParallelKernelBench Exposes LLM Weakness in Multi-GPU Kernels

ParallelKernelBench shows GPT-5.5 and peers struggle with multi-GPU CUDA kernels, solving less than 31% of tasks. Here's why it matters.

NVIDIA Pushes Hardware-Aware LLM Co-Design for AI Efficiency

NVIDIA Pushes Hardware-Aware LLM Co-Design for AI Efficiency

NVIDIA highlights hardware-friendly AI model design to optimize LLM performance. Learn how co-design boosts throughput, latency, and cost-efficiency.

NVIDIA Optimizes JAX LLM Training with Host Offloading

NVIDIA Optimizes JAX LLM Training with Host Offloading

NVIDIA's host offloading for JAX LLM training boosts GPU memory efficiency, enabling larger batch sizes and faster throughput.

Together AI Unveils Advanced Autoscaling for LLM Inference

Together AI Unveils Advanced Autoscaling for LLM Inference

Together AI introduces autoscaling features tailored for large language models, optimizing GPU use and managing latency during traffic spikes.

AMD Silo AI Expands Poro 2 Models with Enhanced Context and Math Capabilities

AMD Silo AI Expands Poro 2 Models with Enhanced Context and Math Capabilities

AMD Silo AI releases Poro 2 8B Long with a 16x larger context window and Poro 2 8B Math Reasoning for multilingual AI development.

NVIDIA NeMo Switchyard Enables Smarter AI Model Routing

NVIDIA NeMo Switchyard Enables Smarter AI Model Routing

NVIDIA NeMo Switchyard optimizes AI workflows by intelligently routing tasks across models, balancing cost, latency, and accuracy.

NVIDIA Dynamo's Shadow Engine Slashes LLM Recovery to 7 Seconds

NVIDIA Dynamo's Shadow Engine Slashes LLM Recovery to 7 Seconds

NVIDIA's Shadow Engine Recovery in Dynamo cuts LLM downtime from minutes to 7 seconds, transforming AI infrastructure resilience.

Evaluating LLMs for Production: Lessons from GitHub's Secret Scanning

Evaluating LLMs for Production: Lessons from GitHub's Secret Scanning

GitHub shares best practices for evaluating LLMs before production, focusing on precision, recall, and real-world workflows to improve security tools.

Best LLM for transcript summaries depends on call length

Best LLM for transcript summaries depends on call length

AssemblyAI says transcript summarization quality varies by voicemail, short calls, support calls and meetings, with cost and latency trade-offs.

AssemblyAI benchmark ranks Gemini 4 Argon first for dictation cleanup

AssemblyAI benchmark ranks Gemini 4 Argon first for dictation cleanup

AssemblyAI’s Oct. 28 bench says Gemini 4 Argon led dictation cleanup, with speed and cost trade-offs varying by length.

AssemblyAI details Voice Standard Bench picks for transcript correction

AssemblyAI details Voice Standard Bench picks for transcript correction

AssemblyAI says Voice Standard Bench ranks LLMs for transcript correction across four lengths, balancing faithfulness, readability, speed and cost.