Transformer Engine
Transformer Engine
NVIDIA Enhances MoE Training for Biological Models with Efficiency Gains
NVIDIA's BioNeMo MoE recipe optimizes training for biological foundation models, achieving 2.21x throughput gains using advanced Transformer Engine techniques.
Transformer Engine
NVIDIA's Transformer Engine Boosts MoE Training in JAX by 10x
NVIDIA's Transformer Engine accelerates Dropless Mixture-of-Experts (MoE) training in JAX, achieving a 10x performance gain and 97% scaling efficiency.
Transformer Engine
Nvidia's New MoE Kernels Promise 93% Speedup for AI Training
Nvidia unveils advanced MoE training kernels, boosting AI model throughput by up to 93% in GPT pre-training and redefining large-scale efficiency.