TRANSFORMER ENGINE
Transformer Engine
Nvidia's New MoE Kernels Promise 93% Speedup for AI Training
Nvidia unveils advanced MoE training kernels, boosting AI model throughput by up to 93% in GPT pre-training and redefining large-scale efficiency.