SKIP SOFTMAX
Skip Softmax
NVIDIA Introduces Skip Softmax for Enhanced LLM Inference Efficiency
NVIDIA's Skip Softmax in TensorRT-LLM offers up to 1.4x faster inference for LLMs by optimizing attention computation, enhancing performance on Hopper and Blackwell architectures.