Moonshot AI's Kimi K3.1 Pro Deployed on AMD Instinct MI355X GPUs

Moonshot AI has announced the deployment of its latest multimodal AI model, Kimi K3.1 Pro, on AMD Instinct MI355X GPUs. The model, which features 2.8 trillion total parameters and 104 billion activated parameters, is designed to handle complex AI tasks with a one-million-token context window. This deployment leverages AMD's ATOM inference engine and vLLM for optimized performance and scalability.

Key Features of Kimi K3.1 Pro

Kimi K3.1 Pro retains the hybrid architecture of its predecessor, Kimi K3, with 69 Kimi Delta Attention (KDA) layers and 24 latent-attention layers. It introduces a sparse indexer to each latent-attention layer, enabling the selection of 2,048 tokens of historical context per query. The model operates with an 896-expert mixture-of-experts (MoE) architecture, where 16 experts are activated per token, and uses MXFP4 quantization-aware training for efficiency.

The model supports both text and image inputs, with a MoonViT-V2 vision encoder comprising 401 million parameters. Additionally, Kimi K3.1 Pro includes a DSpark speculative-decoding draft, which accelerates token generation by proposing draft tokens in parallel with the main decoding process.

Deployment on AMD Instinct MI355X GPUs

AMD's ATOM inference engine powers Kimi K3.1 Pro on eight MI355X GPUs, utilizing tensor parallelism for efficient execution. The released MXFP4 checkpoint is loaded without conversion, and the system supports both long-context and low-latency interactive workloads. During startup, memory usage peaks at approximately 199 GB per GPU out of the available 288 GB, allowing a single one-million-token request to fit on one node.

ATOM optimizations include FP8 re-quantization for attention and dense layers, sparse attention metadata handling, and decode context parallelism for concurrent long-context requests. Meanwhile, vLLM's decode path reduces token generation latency by 27%, according to AMD's internal benchmarks.

Performance and Accuracy

Kimi K3.1 Pro achieved high accuracy on Moonshot AI's Kimi Vendor Verifier benchmarks, scoring 0.871 on OCRBench and 0.991 on tool-call JSON schema validation at maximum reasoning effort. These results highlight the model's robust performance in handling complex reasoning and multimodal tasks.

Moonshot AI's deployment of Kimi K3.1 Pro on AMD Instinct GPUs underscores the scalability and efficiency of AMD's AI hardware and software ecosystem, paving the way for advanced AI applications in both research and industry.