TileRT Achieves 469 tok/s on AMD Instinct MI355X GPUs
Alvin Lang Sep 24, 2026 18:02
TileRT sets a new benchmark on AgentX, delivering 469 tok/s with GLM-5.3 on AMD Instinct MI355X GPUs, surpassing NVIDIA by 100 tok/s.
TileRT has reached a new milestone in AI inference performance, achieving 469 tokens per second (tok/s) for single-user generation throughput on the AgentX benchmark. This result was delivered using GLM-5.3 on 8 AMD Instinct MI355X GPUs and outpaced NVIDIA’s GB300 NVL72 system by over 100 tok/s, according to data published on September 24, 2026.
The AgentX benchmark, developed by SemiAnalysis, focuses on real-world, long-horizon agent workloads that reflect the operational dynamics of coding agents. Unlike traditional benchmarks constrained by fixed prompt lengths, AgentX tests how systems handle multi-turn interactions, growing contexts, and dependencies between agent components over extended sessions.
Why the 469 tok/s Milestone Matters
TileRT’s performance stands out not only for its raw speed but also for its ability to maintain efficiency over long contexts. Testing showed that when input context expanded from 1,000 to 1 million tokens—a 1,000x increase—TileRT still retained about two-thirds of its short-context performance, delivering 425 tok/s at the maximum context length. This sustained throughput is critical in agentic AI applications, where tasks often involve dynamically growing contexts and interactive responses.
The TileRT team attributes these results to its optimization strategies tailored for AMD’s CDNA 4 architecture, which powers the MI355X GPUs. Key innovations include:
- Streaming synchronization to reduce latency by overlapping computation and communication.
- Persistent engine kernels that eliminate the overhead of kernel-by-kernel execution.
- A fused communication-compute model that minimizes synchronization bottlenecks during data transfer.
These innovations align particularly well with AMD’s hardware capabilities, such as larger register files, fine-grained memory control, and instruction-level optimization through inline assembly.
Implications for AI and Inference Markets
TileRT’s advancement underscores the growing importance of optimizing inference speed in AI deployments. The ability to sustain high performance over long-context workloads positions TileRT as a strong contender in latency-sensitive applications like AI-assisted coding, high-frequency trading, and real-time decision-making. This also reflects a broader industry trend: as AI systems scale, inference becomes as critical as training efficiency.
For AMD, the success of TileRT on Instinct MI355X GPUs validates its efforts to compete with NVIDIA in the high-performance AI hardware market. The MI355X’s architectural features, such as its partitioned caches and enhanced register capacity, have proven instrumental in enabling TileRT’s innovations. This could translate to increased adoption of AMD hardware in specialized AI workloads—an area traditionally dominated by NVIDIA.
What’s Next for TileRT?
Looking ahead, the TileRT team plans to expand its support for AMD’s ROCm software ecosystem and introduce optimizations for larger batch sizes. These updates aim to address broader workload demands while maintaining its focus on ultra-low-latency inference. Additionally, its ongoing collaboration with AMD signals further advancements in inference performance.
TileRT is already available on PyPI, making it accessible to developers who want to integrate its runtime into their AI workflows. For those tracking the AI hardware and software arms race, TileRT’s latest results are a clear signal that AMD-backed solutions are gaining traction in the high-stakes inference battle.
Image source: Shutterstock.jpg)