According to @_avichawla, LMCache’s multiprocess caching and CacheBlend deliver 14x faster TTFT and 4x faster decode on Qwen3-235B, cutting costs by up to 90%.

According to soumithchintala, Modal’s DFlash speculator for Inkling delivers 67% higher throughput and interactivity than MTP on SGLang endpoints.

According to @_avichawla, DFlash speeds LLM inference 8.5x via parallel draft tokens, maintaining accuracy and integrating with vLLM, SGLang, and Transformers.

According to AndrewYNg on X, DeepLearning.AI launched a new course, Efficient Inference with SGLang: Text and Image Generation, created with LMSys and RadixArk and taught by Richard Chen of RadixArk. As reported by AndrewYNg, the course targets production LLM cost bottlenecks and latency using SGLang techniques such as kernel fusion, paged attention, continuous batching, and optimized KV cache management for both text and image generation. According to AndrewYNg, the curriculum emphasizes practical deployment patterns for serving large models at scale, highlighting business value through reduced GPU hours, higher throughput per dollar, and improved tail latency—key metrics for inference economics.

According to DeepLearningAI on Twitter, a new course titled Efficient Inference with SGLang: Text and Image Generation is now live, focusing on cutting LLM inference costs by eliminating redundant computation using KV cache and RadixAttention (source: DeepLearning.AI tweet on April 8, 2026). As reported by DeepLearning.AI, the curriculum demonstrates how SGLang accelerates both text and image generation by reusing key value states to reduce recomputation and applying RadixAttention to optimize attention paths for lower latency and memory usage. According to DeepLearning.AI, the course also translates these techniques to vision and diffusion-style workloads, indicating practical deployment benefits such as higher throughput per GPU and reduced serving costs for production inference. As reported by DeepLearning.AI, the material targets practitioners aiming to improve utilization on commodity GPUs and scale serving capacity without proportional hardware spend.

According to OpenClaw on Twitter, the v2026.3.12 release introduces Dashboard v2 with a streamlined control UI, a new /fast mode to speed model interactions, and a plugin-based integration path for Ollama, SGLang, and vLLM that trims the core footprint, enhancing modularity and maintainability (source: OpenClaw Twitter; release notes on GitHub). According to the GitHub release notes, device tokens are now ephemeral to reduce long-lived credential risk, and cron plus Windows reliability fixes address scheduled task stability and cross-platform uptime for on-prem and self-hosted AI deployments (source: GitHub OpenClaw releases). As reported by OpenClaw, these updates target faster inference routing, safer authentication, and easier backend swapping—key for teams orchestrating local LLMs and inference servers in production environments (source: OpenClaw Twitter).

According to @soumithchintala and @ying11231, RadixArk has emerged as a new player in the AI infrastructure sector, aiming to make advanced AI infrastructure open and accessible to everyone (source: https://x.com/ying11231/status/1998079551369593222). The platform is being developed by a core team previously behind SGLang, which gained traction as an open-source AI language stack since its public release in January 2024. RadixArk differentiates itself from established AI infrastructure providers by focusing on community-driven development, openness, and elegant engineering. The company is addressing the inefficiency of repeated infrastructure building across the industry by sharing schedulers, compilers, serving engines, and training pipelines as open tools. This approach creates significant business opportunities for organizations seeking scalable, reliable, and collaborative AI deployment infrastructure, potentially accelerating AI adoption and innovation across sectors (source: @soumithchintala on Twitter, Dec 11, 2025).