Stanford AI Lab: Marin 535B-A23B Training Starts

Stanford AI Lab announced that Percy Liang kicked off training for the Marin 535B-A23B model this week in a fully open process. The run covers pretraining at 80% and midtraining at 20% across 18.75T tokens on 11 x GB200 NVL72 clusters for roughly three months at 2.7e24 FLOPs before post-training begins. Researchers first validated the approach with a four-rung scaling ladder from 1.6B-A61M to 27.7B-A1.2B models on smaller token counts to debug systems and forecast performance, underscoring the AI model pretraining process and scaling laws validation in high-performance computing for AI.