Thinking Machines details NVFP4 end to end 4 bit training
According to Thinking Machines via a company blog post cited by Soumith Chintala on X, the team trained Inkling 2 Small using native 4 bit forward passes from pretraining through final RL rollout.
SourceAnalysis
Thinking Machines detailed its NVFP4 training method in a new blog post, as announced in a tweet by Soumith Chintala on November 13, 2026.
According to the announcement, the team trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout. Training with fewer bits is faster, though quantization error can interfere with learning, and the post explains how the approach mitigates this issue across the full pipeline.
Soumith Chintala
@soumithchintalaCofounded and lead Pytorch at Meta. Also dabble in robotics at NYU.