Skip to main content
Latest Update
2026-11-13 13:55 UTC

Thinking Machines details NVFP4 end to end 4 bit training

Thinking Machines details NVFP4 end to end 4 bit training

According to Thinking Machines via a company blog post cited by Soumith Chintala on X, the team trained Inkling 2 Small using native 4 bit forward passes from pretraining through final RL rollout.

Source

Analysis

Thinking Machines detailed its NVFP4 training method in a new blog post, as announced in a tweet by Soumith Chintala on November 13, 2026.

According to the announcement, the team trained Inkling 2 Small with a native 4-bit forward pass from the first step of pretraining through the final RL rollout. Training with fewer bits is faster, though quantization error can interfere with learning, and the post explains how the approach mitigates this issue across the full pipeline.

Soumith Chintala

@soumithchintala

Cofounded and lead Pytorch at Meta. Also dabble in robotics at NYU.