News

WeMM-Video Sets New Open-Source Standard for Omni-Modal Video Understanding

Tencent's WeMM-Video models achieve state-of-the-art performance in open-source omni-modal video understanding, unifying text, image, audio, and video processing.

Thematic editorial illustration
Editorial illustration; not a product screenshot or event photograph

Tencent has introduced WeMM-Video, a family of Chinese-English bilingual omni-modal foundation models designed to advance the field of video understanding by unifying text, image, audio, and video processing. The models, spanning sizes of 2B, 4B, 9B, and 35B-A3B, aim to address the challenges of joint audio-visual perception and reasoning through a hybrid-thinking paradigm. This includes a fast instruct mode for direct responses and a thinking mode for explicit step-by-step reasoning.

Technical Architecture and Training

WeMM-Video models are trained on a large-scale multimodal corpus of approximately 1.2 trillion tokens. The training process involves progressive pre-training and a multi-stage post-training pipeline that incorporates multi-teacher OPD, anti-repetition MrDPO, and specialist reinforcement learning (RL). These methods are designed to enhance the models' ability to handle complex multimodal tasks.

Benchmark Performance

Evaluated across more than 120 benchmarks, WeMM-Video demonstrates superior performance in omni-modal video understanding and reasoning. The flagship model, WeMM-Video-35B-A3B, establishes a new open-source state of the art, outperforming proprietary models such as Qwen3.5-Omni-Flash and Seed2.0-Mini-0428. It also narrows the performance gap with Gemini-3.0-Flash, a leading proprietary model. Additionally, the compact WeMM-Video-9B surpasses the larger open-source Qwen3-Omni-30B-A3B-Instruct, showcasing its efficiency and capability.

Open-Source Release

The WeMM-Video family is publicly available on GitHub, along with an evaluation and reproduction pipeline to encourage further research and development within the open-source community. The repository includes resources such as a model zoo, quick start guides, and usage examples, supporting both English and Simplified Chinese.

Implications for Multimodal Intelligence

By unifying multiple data modalities and offering state-of-the-art performance, WeMM-Video represents a significant step forward in the field of multimodal intelligence. Its open-source release is expected to foster innovation and collaboration among researchers and developers worldwide.

Sign in to save this article Free account: build your research collection and return to read it anytime.