Most video-language models watch a muted clip. WeMM-Video listens too. Tencent's family, released on 17 November 2026 under Apache-2.0, takes the frames of an MP4 together with its original audio track and answers in Chinese or English text, so a single request can cover what happens on screen, what is said and how the two relate.
It also accepts silent video, standalone audio, still images and plain text. That makes it closer to an omni-modal assistant with a video specialism than to a captioning model.
Four sizes, one recipe
Tencent documents each size in near-identical terms; what changes is scale. We therefore treat the release as one family.
| Checkpoint |
Type |
BF16 weight files |
Notes |
| WeMM-Video-2B |
Dense |
6.12 GiB |
Checked on L20 and H20 GPUs |
| WeMM-Video-4B |
Dense |
10.45 GiB |
Checked on H20 |
| WeMM-Video-9B |
Dense |
19.54 GiB |
Joint leader on audio benchmarks |
| WeMM-Video-35BA3B |
MoE, 35B total / 3B active |
67.39 GiB |
Flagship; serving recipes use two GPUs |
Every size also ships FP8 Block128, AWQ W4A16, GGUF Q4_K_M and NVFP4 W4A16 builds. The mixture-of-experts flagship routes each token through 8 of 256 experts, which is why it fits a 3B active budget despite its size.
Under the hood, separate vision and audio encoders turn media into tokens for a language model, and video frames carry timestamps. The configuration files describe a Qwen3.5-style text model (mostly linear-attention layers with periodic full attention) and a Whisper-type audio encoder feeding a small Q-Former. If you already run Qwen3.5 9B or Qwen3.5 35B-A3B, the serving footprint will feel familiar. Training covered roughly 1.2 trillion multimodal tokens, followed by multi-teacher on-policy distillation, an anti-repetition preference stage and specialist reinforcement learning.
Instruct or Thinking
Instruct mode is the default and replies directly. Pass enable_thinking=True to the chat template (or chat_template_kwargs.enable_thinking over the API) to get a reasoning response first. Keep both in your test plan; reasoning costs latency on long clips.
Running it yourself
There is no hosted endpoint. You load weights, and the custom architecture needs trust_remote_code, so review the repository code and pin a revision.
- Transformers (BF16 only): Python 3.12, CUDA 12.9, FFmpeg,
transformers==5.2.0.
- Patched vLLM 0.18.0: BF16, FP8, AWQ and NVFP4; a patched 0.31.0 also handles BF16.
- Patched SGLang 0.5.16 or 0.5.19: BF16.
- Native runtime: the GGUF build on CPU or CUDA.
The patched servers expose an OpenAI-compatible chat endpoint. Video goes in as video_url, and a use_audio flag decides whether the soundtrack is decoded. One request may carry up to four images, two videos and two audio items.
A context note worth flagging. The configuration allows 262,144 positions, but every published serving command sets an 8,192-token limit. Treat 8K as the tested setting; if you raise it for long videos, measure memory and quality yourself rather than assuming the larger figure just works.
Picking a size for your workload
Tencent reports that the 35B-A3B model sets a new open-source high on omni-modal video understanding, ahead of Qwen3.5-Omni-Flash and Seed2.0-Mini-0428, and that the 35B-A3B and 9B checkpoints lead on general audio understanding. These are publisher figures. Build a small set of your own clips instead: a meeting recording, a product demo, a noisy street scene. Score transcription, event timing and cross-modal questions ("what did the speaker say while the chart appeared?") on 2B, 9B and 35B-A3B side by side. If the 9B answers as well, you save a GPU.
For text-only retrieval over video archives, Tencent's WeMM Embedding 9B is a separate model in the same naming line. More options sit in the video-text-to-text category and on the Tencent provider page.