SSyncDev

WeMM-Video

SyncDev model directory · Profile updated Nov 21, 2026

Compare with another model

WeMM-Video is Tencent's open Chinese-English omni-modal family (2B to 35B-A3B) that reasons over video frames and their soundtrack and answers in text.

Key specifications

Specification score
55.2
Context window
262.14K
License
Apache-2.0

The specification score summarizes recorded features and limits. It is not a benchmark result.

Pricing conditions: Open weights under Apache-2.0; no hosted API price.

Inputs and outputs

Recorded modality support shows what this model can accept and produce. An undocumented modality is shown as unknown rather than assumed to be unsupported.

Accepts

  • Text input
  • Image input
  • Audio input
  • Video input

Produces

  • Text output

Capabilities

ReasoningVisionAudioOpen weights

What is WeMM-Video?

Most video-language models watch a muted clip. WeMM-Video listens too. Tencent's family, released on 17 November 2026 under Apache-2.0, takes the frames of an MP4 together with its original audio track and answers in Chinese or English text, so a single request can cover what happens on screen, what is said and how the two relate.

It also accepts silent video, standalone audio, still images and plain text. That makes it closer to an omni-modal assistant with a video specialism than to a captioning model.

Four sizes, one recipe

Tencent documents each size in near-identical terms; what changes is scale. We therefore treat the release as one family.

Checkpoint Type BF16 weight files Notes
WeMM-Video-2B Dense 6.12 GiB Checked on L20 and H20 GPUs
WeMM-Video-4B Dense 10.45 GiB Checked on H20
WeMM-Video-9B Dense 19.54 GiB Joint leader on audio benchmarks
WeMM-Video-35BA3B MoE, 35B total / 3B active 67.39 GiB Flagship; serving recipes use two GPUs

Every size also ships FP8 Block128, AWQ W4A16, GGUF Q4_K_M and NVFP4 W4A16 builds. The mixture-of-experts flagship routes each token through 8 of 256 experts, which is why it fits a 3B active budget despite its size.

Under the hood, separate vision and audio encoders turn media into tokens for a language model, and video frames carry timestamps. The configuration files describe a Qwen3.5-style text model (mostly linear-attention layers with periodic full attention) and a Whisper-type audio encoder feeding a small Q-Former. If you already run Qwen3.5 9B or Qwen3.5 35B-A3B, the serving footprint will feel familiar. Training covered roughly 1.2 trillion multimodal tokens, followed by multi-teacher on-policy distillation, an anti-repetition preference stage and specialist reinforcement learning.

Instruct or Thinking

Instruct mode is the default and replies directly. Pass enable_thinking=True to the chat template (or chat_template_kwargs.enable_thinking over the API) to get a reasoning response first. Keep both in your test plan; reasoning costs latency on long clips.

Running it yourself

There is no hosted endpoint. You load weights, and the custom architecture needs trust_remote_code, so review the repository code and pin a revision.

  • Transformers (BF16 only): Python 3.12, CUDA 12.9, FFmpeg, transformers==5.2.0.
  • Patched vLLM 0.18.0: BF16, FP8, AWQ and NVFP4; a patched 0.31.0 also handles BF16.
  • Patched SGLang 0.5.16 or 0.5.19: BF16.
  • Native runtime: the GGUF build on CPU or CUDA.

The patched servers expose an OpenAI-compatible chat endpoint. Video goes in as video_url, and a use_audio flag decides whether the soundtrack is decoded. One request may carry up to four images, two videos and two audio items.

A context note worth flagging. The configuration allows 262,144 positions, but every published serving command sets an 8,192-token limit. Treat 8K as the tested setting; if you raise it for long videos, measure memory and quality yourself rather than assuming the larger figure just works.

Picking a size for your workload

Tencent reports that the 35B-A3B model sets a new open-source high on omni-modal video understanding, ahead of Qwen3.5-Omni-Flash and Seed2.0-Mini-0428, and that the 35B-A3B and 9B checkpoints lead on general audio understanding. These are publisher figures. Build a small set of your own clips instead: a meeting recording, a product demo, a noisy street scene. Score transcription, event timing and cross-modal questions ("what did the speaker say while the chart appeared?") on 2B, 9B and 35B-A3B side by side. If the 9B answers as well, you save a GPU.

For text-only retrieval over video archives, Tencent's WeMM Embedding 9B is a separate model in the same naming line. More options sit in the video-text-to-text category and on the Tencent provider page.

How to evaluate WeMM-Video

Compare the model on the work you need to complete.

Tasks and inputs

WeMM-Video is listed for Audio-text-to-text, Automatic speech recognition, Image-text-to-text, Multimodal, Video-text-to-text. It accepts text, image, audio, video and produces text. Use the publisher’s input format and preprocessing for your chosen deployment.

Capacity and cost

Documented limits: 262.14K context. Measure a completed task using the prompts, documents and tools your application will use. Include reasoning and retries in the result.

Choosing a deployment

The weights are downloadable under Apache-2.0. Check runtime support and size the complete model and working context for your expected concurrency.

Frequently asked questions

The 9B dense checkpoint is a sensible first test: Tencent names it alongside 35B-A3B as a leader on audio understanding, and its BF16 weights are about 19.5 GiB. Move to 35B-A3B (two GPUs in the published recipes) only if your own clips show a gap.

These records share a directory category. Check the source card, output type and deployment requirements before treating a model as a replacement.

NVIDIA

Flora-Captioner

Flora-Captioner is NVIDIA's 27B non-commercial video model that writes one action instruction for a chosen interval of a robot manipulation video.

— context

NVIDIA

Flora-Rewarder

Flora-Rewarder is NVIDIA's 27B non-commercial reward model that scores robot episodes for success and grades each half-second of action quality.

— context

perceptron

Perceptron Mk1.5

Perceptron Mk1.5 is an embodied reasoning model for robots, drones and glasses: video, image and audio in; text plus boxes, points and tracks out.

32.77K context

allenai

Molmo2-4B

Molmo2-4B is Ai2's open 4B vision-language model for video, multi-image and image understanding, with pointing and object tracking built in.

— context

Ai2 (Allen Institute for AI)

allenai/MolmoPoint-Vid-4B

allenai/MolmoPoint-Vid-4B is a source-linked Hugging Face repository assigned to Video-text-to-text. Its recorded task interface accepts video, text and produces text.

— context

chenjoya

chenjoya/videollm-online-8b-v1plus

chenjoya/videollm-online-8b-v1plus is a source-linked Hugging Face repository assigned to Video-text-to-text. Its recorded task interface accepts video, text and produces text.

— context