SSyncDev

PixelUMM

SyncDev model directory · Profile updated Nov 19, 2026

Compare with another model

PixelUMM is NVIDIA's encoder-free 15B unified model that understands and generates text, images and video directly in pixel space. Non-commercial.

Key specifications

Specification score
19.7
License
NVIDIA OneWay Noncommercial License (weights); Apache-2.0 (code)

The specification score summarizes recorded features and limits. It is not a benchmark result.

Pricing conditions: Non-commercial research weights; no hosted API or price.

Inputs and outputs

Recorded modality support shows what this model can accept and produce. An undocumented modality is shown as unknown rather than assumed to be unsupported.

Accepts

  • Text input
  • Image input
  • Video input

Produces

  • Text output
  • Image output
  • Video output

Capabilities

VisionOpen weights

What is PixelUMM?

Every popular multimodal generator bolts something onto the language model: a vision encoder to read images, a VAE to write them. PixelUMM throws both away. A single decoder-only Transformer reads raw pixel patches and writes raw pixels, and the same weights handle captioning, video question answering, text-to-image and text-to-video.

NVIDIA researchers built it with the University of Waterloo. The weights appeared on 1 October 2026; no other release date is given.

The design in four lines

  • Images are cut into 16x16 pixel patches; videos into 4-frame tubes.
  • The backbone is Qwen3-8B, with separate understanding and generation experts that share one self-attention over text, clean pixels and noisy pixels.
  • Generation runs as iterative denoising through a pixel head.
  • Total size: 15,199,672,064 parameters.

Which checkpoint to download

Checkpoint Notes
S8-F22-R05 Default; Stage 2 result used for the paper's evaluation
S8-F18-R01 Intermediate checkpoint with 10K extra fine-tuning steps at 480p and 720p short sides; slightly better text-to-video

The checkpoint already holds the learned language weights. You only need Qwen3-8B's config and tokenizer files alongside it. A bundled check_checkpoint.py verifies completeness on CPU before you commit a GPU.

Requirements and terms

Linux, an NVIDIA GPU, a CUDA build of PyTorch and FlashAttention. Text-to-video runs a separate guardrail environment, on by default. Model weights are licensed for non-commercial research or evaluation only under the NVIDIA OneWay Noncommercial License; the source code is Apache-2.0.

NVIDIA is plain about limits: prompts may not be followed reliably, outputs can be semantically or temporally inconsistent, and quality varies with language, domain, duration, resolution and aspect ratio.

Who should try it

Researchers comparing encoder-free and encoder-based designs are the obvious audience. A fair experiment pits it against an encoder-based any-to-any model on the same prompts in both directions: describe an image, then generate one from the description, and score the round trip. If shared pixel representations help, the round trip should hold detail better. For production image or video work, a commercially licensed text-to-video model remains the practical choice.

Other NVIDIA research models are on the provider page.

How to evaluate PixelUMM

Compare the model on the work you need to complete.

Tasks and inputs

PixelUMM is listed for Any-to-any, Text-to-video. It accepts text, image, video and produces text, image, video. Use the publisher’s input format and preprocessing for your chosen deployment.

Capacity and cost

Choose duration, resolution and frame rate together. Compare the cost of an accepted clip, including repeat generations.

Choosing a deployment

The weights are downloadable under NVIDIA OneWay Noncommercial License (weights); Apache-2.0 (code). Check runtime support and size the complete model and working context for your expected concurrency.

Frequently asked questions

It has no VAE and no vision encoder; one decoder-only Transformer reads and generates raw pixel patches for both images and video.

These records share a directory category. Check the source card, output type and deployment requirements before treating a model as a replacement.

Liquid AI

LFM3-Omni-2B

LFM3-Omni-2B is Liquid AI's open any-to-any model: image, video, audio and text in; text and speech out, with multimodal tool calls on-device.

— context

NVIDIA

Nemotron-Labs-Diffusion-Uni-8B

NLD-Uni (Nemotron-Labs-Diffusion-Uni-8B) is NVIDIA's open diffusion-language model that both understands images and generates them, for research.

— context

shanghai-ai-lab

InternLumina-U2

InternLumina-U2 is Shanghai AI Lab's masked-diffusion MoE that understands images, video and 3D and generates and edits images with 8-codebook visual tokens.

— context

sensetime

SenseNova-U1.5-8B-MoT

SenseNova-U1.5-8B-MoT is SenseTime's open unified model for native 4K image generation and instruction-based image editing, Apache-2.0 licensed.

— context

Tencent

Hy-Embodied-RxBrain-1.0

Hy-Embodied-RxBrain-1.0 is Tencent's 6.2B embodied model that reasons in text and imagines goal images, for planning, world-state prediction and video QA.

— context

google

google/gemma-4-E2B-it

google/gemma-4-E2B-it is a source-linked Hugging Face repository assigned to Any-to-any. Its recorded task interface accepts text, image, audio, video and produces text, image, audio, video.

131.07K context