Tasks and inputs
PixelUMM is listed for Any-to-any, Text-to-video. It accepts text, image, video and produces text, image, video. Use the publisher’s input format and preprocessing for your chosen deployment.
PixelUMM is NVIDIA's encoder-free 15B unified model that understands and generates text, images and video directly in pixel space. Non-commercial.
The specification score summarizes recorded features and limits. It is not a benchmark result.
Pricing conditions: Non-commercial research weights; no hosted API or price.
Recorded modality support shows what this model can accept and produce. An undocumented modality is shown as unknown rather than assumed to be unsupported.
Accepts
Produces
Every popular multimodal generator bolts something onto the language model: a vision encoder to read images, a VAE to write them. PixelUMM throws both away. A single decoder-only Transformer reads raw pixel patches and writes raw pixels, and the same weights handle captioning, video question answering, text-to-image and text-to-video.
NVIDIA researchers built it with the University of Waterloo. The weights appeared on 1 October 2026; no other release date is given.
| Checkpoint | Notes |
|---|---|
| S8-F22-R05 | Default; Stage 2 result used for the paper's evaluation |
| S8-F18-R01 | Intermediate checkpoint with 10K extra fine-tuning steps at 480p and 720p short sides; slightly better text-to-video |
The checkpoint already holds the learned language weights. You only need Qwen3-8B's config and tokenizer files alongside it. A bundled check_checkpoint.py verifies completeness on CPU before you commit a GPU.
Linux, an NVIDIA GPU, a CUDA build of PyTorch and FlashAttention. Text-to-video runs a separate guardrail environment, on by default. Model weights are licensed for non-commercial research or evaluation only under the NVIDIA OneWay Noncommercial License; the source code is Apache-2.0.
NVIDIA is plain about limits: prompts may not be followed reliably, outputs can be semantically or temporally inconsistent, and quality varies with language, domain, duration, resolution and aspect ratio.
Researchers comparing encoder-free and encoder-based designs are the obvious audience. A fair experiment pits it against an encoder-based any-to-any model on the same prompts in both directions: describe an image, then generate one from the description, and score the round trip. If shared pixel representations help, the round trip should hold detail better. For production image or video work, a commercially licensed text-to-video model remains the practical choice.
Other NVIDIA research models are on the provider page.
Compare the model on the work you need to complete.
PixelUMM is listed for Any-to-any, Text-to-video. It accepts text, image, video and produces text, image, video. Use the publisher’s input format and preprocessing for your chosen deployment.
Choose duration, resolution and frame rate together. Compare the cost of an accepted clip, including repeat generations.
The weights are downloadable under NVIDIA OneWay Noncommercial License (weights); Apache-2.0 (code). Check runtime support and size the complete model and working context for your expected concurrency.
These records share a directory category. Check the source card, output type and deployment requirements before treating a model as a replacement.
Liquid AI
LFM3-Omni-2B is Liquid AI's open any-to-any model: image, video, audio and text in; text and speech out, with multimodal tool calls on-device.
NVIDIA
NLD-Uni (Nemotron-Labs-Diffusion-Uni-8B) is NVIDIA's open diffusion-language model that both understands images and generates them, for research.
shanghai-ai-lab
InternLumina-U2 is Shanghai AI Lab's masked-diffusion MoE that understands images, video and 3D and generates and edits images with 8-codebook visual tokens.
sensetime
SenseNova-U1.5-8B-MoT is SenseTime's open unified model for native 4K image generation and instruction-based image editing, Apache-2.0 licensed.
Tencent
Hy-Embodied-RxBrain-1.0 is Tencent's 6.2B embodied model that reasons in text and imagines goal images, for planning, world-state prediction and video QA.
google/gemma-4-E2B-it is a source-linked Hugging Face repository assigned to Any-to-any. Its recorded task interface accepts text, image, audio, video and produces text, image, audio, video.