SSyncDev

Bwen 8B

SyncDev model directory · Profile updated Nov 17, 2026

Compare with another model

Bwen 8B is Ai2's byte-level version of Qwen3-8B, retrofitted with the Bolmo byteification method so it reads and writes raw bytes, not tokens.

Key specifications

Specification score
4.8
License
Apache-2.0

The specification score summarizes recorded features and limits. It is not a benchmark result.

Inputs and outputs

Recorded modality support shows what this model can accept and produce. An undocumented modality is shown as unknown rather than assumed to be unsupported.

Accepts

  • Text input

Produces

  • Text output

Capabilities

Open weights

What is Bwen 8B?

Tokenisers are the part of a language model nobody loves. They split words in odd places, mangle rare strings and treat whitespace inconsistently. Bwen 8B removes the tokeniser entirely. Ai2 took Qwen3-8B and retrofitted it to work directly on bytes, using its Bolmo "byteification" method, and released it on 7 October 2026 when that research appeared in Nature.

It is a research checkpoint. It is also, by Ai2's account, its strongest byte-level model so far, ahead of Bolmo 7B on the lab's aggregate evaluation suite.

What byteification means

Building a byte-level model from scratch is expensive. Byteification instead converts an existing subword model through a relatively short extra training run. The original model's knowledge mostly survives, and the new byte-level components learn to feed it. Ai2 also published Stage 1 checkpoints, where the original network is frozen while only the new byte components train; they are meant as a faster starting point for further research.

Why bother? Byte models can handle spelling, unusual strings, mixed scripts and exact whitespace more naturally. Because bytes are the basic unit of all digital data, Ai2 also suggests the approach could eventually extend to images and audio.

The honest trade-off

Byteified models come close to their source models rather than matching them. Follow-up research is still working out the balance between the efficiency of hierarchical byte architectures and fine-grained character understanding. Expect a small quality gap against plain Qwen3-8B on ordinary benchmarks, in exchange for better behaviour on character-level tasks. The model page publishes no benchmark table.

Getting it running

Field Value
Repository allenai/Bwen-8B (plus Bwen-8B-Stage1)
Base Qwen3-8B Base
Size 8B, F32 safetensors
Unit Bytes; max_new_tokens counts bytes
Language English
Licence Apache 2.0

Loading needs Transformers 4.57.3 or later, Python 3.11 and the xlstm 2.0.4 package, with trust_remote_code=True on both the model and tokenizer. vLLM, SGLang and Docker Model Runner commands are also provided. Remember that generation length is measured in bytes, so a limit that suits a token model will cut answers short.

It is a base model with no safety tuning. Ai2 positions it for research and education.

Where it fits

Use it to study tokenisation effects, to build tasks where exact characters matter (code formatting, transliteration, noisy OCR text), or as a starting point for your own byte-level fine-tune. Do not deploy it as an assistant.

A useful first test: take a character-sensitive task, such as spelling correction or counting characters, and run it against Qwen3 32B or another subword model of similar class. Then run a general QA set to see what the byte conversion cost. Its sibling retrofit of Llama 3, Llama-3-Blama-8B, shipped the same day; more from the lab is on the Ai2 provider page.

How to evaluate Bwen 8B

Compare the model on the work you need to complete.

Tasks and inputs

Bwen 8B is listed for Fill-mask, Text generation. It accepts text and produces text. Use the publisher’s input format and preprocessing for your chosen deployment.

Capacity and cost

Measure a completed task using the prompts, documents and tools your application will use. Include reasoning and retries in the result.

Choosing a deployment

The weights are downloadable under Apache-2.0. Check runtime support and size the complete model and working context for your expected concurrency.

Frequently asked questions

A byte-level version of Qwen3-8B Base made with Ai2's Bolmo byteification method, so it processes raw bytes instead of subword tokens.

These records share a directory category. Check the source card, output type and deployment requirements before treating a model as a replacement.

Microsoft

Aion 1.0 Instruct 2.3B

Aion 1.0 Instruct 2.3B is Microsoft's MIT-licensed on-device small language model and Phi Silica successor for Windows, with a 32K context.

32.77K context

Google

Gemini 4 Neon

Gemini 4 Neon is Google's mid-tier Gemini 4 model for complex coding and analysis, sitting between Hydrogen and Carbon at $3/$15 per 1M tokens.

1.05M context

OpenAI

GPT-6.2 Astra

GPT-6.2 Astra is OpenAI's November 2026 flagship for hard professional, scientific and agentic work, at $8/$40 per million tokens with a 1.05M context.

1.05M context

OpenAI

GPT-6.2 Sol

GPT-6.2 Sol is OpenAI's everyday model for coding, agents and writing: $2/$10 per million tokens, 1.05M context, released 17 November 2026.

1.05M context

Alibaba

Qwen4 Max

Qwen4 Max (qwen4-max) is Alibaba's first Qwen4-generation flagship: 1M-token context, text, image and video input, thinking on by default, $1.50/$4.50 per 1M.

1M context

Sarvam

Sarvam-105B-1711

Sarvam-105B-1711 is a refreshed open post-train of Sarvam's 105B MoE, tuned for tool calling, instruction following, agents and Indian languages.

— context