Tasks and inputs
Bwen 8B is listed for Fill-mask, Text generation. It accepts text and produces text. Use the publisher’s input format and preprocessing for your chosen deployment.
Bwen 8B is Ai2's byte-level version of Qwen3-8B, retrofitted with the Bolmo byteification method so it reads and writes raw bytes, not tokens.
The specification score summarizes recorded features and limits. It is not a benchmark result.
Recorded modality support shows what this model can accept and produce. An undocumented modality is shown as unknown rather than assumed to be unsupported.
Accepts
Produces
Tokenisers are the part of a language model nobody loves. They split words in odd places, mangle rare strings and treat whitespace inconsistently. Bwen 8B removes the tokeniser entirely. Ai2 took Qwen3-8B and retrofitted it to work directly on bytes, using its Bolmo "byteification" method, and released it on 7 October 2026 when that research appeared in Nature.
It is a research checkpoint. It is also, by Ai2's account, its strongest byte-level model so far, ahead of Bolmo 7B on the lab's aggregate evaluation suite.
Building a byte-level model from scratch is expensive. Byteification instead converts an existing subword model through a relatively short extra training run. The original model's knowledge mostly survives, and the new byte-level components learn to feed it. Ai2 also published Stage 1 checkpoints, where the original network is frozen while only the new byte components train; they are meant as a faster starting point for further research.
Why bother? Byte models can handle spelling, unusual strings, mixed scripts and exact whitespace more naturally. Because bytes are the basic unit of all digital data, Ai2 also suggests the approach could eventually extend to images and audio.
Byteified models come close to their source models rather than matching them. Follow-up research is still working out the balance between the efficiency of hierarchical byte architectures and fine-grained character understanding. Expect a small quality gap against plain Qwen3-8B on ordinary benchmarks, in exchange for better behaviour on character-level tasks. The model page publishes no benchmark table.
| Field | Value |
|---|---|
| Repository | allenai/Bwen-8B (plus Bwen-8B-Stage1) |
| Base | Qwen3-8B Base |
| Size | 8B, F32 safetensors |
| Unit | Bytes; max_new_tokens counts bytes |
| Language | English |
| Licence | Apache 2.0 |
Loading needs Transformers 4.57.3 or later, Python 3.11 and the xlstm 2.0.4 package, with trust_remote_code=True on both the model and tokenizer. vLLM, SGLang and Docker Model Runner commands are also provided. Remember that generation length is measured in bytes, so a limit that suits a token model will cut answers short.
It is a base model with no safety tuning. Ai2 positions it for research and education.
Use it to study tokenisation effects, to build tasks where exact characters matter (code formatting, transliteration, noisy OCR text), or as a starting point for your own byte-level fine-tune. Do not deploy it as an assistant.
A useful first test: take a character-sensitive task, such as spelling correction or counting characters, and run it against Qwen3 32B or another subword model of similar class. Then run a general QA set to see what the byte conversion cost. Its sibling retrofit of Llama 3, Llama-3-Blama-8B, shipped the same day; more from the lab is on the Ai2 provider page.
Compare the model on the work you need to complete.
Bwen 8B is listed for Fill-mask, Text generation. It accepts text and produces text. Use the publisher’s input format and preprocessing for your chosen deployment.
Measure a completed task using the prompts, documents and tools your application will use. Include reasoning and retries in the result.
The weights are downloadable under Apache-2.0. Check runtime support and size the complete model and working context for your expected concurrency.
These records share a directory category. Check the source card, output type and deployment requirements before treating a model as a replacement.
Microsoft
Aion 1.0 Instruct 2.3B is Microsoft's MIT-licensed on-device small language model and Phi Silica successor for Windows, with a 32K context.
Gemini 4 Neon is Google's mid-tier Gemini 4 model for complex coding and analysis, sitting between Hydrogen and Carbon at $3/$15 per 1M tokens.
OpenAI
GPT-6.2 Astra is OpenAI's November 2026 flagship for hard professional, scientific and agentic work, at $8/$40 per million tokens with a 1.05M context.
OpenAI
GPT-6.2 Sol is OpenAI's everyday model for coding, agents and writing: $2/$10 per million tokens, 1.05M context, released 17 November 2026.
Alibaba
Qwen4 Max (qwen4-max) is Alibaba's first Qwen4-generation flagship: 1M-token context, text, image and video input, thinking on by default, $1.50/$4.50 per 1M.
Sarvam
Sarvam-105B-1711 is a refreshed open post-train of Sarvam's 105B MoE, tuned for tool calling, instruction following, agents and Indian languages.