SSyncDev

Bolmo 7B

SyncDev model directory · Profile updated Nov 19, 2026

Compare with another model

Bolmo 7B is Ai2's open byte-level language model, Olmo 3 7B retrofitted to read and write UTF-8 bytes instead of subword tokens.

Key specifications

Specification score
3.1
License
Apache-2.0

The specification score summarizes recorded features and limits. It is not a benchmark result.

Pricing conditions: Open weights for self-hosting; no hosted API price. Hardware cost is yours.

Inputs and outputs

Recorded modality support shows what this model can accept and produce. An undocumented modality is shown as unknown rather than assumed to be unsupported.

Accepts

  • Text input

Produces

  • Text output

Capabilities

Open weights

What is Bolmo 7B?

Ask a normal language model how many r's are in "strawberry" and it has to guess, because it never saw the letters, only tokens. Bolmo 7B sees every byte. Every one. Ai2 built it by taking Olmo 3 7B and "byteifying" it: keeping the trained transformer and wrapping it in machinery that works on raw UTF-8 instead of a subword vocabulary.

Ai2 introduced it on 15 December 2025 as, to its knowledge, the first fully open byte-level model family competitive with subword models.

The retrofit, step by step

  1. Each byte is embedded and passed through a small local encoder built from mLSTM layers.
  2. A boundary predictor, using one byte of look-ahead, decides where to cut the stream into variable-length patches.
  3. Patches go through the original Olmo 3 transformer.
  4. Outputs are depooled to bytes and refined by a local decoder, which predicts the next byte and the next boundary.

Training came in two stages, and the first was cheap. With the transformer frozen, Ai2 trained only the new parts on 9.8B tokens, about 43B bytes. Then everything was unfrozen for 39.3B more tokens, around 173B bytes. Data came from the Dolma 3 mix, open code and a small slice of character-focused material.

What changes in practice

On Ai2's broad suite Bolmo 7B lands close to the subword Olmo 3 7B. On character-focused tests like CUTE and EXECUTE it pulls far ahead, nearly twenty points on Ai2's character aggregate. Against other byte-level 7B-class models in Ai2's comparison, BLT 7B, TFree-Hat 7B and EvaByte 6.5B, it is the strongest overall.

Item Detail
Repo allenai/Bolmo-7B
Retrofitted from Bolmo-7B-Stage1, itself from Olmo 3 7B
Parameters ~7.6B
Language English
Requirements transformers 4.57.3+, Python 3.11, xlstm==2.0.4
Licence Apache-2.0

One quirk. max_new_tokens counts bytes. Set it roughly four times higher than you would for a tokenised model of English.

Should you switch?

Maybe.

For spelling, string manipulation, rare words and messy whitespace, a byte model is the principled fix. Test with your own failure cases. Product codes, misspellings, mixed scripts and long identifiers are the inputs where subword models stumble most visibly, so collect a few hundred from production logs and compare exact-match accuracy against your current model. Measure speed too.

It is a base model with no safety tuning, so anything user-facing needs your own filtering, instruction tuning and red-teaming before it goes near customers. The same byteifying recipe also produced Bwen 8B from Qwen3 and Llama-3-Blama-8B from Llama 3; a smaller Bolmo 1B exists as well.

How to evaluate Bolmo 7B

Compare the model on the work you need to complete.

Tasks and inputs

Bolmo 7B is listed for Text generation, Text-to-text AI models. It accepts text and produces text. Use the publisher’s input format and preprocessing for your chosen deployment.

Capacity and cost

Measure a completed task using the prompts, documents and tools your application will use. Include reasoning and retries in the result.

Choosing a deployment

The weights are downloadable under Apache-2.0. Check runtime support and size the complete model and working context for your expected concurrency.

Frequently asked questions

It reads and generates raw UTF-8 bytes, grouping them into learned patches, instead of using a fixed subword vocabulary.

These records share a directory category. Check the source card, output type and deployment requirements before treating a model as a replacement.

Microsoft

Aion 1.0 Instruct 2.3B

Aion 1.0 Instruct 2.3B is Microsoft's MIT-licensed on-device small language model and Phi Silica successor for Windows, with a 32K context.

32.77K context

Google

Gemini 4 Neon

Gemini 4 Neon is Google's mid-tier Gemini 4 model for complex coding and analysis, sitting between Hydrogen and Carbon at $3/$15 per 1M tokens.

1.05M context

OpenAI

GPT-6.2 Astra

GPT-6.2 Astra is OpenAI's November 2026 flagship for hard professional, scientific and agentic work, at $8/$40 per million tokens with a 1.05M context.

1.05M context

OpenAI

GPT-6.2 Sol

GPT-6.2 Sol is OpenAI's everyday model for coding, agents and writing: $2/$10 per million tokens, 1.05M context, released 17 November 2026.

1.05M context

Alibaba

Qwen4 Max

Qwen4 Max (qwen4-max) is Alibaba's first Qwen4-generation flagship: 1M-token context, text, image and video input, thinking on by default, $1.50/$4.50 per 1M.

1M context

Sarvam

Sarvam-105B-1711

Sarvam-105B-1711 is a refreshed open post-train of Sarvam's 105B MoE, tuned for tool calling, instruction following, agents and Indian languages.

— context