Guide · Open source

How to use OpenMythos: install and run a looped transformer

Updated 2026-10-29 · written with AI from the project's own documentation and checked against our catalogue data

OpenMythos is an open-source Python library, built on PyTorch, that implements a Recurrent-Depth (looped) Transformer. It is a theoretical reconstruction based on public research, not a replica of any Anthropic model. This guide shows you how to install it, run the example configuration and understand the main options so you can experiment with looped depth at small scale.

OpenMythoskyegomez/OpenMythos · ★ 14.9k · MIT Install options & downloads

Quick start

1Check prerequisites. Make sure you have Python and PyTorch installed. Use a CUDA GPU only if you plan to use the Flash Attention extra.
2Install the package. Run pip install open-mythos, or use uv pip install open-mythos.
3Create a config. Build a MythosConfig with a small vocab_size, dim and max_loop_iters, and choose attn_type as mla or gqa.
4Build and run the model. Instantiate OpenMythos(cfg) and call it with token IDs and n_loops to get logits.
5Generate and inspect. Call model.generate to produce tokens and check the spectral radius of the injection matrix A, which should be below 1.
6Read the API docs. Open docs/open_mythos.md in the repository for the full class reference before changing configs.

What OpenMythos is and what it is not

OpenMythos splits a transformer into three stages: a Prelude of standard layers that runs once, a Recurrent Block that is looped up to max_loop_iters times, and a Coda of standard layers that runs once. The same recurrent weights are reused on every loop instead of stacking many unique layers.

The project describes itself as an independent, community-driven theoretical reconstruction based solely on publicly available research and speculation. It is not affiliated with, endorsed by, or connected to Anthropic. It also does not ship a trained model or a chat interface; the README shows small example configs only.

Library: OpenMythos class and MythosConfig from open_mythos.main
Licence: MIT
Language: Python, built on PyTorch

Install OpenMythos with pip or uv

Install the package from PyPI. You need Python and PyTorch. The README also lists a uv-based install.

To enable Flash Attention 2 in the GQA attention layer, install the extra. The README says this requires CUDA and build tools, so skip it if you do not have a CUDA GPU.

Standard install with pip
Install with uv if you use uv
Optional Flash Attention 2 extra (requires CUDA and build tools)

pip install open-mythos

#uv pip install open-mythos

pip install open-mythos[flash]

Run the example model configuration

The README example builds a small model with dim 256, 8 heads, and 4 loop iterations. Set attn_type to "mla" or "gqa" to pick the attention implementation. The script creates the model, counts parameters, runs a forward pass with n_loops=4, generates 8 new tokens, and prints the spectral radius of the injection matrix A, which the README says must be below 1.

Run the example with random token IDs. The output is not meaningful text because the model is untrained.

vocab_size, dim, n_heads and max_seq_len set the basic shape
max_loop_iters sets the maximum number of loops in the recurrent block
n_experts, n_shared_experts and n_experts_per_tok configure the sparse mixture-of-experts feed-forward
MLA needs kv_lora_rank, q_lora_rank and the RoPE and nope head dimensions; GQA needs n_kv_heads

import torch
from open_mythos.main import OpenMythos, MythosConfig

attn_type = "mla"  # or "gqa"

base = {
    "vocab_size": 1000,
    "dim": 256,
    "n_heads": 8,
    "max_seq_len": 128,
    "max_loop_iters": 4,
    "prelude_layers": 1,
    "coda_layers": 1,
    "n_experts": 8,
    "n_shared_experts": 1,
    "n_experts_per_tok": 2,
    "expert_dim": 64,
    "lora_rank": 8,
    "attn_type": attn_type,
}

if attn_type == "gqa":
    cfg = MythosConfig(**base, n_kv_heads=2)
else:
    cfg = MythosConfig(**base, n_kv_heads=8, kv_lora_rank=32, q_lora_rank=64, qk_rope_head_dim=16, qk_nope_head_dim=16, v_head_dim=16)

model = OpenMythos(cfg)
total = sum(p.numel() for p in model.parameters())
print(f"[{attn_type.upper()}] Parameters: {total:,}")

ids = torch.randint(0, cfg.vocab_size, (2, 16))
logits = model(ids, n_loops=4)
print(f"[{attn_type.upper()}] Logits shape: {logits.shape}")

out = model.generate(ids, max_new_tokens=8, n_loops=8)
print(f"[{attn_type.upper()}] Generated shape: {out.shape}")

Choose attention: MLA or GQA

The attention layer is set with cfg.attn_type. GQA (GQAttention) uses fewer key-value heads than query heads, which the README says reduces KV-cache memory by n_heads / n_kv_heads. MLA (MLAttention) caches a compressed KV latent controlled by kv_lora_rank instead of full keys and values.

GQA uses Flash Attention 2 when flash-attn 2.8.3 or newer is installed, and falls back to manual scaled dot-product attention otherwise.

gqa: GroupedQuery Attention, set n_kv_heads smaller than n_heads
mla: Multi-Latent Attention, set kv_lora_rank and the RoPE/nope head dims

Explore the model variants and training script

The README lists pre-configured variants from 1B to 1T parameters, such as mythos_1b and mythos_3b. Note that the README's own snippet calls mythos_7b(), which is not in its import list, so check the source before copying it. The 3B training script for FineWeb-Edu is in the repository under training/. The README states a target of 30B tokens and a single-GPU and multi-GPU launch method.

Read the API reference in docs/open_mythos.md for constructor, forward, generate and configuration details.

Single GPU: python training/3b_fine_web_edu.py
Multi-GPU: torchrun with --nproc_per_node set to the GPU count
Dataset in the script: HuggingFaceFW/fineweb-edu, sample-10BT by default
Tokenizer: openai/gpt-oss-20b via MythosTokenizer

python training/3b_fine_web_edu.py

What people use it for

Testing how the number of loop iterations (n_loops) changes model behaviour at small scale
Comparing MLA and GQA attention on the same small configuration
Studying how sparse mixture-of-experts layers combine with a looped recurrent block
Reading a compact PyTorch implementation of a looped transformer

Frequently asked questions

Is OpenMythos a free version of Claude?

No. The project says it is an independent, theoretical reconstruction based on public research and speculation, and it is not affiliated with Anthropic. It is not a working replica of Claude models.

What license is OpenMythos under?

OpenMythos is released under the MIT licence, as listed on its repository page. Check the LICENSE file in the repository for the full terms.

What do I need to install OpenMythos?

The README lists Python and PyTorch as prerequisites. Install with pip install open-mythos. The Flash Attention extra additionally requires CUDA and build tools.

Does OpenMythos work without a GPU?

The README says the Flash Attention path requires CUDA and build tools. The base install and the small example config do not require that extra, so you can run them on CPU if PyTorch is installed.

Can I chat with OpenMythos?

Not as shipped. The README shows small example configs and no trained model, so it is not a ready-to-use chat assistant.

What are the alternatives to OpenMythos?

The README lists nanoGPT, Hugging Face transformers, lit-gpt and minGPT as comparable projects. Those are better choices if you want a more established codebase for standard transformer training.

New agent skills, repos and AI tools — every week

The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.

Our verdict

OpenMythos suits researchers, developers and students who want a small, readable PyTorch codebase for looped transformer experiments, including MLA and GQA attention and mixture-of-experts. It is a poor fit if you want a working Claude replica, a trained chat model, or a CUDA-free setup for the Flash Attention path.

Install options & downloads