Skip to content
Get packs

OpenMythos

A community-written, theoretical PyTorch model of a looped transformer, inspired by speculation about Claude M

GitHub Download source
Stars
14.9k
Forks
3.3k
Licence
MIT
Language
Python
Last update
6 months ago
Type
Library

What it is

OpenMythos is a Python library that implements a Recurrent-Depth (looped) Transformer in PyTorch. Instead of stacking many unique layers, a middle block of layers is reused several times per forward pass, with the input re-injected each loop. The project states it is an independent, theoretical reconstruction based on public research and speculation, and is not affiliated with Anthropic.

  • Instantiate a small OpenMythos model from a MythosConfig and count its parameters
  • Run a forward pass with a chosen number of loop iterations using n_loops
  • Generate a few tokens from a randomly initialized model to check the code path
  • Compare MLA and GQA attention variants with the same base configuration
  • Inspect the spectral radius of the injection matrix A to check the stability condition

Great for

  • Researchers exploring looped or recurrent-depth transformer designs
  • Developers who want a small, readable PyTorch codebase to experiment with MLA or GQA attention
  • Students studying how sparse mixture-of-experts and loop-based depth fit together
  • People who want to test compute-adaptive reasoning ideas at small scale

Not ideal for

  • Anyone expecting a working replica of Anthropic's Claude models, since the README says it is speculative and unaffiliated
  • People looking for a ready-to-use chat assistant, as the README shows only small example configs and no trained model
  • Users without a CUDA GPU who need the Flash Attention path, since the README says it requires CUDA and build tools
Step-by-step guide: how to use OpenMythosInstall, first run, examples, tips and FAQs
OpenMythos alternativesminGPT vs OpenMythos

How to get it

  1. 1

    Install with a package manager

    You'll need: Python · PyTorch

    Install from PyPI

    pip install open-mythos

    Install with uv

    uv pip install open-mythos

    Install with Flash Attention 2 extra (requires CUDA and build tools)

    pip install open-mythos[flash]
  2. 2

    Run it

    • Install the package with pip install open-mythos
    • Import MythosConfig and OpenMythos from open_mythos.main
    • Build a small config such as the one in the README (dim 256, vocab 1000)
    • Create the model and call it on random token IDs with n_loops set
    • Call generate() to produce a few tokens
  3. 3

    Get the most out of it

    • Vary max_loop_iters and the n_loops argument to see how depth changes outputs
    • Switch attn_type between "mla" and "gqa" to compare parameter counts and behavior
    • Read docs/open_mythos.md for the full API and configuration reference
    • Check docs/datasets.md for dataset guidance if you plan to train
    • Use the training script in training/3b_fine_web_edu.py as a starting point for multi-GPU runs

Downloads

Source code (.zip)main branch · straight from GitHubOfficial website & docsdiscord.gg/EamjgSaEQf

Strengths

  • Small, documented API with a clear usage example in the README
  • Supports two attention types (MLA and GQA) switchable through one config field
  • Released under the MIT license, permitting reuse

Trade-offs

  • The README describes the work as theoretical and speculative, so claims about Claude Mythos are unverified
  • The README lists pre-configured scales up to 1T parameters, but the example model code and the variant table do not show trained weights or benchmark results
  • The usage example calls mythos_7b() although the listed variants do not include a 7b entry, so the README is internally inconsistent

How it compares

ProjectStarsTypeLevelUpdatedBest for
OpenMythos (this)14.9kLibraryintermediate6 months agoResearchers exploring looped or recurrent-depth transformer designs
nanoGPT64.4kLibraryintermediate12 months agoDevelopers who want to read a compact GPT training implementation
minGPT25kLibraryintermediate2.2 years agoStudents who want to read a working GPT implementation end to end
claude-code-best-practice68.5kLearning resourcebeginner4 days agoDevelopers who are new to Claude Code and want an organized overview of its features
AI-Research-SKILLs14.2kCurated listintermediate26 days agoDevelopers who already use Claude Code, Codex, or Gemini CLI and want their agent to handle ML engineering tasks
ECC295kDev toolintermediate4 days agoDevelopers who already use Claude Code and want a shared workflow for planning, testing and review

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.