Skip to content
Get packs

SkillOpt

Trains reusable text skill files for AI agents by testing edits, with no changes to model weights.

GitHub Download v0.2.0
Stars
18.8k
Forks
1.8k
Licence
MIT
Language
Python
Last update
4 days ago
Type
Dev tool

What it is

SkillOpt is a Python tool that improves a plain-text skill document (best_skill.md) used by a fixed, unchanged language model acting as an agent. It runs the agent on tasks, has an optimizer model propose small add, delete, or replace edits to the skill, and keeps an edit only if it improves a held-out validation score. People use it when they want an agent's instructions to get better over time without retraining the model.

  • Train a best_skill.md file for a benchmark task using a validation-gated loop
  • Run the skillopt-sleep CLI to review past sessions and consolidate validated skills
  • Launch the optional WebUI dashboard to monitor training runs
  • Compare optimized skills across different target models or agent harnesses

Great for

  • Developers who run LLM agents on repeatable tasks and want measurable skill improvements
  • Teams that need a deployable, human-readable skill file instead of fine-tuned weights
  • Researchers reproducing the SkillOpt paper's training loop and benchmarks
  • Users of Claude Code, Codex CLI, or similar coding agents who want offline skill refinement via SkillOpt-Sleep

Not ideal for

  • People who need to change model weights; SkillOpt explicitly does not touch them
  • Users without a labeled or scoreable task set, since training depends on validation scores
  • Non-technical users who expect a point-and-click app; setup uses Python and a CLI
  • Anyone needing guaranteed gains; the README reports results on its own benchmarks only
Step-by-step guide: how to use SkillOptInstall, first run, examples, tips and FAQs

How to get it

  1. 1

    Install with a package manager

    You'll need: Python 3.10+

    Install from PyPI

    pip install skillopt
  2. 2

    Run it

    • Install the package with pip install skillopt
    • Read the versioned docs at docs/index.md for data preparation and training commands
    • Run a training or eval command per the docs, using a supported backend such as OpenAI, Azure, or Claude
    • Inspect the resulting best_skill.md artifact and deploy it with your target model
  3. 3

    Get the most out of it

    • Use the built-in openai_compatible backend for providers that implement the OpenAI Chat Completions protocol
    • Add a custom benchmark via the skillopt/envs/<name>/ package contract; searchqa is the simplest reference
    • Enable the WebUI dashboard with pip install -e ".[webui]" to monitor runs; bind with auth if exposing it
    • Try skillopt-sleep for nightly offline skill consolidation with local coding agents

Downloads

Source code (.zip)main branch · straight from GitHubRelease v0.2.0Release notes and sourceOfficial website & docsaka.ms/skillopt

Strengths

  • Keeps the deployed artifact a small markdown file with zero extra inference-time model calls, per the README
  • Validation-gated updates mean edits are only accepted when they strictly improve held-out score in the default path
  • Extensible: documented contracts for adding new backends and benchmarks

Trade-offs

  • Results are reported on the authors' six benchmarks and paper setup; generalization to your tasks is unproven by the README
  • Many details (installation, configuration, commands) live in external docs not included here, raising setup effort
  • The README notes some experimental controls are not exposed as main CLI flags, so advanced tuning may require the harness code

How it compares

ProjectStarsTypeLevelUpdatedBest for
SkillOpt (this)18.8kDev tooladvanced4 days agoDevelopers who run LLM agents on repeatable tasks and want measurable skill improvements
agentic-awesome-skills49kDev toolintermediate4 days agoDevelopers using Codex or Claude who want agent-chosen instructions for a project
agents42.1kDev toolbeginner4 days agoDevelopers who already use one of the listed AI coding tools and want ready-made specialist agents
planning-with-files27.8kDev toolbeginner6 days agoDevelopers running long multi-step tasks in Claude Code or Codex CLI
distilly26.3kDev toolintermediate24 days agoDevelopers who want an agent to answer in the style of a specific colleague or mentor
autoharness10.6kDev toolbeginner6 days agoDevelopers who use Claude Code heavily and repeat the same kinds of tasks

Stats synced from GitHub. The assessment above was drafted with AI from the project's README on 2026-10-29 and may miss recent changes — check the README before relying on a command. MIT licence: free to use, modify and distribute, including commercially, as long as you keep the copyright notice.