Guide · Agent skill
How to use the ARA Rigor Reviewer agent skill
Updated · written with AI from the project's own documentation and checked against our catalogue data
The ARA Rigor Reviewer is an agent skill that teaches an AI agent to review an Agent-Native Research Artifact, a structured folder of research claims, experiments and logs, the way a strict peer reviewer would. This guide explains what the skill makes the agent do, how to install it, how to ask for a review in plain language, and what to check in the report it writes. No prior experience with agent skills is assumed.
ARA Rigor ReviewerOrchestra-Research/AI-Research-SKILLs · MIT Download the skill Quick start
- 1Download and unzip the skill. Download the skill ZIP from this page and unzip it into ~/.claude/skills/ for personal use or into a project's .claude/skills/ folder.
- 2Or upload it in the Claude apps. In the Claude apps, open Settings, go to Capabilities, and upload the skill under Skills.
- 3Prepare the artifact. Make sure the ARA directory has PAPER.md, logic/ and trace/ files, and that Level 1 structural validation has passed.
- 4Ask for a review in plain language. Tell the agent which artifact to review and that you want a Level 2 rigor review before publication or release.
- 5Wait for level2_report.json. The agent reads the artifact in its fixed order, scores six dimensions from 1 to 5 and writes the report at the artifact root.
- 6Act on the findings. Work through major findings first, then the suggestions, and revise claims, falsification criteria or experiments as needed.
What is an agent skill and what does this one do?
An agent skill is a folder containing a SKILL.md file and any reference files that an AI agent loads when it needs specialised instructions for a task. This skill is designed for one job: a Level 2 semantic epistemic review of an Agent-Native Research Artifact (ARA), which is a directory with PAPER.md, logic/ and trace/ files.
The skill assumes Level 1 structural validation has already passed, so references resolve and required fields exist. Level 2 does not re-check structure. It asks whether the content is sound: whether cited experiments actually support each claim, whether falsification criteria are testable, and whether the research process is honestly documented, including failures.
- Scores six dimensions from 1 to 5: evidence relevance, falsifiability quality, scope calibration, argument coherence, exploration integrity and methodological rigor.
- Lists strengths, weaknesses and suggestions for each dimension.
- Writes a constructive, severity-ranked report to level2_report.json at the artifact root.
- Ends with a recommendation on a scale from Strong Accept to Reject.
Install the ARA Rigor Reviewer skill
Download the ZIP for this skill from this page. Unzip it so that the skill folder sits in one of two locations. For a personal install available to all your projects, use ~/.claude/skills/. For a single project, use the .claude/skills/ folder inside that project.
If you use the Claude apps instead, open Settings, go to Capabilities, and upload the skill under Skills. Either way, the agent needs to be able to read and write files in the artifact folder you want reviewed.
- Personal install: unzip into ~/.claude/skills/
- Project install: unzip into the project's .claude/skills/ folder
- Claude apps: Settings, then Capabilities, then Skills, then upload
Check that your artifact is ready for Level 2
Before you ask for a review, confirm the artifact is in the expected shape. The skill needs an Agent-Native Research Artifact directory containing PAPER.md, logic/ and trace/ files. The skill also expects Level 1 structural validation to have passed. If it has not, fix structure first, because the skill does not judge structure and its scores assume it is sound.
The agent needs file read, write, glob and grep tools. The skill works only with these tools. It does not run code, fetch web pages or consult outside sources.
- PAPER.md at the artifact root
- logic/ and trace/ directories
- Level 1 structural validation already passed
- An agent runtime with file read, write, glob and grep tools
Trigger the skill with a natural request
You do not need a special command. Ask the agent in plain words for an objective critique of the artifact before publication or release, and point it at the artifact directory. The skill description is written for exactly this situation: after Level 1 passes, when an ARA needs an epistemic critique.
Give the path and say what you want. A clear request names the artifact, the purpose and the output you expect, so the agent knows to produce level2_report.json.
- Example: Run a Level 2 rigor review on the ARA in ./my-artifact before I submit it.
- Example: Critique the claims and experiments in this research artifact and write the report.
- Example: Review this artifact for falsifiability and scope calibration, then give a recommendation.
Read the six dimensions and the report
The agent reads files in a fixed order, starting with PAPER.md, then logic/claims.md, logic/experiments.md, the problem, concepts and solution files, related work, trace/exploration_tree.yaml, and a spot-check of evidence files. It then scores each dimension on a 1 to 5 scale. A score of 5 means evidence is type-appropriate for every claim and falsification criteria are specific and independently testable. A score of 1 means the evidence or criteria are largely meaningless.
Use the report as a prioritised to-do list. Start with major findings, since the scoring anchors treat evidence relevance and type-aware entailment as major issues, then work through suggestions.
- D1 Evidence Relevance: does cited evidence support each claim in substance?
- D2 Falsifiability Quality: are criteria actionable, non-trivial and scoped like the claim?
- D3 Scope Calibration: do claims assert no more and no less than the evidence supports?
- D4 Argument Coherence: does the narrative run from problem to solution to evidence?
- D5 Exploration Integrity: does the exploration tree record genuine process, including failures?
- D6 Methodological Rigor: are baselines, ablations and reporting adequate?
Understand the limits and safety of the skill
The skill declares that it runs no scripts and uses no network access. It works only through the agent's own read, write, glob and grep tools, and it writes only its report file. Because it reads only the artifact, it cannot check whether the research is correct in reality or rerun experiments. Its scores reflect textual reasoning about the artifact.
Treat the output as a model judgment. The scores may vary between runs and should inform your own reading, not replace human peer review for grants, hiring or publication decisions.
- Runs scripts: no
- Uses network: no
- Does not verify external sources or rerun experiments
- Scores are model judgments and may be inconsistent
What people use it for
- Pre-submission check of an Agent-Native Research Artifact before sending it to a venue
- Finding claims whose cited experiments do not test what the claim says
- Tightening falsification criteria so an independent researcher could test them
- Checking whether the exploration tree honestly records dead ends and pivots
- Giving a lab group a consistent, rubric-based critique of experiment design and claim scope
- Building or testing ARA tooling that needs a semantic review layer after structural validation
Frequently asked questions
How do I install the ARA Rigor Reviewer skill?
Download the ZIP from this page and unzip it into ~/.claude/skills/ for personal use or into a project's .claude/skills/ folder. In the Claude apps, you can instead upload it under Settings, then Capabilities, then Skills.
What does the ara-rigor-reviewer skill need to run?
It needs an Agent-Native Research Artifact directory with PAPER.md, logic/ and trace/ files, and an agent runtime with file read, write, glob and grep tools. The skill also assumes Level 1 structural validation has already passed.
Is the ARA Rigor Reviewer skill free to use?
The skill is published under the MIT licence, as listed on this page. Check the licence file included with the download for the full terms.
Does the skill check whether my research results are true?
No. The agent reads only the artifact and does not check external sources or rerun experiments, so its scores reflect textual reasoning rather than verified truth.
Why did the skill refuse or give odd results on my artifact?
The skill assumes Level 1 structural validation has passed. If references do not resolve, required fields are missing or the exploration tree does not parse, fix those structural issues first, then ask for the Level 2 review again.
Can this replace human peer review?
No. The scores are model judgments and may be inconsistent, so the skill is not meant for high-stakes decisions such as grants, hiring or publication acceptance. Use it as a critical pre-submission read alongside human feedback.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
This skill suits researchers preparing an Agent-Native Research Artifact who want a strict, rubric-based read before submission, and lab leads who want consistent critiques of experiment design and claim scope. It is less suitable if you need verified checks of whether results are correct, if your artifact has not passed structural validation, or if you need a decision-grade judgment for grants, hiring or publication.
Download the skill