Guide · Agent skill
How to use the Scholar Evaluation agent skill: setup and examples
Updated · written with AI from the project's own documentation and checked against our catalogue data
The scholar-evaluation skill is a set of instructions and local tools that teaches an AI agent to review a scholarly work, such as a paper draft, research protocol, literature review, or research idea, in a qualitative, developmental way. It links each comment to specific evidence locations. This guide covers what the skill does, how to install it, how to trigger it, and where its limits are, so you can use it for feedback on your own work or for a low-stakes review process you are designing.
Scholar EvaluationK-Dense-AI/scientific-agent-skills · MIT Download the skill Quick start
- 1Download the ZIP. Get the scholar-evaluation ZIP from this skill's page on the directory.
- 2Unzip into a skills folder. Place the folder in ~/.claude/skills/ for personal use, or in a project's .claude/skills/ folder.
- 3Or upload it in the Claude apps. Go to Settings, then Capabilities, then Skills, and upload the skill.
- 4Make a natural request. Describe the scholarly work and ask for developmental, evidence-linked feedback.
- 5Provide authorized context. Answer the agent's questions about work type, stage, discipline and the authorized source location.
- 6Run optional checks. With Python 3.11+, run the documented python3 scripts against your local JSON or CSV files.
What is an agent skill, and what does this one do?
An agent skill is a folder of instructions and optional files that an AI agent loads when a task matches its description. This skill is named scholar-evaluation and is defined by a SKILL.md file.
It makes the agent give qualitative, evidence-traceable feedback on a scholarly work, with optional scores that only describe how submitted evidence maps to a predeclared bounded rubric. It can also audit whether a low-stakes assessment process documents its construct, rater quality, uncertainty, privacy and human governance.
- Developmental feedback on a paper, draft, protocol, literature synthesis or research idea
- Checking a low-stakes rubric against bundled templates
- Running local checks on rubric, evaluation and evidence files
Install the scholar-evaluation skill
Download the ZIP from the skill's page on this directory and unzip it. For a personal setup, place the folder in ~/.claude/skills/. For a single project, place it in that project's .claude/skills/ folder.
You can also upload the skill in the Claude apps. Open Settings, go to Capabilities, then Skills, and upload it there.
- Personal install: ~/.claude/skills/scholar-evaluation
- Project install: .claude/skills/scholar-evaluation inside your project
Trigger the skill with a natural request
You do not need a special command. Describe the work and what you want, and say that you want developmental feedback. The skill's description covers reviewing scholarly works and auditing low-stakes rubrics, so requests in those terms are the ones it is built for.
The agent will ask for or record the context it needs before rating anything, such as the work type, stage, discipline, audience and the authorized source location.
- Give me developmental feedback on my draft paper and point to the evidence for each comment.
- Review this research protocol. Tell me where the claims lack supporting evidence.
- Check whether our low-stakes peer-feedback rubric documents its construct, rater training and privacy handling.
Run the bundled local checks (optional)
The optional scripts require Python 3.11 or newer and use only the Python standard library. They read local JSON or CSV files and do not use the network. Run them with python3 from the skill folder. The skill's documentation shows these commands:
- Validate a rubric: scripts/validate_rubric.py with --rubric assets/rubric_template.json
- Calculate bounded scores: scripts/calculate_scores.py with --rubric and --evaluation
- Check evidence traceability: scripts/check_traceability.py with --rubric, --evaluation and --evidence
- Summarise rater agreement: scripts/summarize_agreement.py with --rubric and --ratings
PYTHONDONTWRITEBYTECODE=1 python3 scripts/validate_rubric.py --rubric assets/rubric_template.jsonUnderstand the limits before you start
The skill sets a hard boundary. It must not be used to automate, recommend or score hiring, promotion, tenure, admissions, grants, awards, discipline or other high-impact personnel decisions. It also does not rank people or issue publication-readiness or top-tier judgments.
The skill rejects proxy indicators such as journal impact factor, h-index, citation counts and venue prestige as quality signals. The bundled reference notes that the underlying ScholarEval research is experimental and not validated psychometrics, and the template records content validity as not established.
- Do not put private applications, CVs, letters, reviewer identities or contact details into inputs
- Allowed data classifications: synthetic, public_scholarly_work, deidentified_low_stakes
- No script searches the web, reads credentials, calls a model or launches a process
Safety notes on scripts and network use
The bundled scripts are local and deterministic. The skill's metadata states that it uses no network, credentials, external models or subprocesses. The scripts reject symbolic-link inputs, duplicate JSON keys and unknown fields, and they refuse to overwrite output by default.
Because the scripts run on your machine, read the SKILL.md and the scripts folder before running them, and only run them on data you are authorised to use.
- Inputs are limited to 2 MiB and outputs to 2 MiB
- Output reports are minimised and do not copy evidence excerpts or raw source documents
What people use it for
- Getting structured feedback on a draft manuscript before submission, with each comment tied to a location in the text
- Checking whether the evidence behind a claim in a literature review is observed, interpreted or missing
- Learning how to evaluate the evidence supporting a scholarly claim as a student
- Auditing a low-stakes, developmental peer-feedback rubric for documented construct, rater training and privacy handling
Frequently asked questions
Is the scholar-evaluation skill free to use?
The skill is published under the MIT licence, as stated on its page. Check the licence file in the downloaded ZIP for the full terms.
What do I need to run it?
The agent skill itself needs no special setup beyond a compatible AI agent. The optional bundled scripts need Python 3.11 or newer and use only the standard library. They make no network calls.
Where do I put the skill folder?
Use ~/.claude/skills/ for a personal install or .claude/skills/ inside a project. You can also upload it in the Claude apps under Settings, then Capabilities, then Skills.
Can I use it to help decide hiring, funding or admissions?
No. The skill explicitly prohibits use for hiring, promotion, tenure, admissions, grants, awards, discipline and other high-impact personnel decisions, and the tools are not validated for person assessment.
Does it tell me whether my paper is publishable?
No. The skill does not issue publication-readiness, accept or reject, or top-tier judgments, and it rejects journal impact factor and venue prestige as quality signals.
Why does the agent refuse some requests?
If a request crosses the safety boundary, the skill tells the agent to stop and offer developmental comments on a scholarly work or a process-only audit instead.
New agent skills, repos and AI tools — every week
The best new skills and open-source projects, the fastest-growing repos, and one free prompt. Five minutes to read. Unsubscribe anytime.
Our verdict
This skill suits researchers who want structured, evidence-linked feedback on a draft or protocol, students learning to check evidence behind claims, and groups designing a low-stakes developmental rubric who want a governance checklist. It is the wrong choice for hiring, funding, admissions, publishability decisions or any ranking of people, which the skill itself forbids.
Download the skill