AGENT SKILL EVALUATION

skilledBetter skills start with evidence.

Test what your agent should do. Inspect what it actually did. Improve the Skill and check the result.

Run your first check

Connect your agent

macOS · Linux · Windows Runtime included

skilled / your projectWORKFLOW

Define the expectation

One Skill. A set of repeatable cases.

skilled validate evals/eval.yaml

Run and inspect

Review verdicts alongside the evidence.

skilled run evals/eval.yaml

Revise, then compare

Let the next run show what changed.

Start without model credentials

Validate Skill files and evaluation cases, then preview a run before connecting an engine.

Try a complete example

Keep the evidence

Inspect case verdicts and outputs in HTML. Use JSON for tooling and JUnit in your CI pipeline.

Explore the reports

See what changed

Compare saved runs for regressions, score changes, and coverage differences. No additional model call needed.

Compare two runs

Improve with your agent

The bundled skilled-improve workflow helps your agent diagnose failures, revise a Skill, and rerun the cases.

Connect through MCP

Choose your evaluation

Define cases in YAML and grade them with assertions, scripts, or a model judge. Reuse existing evals.json suites.

Write evaluation cases

Use your engine and runtime

Run a supported agent engine on the host, in Docker, or on a configured OpenSandbox server.

Configure your engine

A CLEAR DIVISION OF WORK

One installation. Two parts that work together.

Component	Responsibility	Where you use it
skilled	Validate, evaluate, and compare	Your terminal or an MCP-compatible agent
skilled-improve	Guide diagnosis, revisions, and regression checks	A bundled workflow your agent follows

skilled-improve is a prompt and workflow, not a separate executable. Your agent uses it with the evaluation tools; you keep the cases and evidence in your project.

YOUR FIRST STEP

Install once. Start with a local check.

The release bundles its own Node runtime and application dependencies. No source build or GitHub account is required.

macOS / LinuxWindows

curl -fsSL https://raw.githubusercontent.com/DeadHead04/skillEd/main/install.sh | bash
export PATH="$HOME/.local/bin:$PATH"
skilled --version

powershell

$installer = Join-Path $env:TEMP 'skilled-install.ps1'
Invoke-WebRequest https://raw.githubusercontent.com/DeadHead04/skillEd/main/install.ps1 -OutFile $installer
& $installer
$env:Path = "$(Join-Path $HOME '.local/bin');$env:Path"
skilled --version

Keep the PATH entry in your shell or user settings for future sessions. Installation options, updates, and uninstall →

Next, create the Hello World example and validate it without calling a model. For a live evaluation, install and authenticate your selected engine first; provider usage may incur charges.

A REPEATABLE PROCESS

From an expectation to an improvement

01
Define
Write the behavior you expect as versioned cases.
02
Evaluate
Run the engine and inspect the report and outputs.
03
Improve
Have your agent revise the Skill using failed cases.
04
Compare
Rerun the suite and check what improved or regressed.

Comparisons use the reports you supply. Background drift monitoring and unrelated conversation capture are not enabled by installing the CLI or connecting MCP. Understand the comparison model →

OPEN SOURCE · TYPESCRIPT · APACHE-2.0

Try it on one Skill.

Start small, keep the report, and make the next change measurable.

Follow the first-evaluation guide → · Read the verification scope · Browse the source