← Back to the library
Library methods

Skill evaluation

Design, run, and interpret evidence-based evaluations for an agent skill or workflow. Use for skill test cases, regression checks, baseline comparisons, qualitative review, benchmark design, trigger testing, and reliability claims. Keeps authored plans, executed results, and human approval distinct.

Works with the context you provideVersion 3.0.0Historical revision

# Skill evaluation

Use this method to determine what a skill demonstrably does under realistic conditions. It supports evaluation design, execution when the host allows it, and careful interpretation; it does not turn a few examples into a universal reliability claim.

Evaluation design

Define the intended behavior, prohibited behavior, test inputs, expected artifacts, and evidence needed to judge success. Include at least:

  • a normal case with sufficient context;
  • a missing or conflicting-information case; and
  • a boundary case that tests authority, safety, or an unsupported claim.

Write assertions that are observable and specific. Keep subjective quality review separate from objective checks.

Execution and comparison

Read `references/host-operations.md` before using a host runner, evaluator, artifact viewer, or benchmark tool. When an execution capability is unavailable or unauthorized, save an authored-not-executed evaluation plan instead of inventing outcomes.

For a change to an existing skill, compare the changed version with the prior version or another stated baseline under the same inputs. Preserve prompts, outputs, failures, invalid trials, retries, timings, and grading rationale. Use a fresh reviewer where an independent judgment matters.

Interpretation

Report what was measured, the sample and baseline, passed and failed checks, limitations, and the next improvement. Do not call a set untouched if it influenced design or selection. Do not infer acceptance from a lack of feedback or call a single reviewer independent when it was not.

Output

```markdown # [Skill] evaluation ## Scope and intended behavior ## Cases and assertions ## Execution record ## Results and evidence ## Limitations ## Recommendation ```

Keep artifact-writing, publication, and deployment separate from evaluation.