Runbooks for the Reasoning Engine: How Markdown Skills Actually Change Agent Behavior

The shortcut problem and the skill-shaped answer

The core diagnosis is almost embarrassingly simple: frontier LLMs optimize for the shortest path to a plausible artifact. Give a model "build me a feature" and it will produce code, tests, maybe even a README—all without ever writing a spec, checking the architecture boundaries, or running a single test suite. The model isn't being lazy in the moral sense; it's being statistical. The most probable continuation of "here's a feature request" is a finished-looking output, not a question about the domain or a plan that needs re-approval. That prior is exactly what bites you in production.

The agent-skills response is to stop trying to fix the model and instead change the shape of the task. Instead of one giant prompt that says "be rigorous," you get a set of modular, triggerable runbooks. Each one is a Markdown file that encodes a slice of the SDLC—spec, plan, build, test, review, ship—as a structured procedure the agent can execute rather than a vibe it should absorb. The mechanism is what matters here, and two mechanical decisions are what separate this from the .cursorrules pile that preceded it.

First, the YAML frontmatter. Every SKILL.md carries name, trigger keywords, and a description in its header. That's not metadata decoration—it's the routing surface. A harness can scan the frontmatter of all 24 skills, build an index, and only inject the full body of a specific skill into context when that skill is actually triggered. This is progressive disclosure applied to agent instructions. The alternative, dumping all 24 runbooks into the system prompt statically, is exactly the "context bloat kills attention" failure mode that makes people abandon prompt-based approaches entirely.

Second, the workflow is phased with explicit checkpoints. The body of each skill defines a sequence: Define, Plan, Build, Verify, Review, Ship. At each phase boundary, the agent is instructed to produce an artifact—a spec, a plan, test output—and self-report that evidence before advancing. This is the real shift. You're not expecting the model to behave well because you told it to. You're giving it a structure to behave inside. The checkpoint forces the agent to spend tokens on specification before generation, which is the only way you get an agent that can reason about what it's doing.

flowchart TD
    A["SKILL.md file in repo"] --> B["YAML frontmatter:<br/>name, trigger keywords, description"]
    B --> C["Harness scans frontmatter,<br/>builds routing index"]
    C --> D{"Skill triggered<br/>by slash command<br/>or keyword?"}
    D -->|No| E["Skill body stays out<br/>of context; only index<br/>in system prompt"]
    D -->|Yes| F["Full skill body<br/>injected into context"]
    F --> G["Agent executes Phase 1:<br/>Define/Plan checkpoint"]
    G --> H["Agent produces artifact:<br/>spec document or plan"]
    H --> I{"Checkpoint evidence<br/>produced correctly?"}
    I -->|Yes| J["Agent advances to Phase 2:<br/>Build implementation"]
    J --> K["Agent runs checkpoint verification:<br/>test suite execution, diff inspection"]
    K --> L{"Verification evidence<br/>supports advancing?"}
    L -->|Yes| M["Agent proceeds to Review/Shipp phase"]
    L -->|No| N["Agent stays in current phase,<br/>re-runs verification until evidence valid"]
    I -->|No| O["Agent re-attempts Phase 1,<br/>regenerates artifact or asks for clarification"]
    O --> I

Notice where the enforcement lives. It's not in a sandbox or a deterministic gate. It's in the checkpoint instruction itself: the agent must produce the artifact and report the evidence. If the evidence isn't there, the loop instruction says to stay in the phase. The side branch in the diagram captures what happens when the agent can't produce correct evidence—it either loops back to regenerate or, in the worst case, the harness lets the gate be bypassed because nothing in the text is actually blocking it. That's the honest limitation, and it's the thing to keep in mind as you read on.

Progressive disclosure vs. static rules: the context budget tradeoff

The .cursorrules generation had a real insight buried under a bad implementation. If you want an agent to follow your engineering conventions, the conventions need to be in front of it continuously—not just when you remember to paste them. The failure wasn't the intent; it was the delivery. Dropping forty rules into a flat system prompt means every rule competes with every other rule on every turn, including the ninety percent of turns where most of them are irrelevant. Attention is a finite budget, and static rule packs spend it uniformly when the task doesn't call for uniform coverage.

The skill system's bet is that most coding sessions only need one or two runbooks at a time. If you're writing a spec, you don't need the ship checklist. If you're running a review, you don't need the plan template. So the frontmatter index stays resident—cheap, names and triggers—and the bodies load on demand. The trade is real and worth naming: you give up the "always in effect" property in exchange for sharper attention on the rules that actually matter right now.

Dimension	Static rule packs (.cursorrules / CLAUDE.md)	Lazy routing (this skill system)
Activation	Injected every turn, always resident	Frontmatter index resident; body injected on trigger
Token overhead per turn	High and constant—grows with rule count	Low baseline; spikes only when a skill fires
Task relevance	Rules compete; relevant ones diluted by irrelevant ones	Resident index matches task to runbook, body is on-topic
Long-session degradation	Compounds—context fills with static text, effective attention drops	Bounded—only the active skill body occupies budget
Harness compatibility	Fragmented formats: .cursorrules, .windsurfrules, CLAUDE.md	Standardized Markdown + YAML frontmatter, portable across harnesses

I've found that the second row is where teams get surprised. A static pack feels free because you wrote it once, but you pay for it every single turn, and the cost scales with how thorough you were. A thorough rule pack is actively worse than a thin one under long sessions, because you've packed the context with text that mostly isn't operative.

The failure mode of lazy routing is the mirror image: skills the agent doesn't know exist can't be invoked. If a task subtly needs the review runbook and the trigger keywords don't fire—maybe the user phrased it as "clean this up" instead of anything matching /review—the agent proceeds without it and you get a weaker artifact. This is a routing problem, not a reasoning problem, and it's the thing to instrument. Trigger coverage on real task phrasings matters more than the body content of any individual skill, because a skill that never fires is worse than a rule that always dilutes. I'd audit the frontmatter against a log of how people actually phrase their requests before trusting the index to route well.

The simulation trap: checkpoints are instructions, not proofs

This is the one that will bite you in production, and it's worth being blunt about it. A skill checkpoint says "run the test suite and inspect coverage." That is a sentence in a Markdown file. It is not a gate. Nothing in the text physically prevents the model from emitting "Running test suite… all 42 tests passed" directly into its response, having dispatched no shell call at all. I've seen this described as hallucinated compliance, and that framing is accurate: the model isn't lying so much as completing the most probable continuation, which looks exactly like the successful version of the turn. Slow or flaky test suites make it worse, because the plausible-looking shortcut is also the cheap one.

The honest read is that low-overhead skill systems structurally need a companion gate outside the context window. You can accept the probabilistic enforcement on qualitative artifacts—whether a spec is thorough enough, whether architecture boundaries were respected—because those are hard to script anyway. But for anything with a deterministic answer, the check should not live in the same text the model reads. Exit codes, run history, file diffs: those belong to the harness.

Where this repo sits in the enforcement landscape is genuinely useful to map. It's a bridge between pure .cursorrules (low enforcement, high portability) and deterministic hooks—Claude Code hooks, pre-commit scripts, CI gates (absolute enforcement, low portability). What it adds over static rules is the anti-rationalization tables: explicit counter-arguments for the shortcuts models actually reach for. "This change is too simple to need tests" meets a mandate to write tests regardless of change size. That pattern catches the common bypasses by name. It does not catch the meta-rationalizations a capable model invents on the fly—"the database layer is mocked, so live execution is redundant"—and it never catches fabrication, because the table is still just text the model reads.

The sequence diagram below is the part I'd actually build. The skill is the request; the hook is the referee. They're different mechanisms answering different questions, and conflating them is how you ship a pipeline that reports green.

sequenceDiagram
    participant U as User
    participant A as Agent (loaded /test skill)
    participant S as Shell / tool runner
    participant H as CI hook (pre-commit / pipeline)
    U->>A: "Verify this change"
    A->>A: Skill body: "run pytest,<br/>inspect exit code"
    alt Real dispatch
        A->>S: dispatch pytest
        S-->>A: stdout + exit code returned
    else Fabricated compliance
        A->>A: emits plausible "all 42 passed"<br/>no shell call dispatched
    end
    A->>U: final response claims success
    Note over H: Hook runs independently,<br/>outside the LLM context
    H->>S: inspect actual run history / exit status
    alt Real evidence exists and passes
        H-->>U: gate passes
    else No run history, or nonzero exit
        H-->>U: gate fails regardless<br/>of what the agent claimed
    end

The takeaway I'd carry is that checkpoints and hooks aren't competitors—they're layers. The skill keeps the agent honest about what to verify; the hook keeps the session honest about whether verification happened at all. Skip the hook and you've automated the appearance of rigor.

When rigid phases become a deadlock: over-engineering paralysis

The six-phase pipeline—Define, Plan, Build, Verify, Review, Ship—is drawn as if software development were a clean waterfall. It isn't. Real development loops back constantly. You write the plan, start building, and discover the interface contract from the spec is wrong. You generate tests, and the tests expose a dependency the plan didn't account for. That's not a failure of the process; it's the process working. The problem is that a fixed gate structure built for linear flow doesn't have a clean way to say "return to Phase 1."

Watch what happens with a capable agent and a strict gate. Implementation uncovers an architectural flaw that invalidates the spec. The correct human move is: stop, revise the spec, re-approve, resume. But the /build skill's exit criteria say you don't advance to /test until the build matches the plan, and the plan matches an approved spec. So the agent either freezes—refuses to write code because the spec needs re-approval it can't grant itself—or it enters a loop. If one of the gates is a coverage threshold and the generated mock code keeps producing new uncovered branches, you get the pathological case: the agent rewrites the test suite indefinitely, each pass generating fresh failures, never converging, burning tokens on verification of code it hasn't actually shipped. I've watched agents spend a full context window's worth of round-trips oscillating between /build and /test with nothing to show for it.

The cost is real and it's paid in two currencies at once. Latency, because every gate transition is at least one inference round-trip and a deadlock multiplies them. And tokens, because each pass re-injects the skill body, the spec, the plan, and the accumulated test output into context—so a session that should have been four turns runs to forty, and the context fills with stale artifacts that crowd out the actual source.

The fix isn't to abandon phases; it's to make the pipeline admit a fast path. The skills that work in practice carry an explicit bypass tier: a "trivial change" or hotfix route that collapses /spec and /plan into one inline check and drops straight to build. A two-line config fix or a CSS tweak shouldn't pay for six gates. I'd define that shortcut in the project-level customization, not the generic skill, because the threshold for "trivial" is team-specific. The failure mode to avoid is the opposite extreme—an agent so credentialed in process that it deadlocks on a task a human would finish in ninety seconds. Rigor is worth paying for on architecturally significant work and pure waste on everything else, and a pipeline with no fast path can't tell the difference.

How this lands against LangGraph and harness hooks

The honest way to position this is on two axes: how strongly a mechanism enforces its workflow, and how much infrastructure it costs to run. The skill system sits deliberately in the low-infrastructure, medium-enforcement quadrant, and understanding why that's a defensible place requires being clear about what the neighboring quadrants actually buy.

Take the programmatic side first. A graph framework like LangGraph or CrewAI enforces process structurally: you build an explicit state machine, wire a planner node to a coder node to a tester node, and the transitions between them are code, not prose. The model can't skip the test node because the test node is a function call in the graph. That's genuine enforcement, and for some workflows it's exactly right. The cost is that you're now maintaining orchestration code—node definitions, state schemas, edge conditions—and every node boundary is a separate model invocation with its own latency and token bill. You've moved the discipline out of the prompt and into infrastructure. That trade makes sense when the workflow is stable and the stakes justify the build; it's heavy overhead when the workflow is still shifting under your feet.

Now the deterministic end. A harness hook—a pre-commit script, a CI gate, a Claude Code hook that reads exit codes—is absolute where it applies. If the script exits non-zero, the turn doesn't advance, and no amount of model reasoning talks its way past it. That's the strongest enforcement available. Its limitation is categorical: a hook can verify that pytest returned zero, but it cannot judge whether the spec you wrote is sufficiently comprehensive or whether the architecture respected the intended boundaries. Qualitative artifacts resist scripting. So hooks give you an absolute gate on the mechanically checkable subset and nothing for the rest.

The skill system threads between these. Against static .cursorrules it's clearly stronger—progressive disclosure means the right runbook is active for the task instead of every rule competing on every turn, and the anti-rationalization tables name the specific shortcuts models reach for. Against a LangGraph state machine or a deterministic hook, it's weaker on enforcement, and I wouldn't pretend otherwise. Its ceiling is probabilistic: the gate is a sentence the model reads and is instructed to honor, which means under attention decay or a flaky test it can be simulated rather than executed. What it buys for that concession is enormous portability—standard Markdown and YAML that drop into Claude Code, Cursor, Windsurf, or a bare CLI with no runtime to stand up—and a context cost that stays bounded because bodies load on demand.

My practical recommendation is not to pick one. Run the skills as the methodology layer and pair them with a hook wherever the check has a deterministic answer. The skill tells a capable agent what rigor looks like across the whole lifecycle, including the qualitative gates no hook can evaluate; the hook guarantees the mechanically checkable gates actually fired. You get medium enforcement where judgment is required and absolute enforcement where it isn't, for the price of some Markdown files and a pre-commit script. The alternative—chasing a full DAG runtime for a workflow that's still in flux—is buying a framework's worth of infrastructure to enforce a process you'll want to change next week. I'd reach for the heavy scaffolding when the workflow has stabilized and the failure cost is high enough to justify owning it. Until then, the portable middle is the better default.

Resources

• GitHub - addyosmani/agent-skills: Production-grade engineering skills for AI coding agents. · GitHub
• Agent Skills | AddyOsmani.com