Intercepting Coding Agents Mid-Flight: An Architectural Breakdown of In-Flight Semantic Review

The In-Flight Evaluation Gap in Autonomous Coding

When autonomous coding agents drift off course, they rarely fail by producing invalid syntax; modern models generate clean, parseable code with ease. The real failure mode is semantic compounding. An agent establishes a flawed domain abstraction in step one—such as introducing a primitive string where a validated domain value object was required—and by step twelve, it has authored four hundred lines of scaffolding, adapters, and unit tests wrapped entirely around that broken premise.

Our existing engineering safeguards are poorly structured for this failure pattern. On one extreme, we rely on passive steering files like AGENTS.md or .cursorrules. I view static prompt instructions as necessary baselines, but as context windows fill with terminal outputs and conversational turns, models suffer from severe instruction drift. Advising an agent to adhere to structural invariants is fundamentally different from validating that it did. On the other extreme, we run deterministic linters and language servers like Semgrep or pyright. These tools execute in milliseconds, but their symbolic AST checks struggle with ambiguous domain semantics—such as recognizing that a newly added function violates an unstated architectural invariant.

quadrantChart
    title Developer Feedback Mechanisms
    x-axis Deterministic Symbolic --> Semantic Heuristic
    y-axis Post-Run CI Latency --> In-Flight Real-Time
    quadrant-1 In-Flight Semantic
    quadrant-2 In-Flight Deterministic
    quadrant-3 Post-Run Deterministic
    quadrant-4 Post-Run Semantic
    Semgrep and LSPs: [0.15, 0.85]
    Hapsland: [0.85, 0.85]
    Passive Agent Prompts: [0.65, 0.35]
    CodeRabbit and PR Bots: [0.85, 0.15]

Most teams attempt to catch semantic debt at the pull request boundary using tools like CodeRabbit or bespoke CI review pipelines. As I explored when analyzing multi-tier review architectures for agent-authored code, post-hoc evaluations arrive far too late in the generation cycle. Catching an architectural error after an agent has committed its changes forces a massive, token-expensive backtracking loop. The agent must dismantle several files of dependent implementations, re-read the entire context, and attempt a second pass while carrying the cognitive baggage of its prior failure.

This creates an in-flight evaluation gap. We need a mechanism that intercepts edits as they occur on the filesystem, evaluates domain invariants via an external heuristic model, and prunes flawed mental models before dependent code is committed. This is the design space targeted by Hapsland (dearlordylord/hapsland). By intercepting edits mid-flight, it shifts semantic review from a post-generation audit into an active execution gate, altering the cost curve of autonomous refactoring before token debt compounds.

How Hapsland Traverses Cross-File Definitions Without Context Dumps

The core technical challenge of in-flight review is assembling sufficient context without overwhelming the evaluator. If you pass an isolated git diff hunk to an external LLM classifier, the model lacks the semantic awareness to know whether a method signature conforms to the broader repository's domain model. Conversely, dumping entire files or concatenating project modules bloats prompt sizes, introduces latency, and leaks irrelevant noise that degrades classifier accuracy.

Hapsland resolves this tension through selective reference harvesting. When an agent touches a file, Hapsland's interceptor parses the local syntax tree, extracts the specific declarations altered by the edit (such as modified structs, functions, or type signatures), and initiates a targeted AST traversal across neighboring files. It follows symbol references, caller-callee bindings, and imported type constraints, extracting only the relevant declarations into a compact semantic bundle. Neither user task descriptions nor conversation histories are bundled; the evaluation payload contains strictly the diff, its cross-file dependency slice, and the declarative rule being tested.

sequenceDiagram
    autonumber
    participant Agent as Coding Agent (Claude Code / Codex CLI)
    participant Interceptor as Hapsland File Interceptor
    participant Walker as AST Reference Walker
    participant Classifier as External Classifier (Jev)

    Agent->>Interceptor: Writes native file modification
    Interceptor->>Walker: Inspects diff & extracts modified AST symbols
    Walker->>Walker: Traverses import graph for referenced definitions
    alt Missing or Inaccessible References
        Walker-->>Interceptor: Reference resolution failed
        Interceptor-->>Agent: Defensive rule skip (No action)
    else Context Successfully Bound
        Walker-->>Interceptor: Return bounded semantic slice
        Interceptor->>Classifier: Batched review payload (Diff + References + Rule)
        Classifier-->>Interceptor: Evaluation verdict & actionable critique
        Interceptor->>Agent: Inject targeted critique directly to editing subagent
    end

The critical engineering tradeoff in this pipeline is Hapsland's handling of unresolved references: if an imported type or dependency path cannot be cleanly resolved, the engine defensively skips the rule rather than guessing. From an operational perspective, this is a double-edged sword. Defensive skipping prevents the external classifier from hallucinating structural critique against incomplete code, which could otherwise send the agent into a corrective tailspin.

However, in dynamically typed codebases or code featuring runtime dependency injection and reflection, static graph traversal frequently fails to locate declarations. In practice, this design can produce silent rule blindness: an agent might commit a severe architectural error, yet no warning is raised simply because the reference walker hit an extraction boundary and skipped the assertion. For teams maintaining strict domain models, designing capture boundaries that respect these traversals is essential to keeping the semantic slice grounded.

Routing Critiques to Subagents Without Poisoning Context

In advanced agentic environments like Claude Code or Codex CLI, code generation is rarely handled by a single monolithic model. Instead, a root orchestrator decomposes a specification and delegates implementation tasks to specialized subagents. A common anti-pattern in developer tooling is broadcasting all warnings, lint failures, and review feedback to the root orchestrator's primary session. Flooding the top-level context with low-level semantic critiques rapidly depletes its attention budget, diluting its high-level strategic roadmap with granular implementation noise.

Hapsland avoids this through runtime identity tracking. By hooking into the native execution primitives of Codex CLI and Claude Code, the interceptor resolves the process metadata of the specific subagent responsible for an individual file write. When the external classifier (such as "Jev") returns an actionable critique, that feedback is injected directly into the active prompt loop of the authoring subagent, leaving the parent orchestrator unburdened.

To keep this real-time loop practical, the engine implements two crucial flow controls: batched dispatch and stale-review discarding. Calling an external classification model on every raw keystroke or discrete file flush would introduce unacceptable latency and inference overhead. Hapsland batches candidate edits into discrete semantic review payloads. More importantly, it continuously reconciles pending reviews against filesystem state. Because external LLM calls typically require several seconds to resolve, an aggressive coding agent may continue writing new code in subsequent steps while an earlier review is in flight.

If the file region under review is modified again before the classifier returns its verdict, Hapsland marks that review as stale and discards it immediately. This discard rule prevents an agent from receiving out-of-date critiques that address code it has already refactored, which would otherwise trigger catastrophic corrective thrashing. However, this creates a subtle operational risk: if an agent writes flawed code in one method and immediately moves to edit another section of the same file, the original flaw's review can be discarded before the critique ever lands. Balancing classifier latency against agent velocity remains the central operational tension of running semantic reviews directly inside the authoring loop.

Architectural Showdown: In-Flight Interception vs LSPs, Guardrails, and PR Bots

Choosing where and how to evaluate code generated by autonomous agents is fundamentally an optimization problem across latency, context breadth, and token expenditure. I evaluate tooling across three primary control boundaries: the local editor/OS boundary, the agent framework loop, and the post-generation repository gate. Hapsland introduces a hybrid paradigm that alters these mechanical tradeoffs.

Tooling Class	Execution Hook Point	Analysis Engine	Context Assembly Mechanism	Token Cost	Feedback Latency
Hapsland	In-flight filesystem interception	External heuristic LLM classifier (e.g., Jev)	AST-driven symbol reference walker stitching cross-file dependencies	Variable (dual-inference billing per candidate edit)	Low-to-medium (1–5s, batched asynchronously)
Semgrep / LSPs	In-editor local save / keystroke	Deterministic AST matching, control-flow graphs, symbol tables	Monolithic project index or local AST symbol resolution	Zero (local compute)	Near-instantaneous (sub-second millisecond execution)
Runtime Guardrails (LangGraph / LlamaIndex)	Step-level agent execution boundary	Pydantic schemas, structural validators, or small evaluator models	Local execution step payload and immediate tool outputs	Low (predictable validation calls)	Immediate per agent step (synchronous blocking)
Post-Gen PR Reviewers (CodeRabbit / PR-Agent)	Git push / pull request creation	Multi-model LLM ensembles with static analysis tooling	Full git diff, repository vector search, and PR metadata	High (broad context prompts over full diffs)	Asynchronous batch (minutes to hours post-run)

Deterministic linters like Semgrep and language LSPs remain irreplaceable for verifying syntax, catching known security anti-patterns, and enforcing type correctness. Their execution is instantaneous and costs zero tokens. However, they are blind to architectural hygiene. A language server will happily accept a function that replaces a strong domain entity with raw string primitives as long as the types check out.

Framework-level guardrails, such as schema validators in LangGraph or LlamaIndex, operate synchronously at the step boundary. They excel at ensuring that an agent's tool call matches a strict schema before execution, but they lack awareness of repository-level code aesthetics or broader file interactions.

Post-generation PR review bots sit at the opposite extreme. Because they run asynchronously against the complete pull request, they possess wide architectural context across all modified files. Yet their intervention comes too late. If an agent spent fifteen minutes writing three hundred lines of domain logic atop a flawed abstraction, catching it on GitHub means forcing the agent—or a human—into a costly rewrite loop.

Hapsland stakes its value on occupying the gap between instantaneous deterministic checks and delayed PR reviews. By splicing cross-file references into a targeted semantic payload, it grants an external LLM sufficient architectural depth without paying the token penalty or waiting for full CI runs. The cost of this position, however, is a dual-inference overhead: you pay token charges not only for the agent writing the implementation, but also for the classifier scrutinizing every intermediate save.

Runtime Fragilities: Race Conditions, Dynamic Graphs, and Silent Blindness

Moving semantic review directly into the authoring loop introduces severe operational fragilities that do not exist in deterministic linters or batch CI jobs. In production, I watch for three distinct breakdown modes: timing races against agent velocity, static traversal limits in dynamic codebases, and global configuration invalidation.

stateDiagram-v2
    [*] --> EditIntercepted: Native File Write
    EditIntercepted --> ReferenceTraversal: Extract Symbols
    ReferenceTraversal --> RuleSkipped: Unresolved Reference Graph
    RuleSkipped --> [*]
    ReferenceTraversal --> BatchedForClassifier: Context Bound
    BatchedForClassifier --> InFlightEvaluation: Dispatch Payload
    InFlightEvaluation --> StaleCheck: Verdict Returned
    StaleCheck --> Discarded: File Buffer Modified
    Discarded --> [*]
    StaleCheck --> FeedbackInjected: File Buffer Valid
    FeedbackInjected --> [*]

The most frequent mechanical failure mode stems from the velocity mismatch between agent execution and classification latency. An external classifier round-trip typically takes between one and five seconds. High-velocity agents like Claude Code frequently emit consecutive edits in rapid succession. To prevent injecting obsolete critiques, Hapsland marks in-flight reviews as stale and discards them if the underlying file region changes before the review resolves.

While discarding stale payloads prevents corrective thrashing against obsolete code, it creates an operational blind spot. If an agent introduces an architectural flaw in step N and immediately modifies adjacent lines in step N+1, the review for step N is dropped. The flawed mental model survives uninspected simply because the agent moved faster than the classification endpoint. Mitigating this requires strictly bounding rule execution to dedicated, low-latency classifier endpoints; standard SaaS model APIs with unpredictable queue times trigger unacceptable drop rates.

The second fragility is silent rule blindness caused by static graph resolution limits. When an agent edits typed code with explicit imports, the AST reference walker effortlessly discovers caller-callee hierarchies and type signatures. But the moment an agent operates in dynamically typed environments—or uses runtime dependency injection, reflection, or dynamic imports—the reference walker fails to resolve the symbol graph.

Because Hapsland's contract defensively skips any rule whose required context cannot be fully resolved, the engine will silently pass the edit without raising an alert. Teams assuming their domain rules are being actively enforced can easily accumulate architectural debt, oblivious to the fact that the walker hit an extraction boundary and skipped the assertion entirely.

Finally, adopting teams face a severe configuration gotcha. Under Hapsland's review contract, rule configuration validation is strictly all-or-nothing:

Invalid selected configuration fails before source capture, even when other selected rules are valid; it cannot fall back to that valid subset.

If a team member commits a minor syntax error or an unrecognized property into a single JSONC rule document, the review engine does not gracefully degrade to run the remaining valid rules. It halts capture across the entire repository. In a collaborative monorepo where multiple teams maintain domain rules, a single malformed configuration commit completely disables in-flight semantic enforcement for every developer and agent operating in that codebase. Any production deployment must treat rule configurations as critical infrastructure, validating them against strict schemas in local pre-commit hooks and pre-merge CI pipelines.

The Data Governance Tradeoff: Sanitized Prompts vs Leaked Source Invariants

Hapsland's privacy contract presents an appealing surface story: conversational histories and prompt instructions are stripped from evaluation dispatches. Only the raw code diff, extracted reference definitions, and declarative rule questions travel over the wire to external review endpoints like Jev. For teams concerned about exposing high-level product specs or conversational instructions, this boundary feels disciplined.

In enterprise codebases, however, this architecture trades one exposure vector for a far more insidious one. By design, the reference walker traverses imported symbols, caller-callee trees, and cross-file interfaces to assemble its semantic context. When an agent touches an interface adapter, the walker pulls referenced entity schemas, domain constants, and adjacent business logic into the evaluation payload. The engine's documentation states plainly that some external classifiers retain transmitted data and reserve arbitrary usage rights. If an adjacent file contains internal valuation algorithms, proprietary data structures, or hardcoded developmental credentials, that source text is automatically bundled and exported off-premise without manual inspection.

┌────────────────────────────────────────────────────────┐
│ Agent Workspace Context (Local Boundary)               │
│ [Conversational History & Prompts] ──(SCRUBBED)──┐     │
│ [Modified Source Function] ───────────┐          │     │
│ [Cross-File Reference Schemas] ───────┼───► [Evaluator Payload] ──► Third-Party Endpoint
│ [Adjacent Business Logic & Keys] ─────┘          │                   (e.g., Jev SaaS)
│ [Agent-Generated Code Comments] ─────────────────┘                   * Data retention risk
└────────────────────────────────────────────────────────┘

A second vulnerability emerges from agent behavior itself. While Hapsland scrubs the chat prompt, it evaluates whatever text the agent commits to disk. When models write implementation code, they routinely paste conversational context, spec snippets, and internal API shapes directly into file headers, docstrings, or inline test fixtures. If an agent echoes a proprietary prompt instruction into a block comment while scaffolding a test, that information bypasses the sanitization boundary and lands straight in the external classifier's log store.

To run this architecture safely, you cannot rely on automated stripping alone. Teams must define strict privacyExcludes patterns, walling off configuration blocks, secret stores, and core IP schemas from the reference crawler. Relying on default extraction boundaries against multi-tenant third-party backends invites data leakage through the very AST links meant to supply semantic grounding.

Practical Blueprints: Building In-Flight Semantic Verification Systems

Deploying in-flight semantic interception requires treating the evaluation loop as a high-throughput runtime compiler rather than a casual linting layer. The most critical operational mandate is establishing a rigorous pre-merge CI pipeline for your declarative rules. Because an invalid JSONC rule document causes the engine to halt capture across the entire repository rather than dropping the offending rule, rule repositories must be guarded by strict schema validation and automated test suites. A malformed regex or an unmapped rule property committed to main instantly disables real-time verification for every active agent and developer.

Observability must focus on defensive skipping rates. When a reference walker encounters unresolved dynamic imports or deeply nested types that exceed capture bounds, it silently passes the edit. If your telemetry tracks only passed and failed evaluations, you will miss instances where ninety percent of agent edits silently bypass checks due to broken traversals. Teams need explicit metrics that track the ratio of evaluated rules to skipped rules, with alerting configured to trigger whenever defensive skips spike after a codebase refactoring.

To avoid paralyzing coding agents, invariant sets must be calibrated around high-confidence structural boundaries. When an external classifier returns ambiguous or stylistic critiques, agents frequently enter an adversarial retry loop—repeatedly editing the same ten lines to satisfy an LLM evaluator rather than advancing the user's primary task.

For readers looking to build this pattern, several production configurations illustrate how to balance semantic enforcement against agent velocity:

If you are pursuing strict domain type enforcement for autonomous refactoring, you can wire Codex CLI or Claude Code edit hooks to an AST parser that inspects function signatures and parameter declarations. Before dispatching to an LLM, run a local Semgrep pre-filter to detect raw string or integer parameters at domain boundaries, forwarding candidate violations to a local quantized model that evaluates whether value objects are missing. The failure mode here is agent retry thrashing, where the model repeatedly wraps primitives in redundant boilerplate. Guard against this by enforcing a hard ceiling on consecutive critique turns and skipping validation entirely for internal serialization layers.

When architecting an invariant gateway for multi-subagent microservices, you can track Claude Code subagent processes to verify that database access patterns respect repository-pattern boundaries before edits compound into service handlers. Hooking a symbol crawler into project-level JSONC rule definitions allows you to resolve cross-module imports and dispatch them to a self-hosted classifier endpoint. The primary operational trap is stale-review discards triggered by rapid subagent writes; you can mitigate this by caching common query patterns and ensuring your local classifier endpoint keeps latency strictly under 800 milliseconds.

Finally, for regulated codebases requiring zero leakage, you can construct a hardened local reviewer that swaps external SaaS classifiers for an on-premise vLLM or Ollama instance hosting a distilled code model. Combine this with explicit privacyExclude configurations and a local audit log that captures the exact diff payloads assembled by the interceptor. The risk to monitor is silent rule blindness caused by overly broad privacy masks; if your exclude rules inadvertently mask shared interface definitions, the AST walker will fail to bind context and quietly skip your most critical security rules.

Resources

• GitHub - dearlordylord/hapsland: Realtime semantic review for coding agents · GitHub
• raw.githubusercontent.com