Skip to content

Verification evidence - 2026-10-31

Historical snapshot only. These results describe the recorded working tree and conditions, not current acceptance. See current verification.

Tool read ergonomics

A model reads an empty read page as a broken tool. read and read_file now carry a notice that names the actual situation: the file exists but is empty (0 bytes), the requested offset/start_line is at or past the end (with the real byte or line count), or content remains and where the next page starts. A page cut by the output budget says so and points at next_offset instead of ending on a bare truncated flag. read also reports estimated_tokens for the returned content, so the model can budget in the unit it is billed in.

FileReadOutputBudget now reserves 256 bytes for fields added after fitting, so notice and estimate cannot push the envelope past the 4 MiB cap. The ModelToolHeadroomRegressionTests case caught that before the reserve existed. The two changed tool descriptions updated the tool-contract golden lock deliberately (description SHA-256 rows for read and read_file).

Broker 1,445 tests passed for this change in isolation (FileReadAdapterTests, FileLineReadTests, headroom and tool-contract tests included).

Tool telemetry counters

The comparison with Claude Code's read tool showed a measured dedup win (18% of reads were same-file collisions). Ru now measures the same question before changing behaviour: the Broker source emits ru.tool.calls (tool.id, tool.status, error.code) and ru.tool.read_pages (tool.id, repeated). repeated is answered by a bounded per-service path tracker that never withholds content; a real "file unchanged" answer needs the Host's compaction-aware read state, so dedup stays a separate decision.

The Host's meter provider now adds the Broker source, without which the counters would be produced but never collected. Tags follow the existing allowlist: bounded tokens only, never paths or arguments. Two listener tests pin the first read (repeated=false), the second (repeated=true) and a failed call's status and bounded error code.

Model persona names the model, not the product

Asked which model it was, the model answered "Ru Agent". The cause was the first sentence of the core contract: "You are Ru Agent". The environment block already carried the serving model id, but it read as metadata next to a first-person identity claim.

The persona block now opens with that id - "You are , a coding model running inside Ru Agent" - states that Ru Agent is the host product and not the speaker's name, and requires model or version questions to be answered with the id from the environment block. The product name in the GUI and terminal is unchanged; only the model-visible identity moved. A test pins the identity sentence and that "You are Ru Agent" no longer appears in the contract.

Deterministic evidence

  • Full .NET matrix on the final tree: Broker 1.447, CLI 2.058, Client 288, Harness.Core 902, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1.612, gui-winui3 1.995 - 9,443 passed, 0 failed (artifacts/persona-matrix.log; the read-ergonomics and telemetry batches ran the earlier artifacts/agent-quality-matrix-3.log).
  • Python: 268 passed, 1 skipped (POSIX-only case).

Flaky and hung runs observed

The first full matrix failed one CLI test, TgSquareWordmarkRasterTests.Frames_use_two_buffers_and_do_not_redraw_or_allocate_during_quiet_time, an allocation-sensitive wordmark check unrelated to these changes. It passed alone (13/13 in the class) and in the final matrix, so the first failure is recorded as environment-sensitive rather than a regression.

A second matrix then hung in RuAgent.Cli.Tests with no output for ~18 minutes. Stale dotnet/testhost processes from the aborted first matrix were still alive; after they were stopped, the CLI project passed alone (2.058 tests, 3m28s) and the full matrix passed cleanly. The hang is recorded as process-state contamination from the aborted run, not a defect in the suite.

Not measured

No real-provider session was run for this change, so the counters have no production sample yet and dedup is not implemented. The read notices and token estimate are covered by unit tests, not by a live model reading a file.

Model-visible git_status tool

Handoff item 1 landed: git_status is a Broker-bound, read-only workspace.read tool. One fixed git status --porcelain=v2 --branch invocation runs through IGitCli and the existing GitStatusParser (no second parser), with status.relativePaths=true, --ahead-behind, --untracked-files=normal and a -- . pathspec, so a workspace that is a subdirectory of a larger repository reports only its own subtree with workspace-relative paths. The report separates staged/unstaged (porcelain code plus original_path for a rename), untracked and unmerged, carries truncated, unparsed_lines and clean, and answers is_repository: false with a notice outside Git instead of failing the call. It is in coding_v2, patch_v2 and coding_code_v1 (now 27/26/29 definitions; catalog 29, Broker bindings 28), is not rememberable for approval, and is not in SubagentRoleTools.

Policy identity was the non-obvious seam: PolicyContextFactory rejects an unlisted id as malformed_call, so an executor alone would have been denied before dispatch. git_status is registered there as a target-free read context, and a test pins that plan (the read-only preset) still allows it.

Deterministic evidence

  • New GitStatusToolTests cover the report shape, entry cap, unparsed lines, fixed argv/environment, failure codes, a real repository change set, the workspace-subtree bound, a plain directory, and the policy context.
  • Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 288, Harness.Core 904, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,615, gui-winui3 1,995 - 9,486 passed, 0 failed (artifacts/git-status-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

The first Broker run failed ApprovalRememberPolicyTests .Every_descriptor_tool_has_an_explicit_allowlist_entry because the new id had no remember-allowlist class. Adding git_status as NotRememberable fixed it and the full matrix then passed; the failure is recorded because it is the reason that allowlist entry exists.

A throwaway harness (artifacts/git-status-live, gitignored, not a repo project) then drove the production v2 path end to end - BrokerSession -> ToolExecutionService -> BrokerExecutionRuntime evaluate/token/execute - over real temporary repositories. Dirty repo, subdirectory workspace, plan preset, plain directory and clean tree each produced the expected policy decision and report; the subdirectory case reported only inside.txt, never the parent repo's tracked.txt or a .. path, and a non-ASCII untracked file decoded correctly.

Live model run

A real-provider run in a throwaway repository (standard preset, the machine's configured OpenRouter model) called the tool through the real Host/CLI path: the durable tool.proposed payload is {"tool_id":"git_status","tool_version":"1.0.0","arguments":{}}, the execution payload returned branch: master, unstaged: [app.txt], untracked: [new-untracked.txt], and the final reply repeated exactly those values.

The first attempt attached to a stale RuBackend Release publish under backend-dotnet/RuBackend/bin/Release/.../publish - the repository layout the CLI resolves by default - so the model answered that git_status was not in its capability list. Re-running with RU_AGENT_HOST_EXE pointing at the Debug host built from this commit, plus isolated RU_DATA_ROOT/RU_LOG_ROOT, produced the call above. Using the tool from the app or the plain CLI without that override requires republishing the Release host (build.bat).

Park reasons in words, and one more self-repair

A parked run explained itself with a raw provider token: the desktop activity row read "Waiting for confirmation" and the status line showed provider_transport_error. Park reasons now resolve through one shared catalog (AgentParkReasonCopy), used by the GUI activity row, the GUI status line and the TUI notice. A transport failure reads "network connection to the provider failed", a timeout "the provider stopped responding", a spent window "the run used its model step budget"; the raw code stays beside it in the tooltip and the terminal notice. The summary-to-status assignment also skips raw state tokens (waiting_user, reconciliation_required), so the GUI line no longer flickers through them on the way to the sentence.

capability_unavailable (a text-selected call naming an unknown tool or version) was terminal even though the model could pick an advertised tool. It is now a model_repairable code: the note names the advertised tool list and the model gets the existing four-note budget. The agent-tools.md recovery counts were also stale (two notes instead of three) and now match AgentRecoveryNotes.

Deterministic evidence

  • New AgentParkReasonCopyTests pin every park code's copy, the network wording and the fallback for unknown codes; ModelStepFailureClassificationTests add capability_unavailable to the repairable path and pin the note guidance.
  • GUI AgentActivityItemTests/AgentRunControllerTests pin the activity row and status line; CLI TuiParkReasonTests/ParkRecoveryRegressionTests pin the notice wording and still assert the raw code remains visible.
  • Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 304, Harness.Core 906, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,615, gui-winui3 1,996 - 9,505 passed, 0 failed (artifacts/park-copy-repair-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

No real-provider run exercised the new pause copy; this is deterministic evidence only. The desktop tests assert the status text and activity row from the feed, not native window pixels.

Host stop mid-model-step no longer leaves a running run

A GUI session was closed while the provider was still generating. The durable stream ended with model.step_started and no completion, but every dispatched tool already had a receipt. On the next start the recovery scanner read that as "simply mid-flight", the drive loop opened another model step on top of the orphaned one, and the session kept its active-run ownership. New prompts queued behind a turn whose model was already gone, and reopening the app showed the session running.

Recovery now tracks the last model step admission against its completion. A run whose step never completed is failed by name: the scanner commits an interrupted completion for that step and run.failed with recovery_model_step_incomplete, releasing the session. A run whose step did complete still resumes exactly as before.

Evidence

  • New HarnessRecoveryScannerTests cases: an orphaned model step fails the run and closes the step; a completed model step with committed execution stays Running.
  • The fix was exercised against a copy of the live harness database in an isolated RU_DATA_ROOT: the stuck run_ec939ff9... moved from state 3 (Running) to 8 (Failed) with terminal_error_code=recovery_model_step_incomplete, after model.step_completed (interrupted) for the orphaned step_f1a1.... The live database was copied and left unchanged.
  • AgentFailureCopy renders the new code as "Model reply interrupted", and its audit test covers it.
  • Focused RuBackend recovery tests (50) and Client copy tests (48) pass.
  • Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 305, Harness.Core 906, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,617, gui-winui3 1,996 - 9,508 passed, 0 failed (artifacts/orphan-step-recovery-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

Automatic budget windows

A user report made the per-run budget visible: one long prompt could stop at 128 model steps and queue the next prompt behind a Continue. The budget was never per session - each chat message is a new run with fresh counters - but stopping a single run was still a pause other agents do not show.

Checked how comparable agents bound one task before changing the policy:

  • Cursor agent docs: "There is no limit on the number of tool calls Agent can make during a task."
  • OpenCode agent option steps: unset means run until the model stops or the user interrupts; when set, the limit forces a summary of remaining work rather than a Continue button.
  • DeepSeek Harness (the installed runtime inspected locally): no per-step or per-tool budget; long work uses goal rounds (default 256, continued automatically), and the goal is blocked with round-limit at the cap.
  • Gemini CLI: maxSessionTurns defaults to -1 (unlimited) with loop detection.
  • Claude Agent SDK: optional maxTurns; unset has no hard limit.

Ru keeps its fail-closed backstop but no longer stops at the first window. A root run opens the next window itself, up to 8 windows (1,024 model steps / 2,048 tool calls) per run; only then does it park with model_step_budget_exhausted / tool_call_budget_exhausted. A manual Continue after that grants one further window, and every later window is manual. Internal subagent runs never take the automatic path (own 32/64 limits), and the family pool stays at 128/256 per window. Checkpoint calls use the same window rule.

Evidence

  • AgentBudgetTests: auto window opens with reset counters; the automatic cap still parks; an internal run never auto-opens; a tool batch opens a window only when it fits one.
  • ContextCompactionCoordinatorTests: a checkpoint at the step limit opens a window; format repair follows it; the cap still parks with zero model calls; a real model step continues with BudgetWindow=2 and no continue.requested.
  • Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 305, Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,617, gui-winui3 1,996 - 9,515 passed, 0 failed (artifacts/auto-budget-window-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

Preset binding and the unbound live run

The GUI reported invalid_response: Invalid create response binding at 17:00 local while the session was set to Standard. The session (ca2cbd67...) had stored plan at version 1. The chat adopted plan, the user chose Standard, and SessionPresetViewModel.EnsureStoredAsync reused the cached version because it compared the cached version with the current Preset (now Standard) instead of the stored value. The create therefore sent standard + version 1; the Host kept the stored plan (the documented stored-selection rule), created run_d9099f77... with plan, and returned 201. The client's strict binding rejected the response, so the GUI showed the error while the Host kept driving the run: 13 model steps and 28 read-only tool calls before the app closed mid-step and left it Running with no membership.

Two fixes:

  • SessionPresetViewModel now caches _storedPreset alongside the version and reuses the version only when the stored preset equals the requested one. A choice made after adoption writes the selection instead of sending a stale pin.
  • AgentRunCreationService rejects a create that pins a version but names a different preset than the stored selection, with 409 preset_mismatch before workspace admission, so a stale client cannot leave a live run it cannot bind or cancel. Legacy requests without a pinned version keep the stored-selection behavior (including full_access without a version creating a Standard run).

The already-orphaned run_d9099f77... is left untouched; the next start runs the existing orphaned-model-step recovery and fails it by name.

Evidence

  • New GUI test: adopting plan, choosing standard, then ensuring writes standard once (23 preset VM tests pass).
  • New Host test: a pinned plan request at the stored standard version is rejected with preset_mismatch, and a correctly pinned Standard request still creates a run afterwards (16 idempotency tests pass).
  • Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 305, Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,618, gui-winui3 1,997 - 9,517 passed, 0 failed (artifacts/preset-binding-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

Review follow-ups: checkpoint calls, input bounds, restore race

Three findings from an external review were checked against the tree; all three reproduced.

  • Checkpoint accounting across a budget window. BuildCheckpointAsync counted a build's admitted calls as run.ModelStepsInWindow - beforeCalls. When the second checkpoint call opened an automatic window, the counter reset and the delta became negative, so callsLeft grew instead of shrinking; the outer loop could admit six calls against the documented four. The build now returns its admitted-call count and the caller subtracts that. A checkpoint at the window boundary still opens the new window, but the allowance is four admissions.
  • Input bounds. The GUI accepted text attachments up to 5 MiB and inlined them into the prompt, while AgentRunCreationService rejects prompts over 16,000 characters. AgentRunLimits.MaxPromptChars is now the shared bound; the GUI checks the composed prompt (typed text plus every inlined document) before sending and before queueing, keeps the content in the composer and shows the length. Separately, RequestSizeMiddleware special-cased only /agent/v1/runs, so /agent/v1/runs/selected fell back to the general 8 MiB cap even though both routes share the image-capable reader; both paths now use the run bound (9 MiB test bodies reach the endpoint and return 400, not 413).
  • Restore race. RestoreWorkspaceChangeAsync compared the target SHA, then decrypted and encrypted the journal, then replaced the file. An editor or another run writing in that interval was silently overwritten. The restore now re-reads the target hash after the temp write and immediately before File.Move, returning workspace_restore_stale when it no longer matches the client's expected bytes. The residual hash-to-rename gap is microseconds, not the journal round trip.

Evidence

  • Checkpoint_calls_stay_capped_when_an_automatic_window_resets_the_counter keeps the boundary case to at most four calls.
  • A_composed_prompt_over_the_host_bound_is_reported_before_sending covers the GUI bound; Agent_run_routes_share_the_larger_body_bound covers both create paths.
  • Restore_target_comparison_is_exact covers hash and missing-file markers; Restore_rechecks_the_target_immediately_before_replacing_it pins the check order.
  • Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 305, Harness.Core 914, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,625, gui-winui3 1,998 - 9,526 passed, 0 failed (artifacts/review-followups-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

Second review: git reads, input bounds, restore race

The second review's medium findings were checked against the tree.

  • Workspace git probe hang and secret inheritance. WorkspaceGitProbe called ReadToEnd() before WaitForExit, so the 5-second timeout could never fire, and it read stdout then stderr sequentially. It also inherited the Host environment. The probe now runs through StreamingGitCli (streamed stdout/ stderr, real 5-second timeout, output cap, sanitized child environment) and the prompt provider awaits it through a SemaphoreSlim instead of blocking a thread under a lock. WorkspaceStateSnapshotService uses the same channel.
  • Repository filters in read paths. git status runs a repo-configured clean filter while comparing a same-size edited file. The first attempt only inspected arguments[0], so GitStatusAdapter's -c status.relativePaths=true ... argv skipped the guard; the verb is now found after global options. Read and unknown verbs neutralize local/worktree filter.*.{clean,smudge,process} keys through GIT_CONFIG_KEY_n (local scope only, so global LFS filters stay active); writes keep all filters. The guard uses ordinal driver names (case-distinct), handles a=b, caps driver/key count, and fails closed if the table cannot be read. Tests go through GitStatusAdapter, prove the filter runs under raw git and not under the Broker read, cover Evil/evil and a=b, and assert a write filter cannot read RU_DESKTOP_TOKEN.
  • Large status truncation. StreamingGitCli used the runner's 8k+8k head/tail defaults, so a status over ~16k was git_output_truncated despite the 256 KiB/1 MiB output caps. ProcessRunSpec can now override the capacities and Git reads keep the whole head; a 300-file (38k) status succeeds through the adapter, the probe reports dirty (300 changed, ...), and the reconciliation fingerprint stays non-null and changes with the workspace.
  • Probe timeout classification. A timeout or unreadable filter table now reports unknown, not not_a_repository; only git_exit_128 with the not-a-repository message maps to not_a_repository.
  • SSE total cap. The raw-stream bound was 5 MiB including framing. It is now 32 MiB with the existing 256 KiB line/event caps; a 6 MiB stream of small events is accepted and a 33 MiB stream still fails response_too_large.
  • Prompt bound drift. AgentRunLimits.MaxPromptChars is now used by the Host, the CLI parser, RPC run.start, ChatSubmissionState and the GUI, and the CLI parser counts characters instead of UTF-8 bytes, so a Vietnamese prompt is not rejected early. The CLI README now says 16,000.
  • Protected-path aliases. RU_PROTECTED_PATHS entries are resolved through a junction/symlink at the entry and at any existing parent; both a junction root and an entry below a junction protect the target files.
  • WebPolicy transition ranges. ::/96 (IPv4-compatible), ::ffff:0:0/96 (IPv4-translated), 64:ff9b:1::/48 (local-use NAT64) and 192.88.99.0/24 (6to4 relay anycast) now fail closed.
  • Managed commit timeout. The managed-commit channel has a 120-second budget for pre-commit hooks; ordinary reads keep the shorter timeout.

Deferred deliberately, not fixed here:

  • Nested run_code calls still bypass HookToolGateway; it only affects the experimental coding_code_v1 profile and needs the program runtime to carry a hook-bound gateway rather than a direct Broker call.
  • SessionInterchangeEndpoints returning {"status":"error"} with HTTP 200 is the documented client contract (SessionInterchangeApiClient raises the message), so changing the status would be a client protocol change. The not-found message was normalized to English.
  • The residual WebSocket-era code has no route in the contract; removing it is a cleanup, not a behavior fix.

Evidence

  • GitFilterHardeningTests: GitStatusAdapter (argv starts with -c) does not run a filter, Evil/evil and a=b drivers are neutralized, a 300-file status succeeds, and a managed add runs its filter without RU_DESKTOP_TOKEN. WorkspaceGitProbeTests: 300-file count, timeout maps to unknown, a real non-repository maps to not_a_repository. WorkspaceStateSnapshotServiceTests: the 300-file live fingerprint is non-null and changes with a new untracked file.
  • Focused Broker 84, RuBackend 7, Runtime.Adapters 19, CLI 81 tests pass.
  • Full .NET matrix on the final tree: Broker 1,484, CLI 2,069, Client 305, Harness.Core 914, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,629, gui-winui3 1,998 - 9,541 passed, 0 failed (artifacts/second-review-followup-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
  • Review scratch repositories under %TEMP%\claude\... were removed without running git inside them; the e7 junction was removed with rmdir first.

Cleanup: WebSocket residue and an unused receipt member

Two dead pieces were removed after checking the whole tracked tree:

  • SystemPromptReceipt.Empty had exactly one occurrence in tracked files: its definition. No test, doc or client referenced it; SystemPromptContext.Empty is different and remains in use. The member was deleted, not the file.
  • app.UseWebSockets() and the 4401 WebSocket branch in DesktopTokenMiddleware were the only production WebSocket references. The route contract and docs already declare zero WebSocket routes, and no current test or client uses the close code. An unauthenticated upgrade is now rejected as 401 JSON; a new RequestBoundaryTests guard pins that response.

The removal does not change any HTTP/SSE route. Authenticated WebSocket upgrades have no endpoint to reach, matching the declared contract.

Evidence

  • Focused Harness.Core prompt tests (314) and RuBackend RequestBoundaryTests/PublicRouteContractTests (8) pass.
  • Full .NET matrix on the final tree (with the worktree removal below): see the counts in the next section. Python 268 passed, 1 skipped.

Remove the CLI worktree convenience

Commit 996e9a31 added agent run --worktree for dogfooding: every headless run otherwise created a git worktree by hand or edited the caller's tree. It was operational convenience, not a product capability: GUI runs in the project folder, and no script in the repository used the flag.

Removed: the --worktree switch, RunCommand.Worktree, the parser branch and worktree_session_conflict, GitWorktreeIsolation/WorktreeLease, its real-git tests, and the CLI README paragraph. --worktree now fails as unknown_switch. Subagent coding worktrees are untouched; they are runtime isolation for delegated runs, not this CLI flag.

Evidence

  • ParseRun_RejectsRemovedWorktreeSwitch pins the removed switch; CLI parser tests (78) pass.
  • Full .NET matrix on the final tree: Broker 1,408, CLI 2,063, Client 305, Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,615, gui-winui3 1,998 - 9,444 passed, 0 failed (artifacts/remove-worktree-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

Compact activity default and auto-collapse

The GUI already had the Compact/Verbose agent-activity choice and its summary disclosure; it was not missing a DSH-style process view. Two changes align the default and the completed-turn behavior:

  • A new install (no Ru/activity-display.json) starts Compact. An existing file keeps the user's saved choice; an explicit false remains Verbose.
  • Verbose still streams every row while a turn runs, then that message auto-collapses once it has an answer and no row is running or waiting. The disclosure is per-message and survives expansion; a failed turn with no answer stays expanded. Historical completed turns collapse the same way when loaded.

Evidence

  • AgentActivityDisplayPreferenceTests: compact default, explicit false, invalid and oversized files fall back to compact.
  • CompactAgentActivityTests: a finished verbose turn collapses and its disclosure preserves the timeline; no answer or a running row stays verbose.
  • Full .NET matrix on the final tree: Broker 1,408, CLI 2,063, Client 305, Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,615, gui-winui3 2,001 - 9,447 passed, 0 failed (artifacts/compact-activity-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

Context popup after a rejected provider request

A provider 4xx fails before any model usage is reported. The popup was showing the prepared request estimate with the label "% of context used", which reads as provider-confirmed usage and contradicts the error row. RetainedContextMeter now exposes whether usage was actually reported; when the retained run is failed and no usage exists, the popup labels the same estimate as "estimated for next request", marks the token count (estimated), and says "Last request failed · no provider usage reported". A failed run that did report an earlier step keeps the normal readings.

Evidence

  • ContextPanelPresenterTests: failed-without-usage sets NextRequestOnly and the failure note; failed-with-usage keeps the normal estimate note.
  • RetainedContextMeterPresenterTests: an estimate does not count as reported usage; a later usage event does.
  • Full .NET matrix on the final tree: Broker 1,408, CLI 2,063, Client 305, Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,615, gui-winui3 2,004 - 9,450 passed, 0 failed (artifacts/context-popup-matrix.log).
  • Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.

Skill tool denied before policy evaluation

A dogfooding run (session f95786307f554e4ca0583266d86c1d4c, run run_8170562a3a51e5bd6b22e201d4f17a8f) recorded tool.execution_requested with policy_denied and tool.execution_completed with malformed_call for a skill call. The tool is advertised in every coding profile (AgentToolProfiles.Common), the Host prompt tells the model to use it to reach skills omitted from the prompt listing, and the Broker binds an executor — but PolicyContextFactory had no skill branch. Create therefore hit its default arm and denied with "unknown tool id" before policy evaluation, which is exactly the registration step 4 in Registering a model-visible tool.

skill is now a target-free Read operation like help and todo_write. Its policy identity stays tool id + full argument hash; a skill body's allowed-tools still grants nothing.

Evidence

  • Reproduced before the fix: the five new tests fail on the pre-change factory and pass after it.
  • PolicyTests.Skill_tool_is_a_target_free_read_context pins the null target and read class; Skill_tool_is_allowed_by_every_preset allows the call under plan, standard and full_access.
  • BrokerExecutionRuntimeTests.Evaluate_then_execute_skill_loads_a_workspace_workflow drives the real evaluate → authorization token → execute path and reads a workspace skills/demo/SKILL.md body.
  • Full .NET matrix on the final tree: Broker 1,413, CLI 2,063, Client 305, Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,615, gui-winui3 2,004 - 9,455 passed, 0 failed (artifacts/skill-policy-matrix.log).
  • Python: 268 passed, 1 skipped (artifacts/skill-policy-python.log).

Follow-up: the defect gets its own code

The same missing-branch denial surfaced as malformed_call / "unknown tool id", the code for a call the model invented. The model had to infer a host bug from a message that blamed its arguments, and the execution contract could only tell it that a policy denial ends the path. PolicyContextFactory now separates the fallback:

  • an unadvertised id still fails as malformed_call with the repair hint;
  • an advertised, Broker-bound id with no switch arm fails as tool_policy_unconfigured, whose hint names the product defect and refuses a retry or a workaround.

The execution contract now says that code is a host defect, not a user decision: report it and stop the path; never route around it with another tool. The shared failure copy renders it on both surfaces as "Tool registration incomplete". A parity test walks every advertised tool the Broker evaluates and fails when one has no branch; ask_user (Host-intercepted) and agent_apply (Host apply service) stay outside that invariant — the first run of the parity test caught agent_apply and made that distinction explicit.

Evidence

  • PolicyTests.Unknown_tool_id_stays_a_malformed_model_call keeps the model error path; An_advertised_tool_without_a_policy_branch_is_a_named_host_defect pins the defect code and hint; Every_broker_evaluated_advertised_tool_has_a_policy_branch is the parity guard (it failed for agent_apply before the Host-evaluated exclusion was documented).
  • CompletionChecklistTests.Execution_contract_names_an_unconfigured_advertised_tool_as_a_host_defect pins the prompt clause; AgentFailureCopyTests includes the new code.
  • Full .NET matrix on the final tree: Broker 1,416, CLI 2,063, Client 306, Harness.Core 914, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,615, gui-winui3 2,004 - 9,460 passed, 0 failed (artifacts/tool-policy-defect-matrix.log).
  • Python: 268 passed, 1 skipped (artifacts/tool-policy-defect-python.log).

Follow-up: skill search matches tokens and stops lying about totals

A second self-review found two defects in the same tool:

  • A multi-word query returned nothing. Matching used one whole Contains over id, name and description, so "debug worktrees" only matched a literal substring and the model had to guess single words. The query now splits on whitespace and every token must appear in one of the three fields (AND), in any order; tokens are deduplicated and capped at eight.
  • A matching entry the model cannot load reported items: [], total as the whole catalog size and truncated: true, which reads as "more results exist". Total now counts entries matching the query, truncated reports a real scan, item-cap or result-cap bound, and the new hidden field counts matching entries excluded because they are disabled or explicit-only.

Evidence

  • SkillBodyToolTests.Skill_tool_query_matches_tokens_in_any_order (a query that is not a substring of any single field), ..._requires_every_token (AND), ..._total_counts_only_matching_entries, and ..._reports_a_hidden_match_instead_of_false_truncation (a manual-only match now reports total: 1, hidden: 1, truncated: false).
  • Full .NET matrix on the final tree: Broker 1,420, CLI 2,063, Client 306, Harness.Core 914, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend 1,615, gui-winui3 2,004 - 9,464 passed, 0 failed (artifacts/skill-query-matrix.log).
  • Python: 268 passed, 1 skipped (artifacts/skill-query-python.log).