Verification evidence - 2026-10-31¶
Historical snapshot only. These results describe the recorded working tree and conditions, not current acceptance. See current verification.
Tool read ergonomics¶
A model reads an empty read page as a broken tool. read and read_file now
carry a notice that names the actual situation: the file exists but is empty
(0 bytes), the requested offset/start_line is at or past the end (with the
real byte or line count), or content remains and where the next page starts. A
page cut by the output budget says so and points at next_offset instead of
ending on a bare truncated flag. read also reports estimated_tokens for the
returned content, so the model can budget in the unit it is billed in.
FileReadOutputBudget now reserves 256 bytes for fields added after fitting, so
notice and estimate cannot push the envelope past the 4 MiB cap. The
ModelToolHeadroomRegressionTests case caught that before the reserve existed.
The two changed tool descriptions updated the tool-contract golden lock
deliberately (description SHA-256 rows for read and read_file).
Broker 1,445 tests passed for this change in isolation (FileReadAdapterTests,
FileLineReadTests, headroom and tool-contract tests included).
Tool telemetry counters¶
The comparison with Claude Code's read tool showed a measured dedup win (18% of
reads were same-file collisions). Ru now measures the same question before
changing behaviour: the Broker source emits ru.tool.calls
(tool.id, tool.status, error.code) and ru.tool.read_pages
(tool.id, repeated). repeated is answered by a bounded per-service path
tracker that never withholds content; a real "file unchanged" answer needs the
Host's compaction-aware read state, so dedup stays a separate decision.
The Host's meter provider now adds the Broker source, without which the counters
would be produced but never collected. Tags follow the existing allowlist:
bounded tokens only, never paths or arguments. Two listener tests pin the first
read (repeated=false), the second (repeated=true) and a failed call's status
and bounded error code.
Model persona names the model, not the product¶
Asked which model it was, the model answered "Ru Agent". The cause was the first sentence of the core contract: "You are Ru Agent". The environment block already carried the serving model id, but it read as metadata next to a first-person identity claim.
The persona block now opens with that id - "You are
Deterministic evidence¶
- Full .NET matrix on the final tree: Broker 1.447, CLI 2.058, Client 288,
Harness.Core 902, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1.612, gui-winui3 1.995 - 9,443 passed, 0 failed
(
artifacts/persona-matrix.log; the read-ergonomics and telemetry batches ran the earlierartifacts/agent-quality-matrix-3.log). - Python: 268 passed, 1 skipped (POSIX-only case).
Flaky and hung runs observed¶
The first full matrix failed one CLI test,
TgSquareWordmarkRasterTests.Frames_use_two_buffers_and_do_not_redraw_or_allocate_during_quiet_time,
an allocation-sensitive wordmark check unrelated to these changes. It passed
alone (13/13 in the class) and in the final matrix, so the first failure is
recorded as environment-sensitive rather than a regression.
A second matrix then hung in RuAgent.Cli.Tests with no output for ~18 minutes.
Stale dotnet/testhost processes from the aborted first matrix were still
alive; after they were stopped, the CLI project passed alone (2.058 tests,
3m28s) and the full matrix passed cleanly. The hang is recorded as process-state
contamination from the aborted run, not a defect in the suite.
Not measured¶
No real-provider session was run for this change, so the counters have no production sample yet and dedup is not implemented. The read notices and token estimate are covered by unit tests, not by a live model reading a file.
Model-visible git_status tool¶
Handoff item 1 landed: git_status is a Broker-bound, read-only
workspace.read tool. One fixed git status --porcelain=v2 --branch invocation
runs through IGitCli and the existing GitStatusParser (no second parser),
with status.relativePaths=true, --ahead-behind, --untracked-files=normal
and a -- . pathspec, so a workspace that is a subdirectory of a larger
repository reports only its own subtree with workspace-relative paths. The
report separates staged/unstaged (porcelain code plus original_path for a
rename), untracked and unmerged, carries truncated, unparsed_lines and
clean, and answers is_repository: false with a notice outside Git instead of
failing the call. It is in coding_v2, patch_v2 and coding_code_v1 (now
27/26/29 definitions; catalog 29, Broker bindings 28), is not rememberable for
approval, and is not in SubagentRoleTools.
Policy identity was the non-obvious seam: PolicyContextFactory rejects an
unlisted id as malformed_call, so an executor alone would have been denied
before dispatch. git_status is registered there as a target-free read
context, and a test pins that plan (the read-only preset) still allows it.
Deterministic evidence¶
- New
GitStatusToolTestscover the report shape, entry cap, unparsed lines, fixed argv/environment, failure codes, a real repository change set, the workspace-subtree bound, a plain directory, and the policy context. - Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 288,
Harness.Core 904, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,615, gui-winui3 1,995 - 9,486 passed, 0 failed
(
artifacts/git-status-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
The first Broker run failed ApprovalRememberPolicyTests
.Every_descriptor_tool_has_an_explicit_allowlist_entry because the new id had
no remember-allowlist class. Adding git_status as NotRememberable fixed it
and the full matrix then passed; the failure is recorded because it is the
reason that allowlist entry exists.
A throwaway harness (artifacts/git-status-live, gitignored, not a repo
project) then drove the production v2 path end to end - BrokerSession ->
ToolExecutionService -> BrokerExecutionRuntime evaluate/token/execute - over
real temporary repositories. Dirty repo, subdirectory workspace, plan preset,
plain directory and clean tree each produced the expected policy decision and
report; the subdirectory case reported only inside.txt, never the parent
repo's tracked.txt or a .. path, and a non-ASCII untracked file decoded
correctly.
Live model run¶
A real-provider run in a throwaway repository (standard preset, the machine's
configured OpenRouter model) called the tool through the real Host/CLI path:
the durable tool.proposed payload is
{"tool_id":"git_status","tool_version":"1.0.0","arguments":{}}, the execution
payload returned branch: master, unstaged: [app.txt],
untracked: [new-untracked.txt], and the final reply repeated exactly those
values.
The first attempt attached to a stale RuBackend Release publish under
backend-dotnet/RuBackend/bin/Release/.../publish - the repository layout the
CLI resolves by default - so the model answered that git_status was not in
its capability list. Re-running with RU_AGENT_HOST_EXE pointing at the Debug
host built from this commit, plus isolated RU_DATA_ROOT/RU_LOG_ROOT,
produced the call above. Using the tool from the app or the plain CLI without
that override requires republishing the Release host (build.bat).
Park reasons in words, and one more self-repair¶
A parked run explained itself with a raw provider token: the desktop activity row
read "Waiting for confirmation" and the status line showed
provider_transport_error. Park reasons now resolve through one shared catalog
(AgentParkReasonCopy), used by the GUI activity row, the GUI status line and
the TUI notice. A transport failure reads "network connection to the provider
failed", a timeout "the provider stopped responding", a spent window "the run
used its model step budget"; the raw code stays beside it in the tooltip and the
terminal notice. The summary-to-status assignment also skips raw state tokens
(waiting_user, reconciliation_required), so the GUI line no longer flickers
through them on the way to the sentence.
capability_unavailable (a text-selected call naming an unknown tool or version)
was terminal even though the model could pick an advertised tool. It is now a
model_repairable code: the note names the advertised tool list and the model
gets the existing four-note budget. The agent-tools.md recovery counts were
also stale (two notes instead of three) and now match AgentRecoveryNotes.
Deterministic evidence¶
- New
AgentParkReasonCopyTestspin every park code's copy, the network wording and the fallback for unknown codes;ModelStepFailureClassificationTestsaddcapability_unavailableto the repairable path and pin the note guidance. - GUI
AgentActivityItemTests/AgentRunControllerTestspin the activity row and status line; CLITuiParkReasonTests/ParkRecoveryRegressionTestspin the notice wording and still assert the raw code remains visible. - Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 304,
Harness.Core 906, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,615, gui-winui3 1,996 - 9,505 passed, 0 failed
(
artifacts/park-copy-repair-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
No real-provider run exercised the new pause copy; this is deterministic evidence only. The desktop tests assert the status text and activity row from the feed, not native window pixels.
Host stop mid-model-step no longer leaves a running run¶
A GUI session was closed while the provider was still generating. The durable
stream ended with model.step_started and no completion, but every dispatched
tool already had a receipt. On the next start the recovery scanner read that as
"simply mid-flight", the drive loop opened another model step on top of the
orphaned one, and the session kept its active-run ownership. New prompts queued
behind a turn whose model was already gone, and reopening the app showed the
session running.
Recovery now tracks the last model step admission against its completion. A
run whose step never completed is failed by name: the scanner commits an
interrupted completion for that step and run.failed with
recovery_model_step_incomplete, releasing the session. A run whose step did
complete still resumes exactly as before.
Evidence¶
- New
HarnessRecoveryScannerTestscases: an orphaned model step fails the run and closes the step; a completed model step with committed execution stays Running. - The fix was exercised against a copy of the live harness database in an
isolated
RU_DATA_ROOT: the stuckrun_ec939ff9...moved from state 3 (Running) to 8 (Failed) withterminal_error_code=recovery_model_step_incomplete, aftermodel.step_completed(interrupted) for the orphanedstep_f1a1.... The live database was copied and left unchanged. AgentFailureCopyrenders the new code as "Model reply interrupted", and its audit test covers it.- Focused RuBackend recovery tests (50) and Client copy tests (48) pass.
- Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 305,
Harness.Core 906, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,617, gui-winui3 1,996 - 9,508 passed, 0 failed
(
artifacts/orphan-step-recovery-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
Automatic budget windows¶
A user report made the per-run budget visible: one long prompt could stop at 128 model steps and queue the next prompt behind a Continue. The budget was never per session - each chat message is a new run with fresh counters - but stopping a single run was still a pause other agents do not show.
Checked how comparable agents bound one task before changing the policy:
- Cursor agent docs: "There is no limit on the number of tool calls Agent can make during a task."
- OpenCode agent option
steps: unset means run until the model stops or the user interrupts; when set, the limit forces a summary of remaining work rather than a Continue button. - DeepSeek Harness (the installed runtime inspected locally): no per-step or
per-tool budget; long work uses goal rounds (default 256, continued
automatically), and the goal is blocked with
round-limitat the cap. - Gemini CLI:
maxSessionTurnsdefaults to -1 (unlimited) with loop detection. - Claude Agent SDK: optional
maxTurns; unset has no hard limit.
Ru keeps its fail-closed backstop but no longer stops at the first window. A
root run opens the next window itself, up to 8 windows (1,024 model steps /
2,048 tool calls) per run; only then does it park with
model_step_budget_exhausted / tool_call_budget_exhausted. A manual Continue
after that grants one further window, and every later window is manual. Internal
subagent runs never take the automatic path (own 32/64 limits), and the family
pool stays at 128/256 per window. Checkpoint calls use the same window rule.
Evidence¶
AgentBudgetTests: auto window opens with reset counters; the automatic cap still parks; an internal run never auto-opens; a tool batch opens a window only when it fits one.ContextCompactionCoordinatorTests: a checkpoint at the step limit opens a window; format repair follows it; the cap still parks with zero model calls; a real model step continues withBudgetWindow=2and nocontinue.requested.- Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 305,
Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,617, gui-winui3 1,996 - 9,515 passed, 0 failed
(
artifacts/auto-budget-window-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
Preset binding and the unbound live run¶
The GUI reported invalid_response: Invalid create response binding at
17:00 local while the session was set to Standard. The session
(ca2cbd67...) had stored plan at version 1. The chat adopted plan,
the user chose Standard, and SessionPresetViewModel.EnsureStoredAsync
reused the cached version because it compared the cached version with the
current Preset (now Standard) instead of the stored value. The create
therefore sent standard + version 1; the Host kept the stored plan (the
documented stored-selection rule), created run_d9099f77... with plan, and
returned 201. The client's strict binding rejected the response, so the GUI
showed the error while the Host kept driving the run: 13 model steps and 28
read-only tool calls before the app closed mid-step and left it Running with
no membership.
Two fixes:
SessionPresetViewModelnow caches_storedPresetalongside the version and reuses the version only when the stored preset equals the requested one. A choice made after adoption writes the selection instead of sending a stale pin.AgentRunCreationServicerejects a create that pins a version but names a different preset than the stored selection, with409 preset_mismatchbefore workspace admission, so a stale client cannot leave a live run it cannot bind or cancel. Legacy requests without a pinned version keep the stored-selection behavior (includingfull_accesswithout a version creating a Standard run).
The already-orphaned run_d9099f77... is left untouched; the next start runs
the existing orphaned-model-step recovery and fails it by name.
Evidence¶
- New GUI test: adopting
plan, choosingstandard, then ensuring writesstandardonce (23 preset VM tests pass). - New Host test: a pinned
planrequest at the storedstandardversion is rejected withpreset_mismatch, and a correctly pinned Standard request still creates a run afterwards (16 idempotency tests pass). - Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 305,
Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,618, gui-winui3 1,997 - 9,517 passed, 0 failed
(
artifacts/preset-binding-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
Review follow-ups: checkpoint calls, input bounds, restore race¶
Three findings from an external review were checked against the tree; all three reproduced.
- Checkpoint accounting across a budget window.
BuildCheckpointAsynccounted a build's admitted calls asrun.ModelStepsInWindow - beforeCalls. When the second checkpoint call opened an automatic window, the counter reset and the delta became negative, socallsLeftgrew instead of shrinking; the outer loop could admit six calls against the documented four. The build now returns its admitted-call count and the caller subtracts that. A checkpoint at the window boundary still opens the new window, but the allowance is four admissions. - Input bounds. The GUI accepted text attachments up to 5 MiB and inlined
them into the prompt, while
AgentRunCreationServicerejects prompts over 16,000 characters.AgentRunLimits.MaxPromptCharsis now the shared bound; the GUI checks the composed prompt (typed text plus every inlined document) before sending and before queueing, keeps the content in the composer and shows the length. Separately,RequestSizeMiddlewarespecial-cased only/agent/v1/runs, so/agent/v1/runs/selectedfell back to the general 8 MiB cap even though both routes share the image-capable reader; both paths now use the run bound (9 MiB test bodies reach the endpoint and return 400, not 413). - Restore race.
RestoreWorkspaceChangeAsynccompared the target SHA, then decrypted and encrypted the journal, then replaced the file. An editor or another run writing in that interval was silently overwritten. The restore now re-reads the target hash after the temp write and immediately beforeFile.Move, returningworkspace_restore_stalewhen it no longer matches the client's expected bytes. The residual hash-to-rename gap is microseconds, not the journal round trip.
Evidence¶
Checkpoint_calls_stay_capped_when_an_automatic_window_resets_the_counterkeeps the boundary case to at most four calls.A_composed_prompt_over_the_host_bound_is_reported_before_sendingcovers the GUI bound;Agent_run_routes_share_the_larger_body_boundcovers both create paths.Restore_target_comparison_is_exactcovers hash and missing-file markers;Restore_rechecks_the_target_immediately_before_replacing_itpins the check order.- Full .NET matrix on the final tree: Broker 1,474, CLI 2,069, Client 305,
Harness.Core 914, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 320, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,625, gui-winui3 1,998 - 9,526 passed, 0 failed
(
artifacts/review-followups-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
Second review: git reads, input bounds, restore race¶
The second review's medium findings were checked against the tree.
- Workspace git probe hang and secret inheritance.
WorkspaceGitProbecalledReadToEnd()beforeWaitForExit, so the 5-second timeout could never fire, and it read stdout then stderr sequentially. It also inherited the Host environment. The probe now runs throughStreamingGitCli(streamed stdout/ stderr, real 5-second timeout, output cap, sanitized child environment) and the prompt provider awaits it through aSemaphoreSliminstead of blocking a thread under a lock.WorkspaceStateSnapshotServiceuses the same channel. - Repository filters in read paths.
git statusruns a repo-configured clean filter while comparing a same-size edited file. The first attempt only inspectedarguments[0], soGitStatusAdapter's-c status.relativePaths=true ...argv skipped the guard; the verb is now found after global options. Read and unknown verbs neutralize local/worktreefilter.*.{clean,smudge,process}keys throughGIT_CONFIG_KEY_n(local scope only, so global LFS filters stay active); writes keep all filters. The guard uses ordinal driver names (case-distinct), handlesa=b, caps driver/key count, and fails closed if the table cannot be read. Tests go throughGitStatusAdapter, prove the filter runs under raw git and not under the Broker read, coverEvil/evilanda=b, and assert a write filter cannot readRU_DESKTOP_TOKEN. - Large status truncation.
StreamingGitCliused the runner's 8k+8k head/tail defaults, so a status over ~16k wasgit_output_truncateddespite the 256 KiB/1 MiB output caps.ProcessRunSpeccan now override the capacities and Git reads keep the whole head; a 300-file (38k) status succeeds through the adapter, the probe reportsdirty (300 changed, ...), and the reconciliation fingerprint stays non-null and changes with the workspace. - Probe timeout classification. A timeout or unreadable filter table now
reports
unknown, notnot_a_repository; onlygit_exit_128with the not-a-repository message maps tonot_a_repository. - SSE total cap. The raw-stream bound was 5 MiB including framing. It is now
32 MiB with the existing 256 KiB line/event caps; a 6 MiB stream of small
events is accepted and a 33 MiB stream still fails
response_too_large. - Prompt bound drift.
AgentRunLimits.MaxPromptCharsis now used by the Host, the CLI parser, RPCrun.start,ChatSubmissionStateand the GUI, and the CLI parser counts characters instead of UTF-8 bytes, so a Vietnamese prompt is not rejected early. The CLI README now says 16,000. - Protected-path aliases.
RU_PROTECTED_PATHSentries are resolved through a junction/symlink at the entry and at any existing parent; both a junction root and an entry below a junction protect the target files. - WebPolicy transition ranges.
::/96(IPv4-compatible),::ffff:0:0/96(IPv4-translated),64:ff9b:1::/48(local-use NAT64) and192.88.99.0/24(6to4 relay anycast) now fail closed. - Managed commit timeout. The managed-commit channel has a 120-second budget for pre-commit hooks; ordinary reads keep the shorter timeout.
Deferred deliberately, not fixed here:
- Nested
run_codecalls still bypassHookToolGateway; it only affects the experimentalcoding_code_v1profile and needs the program runtime to carry a hook-bound gateway rather than a direct Broker call. SessionInterchangeEndpointsreturning{"status":"error"}with HTTP 200 is the documented client contract (SessionInterchangeApiClientraises the message), so changing the status would be a client protocol change. The not-found message was normalized to English.- The residual WebSocket-era code has no route in the contract; removing it is a cleanup, not a behavior fix.
Evidence¶
GitFilterHardeningTests:GitStatusAdapter(argv starts with-c) does not run a filter,Evil/evilanda=bdrivers are neutralized, a 300-file status succeeds, and a managedaddruns its filter withoutRU_DESKTOP_TOKEN.WorkspaceGitProbeTests: 300-file count, timeout maps tounknown, a real non-repository maps tonot_a_repository.WorkspaceStateSnapshotServiceTests: the 300-file live fingerprint is non-null and changes with a new untracked file.- Focused Broker 84, RuBackend 7, Runtime.Adapters 19, CLI 81 tests pass.
- Full .NET matrix on the final tree: Broker 1,484, CLI 2,069, Client 305,
Harness.Core 914, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,629, gui-winui3 1,998 - 9,541 passed, 0 failed
(
artifacts/second-review-followup-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
- Review scratch repositories under
%TEMP%\claude\...were removed without running git inside them; thee7junction was removed withrmdirfirst.
Cleanup: WebSocket residue and an unused receipt member¶
Two dead pieces were removed after checking the whole tracked tree:
SystemPromptReceipt.Emptyhad exactly one occurrence in tracked files: its definition. No test, doc or client referenced it;SystemPromptContext.Emptyis different and remains in use. The member was deleted, not the file.app.UseWebSockets()and the4401WebSocket branch inDesktopTokenMiddlewarewere the only production WebSocket references. The route contract and docs already declare zero WebSocket routes, and no current test or client uses the close code. An unauthenticated upgrade is now rejected as401JSON; a newRequestBoundaryTestsguard pins that response.
The removal does not change any HTTP/SSE route. Authenticated WebSocket upgrades have no endpoint to reach, matching the declared contract.
Evidence¶
- Focused Harness.Core prompt tests (314) and RuBackend
RequestBoundaryTests/PublicRouteContractTests(8) pass. - Full .NET matrix on the final tree (with the worktree removal below): see the counts in the next section. Python 268 passed, 1 skipped.
Remove the CLI worktree convenience¶
Commit 996e9a31 added agent run --worktree for dogfooding: every headless
run otherwise created a git worktree by hand or edited the caller's tree. It
was operational convenience, not a product capability: GUI runs in the project
folder, and no script in the repository used the flag.
Removed: the --worktree switch, RunCommand.Worktree, the parser branch and
worktree_session_conflict, GitWorktreeIsolation/WorktreeLease, its
real-git tests, and the CLI README paragraph. --worktree now fails as
unknown_switch. Subagent coding worktrees are untouched; they are runtime
isolation for delegated runs, not this CLI flag.
Evidence¶
ParseRun_RejectsRemovedWorktreeSwitchpins the removed switch; CLI parser tests (78) pass.- Full .NET matrix on the final tree: Broker 1,408, CLI 2,063, Client 305,
Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,615, gui-winui3 1,998 - 9,444 passed, 0 failed
(
artifacts/remove-worktree-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
Compact activity default and auto-collapse¶
The GUI already had the Compact/Verbose agent-activity choice and its summary disclosure; it was not missing a DSH-style process view. Two changes align the default and the completed-turn behavior:
- A new install (no
Ru/activity-display.json) starts Compact. An existing file keeps the user's saved choice; an explicitfalseremains Verbose. - Verbose still streams every row while a turn runs, then that message auto-collapses once it has an answer and no row is running or waiting. The disclosure is per-message and survives expansion; a failed turn with no answer stays expanded. Historical completed turns collapse the same way when loaded.
Evidence¶
AgentActivityDisplayPreferenceTests: compact default, explicit false, invalid and oversized files fall back to compact.CompactAgentActivityTests: a finished verbose turn collapses and its disclosure preserves the timeline; no answer or a running row stays verbose.- Full .NET matrix on the final tree: Broker 1,408, CLI 2,063, Client 305,
Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,615, gui-winui3 2,001 - 9,447 passed, 0 failed
(
artifacts/compact-activity-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
Context popup after a rejected provider request¶
A provider 4xx fails before any model usage is reported. The popup was showing
the prepared request estimate with the label "% of context used", which reads as
provider-confirmed usage and contradicts the error row. RetainedContextMeter
now exposes whether usage was actually reported; when the retained run is
failed and no usage exists, the popup labels the same estimate as
"estimated for next request", marks the token count (estimated), and says
"Last request failed · no provider usage reported". A failed run that did report
an earlier step keeps the normal readings.
Evidence¶
ContextPanelPresenterTests: failed-without-usage setsNextRequestOnlyand the failure note; failed-with-usage keeps the normal estimate note.RetainedContextMeterPresenterTests: an estimate does not count as reported usage; a later usage event does.- Full .NET matrix on the final tree: Broker 1,408, CLI 2,063, Client 305,
Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,615, gui-winui3 2,004 - 9,450 passed, 0 failed
(
artifacts/context-popup-matrix.log). - Python: 268 passed, 1 skipped; the documentation guards and the route matrix script pass.
Skill tool denied before policy evaluation¶
A dogfooding run (session f95786307f554e4ca0583266d86c1d4c, run
run_8170562a3a51e5bd6b22e201d4f17a8f) recorded tool.execution_requested
with policy_denied and tool.execution_completed with malformed_call for a
skill call. The tool is advertised in every coding profile
(AgentToolProfiles.Common), the Host prompt tells the model to use it to reach
skills omitted from the prompt listing, and the Broker binds an executor — but
PolicyContextFactory had no skill branch. Create therefore hit its default
arm and denied with "unknown tool id" before policy evaluation, which is exactly
the registration step 4 in
Registering a model-visible tool.
skill is now a target-free Read operation like help and todo_write. Its
policy identity stays tool id + full argument hash; a skill body's
allowed-tools still grants nothing.
Evidence¶
- Reproduced before the fix: the five new tests fail on the pre-change factory and pass after it.
PolicyTests.Skill_tool_is_a_target_free_read_contextpins the null target and read class;Skill_tool_is_allowed_by_every_presetallows the call underplan,standardandfull_access.BrokerExecutionRuntimeTests.Evaluate_then_execute_skill_loads_a_workspace_workflowdrives the real evaluate → authorization token → execute path and reads a workspaceskills/demo/SKILL.mdbody.- Full .NET matrix on the final tree: Broker 1,413, CLI 2,063, Client 305,
Harness.Core 913, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,615, gui-winui3 2,004 - 9,455 passed, 0 failed
(
artifacts/skill-policy-matrix.log). - Python: 268 passed, 1 skipped (
artifacts/skill-policy-python.log).
Follow-up: the defect gets its own code¶
The same missing-branch denial surfaced as malformed_call / "unknown tool id",
the code for a call the model invented. The model had to infer a host bug from a
message that blamed its arguments, and the execution contract could only tell it
that a policy denial ends the path. PolicyContextFactory now separates the
fallback:
- an unadvertised id still fails as
malformed_callwith the repair hint; - an advertised, Broker-bound id with no switch arm fails as
tool_policy_unconfigured, whose hint names the product defect and refuses a retry or a workaround.
The execution contract now says that code is a host defect, not a user decision:
report it and stop the path; never route around it with another tool. The shared
failure copy renders it on both surfaces as "Tool registration incomplete". A
parity test walks every advertised tool the Broker evaluates and fails when one
has no branch; ask_user (Host-intercepted) and agent_apply (Host apply
service) stay outside that invariant — the first run of the parity test caught
agent_apply and made that distinction explicit.
Evidence¶
PolicyTests.Unknown_tool_id_stays_a_malformed_model_callkeeps the model error path;An_advertised_tool_without_a_policy_branch_is_a_named_host_defectpins the defect code and hint;Every_broker_evaluated_advertised_tool_has_a_policy_branchis the parity guard (it failed foragent_applybefore the Host-evaluated exclusion was documented).CompletionChecklistTests.Execution_contract_names_an_unconfigured_advertised_tool_as_a_host_defectpins the prompt clause;AgentFailureCopyTestsincludes the new code.- Full .NET matrix on the final tree: Broker 1,416, CLI 2,063, Client 306,
Harness.Core 914, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,615, gui-winui3 2,004 - 9,460 passed, 0 failed
(
artifacts/tool-policy-defect-matrix.log). - Python: 268 passed, 1 skipped (
artifacts/tool-policy-defect-python.log).
Follow-up: skill search matches tokens and stops lying about totals¶
A second self-review found two defects in the same tool:
- A multi-word query returned nothing. Matching used one whole
Containsover id, name and description, so "debug worktrees" only matched a literal substring and the model had to guess single words. The query now splits on whitespace and every token must appear in one of the three fields (AND), in any order; tokens are deduplicated and capped at eight. - A matching entry the model cannot load reported
items: [],totalas the whole catalog size andtruncated: true, which reads as "more results exist".Totalnow counts entries matching the query,truncatedreports a real scan, item-cap or result-cap bound, and the newhiddenfield counts matching entries excluded because they are disabled or explicit-only.
Evidence¶
SkillBodyToolTests.Skill_tool_query_matches_tokens_in_any_order(a query that is not a substring of any single field),..._requires_every_token(AND),..._total_counts_only_matching_entries, and..._reports_a_hidden_match_instead_of_false_truncation(a manual-only match now reportstotal: 1, hidden: 1, truncated: false).- Full .NET matrix on the final tree: Broker 1,420, CLI 2,063, Client 306,
Harness.Core 914, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime
Adapters 321, Runtime Model 142, SessionInterchange 45, Updates 25, RuBackend
1,615, gui-winui3 2,004 - 9,464 passed, 0 failed
(
artifacts/skill-query-matrix.log). - Python: 268 passed, 1 skipped (
artifacts/skill-query-python.log).