Verification evidence - 2026-10-30¶
Historical snapshot only. These results describe the recorded working tree and conditions, not current acceptance. See current verification.
The working tree is based on checkout 2675ab68b1678e2a88f6df515a8900bb91da1782
plus uncommitted scoped work on dev. The open plan docs/plans/2026-10-29-coding-quality.md
records the five changes below in its execution ledger. The plan stays in progress
because the Task 8 live cohort has not run.
Changes verified¶
The LSP connection could hang for 120 seconds and report lsp_timeout when its
reader ended before a request registered. A fenced terminal-error handshake now
reports lsp_server_unavailable. The regression test hung for two minutes before
the fix; a reader-finished test now covers the case. LanguageServers tests passed
18 of 18 on three runs.
The code_intelligence manifest referenced the language_server adapter, which
was missing from the ToolRegistry trusted adapters. 233 Broker failures reported
with invalid_tool_manifest, so the Host catalog could not be composed. The
registry regression was red first. The allowlist entry follows ADR 0037 with
External risk, and the approval behavior is unchanged. Broker (1383 tests) and
Protocol (201 tests, 29 tools) passed afterward.
Triage of eleven failed Kilo runs found spec-clause misses, one run that finished
with no check after edits, and text-only replies under outstanding checks. A
one-time unverified_source_edit hold (cap one, never a failure path) and a
code-aware unverified-note reason with a repair-or-[BLOCKED] line now respond to
these cases. Three Core flow tests were red; Harness.Core now passes 817 tests,
and the family end-to-end test accounts for the extra held step.
The ts-abortable-coalescer and ts-generation-pager graders asserted
calls === 0 before the loader could run, a timing the task does not require.
They now assert exactly one loader call for shared requests, checked by a
four-way oracle per task. Historical scores are unchanged. The two fingerprints
changed, so these tasks no longer match earlier cohorts.
Deterministic evidence¶
The first full scripts/test_dotnet.ps1 -Configuration Debug run stopped at
TgSquareWordmarkRasterTests.Frames_use_two_buffers_and_do_not_redraw_or_allocate_during_quiet_time
in RuAgent.Cli.Tests, on an allocation assertion. The same test failed the first
matrix run on 2026-10-29 and passes in isolation, so it is treated as a recurring
flake unrelated to these changes. The complete rerun passed all 13 projects and
9,162 tests, against 9,094 recorded on 2026-10-29.
| Check | Result |
|---|---|
| .NET matrix, first run | Stopped at the wordmark allocation assertion |
| .NET matrix, rerun | 13 of 13 projects passed |
| Python suite | 268 passed, 1 skipped |
| Documentation guard | tests/test_current_documentation.py passed |
Rerun per-project counts: Broker 1383, Cli 2020, Client 279, Harness.Core 817, Harness.Persistence 370, Hosting 42, Protocol 201, Runtime.Adapters 318, Runtime.Model 124, SessionInterchange 45, Updates 25, RuBackend 1558 and gui-winui3.Tests 1980. The Python suite includes the C# grader-contract tests that need dotnet.
Not measured¶
No real-provider cohort was run after these changes, so the model-level effect on
coding quality is unknown. LSP real-server smoke checks were not run. TypeScript
repository tasks remain unavailable without tsc. Benchmark trajectories still
record only safe summaries, not private payloads.
Requirement checklist and completion review¶
The prompt now asks the agent to record each explicit requirement of a coding
request as its own todo_write item, and to close that item only after a check
exercises it. Before the final reply, the agent re-reads the request and the
changes it made. Completion evidence is durable: a sticky source-edit flag and
the latest successful todo_write list.
Two one-time holds can apply to a non-empty final reply, after the existing
unverified_source_edit hold. open_todos holds when the latest list still has
open items. completion_review holds when source was edited and verified, no
unverified_source_edit note was raised, and no list is fully completed. Neither
hold is a failure path.
| Check | Result |
|---|---|
| Harness.Core unit and coordinator flow tests | 26 new (20 unit, 6 flow); red before the change, green after |
| RuBackend tests that end with open todos | AgentRunActivitiesTests (two) and TodoWriteEndToEndTests script one extra final step |
| Unverified-source-edit flow test | Now asserts the completion_review hold |
| .NET matrix | Full runner stopped at the wordmark flake; other projects rerun individually |
The full runner stopped at TgSquareWordmarkRasterTests.Frames_use_two_buffers_and_do_not_redraw_or_allocate_during_quiet_time (expected 0, actual 992 bytes). This is the fourth recurrence across 2026-10-29 and 2026-10-30; the class passed 13 of 13 in isolation. The remaining projects then ran individually: Broker 1383, Cli 2019/2020 (flake only), Client 279, Harness.Core 843, Harness.Persistence 370, Hosting 42, Protocol 201, Runtime.Adapters 318, Runtime.Model 124, SessionInterchange 45, Updates 25, RuBackend 1558 and gui-winui3.Tests 1980. All were green apart from the Cli wordmark flake. Harness.Core rose from 817 to 843 with the 26 new tests. The 9,188 total counts tests, not passes: the one Cli miss is the flake.
Not measured: no real-provider run was made after this change, because the user chose not to spend provider usage. The model-level effect on coding quality is therefore unknown.
Cost: a coding run with source edits normally takes one extra model step before it finishes, and two when an open checklist is also held.
Remembered approvals¶
The Broker builds every remembered matcher from the approved call's own PolicyContext
(ApprovalRememberPolicy) and returns it with the grant. The evaluator ignores identity-less allow
rules for MCP, computer-use and language-server tools, and the Host parses no tool arguments and saves
nothing before the grant. The change closes a widening: one approval for one MCP tool, computer-use app
or language-server operation silently allowed every later call of that family.
| Check | Result |
|---|---|
RuBackend.Tests/Harness/RememberedApprovalIdentityTests*.cs (widening, failed-grant and root cases) |
12 of 20 red before the change; 22 of 22 green after |
MCP call with a JSON-string arguments (Case9) |
Approvable after the change; the Host re-parsed it and answered approval_stale before |
| Test counts after the change | Broker 1471; RuBackend 1582 |
These counts are reported by the coordinator and were not re-run here. The final full matrix will be added by the coordinator. This documentation step ran no .NET commands.
Open follow-ups, not fixed here:
- New-session external-folder file writes fail at file-tool root selection
(
ToolExecutionService.FileTools.cs, around lines 76 and 272), although policy allows them. Reported by the coordinator; not re-run here. - External-folder durable rules are path-only and do not bind the approved operation.
- A session matcher for a non-shell, non-flag-first terminal launch or a language-server launch is compared as a prefix, so later calls that append arguments also match it.
- Argv glob matching folds case and separators.
Language-server removal¶
The user requested removal of LSP integration. The Broker server host, transport,
tool adapter, configuration reader, policy branch and semantic tool profile are
removed. The catalog now has 28 definitions and 27 Broker executable bindings.
The three remaining coding profiles keep their inventories. A regression test
confirms that a retired profile pin fails with tool_profile_invalid rather than
selecting a replacement; it failed before removal and passes after it. Existing
user configuration files were left untouched. ADR 0037 is retired, current docs
are updated and the completed coding-quality plan is removed.
scripts/test_dotnet.ps1 -Configuration Debug completed with exit code 0:
9,315 passed, 0 failed and 0 skipped across all 13 projects. Counts: Broker 1443,
Cli 2020, Client 279, Harness.Core 898, Harness.Persistence 362, Hosting 42,
Protocol 201, Runtime.Adapters 318, Runtime.Model 124, SessionInterchange 45,
Updates 25, RuBackend 1578 and gui-winui3.Tests 1980. The log is retained locally
at artifacts/remove-lsp-20261030/dotnet-matrix.log.
Focused approval/catalog checks passed 100 tests. Documentation and source
encoding guards (tests/test_current_documentation.py and
tests/test_source_encoding.py) passed 8 tests. A tracked-source scan found no
production LSP implementation or active configuration references. No provider
benchmark, native UI acceptance or Release packaging was run for this removal.
TUI branch integration¶
After committing and pushing the LSP removal (f3b97358), merged
casluminous/tui-welcome-refresh (9d6d8c91, merge e37b7746) and
casluminous/tui-multi-session (44ab02f2) into dev, in that order.
Changelog conflicts retain both features; the removed root work log stays removed.
Both development lines had allocated schema v32 to different changes. Keep the existing v32 execution-request index migration and add provider intents at v33. The v33 migration accepts both deployed development v32 shapes and preserves already recorded intents. Regression cases cover v31 and both v32 variants, retained runs/intents and the resulting execution-request index. A same-version merge first failed two upgrade cases; after the migration and fixture corrections, four focused checks passed. The final matrix also covers the added TUI-v32 case.
The combined route contract has 54 HTTP pairs (29 Agent and 25 non-Agent) and no WebSocket route. A focused guard run first found the stale 53-route validator (1 failed, 15 passed); after aligning the validator, tests and docs, all 16 passed.
Final integrated verification:
scripts/test_dotnet.ps1 -Configuration Debug: exit 0; 9,374 passed, no failures or skips across all 13 projects. Broker 1443, Cli 2058, Client 288, Harness.Core 898, Harness.Persistence 366, Hosting 42, Protocol 201, Runtime.Adapters 318, Runtime.Model 124, SessionInterchange 45, Updates 25, RuBackend 1586 and WinUI 1980..venv/Scripts/python.exe -m pytest tests -q: exit 0; 268 passed, 1 skipped. The POSIX process-group test is platform-skipped on Windows.- Logs:
artifacts/merge-tui-20261030/dotnet-matrix.logandartifacts/merge-tui-20261030/python-matrix.log(local artifacts).
No real-provider, manual terminal/native UI acceptance or Release packaging was run for this integration. Installed binaries and the user's profile were not changed.
GUI image attachment intake¶
Reproduced the picker/drop/clipboard image path using the same
MemoryStream.AsRandomAccessStream() adapter as ChatView: the signature probe
called GetInputStreamAt, which throws NotSupportedException because that
adapter cannot clone streams. Text files bypass this probe. The view logged the
exception without composer feedback, so attaching an image appeared to do nothing.
The existing small-image test used a native InMemoryRandomAccessStream and did
not exercise the failing adapter.
Read the bounded signature directly with DataReader, detaching before disposal
so the decoder retains its input. Intake exceptions now show composer feedback
only while their original session/generation still owns the operation. Limits,
accepted formats and backend contracts are unchanged.
- New managed-stream regression: 1 failed before the fix, 1 passed after it.
dotnet test gui-winui3.Tests/gui-winui3.Tests.csproj --no-restore --nologo: 1,988 passed, no failures or skips. Eight new cases cover the actual GUI stream, PNG/JPEG passthrough and resizing, BMP normalization, empty and invalid input.- Documentation/encoding guards: 8 passed;
git diff --checkpassed.
No desktop automation, manual picker/drop acceptance, provider call or Release installation was performed. The running GUI must use a new build to receive the fix.
Text-only continuation for client image attachments¶
At the user's request, valid attached images no longer cause run admission to
return vision_unsupported for a text-only model. Both create routes retain the
encrypted client-visible image payloads. Request preparation uses the pinned
capability to send either image parts or an explicit unavailable-image notice.
Provider/model selection, validation bounds, unreadable-payload failures and
workspace-image tool gates stay unchanged. ADR 0040 records the behavior.
Six real-protocol request-projection cases and two admission cases failed before the change. The final focused checks passed: Runtime.Model 140 (including 16 new cases), Runtime.Adapters 318 and Host admission/snapshot/idempotency/runner 74. During iteration, three runtime image tests and one Host image test exposed fake model fixtures with no vision declaration; their positive-image fixtures now explicitly declare image input, preserving the original assertions. The new negative cases cover text-only selections, image-only prompts and a false pin overriding a vision-looking model name. Positive cases cover true pins overriding opaque model names. All three protocol bodies are exercised, as are repeated preparation, prompts already in history and ordinary text-only turns.
Final verification on the changed tree:
scripts/test_dotnet.ps1 -Configuration Debug: exit 0; 9,401 passed, no failures or skips across all 13 projects (Runtime.Model 140, Host 1589, GUI 1988)..venv/Scripts/python.exe -m pytest tests -q: exit 0; 268 passed, 1 Windows platform skip for the POSIX process-group test.- Documentation/encoding guards: 8 passed;
git diff --checkpassed. - Local logs:
artifacts/text-only-images-20261030/dotnet-matrix.logandartifacts/text-only-images-20261030/python-matrix.log.
These checks establish admission, persistence and actual protocol-body projection; they do not measure a real model's wording. No real-provider request, native GUI acceptance or Release installation was performed.
Image follow-up context and user message grouping¶
The next user screenshot exposed a gap after text-only image continuation: completed history paired only nonempty text and lost attachment receipt facts. An image-only turn vanished from the context pair. Two new history cases failed before the fix; a pruned-history case passed. The fix carries source association while pairing turns, counts image payloads only for canonical user messages still visible in that session, and captures a model-only receipt fact with the historical prompt. Fork prefixes retain their association. The displayed text is unchanged, no historical image bytes are replayed, and counts cannot expose another session's images. The pinned image capability now appears in the prompt environment with explicit receipt-versus-vision guidance, including on later text-only turns.
The user clarified that image/file attachments must be outside the chat bubble. A source-tree regression failed on the former nested attachments. The new layout keeps the attachment row and text-only bubble separate, supports horizontal overflow, hides an empty text bubble and retains message copy/rewind and image opening. The existing dark theme, reading column and thumbnail treatment remain.
Focused checks passed: Host history/bootstrap/prompt 46, prompt capability 3, GUI layout/attachment/message/radius checks 76, documentation/encoding guards 8. The layout detector reported no findings; it is a source scan, not native pixel acceptance. No desktop automation or real-provider request was used.
OpenCode Go provider¶
The reported symptom was that an OpenCode Go API key could not be added in Settings.
Diagnosis: Models.dev publishes each OpenCode gateway's chat base
(https://opencode.ai/inference/go/openai/v1), where GET /models answers 404. Ru
appends /models to that base for both the catalog and the candidate-key probe, so
the Host recorded catalog_http_error, the probe mapped it to
provider_key_unverified, and the GUI refused to store any key. A second blocker sat
behind it: OpenCode Go answers every completion that carries no x-opencode-session
with HTTP 400 MissingSessionID before it reads the key.
Live probes with a synthetic bearer: GET .../inference/go/openai/v1/models answered
404 with no credential, an invalid bearer and a session header; GET
.../inference/go/v1/models answered 200 without a key and 401 with an invalid bearer,
which is what makes it a usable credential check. Running the Host's own
ProviderCatalogService.CheckCandidateCredentialAsync against the live gateway
returned catalog_http_error for models.dev:opencode-go and models.dev:opencode
before the change, and catalog_unauthorized after it; models.dev:openrouter stayed
ok with 456 models as the control. A key that never got stored is consistent with the
first result, not with a bad key.
The fix is table-driven: ProviderOverrides maps both ids to their gateway model list,
and only while the resolved endpoint still points at opencode.ai, so a catalog entry
that moved keeps its own URL instead of receiving the key there. ProviderSelection.
ModelsUri reads that table; chat completions keep the published base. OpenCode Go's
conversation header is pinned into ProviderBootstrapData, filled from the run's
SessionId when the step request is built, and applied by the Chat Completions
adapter together with Ru's own User-Agent. A pinned provider that needs the header and
has no session fails closed as provider_session_missing.
Checks: Adapters 320, Runtime.Model 142, Harness.Core 901, RuBackend 1598 - all passed
(artifacts/opencode-go-matrix.log). New tests cover the catalog URL for both
gateways, the host-mismatch fallback, the pinned header end to end through the snapshot
factory, the probe's URL and status mapping, header bounds and the missing-session
failure. No real OpenCode Go key was available, so live acceptance of a valid key and a
streamed completion was not measured; the 404/401 evidence does not depend on key
validity. Documentation updated: docs/operations/provider-catalog.md,
docs/architecture/providers.md, CHANGELOG.md.
The Settings Test connection button then still reported catalog_unauthorized while
chat worked. The Host was proven healthy with the stored credential: its own catalog call
returned status=ok with 28 models, and replaying the client's sequence
(POST /settings then GET /models?backend=models.dev:opencode-go&refresh=true) returned
success/ok. The defect was in the desktop client: the 30-second manual-refresh throttle
suppressed a forced refresh and the caller still read State as a fresh verdict, so the
pre-key catalog_unauthorized could be shown after the key was saved. An explicit test and
the refresh that follows a saved key now bypass that throttle (the Host keeps its own retry
throttle), a suppressed refresh is reported as such instead of as a verdict, the verdict
names the tested provider, and [provider-test] backend=… refreshed=… code=… is traced so a
wrong-row test is visible. gui-winui3.Tests 1992 passed. No real-provider acceptance of the
button was possible; the Host-side answers above are the measured evidence.