Verification evidence — 2026-10-20¶
Historical snapshot only. Counts, commands and limitations describe that snapshot and do not establish current acceptance. Use current verification.
GUI review fixes (2026-10-20, scoped)¶
The candidate-key route adds one authenticated non-Agent operation, bringing the current contract to 46 HTTP / 0 WebSocket. The Host checks candidate credentials with a read-only credential source and isolated catalog cache before committing. Rejected, unavailable and cancelled checks preserve the previous credential; a settings-save failure exercises existing rollback. The GUI does not use ordinary settings writes as a fallback when the route is absent on an older Host.
- Host suite: 1,276 passed. A later focused run includes 11 passed for candidate rejection (401/403), rate limiting/outage, cancellation, rollback, live-cache isolation, bearer authentication and mixed-payload rejection. Six of these cases overlap the full-suite run; the figures must not be added together.
- GUI suite: 1,518 passed including the dedicated key route and older-Host 404 behavior. One previous source contract required deleting rejected keys; it was updated to require Host validation and prohibit GUI key removal.
- Route/documentation/metadata guards: 18 passed. The route matrix reports all 46 mapped operations declared. The initial expected-route-set failure and initial GUI contract failure are retained with the subsequent green logs.
- Native UIA and screenshots confirm first-Escape dismissal for both a Settings flyout and ComboBox, a readable stacked form at 800px window width, visible keyboard-focused sidebar actions and correct hide-on-leave/menu behavior, plus accessible names and selected status for all four transcript-size choices.
A task-local self-contained package was built through the 11-stage build script with task-local publish destinations and the broad clean skipped, followed by a fresh GUI build for ComboBox handling. Its authenticated status/shutdown smoke passed. Native checks used that package and an isolated business-data profile; only an empty QA conversation was created, and no real key was edited. The already-running canonical Host and its package were preserved.
Evidence: artifacts/gui-quick-review-2026-10-20/ (including package-build.log,
final-gui-build.log, package-smoke.log, test logs and fixed-*.png). The older
full-matrix/release entries above retain their original snapshot scope. No new
full 13-project matrix, live key validation against a real provider or Narrator
session was performed in this GUI-fix pass.
Live coding trials after reasoning-effort recovery (2026-10-20)¶
One native attempt per task through benchmark.py run-live with the existing
selected model (models.dev:kilo, minimax/minimax-m3.1-flash:free), standard
preset, a published Host built after the change under test (the trial starts
the Host from the publish folder, not from dotnet build output; earlier
same-day experiments that skipped dotnet publish measured the old binary and
are discarded). The pinned request_options_json was read from the harness
database to confirm which effort a run used. Single attempts on a free model; no
statistical claim.
| Condition | Result |
|---|---|
| Everyday suite, saved settings (no effort sent) | 6/6 passed, 94-217 s |
| Nine Terminal/SWE tasks, saved settings, before the change | 1/9; seven ended in model_output_truncated with no source change (reasoning filled the output) |
Same nine, RU_REASONING_EFFORT=low |
6/9 |
| Seven previously truncating tasks, saved settings, with automatic lowest-effort retry after a truncation | 4/7 passed after one truncation each; cs-single-flight-cache failed on a fourth truncation (cumulative cap, since replaced by a consecutive-notes cap); two failed one grader group |
ts-generation-pager, three attempts, saved settings |
0/3, all on same-generation-dedup |
ts-generation-pager fails because its grader also requires that the loader is
not called synchronously from loadNext() (checked before a microtask turn), a
requirement the instruction does not state; implementations returning the shared
promise still fail it. A prompt sentence asking for per-requirement edge-case
checks was tried and reverted: 1/3 on ts-transaction-store against 0/1 earlier
is not evidence. One run of cs-lease-renewal ended in runtime_drive_failed and
did not reproduce on rerun; the structured log now records the exception type
bucket so a recurrence is diagnosable (RU_DRIVE_FAILURE_DIAGNOSTIC also dumps
the full exception).
Deterministic checks for the same change: Harness.Core 537, RuBackend 1264,
Runtime.Adapters 228 and Runtime.Model 91 tests passed; Broker 1067 of 1068
passed in Debug (Assigned_job_kill_takes_down_child_and_grandchild fails with
and without the change in this environment).
Hard-task live run, 2026-10-20 (default settings)¶
Nine hard Terminal/SWE tasks x 2 attempts on minimax/minimax-m3.1-flash:free with the saved settings: 10/18.
Both attempts passed: cs-single-flight-cache, py-migration-rollback, py-stream-parser. One of two: cs-atomic-ledger, cs-lease-renewal, py-dependency-scheduler, ts-transaction-store.
Both failed: ts-generation-pager (grader requires unstated behavior) and ts-session-replay (malformed events are rejected without mutation).
One runtime_drive_failed (ArgumentException right after a tool.proposed) was seen twice across all runs and did not reproduce in three further cs-lease-renewal attempts (0/3 drive failures, 1/3 passes). Cause still unknown; set RU_DRIVE_FAILURE_DIAGNOSTIC to a file path to capture the exception if it recurs.
Single small sample on a free model; treat as variance-limited.
Follow-up with the exception_site Host, four of the variable tasks x 3 attempts: 6/12 (cs-atomic-ledger 2/3, cs-lease-renewal 2/3, py-dependency-scheduler 1/3, ts-transaction-store 1/3). No runtime_drive_failed in these 12 runs, so it has now been seen 2 times in about 35 live runs. The recurring failing groups are snapshot isolation (two tasks) and transition-integrity; checked against the reports: transition-integrity fails because replace(None) raises TypeError where the task text says invalid input raises ValueError; ts-transaction-store fails because the returned snapshot and its values are not frozen although the task asks for an immutable snapshot. These are model misses against stated requirements, not harness faults.
Prompt-guidance check (2026-10-20): nine hard tasks x 1 on the same model after the tool "when to use" guidance, the delegation line and the user-wide instruction file: 4/9 against a baseline of 10/18, no runtime_drive_failed or other system error, mean output tokens about 747 against about 640 to 690. The differences are within the run-to-run variation recorded above, so the change is accepted as a regression guard pass, not as evidence of improvement.