Verification evidence — 2026-10-11¶
Historical snapshot only. Counts, commands and limitations describe that snapshot and do not establish current acceptance. Use current verification.
GUI/backend review fixes (2026-10-11)¶
All seven findings in that review
have implementation changes and regression coverage: owned run workers, stale
stream/history settlement, deleted-run SSE cleanup, provider key probes, latest
Files refresh and export workspace resolution. That full Debug matrix passed
5,957 tests across 12 projects (Backend 1,049; GUI 1,352;
artifacts/gui-backend-fixes-dotnet.log). Its initial export test requested
unsupported writable formats; the test setup was corrected to the catalog's
CanWrite set (artifacts/gui-backend-fixes-dotnet-initial.log).
documentation guards passed 19 tests. Native race acceptance, live providers,
Release publish and the full Python suite were not rerun in this follow-up.
Prior Release/native/Python evidence in this document remains dated evidence.
Transcript scroll regression (2026-10-11)¶
The user reported upward jumps both on session open and while reading history.
Pure viewport-policy tests reproduced the prior calculation errors: 8 failed,
2 passed (artifacts/gui-scroll-red.log). The fix rejects inconsistent extents
even with follow enabled, preserves follow intent during layout changes, realizes
the latest item on open and restores a visible-item anchor on bulk reload.
Deferred callbacks check session, transcript and reader-input generations.
The first compilation failed because importing Windows.System made
DispatcherQueuePriority ambiguous; a VirtualKey alias corrected it. The next
focused run passed 40 tests and failed one source assertion: it prohibited scrolling
even in the existing branch that does not reset the list. That assertion now covers
the post-reset branch, while behavioral tests cover the viewport policy. Logs:
artifacts/gui-scroll-focused.log, artifacts/gui-scroll-focused-green.log.
The scoped command scripts/test_dotnet.ps1 -Configuration Debug -ProjectFilter
gui-winui3 passed 1,339 / 0 failed / 0 skipped
(artifacts/gui-scroll-final-gui.log). These tests alone did not establish
native acceptance; the next user-authorized GUI probe reproduced an additional
virtualized-panel failure.
build.bat passed 11/11 stages (artifacts/gui-scroll-release.log). The
documentation checks (test_current_documentation.py, test_guidance_route_counts.py,
test_readme_counts.py) passed 19 tests (artifacts/gui-scroll-docs.log).
Native GUI follow-up¶
With explicit user authorization, the Release GUI opened the latest stored
session (12 messages). A native UI Automation request to scroll to 100% ended
at 0% after transient measurements around 34%; physical wheel input also
exercised history scrolling. The first calculation-only repair was insufficient.
Evidence: artifacts/gui-scroll-native-bottom-samples.json and
artifacts/gui-scroll-uia-baseline.json. Temporary layout diagnostics showed a
constant message count with changing extents and visible-item indices, rather
than a history reset.
Setting ItemsUpdatingScrollMode="KeepScrollOffset" alone did not settle the
latest position. The final change also schedules a guarded follow attempt after
a completed view change when follow mode is active and realized row heights
have moved the bottom. Reader input still cancels following and deferred work.
Temporary diagnostics were removed from the shipped code.
On the final Release, opening the same session without another scroll
request reached 100% during the observation window and remained there
(artifacts/gui-scroll-uia-final-open.json: 25 samples, 200 ms apart). Wheel
input moved up into history to 79.21%, then back down to 100%; screenshots before,
after and 1.5 seconds after each gesture are under
artifacts/gui-scroll-native-final-history-{up,down}-*.png. This is acceptance
for that stored, idle session; it does not cover every session, active streaming
or a turn-end reload. Initial virtualization still needs layout settling.
Final checks: 1,340 GUI tests / 0 failures / 0 skips
(artifacts/gui-scroll-native-final-gui.log), 11/11 Release stages
(artifacts/gui-scroll-native-final-release.log). The full backend/process
matrix was not rerun for this local GUI change. A diagnostic attempt to publish
only the GUI over the flattened directory made Host startup fail; rebuilding
through build.bat restored the complete distributable before final acceptance.
Review remediation (2026-10-11)¶
An external review pass over the whole tree produced 11 high, 16 medium and a set of low findings; all high findings and the large majority of the rest are repaired in this pass. The highlights, by area:
- CI/tooling.
scripts/check_python_runtime_boundary.pytreated the GUI's vendored browser assets (xterm.js,addon-fit.js) as a shipped interpreter and exited 1 on the repository itself while CI runs it bare. The two paths are now the pinned, exact-path exception; the docs record the enforcement. - Provider runtime. A 120-second idle watchdog in the SSE reader turns a
silent provider into the retryable
provider_timeouton all three protocols; stream-level error events keep their HTTP retryability (rate_limit_error/api_error,server_error/overloaded); a.done-only Responses arguments stream is accepted; emptymodel_toolsomits the key on every protocol; extra sibling fields on a valid Anthropic block are ignored. - Harness. Batch aborts revoke the approvals they registered; cleanup never
replaces the original exception; a retried tool execution publishes a reset
frame (and the client model now parses the always-emitted
reset: nullfield that had made every live tool-output frame unparsable); the two native-history projectors share one matcher with one identity guard. - Persistence. Infrastructure SQLite failures in a work-state update abort the commit; recovery consume for a missing run fails closed; a failed migration rollback no longer masks the error; request-path schema initialization is memoized per database path; the conversation outbox spends a bounded retry budget before failing an intent.
- CLI/RPC/SDK. TUI wind-downs cancel the in-flight run and report discarded
queue entries; Esc between turns is remembered; panel requests answer a
wind-down with a cancel;
run/attachreport a failed cancel; bridge backpressure drains throughRunAsyncwith exit code 3 instead of killing the process from the pump thread; filtered or unreadable durable frames end the subscription withrun.detached;run.detachaccepts a run id for attach cleanup; SDK handler exceptions become warnings; stdin backpressure is respected; cmd metacharacters in.cmdspawn arguments fail closed. - GUI. Session/workspace/diagnostics async-void handlers catch their own failures; the last session click wins a fast switch; settings loading always clears its flag; trace lines go through one bounded background writer.
Evidence: the full 12-project matrix passed 5,909 tests / 0 failures / 0
skips (preset adoption + mode hardening; this run), the Python suite
passed 214 with one Windows/POSIX skip
(artifacts/review-remediation-full-python.log), the SDK typecheck/test/build
passed 22 tests, and the scripted RPC smoke passed end to end (PASS, terminal
run.event at sequence 7). One CLI matrix run flaked once on a timing-sensitive
test (1 failure; two reruns green, including the final full matrix) and is
recorded here rather than retried away. Deferred with reasons: the reply-effect
failure keeps its existing durable best-effort activity marker; a Length-finish
step deliberately does not release held-back text (the whole step is suppressed);
InteractiveLineReader's key loop and the reconciliation-target scan keep
their existing shapes, recorded here rather than rewritten in this pass.
Dialog collision fix (2026-10-11)¶
A GUI bug report showed "Coding Agent could not start: An async operation was
not properly started. Only a single ContentDialog can be open at any time."
after sending a message into a workspace that still needed trust. WinUI allows
one ContentDialog per XamlRoot, so the trust prompt's ShowAsync collided with
another dialog (open or still closing), the trust path projected that as a start
failure, and the run was marked handled — leaving the parked run with no approve
affordance. Every app dialog now goes through ContentDialogGate, which
serializes them and retries the single-dialog condition briefly; a prompt that
still could not open no longer marks the run handled (the monitor refresh
retries it after a short cooldown) and reports a bounded status instead of a
start failure. Source-contract tests pin that every new ContentDialog site
goes through the gate and that no site calls ShowAsync directly. GUI tests
passed 1,303 including the two new gate contracts; the full matrix above is the
verification.
Tool boundary repairs (2026-10-11)¶
The four outstanding findings at 50fd2e28 are repaired: web fetch pages no
longer split UTF-16 surrogate pairs, read windows cannot return a non-EOF
zero-progress page, grep strips the UTF-8 BOM before searching, and glob
traversal observes deadline/caller cancellation after its worker starts.
Small windows and offsets inside a character return named, repairable errors;
malformed UTF-8 at EOF is rejected instead of generating an endless cursor.
Protocol descriptions and the tool guide document these recovery actions.
All 12 initial regression cases failed before repair
(artifacts/tool-boundary-red.log). The affected focused set passed 51 tests
(artifacts/tool-boundary-green.log); adding caller-cancellation coverage
produced 13 passing boundary cases (artifacts/tool-boundary-final-focused.log).
The final 12-project .NET matrix passed 5,879 tests / 0 failures / 0 skips,
including 941 Broker tests (artifacts/tool-boundary-full-dotnet.log).
The original synthetic probe was repeated: BOM/no-BOM searches now agree,
small file windows fail with the repair code, web pages survive strict UTF-8
encoding and JSON roundtrip, and three 1 ms glob calls return timed_out
(artifacts/tool-boundary-smoke-green.log). Cancellation is cooperative;
these timings do not establish a hard real-time deadline.
The Python suite passed 213 tests with one Windows/POSIX skip
(artifacts/tool-boundary-full-python.log). Final bundled Help checks after
README updates passed 12 tests (artifacts/tool-boundary-final-help.log).
build.bat passed all 11 Release stages and refreshed the self-contained GUI,
CLI and Host package (artifacts/tool-boundary-release.log).
The prior discovery/web evidence below remains historical. These are tool
correctness checks, not a new real-provider coding benchmark or native UI trial.
Discovery and output-projection upgrades (2026-10-11)¶
Commit b00282f0 records the preceding web repairs. Seven new discovery
regressions failed before this upgrade (artifacts/tools-discovery-red.log):
recursive globs excluded root files, a literal wildcard placeholder name was
misread, long-line previews omitted their match, exact final pages claimed
more results, no explicit next cursor was returned, and small queries read
all candidate files. The focused discovery run then passed all 41 tests
(artifacts/tools-discovery-green.log). Glob/grep now share a literal-aware
nonbacktracking matcher. Grep consumes ordinal bounded batches, collects only
up to the offset/page/lookahead bound, and exposes actual scanned counts,
preview columns and separate scan-incompleteness/next-page metadata.
A Host projection regression also failed before repair
(artifacts/tools-discovery-projection-red.log): a scan/time bound with no
known next page invented a nonadvancing model cursor. Modern receipts now use
next_offset for continuation; legacy receipts retain the old truncated
compatibility path. Two web projection regressions failed before repair
(artifacts/tools-web-projection-red.log): generic clipping lost structured
JSON/cursors, and a full fetched page with more pages supplied no model cursor.
Structured web projection preserves its wrapper/marker and counts only the
extracted text for a UTF-16 cursor. The final focused Host run passed 32 tests
(artifacts/tools-web-projection-green.log).
The first full discovery matrix stopped on one Help inventory failure:
927 Broker tests passed, but longer documentation displaced op=write from
its bounded excerpt (artifacts/tools-discovery-full-dotnet.log). Splitting
the details into a dedicated documentation section fixed the bundled Help
output without weakening the inventory assertion
(artifacts/tools-discovery-help-green.log). A subsequent 12-project discovery
matrix passed 5864 tests before the two web projection cases were added
(artifacts/tools-discovery-full-dotnet-final.log). The final 12-project
matrix passed 5,866 tests / 0 failures / 0 skips, including 928 Broker and
1043 Host tests (artifacts/tools-upgrade-full-dotnet.log).
An actual-service read-only probe searched the real Protocol checkout through
ToolExecutionService, using a synthetic session with only workspace/system
read scopes, no provider requests or user credentials
(artifacts/tools-upgrade-smoke/report.json). glob **/*.cs returned 37 paths,
including 15 root files. Three grep web_fetch pages followed cursors 0, 1, 2
and ended with no next cursor; the first page searched 8 of 37 discovered
files, while the final page scanned the remainder to establish completion.
This is functional and file-count evidence, not a timing benchmark or
live-model/native-UI acceptance. Free upstream search relevance is unchanged.
The full Python suite passed 213 tests with one POSIX-only skip on Windows
(artifacts/tools-upgrade-full-python.log), and build.bat passed all 11
Release stages (artifacts/tools-upgrade-release.log).
Web-tool repairs and remaining upstream limits (2026-10-11)¶
Eight new replay cases failed on the original implementation
(artifacts/web-quality-red.log): known-engine navigation anchors were accepted,
Bing used a breadcrumb rather than the result heading/destination, JSON scalar
responses escaped as an unbounded parser exception, plain code lost whitespace
and generic syntax, and ordinary documentation exceeded the download bound.
The final repair uses AngleSharp 1.8.4 for local DOM parsing, preserves code,
text and links, and separates the 4 MiB download ceiling from 32000-character
output pages. An initial restore with AngleSharp 1.4.0 was rejected by the
existing NuGet audit policy (artifacts/web-anglesharp-restore.log); the final
version restores and builds without that warning. No audit rule was weakened.
The complete sequential Debug matrix passed 5,856 tests / 0 failures / 0
skips, across all 12 projects (artifacts/web-repair-full-dotnet.log). Broker
contributed 921 passing tests, including 19 new adapter/executor cases. Paging
is also checked through the real service approval path. Documentation guards
passed 15 tests (artifacts/web-repair-docs.log). The first full Python run
had 1 failure, 212 passes and 1 skip because README's older 5809-test badge did
not match the already-recorded 5837-test run
(artifacts/web-repair-full-python.log); README and the current verification
row now reflect the new 5856-test run. The final full Python rerun passed
213 tests with one Windows-inapplicable POSIX skip
(artifacts/web-repair-full-python-final.log). build.bat then passed all
11 stages and published the updated self-contained distribution, including
AngleSharp (artifacts/web-repair-release.log). Historical figures below
remain intact.
The final direct-service public-HTTP rerun made 12 calls, with synthetic
sessions/approvals and the production connection policy, no keys or model
requests (artifacts/web-live-check/report.json,
artifacts/web-live-check/run-final.log). All five text URLs succeeded:
Example Domain, Microsoft Learn C# async, Python sqlite3, TypeScript narrowing,
and the raw TypeScript README. Python sqlite3 was read to completion over four
pages (offsets 0, 32000, 64000, 96000), including transaction control. The PDF
probe still correctly failed as web_content_type_not_supported; PDF extraction
and browser/JavaScript execution are outside this HTTP text tool.
Search no longer accepts DuckDuckGo's homepage or Mojeek navigation as results.
The isolated engine rerun rejected those layouts and Google as
web_search_parse_failed; Bing supplied actual titles, snippets and decoded
destination URLs (artifacts/web-engine-check/report.json). However, all three
live coding queries still returned weak or unrelated upstream Bing results.
The query echoed in Bing's response was complete, and ordinary English-market
and RSS diagnostic variants did not fix relevance. Additional public Brave
and Yahoo probes returned HTTP 429 and HTTP 500 respectively
(artifacts/web-engine-check/extra-sources.log); they were not added to the
engine set. No CAPTCHA was solved and no browser session was used.
This repairs the parser and fetch defects; it does not establish reliable live search relevance. Free engines remain dependent on their upstream availability and ranking. Deterministic tests, live HTTP text retrieval, live-model behavior and interactive UI acceptance are separate evidence.
Live web-tool diagnostic (2026-10-11)¶
On checkout 18180d7c, 47 focused Broker web adapter, executor, capability and
connection-policy tests passed (artifacts/web-tools-focused-tests.log). A
separate diagnostic then called the real ToolExecutionService, with synthetic
session scopes, one-use approvals, the production HTTP connection policy and
default enabled web options (Google plus the fixed fallback race, 256 KiB
response bound). This used public HTTP requests, no model generation, no user
session and no API keys. Report: artifacts/web-live-check/report-before-fix.json.
All three coding-oriented search queries reported success but returned only
the DuckDuckGo homepage with title here, rather than query results. An
engine-by-engine check of the Python query confirmed Google failed to parse,
DuckDuckGo returned that same false result, Bing returned tracking URLs and
placeholder snippets, and Mojeek returned navigation links. DuckDuckGo's saved
HTTP 202 response was a bot-challenge page. Known-engine parsers fall through
to generic absolute-anchor extraction when their expected result blocks are
absent; this allows a challenge/navigation link to win the race and cancel
other engines. Evidence: artifacts/web-engine-check/report-before-fix.json and
artifacts/web-engine-check/duckduckgo-response.html. No challenge was solved.
Of six fetch URLs, Example Domain, the Microsoft Learn C# async page and
the raw TypeScript README succeeded. Python's sqlite3 documentation and the
TypeScript narrowing handbook failed with web_response_too_large. The PDF
probe failed with the expected web_content_type_not_supported. Fetch is
HTTP text extraction without JavaScript execution; its regex sanitizer flattens
whitespace and does not preserve code-block layout or links as Markdown.
This is one small live sample from this machine, not a stability or relevance benchmark. It demonstrates a false-success search defect and default fetch limits that exclude ordinary documentation pages. Production code and web configuration were not changed in this assessment.
Indexed Chat Completions call repairs (2026-10-11)¶
Commit bb27c06f records the preceding terminal-argument and truncation repairs.
The subsequent SSE replay pass reproduced four indexed-call failures through
the real Chat Completions adapter and model runner: interleaved arguments were
attached to the wrong call, a repeated id started a new call, a split function
name was rejected, and out-of-order opening chunks reversed call order. All
four failed before the repair (artifacts/indexed-chat-tools-red.log) and
passed after it (artifacts/indexed-chat-tools-green.log).
The adapter now assembles indexed calls in a separate bounded component and
emits them in index order at completion. Wholly unindexed streams retain the
existing compatibility path; mixed indexed/unindexed metadata, conflicting ids
and oversized arguments are rejected. A replay with two indexed calls lacking
provider ids also reproduced an ambiguous-call failure
(artifacts/indexed-chat-tools-missing-ids-red.log); distinct generated ids now
keep those calls separate. During iteration a regression was caught where an
argument-only chunk's usage was dropped (artifacts/indexed-chat-tools-usage-red.log)
and corrected before final verification. Indexed length output still returns
no executable calls through the preceding truncation repair.
Final verification: scripts/test_dotnet.ps1 -Configuration Debug passed all
12 projects, 5,837 passed / 0 failed / 0 skipped
(artifacts/quality-indexed-chat-full-dotnet.log), including the 15 new indexed
SSE replay cases. Current documentation and route-guidance checks passed 15
tests (artifacts/quality-indexed-chat-docs.log), and git diff --check passed.
These are deterministic SSE replay results, with synthetic credentials and no network provider calls. Another live-model benchmark, Release package build and interactive UI acceptance have not been run for this repair.
Tool argument and truncation repairs (2026-10-11)¶
An omitted bash run-mode argument vector now means an empty vector, matching
the advertised schema. Mode-specific argument parsing retains its named error
instead of flattening it to adapter_error. The model runner preserves a
provider's length finish before parsing native calls or text selections and
returns no executable calls or continuation for that incomplete step. The
Harness's existing bounded truncation recovery stays in charge.
Three terminal regression cases failed before the fix
(artifacts/terminal-arguments-red.log): omitted arguments, missing executable
and missing shell script. All eight launch-feedback cases passed after the fix
(artifacts/terminal-arguments-green.log), and the Broker suite passed 902 tests
(artifacts/terminal-arguments-broker-suite.log). Three model-step cases also
failed before the fix: a complete native call was promoted despite length,
an incomplete native call threw a parse error, and a text selection was
promoted. All three pass after the fix (artifacts/truncated-step-red.log,
artifacts/truncated-step-green.log).
Final verification: scripts/test_dotnet.ps1 -Configuration Debug passed all
12 projects, 5,822 passed / 0 failed / 0 skipped
(artifacts/quality-tool-arguments-truncation-full.log). The focused current
documentation and route-guidance guards passed 15 cases
(artifacts/quality-tool-arguments-truncation-docs.log), and git diff --check
passed. The Release package, full Python suite and interactive native UI were
not revalidated in this repair pass.
Trace inspection of run 3 found that Python's empty write arguments occurred
at 8,192 output tokens, while the provider reported tool_calls, not length.
The truncation repair above does not change that upstream finish marker or
raise the pinned output limit. Python and TypeScript also supplied shell
scripts with mode=run; their existing malformed-call feedback correctly
requested shell mode. These repairs have not yet been evaluated by another
real-provider benchmark, so the latest measured resolution remains 0/3 below.
Post-terminal-fix SWE rerun (2026-10-11)¶
For the subsequent ten-task Standard baseline and its matched follow-up probes, see the single-agent readiness diagnostic (2026-10-13, retained in git history). The runtime, preset and difficulty mix differ from this older three-task run.
The three custom SWE tasks were rerun on committed 96f658c2, using the
rebuilt Release Host/CLI, Kilo stepfun/step-5-preview-free, chat_completions,
confirmed full_access, one fresh workspace/session per task and one attempt
with a 300-second agent deadline. Each task's fixture, effective instruction
hash and Git baseline match run 2. Graders and reference solutions were not
supplied as model input. This rerun does not cover the six Terminal-style tasks.
| Task | Run outcome | Agent seconds | Passing groups | Reward |
|---|---|---|---|---|
cs-lease-renewal |
timeout | 300.016 | 6/7 | 0 |
py-migration-rollback |
timeout | 300.013 | 2/6 | 0 |
ts-session-replay |
timeout | 300.013 | 2/7 | 0 |
C# changed only src/LeaseRegistry.cs and still failed expiry/takeover behavior.
Python and TypeScript left source unchanged and exported empty patches; their
passing groups are baseline preservation behavior. Durable receipts record two
malformed Python calls, one malformed TypeScript call and one nonzero C# command.
All three timed-out runs were durably cancelled, with no nonterminal run left
in the isolated database, and the diagnostic Host was stopped. Runtime binary
hashes remained unchanged throughout the run.
Overall resolution remains 0/3. Run 2's Python candidate passed 6/6 groups;
this attempt produced no Python repair. One stochastic attempt per task cannot
attribute that difference to the terminal fixes or establish stable improvement.
Reports, trajectories and patches are under
artifacts/quality-swe-20261011-run3/; comparison-and-runtime-audit.json
records matching fixtures and final cancellation evidence. This is live-model
evidence on a small custom corpus, not an official SWE-bench score or native UI
acceptance.
Terminal launch and exit-code repairs (2026-10-11)¶
Windows process creation now opens and retains the Process handle while the
child is suspended. A child that exits before the caller waits still retains
its real exit code. Executable and shell resolution run before task
registration, retaining executable_not_found and executable_not_trusted
instead of collapsing them into process_start_failed or reporting an
unresolved background task as started. Execution permissions are unchanged.
The two exit-code regression cases failed on the old implementation
(artifacts/quality-process-exit-red.log) and all six suspended-process tests
passed after the repair (artifacts/quality-process-exit-green.log). The valid
missing-executable regression failed for foreground and background requests
(artifacts/quality-launch-feedback-valid-red.log); the first fixture attempt
omitted the optional argument array and exposed a separate executor/schema
mismatch, now covered by the subsequent argument-vector repair; that first log
is retained separately. Focused launch/streaming checks passed 20 cases
(artifacts/quality-terminal-green.log). A local diagnostic then ran Python
directly and through cmd, retaining exit code zero in both cases
(artifacts/quality-terminal-diagnostic.log). This does not change which bare
executable directories are trusted.
Environment failures are retained: the first full matrix was interrupted after
a 512 MiB fixture could not be allocated on C
(artifacts/quality-terminal-full-dotnet.log). A temporary root inside the
checkout invalidated a no-repository fixture
(artifacts/quality-terminal-full-dotnet-retry.log). Moving that root outside
the checkout on D passed Broker/CLI/client/core checks, then failed five
Windows key-store ACL tests (artifacts/quality-terminal-full-dotnet-final.log).
Both volumes are NTFS; the D-root ACL setup was incompatible with those tests.
The final invocation uses the normal Windows temporary directory after C free
space recovered. No permission tests or key-store implementation were weakened.
Final verification: the normal-temp full .NET matrix passed 5,815 tests in
all 12 projects, with no failures/skips
(artifacts/quality-terminal-full-dotnet-complete.log). Documentation guards
passed 9 tests (artifacts/quality-terminal-docs.log); git diff --check
passed. The full Python suite was not rerun for these C# changes; its preceding
213-pass result remains recorded separately.
Release publishing completed all 11/11 stages
(artifacts/quality-terminal-release.log).
No new live-model benchmark or interactive native UI acceptance is claimed for these launch fixes. The earlier custom SWE results remain unchanged.
Coding recovery and feedback repairs (2026-10-11)¶
Suppressed repeated-tool proposals are retained for internal audit, without orphan native tool calls in model history. Repetition advice still counts toward its existing bound and cannot hide a failed receipt from completion checks. Terminal start/exit/timeout failures now preserve an exit code and bounded, already-redacted diagnostics for model repair. Windows shell guidance names PowerShell explicitly. No provider fallback or permission-policy change was made.
Regression evidence:
- Release distributable rebuilt successfully: 11/11 stages
(
artifacts/quality-release-final.log). - Final full Python tooling suite: 213 passed, 1 skipped
(
artifacts/quality-pytest-complete.log), including the cache-patch regression. The intermediate run passed 212 (artifacts/quality-pytest-final.log); the Windows skip remains the POSIX process-group case. - Final full .NET invocation: 5,809 passed / 0 failed / 0 skipped, all
12 projects (
artifacts/quality-dotnet-complete.log). - The first full matrix stalled in CLI and was interrupted; its log is
artifacts/quality-dotnet-final.log. The isolated CLI diagnostic with--blame-hang-timeout 90s --blame-hang-dump-type nonecompleted all 1,278 tests (artifacts/quality-cli-diagnostic.log). The hang did not reproduce; its cause remains unconfirmed and is not claimed fixed. - Durable SQLite/native-provider history: two causal RED failures in
artifacts/quality-repetition-red3.log, then two passing recovery cases inartifacts/quality-repetition-green2.log. Earlierred/red2logs record corrected test setup/compilation failures, not the reproduced production bug. - Advice-hidden repetition: two RED cases in
artifacts/quality-guard-red.log. Advice-hidden verification failure: one RED case inartifacts/quality-false-success-red.log. Core subsequently passed 481 tests (artifacts/quality-core-green.log). - Missing compiler diagnostics: receipt mapping and a real failed PowerShell
child both failed before the fix (
artifacts/quality-terminal-feedback-red.log). All seven related mapping/history/process checks passed afterward (artifacts/quality-terminal-feedback-green.log). Existing terminal redaction coverage is retained; these diagnostics are private receipt content, not public SSE. - Benchmark instructions previously omitted immutable-file and scratch-file
restrictions enforced by grading. A failing prompt regression reproduced
this omission; all nine benchmark tooling tests passed after disclosing the
constraints. Logs:
artifacts/quality-benchmark-prompt-red.log,artifacts/quality-benchmark-tests.log. Grading criteria remain unchanged; prepared trials now record the effective instruction hash. - Live Python exposed cache files entering patch export despite tree grading
excluding them. The staged-cache round-trip regression failed before the fix
(
artifacts/quality-benchmark-patch-red.log); all 10 benchmark tests passed afterward (artifacts/quality-benchmark-tests-final.log). The actual Python candidate source patch was replayed into a fresh baseline, passed every group, and retained the same candidate hash (artifacts/quality-swe-20261011-run2/py-migration-rollback/patch-validation.json).
Live evidence uses the same configured Kilo stepfun/step-5-preview-free,
chat_completions, one attempt per task, isolated ephemeral Host/data roots,
and disposable workspaces. It does not send reference solutions to the model.
- Small shipping-boundary C#:
standardparked for the existing terminal approval (artifacts/quality-small-csharp-20261011-run1/report.json). A separate explicitly confirmedfull_accesstrial completed in 70.906 s, changed onlysrc/Shipping.cs, and passed the independent grader (artifacts/quality-small-csharp-20261011-full-access-run1/report.json). This used intermediate Debug binaries; it is not a controlled before/after score. -
Initial hard custom SWE diagnostics, 300 s/task,
full_access: C# and Python timed out; TypeScript failedmodel_output_truncatedafter 284.860 s and failed five of seven groups. C# changed a project file; Python left a scratch file. The original prompts did not disclose these integrity restrictions, so the resulting 0/3 is diagnostic evidence with a prompt defect, not a fair quality score. Reports:artifacts/quality-swe-20261011-run1/. -
Final hard diagnostic used rebuilt flat Release binaries, disclosed final-workspace constraints and 300 s/task; binary and instruction hashes are retained in
artifacts/quality-swe-20261011-run2/manifest.jsonand eachtrial.json.
| Task | Agent outcome | Seconds | Grader groups passed | Live reward |
|---|---|---|---|---|
| C# lease renewal | completed; expiry semantics still incorrect | 294.992 | 6/7 | 0 |
| Python migration rollback | timed out; candidate passed every group | 300.017 | 6/6 | 0 |
| TypeScript session replay | failed model_output_truncated |
257.804 | 2/7 | 0 |
Live reward requires both a completed agent run and a fully passing candidate:
0/3 resolved, while 1/3 candidate patches passed all grading groups.
The corrected Python source-only patch also passed fresh-baseline replay;
its original prediction.patch retains the pre-fix cache-export failure as
historical evidence. No model-generated source was repaired by the evaluator.
Receipt metadata from three real failed commands confirms compiler feedback
reached private payloads (artifacts/quality-swe-20261011-run2/failure-feedback-evidence.json).
Reports and trajectories are under artifacts/quality-swe-20261011-run2/.
This run does not establish a controlled improvement rate: several harness and
prompt corrections were applied, there is one attempt, and the deadline is short.
These custom tasks and short single attempts are not official Terminal-Bench or SWE-bench results, a model-independent measure, or native UI acceptance.
Terminal OS sandbox retirement (2026-10-11)¶
ADR 0015 retires AppContainer profiles, ACL grants, capability attributes and sandbox escape UI/arguments. Terminal and managed Git launch as the current Windows user. Ordinary preset and approval policies, process-tree cleanup, cancellation, timeout and output bounds remain. A real standard-preset approved PowerShell child writes outside workspace/TEMP and connects an ephemeral loopback listener without a network capability, proving the former OS boundary is absent.
Credential storage and structured protected-path denials remain. Child processes do not inherit secret environment variables; all stripped values now seed terminal output redaction. Regression tests reproduced the old redactor leaking a key across its emission boundary (three RED cases), then verified chunk splits, longest matches and complete overlapping matches. A real process writes a synthetic key in separate chunks; both frames and receipts redact it. This does not claim that unrestricted current-user commands cannot retrieve credentials independently.
Verification:
- The first full .NET run failed an obsolete sandbox-only approval-ledger test;
that case was removed while ordinary approval-memory coverage was retained
(
artifacts/remove-sandbox-dotnet.log). - The second full run passed the first eleven projects, then failed one GUI
assertion expecting obsolete sandbox copy
(
artifacts/remove-sandbox-dotnet-green.log). The corrected GUI suite passed 1,299; the final Broker suite passed 894, including the fourth redactor regression (artifacts/remove-sandbox-gui-final.log,artifacts/remove-sandbox-broker-final.log). Combined with the unchanged projects from the full run, the total is 5,802 passing tests across 12 projects. A final complete invocation subsequently passed all 5,802 / 0 failed / 0 skipped (artifacts/remove-sandbox-dotnet-final.log). No failed assertion was weakened to permit an execution-policy regression. - Python tooling suite: 211 passed, 1 skipped
(
artifacts/remove-sandbox-pytest-final.log). The first run exposed stale documentation vocabulary and a README badge; both were corrected without changing the guards (artifacts/remove-sandbox-pytest.log). build.bat: 11/11 stages passed (artifacts/remove-sandbox-release.log), refreshing the flat distributable.- Redactor RED and initial GREEN evidence:
artifacts/remove-sandbox-redactor-red.logandartifacts/remove-sandbox-redactor-green.log.
The scripted CLI RPC smoke completed its session/run/attach/event workflow
(artifacts/remove-sandbox-rpc-smoke.log). It used an isolated ephemeral Host,
synthetic model and temporary data root; its ignored runner copy retained the
temporary directory instead of deleting it. This proves transport wiring.
A separate small C# shipping diagnostic used the new flat Release binaries and
the current user-selected Kilo stepfun/step-5-preview-free, chat_completions,
standard, one attempt and a 180-second bound. The pristine baseline failed the
exact-threshold assertion. The live run read files and issued tool calls, then
failed after 15.058 seconds with model_history_invalid, following a
malformed_call receipt and repeated_tool_call recovery note. No source file
changed and independent grading remained red. This trace does not establish
the root cause; sandbox retirement has not demonstrated improved coding-task
completion. Evidence: artifacts/ru-small-csharp-no-sandbox-20261011/manifest.json,
trajectory.json, report.json and prediction.patch. The run preserved the
model selection, shut down its own isolated Host and retained its data root.
No interactive desktop UI was operated.