Skip to content

P1-004 — Current Harness And Automated Verification Reality Check

  • Status owner: P1-010 final ROADMAP schema — transition evidence only
  • Execution result: IN PROGRESS — CLOUD SOURCE REVIEW COMPLETE / LOCAL RO REQUIRED
  • Owning Phase: P1 — Product, Documentation, Benchmark And Repository Foundation
  • Lane: Cloud RO + Local RO
  • Parallel: possible
  • Depends on: none
  • Activation condition: the user's instruction to investigate the P1-004 harness source; no dependency
  • Unlocks: P1-009 after fresh Local execution evidence closes the runtime-behavior unknowns
  • Observation date: 2026-07-30
  • Source revision reviewed: qa@fd9ea998e50581144a9d9ead01b5182e046f4a79
  • Product value: prevents a verification command from being treated as trustworthy merely because it exists; identifies where the current Harness can miss risky changes, overstate evidence, or perform effects outside the declared authorization boundary before P1-009 changes it.
  • Stable planning input: ../requirements/CTC_PRODUCT_PLATFORM_AND_DOCUMENTATION_RESET.md
  • Current mechanism owner: ../../.agents/rules/CODEX_ENGINEERING_HARNESS.md
  • Executable contract: ../../.agents/harness/contract.json
  • Main runner: ../../scripts/verify-change.sh

1. Outcome And Completion Boundary

The following was completed in the Cloud source review scope of P1-004.

  • Cross-checked the Harness contract, planner, dispatcher, isolated command runner, preflight and evidence renderer of the current qa against the real deep-check target.
  • Distinguished the earlier master state from the current qa state, so a defect already fixed was not reported as a current defect.
  • Organised the false negatives, authorization mismatches, coverage gaps and evidence-integrity gaps provable directly from the current source into a findings register.
  • Wrote down, per change type, the difference between the current automatic verification and the minimum verification that is needed.
  • Produced a bounded handoff so that P1-009 can apply the needed corrections in order without building a giant framework.
  • Separated the realistic checks that cannot run on Cloud — deletion-only, committed-range, compiler, local DB, browser, timeout and process cleanup — into an exact Local test contract.

P1-004 as a whole is not yet COMPLETE.

  • This Task's lane is Cloud RO + Local RO.
  • In a connector-only environment the real command exit and result could not be reproduced in a disposable checkout.
  • The local DB, Playwright/browser, the PHP server, the credential source and child-process state after a timeout were not verified.
  • Therefore a finding confirmable through the source control flow is distinguished from a hypothesis that needs Local reproduction.
  • P1-009 must not implement a correction or treat current behaviour as PASS without Local evidence.

2. Evidence Classes

Class Meaning Use
CURRENT_SOURCE executable source/contract read directly from the current qa judging control flow, selection and declared effect
CURRENT_CROSS_FILE a fact confirmed by checking the contract, runner and target script together judging a declaration-versus-reality mismatch
CURRENT_HISTORY a fact about the scope the current qa changed after the prior master separating an already corrected defect from a remaining one
SOURCE_PROVEN_DEFECT the result is determined by the source path alone, with no separate Runtime run usable as P1-009 correction input
LOCAL_REPRO_REQUIRED requires running a process, filesystem, DB, browser, toolchain or timeout no observed PASS/FAIL assumption before Local evidence
POLICY_AMBIGUITY the source behaves consistently but the meaning of the metadata or terms is unclear the contract wording has to be decided in P1-006/P1-009
TARGET_ONLY a correction proposal in this document not a current implementation and not an approved change

3. Reviewed Source Inventory

Source Current role Reviewed blob / boundary Cloud result
.agents/harness/contract.json check order, aliases, surface routing, runner/effect metadata blob 0fe2e3537b3c4080dd3815e64d522fb6b8d60055 18 checks, 12 surface rules; structural declaration only
scripts/verify-change.sh changed-file discovery, plan filtering, dispatch, state aggregation blob 4feeae0885f29139b7efd281381975b783600119 current primary executable authority
scripts/harness/plan-checks.mjs glob routing, type alias resolution, runner inspection blob 97d24a56cc31664e1ccb0ee75403435de35b3940 source-level planner; bounded happy-path self-test
scripts/harness/validate-contract.mjs contract structural validation blob 9ef179c8806d9f3223cb9ff7d661d1874eb9be3a validates shape/reference consistency, not operational semantics
scripts/harness/run-contract-check.mjs allowlisted environment construction, timeout, child execution blob 7af3d2fd7433dff1738c38f6644257a4dce18166 inherited environment isolation exists; effect metadata is not enforced
scripts/harness/ctc-preflight.sh branch/HEAD/dirty/untracked/DB-host-class/recommendation snapshot blob 57f53a5dd3068a4e51b7cb472747b2b18c965a13 working-tree preflight; no candidate-base/range mode
scripts/harness/render-evidence.mjs external runtime-state evidence Markdown renderer blob 58892d4fe7869db065ccd6eafc097082bd87134d manual input renderer; weak cross-file/result validation
scripts/harness/render-evidence.sh renderer happy-path self-test blob cf7729d91ef376d4bd345490eccf4d5b82452f29 only successful fixture path tested
.agents/harness/schemas/*.json intended task/check/review/write/result shapes current qa present but not consumed by renderer or main runner
scripts/run-php-tests.sh flat PHP unit/integration/integration-db suite runner current qa suite discovery/effect boundary requires Local inventory
scripts/api-smoke.js authenticated local API smoke current qa directly loads .env; hidden DEV_ENV password dependency
scripts/smoke-local.sh local app/login smoke current qa loopback HTTP, DB audit and DEV_ENV password dependency
scripts/field_process_crud_smoke.php rollback-safe local DB CRUD/process smoke current qa explicit local-host/write flag and transaction rollback exist
scripts/field_process_ui_smoke.js browser/field E2E with fixtures, DB mutation and screenshots current qa DB/artifact effects; cleanup failure can be swallowed
voice-gateway/package.json, voice-gateway/tsconfig.json real strict TypeScript package build/test contract current qa proves a scoped compiler/test path exists but is not auto-routed
scripts/migrations/*.sql migration source current qa SQL files route only to generic source checks today
AGENTS.md repository-wide current safety/verification rule current qa requires explicit DB-write, smoke and E2E authorization flags

4. Current Execution Graph

working tree + untracked files
        + optional merge-base(base, HEAD)..HEAD
                         |
                         v
              changed-file list
                         |
                         v
        plan-checks.mjs surface/type routing
                         |
                         v
       automatic deep-check omission filter
                         |
                         v
       verify-change.sh runner dispatch
          |             |              |
          v             v              v
  built-in source   run-contract   manual / NOT_RUN
      checks          checker
                         |
                         v
     allowlisted env + timeout + child command
                         |
                         v
       PASS / FAIL / BLOCKED / NOT_RUN / N/A

Separate paths:

ctc-preflight.sh
  -> working-tree state + current HEAD + recommendations

manual task/check/review/write/result JSON
  -> render-evidence.mjs
  -> evidence.md outside repository

Important boundary:

  • verify-change.sh does not produce the structured evidence inputs consumed by render-evidence.mjs.
  • render-evidence.mjs does not verify that its supplied checks came from verify-change.sh or that they correspond to the recorded source SHA.
  • contract.json contains writes and network metadata, but the execution graph does not generically enforce those fields.

5. Current Corrections Already Present And To Preserve

The following prior defects are already corrected in current qa and must not be reintroduced.

Current behavior Result
Automatic path routing omits environment-bound integration/DB/smoke/E2E/Production checks PRESERVE — source-only default is now explicit
--base=<ref> includes committed candidate paths from merge-base to HEAD PRESERVE, while correcting the range-content check in H-004
Explicit --type with remaining NOT_RUN exits non-zero PRESERVE — no false complete result
run-contract-check.mjs starts from a safe base environment and adds only allowlisted keys PRESERVE, while preventing target scripts from bypassing it
api:smoke / app:smoke require --run-smoke PRESERVE
php:integration-db / db:field-crud require --allow-db-write PRESERVE
ui:field requires --run-e2e PRESERVE, but add the missing DB-effect authorization boundary
Contract metadata now admits API/app session/temp writes and UI fixture/artifact writes PRESERVE, then make metadata enforceable
--type=release is documented as source-only range selection rather than Runtime acceptance PRESERVE
Missing command and timeout map to BLOCKED rather than PASS PRESERVE

6. Findings Register

H-001 — Deleted Files And Test-Weakening Changes Can Be Invisible

  • Severity: CRITICAL
  • Evidence class: SOURCE_PROVEN_DEFECT
  • Current source fact:
  • both candidate-range and working-tree discovery use --diff-filter=ACMRTUXB;
  • D is excluded;
  • a deletion-only working tree therefore produces no changed-file plan;
  • the current contract has no test:integrity check;
  • no current runner detects test deletion, skip/only insertion, or removed/weakened assertions.
  • Consequence:
  • deleting one tracked test can return NOT_APPLICABLE with exit 0 under automatic verification;
  • deleting a Runtime/config/doc file also bypasses its surface-specific routing;
  • this directly conflicts with the rule that tests must not be deleted, skipped or weakened to pass.
  • Minimum correction candidate:
  • include deleted paths in discovery;
  • route deletion using the old path;
  • add a bounded test-diff integrity check for deleted tests and obvious skip/only/assertion weakening signals;
  • do not recreate the removed giant integrity framework.
  • Local closure: L-01.

H-002 — Field UI E2E Can Perform DB Writes Without --allow-db-write

  • Severity: CRITICAL
  • Evidence class: CURRENT_CROSS_FILE / SOURCE_PROVEN_DEFECT
  • Current source fact:
  • contract declares ui:field.writes = local-db-fixtures-sessions-and-artifacts;
  • field_process_ui_smoke.js creates users, equipment, loads, transloads, planner, queue, delivery and driver-log fixtures, then performs UI/API mutations;
  • dispatcher gates ui:field only on --run-e2e;
  • --no-db-write does not prevent the command from being launched;
  • AGENTS requires DB writes only with --allow-db-write.
  • Additional cleanup defect:
  • cleanup exceptions are caught and logged in finally but are not rethrown;
  • the process can report the E2E steps as passed even when cleanup leaves data behind.
  • Consequence:
  • authorization state and actual effects diverge;
  • a successful test result does not prove fixture cleanup succeeded.
  • Minimum correction candidate:
  • require both --run-e2e and --allow-db-write for current ui:field, or split read-only browser checks from mutating field E2E;
  • make cleanup failure fail the check;
  • record created/cleaned fixture identity without exposing sensitive data.
  • Local closure: L-06, with disposable local DB authorization only.

H-003 — TypeScript, SQL And Operational Configuration Have Source-Coverage Holes

  • Severity: HIGH
  • Evidence class: SOURCE_PROVEN_DEFECT
  • Current source fact:
  • **/*.ts and **/*.tsx route to node:check;
  • node:check only inspects js|mjs|cjs, so a TS-only change becomes NOT_APPLICABLE;
  • the repository has a real strict TypeScript package under voice-gateway with tsc build and test scripts;
  • scripts/migrations/** and **/*.sql route to PHP/Node checks that do not inspect SQL;
  • generic JSON, tsconfig.json, lockfiles, systemd .service/.timer, Apache/server config and many YAML files fall back to git:diff only.
  • Consequence:
  • syntax/type-invalid TS and malformed migration/config candidates can pass the automatic source gate.
  • Minimum correction candidate:
  • route scoped TS package changes to the package's existing compiler/test command;
  • add bounded SQL/migration static checks and forward/rollback pairing metadata where applicable;
  • parse relevant JSON and validate only known operational config surfaces with their native tool when available;
  • avoid a global dependency install or monolithic build.
  • Local closure: L-03, L-04, L-05.

H-004 — git:diff Does Not Check The Committed Candidate Range It Claims To Cover

  • Severity: HIGH
  • Evidence class: SOURCE_PROVEN_DEFECT
  • Current source fact:
  • changed-file selection can include merge-base(base, HEAD)..HEAD;
  • the git:diff implementation runs only git diff HEAD --check;
  • on a clean worktree that command checks no committed candidate bytes.
  • Consequence:
  • committed trailing whitespace or conflict-marker-like diff defects can receive PASS git:diff in a range verification.
  • Minimum correction candidate:
  • run git diff --check <merge-base>..HEAD when a base exists;
  • additionally check working-tree changes;
  • report the resolved merge-base and HEAD.
  • Local closure: L-02.

H-005 — Declared Environment Allowlist Is Bypassed Or Does Not Match Actual Inputs

  • Severity: HIGH
  • Evidence class: CURRENT_CROSS_FILE
  • Current source fact:
  • the generic runner creates an allowlisted environment;
  • scripts/api-smoke.js then reads .env itself and imports every key not already present;
  • scripts/field_process_ui_smoke.js parses .env directly for DB access;
  • API smoke actually uses HOST, API_SMOKE_PORT, PORT, SMOKE_USERNAME and upload-size variables, while contract metadata lists a different/incomplete set including unused CTC_API_BASE_URL;
  • app smoke actually depends on HOST, PORT, SMOKE_USERNAME, DEV_ENV_PATH and a password entry in private DEV_ENV.md, while those inputs are not faithfully represented in its allowlist;
  • UI smoke declares CTC test credential keys but seeds sessions/users directly and reads DB configuration from .env.
  • Consequence:
  • the contract cannot be used to audit which environment/credential inputs a check consumes;
  • an unallowlisted secret can still become visible to the child script process.
  • Minimum correction candidate:
  • stop target scripts from importing unrestricted .env keys;
  • pass only the check-specific inputs from the runner;
  • make private credential dependencies explicit without committing secret values;
  • align names actually consumed by each script.
  • Local closure: L-07 using a non-sensitive sentinel key only.

H-006 — writes And network Are Descriptive Metadata, Not Enforced Policy

  • Severity: HIGH
  • Evidence class: SOURCE_PROVEN_DEFECT
  • Current source fact:
  • contract validation requires only that writes be a non-empty string and network be boolean;
  • run-contract-check.mjs does not inspect either field;
  • verify-change.sh hardcodes authorization behavior by runner/check ID;
  • arbitrary new writes strings and runner strings pass contract validation if structurally present.
  • Consequence:
  • contract declarations can drift from actual authorization logic;
  • H-002 can recur for any new check.
  • Minimum correction candidate:
  • define a small closed effect vocabulary;
  • validate runner/effect compatibility;
  • derive required flags from effect metadata or assert the hardcoded mapping in self-tests;
  • define network:false as either no external network or no sockets, because current local HTTP checks use loopback networking.
  • Local closure: structural/adversarial contract tests in L-09.

H-007 — Evidence Renderer Can Produce VERIFIED_CHANGE From Contradictory Inputs

  • Severity: HIGH
  • Evidence class: SOURCE_PROVEN_DEFECT
  • Current source fact:
  • renderer checks only check-status enum, result-status enum and write-array presence;
  • it does not load the committed schemas;
  • it does not require known check IDs, PASS acceptance, source SHA, command, candidate match, or review/result consistency;
  • sourceSha defaults to UNKNOWN;
  • no source bridge was found from verify-change.sh --format=json to renderer inputs.
  • Consequence:
  • a hand-authored fixture can contain a failed check and still render a VERIFIED_CHANGE report successfully;
  • evidence format can appear authoritative without proving execution provenance.
  • Minimum correction candidate:
  • validate every input against the existing schemas;
  • cross-check result status against check states and acceptance;
  • require exact source identity for VERIFIED_CHANGE;
  • add a direct, bounded verify-result-to-evidence conversion path;
  • retain runtime evidence outside the repository.
  • Local closure: L-08.

H-008 — Verification Output Does Not Preserve Exact Candidate Or Command Provenance

  • Severity: HIGH
  • Evidence class: SOURCE_PROVEN_DEFECT
  • Current source fact:
  • JSON output records the user-supplied base string, selected IDs and aggregate results;
  • it does not record resolved HEAD, merge-base, changed-file list/digest, dirty byte identity, command argv, cwd or output artifact;
  • preflight records current HEAD but has no base-range mode and is not linked to the verification result.
  • Consequence:
  • after branch movement, commit, merge or local edits, a result cannot independently prove which exact candidate bytes it verified.
  • Minimum correction candidate:
  • include branch, HEAD, resolved merge-base, dirty/untracked list or digest, contract hash and executed argv/cwd in structured output;
  • invalidate/re-run affected checks after candidate bytes change.
  • Local closure: include exact provenance in every Local run artifact.

H-009 — Harness Changes Do Not Automatically Run Harness Behavioral Self-Tests

  • Severity: HIGH
  • Evidence class: SOURCE_PROVEN_DEFECT
  • Current source fact:
  • the harness surface rule selects contract/docs/shell/node source checks;
  • preflight and evidence self-tests are included only by forced --type=harness;
  • planner self-test covers only a few happy-path route cases.
  • Consequence:
  • a normal change to planner/dispatcher/runner/renderer can pass syntax and structural contract validation without testing behavior.
  • Minimum correction candidate:
  • auto-select bounded Harness self-tests for Harness-owned source changes;
  • add adversarial cases for deletion, base range, deep-effect gating, contradictory evidence and unknown runner/effect values.
  • Local closure: L-09.

H-010 — all And incident Type Aliases Are Not Completable By The Generic Runner

  • Severity: MEDIUM
  • Evidence class: SOURCE_PROVEN_DEFECT
  • Current source fact:
  • all includes incident:production-check;
  • incident runner is manual and dispatcher always records NOT_RUN;
  • an explicit type with any NOT_RUN exits non-zero;
  • there is no external-evidence input that can satisfy the manual check.
  • Consequence:
  • --type=all and --type=incident are selection aliases, not executable completion commands, but this is not obvious from the contract.
  • Minimum correction candidate:
  • rename them as incomplete/manual plans, remove them from executable aliases, or support explicit authorized external evidence without auto-running Production writes.
  • Local closure: L-10.

H-011 — Timeout Can Bypass Child Cleanup

  • Severity: MEDIUM
  • Evidence class: LOCAL_REPRO_REQUIRED
  • Current source fact:
  • outer runner applies timeout to the direct child process;
  • API/UI checks start PHP/browser child processes and rely on finally/trap cleanup;
  • forced termination can prevent those handlers from running.
  • Risk:
  • orphan server/browser processes, sessions, artifacts or DB fixtures can remain after BLOCKED timeout.
  • Minimum correction candidate:
  • process-group termination plus post-timeout cleanup/verification;
  • DB fixture cleanup must be independently verifiable.
  • Local closure: L-11.

H-012 — php:integration Effect Classification Is Too Vague For Generic Dispatch

  • Severity: MEDIUM
  • Evidence class: POLICY_AMBIGUITY / LOCAL_REPRO_REQUIRED
  • Current source fact:
  • contract declares task-defined-local-effects;
  • generic argv dispatch runs php:integration without the DB-write gate;
  • the suite runner executes every flat tests/integration/*_test.php file.
  • Unknown:
  • current test-by-test filesystem, session, DB and provider effects were not enumerated in this Cloud pass.
  • Minimum correction candidate:
  • classify the actual integration files;
  • split read-only integration from effectful integration if needed;
  • replace the open-ended effect string with a closed category.
  • Local closure: L-12.

H-013 — Shell Validation Uses Bash Regardless Of Declared Interpreter

  • Severity: MEDIUM
  • Evidence class: SOURCE_PROVEN_GAP
  • Current source fact:
  • every changed .sh file is checked with bash -n;
  • repository scripts include both Bash and /usr/bin/env sh entrypoints.
  • Consequence:
  • Bash syntax checking does not prove POSIX-shell compatibility for sh scripts.
  • Minimum correction candidate:
  • inspect the shebang and use the declared interpreter's syntax mode;
  • retain Bash for Bash scripts.
  • Local closure: native interpreter availability check.

H-014 — Local Loopback Networking Is Not Distinguished From External Networking

  • Severity: LOW / POLICY AMBIGUITY
  • Evidence class: POLICY_AMBIGUITY
  • Current source fact:
  • API/app/UI checks declare network:false;
  • they open or call loopback HTTP endpoints.
  • Consequence:
  • readers cannot tell whether network:false means no external network, no remote network, or no socket use.
  • Minimum correction candidate:
  • use an explicit category such as none, loopback-only, external-read, authorized-external-write.

7. Change-Type Verification Reality Matrix

Change type Current automatic behavior Current gap Minimum target for P1-009
Markdown/docs git:diff + changed document structure/link/transition checks deletion excluded; generated/manual consumer only conditionally covered include deletion; retain bounded changed-doc validation
PHP source changed PHP lint; mapped focused regression on narrow files broad unit/integration mapping is manual/type-driven preserve lint; add only risk-mapped tests
PHP tests unit/focused when path maps deletion/skip/assertion weakening not detected bounded test-integrity diff check
JavaScript node --check for changed JS/MJS/CJS no package build/unit mapping except explicit type/manual command keep syntax; route package-specific tests where owner exists
TypeScript/TSX surface selects node:check, then check is N/A no tsc, package build or test scoped package compiler/test
CSS/UI source diff plus PHP/Node checks where relevant CSS content itself not parsed; no behavior evidence by default source-only default plus explicit viewport/browser acceptance when risk requires
JSON/package package.json JSON parse only generic JSON, tsconfig and lockfile semantics omitted parse known JSON; package-specific lock/build consistency
SQL/migration diff plus inapplicable PHP/Node checks no SQL syntax/pairing/DB compatibility bounded static contract plus disposable DB execution when required
Shell bash -n wrong interpreter for sh scripts shebang-aware syntax check
API source lint/syntax; authenticated smoke only with --run-smoke hidden env/credential inputs; smoke coverage narrow explicit inputs and scoped smoke contract
Local app smoke only with --run-smoke hidden DEV_ENV dependency and loopback metadata ambiguity explicit prerequisite/effect report
DB integration/CRUD explicit --allow-db-write integration effect classification incomplete preserve explicit gate and rollback evidence
Field UI E2E explicit --run-e2e DB-write flag not required; cleanup can fail silently require both effect permissions; cleanup failure is FAIL
Harness source structural contract + source syntax bounded self-tests omitted automatically auto adversarial Harness self-tests
Evidence separate manual renderer no execution provenance/cross-validation schema + result consistency + exact candidate bridge
Deployment/service config usually git:diff fallback no native config validation or Runtime SHA linkage native static check when available; deployment remains separately authorized
Production incident manual NOT_RUN no completion/evidence ingestion path keep manual/authorized; do not imply generic automation

8. False-Pass And False-Complete Scenarios To Prevent

The following statements are not valid completion evidence under current source.

  1. scripts/verify-change.sh returned 0 after a test-only deletion.
  2. git:diff passed for a clean committed candidate range.
  3. node:check was selected for a .ts file.
  4. a migration .sql file received PHP/Node NOT_APPLICABLE and overall source verification passed.
  5. --run-e2e --no-db-write was treated as non-mutating.
  6. UI steps passed while cleanup logged an error.
  7. the environment allowlist was assumed to constrain a script that directly loads .env.
  8. evidence.md says VERIFIED_CHANGE without a known source SHA and consistent successful checks.
  9. a Harness source change passed without running its behavioral self-tests.
  10. --type=all was described as a fully automated gate.
  11. a timed-out browser/API check was assumed to leave no process, session, artifact or fixture.

9. Local Execution Reality-Check Contract

9.1 Safety Boundary

  • use a disposable local clone or disposable worktree;
  • do not push experimental commits;
  • do not run against Production or a non-local DB host;
  • do not use real provider credentials or perform external provider writes;
  • record exact starting qa SHA, resolved HEAD, merge-base and contract hash;
  • preserve unrelated local work;
  • DB/browser cases require an explicitly approved disposable local DB, snapshot/restore boundary and before/after residue query;
  • missing tool, credential or environment is BLOCKED, not PASS;
  • do not install a language/tool merely because an obsolete check expects it.

9.2 Required Source-Only Reproductions

ID Disposable experiment Expected current observation from source Closure evidence
L-01 delete one tracked tests/**/*_test.php file and run automatic JSON verification deleted path absent; likely auto-selection NOT_APPLICABLE/exit 0 if deletion is the only change command, JSON, exit code, git status, changed-file proof
L-02 commit trailing whitespace in an otherwise unclassified source/config file, clean tree, run --base=HEAD^ candidate path selected but git:diff checks only working tree and can PASS merge-base, HEAD, diff, JSON, exit code
L-03 introduce a TypeScript compile error in voice-gateway/src/*.ts; compare automatic verification with npm --prefix voice-gateway test auto Node check N/A while scoped package command fails both command outputs and exit codes
L-04 introduce malformed SQL in a disposable migration file automatic route has no SQL validator plan/result JSON and native/disposable parser result if available
L-05 alter tsconfig.json, a scoped lockfile, .service or .timer file with a native-invalid value generic automatic coverage is diff-only/fallback per-file plan and available native validator result
L-08 build renderer input with a FAIL check plus result.status=VERIFIED_CHANGE renderer is expected to exit 0 and render contradictory evidence fixture, exit code, rendered result
L-09 make a disposable planner/dispatcher/renderer source change and run automatic verification Harness behavioral self-tests are not selected automatically selected check list; then forced --type=harness comparison
L-10 run --type=all --format=json and --type=incident --format=json manual incident remains NOT_RUN; explicit type exits non-zero JSON and exit code
L-13 add a POSIX-incompatible Bash construct to a disposable #!/usr/bin/env sh script bash -n can accept syntax not valid for the declared shell Bash result and declared-shell result

9.3 Environment And Effect Reproductions

ID Experiment Required guard Closure evidence
L-06 run the UI selection path with --type=ui --run-e2e --no-db-write; first use unavailable/disposable DB to prove dispatch, then only with explicit local-write approval run full E2E local-only DB identity, snapshot, no provider writes selection log, DB identity, created/cleaned fixture counts, exit code
L-07 add a non-sensitive sentinel key to disposable .env and instrument only whether the API/UI process can see it; never print values sentinel must contain no secret presence boolean, command, source diff, cleanup
L-11 force a short timeout during a check that starts a local server/child process disposable ports/process namespace; no real DB writes unless separately approved process list/ports before and after, artifact/session residue, exit state
L-12 enumerate and run current tests/integration/*_test.php under tracing/isolated local environment no Production DB/provider; effects logged per-test read/write/network classification and result

9.4 Local Completion Conditions

P1-004 can move to COMPLETE only when:

  • all source-only reproductions are run against a recorded current revision;
  • H-001 through H-010 are confirmed, corrected by newer source, or explicitly disproved with evidence;
  • effectful checks are either safely reproduced or remain BLOCKED with the exact missing prerequisite;
  • timeout/process/session/artifact/DB residue is checked, not assumed;
  • the final current change-type matrix reflects observed results;
  • no Harness correction is mixed into the P1-004 evidence commit;
  • P1-009 receives a ranked correction handoff and one-writer boundary.

10. P1-009 Bounded Correction Order

This is a handoff, not implementation approval.

  1. Candidate discovery integrity
  2. include deleted paths;
  3. check committed range bytes;
  4. preserve exact candidate identity.
  5. Test-integrity minimum
  6. detect test deletion and bounded skip/only/assertion weakening signals;
  7. avoid rebuilding the former large framework.
  8. Effect authorization
  9. make writes/network category enforceable;
  10. require DB authorization for mutating UI E2E;
  11. make cleanup failure fatal.
  12. Missing source validators
  13. scoped TypeScript compiler/test;
  14. bounded SQL/migration contract;
  15. known JSON/lock/config/native validators;
  16. shebang-aware shell syntax.
  17. Environment contract alignment
  18. remove unrestricted child .env loading;
  19. document actual private prerequisites without secret values.
  20. Evidence provenance
  21. schemas, cross-result rules, exact revision, argv/cwd/artifact;
  22. bounded verify-to-evidence bridge.
  23. Harness self-tests
  24. automatic adversarial tests on Harness source changes.
  25. Alias and terminology cleanup
  26. clarify all, incident, release, network and effect categories.

P1-009 must not:

  • turn Harness into an Objective/Phase/Task manager;
  • force every test for every change;
  • install unrelated tools globally;
  • make local DB/browser/Production checks implicit;
  • restore GitHub Actions as quality authority;
  • treat missing prerequisites as PASS;
  • perform Production/provider/deployment writes.

11. Deliverables

  • this current Harness source inventory;
  • corrected-versus-remaining behavior separation;
  • findings H-001 through H-014;
  • current change-type verification reality matrix;
  • exact Local reproduction contract;
  • bounded P1-009 correction order.

12. Acceptance Criteria

  • [x] Current qa, not stale master, is used as the Harness source basis.
  • [x] Contract, planner, dispatcher, isolated runner, preflight and evidence paths are mapped.
  • [x] Actual deep-check targets are compared with their declarations.
  • [x] Previously corrected behavior is separated from remaining defects.
  • [x] Broken/obsolete restriction, false-positive, false-negative and missing-validation classes are recorded.
  • [x] Change-type minimum verification gaps are explicit.
  • [x] Cloud-confirmed fact and Local-required evidence are separated.
  • [x] P1-009 handoff is bounded and does not create a new framework.
  • [ ] Disposable Local execution reproductions are complete.
  • [ ] Local DB/browser/process residue evidence is complete or honestly blocked.
  • [ ] Final observed matrix is attached.

13. Verification

Check Result Notes
Current qa head identification PASS Reviewed from qa@fd9ea998e50581144a9d9ead01b5182e046f4a79.
Current Harness source read PASS Contract and executable scripts fetched from qa; exact blobs recorded in this document.
Contract/planner/dispatcher/target cross-check PASS Findings are based on explicit control-flow and target-script comparison.
Historical-master/current-qa separation PASS Already corrected behavior is listed separately and not reported as a current defect.
Source-only finding traceability PASS Every H finding records fact, consequence, correction candidate and Local closure.
node scripts/docs/check-all.mjs --paths=docs/tasks/P1-004.md NOT_RUN Connector-only environment has no checkout; no false PASS is claimed.
scripts/verify-change.sh --type=harness --base=<review-base> NOT_RUN Requires a local Git worktree and executable toolchain.
Disposable deletion/range/TS/SQL/config reproductions NOT_RUN Local handoff L-01 through L-05/L-08 through L-10/L-13.
Local DB/browser/UI cleanup verification NOT_RUN Requires explicit disposable local DB authorization and Playwright environment.
Timeout/orphan-process verification NOT_RUN Requires Local process/port observation.
Production/provider/deployment verification NOT_APPLICABLE This Task is source research; no external effect is authorized.
GitHub read-after-write REQUIRED Must verify the exact committed file after publication.

14. Runtime / DB / Provider / File / Deployment Impact

  • Repository file added: docs/tasks/P1-004.md
  • Harness source changed: none
  • Runtime changed: none
  • DB changed: none
  • Provider/service changed: none
  • Production/deployment changed: none
  • master changed: none
  • GitHub branch target: qa only

15. Stop Conditions

Stop and do not claim P1-004 completion if:

  • Local commands were not run but source hypotheses are presented as observed Runtime results;
  • a test uses Production/non-local DB or real provider credentials;
  • UI cleanup residue is not checked;
  • the qa Harness source changes and this inventory is not reconciled to the new exact revision;
  • P1-009 code correction is mixed into this source-research evidence without separate scope/review;
  • missing tools or credentials are converted to PASS;
  • a force update, direct master commit, deployment or unrelated-file overwrite would be required.

16. Rollback / Fail-Forward

Source rollback is a revert of the single P1-004 documentation commit on qa.

No Runtime, DB, provider, service or deployment rollback is required because this Task changes documentation only.

If Harness source changes before Local closure:

  1. record the new qa SHA and changed Harness blobs;
  2. re-evaluate only affected findings and Local cases;
  3. preserve superseded observations as dated evidence, not current truth;
  4. update this Task before allowing P1-009 to consume it.

17. Evidence And Handoff

Confirmed current conclusions

  • the current Harness is materially smaller and safer than the prior over-restrictive version;
  • source-only default, explicit deep-check flags and isolated inherited environment are valid improvements;
  • deletion handling, committed-range content checking and test-integrity protection remain critical gaps;
  • mutating field E2E is not bound to the DB-write authorization flag and cleanup failure is not fatal;
  • TypeScript, SQL and operational configuration are not adequately validated by current automatic routing;
  • effect/network metadata is descriptive rather than executable policy;
  • the evidence renderer is not a provenance-verifying gate;
  • exact candidate/command identity is not preserved in the main structured result;
  • Harness source changes do not automatically receive behavioral self-tests.

P1-009 handoff

P1-009 should consume:

  • H-001 through H-014;
  • the current-versus-minimum matrix;
  • Local reproduction outputs;
  • correction order in Section 10;
  • the rule that existing useful minimal behavior must be preserved.

P1-009 should not start implementation until P1-004 Local closure identifies which source-derived hypotheses reproduce on the current toolchain and which have already been superseded.

Engineering / operation states

Dimension State
Cloud source review COMPLETE
Local execution reality check REQUIRED / NOT RUN
Engineering Complete NO — full Task includes Local evidence
Released NO — documentation commit is not a release
User Accepted PENDING USER REVIEW
Stabilized NOT APPLICABLE
Value Verified PENDING P1-009 USE

18. Progress Log

Date Event
2026-07-30 Confirmed P1-004 scope, lane, dependency and independent-output boundary from the reset source.
2026-07-30 Reconciled master and current qa; switched the source basis to the latest qa Harness.
2026-07-30 Reviewed contract, planner, dispatcher, isolated runner, preflight, evidence renderer and schemas.
2026-07-30 Compared API/app/DB/UI target scripts with declared environment/effect metadata.
2026-07-30 Recorded current corrections that must be preserved.
2026-07-30 Registered H-001 through H-014 and built the change-type reality matrix.
2026-07-30 Defined Local disposable reproduction and completion contracts.
2026-07-30 Prepared bounded P1-009 handoff; no Harness implementation, Runtime, DB or deployment change performed.
---

Source: docs/research/platform-foundation/2026-07-30/raw/harness-and-verification-reality-check.md — ctc repository · This page is an English translation of the source document. Identifiers, file paths and code symbols are unchanged. A user instruction quoted inside the text is kept in its original language.