From b792f22f2c2ef4dfb7bd300e7381a186e34e0c18 Mon Sep 17 00:00:00 2001 From: Tianren Dong Date: Sat, 16 May 2026 14:01:00 -0400 Subject: 0.15.25: fix reload after stale singleton --- test-suite/notes/runner-spec.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) (limited to 'test-suite/notes/runner-spec.md') diff --git a/test-suite/notes/runner-spec.md b/test-suite/notes/runner-spec.md index b040cb4..24c97ab 100644 --- a/test-suite/notes/runner-spec.md +++ b/test-suite/notes/runner-spec.md @@ -3,13 +3,14 @@ Future Node/Pi runner should: 1. Read `manifest.json`. -2. Probe environment capabilities (`trusted`, `clipboard`, `touch`, `dialogs`, `downloads`, `fileSystem`). +2. Probe environment capabilities (`trusted`, `clipboard`, `touch`, `dialogs`, `downloads`, `fileSystem`, `strictCspFallback`). 3. For each entry: + - read `gate` (`core`, `conditional`, `quality`) and score in that bucket - skip if `requires` is unsatisfied and expected is `CONDITIONAL` - navigate to challenge URL - execute `recipe` in order - wait `verdictDelayMs || 500` - - evaluate `JSON.stringify({v:window.__verdict,r:window.__reason,d:window.Challenge?.state?.details,e:window.__events?.slice(-20)})` + - evaluate `JSON.stringify({v:window.__verdict,r:window.__reason,d:window.Challenge?.state?.details,e:window.__events?.slice(-20)})`, except strict-CSP pages where eval is expected to fail; read persisted verdict from dashboard/localStorage after leaving the CSP page - compare to `expected[mode]` 4. Output: - Markdown summary @@ -21,6 +22,8 @@ Runner must adapt recipe intent: - expand `${REPO_ROOT}` path placeholders - adapt unsupported shadow/iframe selector notation to snapshot uids or evaluate fallback - preserve hook install ordering for console/network capture tests +- substitute dynamic tab ids from `chrome_tab list` recipes +- support screenshot/coordinate fallback for strict CSP pages where snapshot/evaluate are blocked - record whether trusted/CDP path was used Long-horizon task runner should read `task-manifest.json`, replace `$RUN_ID`, solve task, then evaluate task grader expression. -- cgit v1.2.3