aboutsummaryrefslogtreecommitdiff
path: root/test-suite/notes/browsergym-compat.md
diff options
context:
space:
mode:
authorTianren Dong <tianrendong.dtr@gmail.com>2026-05-13 07:29:41 -0400
committerTianren Dong <tianrendong.dtr@gmail.com>2026-05-13 07:29:41 -0400
commit4c54b05dc15b81e449dc8ac53da8bae536fa2798 (patch)
treef81c5e627dcc4dd81d2fe2fd3b2b584e72a73eca /test-suite/notes/browsergym-compat.md
parent5951b3fb04aaf53b875afe265a57a944d6fce641 (diff)
pi-chrome: snapshot pre-migration state
Diffstat (limited to 'test-suite/notes/browsergym-compat.md')
-rw-r--r--test-suite/notes/browsergym-compat.md70
1 files changed, 70 insertions, 0 deletions
diff --git a/test-suite/notes/browsergym-compat.md b/test-suite/notes/browsergym-compat.md
new file mode 100644
index 0000000..1eaea41
--- /dev/null
+++ b/test-suite/notes/browsergym-compat.md
@@ -0,0 +1,70 @@
+# BrowserGym compatibility contract
+
+Pi-chrome benchmark tasks mirror BrowserGym without importing Gymnasium.
+
+## Task fields
+
+Each `task-manifest.json` entry maps to `AbstractBrowserTask` concepts:
+
+- `taskId` / `id` — stable namespaced id
+- `seed` — deterministic state seed
+- `viewport`, `slowMoMs`, `timeoutMs`, `locale`, `timezoneId`
+- `goal` and `goalObject` — text and OpenAI-style message objects
+- `setupUrl` — page URL that performs setup from `task`, `run`, `seed`
+- `validateScript` — JS validate hook returning reward contract
+- `cheatScript` — gold recipe used to sanity-check grader
+- `maxSteps`, `humanBaseline`, `difficulty`, `tags`
+- `actionSubsets`, `allowedActions` — BrowserGym HighLevelActionSet metadata
+
+## Reset / step return
+
+Runner should expose:
+
+```text
+reset() -> (obs, info)
+step(action) -> (obs, reward, terminated, truncated, info)
+```
+
+`terminated` means task done. `truncated` means max step or timeout.
+
+## Validate return
+
+In-page validators return:
+
+```js
+{ reward, done, message, info }
+```
+
+`reward` is incremental. Current hermetic tasks use binary terminal reward. `info.rubric` carries per-check details.
+
+## Observation schema target
+
+Runner observations should include BrowserGym-like keys:
+
+- `chat_messages`
+- `goal`, `goal_object`
+- `open_pages_urls`, `open_pages_titles`, `active_page_index`
+- `url`, `screenshot`, `dom_object`, `axtree_object`
+- `extra_element_properties`
+- `focused_element_bid`
+- `last_action`, `last_action_error`
+- `elapsed_time`
+
+## BID plan
+
+BrowserGym uses `bid` attributes as stable element handles. Pi-chrome currently returns snapshot `uid`. Compatibility path:
+
+1. During snapshot, assign every interactive element a deterministic `bid`.
+2. Return both `uid` and `bid`.
+3. Add `visibility`, `bbox`, `clickable`, `set_of_marks` metadata.
+4. Maintain uid↔bid mapping so agents can use either addressing mode.
+
+Target attributes:
+
+- `bid`
+- `browsergym_visibility_ratio`
+- `browsergym_set_of_marks`
+
+## Action subsets
+
+See `../browsergym-action-space.json` for BrowserGym-compatible subsets and function signatures.