Done

Embedded browser: support canvas-rendered web apps (Compose Multiplatform / Flutter Web)

From Holm RichterDone 3 weeks ago

Our app is Compose Multiplatform on Kotlin/Wasm. It renders into a canvas inside the open shadow root of #ComposeTarget, with an accessibility overlay whose elements carry our test ids. Two things block browser_click / browser_type today:

  1. Selectors do not pierce open shadow roots. document.querySelector('#login_email') returns null, while the element exists in #ComposeTarget.shadowRoot. Please resolve selectors through open shadow roots (as Playwright does), or accept a >>> / pierce/ syntax.

  2. Input events are synthetic (untrusted). Compose/Skia ignores them, so typing into the login form has no effect. Please offer trusted input through CDP (Input.dispatchKeyEvent, Input.insertText, Input.dispatchMouseEvent) or Electron's sendInputEvent, plus a click at x/y coordinates and a force option for overlays with pointer-events: none.

With both, an agent could log in and verify our screens in the embedded browser. Today only Playwright from the shell can (its selector engine pierces open shadow roots and it dispatches trusted events over CDP), so QA team nodes with "Verify in browser" cannot get past the login screen of such apps.

AgentsRoom diagnostics

  • App version: 1.194.1
  • OS: win32 10.0.26200 (x64)
  • Agent CLI: claude
  • App language: system

Problem to solve

The embedded browser cannot operate canvas-rendered web apps (Compose Multiplatform Wasm, Flutter Web): selectors do not reach elements inside the open shadow root, and synthetic input events are ignored by the canvas input layer. QA agents and team nodes with Verify in browser cannot log in or navigate.

Proposed behavior

Pierce open shadow roots when resolving selectors (or support >>> / pierce/), and dispatch trusted input via CDP or Electron sendInputEvent, including click at x/y coordinates and a force option for overlays with pointer-events: none.

Who benefits

Every team building web UIs with Compose Multiplatform for Web or Flutter Web that wants agents to test in the embedded browser.

Acceptance criteria

An agent can log in and walk a Compose/Flutter web app in the embedded browser, without falling back to hand-written Playwright scripts.

Replies

The team3 weeks ago

Thanks: the bug is fixed and the fix ships in the next release. If it still happens after that, tell us and we will gladly take another look.

Up to 3 images, or paste one straight from your clipboard. Big screenshots are shrunk in your browser before they are sent.