mirror of
https://github.com/NeuralNomadsAI/CodeNomad.git
synced 2026-11-13 17:42:57 +00:00
feat(browser): activate native WebKit agent automation
Expose native driver and device-profile capabilities on macOS and Linux and route the public browser tool through the same authenticated session ownership and parent-channel fences as Windows. Remove host-name refusals while retaining genuine unavailable-host and typed unsupported-operation errors. Integrate current capability documentation in ten languages and preserve native popup, download, permission, dialog and lifetime policies. Add packaged-application qualification using a private OpenCode daemon and deterministic provider issuing actual agent browser tool calls, without touching shared service state.
This commit is contained in:
Pascal André
parent
e0d56d428a
commit
3c68ab5668
43 changed files with 272 additions and 98 deletions
|
|
@ -71,7 +71,7 @@ Electron maps the channel's `Input.insertText` to `WebContents.insertText` and `
|
|||
|
||||
`begin` binds one expiring operation lease to the exact visible registered guest and document generation. Every `send` is bounded and allowlisted to semantic operations; every `check` revalidates visibility, registration and document identity. Responses are fenced again after native I/O. A changed, hidden, ambiguous or missing target fails rather than switching guests. The backend owns refs per session and generation; it issues no refs from an unconfirmed read and never retries a failed mutation. `end` cleans up that exact lease even after cancellation or navigation, without detaching someone else's emulation overrides. A host without channel support fails explicitly.
|
||||
|
||||
`begin` announces `driver: "cdp" | "webkit"` and `contractVersion: 1`. `send` accepts one closed `operation` union: tree, resolve, call, scroll-into-view, geometry, viewport, hit-test, pointer, wheel, keyboard, insert-text, release-handles and checkpoint. Both native hosts reject unknown operation fields, function IDs and argument shapes. Calls carry a registered function ID, opaque target handle and JSON arguments. The hosts obtain executable programs from their own shared fixed registry; the private wire carries operations rather than protocol methods or program text. WebKit implements these operations through its isolated program and target-local native input adapters. Public WebKit automation remains disabled.
|
||||
`begin` announces `driver: "cdp" | "webkit"` and `contractVersion: 1`. `send` accepts one closed `operation` union: tree, resolve, call, scroll-into-view, geometry, viewport, hit-test, pointer, wheel, keyboard, insert-text, release-handles and checkpoint. Both native hosts reject unknown operation fields, function IDs and argument shapes. Calls carry a registered function ID, opaque target handle and JSON arguments. The hosts obtain executable programs from their own shared fixed registry; the private wire carries operations rather than protocol methods or program text. WebKit implements these operations through its isolated program and target-local native input adapters.
|
||||
|
||||
WebKit's fixed isolated-world reader bundles `dom-accessibility-api` 0.7.1 (MIT, notice included in the program and `webkit/LICENSE.dom-accessibility-api`). It computes names, descriptions and roles and adapts native field states to the shared renderer. Password values use the shared read-value policy's maximum sixteen masking bullets. Hidden, inert and ARIA-hidden branches are excluded. Open and captured closed shadow roots and same-origin frames are traversed; cross-origin frames receive an opaque, non-actionable boundary marker. Chromium's main-frame `Accessibility.getFullAXTree` exposes closed-root controls and frame elements, with frame contents available through separate frame-tree requests.
|
||||
|
||||
|
|
@ -95,19 +95,32 @@ Native/HTTP deadlines bound browser tool execution; the tool context's `signal`
|
|||
|
||||
## Native Hosts
|
||||
|
||||
| Desktop host | Native preview/history | Agent browser automation | Device profiles | Pop-ups / downloads / dialogs |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| Electron | Chromium | Supported | Chromium mobile | Supported |
|
||||
| Tauri Windows | WebView2 | Supported | Chromium mobile | Supported |
|
||||
| Tauri macOS 14+ | WKWebView | Supported | Native WebKit mobile | Supported |
|
||||
| Tauri Linux | WebKitGTK | Supported | Native WebKit mobile | Supported |
|
||||
|
||||
Browser automation uses the visible owned session target on every native desktop
|
||||
host. Driver/profile availability follows the compiled native adapter and the
|
||||
host-published capability contract. Unsupported hosts and unknown semantic
|
||||
operations fail explicitly. Engine-specific input, shadow-root, clipboard and
|
||||
mobile-profile limits are specified below and in `BROWSER_DEVICE_EMULATION.md`.
|
||||
|
||||
Electron uses hardened `<webview>` tabs sharing one persistent browser partition (`persist:codenomad-browser-shared-v1`) per data profile/config pair. Attachment verifies guest ownership and accepts credential-free HTTP(S) entry URLs. Owned pop-ups retain the preview partition and native opener; site documents additionally support `about:blank` and same-origin blob URLs. Camera, microphone, geolocation and notifications are denied. Other classified permissions use origin-bound consent or an explicit device/source chooser; unknown requests are denied with an error. Page operations use the guarded channel above.
|
||||
|
||||
Tauri uses a child webview owned by the local application window: WebView2 on Windows, WKWebView on macOS 14+, and WebKitGTK on Linux. Pages load directly in the native view. `embedded_view.rs` owns native creation, geometry, visibility, closure, shared storage and mandatory credential-free HTTP(S) navigation, permission, download and new-window protections; its Windows adapter also installs native accelerators. `browser_controller.rs` owns session registrations, renderer generations, history, events, emulation and automation leases. Application capabilities select primary `local-*` webviews, never their parent-window wildcard: Tauri combines window/webview matching with OR, so a window grant would also authorize an untrusted child, including localhost content. Full primary-renderer navigation disposes its preview children. Final-window decisions count native windows, including multi-webview windows.
|
||||
|
||||
Native views follow the preview bounds and hide beneath local menus, lists and dialogs. Reshow applies the latest bounds before revealing the retained document. Windows/GTK HTML fullscreen fills its owning window, preserves normal guest bounds and the previous window mode, and restores both on exit/hide/close; WKWebView uses native fullscreen. WebKit Back/Forward uses each view's native back-forward list, with registration-owned entry identities and a 32-entry journal; restored sessions load the selected saved URL on explicit opening. WebKit presets apply fitted native CSS viewports and per-view mobile identity, with density/input differences specified in `BROWSER_DEVICE_EMULATION.md`. Element comments are unavailable in native previews.
|
||||
|
||||
Local Tauri primary renderers receive `__CODENOMAD_BROWSER_CAPABILITIES__` during initialization, with independent `embedding`, `history`, `emulation` and `automation` flags. WebKit publishes embedding/history/emulation true, automation false and `reason: "webkit"`; unsupported targets publish `reason: "unsupported-platform"`. UI host selection consumes this contract; registration/history readiness stays independent of device emulation. UI registrations request hidden construction and recheck `localOverlayOpen()`, clipped bounds and display scale before the first reveal, including asynchronous construction beneath a dialog. WebKit registration/update/profile commands await native scale and configuration off the UI thread.
|
||||
Local Tauri primary renderers receive `__CODENOMAD_BROWSER_CAPABILITIES__` during initialization, with independent `embedding`, `history`, `emulation` and `automation` flags. WebKit publishes all four true and `reason: "webkit"` for its native profile hints; unsupported targets publish `reason: "unsupported-platform"`. UI host selection consumes this contract; registration/history readiness stays independent of device emulation. UI registrations request hidden construction and recheck `localOverlayOpen()`, clipped bounds and display scale before the first reveal, including asynchronous construction beneath a dialog. WebKit registration/update/profile commands await native scale and configuration off the UI thread.
|
||||
|
||||
Agent browser automation (`codenomad.browser`, including open, probe and page actions) is unavailable on Tauri macOS/Linux and returns an explicit refusal. Open these previews manually, or use Electron or Tauri Windows for agent automation. Plain-browser and remote-view clients use the sandboxed iframe proxy.
|
||||
Agent browser automation (`codenomad.browser`, including open, probe and page actions) routes through the authenticated backend and guarded native-parent channel on Electron and Tauri Windows/macOS/Linux. Plain-browser and remote-view clients use the sandboxed iframe proxy.
|
||||
|
||||
The Tauri launcher pipes backend stdin and sets `CODENOMAD_NATIVE_PARENT=1` after shell-environment resolution on Windows, macOS and Linux. The private channel fences requests and responses against the backend process generation and absolute deadline. WebKit's internal open, navigate, probe and visible-PNG operations use the common target/open-claim registry, with native navigation completion/failure callbacks and registration/document checks around capture. Native adapter code is compiled only for its target platform.
|
||||
|
||||
WebKit publishes `CODENOMAD_NATIVE_BROWSER_AUTOMATION=0` independently of the transport and retains `CODENOMAD_TAURI_WEBKIT` for host identity. The authenticated bridge returns the public WebKit refusal after session ownership validation; the UI's private open-request subscription follows embedding availability. Mission-only ownership claims retain their normal admission path. Desktop transport/capability markers are stripped from the shared daemon's startup environment.
|
||||
Supported Tauri hosts publish `CODENOMAD_NATIVE_BROWSER_AUTOMATION=1` independently of transport readiness and retain `CODENOMAD_TAURI_WEBKIT` for host identity. The authenticated bridge validates session ownership and native availability; the UI's open-request subscription follows embedding availability. Mission-only ownership claims retain their normal admission path. Desktop transport/capability markers are stripped from the shared daemon's startup environment.
|
||||
|
||||
Private navigation can recover an owned visible guest from an engine-generated error document by loading a validated HTTP(S) destination. Page reads and screenshots require a current credential-free HTTP(S) document. WebKit failure results survive the subsequent load-finished notification; fragment-only navigation completes from the confirmed native URL when document-load callbacks are absent. Capture responses retain the 16 MiB base64 limit, and a hide, close, replacement, navigation or expired backend generation fences the native callback result.
|
||||
|
||||
|
|
|
|||
|
|
@ -18,8 +18,8 @@ the desktop identity. Profile/template changes and desktop reset reload the gues
|
|||
an identical request is a no-op. The sandboxed iframe fallback changes dimensions
|
||||
only. Translated tooltips distinguish Chromium emulation, native WebKit profiles
|
||||
and dimensions-only previews. Native display, history, device emulation and agent
|
||||
automation are separate host capabilities; WebKit previews support manual browsing, while agent browser
|
||||
automation requires Electron or Tauri Windows. Never expose application
|
||||
automation are separate host capabilities; WebKit previews support manual and agent
|
||||
browsing through the native driver. Never expose application
|
||||
privileges or a general CDP bridge to preview pages. Tauri's synchronous WebView2
|
||||
callbacks are awaited off the UI thread. Electron retains its debugger attachment
|
||||
while emulation is active so the overrides survive accessibility automation.
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@ Native Vorschauen teilen Cookies, Anmeldungen und Websitespeicher über Projekte
|
|||
| Linux-Tauri | Natives WebKitGTK-Unterfenster; OAuth-Pop-ups, Downloads und Dialoge | Für sichtbare zugehörige Sitzungsvorschau unterstützt | Mobile WebKit-Identität und CSS-Viewport; native Dichte-/Eingabegrenzen |
|
||||
| Normaler Browser/Remote-Ansicht | Iframe-Ersatz | Kein lokales natives Vorschauziel | Nur Abmessungen |
|
||||
|
||||
Unter Electron und Windows-Tauri wird native Automatisierung beim Desktop-Start vorbereitet und nutzt das sitzungsgebundene Werkzeug `codenomad.browser` zum Öffnen, Navigieren, Schnappschuss, Screenshot und für Seitenaktionen: Klicken, Tippen, Auswählen, Überfahren, Ziehen, Scrollen, Tastendrücke und Warten auf Text. Werkzeugverfügbarkeit autorisiert nicht jedes Fenster: Ausführung prüft zuständiges Backend, native Fähigkeiten, Sitzung und sichtbares Ziel. Eine Aktion, deren Seite inzwischen verborgen, ersetzt oder weiternavigiert wurde, wird abgelehnt statt an eine andere Seite gesendet, und eine Aktion wird nie automatisch wiederholt. Seiteninhalte bleiben nicht vertrauenswürdig. Ein iframe kann scheitern, weil das Ziel Einbettung verbietet; nutzen Sie einen externen Browser, statt Schutzmaßnahmen zu schwächen.
|
||||
Auf allen nativen Desktop-Hosts öffnet und navigiert das sitzungsgebundene Werkzeug `codenomad.browser` die Vorschau, erstellt Schnappschüsse/Screenshots und führt Klick-, Eingabe-, Auswahl-, Hover-, Zieh-, Scroll-, Tasten- und Textwarteaktionen aus. Die Ausführung prüft zuständiges Backend, native Fähigkeiten, Sitzung und sichtbares Ziel. Verborgene, ersetzte oder weiternavigierte Ziele werden abgelehnt; Aktionen werden nie automatisch wiederholt. Seiteninhalte bleiben nicht vertrauenswürdig. Bei blockierter Iframe-Einbettung verwenden Sie einen externen Browser.
|
||||
|
||||
Tauri-Vorschauen unter macOS/Linux werden manuell geöffnet, behalten gemeinsame Anmeldungen über Neustarts hinweg und verwenden den nativen Zurück-/Vorwärts-Verlauf. Native Vorschauen werden unter lokalen Menüs und Dialogen ausgeblendet; Elementkommentare sind im nativen Modus nicht verfügbar. `codenomad.browser` lehnt unter Tauri macOS/Linux alle Aktionen ab, auch bei sichtbarer Vorschau.
|
||||
Native Vorschauen öffnen sich manuell oder durch den Agenten, behalten gemeinsame Anmeldungen über Neustarts und nutzen den nativen Zurück-/Vorwärts-Verlauf. Lokale Menüs und Dialoge blenden die Vorschau aus. Elementkommentare sind im nativen Modus nicht verfügbar. WebKit erfasst skripterzeugte geschlossene Shadow Roots; deklarative geschlossene Wurzeln bleiben undurchsichtig. Unter macOS kann das erste Hover pointerenter auslassen; ⌘A wählt nativ ohne A-keydown aus. GTK-Zwischenablagezugriff erfolgt ausdrücklich mit Strg+V; Schreiben und natives Kopieren/Einfügen funktionieren.
|
||||
|
||||
Native Pop-ups teilen Vorschau-Anmeldungen, behalten `opener`/`postMessage` und schließen mit ihrer Ursprungsseite. Eingegebene/wiederhergestellte Adressen erlauben HTTP(S) ohne Zugangsdaten; Websites können auch leere und gleichursprüngliche Blob-Dokumente öffnen. Datei- und App-Schemata bleiben gesperrt. Downloads landen mit bereinigten, eindeutigen Namen im Downloads-Ordner, ohne Überschreiben oder automatisches Öffnen. Kamera, Mikrofon, Standort und Benachrichtigungen werden verweigert; klassifizierte Berechtigungen benötigen Zustimmung, unbekannte Anfragen zeigen einen Fehler. Tauri und Electron unterstützen `alert`/`confirm`/`prompt`; Electrons Prompt-Dialog gilt für Hauptseiten der Vorschau, Pop-ups und Unterframes. Geräte- und Bildschirmauswahlen zeigen die verfügbaren Einträge in einer scrollbaren Liste. Weitere APIs hängen von Engine und Hardware ab.
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@ Native previews share cookies, sign-ins and site storage across projects/session
|
|||
| Linux Tauri | Native WebKitGTK child; OAuth pop-ups, downloads and dialogs | Supported for the visible owned session preview | WebKit mobile identity and CSS viewport; native density/input limits |
|
||||
| Plain browser/remote view | Iframe fallback | No local native preview target | Dimensions only |
|
||||
|
||||
On Electron and Windows Tauri, native automation is prepared at desktop startup and uses the session-bound `codenomad.browser` tool for open, navigate, snapshot, screenshot and page actions: click, type, select, hover, drag, scroll, key presses and waiting for text. Tool availability is not authorization for every window: execution checks the owning backend, native capabilities, session and visible target. An action whose page was hidden, replaced or navigated meanwhile is refused rather than sent to another page, and an action is never repeated automatically. Page contents remain untrusted. An iframe may fail because the target forbids embedding; use an external browser rather than weakening its protections.
|
||||
On every native desktop host, the session-bound `codenomad.browser` tool opens, navigates, snapshots and screenshots the preview, and performs click, type, select, hover, drag, scroll, key and text-wait actions. Execution checks the owning backend, native capabilities, session and visible target. Hidden, replaced or navigated targets fail safely; actions are never retried automatically. Page content stays untrusted. For sites blocking iframe embedding, use an external browser.
|
||||
|
||||
Tauri macOS/Linux previews open manually, retain shared sign-ins across restarts and use native Back/Forward history. Native previews hide beneath local menus and dialogs; element comments are unavailable in native mode. `codenomad.browser` refuses all actions on Tauri macOS/Linux, even with a visible preview.
|
||||
Native previews open manually or through the agent, retain shared sign-ins across restarts and use native Back/Forward history. Local menus and dialogs hide the preview. Element comments are unavailable in native mode. WebKit snapshots capture script-created closed shadow roots; declarative closed roots stay opaque. On macOS, the first hover can omit pointerenter and ⌘A selects through the native responder without an A keydown. GTK clipboard reads require explicit Ctrl+V; clipboard writes and native copy/paste work.
|
||||
|
||||
Native pop-ups share preview sign-ins, preserve `opener`/`postMessage` and close with their opener. Typed/restored addresses accept credential-free HTTP(S); sites can also open blank and same-origin blob documents. File and app schemes are blocked. Downloads save to Downloads with sanitized unique names, never overwrite files and never auto-open. Camera, microphone, location and notifications are denied; classified permissions use consent, unknown requests show an error. Tauri and Electron support `alert`/`confirm`/`prompt`; Electron's prompt dialog covers top-level preview pages, pop-ups and subframes. Device and screen selectors scroll through the available entries. Other APIs depend on the engine and hardware.
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@ Las vistas previas nativas comparten cookies, inicios de sesión y almacenamient
|
|||
| Tauri para Linux | Ventana hija nativa WebKitGTK; ventanas OAuth, descargas y diálogos | Compatible para la vista previa visible de la sesión propia | Identidad móvil WebKit y viewport CSS; límites nativos de densidad/entrada |
|
||||
| Navegador normal/vista remota | Alternativa iframe | Sin objetivo local de vista previa nativa | Solo dimensiones |
|
||||
|
||||
En Electron y Tauri para Windows, la automatización nativa se prepara al iniciar el escritorio y usa la herramienta `codenomad.browser` vinculada a la sesión para abrir, navegar, obtener una instantánea, capturar pantalla y realizar acciones en la página: hacer clic, escribir, seleccionar, pasar el cursor, arrastrar, desplazarse, pulsar teclas y esperar un texto. Que una herramienta esté disponible no autoriza todas las ventanas: la ejecución comprueba el backend propietario, las capacidades nativas, la sesión y el objetivo visible. Una acción cuya página se ocultó, se sustituyó o navegó mientras tanto se rechaza en lugar de enviarse a otra página, y una acción nunca se repite automáticamente. El contenido de las páginas sigue sin ser fiable. Un iframe puede fallar porque el destino prohíbe incrustarlo; usa un navegador externo en lugar de debilitar sus protecciones.
|
||||
En todos los hosts de escritorio nativos, la herramienta `codenomad.browser` vinculada a la sesión abre, navega y captura la vista previa, y ejecuta clic, escritura, selección, hover, arrastre, desplazamiento, teclas y espera de texto. La ejecución comprueba el backend propietario, capacidades nativas, sesión y objetivo visible. Los objetivos ocultos, sustituidos o navegados se rechazan; las acciones nunca se repiten automáticamente. El contenido sigue sin ser fiable. Si el sitio bloquea iframe, usa un navegador externo.
|
||||
|
||||
Las vistas previas de Tauri para macOS/Linux se abren manualmente, conservan los inicios de sesión compartidos entre reinicios y usan el historial nativo Atrás/Adelante. Las vistas previas nativas se ocultan bajo los menús y diálogos locales; los comentarios sobre elementos no están disponibles en modo nativo. `codenomad.browser` rechaza todas las acciones en Tauri para macOS/Linux, incluso con una vista previa visible.
|
||||
Las vistas previas nativas se abren manualmente o mediante el agente, conservan inicios de sesión entre reinicios y usan el historial nativo Atrás/Adelante. Menús y diálogos locales ocultan la vista. Los comentarios sobre elementos no están disponibles en modo nativo. WebKit captura shadow roots cerrados creados por script; los declarativos cerrados permanecen opacos. En macOS, el primer hover puede omitir pointerenter y ⌘A selecciona de forma nativa sin keydown A. En GTK se lee el portapapeles mediante Ctrl+V explícito; escritura y copiar/pegar nativos funcionan.
|
||||
|
||||
Las ventanas emergentes nativas comparten las sesiones de la vista previa, conservan `opener`/`postMessage` y se cierran con su página de origen. Las direcciones introducidas/restauradas permiten HTTP(S) sin credenciales; los sitios también pueden abrir documentos vacíos y blob del mismo origen. Se bloquean los esquemas de archivos y de la aplicación. Las descargas usan nombres saneados y únicos en Descargas, sin sobrescribir ni abrir archivos automáticamente. Se deniegan cámara, micrófono, ubicación y notificaciones; los permisos clasificados requieren consentimiento y las solicitudes desconocidas muestran un error. Tauri y Electron admiten `alert`/`confirm`/`prompt`; el diálogo prompt de Electron cubre las páginas principales de la vista previa, las ventanas emergentes y los submarcos. Los selectores de dispositivos y pantallas permiten desplazarse por las entradas disponibles. Otras API dependen del motor y del hardware.
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@ Les aperçus natifs partagent cookies, connexions et stockage de site entre proj
|
|||
| Tauri Linux | Vue enfant WebKitGTK native ; pop-ups OAuth, téléchargements et dialogues | Prise en charge pour l’aperçu de session détenu visible | Identité mobile WebKit et viewport CSS ; limites natives de densité/saisie |
|
||||
| Navigateur ordinaire/vue distante | Iframe de repli | Aucune cible locale d’aperçu natif | Dimensions seules |
|
||||
|
||||
Sur Electron et Tauri Windows, l’automatisation native est préparée au démarrage du bureau et utilise l’outil `codenomad.browser` lié à la session pour ouvrir, naviguer, capturer l’état, prendre une capture d’écran et agir sur la page : cliquer, saisir, sélectionner, survoler, glisser, faire défiler, appuyer sur des touches et attendre un texte. La disponibilité d’un outil n’autorise pas toutes les fenêtres : l’exécution vérifie le backend propriétaire, les capacités natives, la session et la cible visible. Une action dont la page a été masquée, remplacée ou a changé d’adresse entre-temps est refusée plutôt qu’envoyée à une autre page, et une action n’est jamais répétée automatiquement. Le contenu des pages reste non fiable. Une iframe peut échouer si la cible interdit l’intégration ; utilisez un navigateur externe plutôt que d’affaiblir ses protections.
|
||||
Sur tous les hôtes de bureau natifs, l’outil `codenomad.browser` lié à la session ouvre, navigue et capture l’aperçu et effectue les actions clic, saisie, sélection, survol, glisser, défilement, touches et attente de texte. L’exécution vérifie le backend propriétaire, les capacités natives, la session et la cible visible. Les cibles masquées, remplacées ou ayant navigué sont refusées ; les actions ne sont jamais répétées automatiquement. Le contenu reste non fiable. Pour les sites bloquant l’iframe, utilisez un navigateur externe.
|
||||
|
||||
Les aperçus Tauri macOS/Linux s’ouvrent manuellement, conservent les connexions partagées entre les redémarrages et utilisent l’historique natif arrière/avant. Les aperçus natifs sont masqués sous les menus et boîtes de dialogue locaux ; les commentaires sur les éléments sont indisponibles en mode natif. `codenomad.browser` refuse toutes les actions sur Tauri macOS/Linux, même avec un aperçu visible.
|
||||
Les aperçus natifs s’ouvrent manuellement ou par l’agent, conservent les connexions partagées entre redémarrages et utilisent l’historique natif arrière/avant. Les menus et dialogues locaux masquent l’aperçu. Les commentaires sur éléments sont indisponibles en mode natif. Les captures WebKit parcourent les shadow roots fermés créés par script ; les racines fermées déclaratives restent opaques. Sur macOS, le premier survol peut omettre pointerenter et ⌘A sélectionne par le responder natif sans keydown A. Sous GTK, la lecture du presse-papiers passe par Ctrl+V explicite ; écriture et copier/coller natifs fonctionnent.
|
||||
|
||||
Les pop-ups natifs partagent les connexions de l’aperçu, conservent `opener`/`postMessage` et se ferment avec leur page d’origine. Les adresses saisies/restaurées acceptent HTTP(S) sans identifiants ; les sites peuvent aussi ouvrir des documents vierges et blob de même origine. Les schémas de fichiers et applicatifs sont bloqués. Les téléchargements vont dans Téléchargements, avec des noms assainis et uniques, sans écrasement ni ouverture automatique. Caméra, microphone, localisation et notifications sont refusés ; les permissions classifiées demandent un consentement, les requêtes inconnues affichent une erreur. Tauri et Electron prennent en charge `alert`/`confirm`/`prompt` ; le dialogue prompt d’Electron couvre les pages principales de l’aperçu, les pop-ups et les sous-frames. Les sélecteurs de périphériques et d’écrans permettent de faire défiler les entrées disponibles. Les autres API dépendent du moteur et du matériel.
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@
|
|||
| Tauri ב־Linux | חלון־בת מקורי של WebKitGTK; חלונות OAuth, הורדות ותיבות דו־שיח | נתמכת לתצוגת הסשן הגלויה שבבעלות היישום | זהות ניידת של WebKit ואזור תצוגת CSS; מגבלות צפיפות וקלט מקוריות |
|
||||
| דפדפן רגיל/תצוגה מרוחקת | חלופת iframe | ללא יעד תצוגה מקומית מקורית | ממדים בלבד |
|
||||
|
||||
ב־Electron וב־Tauri ב־Windows, האוטומציה המקורית מוכנה בהפעלת שולחן העבודה ומשתמשת בכלי `codenomad.browser` הקשור לסשן לפתיחה, לניווט, לתמונת מצב, לצילום מסך ולפעולות בדף: לחיצה, הקלדה, בחירה, ריחוף, גרירה, גלילה, הקשות מקשים והמתנה לטקסט. זמינות כלי אינה הרשאה לכל חלון: ההרצה בודקת את שרת הקצה הבעלים, היכולות המקוריות, הסשן והיעד הגלוי. פעולה שהדף שלה הוסתר, הוחלף או נווט בינתיים נדחית במקום להישלח לדף אחר, ופעולה לעולם אינה חוזרת אוטומטית. תוכן דפים נשאר לא מהימן. iframe עלול להיכשל כי היעד אוסר הטמעה; השתמשו בדפדפן חיצוני במקום להחליש את הגנותיו.
|
||||
בכל מארחי שולחן העבודה המקוריים, הכלי `codenomad.browser` הקשור לסשן פותח, מנווט ומצלם את התצוגה ומבצע לחיצה, הקלדה, בחירה, ריחוף, גרירה, גלילה, מקשים והמתנה לטקסט. ההרצה בודקת שרת בעלים, יכולות מקוריות, סשן ויעד גלוי. יעדים שהוסתרו, הוחלפו או נווטו נדחים; פעולות לעולם אינן חוזרות אוטומטית. תוכן דפים נשאר לא מהימן. לאתר החוסם iframe השתמשו בדפדפן חיצוני.
|
||||
|
||||
תצוגות Tauri ב־macOS/Linux נפתחות ידנית, שומרות התחברויות משותפות גם לאחר הפעלה מחדש ומשתמשות בהיסטוריית אחורה/קדימה מקורית. תצוגות מקוריות מוסתרות מתחת לתפריטים ולתיבות דו־שיח מקומיים; הערות על רכיבים אינן זמינות במצב מקורי. `codenomad.browser` דוחה את כל הפעולות ב־Tauri ב־macOS/Linux, גם כשהתצוגה גלויה.
|
||||
תצוגות מקוריות נפתחות ידנית או בידי הסוכן, שומרות התחברויות אחרי הפעלה מחדש ומשתמשות בהיסטוריית אחורה/קדימה מקורית. תפריטים ותיבות דו־שיח מקומיים מסתירים את התצוגה. הערות רכיבים אינן זמינות במצב מקורי. WebKit לוכד shadow roots סגורים שנוצרו בסקריפט; שורשים סגורים הצהרתיים נשארים אטומים. ב־macOS הריחוף הראשון עשוי להשמיט pointerenter ו־⌘A בוחר באופן מקורי ללא keydown של A. ב־GTK קריאת הלוח מתבצעת עם Ctrl+V מפורש; כתיבה והעתקה/הדבקה מקוריות פועלות.
|
||||
|
||||
חלונות קופצים מקוריים משתפים את הכניסות לחשבון בתצוגה המקדימה, שומרים על `opener`/`postMessage` ונסגרים עם דף המקור. כתובות שהוזנו או שוחזרו מאפשרות HTTP(S) ללא פרטי גישה; אתרים יכולים לפתוח גם מסמכים ריקים ומסמכי blob מאותו מקור. סכמות קבצים ויישום חסומות. הורדות נשמרות בתיקיית ההורדות בשמות מנוקים וייחודיים, ללא דריסה או פתיחה אוטומטית. מצלמה, מיקרופון, מיקום והתראות נדחים; הרשאות מסווגות דורשות הסכמה ובקשות לא מוכרות מציגות שגיאה. Tauri ו־Electron תומכים ב־`alert`/`confirm`/`prompt`; תיבת prompt של Electron זמינה בדפים הראשיים של התצוגה המקדימה, בחלונות קופצים ובמסגרות משנה. בוררי התקנים ומסכים מאפשרים גלילה בין הרשומות הזמינות. ממשקים אחרים תלויים במנוע ובחומרה.
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@
|
|||
| Linux Tauri | ネイティブ WebKitGTK 子。OAuth ポップアップ、ダウンロード、ダイアログに対応 | 所有する表示中のセッションプレビューで対応 | WebKit モバイル識別と CSS ビューポート。密度と入力はネイティブ環境の制約あり |
|
||||
| 通常のブラウザー／リモート表示 | 代替 iframe | ローカルのネイティブプレビュー対象なし | 寸法のみ |
|
||||
|
||||
Electron と Windows Tauri では、ネイティブ自動操作はデスクトップ起動時に準備され、セッションに紐づく `codenomad.browser` ツールで open、navigate、snapshot、screenshot と、ページ操作（クリック、入力、選択、ホバー、ドラッグ、スクロール、キー入力、テキストの出現待ち）を行います。ツールが使えることは、全ウィンドウの操作許可ではありません。実行時に、所有するバックエンド、ネイティブ機能、セッション、表示中の対象を確認します。その間にページが非表示、置き換え、移動された操作は、別のページへ送られずに拒否され、操作が自動的に繰り返されることもありません。ページ内容は引き続き信頼できない情報です。埋め込みを対象が禁止して iframe が失敗する場合は、保護を弱めず外部ブラウザーを使ってください。
|
||||
すべてのネイティブデスクトップホストで、セッションに紐づく `codenomad.browser` がプレビューを開き、移動、スナップショット、スクリーンショット、クリック、入力、選択、ホバー、ドラッグ、スクロール、キー入力、テキスト待機を実行します。所有するバックエンド、ネイティブ機能、セッション、表示中の対象を確認します。非表示、置き換え、移動済みの対象は拒否し、操作を自動で繰り返しません。ページ内容は信頼できない情報です。iframe を禁止するサイトは外部ブラウザーで開いてください。
|
||||
|
||||
macOS/Linux Tauri のプレビューは手動で開き、共有ログインを再起動後も保持し、ネイティブの戻る／進む履歴を使用します。ローカルのメニューやダイアログが開くとネイティブプレビューは非表示になり、ネイティブモードでは要素へのコメントを利用できません。`codenomad.browser` は、プレビューが表示中でも macOS/Linux Tauri でのすべての操作を拒否します。
|
||||
ネイティブプレビューは手動またはエージェントで開き、再起動後もログインを保持し、戻る／進む履歴を使用します。ローカルメニューやダイアログはプレビューを隠します。ネイティブモードでは要素コメントは利用できません。WebKit はスクリプトで作成した閉じた shadow root を取得し、宣言的な閉じた root は不透明なままです。macOS の最初のホバーは pointerenter を省く場合があり、⌘A は A keydown なしでネイティブ選択します。GTK のクリップボード読取は明示的な Ctrl+V で行い、書込とネイティブのコピー／貼付は動作します。
|
||||
|
||||
ネイティブポップアップはプレビューのログイン状態を共有し、`opener`/`postMessage` を保持し、元のページとともに閉じます。入力・復元するアドレスは認証情報のない HTTP(S) のみで、サイトは空白文書や同一オリジンの blob 文書も開けます。ファイル・アプリのスキームは拒否されます。ダウンロードは安全な一意の名前でダウンロードフォルダーに保存され、上書き・自動実行しません。カメラ、マイク、位置情報、通知は拒否され、分類済み権限には同意が必要です。不明な要求はエラーを表示します。Tauri と Electron は `alert`/`confirm`/`prompt` に対応し、Electron の prompt ダイアログはプレビューのトップレベルページ、ポップアップ、サブフレームで利用できます。デバイスと画面の選択リストはスクロールできます。その他の API はエンジンとハードウェアに依存します。
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@
|
|||
| Linux Tauri | नेटिभ WebKitGTK child; OAuth पप-अप, डाउनलोड र संवाद | देखिने स्वामित्वभित्रको session पूर्वावलोकनमा समर्थित | WebKit मोबाइल पहिचान र CSS भ्युपोर्ट; नेटिभ घनत्व/इनपुट सीमा |
|
||||
| सामान्य browser/remote दृश्य | iframe fallback | स्थानीय native preview target छैन | आयाम मात्र |
|
||||
|
||||
Electron र Windows Tauri मा नेटिभ automation desktop startup मै तयार हुन्छ र open, navigate, snapshot, screenshot र page action (click, type, select, hover, drag, scroll, key press र text पर्खने) का लागि session-bound `codenomad.browser` tool प्रयोग गर्छ। tool उपलब्धता हरेक विन्डोको अनुमति होइन: execution ले स्वामी backend, native capability, session र visible target जाँच्छ। बीचमै पृष्ठ लुकेको, बदलिएको वा अर्कै ठाउँ नेभिगेट भएको action अर्को पृष्ठमा पठाइँदैन, अस्वीकार गरिन्छ; कुनै action स्वतः दोहोर्याइँदैन। पृष्ठ सामग्री अविश्वसनीय नै हुन्छ। target ले embedding निषेध गर्दा iframe असफल हुन सक्छ; सुरक्षा कमजोर पार्नुको सट्टा बाह्य browser प्रयोग गर्नुहोस्।
|
||||
सबै नेटिभ desktop host मा session-bound `codenomad.browser` ले पूर्वावलोकन खोल्ने, नेभिगेट गर्ने, snapshot/screenshot लिने र click, type, select, hover, drag, scroll, key तथा text-wait action चलाउँछ। execution ले स्वामी backend, native capability, session र देखिने target जाँच्छ। लुकेको, बदलिएको वा नेभिगेट भएको target अस्वीकार हुन्छ; action स्वतः दोहोरिँदैन। पृष्ठ सामग्री अविश्वसनीय रहन्छ। iframe रोक्ने साइटका लागि बाह्य browser प्रयोग गर्नुहोस्।
|
||||
|
||||
macOS/Linux Tauri पूर्वावलोकन म्यानुअल रूपमा खुल्छन्, पुनःसुरुवातपछि पनि साझा sign-in राख्छन् र नेटिभ Back/Forward इतिहास प्रयोग गर्छन्। स्थानीय menu र dialog खुल्दा नेटिभ पूर्वावलोकन लुक्छन्; नेटिभ मोडमा element comment उपलब्ध छैनन्। `codenomad.browser` ले macOS/Linux Tauri मा पूर्वावलोकन देखिएकै अवस्थामा पनि सबै action अस्वीकार गर्छ।
|
||||
नेटिभ पूर्वावलोकन म्यानुअल वा agent बाट खुल्छन्, पुनःसुरुवातपछि sign-in राख्छन् र नेटिभ Back/Forward इतिहास प्रयोग गर्छन्। स्थानीय menu/dialog ले पूर्वावलोकन लुकाउँछन्। नेटिभ मोडमा element comment उपलब्ध छैन। WebKit ले script-created closed shadow root पढ्छ; declarative closed root अपारदर्शी रहन्छ। macOS को पहिलो hover ले pointerenter छुटाउन सक्छ र ⌘A ले A keydown बिना नेटिभ चयन गर्छ। GTK clipboard read का लागि स्पष्ट Ctrl+V प्रयोग हुन्छ; write र नेटिभ copy/paste चल्छन्।
|
||||
|
||||
नेटिभ पप-अपले पूर्वावलोकनका साइन-इन साझा गर्छन्, `opener`/`postMessage` कायम राख्छन् र मूल पृष्ठसँगै बन्द हुन्छन्। लेखिएका वा पुनर्स्थापित ठेगानामा परिचयपत्ररहित HTTP(S) मात्र अनुमति छ; साइटले खाली र उही मूलका blob कागजात पनि खोल्न सक्छन्। फाइल र एप स्किम रोकिएका छन्। डाउनलोडहरू सुरक्षित अद्वितीय नाममा Downloads मा राखिन्छन्, फाइल मेटिँदैन र स्वतः खोलिँदैन। क्यामेरा, माइक्रोफोन, स्थान र सूचना अस्वीकार गरिन्छन्; वर्गीकृत अनुमतिमा सहमति मागिन्छ र अज्ञात अनुरोधमा त्रुटि देखाइन्छ। Tauri र Electron मा `alert`/`confirm`/`prompt` उपलब्ध छन्; Electron को prompt संवाद पूर्वावलोकनका मुख्य पृष्ठ, पप-अप र उप-फ्रेममा उपलब्ध छ। उपकरण र स्क्रिन छनोट सूचीका उपलब्ध प्रविष्टिहरू स्क्रोल गर्न सकिन्छ। अन्य API इन्जिन र हार्डवेयरमा निर्भर छन्।
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@
|
|||
| Linux Tauri | Нативный дочерний WebKitGTK; окна OAuth, загрузки и диалоги | Поддерживается для видимого принадлежащего сессии предпросмотра | Мобильная идентификация WebKit и CSS-область; нативные ограничения плотности/ввода |
|
||||
| Обычный браузер/удалённый вид | Резервный iframe | Нет локальной нативной цели предпросмотра | Только размеры |
|
||||
|
||||
В Electron и Windows Tauri нативная автоматизация подготавливается при запуске настольного приложения и использует связанный с сессией инструмент `codenomad.browser` для открытия, навигации, снимка структуры, снимка экрана и действий на странице: нажатия, ввода, выбора, наведения, перетаскивания, прокрутки, нажатия клавиш и ожидания текста. Доступность инструмента не разрешает доступ к любому окну: выполнение проверяет серверную часть-владельца, нативные возможности, сессию и видимую цель. Действие, страница которого тем временем была скрыта, заменена или перешла на другой адрес, отклоняется, а не отправляется на другую страницу, и действие никогда не повторяется автоматически. Контент страниц остаётся недоверенным. Iframe может не работать из-за запрета встраивания у цели; используйте внешний браузер, не ослабляйте защиту.
|
||||
На всех нативных настольных хостах связанный с сессией `codenomad.browser` открывает и навигирует предпросмотр, создаёт снимки структуры/экрана и выполняет нажатие, ввод, выбор, наведение, перетаскивание, прокрутку, клавиши и ожидание текста. Проверяются владеющий сервер, нативные возможности, сессия и видимая цель. Скрытые, заменённые или перешедшие цели отклоняются; действия никогда не повторяются автоматически. Контент остаётся недоверенным. Для сайтов, блокирующих iframe, используйте внешний браузер.
|
||||
|
||||
Предпросмотры Tauri в macOS/Linux открываются вручную, сохраняют общие входы после перезапуска и используют нативную историю переходов назад/вперёд. Нативные предпросмотры скрываются под локальными меню и диалогами; комментарии к элементам в нативном режиме недоступны. `codenomad.browser` отклоняет все действия в Tauri macOS/Linux, даже при видимом предпросмотре.
|
||||
Нативные предпросмотры открываются вручную или агентом, сохраняют входы после перезапуска и используют нативную историю назад/вперёд. Локальные меню и диалоги скрывают предпросмотр. Комментарии к элементам в нативном режиме недоступны. WebKit читает закрытые shadow roots, созданные скриптом; декларативные закрытые корни остаются непрозрачными. На macOS первое наведение может пропустить pointerenter, а ⌘A выбирает нативно без A keydown. Чтение буфера GTK выполняется явным Ctrl+V; запись и нативное копирование/вставка работают.
|
||||
|
||||
Нативные всплывающие окна используют общие входы предпросмотра, сохраняют `opener`/`postMessage` и закрываются вместе с исходной страницей. Введённые и восстановленные адреса допускают HTTP(S) без учётных данных; сайты также могут открывать пустые и одноисточниковые blob-документы. Схемы файлов и приложения заблокированы. Загрузки сохраняются в папку загрузок с очищенными уникальными именами, без перезаписи и автоматического открытия. Камера, микрофон, местоположение и уведомления запрещены; классифицированные разрешения требуют согласия, неизвестные запросы показывают ошибку. Tauri и Electron поддерживают `alert`/`confirm`/`prompt`; диалог prompt в Electron доступен на основных страницах предпросмотра, во всплывающих окнах и подфреймах. Списки устройств и экранов позволяют прокручивать доступные элементы. Остальные API зависят от движка и оборудования.
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@ Native önizlemeler aynı masaüstü motoru ve profil/yapılandırma kapsamında
|
|||
| Linux Tauri | Yerel WebKitGTK alt öğesi; OAuth pencereleri, indirmeler ve iletişim kutuları | Görünür sahip olunan oturum önizlemesi için desteklenir | WebKit mobil kimliği ve CSS görünümü; yerel yoğunluk/giriş sınırları |
|
||||
| Normal tarayıcı/uzak görünüm | Alternatif iframe görünümü | Yerel native önizleme hedefi yok | Yalnızca boyutlar |
|
||||
|
||||
Electron ve Windows Tauri'de yerel otomasyon, masaüstü başlangıcında hazırlanır ve açma, gezinme, anlık görüntü, ekran görüntüsü ve sayfa eylemleri için oturuma bağlı `codenomad.browser` aracını kullanır: tıklama, yazma, seçme, üzerine gelme, sürükleme, kaydırma, tuşa basma ve metin bekleme. Aracın kullanılabilirliği her pencere için yetkilendirme değildir: yürütme, sahip olunan arka ucu, yerel yetenekleri, oturumu ve görünür hedefi kontrol eder. Sayfası bu arada gizlenmiş, değiştirilmiş veya başka bir adrese gitmiş bir eylem başka bir sayfaya gönderilmek yerine reddedilir ve hiçbir eylem otomatik olarak tekrarlanmaz. Sayfa içerikleri güvenilmez durumda kalıyor. Hedef yerleştirmeyi yasakladığından iframe başarısız olabilir; korumalarını zayıflatmak yerine harici bir tarayıcı kullanın.
|
||||
Tüm yerel masaüstü sunucularında oturuma bağlı `codenomad.browser`, önizlemeyi açar, gezinir, görüntüler ve tıklama, yazma, seçme, üzerine gelme, sürükleme, kaydırma, tuş ve metin bekleme eylemlerini yürütür. Sahip arka uç, yerel yetenekler, oturum ve görünür hedef kontrol edilir. Gizlenmiş, değiştirilmiş veya gezinmiş hedefler reddedilir; eylemler otomatik tekrarlanmaz. Sayfa içeriği güvenilmez kalır. iframe engelleyen siteler için harici tarayıcı kullanın.
|
||||
|
||||
Tauri macOS/Linux önizlemeleri elle açılır, paylaşılan oturum açma bilgilerini yeniden başlatmalar arasında korur ve yerel Geri/İleri geçmişini kullanır. Yerel önizlemeler, uygulama içindeki menü ve iletişim kutularının altında gizlenir; yerel modda öğe yorumları kullanılamaz. `codenomad.browser`, görünür bir önizleme olsa bile Tauri macOS/Linux'ta tüm eylemleri reddeder.
|
||||
Yerel önizlemeler elle veya ajan tarafından açılır, oturum açmayı yeniden başlatmalarda korur ve yerel Geri/İleri geçmişini kullanır. Yerel menü ve iletişim kutuları önizlemeyi gizler. Yerel modda öğe yorumları kullanılamaz. WebKit, betikle oluşturulan kapalı shadow root'ları okur; bildirimsel kapalı kökler opak kalır. macOS ilk hover'da pointerenter atlayabilir; ⌘A, A keydown olmadan yerel seçim yapar. GTK pano okuması açık Ctrl+V ile yapılır; yazma ve yerel kopyala/yapıştır çalışır.
|
||||
|
||||
Yerel açılır pencereler önizleme oturumlarını paylaşır, `opener`/`postMessage` ilişkisini korur ve açan sayfayla kapanır. Yazılan veya geri yüklenen adresler kimlik bilgisiz HTTP(S) ile sınırlıdır; siteler boş ve aynı kökenli blob belgeleri de açabilir. Dosya ve uygulama şemaları engellenir. İndirmeler temizlenmiş benzersiz adlarla İndirilenler'e kaydedilir; üzerine yazılmaz ve otomatik açılmaz. Kamera, mikrofon, konum ve bildirimler reddedilir; sınıflandırılmış izinler onay ister, bilinmeyen istekler hata gösterir. Tauri ve Electron `alert`/`confirm`/`prompt` destekler; Electron'un prompt iletişim kutusu önizlemenin ana sayfalarında, açılır pencerelerde ve alt çerçevelerde kullanılabilir. Aygıt ve ekran seçicilerinde mevcut girdiler arasında kaydırma yapılabilir. Diğer API'ler motora ve donanıma bağlıdır.
|
||||
|
||||
|
|
|
|||
|
|
@ -24,9 +24,9 @@
|
|||
| Linux Tauri | 原生 WebKitGTK 子窗口；支持 OAuth 弹窗、下载和对话框 | 支持可见且归属已确认的会话预览 | WebKit 移动标识和 CSS 视口；密度和输入受原生限制 |
|
||||
| 普通浏览器/远程视图 | iframe 后备实现 | 无本地原生预览目标 | 仅尺寸 |
|
||||
|
||||
在 Electron 和 Windows Tauri 上，原生自动化在桌面启动时准备好，使用会话绑定的 `codenomad.browser` 工具进行打开、导航、快照、截图和页面操作：点击、输入、选择、悬停、拖动、滚动、按键以及等待文本出现。工具可用不代表对每个窗口都已获授权：执行会检查所属后端、原生能力、会话和可见目标。如果操作所针对的页面在此期间已被隐藏、替换或导航离开，该操作会被拒绝，而不会发送到其他页面；操作也绝不会被自动重复。页面内容仍不可信。目标禁止嵌入时 iframe 可能失败；请使用外部浏览器，不要削弱保护。
|
||||
在所有原生桌面宿主上，会话绑定的 `codenomad.browser` 都可打开、导航、快照和截图预览，并执行点击、输入、选择、悬停、拖动、滚动、按键和等待文本操作。执行时会检查所属后端、原生能力、会话和可见目标。隐藏、替换或导航后的目标会被拒绝；操作绝不自动重试。页面内容仍不可信。对于阻止 iframe 嵌入的网站，请使用外部浏览器。
|
||||
|
||||
Tauri macOS/Linux 预览需手动打开，共享登录状态在重启后仍会保留，并使用原生后退/前进历史。原生预览会在应用内菜单和对话框下方隐藏；原生模式不支持元素评论。即使预览可见，`codenomad.browser` 也会拒绝 Tauri macOS/Linux 上的所有操作。
|
||||
原生预览可手动或由智能体打开，重启后保留登录，并使用原生后退/前进历史。本地菜单和对话框会隐藏预览。原生模式不支持元素评论。WebKit 可读取脚本创建的封闭 shadow root；声明式封闭根保持不透明。macOS 首次悬停可能省略 pointerenter；⌘A 通过原生响应器选择，不产生 A keydown。GTK 剪贴板读取需显式 Ctrl+V；写入和原生复制/粘贴可用。
|
||||
|
||||
原生弹窗共享预览登录状态，保留 `opener`/`postMessage`，并随打开它的页面关闭。输入或恢复的地址仅允许无凭据的 HTTP(S)；网站还可打开空白文档和同源 blob 文档。文件及应用协议被阻止。下载使用清理后的唯一名称保存到“下载”文件夹，不覆盖文件，也不自动打开。摄像头、麦克风、位置和通知始终被拒绝；已分类权限需要同意，未知请求显示错误。Tauri 和 Electron 支持 `alert`/`confirm`/`prompt`；Electron 的 prompt 对话框适用于预览顶层页面、弹窗和子框架。设备及屏幕选择列表可滚动查看可用项目。其他 API 取决于引擎和硬件。
|
||||
|
||||
|
|
|
|||
|
|
@ -2,13 +2,16 @@ import assert from "node:assert/strict"
|
|||
import test from "node:test"
|
||||
import { NativeParent, NATIVE_REQUEST_PREFIX, NATIVE_RESPONSE_PREFIX } from "./native-parent"
|
||||
|
||||
test("WebKit's explicit public automation gate remains closed with an available private native transport", () => {
|
||||
test("WebKit transport exposes browser automation and respects explicit missing capabilities", () => {
|
||||
const previous = process.env.CODENOMAD_TAURI_WEBKIT
|
||||
const previousCapability = process.env.CODENOMAD_NATIVE_BROWSER_AUTOMATION
|
||||
try {
|
||||
process.env.CODENOMAD_TAURI_WEBKIT = "1"
|
||||
delete process.env.CODENOMAD_NATIVE_BROWSER_AUTOMATION
|
||||
assert.equal(new NativeParent(undefined, true).browserAutomationUnavailable, false)
|
||||
assert.equal(new NativeParent(undefined, false).browserAutomationUnavailable, false)
|
||||
process.env.CODENOMAD_NATIVE_BROWSER_AUTOMATION = "1"
|
||||
assert.equal(new NativeParent(undefined, true).browserAutomationUnavailable, false)
|
||||
process.env.CODENOMAD_NATIVE_BROWSER_AUTOMATION = "0"
|
||||
assert.equal(new NativeParent(undefined, true).browserAutomationUnavailable, true)
|
||||
assert.equal(new NativeParent(undefined, false).browserAutomationUnavailable, true)
|
||||
|
|
@ -20,7 +23,7 @@ test("WebKit's explicit public automation gate remains closed with an available
|
|||
}
|
||||
})
|
||||
|
||||
test("WebKit browser refusal is independent of the native transport", async () => {
|
||||
test("a genuinely unavailable native transport rejects requests", async () => {
|
||||
const parent = new NativeParent({ write: () => { throw new Error("Unexpected native request") } }, false, true)
|
||||
assert.equal(parent.browserAutomationUnavailable, true)
|
||||
await assert.rejects(parent.request("browser.probe", {}), /Native control is unavailable/)
|
||||
|
|
|
|||
|
|
@ -26,8 +26,7 @@ export class NativeParent {
|
|||
constructor(
|
||||
private readonly output: Pick<NodeJS.WriteStream, "write"> = process.stdout,
|
||||
readonly available = process.env.CODENOMAD_NATIVE_PARENT === "1",
|
||||
readonly browserAutomationUnavailable = process.env.CODENOMAD_NATIVE_BROWSER_AUTOMATION === "0"
|
||||
|| (!available && process.env.CODENOMAD_TAURI_WEBKIT === "1"),
|
||||
readonly browserAutomationUnavailable = process.env.CODENOMAD_NATIVE_BROWSER_AUTOMATION === "0",
|
||||
) {}
|
||||
|
||||
request<T>(method: string, params: unknown, timeoutMs = DEFAULT_TIMEOUT_MS): Promise<T> {
|
||||
|
|
|
|||
|
|
@ -8,7 +8,6 @@ import childProcess from "node:child_process"
|
|||
import { syncBuiltinESMExports } from "node:module"
|
||||
import {
|
||||
AUTOMATION_BRIDGE_PATH,
|
||||
BROWSER_AUTOMATION_UNAVAILABLE,
|
||||
automationBridgeDirectory,
|
||||
automationBridgeDirectories,
|
||||
discoverAutomationBridgeDirectories,
|
||||
|
|
@ -180,22 +179,21 @@ test("mission discovery waits for cold ownership validation and still rejects co
|
|||
}
|
||||
})
|
||||
|
||||
test("browser discovery preserves WebKit refusals without executing a preview action", async () => {
|
||||
const root = await mkdtemp(path.join(os.tmpdir(), "codenomad-webkit-refusal-"))
|
||||
test("browser discovery executes native WebKit open, snapshot and screenshot actions", async () => {
|
||||
const root = await mkdtemp(path.join(os.tmpdir(), "codenomad-webkit-browser-"))
|
||||
const restore = isolateAutomationBridgeRegistry(root)
|
||||
const calls: unknown[] = []
|
||||
let remove: (() => Promise<void>) | undefined
|
||||
const { server, url } = await listen(async (body, response) => {
|
||||
calls.push(body.mode)
|
||||
response.statusCode = 409
|
||||
return { error: BROWSER_AUTOMATION_UNAVAILABLE }
|
||||
return { result: body.mode === "browser-execute" ? { text: "WebKit native result" } : { available: true } }
|
||||
})
|
||||
try {
|
||||
remove = await publishAutomationBridge(createAutomationBridgeRegistration(url))
|
||||
for (const command of [{ action: "open", url: "https://example.com" }, { action: "snapshot" }, { action: "screenshot" }]) {
|
||||
await assert.rejects(executeBrowserTool("ses_webkit", command), { message: BROWSER_AUTOMATION_UNAVAILABLE })
|
||||
assert.match((await executeBrowserTool("ses_webkit", command)).content as string, /WebKit native result/)
|
||||
}
|
||||
assert.deepEqual(calls, ["browser-claim", "browser-probe", "browser-probe"])
|
||||
assert.deepEqual(calls, ["browser-claim", "browser-execute", "browser-probe", "browser-execute", "browser-probe", "browser-execute"])
|
||||
} finally {
|
||||
await remove?.()
|
||||
await closeServer(server)
|
||||
|
|
|
|||
|
|
@ -547,8 +547,6 @@ async function probeBridges(
|
|||
return found.slice(0, limit)
|
||||
}
|
||||
|
||||
export const BROWSER_AUTOMATION_UNAVAILABLE = "Agent browser automation is unavailable in Tauri on macOS and Linux. Open the web preview manually, or use Electron for agent browser automation."
|
||||
|
||||
async function probeBrowserBridges(
|
||||
active: DiscoveredBridgeRegistration[],
|
||||
sessionID: string,
|
||||
|
|
@ -557,12 +555,10 @@ async function probeBrowserBridges(
|
|||
purpose?: "mission",
|
||||
): Promise<DiscoveredBridgeRegistration[]> {
|
||||
const found: DiscoveredBridgeRegistration[] = []
|
||||
let unavailable = false
|
||||
for (let index = 0; index < active.length && found.length < 2; index += PROBE_CONCURRENCY) {
|
||||
const batch = await Promise.all(active.slice(index, index + PROBE_CONCURRENCY).map(async (registration) => {
|
||||
try {
|
||||
const response = await callBridge(registration, { mode, sessionID, ...(purpose ? { purpose } : {}) }, timeoutMs)
|
||||
if (response.body.error === BROWSER_AUTOMATION_UNAVAILABLE) unavailable = true
|
||||
return response.status === 200 ? registration : undefined
|
||||
} catch {
|
||||
return undefined
|
||||
|
|
@ -572,7 +568,6 @@ async function probeBrowserBridges(
|
|||
if (registration) found.push(registration)
|
||||
}
|
||||
}
|
||||
if (unavailable) throw new Error(BROWSER_AUTOMATION_UNAVAILABLE)
|
||||
return found.slice(0, 2)
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
import assert from "node:assert/strict"
|
||||
import test from "node:test"
|
||||
import Fastify from "fastify"
|
||||
import { AUTOMATION_BRIDGE_PATH, BROWSER_AUTOMATION_UNAVAILABLE } from "../../opencode/automation-plugin"
|
||||
import { AUTOMATION_BRIDGE_PATH } from "../../opencode/automation-plugin"
|
||||
import { registerAutomationPluginRoute } from "./automation-plugin"
|
||||
|
||||
test("a slow unrelated workspace cannot hide an already verified automation owner", { timeout: 2_000 }, async t => {
|
||||
|
|
@ -173,12 +173,11 @@ test("browser works with an unavailable developer target and unrelated inventory
|
|||
})
|
||||
|
||||
assert.equal((await request({ mode: "browser-claim", sessionID: "session-1" })).statusCode, 200)
|
||||
assert.deepEqual(nativeCalls, [{ method: "browser.probe", params: { sessionID: "session-1" } }])
|
||||
assert.equal(nativeCalls.length, 0)
|
||||
assert.equal((await request({ mode: "browser-probe", sessionID: "session-1" })).statusCode, 200)
|
||||
const open = await request({ mode: "browser-execute", sessionID: "session-1", command: { action: "open", url: "https://example.com" } })
|
||||
assert.equal(open.statusCode, 200)
|
||||
assert.deepEqual(nativeCalls, [
|
||||
{ method: "browser.probe", params: { sessionID: "session-1" } },
|
||||
{ method: "browser.probe", params: { sessionID: "session-1" } },
|
||||
{
|
||||
method: "browser.execute",
|
||||
|
|
@ -198,15 +197,15 @@ test("browser works with an unavailable developer target and unrelated inventory
|
|||
await app.close()
|
||||
})
|
||||
|
||||
for (const transport of [true, false]) test(`WebKit browser refusals preserve Mission-only claims (native transport: ${transport})`, async t => {
|
||||
test("WebKit browser claims, probes and native actions succeed independently of developer CDP", async t => {
|
||||
const app = Fastify({ logger: false })
|
||||
t.after(() => app.close())
|
||||
const nativeCalls: string[] = []
|
||||
registerAutomationPluginRoute(app, {
|
||||
authManager: { isLoopbackRequest: () => true }, bridgeToken: "secret",
|
||||
nativeParent: { available: transport, browserAutomationUnavailable: !transport, request: async (method: string) => {
|
||||
nativeParent: { available: true, browserAutomationUnavailable: false, request: async (method: string) => {
|
||||
nativeCalls.push(method)
|
||||
throw new Error(BROWSER_AUTOMATION_UNAVAILABLE)
|
||||
return method === "browser.probe" ? { available: true } : { text: "WebKit native result" }
|
||||
} },
|
||||
workspaceManager: {
|
||||
getSharedServiceClient: async () => ({ session: { get: async () => ({ location: { directory: "/repo" } }) } }),
|
||||
|
|
@ -219,22 +218,18 @@ for (const transport of [true, false]) test(`WebKit browser refusals preserve Mi
|
|||
assert.equal((await request({ mode: "browser-claim", purpose: "mission" })).statusCode, 200)
|
||||
assert.deepEqual(nativeCalls, [])
|
||||
const claim = await request({ mode: "browser-claim" })
|
||||
assert.equal(claim.statusCode, 409)
|
||||
assert.equal(claim.json().error, BROWSER_AUTOMATION_UNAVAILABLE)
|
||||
assert.deepEqual(nativeCalls, transport ? ["browser.probe"] : [], "claim refuses without requesting preview creation")
|
||||
assert.equal(claim.statusCode, 200)
|
||||
assert.deepEqual(nativeCalls, [], "claim can open the first preview without an existing target")
|
||||
const probe = await request({ mode: "browser-probe" })
|
||||
assert.equal(probe.statusCode, transport ? 404 : 409)
|
||||
assert.equal(probe.json().error, BROWSER_AUTOMATION_UNAVAILABLE)
|
||||
assert.equal(probe.statusCode, 200)
|
||||
assert.equal(probe.json().result.available, true)
|
||||
for (const command of [
|
||||
{ action: "open", url: "https://example.com" }, { action: "navigate", url: "https://example.com" },
|
||||
{ action: "screenshot" }, { action: "snapshot" }, { action: "click", ref: "e1" },
|
||||
{ action: "type", ref: "e1", text: "x" }, { action: "select", ref: "e1", values: ["x"] },
|
||||
{ action: "hover", ref: "e1" }, { action: "drag", ref: "e1", to: "e2" },
|
||||
{ action: "scroll", direction: "down" }, { action: "press", key: "Enter" }, { action: "wait", text: "Ready" },
|
||||
{ action: "screenshot" },
|
||||
]) {
|
||||
const response = await request({ mode: "browser-execute", command })
|
||||
assert.equal(response.statusCode, 409, command.action)
|
||||
assert.equal(response.json().error, BROWSER_AUTOMATION_UNAVAILABLE, command.action)
|
||||
assert.equal(response.statusCode, 200, command.action)
|
||||
assert.equal(response.json().result.text, "WebKit native result", command.action)
|
||||
}
|
||||
})
|
||||
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import type { AuthManager } from "../../auth/manager"
|
|||
import type { DeveloperCdp } from "../../developer-cdp"
|
||||
import type { DeveloperCdpIdentity, DeveloperCdpSelection } from "../../developer-cdp"
|
||||
import type { NativeParent } from "../../native-parent"
|
||||
import { AUTOMATION_BRIDGE_PATH, BROWSER_AUTOMATION_UNAVAILABLE, parseBrowserAction, parseDeveloperAction } from "../../opencode/automation-plugin"
|
||||
import { AUTOMATION_BRIDGE_PATH, parseBrowserAction, parseDeveloperAction } from "../../opencode/automation-plugin"
|
||||
import type { WorkspaceManager } from "../../workspaces/manager"
|
||||
import type { WorktreeDeletionFence } from "../../workspaces/worktree-session-evacuation"
|
||||
import { admitMissionInput } from "./mission-input"
|
||||
|
|
@ -155,7 +155,7 @@ export function registerAutomationPluginRoute(app: FastifyInstance, deps: Automa
|
|||
|
||||
if (["browser-claim", "browser-probe", "browser-execute"].includes(String(body.mode)) && !(body.mode === "browser-claim" && body.purpose === "mission")
|
||||
&& deps.nativeParent.browserAutomationUnavailable) {
|
||||
return reply.code(409).send({ error: BROWSER_AUTOMATION_UNAVAILABLE })
|
||||
return reply.code(409).send({ code: "unsupported", error: "Browser automation is unavailable in this host" })
|
||||
}
|
||||
|
||||
if (body.mode === "browser-claim") {
|
||||
|
|
@ -165,16 +165,6 @@ export function registerAutomationPluginRoute(app: FastifyInstance, deps: Automa
|
|||
if (!deps.nativeParent.available && body.purpose !== "mission") {
|
||||
return reply.code(404).send({ error: "Browser automation is unavailable in this host" })
|
||||
}
|
||||
if (body.purpose !== "mission") {
|
||||
try {
|
||||
await deps.nativeParent.request("browser.probe", { sessionID: body.sessionID })
|
||||
} catch (error) {
|
||||
// A supported host can open a preview before it has a visible target.
|
||||
if (error instanceof Error && error.message === BROWSER_AUTOMATION_UNAVAILABLE) {
|
||||
return reply.code(409).send({ error: error.message })
|
||||
}
|
||||
}
|
||||
}
|
||||
return reply.send({ result: { available: true } })
|
||||
}
|
||||
if (body.mode === "browser-probe") {
|
||||
|
|
|
|||
|
|
@ -220,7 +220,7 @@ fn main() {
|
|||
.arg(&args[3])
|
||||
.env("CODENOMAD_NATIVE_PARENT", "1")
|
||||
.env("CODENOMAD_TAURI_WEBKIT", "1")
|
||||
.env("CODENOMAD_NATIVE_BROWSER_AUTOMATION", "0")
|
||||
.env("CODENOMAD_NATIVE_BROWSER_AUTOMATION", "1")
|
||||
.stdin(Stdio::piped())
|
||||
.stdout(Stdio::piped())
|
||||
.stderr(Stdio::inherit())
|
||||
|
|
|
|||
|
|
@ -1372,7 +1372,7 @@ fn no_browser_target_error() -> String {
|
|||
|
||||
#[cfg(any(test, not(windows)))]
|
||||
fn unsupported() -> String {
|
||||
"Agent browser automation is unavailable in Tauri on macOS and Linux. Open the web preview manually, or use Electron for agent browser automation.".to_string()
|
||||
"Native browser previews are unavailable on this platform.".to_string()
|
||||
}
|
||||
|
||||
#[cfg(any(windows, target_os = "linux", target_os = "macos"))]
|
||||
|
|
@ -1604,8 +1604,11 @@ mod regressions;
|
|||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn webkit_agent_refusal_offers_manual_preview_and_electron() {
|
||||
assert_eq!(super::unsupported(), "Agent browser automation is unavailable in Tauri on macOS and Linux. Open the web preview manually, or use Electron for agent browser automation.");
|
||||
fn webkit_hosts_publish_native_browser_profiles_and_automation() {
|
||||
let source = include_str!("local_windows.rs");
|
||||
let webkit = source.lines().find(|line| line.contains("reason:'webkit'")).unwrap();
|
||||
assert!(webkit.contains("emulation:true,automation:true"));
|
||||
assert!(include_str!("cli_manager.rs").contains("cfg!(any(windows, target_os = \"macos\", target_os = \"linux\"))"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
|
|
|||
|
|
@ -1261,7 +1261,7 @@ impl CliProcessManager {
|
|||
},
|
||||
);
|
||||
c.env("CODENOMAD_NATIVE_PARENT", "1");
|
||||
c.env("CODENOMAD_NATIVE_BROWSER_AUTOMATION", if cfg!(windows) { "1" } else { "0" });
|
||||
c.env("CODENOMAD_NATIVE_BROWSER_AUTOMATION", if cfg!(any(windows, target_os = "macos", target_os = "linux")) { "1" } else { "0" });
|
||||
configure_spawn(&mut c);
|
||||
if let Some(ref cwd) = cwd {
|
||||
c.current_dir(cwd);
|
||||
|
|
|
|||
|
|
@ -11,7 +11,7 @@ const MAX_FOLDER_ATTEMPTS: u8 = 3;
|
|||
#[cfg(target_os = "windows")]
|
||||
const LOCAL_WINDOW_CONTEXT_SCRIPT: &str = "window.__CODENOMAD_RUNTIME_HOST__ = 'tauri'; window.__CODENOMAD_WINDOW_CONTEXT__ = 'local'; window.__CODENOMAD_BROWSER_CAPABILITIES__ = Object.freeze({embedding:true,history:true,emulation:true,automation:true});";
|
||||
#[cfg(any(target_os = "macos", target_os = "linux"))]
|
||||
const LOCAL_WINDOW_CONTEXT_SCRIPT: &str = "window.__CODENOMAD_RUNTIME_HOST__ = 'tauri'; window.__CODENOMAD_WINDOW_CONTEXT__ = 'local'; window.__CODENOMAD_BROWSER_CAPABILITIES__ = Object.freeze({embedding:true,history:true,emulation:true,automation:false,reason:'webkit'});";
|
||||
const LOCAL_WINDOW_CONTEXT_SCRIPT: &str = "window.__CODENOMAD_RUNTIME_HOST__ = 'tauri'; window.__CODENOMAD_WINDOW_CONTEXT__ = 'local'; window.__CODENOMAD_BROWSER_CAPABILITIES__ = Object.freeze({embedding:true,history:true,emulation:true,automation:true,reason:'webkit'});";
|
||||
#[cfg(not(any(target_os = "windows", target_os = "macos", target_os = "linux")))]
|
||||
const LOCAL_WINDOW_CONTEXT_SCRIPT: &str = "window.__CODENOMAD_RUNTIME_HOST__ = 'tauri'; window.__CODENOMAD_WINDOW_CONTEXT__ = 'local'; window.__CODENOMAD_BROWSER_CAPABILITIES__ = Object.freeze({embedding:false,history:false,emulation:false,automation:false,reason:'unsupported-platform'});";
|
||||
|
||||
|
|
|
|||
|
|
@ -17,7 +17,9 @@ python3 packages/tauri-app/scripts/test-browser-parent-webkit.py \
|
|||
|
||||
Requirements: Rust, Node, OpenSSL, and the host's native development libraries (GTK 3 / WebKitGTK 4.1, version 2.40+ for isolated-world `evaluate_javascript`, on Linux). Linux inherits display/session variables from the user manager; `GDK_BACKEND=x11` or `wayland` selects the engine display backend. `--nested-kde` owns a separate virtual KWin Wayland compositor alongside an existing desktop and terminates only that compositor. The Cargo target is caller-owned and reusable. The initial offline build reuses the checkout lockfile's resolved versions and the production native build helper; it has a five-minute bound. Native execution is bounded to 150 seconds. Output includes `parent-result.json` and the native `parent-capture.png`.
|
||||
|
||||
The harness compiles production `BrowserController`, `EmbeddedViews`, local-window identity validation, native adapters and `native_request::dispatch`. Its child uses production `NativeParent`, real prefixed stdout requests and piped stdin responses. The owning-window inventory and renderer's open-claim callback are fixture-supplied; the existing application UI subscription has its own regression. The harness starts its own fixture child and HTTP/self-signed TLS servers. Keep its temporary data profile separate from application profiles. The public automation capability stays disabled while private requests are exercised directly.
|
||||
The harness compiles production `BrowserController`, `EmbeddedViews`, local-window identity validation, native adapters and `native_request::dispatch`. Its child uses production `NativeParent`, real prefixed stdout requests and piped stdin responses. The owning-window inventory and renderer's open-claim callback are fixture-supplied; the existing application UI subscription has its own regression. The harness starts its own fixture child and HTTP/self-signed TLS servers. Keep its temporary data profile separate from application profiles. Native browser automation is available on all supported desktop hosts.
|
||||
|
||||
The full application qualification runs with `node scripts/qualify-native-browser-app.mjs <app-executable> <opencode-executable> <output-directory>`. It starts a private OpenCode daemon, a deterministic provider issuing actual `codenomad.browser` agent tool calls, the packaged application under a unique data profile/configuration, and a local page. Lifecycle discovery is pinned to that retained private daemon by a fixture CLI adapter; shared services remain untouched. The real UI opens the session preview through the real backend and native-parent channel. Checks cover open, snapshot, all page actions, screenshot, OAuth opener/postMessage, a downloaded file and a native JavaScript dialog. Artifacts contain completed native tool messages, the app hash, PNG and process logs. Run on a graphical desktop; dialog acceptance targets the owned app process.
|
||||
|
||||
The fixed fixture-only operations control visibility, a renderer dialog, duplicate ownership, close/reopen, response delay and backend retirement. Native capture-completion tests revoke authority at the callback guard's acceptance boundary, after the real engine has produced its result. The controller's native navigation and capture operations remain unchanged by the fixture. These operations belong only to this example executable, never renderer IPC or the application parent-method allowlist.
|
||||
|
||||
|
|
|
|||
|
|
@ -16,7 +16,7 @@ const request=<T=any>(method:string,params:object={},timeout=20_000)=>parent.req
|
|||
const control=(kind:string)=>request('fixture.action-control',{kind})
|
||||
const observe=()=>request('fixture.action-probe')
|
||||
async function main(){
|
||||
assert.equal(parent.browserAutomationUnavailable,true)
|
||||
assert.equal(parent.browserAutomationUnavailable,false)
|
||||
await request('browser.execute',{command:{action:'open',url:`${base}/read/preview`}})
|
||||
const result=await actionScenarios({
|
||||
navigate:name=>{console.error(`action fixture: ${name}`);return request('browser.execute',{command:{action:'navigate',url:`${base}/read/${name}`}})},
|
||||
|
|
|
|||
22
packages/tauri-app/tests/browser-parent/app.html
Normal file
22
packages/tauri-app/tests/browser-parent/app.html
Normal file
|
|
@ -0,0 +1,22 @@
|
|||
<!doctype html><meta charset="utf-8"><title>Native app qualification</title>
|
||||
<style>body{font:16px sans-serif;margin:16px}button,input,select{margin:5px;padding:6px}.row{display:flex;gap:15px}#source,#target{display:inline-block;padding:15px;border:1px solid;touch-action:none}#scroller{height:80px;overflow:auto;width:280px;border:1px solid}#scroller div{height:500px}</style>
|
||||
<h1>Ready native browser</h1>
|
||||
<button id="increment">Increment</button><output id="count">Count 0</output>
|
||||
<label>Name<input id="name"></label><label>Plan<select id="plan"><option>Standard</option><option>Pro</option></select></label>
|
||||
<output id="keys">Keys idle</output><button id="hover">Hover target</button><output id="hovered">Hover idle</output>
|
||||
<div class="row"><div id="source" role="button" aria-label="Drag source">Drag source</div><div id="target" role="button" aria-label="Drop target">Drop target</div><output id="dragged">Drag idle</output></div>
|
||||
<div id="scroller" role="region" aria-label="Scroll box"><div>Scrollable content</div></div><output id="scrolled">Scroll idle</output>
|
||||
<button id="oauth">OAuth popup</button><output id="oauthState">OAuth idle</output>
|
||||
<a id="download" href="/download" download>Download fixture</a><button id="dialog">Native dialog</button><output id="dialogState">Dialog idle</output>
|
||||
<script>
|
||||
let count=0,drag=false;
|
||||
increment.onclick=()=>document.querySelector('#count').textContent='Count '+ ++count;
|
||||
document.querySelector('#name').onkeydown=e=>{if(e.key==='Enter')keys.textContent='Pressed Enter'};
|
||||
document.querySelector('#plan').onchange=e=>keys.textContent='Plan '+e.target.value;
|
||||
hover.onmouseover=()=>hovered.textContent='Hovered';
|
||||
source.onpointerdown=()=>drag=true;document.onpointerup=e=>{if(drag&&e.clientX>target.getBoundingClientRect().left)dragged.textContent='Dragged';drag=false};
|
||||
scroller.onscroll=()=>scrolled.textContent='Scrolled';
|
||||
oauth.onclick=()=>window.open('/oauth','p7-oauth','width=360,height=240');
|
||||
onmessage=e=>{if(e.origin===location.origin&&e.data==='oauth-complete')oauthState.textContent='OAuth complete'};
|
||||
dialog.onclick=()=>{alert('Native app qualification');dialogState.textContent='Dialog accepted'};
|
||||
</script>
|
||||
|
|
@ -12,7 +12,7 @@ const execute = <T = {url: string}>(action: string, url?: string, timeout?: numb
|
|||
const checks: string[] = []
|
||||
async function check(name: string, run: () => Promise<void>) { await run(); checks.push(name); console.error(`PASS ${name}`) }
|
||||
try {
|
||||
assert.equal(parent.browserAutomationUnavailable, true, "public automation gate remains closed")
|
||||
assert.equal(parent.browserAutomationUnavailable, false, "native browser automation is available")
|
||||
await check("no preview", async () => { await assert.rejects(request("browser.probe"), /No visible/ ) })
|
||||
await check("autonomous open and redirect", async () => {
|
||||
assert.equal((await execute("open", `${base}/redirect`)).url, `${base}/ok`)
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ const [base,output]=process.argv.slice(2),request=(method:string,input:object={}
|
|||
const probe=(target?:string)=>request('fixture.mobile-probe',{target})
|
||||
async function until(check:(v:any)=>boolean,target?:string){const end=Date.now()+8000;let last;do{try{last=await probe(target);if(check(last))return last}catch{}await new Promise(r=>setTimeout(r,40))}while(Date.now()<end);throw new Error('Mobile observation did not settle: '+JSON.stringify(last))}
|
||||
async function main(){
|
||||
assert.equal(parent.browserAutomationUnavailable,true)
|
||||
assert.equal(parent.browserAutomationUnavailable,false)
|
||||
await request('browser.execute',{command:{action:'open',url:base+'/mobile'}})
|
||||
await request('fixture.mobile-init',{url:base+'/mobile'})
|
||||
const primary=await until(v=>v.ua===v.httpUA,'primary'),other=await until(v=>v.ua===v.httpUA,'other'),baseline=await until(v=>v.ua===v.httpUA)
|
||||
|
|
|
|||
|
|
@ -27,7 +27,7 @@ async function lease() {
|
|||
}
|
||||
}
|
||||
async function main() {
|
||||
assert.equal(parent.browserAutomationUnavailable,true)
|
||||
assert.equal(parent.browserAutomationUnavailable,false)
|
||||
await request("browser.execute",{command:{action:"open",url:`${base}/read/preview`}})
|
||||
for(const name of ["preview","forms","edges","inputs","advanced"]) {
|
||||
await navigate(`${base}/read/${name}`)
|
||||
|
|
|
|||
|
|
@ -944,7 +944,7 @@ export const BrowserFrame: Component<BrowserFrameProps> = (props) => {
|
|||
aria-label={option.getLabel(props.labels)}
|
||||
disabled={nativeMode() && capabilities.emulation && emulationProfile(option.id) !== "none" && !emulationReady()}
|
||||
title={emulationProfile(option.id) !== "none"
|
||||
? t(nativeMode() && webkitEmulation ? "browser.viewport.webkitEmulationHint" : nativeMode() && capabilities.emulation ? "browser.viewport.emulationHint" : "browser.viewport.emulationUnavailable")
|
||||
? t(nativeMode() && webkitEmulation ? "browser.viewport.webkitEmulationHint" : nativeMode() && capabilities.emulation ? "browser.viewport.emulationHint" : "browser.viewport.dimensionsOnly")
|
||||
: option.getLabel(props.labels)}
|
||||
onClick={() => void selectViewport(option.id)}
|
||||
>
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "Mobile WebKit-Kennung und CSS-Viewport. Pixeldichte und Eingabe unterliegen nativen Grenzen.",
|
||||
"browser.viewport.emulationHint": "Lädt mit mobiler Kennung, Touch und Pixeldichte neu. Browser-Engine, Tastatur und Systemleisten werden nicht simuliert.",
|
||||
"browser.viewport.emulationUnavailable": "Nur Abmessungen. Geräteemulation ist in dieser Vorschau nicht verfügbar.",
|
||||
"browser.viewport.dimensionsOnly": "Nur Abmessungen.",
|
||||
"sessionList.filter.includeMainSessions": "Hauptsitzungen",
|
||||
"sessionList.filter.includeSubsessions": "Untersitzungen",
|
||||
"session.pruning.maintenance_required": "Die Sitzung oder ihr Speicher ist beschäftigt. Warte, bis der aktuelle Vorgang abgeschlossen ist, und versuche die Bereinigung erneut.",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "WebKit mobile identity and CSS viewport. Native density and input limits apply.",
|
||||
"browser.viewport.emulationHint": "Reloads with mobile identity, touch and pixel density. Browser engine, keyboard and system bars are not simulated.",
|
||||
"browser.viewport.emulationUnavailable": "Dimensions only. Device emulation is unavailable in this preview.",
|
||||
"browser.viewport.dimensionsOnly": "Dimensions only.",
|
||||
"sessionList.filter.includeMainSessions": "Main sessions",
|
||||
"sessionList.filter.includeSubsessions": "Subsessions",
|
||||
"session.pruning.maintenance_required": "The session or its storage is busy. Wait for the current operation to finish, then retry cleanup.",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "Identidad móvil WebKit y viewport CSS. Se aplican límites nativos de densidad y entrada.",
|
||||
"browser.viewport.emulationHint": "Recarga con identidad móvil, entrada táctil y densidad de píxeles. No simula el motor del navegador, el teclado ni las barras del sistema.",
|
||||
"browser.viewport.emulationUnavailable": "Solo dimensiones. La emulación de dispositivos no está disponible en esta vista previa.",
|
||||
"browser.viewport.dimensionsOnly": "Solo dimensiones.",
|
||||
"sessionList.filter.includeMainSessions": "Sesiones principales",
|
||||
"sessionList.filter.includeSubsessions": "Subsesiones",
|
||||
"session.pruning.maintenance_required": "La sesión o su almacenamiento está ocupado. Espera a que termine la operación actual y vuelve a intentar la limpieza.",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "Identité mobile WebKit et viewport CSS. Densité et saisie soumises aux limites natives.",
|
||||
"browser.viewport.emulationHint": "Recharge avec identité mobile, tactile et densité de pixels. Le moteur du navigateur, le clavier et les barres système ne sont pas simulés.",
|
||||
"browser.viewport.emulationUnavailable": "Dimensions uniquement. L’émulation d’appareil est indisponible dans cet aperçu.",
|
||||
"browser.viewport.dimensionsOnly": "Dimensions uniquement.",
|
||||
"sessionList.filter.includeMainSessions": "Sessions principales",
|
||||
"sessionList.filter.includeSubsessions": "Sous-sessions",
|
||||
"session.pruning.maintenance_required": "La session ou son stockage est occupé. Attendez la fin de l’opération en cours, puis réessayez le nettoyage.",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "זיהוי נייד של WebKit ואזור תצוגה ב־CSS. חלות מגבלות מקוריות על צפיפות פיקסלים וקלט.",
|
||||
"browser.viewport.emulationHint": "טוען מחדש עם זיהוי נייד, מגע וצפיפות פיקסלים. מנוע הדפדפן, המקלדת וסרגלי המערכת אינם מדומים.",
|
||||
"browser.viewport.emulationUnavailable": "מידות בלבד. הדמיית מכשיר אינה זמינה בתצוגה מקדימה זו.",
|
||||
"browser.viewport.dimensionsOnly": "מידות בלבד.",
|
||||
"sessionList.filter.includeMainSessions": "סשנים ראשיים",
|
||||
"sessionList.filter.includeSubsessions": "תת־סשנים",
|
||||
"session.pruning.maintenance_required": "הסשן או האחסון שלו עסוקים. יש להמתין לסיום הפעולה הנוכחית ולנסות לנקות שוב.",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "WebKitのモバイル識別とCSSビューポート。画素密度と入力にはネイティブ環境の制限があります。",
|
||||
"browser.viewport.emulationHint": "モバイル識別情報、タッチ、ピクセル密度を設定して再読み込みします。ブラウザーエンジン、キーボード、システムバーは再現しません。",
|
||||
"browser.viewport.emulationUnavailable": "サイズのみ。このプレビューではデバイスのエミュレーションは利用できません。",
|
||||
"browser.viewport.dimensionsOnly": "サイズのみ。",
|
||||
"sessionList.filter.includeMainSessions": "メインセッション",
|
||||
"sessionList.filter.includeSubsessions": "サブセッション",
|
||||
"session.pruning.maintenance_required": "セッションまたはストレージが使用中です。現在の処理が完了してから、クリーンアップを再試行してください。",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "WebKit मोबाइल पहिचान र CSS भ्युपोर्ट। पिक्सेल घनत्व र इनपुटमा नेटिभ सीमाहरू लागू हुन्छन्।",
|
||||
"browser.viewport.emulationHint": "मोबाइल पहिचान, टच र पिक्सेल घनत्वसहित पुनः लोड गर्छ। ब्राउजर इन्जिन, किबोर्ड र प्रणाली बारहरू अनुकरण गरिँदैनन्।",
|
||||
"browser.viewport.emulationUnavailable": "आयाम मात्र। यस पूर्वावलोकनमा उपकरण अनुकरण उपलब्ध छैन।",
|
||||
"browser.viewport.dimensionsOnly": "आयाम मात्र।",
|
||||
"sessionList.filter.includeMainSessions": "मुख्य सत्रहरू",
|
||||
"sessionList.filter.includeSubsessions": "उप-सत्रहरू",
|
||||
"session.pruning.maintenance_required": "सत्र वा यसको भण्डारण व्यस्त छ। हालको कार्य सकिएपछि सफाइ फेरि प्रयास गर्नुहोस्।",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "Мобильная идентификация WebKit и CSS-область просмотра. Действуют нативные ограничения плотности пикселей и ввода.",
|
||||
"browser.viewport.emulationHint": "Перезагрузка с мобильной идентификацией, сенсорным вводом и плотностью пикселей. Движок браузера, клавиатура и системные панели не эмулируются.",
|
||||
"browser.viewport.emulationUnavailable": "Только размеры. Эмуляция устройства недоступна в этом предпросмотре.",
|
||||
"browser.viewport.dimensionsOnly": "Только размеры.",
|
||||
"sessionList.filter.includeMainSessions": "Основные сессии",
|
||||
"sessionList.filter.includeSubsessions": "Подсессии",
|
||||
"session.pruning.maintenance_required": "Сессия или её хранилище заняты. Дождитесь завершения текущей операции и повторите очистку.",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "WebKit mobil kimliği ve CSS görünümü. Yerel piksel yoğunluğu ve giriş sınırlamaları geçerlidir.",
|
||||
"browser.viewport.emulationHint": "Mobil kimlik, dokunma ve piksel yoğunluğuyla yeniden yükler. Tarayıcı motoru, klavye ve sistem çubukları simüle edilmez.",
|
||||
"browser.viewport.emulationUnavailable": "Yalnızca boyutlar. Bu önizlemede cihaz emülasyonu kullanılamıyor.",
|
||||
"browser.viewport.dimensionsOnly": "Yalnızca boyutlar.",
|
||||
"sessionList.filter.includeMainSessions": "Ana oturumlar",
|
||||
"sessionList.filter.includeSubsessions": "Alt oturumlar",
|
||||
"session.pruning.maintenance_required": "Oturum veya depolama alanı meşgul. Geçerli işlem bitince temizlemeyi yeniden deneyin.",
|
||||
|
|
|
|||
|
|
@ -1,7 +1,7 @@
|
|||
export const sessionMessages = {
|
||||
"browser.viewport.webkitEmulationHint": "WebKit 移动标识和 CSS 视口。像素密度和输入受原生环境限制。",
|
||||
"browser.viewport.emulationHint": "使用移动设备标识、触控和像素密度重新加载。不模拟浏览器引擎、键盘或系统栏。",
|
||||
"browser.viewport.emulationUnavailable": "仅调整尺寸。此预览不支持设备模拟。",
|
||||
"browser.viewport.dimensionsOnly": "仅调整尺寸。",
|
||||
"sessionList.filter.includeMainSessions": "主会话",
|
||||
"sessionList.filter.includeSubsessions": "子会话",
|
||||
"session.pruning.maintenance_required": "会话或其存储正忙。请等待当前操作完成后重试清理。",
|
||||
|
|
|
|||
|
|
@ -3,14 +3,14 @@ import { describe, it } from "node:test"
|
|||
import { browserHostCapabilities, nativeBrowserHost, onNativeBrowserOpen, physicalBrowserBounds, selectBrowserOpenOwner } from "./browser.ts"
|
||||
|
||||
describe("native browser adapter", () => {
|
||||
it("subscribes to private WebKit open requests while public automation remains disabled", async () => {
|
||||
it("subscribes to WebKit open requests with native automation available", async () => {
|
||||
const previous = Object.getOwnPropertyDescriptor(globalThis, "window")
|
||||
const calls: string[] = []
|
||||
let dispatch: (event: { payload: unknown }) => void = () => {}
|
||||
try {
|
||||
Object.defineProperty(globalThis, "window", { configurable: true, value: {
|
||||
__CODENOMAD_RUNTIME_HOST__: "tauri", __CODENOMAD_WINDOW_CONTEXT__: "local",
|
||||
__CODENOMAD_BROWSER_CAPABILITIES__: { embedding: true, history: true, emulation: false, automation: false },
|
||||
__CODENOMAD_BROWSER_CAPABILITIES__: { embedding: true, history: true, emulation: true, automation: true, reason: "webkit" },
|
||||
__TAURI_INTERNALS__: {
|
||||
transformCallback: (callback: typeof dispatch) => { dispatch = callback; return 1 },
|
||||
invoke: async (command: string) => { calls.push(command); return 1 },
|
||||
|
|
@ -23,7 +23,7 @@ describe("native browser adapter", () => {
|
|||
const payload = { sessionID: "session", url: "https://example.com/", requestID: "request" }
|
||||
dispatch({ payload })
|
||||
assert.deepEqual(received, payload)
|
||||
assert.equal(browserHostCapabilities({ host: "tauri", windowContext: "local" }).automation, false)
|
||||
assert.equal(browserHostCapabilities({ host: "tauri", windowContext: "local" }).automation, true)
|
||||
await unsubscribe()
|
||||
} finally {
|
||||
if (previous) Object.defineProperty(globalThis, "window", previous)
|
||||
|
|
|
|||
|
|
@ -77,7 +77,7 @@ for (const platform of ["macos", "linux"]) {
|
|||
assert.equal(await options.count(), 6)
|
||||
const mobile = page.getByRole("menuitemradio", { name: /390 × 844/ })
|
||||
assert.equal(await mobile.isEnabled(), true)
|
||||
assert.equal(await mobile.getAttribute("title"), "Dimensions only. Device emulation is unavailable in this preview.")
|
||||
assert.equal(await mobile.getAttribute("title"), "Dimensions only.")
|
||||
await mobile.click()
|
||||
await page.evaluate(() => {
|
||||
const f = (window as any).nativeFixture
|
||||
|
|
|
|||
|
|
@ -58,7 +58,7 @@ window.__CODENOMAD_RUNTIME_HOST__ = host
|
|||
window.__CODENOMAD_WINDOW_CONTEXT__ = "local"
|
||||
window.__CODENOMAD_BROWSER_CAPABILITIES__ = { embedding: true, history: true,
|
||||
emulation: !new URLSearchParams(location.search).has("dimensionsOnly"),
|
||||
automation: !["macos", "linux"].includes(new URLSearchParams(location.search).get("platform") ?? ""),
|
||||
automation: true,
|
||||
...(["macos","linux"].includes(new URLSearchParams(location.search).get("platform")??"") ? {reason:"webkit" as const}: {}) }
|
||||
Object.assign(window, {
|
||||
nativeFixture,
|
||||
|
|
|
|||
154
scripts/qualify-native-browser-app.mjs
Normal file
154
scripts/qualify-native-browser-app.mjs
Normal file
|
|
@ -0,0 +1,154 @@
|
|||
// Real packaged app + backend + native parent, with a private OpenCode daemon and
|
||||
// deterministic provider producing ordinary codenomad.browser agent tool calls.
|
||||
import assert from "node:assert/strict"
|
||||
import { spawn, execFileSync } from "node:child_process"
|
||||
import { createServer } from "node:http"
|
||||
import { mkdir, readFile, writeFile, chmod, readdir } from "node:fs/promises"
|
||||
import { createHash, randomUUID } from "node:crypto"
|
||||
import os from "node:os"
|
||||
import path from "node:path"
|
||||
import { setTimeout as delay } from "node:timers/promises"
|
||||
import { tsImport } from "tsx/esm/api"
|
||||
|
||||
const [appPath, cli, outputPath] = process.argv.slice(2)
|
||||
if (![appPath, cli, outputPath].every(value => value && path.isAbsolute(value))) throw Error("Usage: node scripts/qualify-native-browser-app.mjs <app-executable> <opencode-executable> <output-directory>")
|
||||
const output = outputPath, root = path.join(output, "private"), project = path.join(root, "project")
|
||||
await mkdir(project, { recursive: true })
|
||||
const run = randomUUID().slice(0, 8), password = `p7-${randomUUID()}`, started = Date.now()
|
||||
let desired, issued, providerError, app, daemon, daemonOutput = "", appOutput = ""
|
||||
const tools = [], evidence = []
|
||||
const fixtureHtml = await readFile(new URL("../packages/tauri-app/tests/browser-parent/app.html", import.meta.url), "utf8")
|
||||
const fixture = createServer((req, res) => {
|
||||
if (req.url === "/download") {
|
||||
res.setHeader("Content-Disposition", `attachment; filename="p7-${run}.txt"`)
|
||||
res.end(`native-download-${run}`)
|
||||
} else if (req.url === "/oauth") {
|
||||
res.setHeader("Content-Type", "text/html")
|
||||
res.end("<title>OAuth fixture</title><script>opener.postMessage('oauth-complete',location.origin);setTimeout(()=>close(),300)</script>")
|
||||
} else { res.setHeader("Content-Type", "text/html"); res.end(fixtureHtml) }
|
||||
})
|
||||
const provider = createServer(async (req, res) => {
|
||||
try {
|
||||
let raw = ""; for await (const chunk of req) raw += chunk
|
||||
const body = JSON.parse(raw), catalog = (body.tools || []).map(tool => tool.function?.name)
|
||||
const command = catalog.includes("codenomad_browser") ? desired : undefined
|
||||
if (command) { desired = undefined; issued = true; tools.push(command) }
|
||||
if (!body.stream) { res.setHeader("Content-Type", "application/json"); res.end(JSON.stringify({ id: run, choices: [{ message: { role: "assistant", content: "Done" }, finish_reason: "stop" }] })); return }
|
||||
res.setHeader("Content-Type", "text/event-stream")
|
||||
const chunk = (delta, finish_reason = null) => res.write(`data: ${JSON.stringify({ id: run, object: "chat.completion.chunk", model: "fixture", choices: [{ index: 0, delta, finish_reason }] })}\n\n`)
|
||||
chunk({ role: "assistant" })
|
||||
if (command) chunk({ tool_calls: [{ index: 0, id: `p7-${tools.length}`, type: "function", function: { name: "codenomad_browser", arguments: JSON.stringify(command) } }] })
|
||||
else chunk({ content: "Done" })
|
||||
chunk({}, command ? "tool_calls" : "stop"); res.end("data: [DONE]\n\n")
|
||||
} catch (error) { providerError = error; res.destroy(error) }
|
||||
})
|
||||
await Promise.all([fixture, provider].map(server => new Promise(done => server.listen(0, "127.0.0.1", done))))
|
||||
const env = Object.fromEntries(Object.entries(process.env).filter(([key]) => !/^(OPENCODE_|CODENOMAD_)/.test(key)))
|
||||
for (const key of ["XDG_DATA_HOME", "XDG_CONFIG_HOME", "XDG_STATE_HOME", "XDG_CACHE_HOME"]) env[key] = path.join(root, key)
|
||||
Object.assign(env, { HOME: root, USERPROFILE: root, OPENCODE_TEST_HOME: root,
|
||||
OPENCODE_CONFIG_DIR: path.join(root, "config"), OPENCODE_DB: path.join(root, "fixture.db"), OPENCODE_SERVER_PASSWORD: password,
|
||||
OPENCODE_CONFIG_PROJECT_DISABLE: "1", OPENCODE_DISABLE_MODELS_FETCH: "1", OPENCODE_DISABLE_FFF: "1" })
|
||||
await mkdir(env.OPENCODE_CONFIG_DIR, { recursive: true })
|
||||
await writeFile(path.join(env.OPENCODE_CONFIG_DIR, "opencode.json"), "{}")
|
||||
env.OPENCODE_CONFIG_CONTENT = JSON.stringify({ model: "fixture/fixture", permissions: [{ action: "*", resource: "*", effect: "allow" }],
|
||||
providers: { fixture: { package: "@opencode/ai/providers/openai-compatible", settings: { baseURL: `http://127.0.0.1:${provider.address().port}/v1`, apiKey: "fixture" }, models: { fixture: {} } } } })
|
||||
async function until(check, timeout = 60000) {
|
||||
const end = Date.now() + timeout
|
||||
do { if (providerError) throw providerError; const value = await check(); if (value) return value; await delay(150) } while (Date.now() < end)
|
||||
throw Error("Qualification condition timed out")
|
||||
}
|
||||
function strings(value) {
|
||||
if (typeof value === "string") return value
|
||||
if (Array.isArray(value)) return value.map(strings).join("\n")
|
||||
return value && typeof value === "object" ? Object.values(value).map(strings).join("\n") : ""
|
||||
}
|
||||
try {
|
||||
daemon = spawn(cli, ["serve", "--hostname", "127.0.0.1", "--port", "0", "--print-logs"], { cwd: root, env, windowsHide: true })
|
||||
daemon.stdout.on("data", chunk => { daemonOutput += chunk }); daemon.stderr.on("data", chunk => { daemonOutput += chunk })
|
||||
await until(() => /http:\/\/127\.0\.0\.1:\d+/.test(daemonOutput))
|
||||
const baseUrl = daemonOutput.match(/http:\/\/127\.0\.0\.1:\d+/)[0]
|
||||
// The fixture CLI adapter pins lifecycle discovery to this retained private daemon.
|
||||
// All non-lifecycle commands run the real CLI under the same isolated environment.
|
||||
const wrapperModule = path.join(root, "cli.cjs"), wrapper = path.join(root, process.platform === "win32" ? "opencode.cmd" : "opencode")
|
||||
await writeFile(wrapperModule, `const {spawnSync}=require('node:child_process'); const a=process.argv.slice(2); if(a.join(' ')==='service status')console.log(${JSON.stringify(baseUrl)}); else if(a.join(' ')==='service get password')console.log(${JSON.stringify(password)}); else if(a[0]==='service'){console.error('Fixture lifecycle is externally owned');process.exit(1)}else {const r=spawnSync(${JSON.stringify(cli)},a,{env:${JSON.stringify(env)},stdio:'inherit',windowsHide:true});process.exit(r.status??1)}`)
|
||||
await writeFile(wrapper, process.platform === "win32" ? `@echo off\r\n"${process.execPath}" "${wrapperModule}" %*\r\n` : `#!/bin/sh\nexec '${process.execPath.replaceAll("'", "'\\''")}' '${wrapperModule.replaceAll("'", "'\\''")}' "$@"\n`)
|
||||
if (process.platform !== "win32") await chmod(wrapper, 0o755)
|
||||
const { OpenCodeCliService } = await tsImport("../packages/server/src/workspaces/opencode-cli-service.ts", import.meta.url)
|
||||
const { OpenCodeSharedService } = await tsImport("../packages/server/src/workspaces/opencode-service.ts", import.meta.url)
|
||||
const lifecycle = new OpenCodeCliService({ label: "Private app qualification", timeoutMs: 10000, command: args => ({ command: cli, args, options: {} }) },
|
||||
{ execFile: async (_file, args) => ({ stdout: args.join(" ") === "service get password" ? password : baseUrl, stderr: "" }) })
|
||||
const shared = new OpenCodeSharedService(), client = await shared.client({ kind: "lifecycle", identity: `p7-${run}`, lifecycle })
|
||||
const session = await client.session.create({ location: { directory: project }, title: "Native browser qualification" })
|
||||
const config = path.join(root, "config.yaml")
|
||||
await writeFile(config, JSON.stringify({ server: { opencodeBinary: wrapper }, preferences: { lastUsedBinary: wrapper, locale: "en" }, opencodeBinaries: [{ path: wrapper, lastUsed: Date.now() }] }))
|
||||
const appEnv = { ...process.env, CODENOMAD_PROFILE: `p7-${run}`, CLI_CONFIG: config }
|
||||
delete appEnv.VITE_DEV_SERVER_URL; delete appEnv.ELECTRON_RUN_AS_NODE
|
||||
app = spawn(appPath, [project], { cwd: path.dirname(appPath), env: appEnv, windowsHide: false })
|
||||
app.stdout.on("data", chunk => { appOutput += chunk }); app.stderr.on("data", chunk => { appOutput += chunk })
|
||||
await until(async () => {
|
||||
if (app.exitCode !== null) throw Error(`App exited ${app.exitCode}: ${appOutput.slice(-4000)}`)
|
||||
return (await client.plugin.list({ location: { directory: project } })).data.some(plugin => plugin.id === "codenomad.automation" && plugin.state.status === "active")
|
||||
}, 90000)
|
||||
async function call(command) {
|
||||
issued = false; desired = command
|
||||
await client.session.prompt({ sessionID: session.id, text: `Qualification step ${evidence.length + 1}. Use the browser tool.` })
|
||||
await client.session.wait({ sessionID: session.id }, { signal: AbortSignal.timeout(95000) })
|
||||
assert.ok(issued, "Native model tool catalog must invoke codenomad.browser")
|
||||
const messages = await client.message.list({ sessionID: session.id, limit: 100, order: "asc" })
|
||||
const tool = messages.data.flatMap(message => message.type === "assistant" ? message.content : [])
|
||||
.filter(part => part.type === "tool" && part.name.includes("browser")).at(-1)
|
||||
assert.equal(tool?.state.status, "completed", JSON.stringify(tool?.state))
|
||||
const result = tool.state.content
|
||||
evidence.push({ command, toolID: tool.id, status: tool.state.status, content: result })
|
||||
await writeFile(path.join(output, "tool-results.json"), JSON.stringify(evidence, null, 2))
|
||||
console.log(`PASS ${command.action}`)
|
||||
return result
|
||||
}
|
||||
const url = `http://127.0.0.1:${fixture.address().port}/`
|
||||
await call({ action: "open", url })
|
||||
let snapshot = strings(await call({ action: "snapshot" }))
|
||||
const ref = label => {
|
||||
const line = snapshot.split("\n").find(line => line.includes(`"${label}"`))
|
||||
assert.ok(line, `Missing ${label}: ${snapshot}`)
|
||||
return /\[([^\]]+)\]/.exec(line)[1]
|
||||
}
|
||||
await call({ action: "click", ref: ref("Increment") })
|
||||
await call({ action: "type", ref: ref("Name"), text: "Native parity" })
|
||||
await call({ action: "select", ref: ref("Plan"), values: ["Pro"] })
|
||||
await call({ action: "press", ref: ref("Name"), key: "Enter" })
|
||||
await call({ action: "hover", ref: ref("Hover target") })
|
||||
await call({ action: "drag", ref: ref("Drag source"), to: ref("Drop target") })
|
||||
await call({ action: "scroll", ref: ref("Scroll box"), direction: "down", amount: 150 })
|
||||
for (const text of ["Count 1", "Pressed Enter", "Hovered", "Dragged", "Scrolled"]) await call({ action: "wait", text })
|
||||
await call({ action: "click", ref: ref("OAuth popup") })
|
||||
await call({ action: "wait", text: "OAuth complete" })
|
||||
await call({ action: "click", ref: ref("Download fixture") })
|
||||
const downloads = path.join(os.homedir(), "Downloads")
|
||||
const download = await until(async () => (await readdir(downloads)).find(name => name.includes(`p7-${run}.txt`)))
|
||||
assert.equal(await readFile(path.join(downloads, download), "utf8"), `native-download-${run}`)
|
||||
const dialog = call({ action: "click", ref: ref("Native dialog") })
|
||||
await delay(1500)
|
||||
if (process.platform === "darwin") execFileSync("osascript", ["-e", `tell application "System Events" to tell (first process whose unix id is ${app.pid}) to set frontmost to true`, "-e", 'tell application "System Events" to key code 36'])
|
||||
else if (process.platform === "linux") {
|
||||
const windows = execFileSync("xdotool", ["search", "--onlyvisible", "--pid", String(app.pid)], { encoding: "utf8" }).trim().split("\n")
|
||||
execFileSync("xdotool", ["windowactivate", "--sync", windows.at(-1)])
|
||||
execFileSync("xdotool", ["key", "Return"])
|
||||
} else {
|
||||
execFileSync("pwsh", ["-NoProfile", "-Command", `$w=New-Object -ComObject WScript.Shell; $w.AppActivate(${app.pid}) | Out-Null; Start-Sleep -Milliseconds 150; $w.SendKeys('{ENTER}')`], { windowsHide: true })
|
||||
}
|
||||
await dialog
|
||||
await call({ action: "wait", text: "Dialog accepted" })
|
||||
snapshot = strings(await call({ action: "snapshot" }))
|
||||
assert.match(snapshot, /Native parity/)
|
||||
const image = strings(await call({ action: "screenshot" }))
|
||||
const png = /data:image\/png;base64,([A-Za-z0-9+/=]+)/.exec(image)
|
||||
if (png) await writeFile(path.join(output, "browser.png"), Buffer.from(png[1], "base64"))
|
||||
await writeFile(path.join(output, "result.json"), JSON.stringify({ passed: true, run, sessionID: session.id, project, appPath,
|
||||
appSHA256: createHash("sha256").update(await readFile(appPath)).digest("hex"), tools: tools.map(command => command.action), download, screenshot: !!png }, null, 2))
|
||||
await shared.shutdown()
|
||||
} finally {
|
||||
app?.kill(); daemon?.kill()
|
||||
await writeFile(path.join(output, "app.log"), appOutput)
|
||||
await writeFile(path.join(output, "opencode.log"), daemonOutput)
|
||||
await Promise.all([fixture, provider].map(server => new Promise(done => server.close(done))))
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue