add interactive browser
This commit is contained in:
@@ -0,0 +1 @@
|
||||
node_modules/
|
||||
@@ -0,0 +1,310 @@
|
||||
---
|
||||
name: interactive-browser
|
||||
description: "Launch a visible Playwright browser and interact with web pages with human-in-the-loop assistance. Use when the user wants the agent to browse websites, fill forms, click buttons, or automate web tasks while the user watches and guides. The browser window is visible (not headless) and the agent takes screenshots to show the user the current page state."
|
||||
---
|
||||
|
||||
# Interactive Browser
|
||||
|
||||
Launch a visible Playwright browser and interact with web pages while the user watches and provides guidance. This is a **human-in-the-loop** browser automation skill — the agent performs actions but the user sees everything and directs what happens next.
|
||||
|
||||
## Setup (one-time)
|
||||
|
||||
The skill uses Playwright with Chromium. Ensure Playwright is available:
|
||||
|
||||
```bash
|
||||
npx playwright --version
|
||||
```
|
||||
|
||||
If Chromium is not installed, install it:
|
||||
|
||||
```bash
|
||||
npx playwright install chromium
|
||||
```
|
||||
|
||||
The skill installs its own Playwright dependency on first use.
|
||||
|
||||
## How It Works
|
||||
|
||||
The skill runs a lightweight HTTP server that keeps a visible browser open. The agent communicates with the server via `curl` commands. All screenshots are saved to a temp directory and can be displayed to the user.
|
||||
|
||||
## Starting the Browser Server
|
||||
|
||||
```bash
|
||||
# Kill any stale server on the port first
|
||||
kill $(lsof -ti:9876) 2>/dev/null; sleep 2
|
||||
|
||||
# Start detached with logging so it survives shell exit
|
||||
cd /var/home/ducoterra/.pi/agent/skills/interactive-browser && \
|
||||
node scripts/browser-server.js 9876 /tmp/pi-browser-screenshots > /tmp/browser-server.log 2>&1 &
|
||||
sleep 3
|
||||
|
||||
# ALWAYS verify it started
|
||||
curl -s -g -X POST "http://localhost:9876/status"
|
||||
```
|
||||
|
||||
The server runs on port **9876** by default. Screenshots are saved to the specified directory. Action logs go to `/tmp/browser-server.log` — check it whenever a request misbehaves.
|
||||
|
||||
> **Why the redirect?** A background process writing to a dead terminal can get SIGHUP/SIGPIPE and die silently. Always redirect output to a log file and verify with `/status` before use.
|
||||
|
||||
## Commands (via curl)
|
||||
|
||||
All commands use `POST` requests to `http://localhost:9876`.
|
||||
|
||||
**Always use `curl -g`** — selectors like `a[href='/user/login']` contain brackets that curl would otherwise interpret as URL globs (exit code 3: malformed URL).
|
||||
|
||||
Parameters can go in the query string OR the request body (JSON or url-encoded — body wins). For complex selectors/JS with special characters, prefer `--data-urlencode`:
|
||||
|
||||
```bash
|
||||
curl -s -g -X POST "http://localhost:9876/click" --data-urlencode "selector=button:has-text('Submit')"
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
| Param | Applies to | Description |
|
||||
|-------|-----------|-------------|
|
||||
| `url` | launch, go | Target URL |
|
||||
| `selector` | click, dblclick, fill, select, hover, get_text | CSS selector |
|
||||
| `value` | fill, select | Text to fill / option value |
|
||||
| `expr` | evaluate | JavaScript expression to run in page |
|
||||
| `force` | click | `true` skips Playwright actionability checks (use when element is obscured or animating) |
|
||||
|
||||
### Launch & Navigate
|
||||
|
||||
```bash
|
||||
# Launch browser and navigate to a URL
|
||||
curl -s -g -X POST "http://localhost:9876/launch?url=https://example.com"
|
||||
|
||||
# Navigate to a different URL (browser stays open)
|
||||
curl -s -g -X POST "http://localhost:9876/go?url=https://other-site.com"
|
||||
|
||||
# Reload current page
|
||||
curl -s -g -X POST "http://localhost:9876/reload"
|
||||
```
|
||||
|
||||
### Interact
|
||||
|
||||
```bash
|
||||
# Click an element (add force=true if it's covered/animating)
|
||||
curl -s -g -X POST "http://localhost:9876/click?selector=#login-btn"
|
||||
|
||||
# Double-click
|
||||
curl -s -g -X POST "http://localhost:9876/dblclick?selector=.item"
|
||||
|
||||
# Fill a form field — use --data-urlencode for selectors with special chars
|
||||
curl -s -g -X POST "http://localhost:9876/fill" --data-urlencode "selector=input[name='email']" --data-urlencode "value=user@example.com"
|
||||
|
||||
# Select an option from a dropdown
|
||||
curl -s -g -X POST "http://localhost:9876/select" --data-urlencode "selector=select#country" --data-urlencode "value=US"
|
||||
|
||||
# Hover over an element
|
||||
curl -s -g -X POST "http://localhost:9876/hover?selector=.dropdown-trigger"
|
||||
```
|
||||
|
||||
### Inspect
|
||||
|
||||
```bash
|
||||
# Take a screenshot of the current page
|
||||
curl -s -g -X POST "http://localhost:9876/screenshot"
|
||||
|
||||
# Get text content of an element
|
||||
curl -s -g -X POST "http://localhost:9876/get_text?selector=#main-heading"
|
||||
|
||||
# Evaluate JavaScript on the page (always via --data-urlencode for complex expressions)
|
||||
curl -s -g -X POST "http://localhost:9876/evaluate" --data-urlencode "expr=document.title"
|
||||
```
|
||||
|
||||
### Control
|
||||
|
||||
```bash
|
||||
# Check browser status
|
||||
curl -s -g -X POST "http://localhost:9876/status"
|
||||
|
||||
# Close the browser
|
||||
curl -s -g -X POST "http://localhost:9876/close"
|
||||
```
|
||||
|
||||
## Output Format
|
||||
|
||||
Each command outputs JSON to stdout:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"command": "click",
|
||||
"selector": "#login-btn",
|
||||
"url": "https://example.com/page",
|
||||
"title": "Example Page",
|
||||
"screenshot": "/tmp/pi-browser-screenshots/click-2024-01-01T12-00-00.png"
|
||||
}
|
||||
```
|
||||
|
||||
On error:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "error",
|
||||
"message": "Server error",
|
||||
"details": "Target closed"
|
||||
}
|
||||
```
|
||||
|
||||
## Workflow: Human-in-the-Loop Interaction
|
||||
|
||||
This is the recommended pattern for interactive browser automation:
|
||||
|
||||
### 1. Start the Server
|
||||
|
||||
See [Starting the Browser Server](#starting-the-browser-server) — kill stale instances, start with log redirect, verify with `/status`.
|
||||
|
||||
### 2. Launch the Browser
|
||||
|
||||
```bash
|
||||
curl -s -g -X POST "http://localhost:9876/launch?url=https://target-site.com"
|
||||
```
|
||||
|
||||
### 3. Show the User the Page
|
||||
|
||||
Read the screenshot file from the JSON output and display it. Tell the user what you see and ask what they want to do.
|
||||
|
||||
```bash
|
||||
read /tmp/pi-browser-screenshots/launch-*.png
|
||||
```
|
||||
|
||||
### 4. Get User Instructions
|
||||
|
||||
The user looks at the visible browser window AND the screenshot, then tells you what to do:
|
||||
|
||||
> "Click the 'Sign In' button in the top right"
|
||||
> "Fill the email field with my address"
|
||||
> "Take a screenshot, I need to see what's on the page"
|
||||
|
||||
### 5. Execute the Action
|
||||
|
||||
Run the appropriate curl command:
|
||||
|
||||
```bash
|
||||
curl -s -g -X POST "http://localhost:9876/click?selector=.nav-signin"
|
||||
```
|
||||
|
||||
### 6. Show the Result
|
||||
|
||||
Read the new screenshot and show it to the user. Repeat from step 4.
|
||||
|
||||
### 7. Close When Done
|
||||
|
||||
```bash
|
||||
curl -s -g -X POST "http://localhost:9876/close"
|
||||
```
|
||||
|
||||
## Finding Selectors (Don't Guess)
|
||||
|
||||
**When a click fails with `Timeout ... waiting for locator(...)`, the selector doesn't match the page** — the site uses different markup than you assumed (e.g. Gitea's sign-in link is `a[href='/user/login']`, not `/user/sign_in`). Never guess hrefs. Enumerate elements first:
|
||||
|
||||
```bash
|
||||
# List all links with their text and href
|
||||
curl -s -g -X POST "http://localhost:9876/evaluate" \
|
||||
--data-urlencode "expr=Array.from(document.querySelectorAll('a')).map(function(a){return a.textContent.trim()+' -> '+a.getAttribute('href')}).join('\n')"
|
||||
|
||||
# List form fields
|
||||
curl -s -g -X POST "http://localhost:9876/evaluate" \
|
||||
--data-urlencode "expr=Array.from(document.querySelectorAll('input,textarea,select')).map(function(e){return e.tagName+' name='+e.getAttribute('name')+' type='+e.getAttribute('type')}).join('\n')"
|
||||
```
|
||||
|
||||
Selector tips:
|
||||
- Use CSS selectors: `#id`, `.class`, `tag`, `[attr="value"]`
|
||||
- For form fields: `input[type="email"]`, `textarea[name="message"]`, `select#country`
|
||||
- For precise targeting: `input[name="username"]` is more stable than text-matching
|
||||
- Use `force=true` when an element is found but covered/animating: `.../click?selector=X&force=true`
|
||||
|
||||
## Reading Page Content (DOM > Screenshots)
|
||||
|
||||
For extracting text (post lists, comments, tables), use `evaluate` on the DOM — it's exact, cheap, and scales to whole pages. Screenshots are for *showing the user state*, not for reading data.
|
||||
|
||||
```bash
|
||||
# Example: extract Reddit post titles + scores
|
||||
curl -s -g -X POST "http://localhost:9876/evaluate" \
|
||||
--data-urlencode "expr=Array.from(document.querySelectorAll('shreddit-post')).map(function(p){return p.getAttribute('score')+' pts | '+p.getAttribute('post-title')}).join('\n')"
|
||||
```
|
||||
|
||||
### Lazy-loaded / infinite-scroll pages
|
||||
|
||||
Pages like Reddit load content as you scroll. Scroll several times with pauses before extracting:
|
||||
|
||||
```bash
|
||||
for i in 1 2 3 4 5; do
|
||||
curl -s -g -X POST "http://localhost:9876/evaluate" --data-urlencode "expr=window.scrollBy(0, 1200)" > /dev/null
|
||||
sleep 0.8
|
||||
done
|
||||
# then run your extraction evaluate
|
||||
```
|
||||
|
||||
### Writing JS for `evaluate`
|
||||
|
||||
- Pass expressions via `--data-urlencode` — never hand-encode in the query string
|
||||
- Prefer `function(){}` over arrow functions, and IIFEs `(function(){...})()` for multi-statement code
|
||||
- Return a single string/number (the result is `String()`-ed); join multi-line output with `'\n'`
|
||||
|
||||
## Important Notes
|
||||
|
||||
- **Requires a display** — the browser is visible, so this needs X11/Wayland (or X forwarding / a virtual display). It will fail on a headless box with no display server.
|
||||
- **The browser is visible** — the user should see it open on their screen
|
||||
- **slowMo is set to 100ms** — actions have a slight delay so the user can follow along
|
||||
- **Viewport is 1280x900** — reasonably sized for most tasks
|
||||
- **Persistent session** — the browser keeps state (cookies, localStorage) between commands, so logged-in sessions persist
|
||||
- **One browser, one page** — the server drives a single page; no parallel tabs
|
||||
- **Secrets** — when possible, ask the user to type passwords themselves in the visible window rather than passing them through chat/commands
|
||||
- **Screenshots are saved** to the temp directory with timestamps — you can review them later
|
||||
- **Interaction timeouts are 8s** — failures return a JSON error fast; check the error's `details` for Playwright's call log
|
||||
- **Always close the browser** when done to free resources
|
||||
- **The server stays running** until explicitly closed — you can launch, do other work, then come back and continue interacting
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Symptom | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| curl exit 3 (URL malformed) | Selector contains `[...]` interpreted as curl glob | Add `-g` to every curl call |
|
||||
| curl exit 52 (empty reply) | Server dead | Restart it (see Starting the Browser Server), check `/tmp/browser-server.log` |
|
||||
| curl exit 28 (timeout) | Server request stuck | Should not happen anymore (8s internal timeouts); check the log |
|
||||
| `Timeout ... waiting for locator` | Selector doesn't match the page | Enumerate elements with `evaluate` (see Finding Selectors) |
|
||||
| Click succeeds but nothing happens | Element is covered or JS-gated | Retry with `force=true`, then screenshot to verify |
|
||||
| `Missing required parameter: X` | Param in URL but not decoded properly | Move it to the body with `--data-urlencode "X=..."` |
|
||||
| Stale page content after navigation | Page still loading / lazy content | `reload`, or scroll before extracting |
|
||||
| Server port busy | Old server instance | `kill $(lsof -ti:9876)` before starting |
|
||||
|
||||
## Example Session
|
||||
|
||||
```
|
||||
User: "Help me log into my GitHub account"
|
||||
|
||||
Agent: "I'll open a visible browser so you can see everything. Starting the server..."
|
||||
Agent: $ kill $(lsof -ti:9876) 2>/dev/null; sleep 2
|
||||
Agent: $ cd .../interactive-browser && node scripts/browser-server.js 9876 /tmp/pi-browser-screenshots > /tmp/browser-server.log 2>&1 &
|
||||
Agent: $ curl -s -g -X POST "http://localhost:9876/status" # verify it started
|
||||
Agent: $ curl -s -g -X POST "http://localhost:9876/launch?url=https://github.com/login"
|
||||
Agent: [reads screenshot, shows it to user]
|
||||
Agent: "I can see the GitHub login page. What would you like to do?"
|
||||
|
||||
User: "Fill in my email: ducoterra@example.com"
|
||||
|
||||
Agent: $ curl -s -g -X POST "http://localhost:9876/fill" --data-urlencode "selector=input[name='login']" --data-urlencode "value=ducoterra@example.com"
|
||||
Agent: [shows new screenshot]
|
||||
Agent: "Email filled in. What next?"
|
||||
|
||||
User: "Click the 'Continue' button"
|
||||
|
||||
Agent: $ curl -s -g -X POST "http://localhost:9876/click" --data-urlencode "selector=input[name='commit']"
|
||||
Agent: [shows new screenshot]
|
||||
Agent: "It's asking for a password. What should I do?"
|
||||
|
||||
User: "I'll type the password myself. You wait."
|
||||
|
||||
[User types password in the visible browser]
|
||||
|
||||
User: "Done, click Sign in"
|
||||
|
||||
Agent: $ curl -s -g -X POST "http://localhost:9876/click" --data-urlencode "selector=input[data-commit-button]"
|
||||
Agent: [shows new screenshot]
|
||||
Agent: "You're logged in! Welcome to your dashboard."
|
||||
|
||||
Agent: $ curl -s -g -X POST "http://localhost:9876/close"
|
||||
```
|
||||
Generated
+59
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"name": "interactive-browser",
|
||||
"version": "1.0.0",
|
||||
"lockfileVersion": 3,
|
||||
"requires": true,
|
||||
"packages": {
|
||||
"": {
|
||||
"name": "interactive-browser",
|
||||
"version": "1.0.0",
|
||||
"dependencies": {
|
||||
"playwright": "^1.62.1"
|
||||
}
|
||||
},
|
||||
"node_modules/fsevents": {
|
||||
"version": "2.3.2",
|
||||
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
|
||||
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
|
||||
"hasInstallScript": true,
|
||||
"license": "MIT",
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/playwright/-/playwright-1.62.1.tgz",
|
||||
"integrity": "sha512-0M+L3LAD8/nm554LOla9Ayx0j0tmFZ0FBcoQ7F1VuVHpM/XpiC8RcDzBQB8W5+hA8L22THxELzeF+2WcUzvcLg==",
|
||||
"license": "Apache-2.0",
|
||||
"dependencies": {
|
||||
"playwright-core": "1.62.1"
|
||||
},
|
||||
"bin": {
|
||||
"playwright": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
},
|
||||
"optionalDependencies": {
|
||||
"fsevents": "2.3.2"
|
||||
}
|
||||
},
|
||||
"node_modules/playwright-core": {
|
||||
"version": "1.62.1",
|
||||
"resolved": "https://registry.npmjs.org/playwright-core/-/playwright-core-1.62.1.tgz",
|
||||
"integrity": "sha512-wPYSwEBJY9GHraISXqyqtx0na0LpO3XEX7jNDhntbex7tzUS7kLnZsOlFruFJB4Hi/rhDMjXGqHewDZ68nYZVw==",
|
||||
"license": "Apache-2.0",
|
||||
"bin": {
|
||||
"playwright-core": "cli.js"
|
||||
},
|
||||
"engines": {
|
||||
"node": ">=20"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,14 @@
|
||||
{
|
||||
"name": "interactive-browser",
|
||||
"version": "1.0.0",
|
||||
"description": "",
|
||||
"main": "index.js",
|
||||
"scripts": {
|
||||
"test": "echo \"Error: no test specified\" && exit 1"
|
||||
},
|
||||
"keywords": [],
|
||||
"type": "commonjs",
|
||||
"dependencies": {
|
||||
"playwright": "^1.62.1"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,472 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* Interactive Browser Server
|
||||
*
|
||||
* A long-running HTTP server that keeps a visible Playwright browser open
|
||||
* and accepts commands via HTTP. Designed for human-in-the-loop interaction.
|
||||
*
|
||||
* Usage:
|
||||
* node browser-server.js [port]
|
||||
*
|
||||
* Default port: 9876
|
||||
*
|
||||
* API Endpoints:
|
||||
* POST /launch?url=<url> - Launch browser and navigate
|
||||
* POST /go?url=<url> - Navigate to URL
|
||||
* POST /reload - Reload current page
|
||||
* POST /screenshot - Take screenshot
|
||||
* POST /click?selector=<sel> - Click element
|
||||
* POST /dblclick?selector=<sel> - Double-click element
|
||||
* POST /fill?selector=<sel>&value=<val> - Fill form field
|
||||
* POST /select?selector=<sel>&value=<val> - Select dropdown option
|
||||
* POST /hover?selector=<sel> - Hover over element
|
||||
* POST /evaluate?expr=<js> - Evaluate JS expression
|
||||
* POST /get_text?selector=<sel> - Get element text
|
||||
* POST /close - Close browser
|
||||
*
|
||||
* All responses are JSON with screenshot paths.
|
||||
* Screenshots are saved to the specified output directory.
|
||||
*/
|
||||
|
||||
const { chromium } = require('playwright');
|
||||
const http = require('http');
|
||||
const { URL } = require('url');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
const os = require('os');
|
||||
|
||||
const PORT = parseInt(process.argv[2]) || 9876;
|
||||
const OUTPUT_DIR = process.argv[3] || path.join(os.tmpdir(), 'pi-browser-screenshots');
|
||||
const STATE_FILE = path.join(os.tmpdir(), 'pi-browser-state.json');
|
||||
|
||||
let browser = null;
|
||||
let context = null;
|
||||
let page = null;
|
||||
let wasClosed = false;
|
||||
|
||||
// Ensure output directory exists
|
||||
fs.mkdirSync(OUTPUT_DIR, { recursive: true });
|
||||
|
||||
// Never let an unhandled async error kill the server — log it instead.
|
||||
process.on('unhandledRejection', (err) => {
|
||||
console.error('[unhandledRejection]', err);
|
||||
});
|
||||
process.on('uncaughtException', (err) => {
|
||||
console.error('[uncaughtException]', err);
|
||||
});
|
||||
|
||||
async function ensureBrowser() {
|
||||
if (!browser) {
|
||||
browser = await chromium.launch({
|
||||
headless: false,
|
||||
slowMo: 100,
|
||||
args: [
|
||||
'--start-maximized',
|
||||
'--disable-blink-features=AutomationControlled',
|
||||
],
|
||||
});
|
||||
context = await browser.newContext({
|
||||
viewport: { width: 1280, height: 900 },
|
||||
locale: 'en-US',
|
||||
timezoneId: 'America/New_York',
|
||||
});
|
||||
page = await context.newPage();
|
||||
page.on('console', (msg) => {
|
||||
console.error(`[Browser Console] ${msg.type()}: ${msg.text()}`);
|
||||
});
|
||||
page.on('pageerror', (err) => {
|
||||
console.error(`[Browser Page Error] ${err.message}`);
|
||||
});
|
||||
wasClosed = false;
|
||||
}
|
||||
}
|
||||
|
||||
async function saveScreenshot(page, suffix = 'screenshot') {
|
||||
const timestamp = new Date().toISOString().replace(/[:.]/g, '-');
|
||||
const filename = `${suffix}-${timestamp}.png`;
|
||||
const filepath = path.join(OUTPUT_DIR, filename);
|
||||
await page.screenshot({ path: filepath, fullPage: false });
|
||||
return filepath;
|
||||
}
|
||||
|
||||
async function getPageInfo(page) {
|
||||
try {
|
||||
return {
|
||||
url: page.url(),
|
||||
title: await page.title(),
|
||||
};
|
||||
} catch {
|
||||
return { url: 'about:blank', title: '' };
|
||||
}
|
||||
}
|
||||
|
||||
function jsonResponse(res, statusCode, data) {
|
||||
res.writeHead(statusCode, { 'Content-Type': 'application/json' });
|
||||
res.end(JSON.stringify(data, null, 2));
|
||||
}
|
||||
|
||||
function errorResponse(res, statusCode, message, details) {
|
||||
console.error(`[Error] ${message}`);
|
||||
if (details) console.error(details);
|
||||
jsonResponse(res, statusCode, { status: 'error', message, details });
|
||||
}
|
||||
|
||||
function parseBody(bodyStr) {
|
||||
const params = new URLSearchParams();
|
||||
if (bodyStr) {
|
||||
try {
|
||||
// Try JSON first
|
||||
const obj = JSON.parse(bodyStr);
|
||||
for (const [k, v] of Object.entries(obj)) {
|
||||
params.set(k, v);
|
||||
}
|
||||
} catch {
|
||||
// Fall back to URL-encoded
|
||||
const sp = new URLSearchParams(bodyStr);
|
||||
for (const [k, v] of sp) params.set(k, v);
|
||||
}
|
||||
}
|
||||
return params;
|
||||
}
|
||||
|
||||
async function handleRequest(req, res) {
|
||||
const parsedUrl = new URL(req.url, `http://localhost:${PORT}`);
|
||||
const pathname = parsedUrl.pathname;
|
||||
const urlParams = parsedUrl.searchParams;
|
||||
|
||||
// CORS support
|
||||
res.setHeader('Access-Control-Allow-Origin', '*');
|
||||
res.setHeader('Access-Control-Allow-Methods', 'POST, OPTIONS');
|
||||
res.setHeader('Access-Control-Allow-Headers', 'Content-Type');
|
||||
|
||||
if (req.method === 'OPTIONS') {
|
||||
res.writeHead(204);
|
||||
res.end();
|
||||
return;
|
||||
}
|
||||
|
||||
if (req.method !== 'POST') {
|
||||
errorResponse(res, 405, 'Method not allowed. Use POST.');
|
||||
return;
|
||||
}
|
||||
|
||||
let body = '';
|
||||
for await (const chunk of req) {
|
||||
body += chunk;
|
||||
}
|
||||
|
||||
// Merge URL params and body params (body takes precedence)
|
||||
const bodyParams = parseBody(body);
|
||||
const params = new URLSearchParams();
|
||||
for (const [k, v] of urlParams) params.set(k, v);
|
||||
for (const [k, v] of bodyParams) params.set(k, v);
|
||||
|
||||
try {
|
||||
switch (pathname) {
|
||||
case '/launch': {
|
||||
const url = params.get('url');
|
||||
if (!url) {
|
||||
errorResponse(res, 400, 'Missing required parameter: url');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
console.error(`[Server] Navigating to: ${url}`);
|
||||
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
|
||||
await page.waitForTimeout(1000);
|
||||
const info = await getPageInfo(page);
|
||||
const screenshotPath = await saveScreenshot(page, 'launch');
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'launch',
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/go': {
|
||||
const url = params.get('url');
|
||||
if (!url) {
|
||||
errorResponse(res, 400, 'Missing required parameter: url');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
console.error(`[Server] Navigating to: ${url}`);
|
||||
await page.goto(url, { waitUntil: 'domcontentloaded', timeout: 30000 });
|
||||
await page.waitForTimeout(1000);
|
||||
const info = await getPageInfo(page);
|
||||
const screenshotPath = await saveScreenshot(page, 'go');
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'go',
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/reload': {
|
||||
await ensureBrowser();
|
||||
console.error('[Server] Reloading page...');
|
||||
await page.reload({ waitUntil: 'domcontentloaded', timeout: 30000 });
|
||||
await page.waitForTimeout(1000);
|
||||
const info = await getPageInfo(page);
|
||||
const screenshotPath = await saveScreenshot(page, 'reload');
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'reload',
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/screenshot': {
|
||||
await ensureBrowser();
|
||||
const screenshotPath = await saveScreenshot(page, 'screenshot');
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'screenshot',
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/click': {
|
||||
const selector = params.get('selector');
|
||||
if (!selector) {
|
||||
errorResponse(res, 400, 'Missing required parameter: selector');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
const force = params.get('force') === 'true';
|
||||
console.error(`[Server] Clicking: ${selector}${force ? ' (force)' : ''}`);
|
||||
await page.click(selector, { timeout: 8000, force });
|
||||
await page.waitForTimeout(500);
|
||||
const screenshotPath = await saveScreenshot(page, 'click');
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'click',
|
||||
selector,
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/dblclick': {
|
||||
const selector = params.get('selector');
|
||||
if (!selector) {
|
||||
errorResponse(res, 400, 'Missing required parameter: selector');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
console.error(`[Server] Double-clicking: ${selector}`);
|
||||
await page.dblclick(selector, { timeout: 8000 });
|
||||
await page.waitForTimeout(500);
|
||||
const screenshotPath = await saveScreenshot(page, 'dblclick');
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'dblclick',
|
||||
selector,
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/fill': {
|
||||
const selector = params.get('selector');
|
||||
const value = params.get('value');
|
||||
if (!selector || value === null) {
|
||||
errorResponse(res, 400, 'Missing required parameters: selector, value');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
console.error(`[Server] Filling "${value}" into: ${selector}`);
|
||||
await page.fill(selector, value, { timeout: 8000 });
|
||||
await page.waitForTimeout(300);
|
||||
const screenshotPath = await saveScreenshot(page, 'fill');
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'fill',
|
||||
selector,
|
||||
value,
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/select': {
|
||||
const selector = params.get('selector');
|
||||
const value = params.get('value');
|
||||
if (!selector || !value) {
|
||||
errorResponse(res, 400, 'Missing required parameters: selector, value');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
console.error(`[Server] Selecting "${value}" in: ${selector}`);
|
||||
await page.selectOption(selector, value, { timeout: 8000 });
|
||||
await page.waitForTimeout(300);
|
||||
const screenshotPath = await saveScreenshot(page, 'select');
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'select',
|
||||
selector,
|
||||
value,
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/hover': {
|
||||
const selector = params.get('selector');
|
||||
if (!selector) {
|
||||
errorResponse(res, 400, 'Missing required parameter: selector');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
console.error(`[Server] Hovering: ${selector}`);
|
||||
await page.hover(selector, { timeout: 8000 });
|
||||
await page.waitForTimeout(300);
|
||||
const screenshotPath = await saveScreenshot(page, 'hover');
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'hover',
|
||||
selector,
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/evaluate': {
|
||||
const expr = params.get('expr');
|
||||
if (!expr) {
|
||||
errorResponse(res, 400, 'Missing required parameter: expr');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
console.error(`[Server] Evaluating: ${expr}`);
|
||||
const result = await page.evaluate(expr);
|
||||
const screenshotPath = await saveScreenshot(page, 'evaluate');
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'evaluate',
|
||||
expression: expr,
|
||||
result: String(result),
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/get_text': {
|
||||
const selector = params.get('selector');
|
||||
if (!selector) {
|
||||
errorResponse(res, 400, 'Missing required parameter: selector');
|
||||
return;
|
||||
}
|
||||
await ensureBrowser();
|
||||
console.error(`[Server] Getting text from: ${selector}`);
|
||||
const text = await page.textContent(selector);
|
||||
const screenshotPath = await saveScreenshot(page, 'get_text');
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'get_text',
|
||||
selector,
|
||||
text: text || '',
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
screenshot: screenshotPath,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/close': {
|
||||
console.error('[Server] Closing browser...');
|
||||
if (browser) {
|
||||
await browser.close();
|
||||
}
|
||||
wasClosed = true;
|
||||
browser = null;
|
||||
context = null;
|
||||
page = null;
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
command: 'close',
|
||||
message: 'Browser closed',
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
case '/status': {
|
||||
if (wasClosed || !browser) {
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
state: 'idle',
|
||||
message: 'Browser is not running',
|
||||
});
|
||||
} else {
|
||||
const info = await getPageInfo(page);
|
||||
jsonResponse(res, 200, {
|
||||
status: 'ok',
|
||||
state: 'running',
|
||||
url: info.url,
|
||||
title: info.title,
|
||||
});
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
default:
|
||||
errorResponse(res, 404, `Unknown endpoint: ${pathname}`);
|
||||
}
|
||||
} catch (err) {
|
||||
errorResponse(res, 500, `Server error`, err.message);
|
||||
}
|
||||
}
|
||||
|
||||
const server = http.createServer(handleRequest);
|
||||
|
||||
server.listen(PORT, () => {
|
||||
console.error(`Interactive Browser Server running on http://localhost:${PORT}`);
|
||||
console.error(`Screenshots saved to: ${OUTPUT_DIR}`);
|
||||
console.error('Press Ctrl+C to stop');
|
||||
});
|
||||
|
||||
// Graceful shutdown
|
||||
process.on('SIGINT', async () => {
|
||||
console.error('\n[Server] Shutting down...');
|
||||
if (browser) await browser.close();
|
||||
server.close();
|
||||
process.exit(0);
|
||||
});
|
||||
|
||||
process.on('SIGTERM', async () => {
|
||||
console.error('\n[Server] Shutting down...');
|
||||
if (browser) await browser.close();
|
||||
server.close();
|
||||
process.exit(0);
|
||||
});
|
||||
Reference in New Issue
Block a user