From 5bb078094c8eaa194f5bfcd385654f73473078db Mon Sep 17 00:00:00 2001 From: meleki Date: Sun, 18 Oct 2026 10:39:43 +0200 Subject: [PATCH] MCP: let an AI agent see the open editor (#429) New page MCP: setting up Claude Code with an agent token, switching the agent on in the editor, the tools and the rules. - API: the routes for agent tokens and the relay. - Architecture: the relay and the agent parts of the editor. - Development: the relay in deployment and CI. - Sidebar: link to MCP. --- API.md | 27 +++++++++++++- Architecture.md | 8 +++++ Development.md | 20 +++++++++-- MCP.md | 95 +++++++++++++++++++++++++++++++++++++++++++++++++ _Sidebar.md | 1 + 5 files changed, 147 insertions(+), 4 deletions(-) create mode 100644 MCP.md diff --git a/API.md b/API.md index 3891dd0..ba6aa87 100644 --- a/API.md +++ b/API.md @@ -41,6 +41,12 @@ All routes go through `api/index.php?r=…` (the site's nginx does not pass `PAT | POST | `pixels` | **raw cel bytes** → `{file,bytes}` (201) — the pixels of one layer on one frame of [pixel art](Pixel-Art); **422** for anything that is not a cel. No listing and no delete: a cel is found by the name in the project | | GET | `plugins` | `[{file,bytes,updatedAt}]` — every `.js` in `data/plugins/`, sorted by file name. Also without a session: a guest's demo may switch one on | | GET | `plugins/{file}` | the module source as `text/javascript` (`nosniff`, `no-cache`); 404 for anything but a bare file name there — see [Plugins](Plugins) | +| GET | `agent-tokens` | `[{id,name,createdAt,expiresAt,usedAt}]`, oldest first: the caller's agent tokens (#429), never a token itself. `usedAt` is null until an agent used it | +| POST | `agent-tokens` | `{name, days?}` → `{id,name,createdAt,expiresAt,usedAt,token}` (201). The one answer the token `fwa_…` is ever in. `days` is 1 to 365, 90 without. **422** for a name over 60 characters or a lifetime out of range, **409** for a 21st token | +| DELETE | `agent-tokens/{id}` | 204, the token stops working at the agent's next request; **404** for none of the caller's | +| POST | `agent/ticket` | → `{ticket, expiresIn}`: a ticket for the caller's tab to get onto the MCP relay with, good for 60 seconds and once | +| POST | `agent/verify` | `{token, address}` → `{user, role}`; **401** for an unknown, expired or revoked token, **429** while `address` is locked out. The relay's own: without its key the route answers **404** | +| POST | `agent/redeem` | `{ticket}` → `{user, role}`; **401** for an unknown, used or expired ticket. The relay's own, like `agent/verify` | | GET | `users` | `[{username,role,createdAt,projects:[{id,name,revision,updatedAt}]}]` — every account with its projects, never a password. Administrators only | | POST | `users` | `{username,password,role}` → `{username,role,createdAt}` (201). **409** for a taken name, **422** for a bad name, a password under 8 characters or an unknown role. Administrators only | | PATCH | `users/{name}` | `{username?,role?,password?}` → `{username,role,createdAt}`. Only what is sent changes, and everything is checked before anything is written. A new name moves the account's projects and signs it out everywhere; a new password ends its sessions (the caller's own stays). **404** unknown account, **409** taken name, **422** for the caller's own name or role and for demoting the last administrator. Administrators only | @@ -56,7 +62,7 @@ its owner (#272): an administrator gets **403** `{"error":"Only its owner may de also for a file without an owner. Snapshots and the release of the project are theirs to manage all the same. The `users` routes answer anybody but an administrator **403** `{"error":"Administrators only"}`. Cels and plugins are shared between accounts; uploading a cel -still takes a session. See [Accounts](Accounts). +still takes a session. A guest has no agent tokens and gets no ticket. See [Accounts](Accounts). ## What the server refuses (#427) @@ -87,6 +93,25 @@ as somebody signed out, as the owner, as another account and as an administrator route list at the top of `api/index.php` and fails for a route the matrix does not ask about, so a new route has to be put there with what it expects. +## Agent tokens and the MCP relay (#429) + +An agent such as Claude Code works in the editor through the MCP relay, which passes its tool calls +to the user's open tab (see [MCP](MCP)). The API decides who is who for it; the relay keeps no +accounts. + +- An **agent token** (`fwa_` and 64 hex digits) is kept as `data/agent-tokens/{sha256}.json` + (`api/lib/AgentTokens.php`, 0640), like a session: the file cannot be replayed as the token. It + opens the relay and nothing else, no route of the API takes it. It ends at its expiry, when its + owner revokes it, and with every login of the account when the password changes (an + administrator's reset included) or the account is renamed or deleted. +- A **ticket** is how a tab gets onto the relay. The session cookie belongs to the folder of the + API and never reaches the relay's path, so the tab asks `agent/ticket` with it and hands the + ticket over on its WebSocket. It lies in `data/agent-tickets/` for a minute and is taken once. +- `agent/verify` and `agent/redeem` are the relay's. It sends the key in `X-Relay-Key` that lies in + `data/mcp-relay.key` for both sides; without it the two routes are not there. A wrong token + counts as a wrong password to the brake above, against the agent's address, which the relay + passes along. + ## Demo copies and demo material (#428) A guest tries the editor without an account (see [Accounts](Accounts)), and the server writes diff --git a/Architecture.md b/Architecture.md index b577041..bea961e 100644 --- a/Architecture.md +++ b/Architecture.md @@ -335,6 +335,9 @@ src/editor/ App → Editor (layout + ticker), PlaySession (the run), render, sound, pattern, song, routing, bake, video, curve — `tools/` sees one flat object), textProbe (finding widgets by their text — the ground floor of the test hook, shared with the sign-in screen), auth/LoginScreen + auth/PasswordDialog + auth/loginRules + auth/FormStatus (the sign-in screen and the change-password dialog in the canvas, what either may complain about, and the warning line and busy state both show, #177/#182), session (what a guest or an account may do, `sessionAllows`, #428), guestStores (a guest's stores: they list what the server offers and refuse every write before a request is made, #428), autosave, + agent/ (#429: AgentLink, the tab's WebSocket to the MCP relay with its retries; agentView, what + `get_view` answers and the one line the agent is told when it changes, as plain functions; + agentTools, the tools of tools.json, which the relay reads as well; screenshot; AgentTokensDialog), polygonEdit (points of an outline: what is under the pointer, whether a point is inside, put / move / take away — scene view and Texture tab; and the same for a ring of a vector art shape, where a point may carry Bezier handles: `ringGripAt`, splitting a curved segment @@ -551,12 +554,17 @@ api/ PHP backend: index.php (routing and the sign-in gate; `ro lib/LoginThrottle.php (#427: five wrong passwords from one address, then 15 minutes of 429; data/sign-ins/), lib/Quota.php (#427: 1 GB per account for uploads, projects and snapshots, 413 for a write that would grow past it) + lib/AgentTokens.php (#429: agent tokens in data/agent-tokens/ and the tabs' one-minute tickets in + data/agent-tickets/, both under the hash of what they keep) lib/{VideoStore,VideoQueue,VideoFormat,VideoWorker}.php (#402: a video named after its upload, the queue in data/videos/queue/, the one target format as plain rules, the conversion with ffmpeg; run by tools/video-worker.php in the container of tools/video/ and in CI) data/projects/ persisted projects, data/textures/ + data/fonts/ + data/sounds/ + data/videos/ content-addressed uploads, data/plugins/ editor plugins, filled from plugins/ by the build and the seeder; which of them run is written in each project (#119) +tools/mcp/ the MCP relay (#429): relay (agents over Streamable HTTP, tabs over a WebSocket), websocket (the + server half of RFC 6455), api (its two questions to the API), serve (start from the environment), + compose.yml (the container feuerwerk-mcp on stein2) tests/ Vitest unit tests tools/ seed-*, e2e (steps in tools/e2e/, rig in tools/e2e/lib/), smoke-*, lib/, assets/ (the walk demo's sheet) ``` diff --git a/Development.md b/Development.md index 3eeb36d..a8e5c2b 100644 --- a/Development.md +++ b/Development.md @@ -22,6 +22,7 @@ php tests/php/files.test.php # uploaded files i php tests/php/videos.test.php # the video store, its queue and the worker (#402), with the # real ffmpeg: the CI image and feuerwerk-video have it php tests/php/limits.test.php # the brake on wrong passwords and the quota (#427) +php tests/php/agents.test.php # agent tokens and the relay's tickets (#429) php tests/php/rights.test.php # every route as signed out, owner, stranger and admin (#427): # starts the API under `php -S` on a scratch directory node tools/seed-e2e.mjs && node tools/e2e.mjs # 2088 editor checks, screenshots in /tmp/gm-e2e-*.png @@ -317,6 +318,17 @@ The video worker (`tools/video/compose.yml`) runs without network, without capab checkout read-only and only `data/videos/` writable, and with 2 GB of memory and 256 processes at most: ffmpeg reads what strangers upload. +The **MCP relay** (#429, see [MCP](MCP)) is a small Node service in the container `feuerwerk-mcp` +(`tools/mcp/compose.yml`, `docker compose -f tools/mcp/compose.yml up -d`). nginx passes the one +path `/feuerwerk/mcp` on to it, plain requests, event streams and the WebSocket upgrade alike, to +`172.16.0.1:8091` on the docker bridge's gateway, where nothing from outside reaches it. It is named +by address, so nginx starts while the relay is down; that path answers 502 then and nothing else +notices. The container sees only `tools/mcp/`, `src/editor/agent/` (for the tool list, which it +reads on every use) and `data/mcp-relay.key`, read-only, with no capabilities, 256 MB and 64 +processes. It asks the API through the web stack's port, with the key, and takes the agent's address +from nginx's `X-Real-IP`. A change to `tools/mcp/` needs `docker restart feuerwerk-mcp`; new tools +in `tools.json` need only the build. + ## Commit messages A commit message is read in a log viewer, in `git log --oneline` and in the Gitea history, and @@ -358,10 +370,12 @@ Every push runs the whole proof on the Gitea runner: `.gitea/workflows/ci.yml` i `./check.sh` and the PHP tests (`tests/php/`, #179 — this machine has no PHP, so they are not in `check.sh`; here they run in the web server's container), builds, then `tools/ci/run-e2e.sh` (the editor suite) and `tools/ci/run-smokes.sh` (the three games). Both scripts source `tools/ci/serve.sh`, which puts the checkout behind -`http://127.0.0.1:8080/feuerwerk/` with `php -S` — the build wires its assets absolutely under -`/feuerwerk/dist/`, so the path segment is not optional — and writes the empty project files the +`http://127.0.0.1:8080/feuerwerk/` with `php -S`, under the same path as on stein2, and writes the empty project files the seeders need (`PUT` does not create a project, `POST` hands out a random id). Every tool takes its -server from `FEUERWERK_BASE_URL`. +server from `FEUERWERK_BASE_URL`. The MCP relay (#429) stands in front of the same server on +`http://127.0.0.1:8081/feuerwerk/` (`FEUERWERK_MCP_BASE_URL`): it passes everything but its own +path on, so the editor and the relay share an origin there as they do behind nginx. Only step 71 +opens the editor on that port. The jobs run in the image from `tools/ci/Dockerfile`: chromium with the Vulkan drivers and `xvfb`, `php-cli`, `ffmpeg` for the video worker (#402, `serve.sh` starts it beside the server), diff --git a/MCP.md b/MCP.md new file mode 100644 index 0000000..d0001e3 --- /dev/null +++ b/MCP.md @@ -0,0 +1,95 @@ +An AI agent such as Claude Code can work in the editor (#429). It works **inside your open editor +tab**, not on the stored project: you see everything it does as it happens, and you take it back +with Ctrl+Z like your own work. It speaks MCP (the Model Context Protocol) to a relay on the +server, and the relay passes its tool calls to your tab. + +So far the agent sees what you see; changing the project comes next (see the issue). + +## Setting it up + +1. **Make a token.** Account menu (top right) → *Agent tokens…* → a name, a lifetime (30 days, + 90 days or a year) → *New token*. The dialog shows the token once, with *Copy token*, and the + line that sets it up in Claude Code, with *Copy command*: + + ``` + claude mcp add --transport http feuerwerk https://feuerwerk.nockal.com/mcp --header "Authorization: Bearer fwa_..." + ``` + + The server keeps only a hash of the token; close the dialog without copying it and it is gone, + make a new one. +2. **Switch the agent on** in the tab you want it to work in: account menu → *Agent*. A badge + appears beside the account menu and shows how many agents are connected. A click on the badge + switches the agent off again. +3. Ask Claude Code about the editor. Without a tab switched on, every tool answers that no editor + is connected and how to fix that. + +Open the editor at **https://feuerwerk.nockal.com/** for the agent. At `nockal.com/feuerwerk/` the +proxy in front of the server passes the tab's WebSocket messages only one way, and the badge stays +at *Agent offline*. The agent itself reaches the relay at either address. + +## Tokens + +- A token opens the relay for an agent of your account, **and nothing else**: no route of the API + takes it, so it cannot read or write a project behind the tab's back. +- *Agent tokens…* lists your tokens with the day each was made, expires and was last used, and + *Revoke* ends one at the agent's next request. +- Every token of an account ends when its password changes, also when an administrator sets it, + and when the account is renamed or deleted. +- An account has at most 20. Wrong tokens count as wrong passwords: five from one address within + 15 minutes lock that address out for 15 minutes. +- A guest (see [Accounts](Accounts)) has no tokens and no *Agent* switch. + +## One tab at a time + +The agent works in the tab of your account that switched it on last: switching it on in a second +tab switches it off in the first, which says so. The switch is kept for the browser tab, so a reload +(a deployment) comes back with the agent on, and signing out switches it off. If the connection to +the relay is lost, the tab tries again on its own, every few seconds at first and then every 30; +the badge says *Agent offline* meanwhile. + +## What the agent can do + +| Tool | What it does | +|---|---| +| `get_view` | What you are looking at: the project, the tab and the document it shows, the selection with names (objects, nodes by type, frames, keys ...), the camera of the view, whether the game is playing or paused, tabs in windows of their own, and the menu or dialog on top | +| `screenshot` | The whole editor as drawn, as a PNG, at most `maxWidth` pixels wide (1280 unless asked otherwise) | + +There is also a resource, `feuerwerk://view`, with the same answer as `get_view`. An agent that +subscribes to it hears when you switch the tab, the document or the selection, start or stop +playing, or open a dialog; moving the camera is no news. Not every client follows subscriptions, +so the relay also tells the agent with its next tool answer that the view has changed since it +last looked, in one line. + +## How it works + +``` +Claude Code --- MCP over HTTPS ---> /mcp ---> relay (tools/mcp/) <--- WebSocket --- your editor tab + | + +--- whose token, whose ticket? ---> api/index.php +``` + +- The **relay** (`tools/mcp/relay.mjs`) is a Node service without dependencies in the container + `feuerwerk-mcp`. The agent's side is MCP's *Streamable HTTP*: JSON-RPC by POST, an event stream by + GET for notifications, DELETE to end a session. Your tab's side is a WebSocket on the same path, + with an `Origin` check. +- It knows no accounts. Every request of an agent asks the API who its token belongs to, so a + revoked token stops at once. A tab gets onto it with a one-minute ticket the API gives it for its + session (the session cookie does not travel to the relay's path). +- The **tools live in the editor** (`src/editor/agent/`). The relay knows only their list, + `src/editor/agent/tools.json`, and passes a call to the tab of the token's account and the answer + back. A call to an account without a tab on gets "no editor connected" and never reaches another + account's tab. + +What is behind it on the server, and the routes it uses: [API](API), [Development](Development). + +## Tests + +- `tests/mcp/relay.test.ts`: the relay between a fake API, MCP clients and fake tabs, without a + browser: tokens refused and braked, sessions per account, tabs refused from another origin or + without a ticket, one tab per account, timeouts, the view resource and its notifications. +- `tests/php/agents.test.php`: tokens and tickets on disk. `tests/php/rights.test.php` asks the new + routes as everybody, with and without the relay's key. +- `tests/editor/agent-view.test.ts`: what `get_view` answers, its one line, the tools' arguments. +- e2e step 71 (`tools/e2e/71-mcp.mjs`): a token made in the dialog, the agent switched on in the + account menu, an MCP client reading the view, hearing a tab switch and taking a screenshot, + and the badge switching it off. diff --git a/_Sidebar.md b/_Sidebar.md index 6c50b8d..da55ea4 100644 --- a/_Sidebar.md +++ b/_Sidebar.md @@ -2,6 +2,7 @@ - [Concepts](Concepts) - [Accounts](Accounts) +- [Agents (MCP)](MCP) - [Scenes](Scenes) - [Scene editing](Scene-editing) - [Behaviors](Behaviors)