diff --git a/MCP.md b/MCP.md index c9f48a7..f8a61ef 100644 --- a/MCP.md +++ b/MCP.md @@ -65,7 +65,9 @@ the badge says *Agent offline* meanwhile. | `run_commands` | Changes the project, see below | | `open` | Brings up an asset's tab, or the scene or game tab | | `select` | Selects an asset, objects of a scene or container, or nodes of a graph | -| `play` | Play, pause, resume, stop | +| `play` | Play, pause, resume, stop. Says so when no camera is on, since the game then shows nothing | +| `play_input` | Keys and the mouse in the running game, step by step, see below | +| `get_runtime` | The running game: time, pause, camera, keys held, mouse, every object with its position (behaviors with their variables), the Variables assets, the last console lines | | `undo`, `redo` | Like Ctrl+Z and Ctrl+Y, up to 20 steps at once | There is also a resource, `feuerwerk://view`, with the same answer as `get_view`. An agent that @@ -122,6 +124,31 @@ A behavior acts on the object it is placed under. The smallest walking object is sprite, a behavior placed with the sprite as `parent`, and in the behavior an `OnKeyHeld` whose `exec` and `dt` feed a `Move`, with an `Owner` on its `object`. +## Playing the game + +The agent can test what it built without you pressing a key: `play`, then `play_input` to play +it, then `get_runtime` to see what happened. + +- **`play_input` takes steps, and each does one thing.** `{"key": "ArrowLeft", "hold": 0.5}` holds a + key, `{"key": "Space"}` taps it for one frame, `"keep": "down"` leaves a key held after the call + and `"keep": "up"` lets it go. `{"mouse": "click", "at": {"object": ""}}` clicks an object, + `"at": {"x": .., "y": ..}` a point in world units; `down`, `up` and `move` do just that. + `{"wheel": -1}` turns the wheel, `{"wait": 1}` lets the game run on. Keys go by their + `KeyboardEvent.key` name (`a`, `ArrowUp`, `Enter`), and `Space` is the space bar. +- **Time is the game's.** Every step takes at least one frame, so a tap is seen as a press and then + a release. Hold and wait count game seconds, at most 30 per call. A call over 5 seconds tells the + relay it takes longer. +- **What the answer tells:** the game time it took, the keys still held, where the mouse is, + whether a click on an object landed on it, and the new console lines. +- **A tapped or held key is let go again**, even when a step fails. Only `keep` leaves one down. +- **It needs the game playing, not paused, and the editor tab in front.** A tab in the background + runs no frames, so the call says so right away. If the game time stands still for 25 seconds + (paused by a behavior, the tab sent to the background), the call stops and names the step. +- **Your own mouse over the game view** still moves the game's mouse. While the agent clicks, leave + it outside the Game tab. +- `get_view` warns when the scene Play starts has no camera switched on, and while playing when the + game has none. + ## How it works ``` @@ -141,8 +168,8 @@ Claude Code --- MCP over HTTPS ---> /mcp ---> relay (tools/mcp/) <--- WebSocket `src/editor/agent/tools.json`, and passes a call to the tab of the token's account and the answer back. A call to an account without a tab on gets "no editor connected" and never reaches another account's tab. -- A call has 30 seconds. One that asks you something (a delete) tells the relay to wait, and then - has two minutes. +- A call has 30 seconds. One that asks you something (a delete) or plays the game for more than 5 + seconds tells the relay to wait, and then has two minutes. - `tools.json` holds each tool's argument schema; the editor checks every call against it (`argCheck.ts`). The commands and their schemas are in `commandCatalog.ts`, the all-or-nothing undo step in `editPlan.ts`, the reading tools in `readTools.ts`. @@ -166,11 +193,16 @@ What is behind it on the server, and the routes it uses: [API](API), [Developmen the arrow keys" (no overlaps, no link over a node, none backwards, at most 8 crossings with one Owner for four moves and 4 with one each), chains running straight, nodes that stay, groups kept clear, loops, and what `lint_graph` counts. +- `tests/editor/agent-play.test.ts`: a held key walks the object as far as the game seconds say, a + tapped one fires a key-down once, a kept one stays down across calls, a click on an object lands + and one beside it does not, a call that gives up when the game stands still and lets go of its key, + calls turned down before anything runs, `get_runtime`, and the camera hint. - `tests/editor/graph-framing.test.ts`: how far the camera goes after nodes: not at all when they are in sight, just far enough otherwise, zooming out only for what does not fit. - e2e step 71 (`tools/e2e/71-mcp.mjs`): a token made in the dialog, the agent switched on in the account menu, an MCP client reading the view, hearing a tab switch and taking a screenshot; the agent building a walking object whose graph comes up with its nodes selected and arranged, Arrange in the header untangling it as one undo step, the open tab's camera going after a node - added far away, the object moving when played, one Ctrl+Z taking it back, a delete waiting for + added far away, the object moving when played, the agent holding the arrow and clicking the + object itself and reading with `get_runtime` how far it went, one Ctrl+Z taking it back, a delete waiting for the user's Cancel; and the badge switching it off.