adminandClaude Opus 5.5
1943539a7a
An agent could build a game object but not see whether it works. It had to ask the user to press the keys. Now it can play the game in the editor itself, send keys and mouse input and read what happened. - play_input sends steps into the running game: hold or tap a key, keep it down or let it go, click, press, release or move the mouse, turn the wheel, wait. Holding and waiting count game seconds. - A mouse target is a point in world units or an object by id or name. A click on an object reports whether it landed on it. - The answer names the game time used, the keys still held, the mouse and the new console lines. - One tool instead of press_keys and pointer from the plan. A step list can then mix both, like holding Shift during a click. - get_runtime reads the running game: objects with id, type and position, the own variables of their behaviors, the Variables assets, the camera, keys, mouse and the console. Named objects come with all their properties. - A hidden editor tab gets no frames. play_input says so at once. If game time stands still for 25 s it stops, names the step and lets go of every key it held. - get_view and play say when the scene has no active camera. The first agent test missed that and only saw it on a screenshot. - The relay instructions tell the agent to test what it built. Found in the live test on a real project: - A click on an object by its name reported hit false. The hit test looked the target up by id only. - get_runtime listed the built-in switches of a behavior (enabled, runInEditor) as its variables. Tests: - agent-play (new), agent-edit, agent-view, relay green. - 5 counter-checks red: hold counts one frame, key never released, click ignores the object, hidden tab not refused, no variable filter. - e2e 71: 21/21. The agent holds ArrowLeft 0.5 s (x -120 to -270) and clicks the object (hit true). - Gate: CHECK EXIT 0, 319 files, 4944 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
269 lines
13 KiB
JSON
269 lines
13 KiB
JSON
[
|
||
{
|
||
"name": "get_view",
|
||
"title": "What the user sees",
|
||
"description": "What the user is looking at in the editor right now: the project, the open tab and document, the selection (asset, scene objects, graph nodes, keys, ...), the camera of the view showing, whether the game is playing or paused, and an open menu or dialog. Call it first, and again whenever you were told the view changed.",
|
||
"inputSchema": { "type": "object", "properties": {}, "additionalProperties": false },
|
||
"annotations": { "readOnlyHint": true, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "screenshot",
|
||
"title": "Screenshot of the editor",
|
||
"description": "The whole editor as the user sees it, as a PNG: toolbar, panels, the open tab, menus and dialogs included. Scaled down to maxWidth pixels (default 1280) if it is wider.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": {
|
||
"maxWidth": {
|
||
"type": "integer",
|
||
"minimum": 320,
|
||
"maximum": 3840,
|
||
"description": "Widest the picture may be, in pixels."
|
||
}
|
||
},
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": true, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "list_assets",
|
||
"title": "List the assets",
|
||
"description": "The project's assets with their ids, names and types, and how many objects or nodes each holds. Filter by type and by a part of the name.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": {
|
||
"type": {
|
||
"type": "string",
|
||
"description": "Only assets of this type (scene, sprite, behavior, ...)."
|
||
},
|
||
"query": { "type": "string", "description": "Only assets whose name contains this." }
|
||
},
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": true, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "get_asset",
|
||
"title": "Read an asset",
|
||
"description": "One asset as stored: its property values, the rows of its properties (key, kind, default), the objects of a scene or container with their names and positions. A graph is only counted here; read it with get_graph.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": { "asset": { "type": "string", "description": "Id of the asset." } },
|
||
"required": ["asset"],
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": true, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "get_graph",
|
||
"title": "Read a graph",
|
||
"description": "The graph of a behavior, state machine, shader, particles, sound FX or instrument: each node with its settings and its input and output ports as they are now, the edges between ports, and the groups and notes.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": { "asset": { "type": "string", "description": "Id of the asset holding the graph." } },
|
||
"required": ["asset"],
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": true, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "lint_graph",
|
||
"title": "Check how a graph reads",
|
||
"description": "Counts what makes a graph hard to read, as the user sees it: nodes on top of each other (overlaps), links that cross (crossings), links running over a node that is not theirs (throughNodes), links running right to left (backwards), and the longest links, with a few ids of each. Aim for 0 overlaps and 0 throughNodes. To mend: the command arrange_graph, or a second Get or Owner node next to a far-off user instead of one long link.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": { "asset": { "type": "string", "description": "Id of the asset holding the graph." } },
|
||
"required": ["asset"],
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": true, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "catalog",
|
||
"title": "What can be made",
|
||
"description": "Looks things up before an edit. topic \"commands\": every command run_commands knows, with its arguments. topic \"nodes\" with asset: the nodes that graph can hold; with query, the best matches in full (ports, settings). topic \"asset_types\": the kinds of asset; with type, that kind's properties. topic \"variable_kinds\": what a variable can hold.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": {
|
||
"topic": { "type": "string", "enum": ["commands", "nodes", "asset_types", "variable_kinds"] },
|
||
"asset": { "type": "string", "description": "For nodes: id of the graph asset." },
|
||
"query": {
|
||
"type": "string",
|
||
"description": "For nodes: words to search for, like \"key held\" or \"move\"."
|
||
},
|
||
"type": { "type": "string", "description": "For asset_types: one type to describe in full." }
|
||
},
|
||
"required": ["topic"],
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": true, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "run_commands",
|
||
"title": "Change the project",
|
||
"description": "Runs commands (catalog topic \"commands\") in order as ONE undo step: if one fails, none of them stays, and the answer names the step. A step with \"as\": \"name\" lets later steps use what it made: \"$name\" is its id, \"$name.closing\" another id it names. Each step is built after the one before has run, so a step can add nodes to a behavior made in the same call. Deleting an asset asks the user first. Unless the user switched it off, the editor then shows what the call worked on last: the graph's tab with the nodes it touched selected, the scene (switching to it within the same undo step) or container with its objects selected, or the asset in the inspector.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": {
|
||
"steps": {
|
||
"type": "array",
|
||
"minItems": 1,
|
||
"items": {
|
||
"type": "object",
|
||
"properties": {
|
||
"command": { "type": "string", "minLength": 1, "description": "Name of the command." },
|
||
"args": { "type": "object", "description": "Its arguments, as the catalog lists them." },
|
||
"as": { "type": "string", "minLength": 1, "description": "A name for what this step makes." }
|
||
},
|
||
"required": ["command"],
|
||
"additionalProperties": false
|
||
}
|
||
},
|
||
"label": {
|
||
"type": "string",
|
||
"minLength": 1,
|
||
"description": "What the edit does, in a few words: the user sees it as the undo step."
|
||
}
|
||
},
|
||
"required": ["steps"],
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": {
|
||
"readOnlyHint": false,
|
||
"destructiveHint": true,
|
||
"idempotentHint": false,
|
||
"openWorldHint": false
|
||
}
|
||
},
|
||
{
|
||
"name": "open",
|
||
"title": "Show something",
|
||
"description": "Brings up an asset's tab (a scene becomes the scene being edited), or the scene or game tab. run_commands shows what it changed by itself; this is for anything else the user should see.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": {
|
||
"asset": { "type": "string", "description": "Id of the asset to open." },
|
||
"tab": { "type": "string", "enum": ["scene", "game"] }
|
||
},
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": false, "destructiveHint": false, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "select",
|
||
"title": "Select",
|
||
"description": "Selects an asset, placed objects of a scene or container, or nodes of a graph – what the inspector then shows. kind \"none\" clears the selection.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": {
|
||
"kind": { "type": "string", "enum": ["none", "asset", "objects", "nodes"] },
|
||
"asset": {
|
||
"type": "string",
|
||
"description": "The asset (kind asset), or the graph asset (kind nodes)."
|
||
},
|
||
"in": { "type": "string", "description": "Kind objects: the scene or container." },
|
||
"objects": { "type": "array", "items": { "type": "string" } },
|
||
"nodes": { "type": "array", "items": { "type": "string" } }
|
||
},
|
||
"required": ["kind"],
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": false, "destructiveHint": false, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "play",
|
||
"title": "Play the game",
|
||
"description": "Plays the game in the editor, pauses, resumes or stops it. Edits made while it runs change the running copy only and are gone when it stops.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": { "action": { "type": "string", "enum": ["play", "pause", "resume", "stop"] } },
|
||
"required": ["action"],
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": false, "destructiveHint": false, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "play_input",
|
||
"title": "Keys and mouse in the running game",
|
||
"description": "Plays keys and the mouse into the running game like a player, step by step; each step takes at least one frame, and hold and wait count game seconds. A step does one thing: {\"key\": \"ArrowLeft\", \"hold\": 0.5} holds a key; {\"key\": \"Space\"} taps it for one frame; {\"key\": \"Shift\", \"keep\": \"down\"} presses it and leaves it held after the call, \"keep\": \"up\" lets it go; {\"mouse\": \"click\", \"at\": {\"object\": \"<object id>\"}} or {\"mouse\": \"click\", \"at\": {\"x\": 10, \"y\": -20}} clicks the left button there (world units, as get_runtime gives them); \"down\", \"up\" and \"move\" do just that; {\"wheel\": -1} turns the wheel; {\"wait\": 1} lets the game run on. Keys go by their KeyboardEvent.key name: \"a\", \"ArrowUp\", \"Enter\", \"Space\". Answers the game time it took, the keys still held, where the mouse is, whether a click on an object landed on it, and the new console lines; read positions and variables with get_runtime. Needs the game playing, not paused, and the editor tab in front.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": {
|
||
"steps": {
|
||
"type": "array",
|
||
"minItems": 1,
|
||
"items": {
|
||
"type": "object",
|
||
"properties": {
|
||
"key": { "type": "string", "minLength": 1 },
|
||
"hold": {
|
||
"type": "number",
|
||
"minimum": 0,
|
||
"maximum": 10,
|
||
"description": "Seconds the key is held."
|
||
},
|
||
"keep": { "type": "string", "enum": ["down", "up"] },
|
||
"mouse": { "type": "string", "enum": ["click", "down", "up", "move"] },
|
||
"at": {
|
||
"type": "object",
|
||
"properties": {
|
||
"x": { "type": "number" },
|
||
"y": { "type": "number" },
|
||
"object": { "type": "string", "description": "An object id (or name) of the running game." }
|
||
},
|
||
"additionalProperties": false
|
||
},
|
||
"wheel": { "type": "integer", "minimum": -10, "maximum": 10 },
|
||
"wait": { "type": "number", "minimum": 0, "maximum": 10, "description": "Game seconds." }
|
||
},
|
||
"additionalProperties": false
|
||
}
|
||
}
|
||
},
|
||
"required": ["steps"],
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": false, "destructiveHint": false, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "get_runtime",
|
||
"title": "Read the running game",
|
||
"description": "The game as it runs now: game time, whether it is paused, the camera, the keys held and the mouse, every object with its id, type and position in world units (behaviors with their variables), the values of the Variables assets, and the last console lines. Name objects (ids or names) to get all their properties.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": {
|
||
"objects": { "type": "array", "items": { "type": "string" } },
|
||
"console": {
|
||
"type": "integer",
|
||
"minimum": 0,
|
||
"maximum": 500,
|
||
"description": "How many of the last console lines to give (default 30)."
|
||
}
|
||
},
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": true, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "undo",
|
||
"title": "Undo",
|
||
"description": "Takes back the last edits, the user's as well as yours, like Ctrl+Z; one run_commands call is one step.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": { "steps": { "type": "integer", "minimum": 1, "maximum": 20 } },
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": false, "destructiveHint": false, "openWorldHint": false }
|
||
},
|
||
{
|
||
"name": "redo",
|
||
"title": "Redo",
|
||
"description": "Does the edits again that undo took back, like Ctrl+Y.",
|
||
"inputSchema": {
|
||
"type": "object",
|
||
"properties": { "steps": { "type": "integer", "minimum": 1, "maximum": 20 } },
|
||
"additionalProperties": false
|
||
},
|
||
"annotations": { "readOnlyHint": false, "destructiveHint": false, "openWorldHint": false }
|
||
}
|
||
]
|