From 9154ac85787080affe44a0008ac368b0248fea8f Mon Sep 17 00:00:00 2001 From: MelekTaus Date: Sun, 18 Oct 2026 17:32:52 +0200 Subject: [PATCH] MCP: music and composition areas (#429) --- MCP.md | 28 +++++++++++++++++++++++++--- 1 file changed, 25 insertions(+), 3 deletions(-) diff --git a/MCP.md b/MCP.md index e5f6db3..eb1d5b2 100644 --- a/MCP.md +++ b/MCP.md @@ -112,7 +112,7 @@ it. One Ctrl+Z takes the whole call back. answer, up to 110 seconds; Cancel, Escape or no answer leave everything as it was. - **You see what the agent works on.** After each call the editor brings up what its last step worked on: the graph's tab with every node the call touched there selected, the scene or - container with its objects selected, the tab of an animation, curve, texture, tile, tilemap, pixel art or vector art + container with its objects selected, the tab of an animation, curve, texture, tile, tilemap, pixel art, vector art, pattern, song, sound, video or composition (with the frame picked when the call worked on one), or the asset in the inspector. Objects in another scene switch the scene, inside the agent's undo step, so Ctrl+Z goes back to the scene you had. Account menu → *Follow agent* switches this off (kept in this browser); while the game runs the @@ -137,10 +137,12 @@ for an area or for command names asked for. | tiles | `add_tile_picture`, `remove_tile_picture`, `move_tile_picture`, `paint_tiles`, `add_tilemap_layer`, `duplicate_tilemap_layer`, `remove_tilemap_layer`, `move_tilemap_layer`, `set_tilemap_layer`, `resize_placed_tilemap` | | pixelart | `paint_pixels`, `add_pixel_layer`, `duplicate_pixel_layer`, `remove_pixel_layer`, `set_pixel_layer`, `move_pixel_layer`, `merge_pixel_layer_down`, `add_pixel_frame`, `remove_pixel_frame`, `move_pixel_frame`, `resize_pixel_canvas`, `set_palette_colors`, `remove_palette_color` | | vectorart | `add_vector_shape`, `set_vector_shape_outline`, `set_vector_shape_style`, `transform_vector_shapes`, `duplicate_vector_shape`, `remove_vector_shapes`, `combine_vector_shapes`, `rename_vector_shape`, `order_vector_shape`, `move_vector_shapes`, `group_vector_shapes`, `ungroup_vector_shapes`, `add_vector_layer`, `remove_vector_layer`, `move_vector_layer`, `set_vector_layer` | +| music | `add_notes`, `change_notes`, `remove_notes`, `set_loop`, `add_song_track`, `change_song_track`, `move_song_track`, `remove_song_track`, `add_song_item`, `change_song_item`, `remove_song_items`, `set_send`, `add_clip`, `change_clip`, `remove_clip` | +| composition | `add_composition_track`, `change_composition_track`, `move_composition_track`, `remove_composition_track`, `add_composition_item`, `change_composition_item`, `remove_composition_items`, `add_composition_clip`, `change_composition_clip` | | project | `rename_project`, `set_layer_names`, `set_play_from`, `set_while_playing`, `set_game_link_zoom`, `set_symbol_scale`, `set_master_volume`, `set_master_fx` | A change to the fields of a Struct asset reaches every variable and node of that type, as it does -from the inspector. Songs, patterns and compositions come next (#429, stage 5d). +from the inspector. - **Animations** name a row by its target and property, as `get_asset` lists them, and a key by its row and frame. An easing is a name like `sine-out` or `curve:` for a Curve asset; how @@ -160,6 +162,16 @@ from the inspector. Songs, patterns and compositions come next (#429, stage 5d). later with `set_vector_shape_style`; `paint` of type `solid` and an outline of width 0 take them away again. `transform_vector_shapes` moves, scales, turns and mirrors shapes and groups with their paints, round their middle unless `about` names a point. +- **Notes** are counted in ticks, 480 to a quarter note, and a pitch is a MIDI number or a name + like `C4` or `F#3`. `get_asset` reads a pattern's notes with their names and how many ticks it is + long; a note starting past its end is refused. The routing of a song and the master routing + (`#routing`) are graphs: links are drawn with `connect` and set with `set_send`. +- **Song and composition items** without a length are as long as their pattern, their sound or + video, or what they play, else 3 s; on a composition such an item keeps following it. How long a + sound lasts is only known once the editor has loaded it (`get_asset` says `seconds`), so a sound + not loaded yet needs a length. An object item's object is set with the commands of area objects, + in = the composition. Moving an item takes the clips that belong to it along, as dragging does, + and a clip that would come to lie over another is refused. Not for the agent, on purpose: textures, fonts, sounds and videos need a file, pasting needs the clipboard (the agent builds directly instead), and switching a plugin on would run its code. @@ -249,6 +261,15 @@ What is behind it on the server, and the routes it uses: [API](API), [Developmen out with its pixels, a big canvas only counted; an ellipse lying on its box all the way round, a rect turned round its middle, a gradient and an outline set and taken away, a shape moved on top of a group, and unknown shapes refused. +- `tests/editor/agent-music.test.ts`: notes added by number and by name and read back with their + names, a note past the end and a name there is none of refused, notes changed, removed and + replaced, a loop set and cleared; a pattern laid on a song as long as it is, a sound by its loaded + length and refused while not loaded, tracks added, changed, moved and removed (the last one kept) + with the routing following, sends of a song and of the master routing (a sound only after its + level); a clip of a sound added, changed and removed; an object, a picture and a pattern laid on a + composition, an animation and the composition itself refused as items, an item moved with its + clip, a clip over another refused, fades only on what sounds, an item removed with its object, and + the tracks of a composition. - `tests/editor/graph-arrange.test.ts`: the layout on the graph an agent really built for "walk with the arrow keys" (no overlaps, no link over a node, none backwards, at most 8 crossings with one Owner for four moves and 4 with one each), chains running straight, the flow on top when two @@ -265,6 +286,7 @@ What is behind it on the server, and the routes it uses: [API](API), [Developmen agent building a walking object whose graph comes up with its nodes selected and arranged, Arrange in the header untangling it as one undo step, the open tab's camera going after a node added far away, a Variables node with a port and a group framing every drawn node, an animation keyed by - the agent coming up in its tab, pixel art it paints read back as rows and its cel saved to the server, the object moving when played, the agent holding the arrow and clicking the + the agent coming up in its tab, pixel art it paints read back as rows and its cel saved to the server, notes it writes read back by + name and a pattern and a sprite it lays on a composition opening in its tab, the object moving when played, the agent holding the arrow and clicking the object itself and reading with `get_runtime` how far it went, one Ctrl+Z taking it back, a delete waiting for the user's Cancel; and the badge switching it off.