Let the agent write music and build compositions (#429)
feuerwerk / gate (push) Successful in 5m17s
feuerwerk / suite (push) Successful in 18m54s

The agent could already draw, animate and wire graphs, but songs,
patterns, sound clips and compositions were out of its reach. This
last step of #429 adds the two missing catalog areas, so the agent
can now use 168 of the editor's commands.

- Area music (15 commands): notes, pattern and song loops, song
  tracks and items, sends, and clips of sounds and videos.
- Area composition (9 commands): tracks, items for objects, images
  and audio, changes with fades, volume and bus, removal, and
  animation clips.
- A pitch is a MIDI number or a name like C4 or F#3. A note past the
  end of its pattern is refused.
- Song and master routing stay graphs. The agent links them with
  connect, and set_send sets level and tap. get_graph now shows the
  send of every link.
- Lengths follow the same rules as dragging in the tab. A sound has
  a length only after the editor loaded it. get_asset reports it as
  seconds, and without it the agent has to give a length.
- Moving a composition item takes its clips along, as dragging does.
  A clip that would then overlap another one is refused. Fades are
  only allowed on items with sound.
- get_asset reads notes back with their names and gives the length
  of a pattern in ticks.

Found while testing live: two song tracks in the same column made
lint_graph report a backwards link. arrange_graph cleared it, as the
agent instructions ask.

Tests:
- agent-music (new, 9 tests) and the other agent and relay tests:
  109/109.
- 9 mutations of the new code, each one red in its own test.
- e2e 71 on melektaus-atx: 25/25. New check: notes are read back by
  name, and a composition with a pattern and a sprite opens in its
  tab.
- Gate on melektaus-atx: CHECK EXIT 0, 323 files, 4987 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-18 17:32:44 +02:00
1 parent 0684e76dc5
commit 10f5e3ba8f
7 files changed
+1449 -2

No files matched your search

+53
View File
@@ -463,6 +463,59 @@ export async function mcp(t) {
}),
);
// 71g8. Music and compositions (#429 stage 5d): notes the agent writes, and the pattern and the
// sprite laid on a composition, which comes up in its tab.
const composed = await callTool('run_commands', {
label: 'compose',
steps: [
{ command: 'create_asset', args: { type: 'pattern', name: `${TOKEN_NAME} riff` }, as: 'riff' },
{
command: 'add_notes',
args: {
asset: '$riff',
notes: [
{ tick: 0, length: 480, note: 'C4' },
{ tick: 480, length: 480, note: 'E4' },
],
},
},
{ command: 'create_asset', args: { type: 'composition', name: `${TOKEN_NAME} cut` }, as: 'cut' },
{ command: 'add_composition_track', args: { asset: '$cut', name: 'Music' }, as: 'music' },
{
command: 'add_composition_item',
args: { asset: '$cut', track: '$music', place: '$riff', start: 0 },
},
{
command: 'add_composition_item',
args: { asset: '$cut', track: '$music', place: heroId, start: 30 },
},
],
});
await settle();
const composedMade = composed?.structuredContent?.made ?? [];
const riffId = composedMade.find((entry) => entry.step === 1)?.id;
const cutId = composedMade.find((entry) => entry.step === 3)?.id;
const cutView = await viewNow();
const riffRead = riffId ? (await callTool('get_asset', { asset: riffId }))?.structuredContent : null;
const cutRead = cutId ? (await callTool('get_asset', { asset: cutId }))?.structuredContent : null;
await callTool('undo');
await callTool('open', { asset: walkId });
await settle();
check(
'notes the agent writes read back by name, and a pattern and a sprite it lays on a composition open in its tab',
composed?.isError === false &&
cutView?.document?.asset?.id === cutId &&
JSON.stringify(riffRead?.notes?.map((note) => note.name)) === '["C4","E4"]' &&
JSON.stringify(cutRead?.items?.map((item) => [item.start, item.length])) === '[[0,480],[30,180]]' &&
cutRead?.objects?.length === 1,
JSON.stringify({
text: composed?.content?.[0]?.text?.slice(0, 300),
document: cutView?.document,
notes: riffRead?.notes,
items: cutRead?.items,
}),
);
// 71h. Played by the agent, the object walks left while the user holds the arrow.
const started = await callTool('play', { action: 'play' });
await waitUntil('__gm.state.mode === "play"', { timeout: 10000, label: 'the game runs' });