Play a sound through a graph of gain, pitch and EQ effects #311

A sound could only be played as it was uploaded. A Sound FX asset is
a node graph like a shader: Input, effects, Output. Play sound has a
new input fx that sends the voice through it.

- New asset type soundfx ("Sound FX"), not placeable, with a graph
  and its own tab. The heading in the asset panel is "Sound effects",
  so it does not take the click meant for the + menu entry.
- Nodes in core/soundfx/nodes.ts, category sound, port type audio
  that only fits audio: Input, Output, Gain (dB), Pitch (semitones),
  EQ (80 Hz and 12 kHz shelves, peaks at 250 Hz, 1 kHz and 4 kHz).
- compileSoundFx walks back from the Output with the shared graph
  emitter. An input takes one edge, so the result is one line: a
  pitch and a list of gain and filter stages. A node without input
  or a graph without Output reports a problem on the node.
- Pitch is the playback rate of the source. It applies to the whole
  sound and also changes its length, the node description says so.
- The BehaviorSystem resolves fx and hands the line to the AudioSink.
  An fx that is missing or does not compile plays the sound dry.
- SoundLibrary wires source, stages, volume, destination in that
  order. stats().last now also names pitch and stage count.
- The tab header picks one of the project's sounds and plays it
  through the graph as it stands. Stop there ends only its own voice.

Tests:
- soundfx.test.ts (new, 11), sound-library (9, +2 for the wiring),
  node-catalog (+1 for fx), graph and asset-groups adjusted
- e2e 34: ALL PASSED - 26 checks (+5: + menu opens the tab, palette,
  2 effects, tab Play and game Play through the FX at -12 st)
- Gate on melektaus-atx: typecheck, lint, prettier clean, 239 of 240
  test files green. The red one was sound-library (its fake
  AudioContext had no playbackRate), fixed afterwards, gate not
  re-run in full.
- Counter-check: without the audio rule in portsCompatible the port
  test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-05 16:45:42 +02:00
1 parent 93c0a80b7d
commit de14f0f9e2
34 files changed
+899 -43

No files matched your search

+86
View File
@@ -270,6 +270,92 @@ export async function sound(t) {
);
await setDetached(false);
// 34g-3. A sound FX asset (#311): "+ → Sound FX" opens its tab, which says what keeps the graph
// from compiling; built into Input → Pitch → Gain → Output, the header plays the sound
// through it, and a `Play sound` naming it sends the game's voice the same way.
await showTab('scene');
await clickText('+');
await pickAssetType(t, 'Sound FX');
const fxCreated = await json(`{
tab: __gm.tab(),
current: __gm.soundfx.current(),
info: __gm.soundfx.info(),
preview: __gm.soundfx.preview(),
assets: __gm.state.project.assets.filter((a) => a.type === 'soundfx').map((a) => a.id),
}`);
check(
'the + menu creates a sound FX asset and opens its tab, with no Output to compile and no Play',
fxCreated.tab === 'soundfx' &&
fxCreated.assets.length === 1 &&
fxCreated.current === fxCreated.assets[0] &&
fxCreated.info === 'No Output node' &&
fxCreated.preview.sound === soundId &&
fxCreated.preview.play === false,
JSON.stringify(fxCreated),
);
const fxId = fxCreated.assets[0];
const fxCanvas = await json('__gm.soundfx.canvasRect()');
await click(fxCanvas.x + 80, fxCanvas.y + 60, 'right');
const fxPalette = await json('__gm.overlayTexts()');
check(
'its palette offers the sound effects and no behavior node',
['Input', 'Output', 'Gain', 'Pitch', 'EQ'].every((label) => fxPalette.includes(label)) &&
!fxPalette.includes('Branch'),
JSON.stringify(fxPalette.slice(0, 12)),
);
await press('Escape');
await mutate(`
const fx = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)});
fx.name = 'Low';
fx.graph = ${JSON.stringify({
nodes: [
node('fx-in', 'SoundFxInput', 0, 0),
node('fx-pitch', 'SoundFxPitch', 220, 0, { semitones: -12 }),
node('fx-gain', 'SoundFxGain', 440, 0, { gain: -6 }),
node('fx-out', 'SoundFxOutput', 660, 0),
],
edges: [
edge('fx-e1', 'fx-in', 'out', 'fx-pitch', 'in'),
edge('fx-e2', 'fx-pitch', 'out', 'fx-gain', 'in'),
edge('fx-e3', 'fx-gain', 'out', 'fx-out', 'in'),
],
})};`);
const fxBuilt = await json('{ info: __gm.soundfx.info(), preview: __gm.soundfx.preview() }');
check(
'Input → Pitch → Gain → Output compiles to two effects, and Play is offered',
fxBuilt.info === '2 effects' && fxBuilt.preview.play === true,
JSON.stringify(fxBuilt),
);
await evaluate('__gm.sound.stopAll()');
const beforeFx = await stats();
const fxPlayAt = await json('__gm.soundfx.playAt()');
await click(fxPlayAt.x, fxPlayAt.y);
const heardFx = await waitFor(stats, (s) => s.started > beforeFx.started);
check(
'the tab’s Play sends the sound through the graph: an octave down and one gain stage',
heardFx.last?.file === blip.file && heardFx.last.pitch === -12 && heardFx.last.stages === 1,
JSON.stringify(heardFx),
);
await evaluate('__gm.sound.stopAll()');
await showTab('scene');
await mutate(`
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
Object.assign(graph.nodes.find((n) => n.id === 'sn-play').params, { loop: 'false', fx: ${JSON.stringify(fxId)} });`);
const beforeGameFx = await stats();
await play();
const gameFx = await waitFor(stats, (s) => s.started > beforeGameFx.started);
check(
'a Play sound naming the asset plays the game’s voice through it',
gameFx.last?.pitch === -12 && gameFx.last.stages === 1,
JSON.stringify(gameFx),
);
await stop();
await mutate(`
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
Object.assign(graph.nodes.find((n) => n.id === 'sn-play').params, { loop: '', fx: '' });
const project = __gm.state.project;
project.assets = project.assets.filter((a) => a.id !== ${JSON.stringify(fxId)});`);
// 34h. Dropping a sound file onto the editor uploads it and makes an asset named after the file.
const dropped = await evaluate(`
(async () => {