Play a sound through a graph of gain, pitch and EQ effects #311

A sound could only be played as it was uploaded. A Sound FX asset is
a node graph like a shader: Input, effects, Output. Play sound has a
new input fx that sends the voice through it.

- New asset type soundfx ("Sound FX"), not placeable, with a graph
  and its own tab. The heading in the asset panel is "Sound effects",
  so it does not take the click meant for the + menu entry.
- Nodes in core/soundfx/nodes.ts, category sound, port type audio
  that only fits audio: Input, Output, Gain (dB), Pitch (semitones),
  EQ (80 Hz and 12 kHz shelves, peaks at 250 Hz, 1 kHz and 4 kHz).
- compileSoundFx walks back from the Output with the shared graph
  emitter. An input takes one edge, so the result is one line: a
  pitch and a list of gain and filter stages. A node without input
  or a graph without Output reports a problem on the node.
- Pitch is the playback rate of the source. It applies to the whole
  sound and also changes its length, the node description says so.
- The BehaviorSystem resolves fx and hands the line to the AudioSink.
  An fx that is missing or does not compile plays the sound dry.
- SoundLibrary wires source, stages, volume, destination in that
  order. stats().last now also names pitch and stage count.
- The tab header picks one of the project's sounds and plays it
  through the graph as it stands. Stop there ends only its own voice.

Tests:
- soundfx.test.ts (new, 11), sound-library (9, +2 for the wiring),
  node-catalog (+1 for fx), graph and asset-groups adjusted
- e2e 34: ALL PASSED - 26 checks (+5: + menu opens the tab, palette,
  2 effects, tab Play and game Play through the FX at -12 st)
- Gate on melektaus-atx: typecheck, lint, prettier clean, 239 of 240
  test files green. The red one was sound-library (its fake
  AudioContext had no playbackRate), fixed afterwards, gate not
  re-run in full.
- Counter-check: without the audio rule in portsCompatible the port
  test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-05 16:45:42 +02:00
1 parent 93c0a80b7d
commit de14f0f9e2
34 files changed
+899 -43

No files matched your search

+5 -2
View File
@@ -1,3 +1,5 @@
import type { FxChain } from '../../core/soundfx/nodes';
/**
* What the runtime needs of the outside world to make a noise – the counterpart of `Input` on the
* way out. The browser side of it is `viewport/SoundLibrary`; a test (and a run started before
@@ -11,9 +13,10 @@ export interface AudioSink {
* Starts one voice. `volume` is final (0–1) and every call overlaps whatever is already playing.
* `owner` is the behavior the sound belongs to: when that behavior is destroyed the voice goes
* with it, because a sound is something a thing in the world is doing, not a thing of its own.
* `''` means the voice has no owner and outlives whatever started it.
* `''` means the voice has no owner and outlives whatever started it. `fx` is the line of effects
* the voice passes on its way out (#311); `DRY` passes none.
*/
play(file: string, volume: number, loop: boolean, owner: string): void;
play(file: string, volume: number, loop: boolean, owner: string, fx?: FxChain): void;
/** Silences every voice of that file; `''` silences all of them. */
stop(file: string): void;
/** Silences every voice started by that owner – what destroying a behavior does. */
+9 -4
View File
@@ -15,6 +15,8 @@ import { soundFileOf } from '../../core/model/sound';
import { hasRenderLayer } from '../../core/properties/PropertySchema';
import { resolveAssetProps } from '../../core/properties/resolve';
import { loopsOf, volumeOf } from '../../core/properties/sounds';
import { soundFxBuildFor } from '../../core/soundfx/compile';
import { DRY } from '../../core/soundfx/nodes';
import type { Entity } from '../Entity';
import { PhysicsSystem } from '../physics/PhysicsSystem';
import type { World } from '../World';
@@ -162,8 +164,8 @@ export function behaviorContext(
// eslint-disable-next-line no-console
console.log(`[behavior ${entity.name}]`, message);
},
playSound: (assetId, volume, loop, detached) =>
playSound(run, assetId, volume, loop, detached ? '' : entity.id),
playSound: (assetId, volume, loop, detached, fx) =>
playSound(run, assetId, volume, loop, detached ? '' : entity.id, fx),
// An id nothing resolves stops nothing, rather than silencing the whole run by accident;
// only the empty id means "everything", which is what the node promises.
stopSound: (assetId) => {
@@ -214,7 +216,8 @@ export function behaviorContext(
/**
* What the graph asked for, as the sink understands it: the asset's own volume scaled by the
* node's, and its loop flag unless the node overrules it either way. The asset is what a sound
* *is* – a node that says nothing therefore plays it as the project describes it.
* *is* – a node that says nothing therefore plays it as the project describes it. An FX asset that
* does not compile plays the sound dry: a broken effect should not silence the game.
*/
function playSound(
{ project, audio }: ContextSources,
@@ -222,12 +225,14 @@ function playSound(
volume: number,
loop: boolean | undefined,
owner: string,
fxId: string,
): void {
const asset = findAssetOf(project, assetId, 'sound');
if (!asset?.file) return;
const props = resolveAssetProps(project, asset);
const level = Math.max(0, Math.min(1, volumeOf(props) * volume));
audio.play(asset.file, level, loop ?? loopsOf(props), owner);
const fx = fxId ? findAssetOf(project, fxId, 'soundfx') : undefined;
audio.play(asset.file, level, loop ?? loopsOf(props), owner, (fx && soundFxBuildFor(fx).compiled) || DRY);
}
/** Where a ray of that length ends; where it has no direction, where it starts. */