Play a sound through a graph of gain, pitch and EQ effects #311

A sound could only be played as it was uploaded. A Sound FX asset is
a node graph like a shader: Input, effects, Output. Play sound has a
new input fx that sends the voice through it.

- New asset type soundfx ("Sound FX"), not placeable, with a graph
  and its own tab. The heading in the asset panel is "Sound effects",
  so it does not take the click meant for the + menu entry.
- Nodes in core/soundfx/nodes.ts, category sound, port type audio
  that only fits audio: Input, Output, Gain (dB), Pitch (semitones),
  EQ (80 Hz and 12 kHz shelves, peaks at 250 Hz, 1 kHz and 4 kHz).
- compileSoundFx walks back from the Output with the shared graph
  emitter. An input takes one edge, so the result is one line: a
  pitch and a list of gain and filter stages. A node without input
  or a graph without Output reports a problem on the node.
- Pitch is the playback rate of the source. It applies to the whole
  sound and also changes its length, the node description says so.
- The BehaviorSystem resolves fx and hands the line to the AudioSink.
  An fx that is missing or does not compile plays the sound dry.
- SoundLibrary wires source, stages, volume, destination in that
  order. stats().last now also names pitch and stage count.
- The tab header picks one of the project's sounds and plays it
  through the graph as it stands. Stop there ends only its own voice.

Tests:
- soundfx.test.ts (new, 11), sound-library (9, +2 for the wiring),
  node-catalog (+1 for fx), graph and asset-groups adjusted
- e2e 34: ALL PASSED - 26 checks (+5: + menu opens the tab, palette,
  2 effects, tab Play and game Play through the FX at -12 st)
- Gate on melektaus-atx: typecheck, lint, prettier clean, 239 of 240
  test files green. The red one was sound-library (its fake
  AudioContext had no playbackRate), fixed afterwards, gate not
  re-run in full.
- Counter-check: without the audio rule in portsCompatible the port
  test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-05 16:45:42 +02:00
1 parent 93c0a80b7d
commit de14f0f9e2
34 files changed
+899 -43

No files matched your search

+4 -1
View File
@@ -28,6 +28,7 @@ import type { GraphAsset } from '../../core/model/Asset';
import { graphLookup } from '../../core/model/graphLookup';
import { particleLibrary } from '../../viewport/ParticleLibrary';
import { shaderLibrary } from '../../viewport/ShaderLibrary';
import { soundFxBuildFor } from '../../core/soundfx/compile';
import { findAsset, findGraphAsset } from '../../core/model/Project';
import { type Trace, TRACE_FADE } from '../../runtime/systems/BehaviorSystem';
import { ViewTween } from '../viewTween';
@@ -305,7 +306,9 @@ export class GraphCanvas extends Widget {
? shaderLibrary.checkFor(asset, this.state.project).problems
: asset.type === 'particles'
? particleLibrary.buildFor(asset).problems
: null;
: asset.type === 'soundfx'
? soundFxBuildFor(asset).problems
: null;
for (const node of asset.graph.nodes) {
const resolved = resolveDef(node, asset.graph, lookup);
if (!resolved) continue;
+4
View File
@@ -67,6 +67,8 @@ function categoryColor(category: NodeCategory): number {
return c.categoryShader;
case 'particles':
return c.categoryParticles;
case 'sound':
return c.categorySound;
}
}
@@ -91,6 +93,8 @@ export function portColor(type: PortType): number {
return c.portVec3;
case 'picture':
return c.portPicture;
case 'audio':
return c.portAudio;
case 'any':
return c.portAny;
}
+10 -2
View File
@@ -41,6 +41,7 @@ import {
import { nextParticlePropertyName, PARTICLE_NODES, PARTICLE_OUTPUT } from '../../core/particles/nodes';
import { offeredIn, ShaderFunctionInput, ShaderFunctionOutput } from '../../core/shader/functions';
import { SHADER_NODES, SHADER_OUTPUT, SHADER_VERTEX_OUTPUT } from '../../core/shader/nodes';
import { SOUND_FX_INPUT, SOUND_FX_NODES, SOUND_FX_OUTPUT } from '../../core/soundfx/nodes';
/**
* One row of the palette. `def` is the definition the node would actually have once placed, so the
@@ -125,7 +126,8 @@ function shaderEntries(project: Project, asset: ShaderAsset | ShaderFunctionAsse
/**
* The node types offerable in `asset`'s graph. A state machine holds states and nothing else, each
* new one named by its number; a shader holds shader nodes (`shaderEntries`). In a behavior, every other behavior appears under its own name instead of behind a
* new one named by its number; a shader holds shader nodes (`shaderEntries`), a sound FX its effects
* with one Input and one Output. In a behavior, every other behavior appears under its own name instead of behind a
* generic "Run behavior" – and once more as the event "On <name>" if it has an End to be heard
* through (#294) –, and the two terminals drop out once the graph has them.
*/
@@ -135,6 +137,12 @@ export function paletteEntries(project: Project, asset: GraphAsset): PaletteEntr
return [{ id: State.type, hint: State.category, def: State, params }];
}
if (asset.type === 'shader' || asset.type === 'shaderfunction') return shaderEntries(project, asset);
if (asset.type === 'soundfx') {
const placed = new Set(asset.graph.nodes.map((n) => n.type));
return SOUND_FX_NODES.filter(
(def) => !(def.type === SOUND_FX_INPUT || def.type === SOUND_FX_OUTPUT) || !placed.has(def.type),
).map((def) => ({ id: def.type, hint: def.category, def, params: {} }));
}
if (asset.type === 'particles') {
const hasOutput = asset.graph.nodes.some((n) => n.type === PARTICLE_OUTPUT);
const taken = new Set(asset.graph.nodes.filter(isParticlePropertyNode).map(propertyNameOf));
@@ -147,7 +155,7 @@ export function paletteEntries(project: Project, asset: GraphAsset): PaletteEntr
const entries: PaletteEntry[] = [];
for (const def of allNodeDefs()) {
if (namesBehavior(def.type) || def.category === 'state') continue;
if (def.category === 'shader' || def.category === 'particles') continue;
if (def.category === 'shader' || def.category === 'particles' || def.category === 'sound') continue;
if (isTerminalType(def.type) && terminalNode(asset.graph, def.type)) continue;
entries.push({ id: def.type, hint: def.category, def, params: {} });
if (def.type === FIND) entries.push(filterEntry(def));