Change the sound FX parameters of a sound while it plays #315
Until now a value handed to a sound FX counted from the start of the voice only. An engine whose pitch follows the speed, or a loop that gets quieter, could not be built. Every play of Play sound now gives out the number of its voice, and a new node lets the knobs of that one voice glide to new values while it plays. - Play sound has a new output voice: the number of this one play, or 0 where nothing was played. Kept in a Number variable it reaches the voice from another event. Two plays of one sound are two voices. - New action node Set sound FX parameters (exec, voice, fx params). It reaches exactly the voice it is given, a detached one too. A voice that has ended, or voice 0, is not reached. - AudioSink.play returns the voice number and takes the plan and the values (FxVoice) instead of a finished chain. AudioSink.tune takes a voice number and new values. A name the voice's FX does not have is left out, one not given keeps its value. - SoundLibrary hands out the number at once, before the bytes are there, and never twice. A voice still waiting for its bytes starts with the values handed in meanwhile. - Every knob of a voice moves with setTargetAtTime (30 ms time constant). A knob whose value did not change is not sent again, so a tune every frame costs next to nothing. - A pitch on a playing voice glides by playbackRate. A new WSOLA copy in the middle of a sound is not worked out, so the voice gets shorter or longer while it is moved. On a loop that is not heard. - fxChainOf now gives a chain whose shape depends on the plan alone. A stage fed by a parameter stays at 0 dB, and a Mix fed by a parameter keeps both ways. A typed-in 0 or Mix at 0/1 is still left out by the compiler. - Every way of a voice has a source of its own, also at the same pitch, so each can glide on its own. The pitched copy is shared. - The tab keeps the numbers of what it plays, so moving a parameter field is heard at once. - Test hook __gm.sound.levels(file) reads the knobs of every voice of a file, oldest first, and stats() counts the voices tuned. Tests: - soundfx.test.ts: 29 tests (+2 for the node in a running world: the voice of the first of two plays of one sound, and a play that played nothing) - sound-library.test.ts: 15 tests (+3: glide, reach by number, pending voice) - node-catalog.test.ts: 370 tests (+2, Play sound checks its voice) - sounds + keep-on-scene-load: 31 tests, unchanged - Counter-test: a tune that reaches every voice of the same file makes the reach test fail. - e2e 34 (melektaus-atx): ALL PASSED - 30 checks (+1). In Chrome two looping voices of one sound start at depth 7. The one whose number sits in a variable glides to 0.7938 (2^(-4/12) = 0.7937), the other stays at 1. Both gains stay at 0.5012. - The gate ran before voices got a number (CHECK EXIT 0, 241 files, 3658 tests), not after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
072537b2e8
commit
c7e9769f90
18 files changed
+563
-106
No files matched your search
@@ -240,6 +240,7 @@ export interface NodeContext {
|
||||
* the one case where the rule is in the way: the death sound of an object that destroys itself.
|
||||
* `fx` names a sound FX asset the voice is played through (#311); `''` plays it as it is.
|
||||
* `fxParams` are values for its parameters; one not given is the default the FX declares.
|
||||
* Returns the number of the voice for `tuneSound`, or 0 where nothing is played.
|
||||
*/
|
||||
playSound(
|
||||
assetId: string,
|
||||
@@ -248,9 +249,14 @@ export interface NodeContext {
|
||||
detached: boolean,
|
||||
fx: string,
|
||||
fxParams: FxParams,
|
||||
): void;
|
||||
): number;
|
||||
/** Silences every voice of that sound; `''` silences everything this run started. */
|
||||
stopSound(assetId: string): void;
|
||||
/**
|
||||
* Hands new values to the sound FX parameters of one voice, by the number `playSound` gave it
|
||||
* (#315), so a running sound follows the game. A voice that has ended is not reached.
|
||||
*/
|
||||
tuneSound(voice: number, fxParams: FxParams): void;
|
||||
/**
|
||||
* Asks for another scene, by asset id. The swap happens once the frame is over – a graph must be
|
||||
* able to finish the flow it is in, and half a frame in the old world and half in the new one is
|
||||
|
||||
@@ -59,7 +59,9 @@ export const PlaySound: NodeDef = {
|
||||
'the node only colors it. Each play is a voice of its own: firing this twice in a row is heard ' +
|
||||
'twice, overlapping. The sound belongs to this object and stops when the object is destroyed, ' +
|
||||
'unless "detached" is on. "fx" sends it through a sound FX asset; left empty, it plays as it is. ' +
|
||||
'"fx params" takes values for its parameters from a Sound FX parameters node.',
|
||||
'"fx params" takes values for its parameters from a Sound FX parameters node. "voice" is the ' +
|
||||
'number of this one play, for Set sound FX parameters to change it while it plays; 0 where ' +
|
||||
'nothing was played.',
|
||||
inputs: [
|
||||
exec(),
|
||||
soundPort(),
|
||||
@@ -69,25 +71,26 @@ export const PlaySound: NodeDef = {
|
||||
soundFxPort(),
|
||||
{ key: 'fxParams', label: 'fx params', type: 'fxparams' },
|
||||
],
|
||||
outputs: [exec()],
|
||||
outputs: [exec(), num('voice')],
|
||||
params: [],
|
||||
run: (ctx, inputs) => {
|
||||
const sound = asString(inputs.sound);
|
||||
if (sound)
|
||||
ctx.playSound(
|
||||
sound,
|
||||
asNumber(inputs.volume),
|
||||
loopChoice(inputs.loop),
|
||||
asBoolean(inputs.detached),
|
||||
asString(inputs.fx),
|
||||
fxParamsOf(inputs.fxParams),
|
||||
);
|
||||
return { next: 'exec' };
|
||||
const voice = sound
|
||||
? ctx.playSound(
|
||||
sound,
|
||||
asNumber(inputs.volume),
|
||||
loopChoice(inputs.loop),
|
||||
asBoolean(inputs.detached),
|
||||
asString(inputs.fx),
|
||||
fxParamsOf(inputs.fxParams),
|
||||
)
|
||||
: 0;
|
||||
return { next: 'exec', outputs: { voice } };
|
||||
},
|
||||
};
|
||||
|
||||
/** The numbers of a parameter bundle; anything else hands in none. */
|
||||
function fxParamsOf(value: Value | undefined): FxParams {
|
||||
export function fxParamsOf(value: Value | undefined): FxParams {
|
||||
if (!isStruct(value)) return {};
|
||||
const params: Record<string, number> = {};
|
||||
for (const [key, item] of Object.entries(value)) if (typeof item === 'number') params[key] = item;
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
import type { NodeDef } from '../../NodeRegistry';
|
||||
import { asNumber } from '../../values';
|
||||
import { exec, num } from '../shared';
|
||||
import { fxParamsOf } from './PlaySound';
|
||||
|
||||
/**
|
||||
* Changes the sound FX parameters of a sound that is already playing (#315). It reaches one voice,
|
||||
* by the number its `Play sound` handed out: every play is a sound of its own, and two objects with
|
||||
* the same engine sound – or one object playing it twice – each keep theirs.
|
||||
*/
|
||||
export const SetSoundFxParameters: NodeDef = {
|
||||
type: 'SetSoundFxParameters',
|
||||
label: 'Set sound FX parameters',
|
||||
category: 'action',
|
||||
description:
|
||||
'Changes the sound FX parameters of a sound while it plays: the voice a Play sound handed out ' +
|
||||
'glides to the values from a Sound FX parameters node. Keep the voice in a variable to reach it ' +
|
||||
'from another event. A parameter not given keeps its value, a voice that has ended is not ' +
|
||||
'reached. A pitch changes by speed here, so the sound gets shorter or longer while it is moved.',
|
||||
inputs: [
|
||||
exec(),
|
||||
{ ...num('voice'), inline: false },
|
||||
{ key: 'fxParams', label: 'fx params', type: 'fxparams' },
|
||||
],
|
||||
outputs: [exec()],
|
||||
params: [],
|
||||
run: (ctx, inputs) => {
|
||||
const voice = asNumber(inputs.voice);
|
||||
if (voice > 0) ctx.tuneSound(voice, fxParamsOf(inputs.fxParams));
|
||||
return { next: 'exec' };
|
||||
},
|
||||
};
|
||||
@@ -6,14 +6,14 @@ import { asNumber } from '../../values';
|
||||
/**
|
||||
* Values for the parameters of one sound FX (#311), picked from those its Input declares the way a
|
||||
* Variables node picks variables. Each picked one is a number input starting at the FX's default;
|
||||
* the bundle goes into the `fx params` input of a `Play sound`.
|
||||
* the bundle goes into the `fx params` input of a `Play sound` or a `Set sound FX parameters`.
|
||||
*/
|
||||
export const SoundFxParameters: NodeDef = {
|
||||
type: SOUND_FX_PARAMETERS,
|
||||
label: 'Sound FX parameters',
|
||||
category: 'data',
|
||||
description:
|
||||
'Values for the parameters of a sound FX, for the "fx params" input of a Play sound. Pick the parameters to set; one not picked keeps the default its Input gives it.',
|
||||
'Values for the parameters of a sound FX, for the "fx params" input of a Play sound or a Set sound FX parameters. Pick the parameters to set; one not picked keeps the default its Input gives it, or on a playing sound the value it has.',
|
||||
inputs: [],
|
||||
outputs: [{ key: 'params', label: 'params', type: 'fxparams' }],
|
||||
params: [SOUND_FX_SETTING],
|
||||
|
||||
@@ -23,6 +23,7 @@ import { Raycast } from './action/Raycast';
|
||||
import { RaycastAll } from './action/RaycastAll';
|
||||
import { RestartGame } from './action/RestartGame';
|
||||
import { SetPosition } from './action/SetPosition';
|
||||
import { SetSoundFxParameters } from './action/SetSoundFxParameters';
|
||||
import { SetZ } from './action/SetZ';
|
||||
import { Spawn } from './action/Spawn';
|
||||
import { StopGame } from './action/StopGame';
|
||||
@@ -298,6 +299,7 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
|
||||
StopAnimation,
|
||||
PlaySound,
|
||||
StopSound,
|
||||
SetSoundFxParameters,
|
||||
SoundFxParameters,
|
||||
LoadScene,
|
||||
Log,
|
||||
|
||||
+19
-10
@@ -50,6 +50,14 @@ export const EMPTY_PLAN: FxPlan = { pitch: [], stages: [] };
|
||||
/** Values for the parameters, by name; a name the plan does not have is left out. */
|
||||
export type FxParams = Readonly<Record<string, number>>;
|
||||
|
||||
/** How a voice goes through its sound FX: the plan and the values it starts with, kept so they can change while it plays (#315). */
|
||||
export interface FxVoice {
|
||||
plan: FxPlan;
|
||||
params: FxParams;
|
||||
}
|
||||
|
||||
export const DRY_VOICE: FxVoice = { plan: EMPTY_PLAN, params: {} };
|
||||
|
||||
/**
|
||||
* A node of a sound FX graph (#311). It never runs: `emit` says what it does to the sounds arriving
|
||||
* at its audio inputs, by input key, given the numbers at its other inputs. Input and Output have no
|
||||
@@ -239,23 +247,24 @@ function amountOf(amount: FxAmount, params: FxParams): number {
|
||||
return Math.max(amount.min ?? -Infinity, Math.min(amount.max ?? Infinity, given));
|
||||
}
|
||||
|
||||
/** A stage fed by a parameter stays at 0 as well: a value handed in later has to find it there. */
|
||||
function stagesOf(plan: FxPlan, params: FxParams): FxStage[] {
|
||||
const stages: FxStage[] = [];
|
||||
for (const stage of plan.stages) {
|
||||
if (stage.kind === 'gain') {
|
||||
const db = amountOf(stage.db, params);
|
||||
if (db !== 0) stages.push({ kind: 'gain', gain: 10 ** (db / 20) });
|
||||
if (db !== 0 || stage.db.param !== undefined) stages.push({ kind: 'gain', gain: 10 ** (db / 20) });
|
||||
continue;
|
||||
}
|
||||
const gain = amountOf(stage.gain, params);
|
||||
if (gain !== 0) stages.push({ ...stage, gain });
|
||||
if (gain !== 0 || stage.gain.param !== undefined) stages.push({ ...stage, gain });
|
||||
}
|
||||
return stages;
|
||||
}
|
||||
|
||||
/**
|
||||
* The chain one voice is sent along: the plan with every parameter filled in. A Mix that ends at 0
|
||||
* or 1 is left out with the sound it silences, so that one is not even started.
|
||||
* The chain one voice is sent along: the plan with every parameter filled in. Its shape depends on
|
||||
* the plan alone, never on the values, so the values of a playing voice can change in place (#315).
|
||||
*/
|
||||
export function fxChainOf(plan: FxPlan, params: FxParams = {}): FxChain {
|
||||
const stages = stagesOf(plan, params);
|
||||
@@ -263,10 +272,10 @@ export function fxChainOf(plan: FxPlan, params: FxParams = {}): FxChain {
|
||||
const pitch = plan.pitch.reduce((sum, amount) => sum + amountOf(amount, params), 0);
|
||||
return { pitch, stages };
|
||||
}
|
||||
const mix = amountOf(plan.mix, params);
|
||||
if (mix <= 0 || mix >= 1) {
|
||||
const heard = fxChainOf(mix <= 0 ? plan.a : plan.b, params);
|
||||
return { ...heard, stages: [...heard.stages, ...stages] };
|
||||
}
|
||||
return { mix, a: fxChainOf(plan.a, params), b: fxChainOf(plan.b, params), stages };
|
||||
return {
|
||||
mix: amountOf(plan.mix, params),
|
||||
a: fxChainOf(plan.a, params),
|
||||
b: fxChainOf(plan.b, params),
|
||||
stages,
|
||||
};
|
||||
}
|
||||
@@ -41,6 +41,8 @@ export function installDebugHook(app: Application, state: EditorState, editor: E
|
||||
sound: {
|
||||
stats: () => soundLibrary.stats(),
|
||||
duration: (file: string) => soundLibrary.duration(file),
|
||||
/** Where the knobs of each voice of that file are now, oldest first, a pitch as its speed (#315). */
|
||||
levels: (file: string) => soundLibrary.levels(file),
|
||||
/** Decodes ahead of a check, so a play can be asserted without racing the network. */
|
||||
load: (file: string) => soundLibrary.load(file).then((buffer) => buffer?.duration ?? null),
|
||||
stopAll: () => soundLibrary.stopAll(),
|
||||
|
||||
@@ -3,7 +3,6 @@ import { findAssetOf } from '../../core/model/Project';
|
||||
import { resolveAssetProps } from '../../core/properties/resolve';
|
||||
import { volumeOf } from '../../core/properties/sounds';
|
||||
import { soundFxBuildFor, soundFxParams } from '../../core/soundfx/compile';
|
||||
import { fxChainOf } from '../../core/soundfx/nodes';
|
||||
import { Button, Dropdown, HStack, Label, Menu, NumberField, Stack, Theme } from '../../ui';
|
||||
import { soundLibrary } from '../../viewport/SoundLibrary';
|
||||
import { AssetTabEditor } from '../AssetTabEditor';
|
||||
@@ -40,6 +39,8 @@ export class SoundFxEditor extends AssetTabEditor<SoundFxAsset, GraphCanvas> {
|
||||
private defaults: Record<string, number> = {};
|
||||
private paramsKey = '';
|
||||
private valuesFor = '';
|
||||
/** The voices the tab started that may still be playing – what a moved field reaches. */
|
||||
private voices: number[] = [];
|
||||
|
||||
constructor(state: EditorState) {
|
||||
super(state, { infoWidth: INFO_WIDTH });
|
||||
@@ -89,8 +90,8 @@ export class SoundFxEditor extends AssetTabEditor<SoundFxAsset, GraphCanvas> {
|
||||
/** Sets one parameter the header plays with, as typing into its field does. */
|
||||
setPreviewValue(key: string, value: number): void {
|
||||
if (!(key in this.values)) return;
|
||||
this.values[key] = value;
|
||||
this.paramFields.get(key)?.setNumber(value);
|
||||
this.setValue(key, value);
|
||||
}
|
||||
|
||||
/** Plays the picked sound through the graph; a graph that does not compile is not played at all. */
|
||||
@@ -100,11 +101,22 @@ export class SoundFxEditor extends AssetTabEditor<SoundFxAsset, GraphCanvas> {
|
||||
const plan = asset && soundFxBuildFor(asset).compiled;
|
||||
if (!sound?.file || !plan) return;
|
||||
const volume = volumeOf(resolveAssetProps(this.state.project, sound));
|
||||
soundLibrary.play(sound.file, volume, false, PREVIEW_OWNER, fxChainOf(plan, this.values));
|
||||
const voice = soundLibrary.play(sound.file, volume, false, PREVIEW_OWNER, {
|
||||
plan,
|
||||
params: { ...this.values },
|
||||
});
|
||||
if (voice) this.voices.push(voice);
|
||||
}
|
||||
|
||||
/** A moved field is heard at once on what the tab is playing (#315); an ended voice is let go. */
|
||||
private setValue(key: string, value: number): void {
|
||||
this.values[key] = value;
|
||||
this.voices = this.voices.filter((voice) => soundLibrary.tune(voice, { [key]: value }));
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
soundLibrary.stopOwner(PREVIEW_OWNER);
|
||||
this.voices = [];
|
||||
}
|
||||
|
||||
protected override infoText(asset: SoundFxAsset): string {
|
||||
@@ -152,8 +164,8 @@ export class SoundFxEditor extends AssetTabEditor<SoundFxAsset, GraphCanvas> {
|
||||
const field = new NumberField(values[port.key], {
|
||||
width: PARAM_WIDTH,
|
||||
step: 0.1,
|
||||
onInput: (value) => (this.values[port.key] = value),
|
||||
onChange: (value) => (this.values[port.key] = value),
|
||||
onInput: (value) => this.setValue(port.key, value),
|
||||
onChange: (value) => this.setValue(port.key, value),
|
||||
});
|
||||
this.paramFields.set(port.key, field);
|
||||
this.paramBar.add(field);
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import type { FxChain } from '../../core/soundfx/nodes';
|
||||
import type { FxParams, FxVoice } from '../../core/soundfx/nodes';
|
||||
|
||||
/**
|
||||
* What the runtime needs of the outside world to make a noise – the counterpart of `Input` on the
|
||||
@@ -13,10 +13,17 @@ export interface AudioSink {
|
||||
* Starts one voice. `volume` is final (0–1) and every call overlaps whatever is already playing.
|
||||
* `owner` is the behavior the sound belongs to: when that behavior is destroyed the voice goes
|
||||
* with it, because a sound is something a thing in the world is doing, not a thing of its own.
|
||||
* `''` means the voice has no owner and outlives whatever started it. `fx` is the line of effects
|
||||
* the voice passes on its way out (#311); `DRY` passes none.
|
||||
* `''` means the voice has no owner and outlives whatever started it. `fx` is the sound FX the
|
||||
* voice passes on its way out (#311), with the values of its parameters; `DRY_VOICE` passes none.
|
||||
* Returns the number of the voice, never used twice, or 0 where nothing is going to play.
|
||||
*/
|
||||
play(file: string, volume: number, loop: boolean, owner: string, fx?: FxChain): void;
|
||||
play(file: string, volume: number, loop: boolean, owner: string, fx?: FxVoice): number;
|
||||
/**
|
||||
* Hands new values to the sound FX parameters of one voice, by the number `play` gave it (#315).
|
||||
* A name its FX does not have is left out, one not given keeps its value. A voice that has ended
|
||||
* is not there any more and nothing happens – a detached one is reached like any other.
|
||||
*/
|
||||
tune(voice: number, params: FxParams): void;
|
||||
/** Silences every voice of that file; `''` silences all of them. */
|
||||
stop(file: string): void;
|
||||
/** Silences every voice started by that owner – what destroying a behavior does. */
|
||||
@@ -24,7 +31,8 @@ export interface AudioSink {
|
||||
}
|
||||
|
||||
export const SILENT_AUDIO: AudioSink = {
|
||||
play: () => {},
|
||||
play: () => 0,
|
||||
tune: () => {},
|
||||
stop: () => {},
|
||||
stopOwner: () => {},
|
||||
};
|
||||
@@ -16,7 +16,7 @@ import { hasRenderLayer } from '../../core/properties/PropertySchema';
|
||||
import { resolveAssetProps } from '../../core/properties/resolve';
|
||||
import { loopsOf, volumeOf } from '../../core/properties/sounds';
|
||||
import { soundFxBuildFor } from '../../core/soundfx/compile';
|
||||
import { DRY, fxChainOf, type FxParams } from '../../core/soundfx/nodes';
|
||||
import { DRY_VOICE, type FxParams } from '../../core/soundfx/nodes';
|
||||
import type { Entity } from '../Entity';
|
||||
import { PhysicsSystem } from '../physics/PhysicsSystem';
|
||||
import type { World } from '../World';
|
||||
@@ -172,6 +172,7 @@ export function behaviorContext(
|
||||
const file = assetId === '' ? '' : soundFileOf(project, assetId);
|
||||
if (assetId === '' || file) run.audio.stop(file);
|
||||
},
|
||||
tuneSound: (voice, fxParams) => run.audio.tune(voice, fxParams),
|
||||
spawn: (assetId, position, parent) => world.spawn(assetId, position, resolve(parent) ?? null),
|
||||
addJoint: (assetId, owner, connected, at) => {
|
||||
const on = resolve(owner);
|
||||
@@ -218,7 +219,7 @@ export function behaviorContext(
|
||||
* node's, and its loop flag unless the node overrules it either way. The asset is what a sound
|
||||
* *is* – a node that says nothing therefore plays it as the project describes it. An FX asset that
|
||||
* does not compile plays the sound dry: a broken effect should not silence the game. Its parameters
|
||||
* take the values handed in, the others their defaults.
|
||||
* take the values handed in, the others their defaults. Returns the number of the voice, 0 for none.
|
||||
*/
|
||||
function playSound(
|
||||
{ project, audio }: ContextSources,
|
||||
@@ -228,14 +229,20 @@ function playSound(
|
||||
owner: string,
|
||||
fxId: string,
|
||||
fxParams: FxParams,
|
||||
): void {
|
||||
): number {
|
||||
const asset = findAssetOf(project, assetId, 'sound');
|
||||
if (!asset?.file) return;
|
||||
if (!asset?.file) return 0;
|
||||
const props = resolveAssetProps(project, asset);
|
||||
const level = Math.max(0, Math.min(1, volumeOf(props) * volume));
|
||||
const fx = fxId ? findAssetOf(project, fxId, 'soundfx') : undefined;
|
||||
const plan = fx && soundFxBuildFor(fx).compiled;
|
||||
audio.play(asset.file, level, loop ?? loopsOf(props), owner, plan ? fxChainOf(plan, fxParams) : DRY);
|
||||
return audio.play(
|
||||
asset.file,
|
||||
level,
|
||||
loop ?? loopsOf(props),
|
||||
owner,
|
||||
plan ? { plan, params: fxParams } : DRY_VOICE,
|
||||
);
|
||||
}
|
||||
|
||||
/** Where a ray of that length ends; where it has no direction, where it starts. */
|
||||
|
||||
+143
-31
@@ -1,34 +1,62 @@
|
||||
import { DRY, type FxChain, type FxStage } from '../core/soundfx/nodes';
|
||||
import {
|
||||
DRY_VOICE,
|
||||
fxChainOf,
|
||||
type FxChain,
|
||||
type FxParams,
|
||||
type FxStage,
|
||||
type FxVoice,
|
||||
} from '../core/soundfx/nodes';
|
||||
import { pitchShift } from '../core/soundfx/pitchShift';
|
||||
import type { AudioSink } from '../runtime/systems/Audio';
|
||||
|
||||
const DEFAULT_BASE = '/game-maker/data/sounds';
|
||||
/** How many pitched copies are kept at once. */
|
||||
const SHIFTED_KEPT = 32;
|
||||
/** Seconds a changed parameter takes to get most of the way there: quick, but no click (#315). */
|
||||
const GLIDE = 0.03;
|
||||
|
||||
/**
|
||||
* One number of a voice that can still change while it plays (#315): a level, a share of a Mix, the
|
||||
* gain of a band – or the pitch of a source, which glides by speed from the pitch its copy was made at.
|
||||
*/
|
||||
type Knob =
|
||||
| { kind: 'level'; param: AudioParam; target: number }
|
||||
| { kind: 'pitch'; source: AudioBufferSourceNode; base: number; target: number };
|
||||
|
||||
/**
|
||||
* One sound in flight: its sources, its own gain, the file it came from and who is making it. A
|
||||
* voice has a source per pitch its sound FX plays the file at (#316), started together and as long
|
||||
* as each other.
|
||||
* voice has a source per way through its sound FX (#316), started together and as long as each
|
||||
* other. Its knobs are in the order `levelsOf` lists the numbers of its chain.
|
||||
*/
|
||||
interface Voice {
|
||||
/** What `play` returned for it, the handle a `tune` reaches it by. */
|
||||
id: number;
|
||||
file: string;
|
||||
/** The behavior this voice belongs to; `''` for one nothing owns (see `AudioSink.play`). */
|
||||
owner: string;
|
||||
sources: AudioBufferSourceNode[];
|
||||
gain: GainNode;
|
||||
fx: FxVoice;
|
||||
knobs: Knob[];
|
||||
}
|
||||
|
||||
/** What one voice is wired from: the file, and its sources by pitch so far. */
|
||||
/** What one voice is wired from: the file, and what has been made for it so far. */
|
||||
interface Wiring {
|
||||
context: AudioContext;
|
||||
file: string;
|
||||
buffer: AudioBuffer;
|
||||
loop: boolean;
|
||||
sources: Map<number, AudioBufferSourceNode>;
|
||||
sources: { source: AudioBufferSourceNode; pitch: number }[];
|
||||
knobs: Knob[];
|
||||
stages: number;
|
||||
}
|
||||
|
||||
/** A play still waiting on its bytes: what a `tune` in the meantime writes into. */
|
||||
interface Pending {
|
||||
id: number;
|
||||
fx: FxVoice;
|
||||
}
|
||||
|
||||
/**
|
||||
* Plays uploaded sounds, keyed by file name – the texture library's twin on the audio side. Files
|
||||
* are content addressed, so a decoded buffer can never go stale and nothing is evicted.
|
||||
@@ -52,8 +80,12 @@ export class SoundLibrary implements AudioSink {
|
||||
/** Never retried, so one broken reference cannot spam the network. */
|
||||
private readonly failed = new Set<string>();
|
||||
private voices: Voice[] = [];
|
||||
private pending: Pending[] = [];
|
||||
/** The number the next play is given; 0 stays free for "nothing plays". */
|
||||
private nextId = 1;
|
||||
private started = 0;
|
||||
private stopped = 0;
|
||||
private tuned = 0;
|
||||
/** The last voice actually started, with the values it was started at – what a check reads back. */
|
||||
private last: LastVoice | null = null;
|
||||
/**
|
||||
@@ -73,20 +105,44 @@ export class SoundLibrary implements AudioSink {
|
||||
this.base = base.replace(/\/+$/, '');
|
||||
}
|
||||
|
||||
play(file: string, volume: number, loop: boolean, owner = '', fx: FxChain = DRY): void {
|
||||
if (file === '' || this.failed.has(file)) return;
|
||||
/**
|
||||
* The number is handed out at once, before the bytes are there: a graph can tune a voice in the
|
||||
* same frame it asked for it, and the voice then starts with those values.
|
||||
*/
|
||||
play(file: string, volume: number, loop: boolean, owner = '', fx: FxVoice = DRY_VOICE): number {
|
||||
if (file === '' || this.failed.has(file)) return 0;
|
||||
const id = this.nextId++;
|
||||
const buffer = this.buffers.get(file);
|
||||
if (buffer) {
|
||||
this.start(file, buffer, volume, loop, owner, fx);
|
||||
return;
|
||||
this.start(id, file, buffer, volume, loop, owner, fx);
|
||||
return id;
|
||||
}
|
||||
const askedAt = this.cancelsFor(file, owner);
|
||||
const waiting: Pending = { id, fx: { plan: fx.plan, params: { ...fx.params } } };
|
||||
this.pending.push(waiting);
|
||||
void this.load(file).then((loaded) => {
|
||||
this.pending = this.pending.filter((other) => other !== waiting);
|
||||
// A `Stop sound` – or the destruction of the behavior that asked – that happened while the
|
||||
// bytes were in flight has to win, or a sound the run already ended would start afterwards.
|
||||
if (loaded && this.cancelsFor(file, owner) === askedAt)
|
||||
this.start(file, loaded, volume, loop, owner, fx);
|
||||
this.start(id, file, loaded, volume, loop, owner, waiting.fx);
|
||||
});
|
||||
return id;
|
||||
}
|
||||
|
||||
/** Says whether the voice was still there to be reached – playing, or waiting on its bytes. */
|
||||
tune(id: number, params: FxParams): boolean {
|
||||
const waiting = this.pending.find((other) => other.id === id);
|
||||
if (waiting) {
|
||||
waiting.fx = { plan: waiting.fx.plan, params: { ...waiting.fx.params, ...params } };
|
||||
return true;
|
||||
}
|
||||
const voice = this.voices.find((other) => other.id === id);
|
||||
if (!voice) return false;
|
||||
voice.fx = { plan: voice.fx.plan, params: { ...voice.fx.params, ...params } };
|
||||
this.glide(voice);
|
||||
this.tuned += 1;
|
||||
return true;
|
||||
}
|
||||
|
||||
stop(file: string): void {
|
||||
@@ -123,6 +179,7 @@ export class SoundLibrary implements AudioSink {
|
||||
playing: number;
|
||||
started: number;
|
||||
stopped: number;
|
||||
tuned: number;
|
||||
decoded: number;
|
||||
failed: number;
|
||||
last: LastVoice | null;
|
||||
@@ -131,12 +188,25 @@ export class SoundLibrary implements AudioSink {
|
||||
playing: this.voices.length,
|
||||
started: this.started,
|
||||
stopped: this.stopped,
|
||||
tuned: this.tuned,
|
||||
decoded: this.buffers.size,
|
||||
failed: this.failed.size,
|
||||
last: this.last,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Where the knobs of each voice of that file are right now, oldest first, a pitch as the speed it
|
||||
* plays at – what a check reads to see a change arrive at one voice and not another (test hook).
|
||||
*/
|
||||
levels(file: string): number[][] {
|
||||
return this.voices
|
||||
.filter((voice) => voice.file === file)
|
||||
.map((voice) =>
|
||||
voice.knobs.map((knob) => (knob.kind === 'pitch' ? knob.source.playbackRate : knob.param).value),
|
||||
);
|
||||
}
|
||||
|
||||
/** How long a decoded sound runs, in seconds; null while it has not been loaded (test hook). */
|
||||
duration(file: string): number | null {
|
||||
return this.buffers.get(file)?.duration ?? null;
|
||||
@@ -177,12 +247,13 @@ export class SoundLibrary implements AudioSink {
|
||||
}
|
||||
|
||||
private start(
|
||||
id: number,
|
||||
file: string,
|
||||
buffer: AudioBuffer,
|
||||
volume: number,
|
||||
loop: boolean,
|
||||
owner: string,
|
||||
fx: FxChain,
|
||||
fx: FxVoice,
|
||||
): void {
|
||||
const context = this.audio();
|
||||
// A context starts suspended where the page has not been interacted with; without this the
|
||||
@@ -190,18 +261,18 @@ export class SoundLibrary implements AudioSink {
|
||||
if (context.state === 'suspended') void context.resume();
|
||||
const gain = context.createGain();
|
||||
gain.gain.value = Math.max(0, Math.min(1, volume));
|
||||
const wiring: Wiring = { context, file, buffer, loop, sources: new Map(), stages: 0 };
|
||||
this.wire(wiring, fx).connect(gain).connect(context.destination);
|
||||
const sources = [...wiring.sources.values()];
|
||||
const wiring: Wiring = { context, file, buffer, loop, sources: [], knobs: [], stages: 0 };
|
||||
this.wire(wiring, fxChainOf(fx.plan, fx.params)).connect(gain).connect(context.destination);
|
||||
const sources = wiring.sources.map((made) => made.source);
|
||||
this.last = {
|
||||
file,
|
||||
volume: gain.gain.value,
|
||||
loop,
|
||||
pitches: [...wiring.sources.keys()],
|
||||
pitches: wiring.sources.map((made) => made.pitch),
|
||||
stages: wiring.stages,
|
||||
duration: sources[0]!.buffer!.duration,
|
||||
};
|
||||
const voice: Voice = { file, owner, sources, gain };
|
||||
const voice: Voice = { id, file, owner, sources, gain, fx, knobs: wiring.knobs };
|
||||
let playing = sources.length;
|
||||
for (const source of sources)
|
||||
source.onended = () => {
|
||||
@@ -213,35 +284,60 @@ export class SoundLibrary implements AudioSink {
|
||||
}
|
||||
|
||||
/**
|
||||
* The nodes a chain becomes, up to the one that carries its sound on. Two ways at the same pitch
|
||||
* share a source; a Mix weighs each of its sounds by its share before they are summed.
|
||||
* The nodes a chain becomes, up to the one that carries its sound on. Every way has a source of
|
||||
* its own, so each can glide to its own pitch; a Mix weighs each of its sounds by its share before
|
||||
* they are summed.
|
||||
*/
|
||||
private wire(wiring: Wiring, chain: FxChain): AudioNode {
|
||||
const { context } = wiring;
|
||||
const { context, knobs } = wiring;
|
||||
let tail: AudioNode;
|
||||
if ('mix' in chain) {
|
||||
tail = context.createGain();
|
||||
this.wire(wiring, chain.a)
|
||||
.connect(gainNode(context, 1 - chain.mix))
|
||||
.connect(tail);
|
||||
this.wire(wiring, chain.b).connect(gainNode(context, chain.mix)).connect(tail);
|
||||
const wayA = this.wire(wiring, chain.a);
|
||||
const shareA = gainNode(context, 1 - chain.mix);
|
||||
wayA.connect(shareA).connect(tail);
|
||||
knobs.push({ kind: 'level', param: shareA.gain, target: 1 - chain.mix });
|
||||
const wayB = this.wire(wiring, chain.b);
|
||||
const shareB = gainNode(context, chain.mix);
|
||||
wayB.connect(shareB).connect(tail);
|
||||
knobs.push({ kind: 'level', param: shareB.gain, target: chain.mix });
|
||||
} else tail = this.source(wiring, chain.pitch);
|
||||
for (const stage of chain.stages) tail = tail.connect(fxNode(context, stage));
|
||||
for (const stage of chain.stages) {
|
||||
const [node, param] = fxNode(context, stage);
|
||||
tail = tail.connect(node);
|
||||
knobs.push({ kind: 'level', param, target: stage.gain });
|
||||
}
|
||||
wiring.stages += chain.stages.length;
|
||||
return tail;
|
||||
}
|
||||
|
||||
private source(wiring: Wiring, pitch: number): AudioBufferSourceNode {
|
||||
const held = wiring.sources.get(pitch);
|
||||
if (held) return held;
|
||||
const { context, file, buffer, loop } = wiring;
|
||||
const source = context.createBufferSource();
|
||||
source.buffer = pitch === 0 ? buffer : this.pitched(context, file, buffer, pitch, loop);
|
||||
source.loop = loop;
|
||||
wiring.sources.set(pitch, source);
|
||||
wiring.sources.push({ source, pitch });
|
||||
wiring.knobs.push({ kind: 'pitch', source, base: pitch, target: pitch });
|
||||
return source;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sends every knob of a voice towards what its values say now. A pitch cannot get a new copy in
|
||||
* the middle of a sound, so it glides by speed: the voice gets that much shorter or longer.
|
||||
*/
|
||||
private glide(voice: Voice): void {
|
||||
const levels = levelsOf(fxChainOf(voice.fx.plan, voice.fx.params));
|
||||
const now = this.audio().currentTime;
|
||||
voice.knobs.forEach((knob, i) => {
|
||||
const level = levels[i];
|
||||
if (level === undefined || level === knob.target) return;
|
||||
knob.target = level;
|
||||
if (knob.kind === 'pitch')
|
||||
knob.source.playbackRate.setTargetAtTime(2 ** ((level - knob.base) / 12), now, GLIDE);
|
||||
else knob.param.setTargetAtTime(level, now, GLIDE);
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* The sound at another pitch and its own length (#312), worked out once per file, pitch and loop
|
||||
* and kept for the next voice. The oldest goes first, so a pitch drawn at random every play cannot
|
||||
@@ -286,7 +382,7 @@ interface LastVoice {
|
||||
file: string;
|
||||
volume: number;
|
||||
loop: boolean;
|
||||
/** Semitones of each source, one per pitch (#316), and how many gain and filter nodes it passed. */
|
||||
/** Semitones of each source, one per way (#316), and how many gain and filter nodes it passed. */
|
||||
pitches: number[];
|
||||
stages: number;
|
||||
/** Seconds of what was played: a pitched voice is as long as the sound itself (#312). */
|
||||
@@ -299,14 +395,30 @@ function gainNode(context: AudioContext, level: number): GainNode {
|
||||
return gain;
|
||||
}
|
||||
|
||||
function fxNode(context: AudioContext, stage: FxStage): AudioNode {
|
||||
if (stage.kind === 'gain') return gainNode(context, stage.gain);
|
||||
/** The node a stage becomes, with the number of it a parameter can move. */
|
||||
function fxNode(context: AudioContext, stage: FxStage): [AudioNode, AudioParam] {
|
||||
if (stage.kind === 'gain') {
|
||||
const gain = gainNode(context, stage.gain);
|
||||
return [gain, gain.gain];
|
||||
}
|
||||
const filter = context.createBiquadFilter();
|
||||
filter.type = stage.filter;
|
||||
filter.frequency.value = stage.frequency;
|
||||
filter.gain.value = stage.gain;
|
||||
filter.Q.value = stage.q;
|
||||
return filter;
|
||||
return [filter, filter.gain];
|
||||
}
|
||||
|
||||
/** The numbers of a chain in the order `wire` makes their knobs: a stage's `gain` is what its node holds. */
|
||||
function levelsOf(chain: FxChain, into: number[] = []): number[] {
|
||||
if ('mix' in chain) {
|
||||
levelsOf(chain.a, into);
|
||||
into.push(1 - chain.mix);
|
||||
levelsOf(chain.b, into);
|
||||
into.push(chain.mix);
|
||||
} else into.push(chain.pitch);
|
||||
for (const stage of chain.stages) into.push(stage.gain);
|
||||
return into;
|
||||
}
|
||||
|
||||
export const soundLibrary = new SoundLibrary();
|
||||
Reference in new issue
Block a user