Change the sound FX parameters of a sound while it plays #315
Until now a value handed to a sound FX counted from the start of the voice only. An engine whose pitch follows the speed, or a loop that gets quieter, could not be built. Every play of Play sound now gives out the number of its voice, and a new node lets the knobs of that one voice glide to new values while it plays. - Play sound has a new output voice: the number of this one play, or 0 where nothing was played. Kept in a Number variable it reaches the voice from another event. Two plays of one sound are two voices. - New action node Set sound FX parameters (exec, voice, fx params). It reaches exactly the voice it is given, a detached one too. A voice that has ended, or voice 0, is not reached. - AudioSink.play returns the voice number and takes the plan and the values (FxVoice) instead of a finished chain. AudioSink.tune takes a voice number and new values. A name the voice's FX does not have is left out, one not given keeps its value. - SoundLibrary hands out the number at once, before the bytes are there, and never twice. A voice still waiting for its bytes starts with the values handed in meanwhile. - Every knob of a voice moves with setTargetAtTime (30 ms time constant). A knob whose value did not change is not sent again, so a tune every frame costs next to nothing. - A pitch on a playing voice glides by playbackRate. A new WSOLA copy in the middle of a sound is not worked out, so the voice gets shorter or longer while it is moved. On a loop that is not heard. - fxChainOf now gives a chain whose shape depends on the plan alone. A stage fed by a parameter stays at 0 dB, and a Mix fed by a parameter keeps both ways. A typed-in 0 or Mix at 0/1 is still left out by the compiler. - Every way of a voice has a source of its own, also at the same pitch, so each can glide on its own. The pitched copy is shared. - The tab keeps the numbers of what it plays, so moving a parameter field is heard at once. - Test hook __gm.sound.levels(file) reads the knobs of every voice of a file, oldest first, and stats() counts the voices tuned. Tests: - soundfx.test.ts: 29 tests (+2 for the node in a running world: the voice of the first of two plays of one sound, and a play that played nothing) - sound-library.test.ts: 15 tests (+3: glide, reach by number, pending voice) - node-catalog.test.ts: 370 tests (+2, Play sound checks its voice) - sounds + keep-on-scene-load: 31 tests, unchanged - Counter-test: a tune that reaches every voice of the same file makes the reach test fail. - e2e 34 (melektaus-atx): ALL PASSED - 30 checks (+1). In Chrome two looping voices of one sound start at depth 7. The one whose number sits in a variable glides to 0.7938 (2^(-4/12) = 0.7937), the other stays at 1. Both gains stay at 0.5012. - The gate ran before voices got a number (CHECK EXIT 0, 241 files, 3658 tests), not after. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
072537b2e8
commit
c7e9769f90
18 files changed
+557
-100
No files matched your search
@@ -240,6 +240,7 @@ export interface NodeContext {
|
|||||||
* the one case where the rule is in the way: the death sound of an object that destroys itself.
|
* the one case where the rule is in the way: the death sound of an object that destroys itself.
|
||||||
* `fx` names a sound FX asset the voice is played through (#311); `''` plays it as it is.
|
* `fx` names a sound FX asset the voice is played through (#311); `''` plays it as it is.
|
||||||
* `fxParams` are values for its parameters; one not given is the default the FX declares.
|
* `fxParams` are values for its parameters; one not given is the default the FX declares.
|
||||||
|
* Returns the number of the voice for `tuneSound`, or 0 where nothing is played.
|
||||||
*/
|
*/
|
||||||
playSound(
|
playSound(
|
||||||
assetId: string,
|
assetId: string,
|
||||||
@@ -248,9 +249,14 @@ export interface NodeContext {
|
|||||||
detached: boolean,
|
detached: boolean,
|
||||||
fx: string,
|
fx: string,
|
||||||
fxParams: FxParams,
|
fxParams: FxParams,
|
||||||
): void;
|
): number;
|
||||||
/** Silences every voice of that sound; `''` silences everything this run started. */
|
/** Silences every voice of that sound; `''` silences everything this run started. */
|
||||||
stopSound(assetId: string): void;
|
stopSound(assetId: string): void;
|
||||||
|
/**
|
||||||
|
* Hands new values to the sound FX parameters of one voice, by the number `playSound` gave it
|
||||||
|
* (#315), so a running sound follows the game. A voice that has ended is not reached.
|
||||||
|
*/
|
||||||
|
tuneSound(voice: number, fxParams: FxParams): void;
|
||||||
/**
|
/**
|
||||||
* Asks for another scene, by asset id. The swap happens once the frame is over – a graph must be
|
* Asks for another scene, by asset id. The swap happens once the frame is over – a graph must be
|
||||||
* able to finish the flow it is in, and half a frame in the old world and half in the new one is
|
* able to finish the flow it is in, and half a frame in the old world and half in the new one is
|
||||||
|
|||||||
@@ -59,7 +59,9 @@ export const PlaySound: NodeDef = {
|
|||||||
'the node only colors it. Each play is a voice of its own: firing this twice in a row is heard ' +
|
'the node only colors it. Each play is a voice of its own: firing this twice in a row is heard ' +
|
||||||
'twice, overlapping. The sound belongs to this object and stops when the object is destroyed, ' +
|
'twice, overlapping. The sound belongs to this object and stops when the object is destroyed, ' +
|
||||||
'unless "detached" is on. "fx" sends it through a sound FX asset; left empty, it plays as it is. ' +
|
'unless "detached" is on. "fx" sends it through a sound FX asset; left empty, it plays as it is. ' +
|
||||||
'"fx params" takes values for its parameters from a Sound FX parameters node.',
|
'"fx params" takes values for its parameters from a Sound FX parameters node. "voice" is the ' +
|
||||||
|
'number of this one play, for Set sound FX parameters to change it while it plays; 0 where ' +
|
||||||
|
'nothing was played.',
|
||||||
inputs: [
|
inputs: [
|
||||||
exec(),
|
exec(),
|
||||||
soundPort(),
|
soundPort(),
|
||||||
@@ -69,25 +71,26 @@ export const PlaySound: NodeDef = {
|
|||||||
soundFxPort(),
|
soundFxPort(),
|
||||||
{ key: 'fxParams', label: 'fx params', type: 'fxparams' },
|
{ key: 'fxParams', label: 'fx params', type: 'fxparams' },
|
||||||
],
|
],
|
||||||
outputs: [exec()],
|
outputs: [exec(), num('voice')],
|
||||||
params: [],
|
params: [],
|
||||||
run: (ctx, inputs) => {
|
run: (ctx, inputs) => {
|
||||||
const sound = asString(inputs.sound);
|
const sound = asString(inputs.sound);
|
||||||
if (sound)
|
const voice = sound
|
||||||
ctx.playSound(
|
? ctx.playSound(
|
||||||
sound,
|
sound,
|
||||||
asNumber(inputs.volume),
|
asNumber(inputs.volume),
|
||||||
loopChoice(inputs.loop),
|
loopChoice(inputs.loop),
|
||||||
asBoolean(inputs.detached),
|
asBoolean(inputs.detached),
|
||||||
asString(inputs.fx),
|
asString(inputs.fx),
|
||||||
fxParamsOf(inputs.fxParams),
|
fxParamsOf(inputs.fxParams),
|
||||||
);
|
)
|
||||||
return { next: 'exec' };
|
: 0;
|
||||||
|
return { next: 'exec', outputs: { voice } };
|
||||||
},
|
},
|
||||||
};
|
};
|
||||||
|
|
||||||
/** The numbers of a parameter bundle; anything else hands in none. */
|
/** The numbers of a parameter bundle; anything else hands in none. */
|
||||||
function fxParamsOf(value: Value | undefined): FxParams {
|
export function fxParamsOf(value: Value | undefined): FxParams {
|
||||||
if (!isStruct(value)) return {};
|
if (!isStruct(value)) return {};
|
||||||
const params: Record<string, number> = {};
|
const params: Record<string, number> = {};
|
||||||
for (const [key, item] of Object.entries(value)) if (typeof item === 'number') params[key] = item;
|
for (const [key, item] of Object.entries(value)) if (typeof item === 'number') params[key] = item;
|
||||||
|
|||||||
@@ -0,0 +1,32 @@
|
|||||||
|
import type { NodeDef } from '../../NodeRegistry';
|
||||||
|
import { asNumber } from '../../values';
|
||||||
|
import { exec, num } from '../shared';
|
||||||
|
import { fxParamsOf } from './PlaySound';
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Changes the sound FX parameters of a sound that is already playing (#315). It reaches one voice,
|
||||||
|
* by the number its `Play sound` handed out: every play is a sound of its own, and two objects with
|
||||||
|
* the same engine sound – or one object playing it twice – each keep theirs.
|
||||||
|
*/
|
||||||
|
export const SetSoundFxParameters: NodeDef = {
|
||||||
|
type: 'SetSoundFxParameters',
|
||||||
|
label: 'Set sound FX parameters',
|
||||||
|
category: 'action',
|
||||||
|
description:
|
||||||
|
'Changes the sound FX parameters of a sound while it plays: the voice a Play sound handed out ' +
|
||||||
|
'glides to the values from a Sound FX parameters node. Keep the voice in a variable to reach it ' +
|
||||||
|
'from another event. A parameter not given keeps its value, a voice that has ended is not ' +
|
||||||
|
'reached. A pitch changes by speed here, so the sound gets shorter or longer while it is moved.',
|
||||||
|
inputs: [
|
||||||
|
exec(),
|
||||||
|
{ ...num('voice'), inline: false },
|
||||||
|
{ key: 'fxParams', label: 'fx params', type: 'fxparams' },
|
||||||
|
],
|
||||||
|
outputs: [exec()],
|
||||||
|
params: [],
|
||||||
|
run: (ctx, inputs) => {
|
||||||
|
const voice = asNumber(inputs.voice);
|
||||||
|
if (voice > 0) ctx.tuneSound(voice, fxParamsOf(inputs.fxParams));
|
||||||
|
return { next: 'exec' };
|
||||||
|
},
|
||||||
|
};
|
||||||
@@ -6,14 +6,14 @@ import { asNumber } from '../../values';
|
|||||||
/**
|
/**
|
||||||
* Values for the parameters of one sound FX (#311), picked from those its Input declares the way a
|
* Values for the parameters of one sound FX (#311), picked from those its Input declares the way a
|
||||||
* Variables node picks variables. Each picked one is a number input starting at the FX's default;
|
* Variables node picks variables. Each picked one is a number input starting at the FX's default;
|
||||||
* the bundle goes into the `fx params` input of a `Play sound`.
|
* the bundle goes into the `fx params` input of a `Play sound` or a `Set sound FX parameters`.
|
||||||
*/
|
*/
|
||||||
export const SoundFxParameters: NodeDef = {
|
export const SoundFxParameters: NodeDef = {
|
||||||
type: SOUND_FX_PARAMETERS,
|
type: SOUND_FX_PARAMETERS,
|
||||||
label: 'Sound FX parameters',
|
label: 'Sound FX parameters',
|
||||||
category: 'data',
|
category: 'data',
|
||||||
description:
|
description:
|
||||||
'Values for the parameters of a sound FX, for the "fx params" input of a Play sound. Pick the parameters to set; one not picked keeps the default its Input gives it.',
|
'Values for the parameters of a sound FX, for the "fx params" input of a Play sound or a Set sound FX parameters. Pick the parameters to set; one not picked keeps the default its Input gives it, or on a playing sound the value it has.',
|
||||||
inputs: [],
|
inputs: [],
|
||||||
outputs: [{ key: 'params', label: 'params', type: 'fxparams' }],
|
outputs: [{ key: 'params', label: 'params', type: 'fxparams' }],
|
||||||
params: [SOUND_FX_SETTING],
|
params: [SOUND_FX_SETTING],
|
||||||
|
|||||||
@@ -23,6 +23,7 @@ import { Raycast } from './action/Raycast';
|
|||||||
import { RaycastAll } from './action/RaycastAll';
|
import { RaycastAll } from './action/RaycastAll';
|
||||||
import { RestartGame } from './action/RestartGame';
|
import { RestartGame } from './action/RestartGame';
|
||||||
import { SetPosition } from './action/SetPosition';
|
import { SetPosition } from './action/SetPosition';
|
||||||
|
import { SetSoundFxParameters } from './action/SetSoundFxParameters';
|
||||||
import { SetZ } from './action/SetZ';
|
import { SetZ } from './action/SetZ';
|
||||||
import { Spawn } from './action/Spawn';
|
import { Spawn } from './action/Spawn';
|
||||||
import { StopGame } from './action/StopGame';
|
import { StopGame } from './action/StopGame';
|
||||||
@@ -298,6 +299,7 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
|
|||||||
StopAnimation,
|
StopAnimation,
|
||||||
PlaySound,
|
PlaySound,
|
||||||
StopSound,
|
StopSound,
|
||||||
|
SetSoundFxParameters,
|
||||||
SoundFxParameters,
|
SoundFxParameters,
|
||||||
LoadScene,
|
LoadScene,
|
||||||
Log,
|
Log,
|
||||||
|
|||||||
+19
-10
@@ -50,6 +50,14 @@ export const EMPTY_PLAN: FxPlan = { pitch: [], stages: [] };
|
|||||||
/** Values for the parameters, by name; a name the plan does not have is left out. */
|
/** Values for the parameters, by name; a name the plan does not have is left out. */
|
||||||
export type FxParams = Readonly<Record<string, number>>;
|
export type FxParams = Readonly<Record<string, number>>;
|
||||||
|
|
||||||
|
/** How a voice goes through its sound FX: the plan and the values it starts with, kept so they can change while it plays (#315). */
|
||||||
|
export interface FxVoice {
|
||||||
|
plan: FxPlan;
|
||||||
|
params: FxParams;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const DRY_VOICE: FxVoice = { plan: EMPTY_PLAN, params: {} };
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* A node of a sound FX graph (#311). It never runs: `emit` says what it does to the sounds arriving
|
* A node of a sound FX graph (#311). It never runs: `emit` says what it does to the sounds arriving
|
||||||
* at its audio inputs, by input key, given the numbers at its other inputs. Input and Output have no
|
* at its audio inputs, by input key, given the numbers at its other inputs. Input and Output have no
|
||||||
@@ -239,23 +247,24 @@ function amountOf(amount: FxAmount, params: FxParams): number {
|
|||||||
return Math.max(amount.min ?? -Infinity, Math.min(amount.max ?? Infinity, given));
|
return Math.max(amount.min ?? -Infinity, Math.min(amount.max ?? Infinity, given));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** A stage fed by a parameter stays at 0 as well: a value handed in later has to find it there. */
|
||||||
function stagesOf(plan: FxPlan, params: FxParams): FxStage[] {
|
function stagesOf(plan: FxPlan, params: FxParams): FxStage[] {
|
||||||
const stages: FxStage[] = [];
|
const stages: FxStage[] = [];
|
||||||
for (const stage of plan.stages) {
|
for (const stage of plan.stages) {
|
||||||
if (stage.kind === 'gain') {
|
if (stage.kind === 'gain') {
|
||||||
const db = amountOf(stage.db, params);
|
const db = amountOf(stage.db, params);
|
||||||
if (db !== 0) stages.push({ kind: 'gain', gain: 10 ** (db / 20) });
|
if (db !== 0 || stage.db.param !== undefined) stages.push({ kind: 'gain', gain: 10 ** (db / 20) });
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
const gain = amountOf(stage.gain, params);
|
const gain = amountOf(stage.gain, params);
|
||||||
if (gain !== 0) stages.push({ ...stage, gain });
|
if (gain !== 0 || stage.gain.param !== undefined) stages.push({ ...stage, gain });
|
||||||
}
|
}
|
||||||
return stages;
|
return stages;
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The chain one voice is sent along: the plan with every parameter filled in. A Mix that ends at 0
|
* The chain one voice is sent along: the plan with every parameter filled in. Its shape depends on
|
||||||
* or 1 is left out with the sound it silences, so that one is not even started.
|
* the plan alone, never on the values, so the values of a playing voice can change in place (#315).
|
||||||
*/
|
*/
|
||||||
export function fxChainOf(plan: FxPlan, params: FxParams = {}): FxChain {
|
export function fxChainOf(plan: FxPlan, params: FxParams = {}): FxChain {
|
||||||
const stages = stagesOf(plan, params);
|
const stages = stagesOf(plan, params);
|
||||||
@@ -263,10 +272,10 @@ export function fxChainOf(plan: FxPlan, params: FxParams = {}): FxChain {
|
|||||||
const pitch = plan.pitch.reduce((sum, amount) => sum + amountOf(amount, params), 0);
|
const pitch = plan.pitch.reduce((sum, amount) => sum + amountOf(amount, params), 0);
|
||||||
return { pitch, stages };
|
return { pitch, stages };
|
||||||
}
|
}
|
||||||
const mix = amountOf(plan.mix, params);
|
return {
|
||||||
if (mix <= 0 || mix >= 1) {
|
mix: amountOf(plan.mix, params),
|
||||||
const heard = fxChainOf(mix <= 0 ? plan.a : plan.b, params);
|
a: fxChainOf(plan.a, params),
|
||||||
return { ...heard, stages: [...heard.stages, ...stages] };
|
b: fxChainOf(plan.b, params),
|
||||||
}
|
stages,
|
||||||
return { mix, a: fxChainOf(plan.a, params), b: fxChainOf(plan.b, params), stages };
|
};
|
||||||
}
|
}
|
||||||
@@ -41,6 +41,8 @@ export function installDebugHook(app: Application, state: EditorState, editor: E
|
|||||||
sound: {
|
sound: {
|
||||||
stats: () => soundLibrary.stats(),
|
stats: () => soundLibrary.stats(),
|
||||||
duration: (file: string) => soundLibrary.duration(file),
|
duration: (file: string) => soundLibrary.duration(file),
|
||||||
|
/** Where the knobs of each voice of that file are now, oldest first, a pitch as its speed (#315). */
|
||||||
|
levels: (file: string) => soundLibrary.levels(file),
|
||||||
/** Decodes ahead of a check, so a play can be asserted without racing the network. */
|
/** Decodes ahead of a check, so a play can be asserted without racing the network. */
|
||||||
load: (file: string) => soundLibrary.load(file).then((buffer) => buffer?.duration ?? null),
|
load: (file: string) => soundLibrary.load(file).then((buffer) => buffer?.duration ?? null),
|
||||||
stopAll: () => soundLibrary.stopAll(),
|
stopAll: () => soundLibrary.stopAll(),
|
||||||
|
|||||||
@@ -3,7 +3,6 @@ import { findAssetOf } from '../../core/model/Project';
|
|||||||
import { resolveAssetProps } from '../../core/properties/resolve';
|
import { resolveAssetProps } from '../../core/properties/resolve';
|
||||||
import { volumeOf } from '../../core/properties/sounds';
|
import { volumeOf } from '../../core/properties/sounds';
|
||||||
import { soundFxBuildFor, soundFxParams } from '../../core/soundfx/compile';
|
import { soundFxBuildFor, soundFxParams } from '../../core/soundfx/compile';
|
||||||
import { fxChainOf } from '../../core/soundfx/nodes';
|
|
||||||
import { Button, Dropdown, HStack, Label, Menu, NumberField, Stack, Theme } from '../../ui';
|
import { Button, Dropdown, HStack, Label, Menu, NumberField, Stack, Theme } from '../../ui';
|
||||||
import { soundLibrary } from '../../viewport/SoundLibrary';
|
import { soundLibrary } from '../../viewport/SoundLibrary';
|
||||||
import { AssetTabEditor } from '../AssetTabEditor';
|
import { AssetTabEditor } from '../AssetTabEditor';
|
||||||
@@ -40,6 +39,8 @@ export class SoundFxEditor extends AssetTabEditor<SoundFxAsset, GraphCanvas> {
|
|||||||
private defaults: Record<string, number> = {};
|
private defaults: Record<string, number> = {};
|
||||||
private paramsKey = '';
|
private paramsKey = '';
|
||||||
private valuesFor = '';
|
private valuesFor = '';
|
||||||
|
/** The voices the tab started that may still be playing – what a moved field reaches. */
|
||||||
|
private voices: number[] = [];
|
||||||
|
|
||||||
constructor(state: EditorState) {
|
constructor(state: EditorState) {
|
||||||
super(state, { infoWidth: INFO_WIDTH });
|
super(state, { infoWidth: INFO_WIDTH });
|
||||||
@@ -89,8 +90,8 @@ export class SoundFxEditor extends AssetTabEditor<SoundFxAsset, GraphCanvas> {
|
|||||||
/** Sets one parameter the header plays with, as typing into its field does. */
|
/** Sets one parameter the header plays with, as typing into its field does. */
|
||||||
setPreviewValue(key: string, value: number): void {
|
setPreviewValue(key: string, value: number): void {
|
||||||
if (!(key in this.values)) return;
|
if (!(key in this.values)) return;
|
||||||
this.values[key] = value;
|
|
||||||
this.paramFields.get(key)?.setNumber(value);
|
this.paramFields.get(key)?.setNumber(value);
|
||||||
|
this.setValue(key, value);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Plays the picked sound through the graph; a graph that does not compile is not played at all. */
|
/** Plays the picked sound through the graph; a graph that does not compile is not played at all. */
|
||||||
@@ -100,11 +101,22 @@ export class SoundFxEditor extends AssetTabEditor<SoundFxAsset, GraphCanvas> {
|
|||||||
const plan = asset && soundFxBuildFor(asset).compiled;
|
const plan = asset && soundFxBuildFor(asset).compiled;
|
||||||
if (!sound?.file || !plan) return;
|
if (!sound?.file || !plan) return;
|
||||||
const volume = volumeOf(resolveAssetProps(this.state.project, sound));
|
const volume = volumeOf(resolveAssetProps(this.state.project, sound));
|
||||||
soundLibrary.play(sound.file, volume, false, PREVIEW_OWNER, fxChainOf(plan, this.values));
|
const voice = soundLibrary.play(sound.file, volume, false, PREVIEW_OWNER, {
|
||||||
|
plan,
|
||||||
|
params: { ...this.values },
|
||||||
|
});
|
||||||
|
if (voice) this.voices.push(voice);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** A moved field is heard at once on what the tab is playing (#315); an ended voice is let go. */
|
||||||
|
private setValue(key: string, value: number): void {
|
||||||
|
this.values[key] = value;
|
||||||
|
this.voices = this.voices.filter((voice) => soundLibrary.tune(voice, { [key]: value }));
|
||||||
}
|
}
|
||||||
|
|
||||||
stop(): void {
|
stop(): void {
|
||||||
soundLibrary.stopOwner(PREVIEW_OWNER);
|
soundLibrary.stopOwner(PREVIEW_OWNER);
|
||||||
|
this.voices = [];
|
||||||
}
|
}
|
||||||
|
|
||||||
protected override infoText(asset: SoundFxAsset): string {
|
protected override infoText(asset: SoundFxAsset): string {
|
||||||
@@ -152,8 +164,8 @@ export class SoundFxEditor extends AssetTabEditor<SoundFxAsset, GraphCanvas> {
|
|||||||
const field = new NumberField(values[port.key], {
|
const field = new NumberField(values[port.key], {
|
||||||
width: PARAM_WIDTH,
|
width: PARAM_WIDTH,
|
||||||
step: 0.1,
|
step: 0.1,
|
||||||
onInput: (value) => (this.values[port.key] = value),
|
onInput: (value) => this.setValue(port.key, value),
|
||||||
onChange: (value) => (this.values[port.key] = value),
|
onChange: (value) => this.setValue(port.key, value),
|
||||||
});
|
});
|
||||||
this.paramFields.set(port.key, field);
|
this.paramFields.set(port.key, field);
|
||||||
this.paramBar.add(field);
|
this.paramBar.add(field);
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
import type { FxChain } from '../../core/soundfx/nodes';
|
import type { FxParams, FxVoice } from '../../core/soundfx/nodes';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* What the runtime needs of the outside world to make a noise – the counterpart of `Input` on the
|
* What the runtime needs of the outside world to make a noise – the counterpart of `Input` on the
|
||||||
@@ -13,10 +13,17 @@ export interface AudioSink {
|
|||||||
* Starts one voice. `volume` is final (0–1) and every call overlaps whatever is already playing.
|
* Starts one voice. `volume` is final (0–1) and every call overlaps whatever is already playing.
|
||||||
* `owner` is the behavior the sound belongs to: when that behavior is destroyed the voice goes
|
* `owner` is the behavior the sound belongs to: when that behavior is destroyed the voice goes
|
||||||
* with it, because a sound is something a thing in the world is doing, not a thing of its own.
|
* with it, because a sound is something a thing in the world is doing, not a thing of its own.
|
||||||
* `''` means the voice has no owner and outlives whatever started it. `fx` is the line of effects
|
* `''` means the voice has no owner and outlives whatever started it. `fx` is the sound FX the
|
||||||
* the voice passes on its way out (#311); `DRY` passes none.
|
* voice passes on its way out (#311), with the values of its parameters; `DRY_VOICE` passes none.
|
||||||
|
* Returns the number of the voice, never used twice, or 0 where nothing is going to play.
|
||||||
*/
|
*/
|
||||||
play(file: string, volume: number, loop: boolean, owner: string, fx?: FxChain): void;
|
play(file: string, volume: number, loop: boolean, owner: string, fx?: FxVoice): number;
|
||||||
|
/**
|
||||||
|
* Hands new values to the sound FX parameters of one voice, by the number `play` gave it (#315).
|
||||||
|
* A name its FX does not have is left out, one not given keeps its value. A voice that has ended
|
||||||
|
* is not there any more and nothing happens – a detached one is reached like any other.
|
||||||
|
*/
|
||||||
|
tune(voice: number, params: FxParams): void;
|
||||||
/** Silences every voice of that file; `''` silences all of them. */
|
/** Silences every voice of that file; `''` silences all of them. */
|
||||||
stop(file: string): void;
|
stop(file: string): void;
|
||||||
/** Silences every voice started by that owner – what destroying a behavior does. */
|
/** Silences every voice started by that owner – what destroying a behavior does. */
|
||||||
@@ -24,7 +31,8 @@ export interface AudioSink {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export const SILENT_AUDIO: AudioSink = {
|
export const SILENT_AUDIO: AudioSink = {
|
||||||
play: () => {},
|
play: () => 0,
|
||||||
|
tune: () => {},
|
||||||
stop: () => {},
|
stop: () => {},
|
||||||
stopOwner: () => {},
|
stopOwner: () => {},
|
||||||
};
|
};
|
||||||
@@ -16,7 +16,7 @@ import { hasRenderLayer } from '../../core/properties/PropertySchema';
|
|||||||
import { resolveAssetProps } from '../../core/properties/resolve';
|
import { resolveAssetProps } from '../../core/properties/resolve';
|
||||||
import { loopsOf, volumeOf } from '../../core/properties/sounds';
|
import { loopsOf, volumeOf } from '../../core/properties/sounds';
|
||||||
import { soundFxBuildFor } from '../../core/soundfx/compile';
|
import { soundFxBuildFor } from '../../core/soundfx/compile';
|
||||||
import { DRY, fxChainOf, type FxParams } from '../../core/soundfx/nodes';
|
import { DRY_VOICE, type FxParams } from '../../core/soundfx/nodes';
|
||||||
import type { Entity } from '../Entity';
|
import type { Entity } from '../Entity';
|
||||||
import { PhysicsSystem } from '../physics/PhysicsSystem';
|
import { PhysicsSystem } from '../physics/PhysicsSystem';
|
||||||
import type { World } from '../World';
|
import type { World } from '../World';
|
||||||
@@ -172,6 +172,7 @@ export function behaviorContext(
|
|||||||
const file = assetId === '' ? '' : soundFileOf(project, assetId);
|
const file = assetId === '' ? '' : soundFileOf(project, assetId);
|
||||||
if (assetId === '' || file) run.audio.stop(file);
|
if (assetId === '' || file) run.audio.stop(file);
|
||||||
},
|
},
|
||||||
|
tuneSound: (voice, fxParams) => run.audio.tune(voice, fxParams),
|
||||||
spawn: (assetId, position, parent) => world.spawn(assetId, position, resolve(parent) ?? null),
|
spawn: (assetId, position, parent) => world.spawn(assetId, position, resolve(parent) ?? null),
|
||||||
addJoint: (assetId, owner, connected, at) => {
|
addJoint: (assetId, owner, connected, at) => {
|
||||||
const on = resolve(owner);
|
const on = resolve(owner);
|
||||||
@@ -218,7 +219,7 @@ export function behaviorContext(
|
|||||||
* node's, and its loop flag unless the node overrules it either way. The asset is what a sound
|
* node's, and its loop flag unless the node overrules it either way. The asset is what a sound
|
||||||
* *is* – a node that says nothing therefore plays it as the project describes it. An FX asset that
|
* *is* – a node that says nothing therefore plays it as the project describes it. An FX asset that
|
||||||
* does not compile plays the sound dry: a broken effect should not silence the game. Its parameters
|
* does not compile plays the sound dry: a broken effect should not silence the game. Its parameters
|
||||||
* take the values handed in, the others their defaults.
|
* take the values handed in, the others their defaults. Returns the number of the voice, 0 for none.
|
||||||
*/
|
*/
|
||||||
function playSound(
|
function playSound(
|
||||||
{ project, audio }: ContextSources,
|
{ project, audio }: ContextSources,
|
||||||
@@ -228,14 +229,20 @@ function playSound(
|
|||||||
owner: string,
|
owner: string,
|
||||||
fxId: string,
|
fxId: string,
|
||||||
fxParams: FxParams,
|
fxParams: FxParams,
|
||||||
): void {
|
): number {
|
||||||
const asset = findAssetOf(project, assetId, 'sound');
|
const asset = findAssetOf(project, assetId, 'sound');
|
||||||
if (!asset?.file) return;
|
if (!asset?.file) return 0;
|
||||||
const props = resolveAssetProps(project, asset);
|
const props = resolveAssetProps(project, asset);
|
||||||
const level = Math.max(0, Math.min(1, volumeOf(props) * volume));
|
const level = Math.max(0, Math.min(1, volumeOf(props) * volume));
|
||||||
const fx = fxId ? findAssetOf(project, fxId, 'soundfx') : undefined;
|
const fx = fxId ? findAssetOf(project, fxId, 'soundfx') : undefined;
|
||||||
const plan = fx && soundFxBuildFor(fx).compiled;
|
const plan = fx && soundFxBuildFor(fx).compiled;
|
||||||
audio.play(asset.file, level, loop ?? loopsOf(props), owner, plan ? fxChainOf(plan, fxParams) : DRY);
|
return audio.play(
|
||||||
|
asset.file,
|
||||||
|
level,
|
||||||
|
loop ?? loopsOf(props),
|
||||||
|
owner,
|
||||||
|
plan ? { plan, params: fxParams } : DRY_VOICE,
|
||||||
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Where a ray of that length ends; where it has no direction, where it starts. */
|
/** Where a ray of that length ends; where it has no direction, where it starts. */
|
||||||
|
|||||||
+143
-31
@@ -1,34 +1,62 @@
|
|||||||
import { DRY, type FxChain, type FxStage } from '../core/soundfx/nodes';
|
import {
|
||||||
|
DRY_VOICE,
|
||||||
|
fxChainOf,
|
||||||
|
type FxChain,
|
||||||
|
type FxParams,
|
||||||
|
type FxStage,
|
||||||
|
type FxVoice,
|
||||||
|
} from '../core/soundfx/nodes';
|
||||||
import { pitchShift } from '../core/soundfx/pitchShift';
|
import { pitchShift } from '../core/soundfx/pitchShift';
|
||||||
import type { AudioSink } from '../runtime/systems/Audio';
|
import type { AudioSink } from '../runtime/systems/Audio';
|
||||||
|
|
||||||
const DEFAULT_BASE = '/game-maker/data/sounds';
|
const DEFAULT_BASE = '/game-maker/data/sounds';
|
||||||
/** How many pitched copies are kept at once. */
|
/** How many pitched copies are kept at once. */
|
||||||
const SHIFTED_KEPT = 32;
|
const SHIFTED_KEPT = 32;
|
||||||
|
/** Seconds a changed parameter takes to get most of the way there: quick, but no click (#315). */
|
||||||
|
const GLIDE = 0.03;
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One number of a voice that can still change while it plays (#315): a level, a share of a Mix, the
|
||||||
|
* gain of a band – or the pitch of a source, which glides by speed from the pitch its copy was made at.
|
||||||
|
*/
|
||||||
|
type Knob =
|
||||||
|
| { kind: 'level'; param: AudioParam; target: number }
|
||||||
|
| { kind: 'pitch'; source: AudioBufferSourceNode; base: number; target: number };
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* One sound in flight: its sources, its own gain, the file it came from and who is making it. A
|
* One sound in flight: its sources, its own gain, the file it came from and who is making it. A
|
||||||
* voice has a source per pitch its sound FX plays the file at (#316), started together and as long
|
* voice has a source per way through its sound FX (#316), started together and as long as each
|
||||||
* as each other.
|
* other. Its knobs are in the order `levelsOf` lists the numbers of its chain.
|
||||||
*/
|
*/
|
||||||
interface Voice {
|
interface Voice {
|
||||||
|
/** What `play` returned for it, the handle a `tune` reaches it by. */
|
||||||
|
id: number;
|
||||||
file: string;
|
file: string;
|
||||||
/** The behavior this voice belongs to; `''` for one nothing owns (see `AudioSink.play`). */
|
/** The behavior this voice belongs to; `''` for one nothing owns (see `AudioSink.play`). */
|
||||||
owner: string;
|
owner: string;
|
||||||
sources: AudioBufferSourceNode[];
|
sources: AudioBufferSourceNode[];
|
||||||
gain: GainNode;
|
gain: GainNode;
|
||||||
|
fx: FxVoice;
|
||||||
|
knobs: Knob[];
|
||||||
}
|
}
|
||||||
|
|
||||||
/** What one voice is wired from: the file, and its sources by pitch so far. */
|
/** What one voice is wired from: the file, and what has been made for it so far. */
|
||||||
interface Wiring {
|
interface Wiring {
|
||||||
context: AudioContext;
|
context: AudioContext;
|
||||||
file: string;
|
file: string;
|
||||||
buffer: AudioBuffer;
|
buffer: AudioBuffer;
|
||||||
loop: boolean;
|
loop: boolean;
|
||||||
sources: Map<number, AudioBufferSourceNode>;
|
sources: { source: AudioBufferSourceNode; pitch: number }[];
|
||||||
|
knobs: Knob[];
|
||||||
stages: number;
|
stages: number;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/** A play still waiting on its bytes: what a `tune` in the meantime writes into. */
|
||||||
|
interface Pending {
|
||||||
|
id: number;
|
||||||
|
fx: FxVoice;
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Plays uploaded sounds, keyed by file name – the texture library's twin on the audio side. Files
|
* Plays uploaded sounds, keyed by file name – the texture library's twin on the audio side. Files
|
||||||
* are content addressed, so a decoded buffer can never go stale and nothing is evicted.
|
* are content addressed, so a decoded buffer can never go stale and nothing is evicted.
|
||||||
@@ -52,8 +80,12 @@ export class SoundLibrary implements AudioSink {
|
|||||||
/** Never retried, so one broken reference cannot spam the network. */
|
/** Never retried, so one broken reference cannot spam the network. */
|
||||||
private readonly failed = new Set<string>();
|
private readonly failed = new Set<string>();
|
||||||
private voices: Voice[] = [];
|
private voices: Voice[] = [];
|
||||||
|
private pending: Pending[] = [];
|
||||||
|
/** The number the next play is given; 0 stays free for "nothing plays". */
|
||||||
|
private nextId = 1;
|
||||||
private started = 0;
|
private started = 0;
|
||||||
private stopped = 0;
|
private stopped = 0;
|
||||||
|
private tuned = 0;
|
||||||
/** The last voice actually started, with the values it was started at – what a check reads back. */
|
/** The last voice actually started, with the values it was started at – what a check reads back. */
|
||||||
private last: LastVoice | null = null;
|
private last: LastVoice | null = null;
|
||||||
/**
|
/**
|
||||||
@@ -73,20 +105,44 @@ export class SoundLibrary implements AudioSink {
|
|||||||
this.base = base.replace(/\/+$/, '');
|
this.base = base.replace(/\/+$/, '');
|
||||||
}
|
}
|
||||||
|
|
||||||
play(file: string, volume: number, loop: boolean, owner = '', fx: FxChain = DRY): void {
|
/**
|
||||||
if (file === '' || this.failed.has(file)) return;
|
* The number is handed out at once, before the bytes are there: a graph can tune a voice in the
|
||||||
|
* same frame it asked for it, and the voice then starts with those values.
|
||||||
|
*/
|
||||||
|
play(file: string, volume: number, loop: boolean, owner = '', fx: FxVoice = DRY_VOICE): number {
|
||||||
|
if (file === '' || this.failed.has(file)) return 0;
|
||||||
|
const id = this.nextId++;
|
||||||
const buffer = this.buffers.get(file);
|
const buffer = this.buffers.get(file);
|
||||||
if (buffer) {
|
if (buffer) {
|
||||||
this.start(file, buffer, volume, loop, owner, fx);
|
this.start(id, file, buffer, volume, loop, owner, fx);
|
||||||
return;
|
return id;
|
||||||
}
|
}
|
||||||
const askedAt = this.cancelsFor(file, owner);
|
const askedAt = this.cancelsFor(file, owner);
|
||||||
|
const waiting: Pending = { id, fx: { plan: fx.plan, params: { ...fx.params } } };
|
||||||
|
this.pending.push(waiting);
|
||||||
void this.load(file).then((loaded) => {
|
void this.load(file).then((loaded) => {
|
||||||
|
this.pending = this.pending.filter((other) => other !== waiting);
|
||||||
// A `Stop sound` – or the destruction of the behavior that asked – that happened while the
|
// A `Stop sound` – or the destruction of the behavior that asked – that happened while the
|
||||||
// bytes were in flight has to win, or a sound the run already ended would start afterwards.
|
// bytes were in flight has to win, or a sound the run already ended would start afterwards.
|
||||||
if (loaded && this.cancelsFor(file, owner) === askedAt)
|
if (loaded && this.cancelsFor(file, owner) === askedAt)
|
||||||
this.start(file, loaded, volume, loop, owner, fx);
|
this.start(id, file, loaded, volume, loop, owner, waiting.fx);
|
||||||
});
|
});
|
||||||
|
return id;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Says whether the voice was still there to be reached – playing, or waiting on its bytes. */
|
||||||
|
tune(id: number, params: FxParams): boolean {
|
||||||
|
const waiting = this.pending.find((other) => other.id === id);
|
||||||
|
if (waiting) {
|
||||||
|
waiting.fx = { plan: waiting.fx.plan, params: { ...waiting.fx.params, ...params } };
|
||||||
|
return true;
|
||||||
|
}
|
||||||
|
const voice = this.voices.find((other) => other.id === id);
|
||||||
|
if (!voice) return false;
|
||||||
|
voice.fx = { plan: voice.fx.plan, params: { ...voice.fx.params, ...params } };
|
||||||
|
this.glide(voice);
|
||||||
|
this.tuned += 1;
|
||||||
|
return true;
|
||||||
}
|
}
|
||||||
|
|
||||||
stop(file: string): void {
|
stop(file: string): void {
|
||||||
@@ -123,6 +179,7 @@ export class SoundLibrary implements AudioSink {
|
|||||||
playing: number;
|
playing: number;
|
||||||
started: number;
|
started: number;
|
||||||
stopped: number;
|
stopped: number;
|
||||||
|
tuned: number;
|
||||||
decoded: number;
|
decoded: number;
|
||||||
failed: number;
|
failed: number;
|
||||||
last: LastVoice | null;
|
last: LastVoice | null;
|
||||||
@@ -131,12 +188,25 @@ export class SoundLibrary implements AudioSink {
|
|||||||
playing: this.voices.length,
|
playing: this.voices.length,
|
||||||
started: this.started,
|
started: this.started,
|
||||||
stopped: this.stopped,
|
stopped: this.stopped,
|
||||||
|
tuned: this.tuned,
|
||||||
decoded: this.buffers.size,
|
decoded: this.buffers.size,
|
||||||
failed: this.failed.size,
|
failed: this.failed.size,
|
||||||
last: this.last,
|
last: this.last,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Where the knobs of each voice of that file are right now, oldest first, a pitch as the speed it
|
||||||
|
* plays at – what a check reads to see a change arrive at one voice and not another (test hook).
|
||||||
|
*/
|
||||||
|
levels(file: string): number[][] {
|
||||||
|
return this.voices
|
||||||
|
.filter((voice) => voice.file === file)
|
||||||
|
.map((voice) =>
|
||||||
|
voice.knobs.map((knob) => (knob.kind === 'pitch' ? knob.source.playbackRate : knob.param).value),
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
/** How long a decoded sound runs, in seconds; null while it has not been loaded (test hook). */
|
/** How long a decoded sound runs, in seconds; null while it has not been loaded (test hook). */
|
||||||
duration(file: string): number | null {
|
duration(file: string): number | null {
|
||||||
return this.buffers.get(file)?.duration ?? null;
|
return this.buffers.get(file)?.duration ?? null;
|
||||||
@@ -177,12 +247,13 @@ export class SoundLibrary implements AudioSink {
|
|||||||
}
|
}
|
||||||
|
|
||||||
private start(
|
private start(
|
||||||
|
id: number,
|
||||||
file: string,
|
file: string,
|
||||||
buffer: AudioBuffer,
|
buffer: AudioBuffer,
|
||||||
volume: number,
|
volume: number,
|
||||||
loop: boolean,
|
loop: boolean,
|
||||||
owner: string,
|
owner: string,
|
||||||
fx: FxChain,
|
fx: FxVoice,
|
||||||
): void {
|
): void {
|
||||||
const context = this.audio();
|
const context = this.audio();
|
||||||
// A context starts suspended where the page has not been interacted with; without this the
|
// A context starts suspended where the page has not been interacted with; without this the
|
||||||
@@ -190,18 +261,18 @@ export class SoundLibrary implements AudioSink {
|
|||||||
if (context.state === 'suspended') void context.resume();
|
if (context.state === 'suspended') void context.resume();
|
||||||
const gain = context.createGain();
|
const gain = context.createGain();
|
||||||
gain.gain.value = Math.max(0, Math.min(1, volume));
|
gain.gain.value = Math.max(0, Math.min(1, volume));
|
||||||
const wiring: Wiring = { context, file, buffer, loop, sources: new Map(), stages: 0 };
|
const wiring: Wiring = { context, file, buffer, loop, sources: [], knobs: [], stages: 0 };
|
||||||
this.wire(wiring, fx).connect(gain).connect(context.destination);
|
this.wire(wiring, fxChainOf(fx.plan, fx.params)).connect(gain).connect(context.destination);
|
||||||
const sources = [...wiring.sources.values()];
|
const sources = wiring.sources.map((made) => made.source);
|
||||||
this.last = {
|
this.last = {
|
||||||
file,
|
file,
|
||||||
volume: gain.gain.value,
|
volume: gain.gain.value,
|
||||||
loop,
|
loop,
|
||||||
pitches: [...wiring.sources.keys()],
|
pitches: wiring.sources.map((made) => made.pitch),
|
||||||
stages: wiring.stages,
|
stages: wiring.stages,
|
||||||
duration: sources[0]!.buffer!.duration,
|
duration: sources[0]!.buffer!.duration,
|
||||||
};
|
};
|
||||||
const voice: Voice = { file, owner, sources, gain };
|
const voice: Voice = { id, file, owner, sources, gain, fx, knobs: wiring.knobs };
|
||||||
let playing = sources.length;
|
let playing = sources.length;
|
||||||
for (const source of sources)
|
for (const source of sources)
|
||||||
source.onended = () => {
|
source.onended = () => {
|
||||||
@@ -213,35 +284,60 @@ export class SoundLibrary implements AudioSink {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The nodes a chain becomes, up to the one that carries its sound on. Two ways at the same pitch
|
* The nodes a chain becomes, up to the one that carries its sound on. Every way has a source of
|
||||||
* share a source; a Mix weighs each of its sounds by its share before they are summed.
|
* its own, so each can glide to its own pitch; a Mix weighs each of its sounds by its share before
|
||||||
|
* they are summed.
|
||||||
*/
|
*/
|
||||||
private wire(wiring: Wiring, chain: FxChain): AudioNode {
|
private wire(wiring: Wiring, chain: FxChain): AudioNode {
|
||||||
const { context } = wiring;
|
const { context, knobs } = wiring;
|
||||||
let tail: AudioNode;
|
let tail: AudioNode;
|
||||||
if ('mix' in chain) {
|
if ('mix' in chain) {
|
||||||
tail = context.createGain();
|
tail = context.createGain();
|
||||||
this.wire(wiring, chain.a)
|
const wayA = this.wire(wiring, chain.a);
|
||||||
.connect(gainNode(context, 1 - chain.mix))
|
const shareA = gainNode(context, 1 - chain.mix);
|
||||||
.connect(tail);
|
wayA.connect(shareA).connect(tail);
|
||||||
this.wire(wiring, chain.b).connect(gainNode(context, chain.mix)).connect(tail);
|
knobs.push({ kind: 'level', param: shareA.gain, target: 1 - chain.mix });
|
||||||
|
const wayB = this.wire(wiring, chain.b);
|
||||||
|
const shareB = gainNode(context, chain.mix);
|
||||||
|
wayB.connect(shareB).connect(tail);
|
||||||
|
knobs.push({ kind: 'level', param: shareB.gain, target: chain.mix });
|
||||||
} else tail = this.source(wiring, chain.pitch);
|
} else tail = this.source(wiring, chain.pitch);
|
||||||
for (const stage of chain.stages) tail = tail.connect(fxNode(context, stage));
|
for (const stage of chain.stages) {
|
||||||
|
const [node, param] = fxNode(context, stage);
|
||||||
|
tail = tail.connect(node);
|
||||||
|
knobs.push({ kind: 'level', param, target: stage.gain });
|
||||||
|
}
|
||||||
wiring.stages += chain.stages.length;
|
wiring.stages += chain.stages.length;
|
||||||
return tail;
|
return tail;
|
||||||
}
|
}
|
||||||
|
|
||||||
private source(wiring: Wiring, pitch: number): AudioBufferSourceNode {
|
private source(wiring: Wiring, pitch: number): AudioBufferSourceNode {
|
||||||
const held = wiring.sources.get(pitch);
|
|
||||||
if (held) return held;
|
|
||||||
const { context, file, buffer, loop } = wiring;
|
const { context, file, buffer, loop } = wiring;
|
||||||
const source = context.createBufferSource();
|
const source = context.createBufferSource();
|
||||||
source.buffer = pitch === 0 ? buffer : this.pitched(context, file, buffer, pitch, loop);
|
source.buffer = pitch === 0 ? buffer : this.pitched(context, file, buffer, pitch, loop);
|
||||||
source.loop = loop;
|
source.loop = loop;
|
||||||
wiring.sources.set(pitch, source);
|
wiring.sources.push({ source, pitch });
|
||||||
|
wiring.knobs.push({ kind: 'pitch', source, base: pitch, target: pitch });
|
||||||
return source;
|
return source;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Sends every knob of a voice towards what its values say now. A pitch cannot get a new copy in
|
||||||
|
* the middle of a sound, so it glides by speed: the voice gets that much shorter or longer.
|
||||||
|
*/
|
||||||
|
private glide(voice: Voice): void {
|
||||||
|
const levels = levelsOf(fxChainOf(voice.fx.plan, voice.fx.params));
|
||||||
|
const now = this.audio().currentTime;
|
||||||
|
voice.knobs.forEach((knob, i) => {
|
||||||
|
const level = levels[i];
|
||||||
|
if (level === undefined || level === knob.target) return;
|
||||||
|
knob.target = level;
|
||||||
|
if (knob.kind === 'pitch')
|
||||||
|
knob.source.playbackRate.setTargetAtTime(2 ** ((level - knob.base) / 12), now, GLIDE);
|
||||||
|
else knob.param.setTargetAtTime(level, now, GLIDE);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The sound at another pitch and its own length (#312), worked out once per file, pitch and loop
|
* The sound at another pitch and its own length (#312), worked out once per file, pitch and loop
|
||||||
* and kept for the next voice. The oldest goes first, so a pitch drawn at random every play cannot
|
* and kept for the next voice. The oldest goes first, so a pitch drawn at random every play cannot
|
||||||
@@ -286,7 +382,7 @@ interface LastVoice {
|
|||||||
file: string;
|
file: string;
|
||||||
volume: number;
|
volume: number;
|
||||||
loop: boolean;
|
loop: boolean;
|
||||||
/** Semitones of each source, one per pitch (#316), and how many gain and filter nodes it passed. */
|
/** Semitones of each source, one per way (#316), and how many gain and filter nodes it passed. */
|
||||||
pitches: number[];
|
pitches: number[];
|
||||||
stages: number;
|
stages: number;
|
||||||
/** Seconds of what was played: a pitched voice is as long as the sound itself (#312). */
|
/** Seconds of what was played: a pitched voice is as long as the sound itself (#312). */
|
||||||
@@ -299,14 +395,30 @@ function gainNode(context: AudioContext, level: number): GainNode {
|
|||||||
return gain;
|
return gain;
|
||||||
}
|
}
|
||||||
|
|
||||||
function fxNode(context: AudioContext, stage: FxStage): AudioNode {
|
/** The node a stage becomes, with the number of it a parameter can move. */
|
||||||
if (stage.kind === 'gain') return gainNode(context, stage.gain);
|
function fxNode(context: AudioContext, stage: FxStage): [AudioNode, AudioParam] {
|
||||||
|
if (stage.kind === 'gain') {
|
||||||
|
const gain = gainNode(context, stage.gain);
|
||||||
|
return [gain, gain.gain];
|
||||||
|
}
|
||||||
const filter = context.createBiquadFilter();
|
const filter = context.createBiquadFilter();
|
||||||
filter.type = stage.filter;
|
filter.type = stage.filter;
|
||||||
filter.frequency.value = stage.frequency;
|
filter.frequency.value = stage.frequency;
|
||||||
filter.gain.value = stage.gain;
|
filter.gain.value = stage.gain;
|
||||||
filter.Q.value = stage.q;
|
filter.Q.value = stage.q;
|
||||||
return filter;
|
return [filter, filter.gain];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** The numbers of a chain in the order `wire` makes their knobs: a stage's `gain` is what its node holds. */
|
||||||
|
function levelsOf(chain: FxChain, into: number[] = []): number[] {
|
||||||
|
if ('mix' in chain) {
|
||||||
|
levelsOf(chain.a, into);
|
||||||
|
into.push(1 - chain.mix);
|
||||||
|
levelsOf(chain.b, into);
|
||||||
|
into.push(chain.mix);
|
||||||
|
} else into.push(chain.pitch);
|
||||||
|
for (const stage of chain.stages) into.push(stage.gain);
|
||||||
|
return into;
|
||||||
}
|
}
|
||||||
|
|
||||||
export const soundLibrary = new SoundLibrary();
|
export const soundLibrary = new SoundLibrary();
|
||||||
@@ -54,7 +54,8 @@ function keep(f: Fixture, sceneId: string, objectId: string): void {
|
|||||||
function start(f: Fixture, audio?: AudioSink) {
|
function start(f: Fixture, audio?: AudioSink) {
|
||||||
const stoppedOwners: string[] = [];
|
const stoppedOwners: string[] = [];
|
||||||
const sink: AudioSink = audio ?? {
|
const sink: AudioSink = audio ?? {
|
||||||
play: () => {},
|
play: () => 0,
|
||||||
|
tune: () => {},
|
||||||
stop: () => {},
|
stop: () => {},
|
||||||
stopOwner: (owner: string) => stoppedOwners.push(owner),
|
stopOwner: (owner: string) => stoppedOwners.push(owner),
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -1940,12 +1940,14 @@ const CASES: Record<string, Case[]> = {
|
|||||||
name: 'no sound is a no-op that still continues the flow',
|
name: 'no sound is a no-op that still continues the flow',
|
||||||
inputs: { sound: '', volume: 1, loop: '' },
|
inputs: { sound: '', volume: 1, loop: '' },
|
||||||
next: 'exec',
|
next: 'exec',
|
||||||
|
outputs: { voice: 0 },
|
||||||
after: (ctx) => expect(ctx.played).toEqual([]),
|
after: (ctx) => expect(ctx.played).toEqual([]),
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: 'hands the id on with the volume and loop it was given',
|
name: 'hands the id on with the volume and loop it was given, and gives out the voice (#315)',
|
||||||
inputs: { sound: 'snd-1', volume: 0.5, loop: 'true' },
|
inputs: { sound: 'snd-1', volume: 0.5, loop: 'true' },
|
||||||
next: 'exec',
|
next: 'exec',
|
||||||
|
outputs: { voice: 1 },
|
||||||
// What the two mean – a factor on the asset's volume, an answer that overrules its loop
|
// What the two mean – a factor on the asset's volume, an answer that overrules its loop
|
||||||
// flag – is the BehaviorSystem's to resolve, since only it holds the project (see
|
// flag – is the BehaviorSystem's to resolve, since only it holds the project (see
|
||||||
// `sounds.test.ts`).
|
// `sounds.test.ts`).
|
||||||
@@ -2025,6 +2027,20 @@ const CASES: Record<string, Case[]> = {
|
|||||||
after: (ctx) => expect(ctx.silenced).toEqual(['']),
|
after: (ctx) => expect(ctx.silenced).toEqual(['']),
|
||||||
},
|
},
|
||||||
],
|
],
|
||||||
|
SetSoundFxParameters: [
|
||||||
|
{
|
||||||
|
name: 'hands the numbers of the bundle to the voice it is given',
|
||||||
|
inputs: { voice: 4, fxParams: { depth: 3, label: 'x' } },
|
||||||
|
next: 'exec',
|
||||||
|
after: (ctx) => expect(ctx.tuned).toEqual([{ voice: 4, fxParams: { depth: 3 } }]),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
name: 'voice 0 – nothing was played – reaches nothing, and the flow carries on',
|
||||||
|
inputs: { voice: 0, fxParams: { depth: 3 } },
|
||||||
|
next: 'exec',
|
||||||
|
after: (ctx) => expect(ctx.tuned).toEqual([]),
|
||||||
|
},
|
||||||
|
],
|
||||||
Log: [
|
Log: [
|
||||||
{
|
{
|
||||||
name: 'logs its message and continues',
|
name: 'logs its message and continues',
|
||||||
|
|||||||
+81
-16
@@ -12,7 +12,14 @@ import type { BehaviorAsset, SoundFxAsset } from '../../src/core/model/Asset';
|
|||||||
import { isPlaceableType } from '../../src/core/model/AssetTypes';
|
import { isPlaceableType } from '../../src/core/model/AssetTypes';
|
||||||
import { assetRefChoices } from '../../src/core/properties/assetRef';
|
import { assetRefChoices } from '../../src/core/properties/assetRef';
|
||||||
import { compileSoundFx, soundFxBuildFor } from '../../src/core/soundfx/compile';
|
import { compileSoundFx, soundFxBuildFor } from '../../src/core/soundfx/compile';
|
||||||
import { DRY, EMPTY_PLAN, type FxChain, fxChainOf, type FxPlan } from '../../src/core/soundfx/nodes';
|
import {
|
||||||
|
DRY,
|
||||||
|
EMPTY_PLAN,
|
||||||
|
type FxChain,
|
||||||
|
fxChainOf,
|
||||||
|
type FxPlan,
|
||||||
|
type FxVoice,
|
||||||
|
} from '../../src/core/soundfx/nodes';
|
||||||
import { deserializeProject } from '../../src/core/store/serialize';
|
import { deserializeProject } from '../../src/core/store/serialize';
|
||||||
import type { AudioSink } from '../../src/runtime/systems/Audio';
|
import type { AudioSink } from '../../src/runtime/systems/Audio';
|
||||||
import { paletteEntries } from '../../src/editor/graph/paletteEntries';
|
import { paletteEntries } from '../../src/editor/graph/paletteEntries';
|
||||||
@@ -214,15 +221,15 @@ describe('a Mix in a sound FX graph (#316)', () => {
|
|||||||
expect(b).toMatchObject({ pitch: 0, stages: [{ kind: 'gain' }, { kind: 'filter' }] });
|
expect(b).toMatchObject({ pitch: 0, stages: [{ kind: 'gain' }, { kind: 'filter' }] });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('takes its share from a parameter, held between 0 and 1', () => {
|
it('takes its share from a parameter, held between 0 and 1, and keeps both ways for it (#315)', () => {
|
||||||
const f = forked();
|
const f = forked();
|
||||||
f.input.ports = [{ key: 'blend', kind: 'number', default: 0.25 }];
|
f.input.ports = [{ key: 'blend', kind: 'number', default: 0.25 }];
|
||||||
f.edit.connect(f.input, 'blend', f.mix, 'mix');
|
f.edit.connect(f.input, 'blend', f.mix, 'mix');
|
||||||
const plan = compileSoundFx(f.fx.graph).compiled!;
|
const plan = compileSoundFx(f.fx.graph).compiled!;
|
||||||
expect(fxChainOf(plan)).toMatchObject({ mix: 0.25 });
|
expect(fxChainOf(plan)).toMatchObject({ mix: 0.25 });
|
||||||
expect(fxChainOf(plan, { blend: 0.8 })).toMatchObject({ mix: 0.8 });
|
expect(fxChainOf(plan, { blend: 0.8 })).toMatchObject({ mix: 0.8 });
|
||||||
expect(fxChainOf(plan, { blend: -3 })).toMatchObject({ pitch: 12 });
|
expect(fxChainOf(plan, { blend: -3 })).toMatchObject({ mix: 0, a: { pitch: 12 }, b: { pitch: 0 } });
|
||||||
expect(fxChainOf(plan, { blend: 3 })).toMatchObject({ pitch: 0 });
|
expect(fxChainOf(plan, { blend: 3 })).toMatchObject({ mix: 1, a: { pitch: 12 }, b: { pitch: 0 } });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('hands a Pitch behind it to both of its sounds', () => {
|
it('hands a Pitch behind it to both of its sounds', () => {
|
||||||
@@ -249,12 +256,20 @@ describe('a Mix in a sound FX graph (#316)', () => {
|
|||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
/** Records what it was asked to play, with the line of effects. */
|
/**
|
||||||
function fakeAudio(): AudioSink & { played: { file: string; fx: FxChain | undefined }[] } {
|
* Records what it was asked to play, with the line of effects its values make, and every tune. A
|
||||||
|
* voice is numbered by how many have been played, so the first is 1.
|
||||||
|
*/
|
||||||
|
function fakeAudio(): AudioSink & {
|
||||||
|
played: { file: string; owner: string; fx: FxChain | undefined }[];
|
||||||
|
tuned: { voice: number; params: Readonly<Record<string, number>> }[];
|
||||||
|
} {
|
||||||
const sink = {
|
const sink = {
|
||||||
played: [] as { file: string; fx: FxChain | undefined }[],
|
played: [] as { file: string; owner: string; fx: FxChain | undefined }[],
|
||||||
play: (file: string, _volume: number, _loop: boolean, _owner: string, fx?: FxChain) =>
|
tuned: [] as { voice: number; params: Readonly<Record<string, number>> }[],
|
||||||
sink.played.push({ file, fx }),
|
play: (file: string, _volume: number, _loop: boolean, owner: string, fx?: FxVoice) =>
|
||||||
|
sink.played.push({ file, owner, fx: fx && fxChainOf(fx.plan, fx.params) }),
|
||||||
|
tune: (voice: number, params: Readonly<Record<string, number>>) => sink.tuned.push({ voice, params }),
|
||||||
stop: () => {},
|
stop: () => {},
|
||||||
stopOwner: () => {},
|
stopOwner: () => {},
|
||||||
};
|
};
|
||||||
@@ -286,7 +301,7 @@ describe('Play sound with fx', () => {
|
|||||||
const f = playingScene((s) => s.fx.id);
|
const f = playingScene((s) => s.fx.id);
|
||||||
const audio = fakeAudio();
|
const audio = fakeAudio();
|
||||||
startWorld(f, { behaviors: true, audio }).frame();
|
startWorld(f, { behaviors: true, audio }).frame();
|
||||||
expect(audio.played).toEqual([{ file: FILE, fx: { pitch: -12, stages: [] } }]);
|
expect(audio.played).toMatchObject([{ file: FILE, fx: { pitch: -12, stages: [] } }]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('plays it dry without an fx, with one that is gone, and with one that does not compile', () => {
|
it('plays it dry without an fx, with one that is gone, and with one that does not compile', () => {
|
||||||
@@ -305,7 +320,8 @@ describe('Play sound with fx', () => {
|
|||||||
expect(soundFxBuildFor(broken.fx).compiled).toBeNull();
|
expect(soundFxBuildFor(broken.fx).compiled).toBeNull();
|
||||||
const brokenAudio = fakeAudio();
|
const brokenAudio = fakeAudio();
|
||||||
startWorld(broken, { behaviors: true, audio: brokenAudio }).frame();
|
startWorld(broken, { behaviors: true, audio: brokenAudio }).frame();
|
||||||
for (const audio of [none, gone, brokenAudio]) expect(audio.played).toEqual([{ file: FILE, fx: DRY }]);
|
for (const audio of [none, gone, brokenAudio])
|
||||||
|
expect(audio.played).toMatchObject([{ file: FILE, fx: DRY }]);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -329,11 +345,11 @@ describe('the parameters of a sound FX', () => {
|
|||||||
expect(pitchOf(fxChainOf(plan, { other: 3 }))).toBe(-12);
|
expect(pitchOf(fxChainOf(plan, { other: 3 }))).toBe(-12);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('keep a stage that is 0 by default, so a value handed in can open it', () => {
|
it('keep a stage that is 0 by default, so a value handed in can open it, even on a playing voice', () => {
|
||||||
const f = withParam('SoundFxGain', 'gain', [{ key: 'loud', kind: 'number' }]);
|
const f = withParam('SoundFxGain', 'gain', [{ key: 'loud', kind: 'number' }]);
|
||||||
const plan = soundFxBuildFor(f.fx).compiled!;
|
const plan = soundFxBuildFor(f.fx).compiled!;
|
||||||
expect(plan.stages).toHaveLength(1);
|
expect(plan.stages).toHaveLength(1);
|
||||||
expect(fxChainOf(plan)).toEqual(DRY);
|
expect(fxChainOf(plan)).toEqual({ pitch: 0, stages: [{ kind: 'gain', gain: 1 }] });
|
||||||
expect(fxChainOf(plan, { loud: 6 }).stages[0]).toMatchObject({ kind: 'gain' });
|
expect(fxChainOf(plan, { loud: 6 }).stages[0]).toMatchObject({ kind: 'gain' });
|
||||||
expect((fxChainOf(plan, { loud: 6 }).stages[0] as { gain: number }).gain).toBeCloseTo(10 ** (6 / 20));
|
expect((fxChainOf(plan, { loud: 6 }).stages[0] as { gain: number }).gain).toBeCloseTo(10 ** (6 / 20));
|
||||||
});
|
});
|
||||||
@@ -437,7 +453,9 @@ describe('Sound FX parameters and Play sound', () => {
|
|||||||
const f = scene({ pick: ['depth'], values: { depth: 7 } });
|
const f = scene({ pick: ['depth'], values: { depth: 7 } });
|
||||||
const audio = fakeAudio();
|
const audio = fakeAudio();
|
||||||
startWorld(f, { behaviors: true, audio }).frame();
|
startWorld(f, { behaviors: true, audio }).frame();
|
||||||
expect(audio.played).toEqual([{ file: FILE, fx: { pitch: 7, stages: [] } }]);
|
expect(audio.played).toMatchObject([
|
||||||
|
{ file: FILE, fx: { pitch: 7, stages: [{ kind: 'gain', gain: 1 }] } },
|
||||||
|
]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('starts a picked parameter at the default of the FX', () => {
|
it('starts a picked parameter at the default of the FX', () => {
|
||||||
@@ -449,7 +467,7 @@ describe('Sound FX parameters and Play sound', () => {
|
|||||||
]);
|
]);
|
||||||
const audio = fakeAudio();
|
const audio = fakeAudio();
|
||||||
startWorld(f, { behaviors: true, audio }).frame();
|
startWorld(f, { behaviors: true, audio }).frame();
|
||||||
expect(audio.played[0]!.fx).toEqual({ pitch: -12, stages: [] });
|
expect(audio.played[0]!.fx).toEqual({ pitch: -12, stages: [{ kind: 'gain', gain: 1 }] });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('says so at the node when no FX is picked or it is gone', () => {
|
it('says so at the node when no FX is picked or it is gone', () => {
|
||||||
@@ -475,9 +493,56 @@ describe('Sound FX parameters and Play sound', () => {
|
|||||||
expect(f.params.params.drop).toBe(7);
|
expect(f.params.params.drop).toBe(7);
|
||||||
const audio = fakeAudio();
|
const audio = fakeAudio();
|
||||||
startWorld(f, { behaviors: true, audio }).frame();
|
startWorld(f, { behaviors: true, audio }).frame();
|
||||||
expect(audio.played[0]!.fx).toEqual({ pitch: 7, stages: [] });
|
expect(audio.played[0]!.fx).toEqual({ pitch: 7, stages: [{ kind: 'gain', gain: 1 }] });
|
||||||
f.stack.undo();
|
f.stack.undo();
|
||||||
expect(f.params.ports).toEqual([{ key: 'depth', kind: 'number' }]);
|
expect(f.params.ports).toEqual([{ key: 'depth', kind: 'number' }]);
|
||||||
expect(f.params.params.depth).toBe(7);
|
expect(f.params.params.depth).toBe(7);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('Set sound FX parameters (#315)', () => {
|
||||||
|
/**
|
||||||
|
* Pressing x plays Jump through an FX twice, the first `sound` – then hands `depth` to the voice
|
||||||
|
* the first play gave out.
|
||||||
|
*/
|
||||||
|
function tuning(sound?: string) {
|
||||||
|
const f = makeSpriteScene('Box');
|
||||||
|
const { fx } = makeFx(f);
|
||||||
|
const created = new CreateSoundCommand('Jump', FILE);
|
||||||
|
f.stack.execute(created);
|
||||||
|
const behavior = makeBehavior(f, 'Logic');
|
||||||
|
place(f, behavior.id, { x: 0, y: 0 }, place(f, f.spriteId, { x: 0, y: 0 }));
|
||||||
|
const edit = graphEditor(f, behavior);
|
||||||
|
const first = edit.addNode('PlaySound', { sound: sound ?? created.asset.id, fx: fx.id });
|
||||||
|
const second = edit.addNode('PlaySound', { sound: created.asset.id, fx: fx.id });
|
||||||
|
const set = edit.addNode('SetSoundFxParameters');
|
||||||
|
const params = edit.addNode('SoundFxParameters', { soundFxAsset: fx.id, depth: 3 });
|
||||||
|
params.ports = [{ key: 'depth', kind: 'number' }];
|
||||||
|
edit.wire([
|
||||||
|
[edit.addNode('OnKeyDown', { key: 'x' }), 'exec', first, 'exec'],
|
||||||
|
[first, 'exec', second, 'exec'],
|
||||||
|
[second, 'exec', set, 'exec'],
|
||||||
|
[first, 'voice', set, 'voice'],
|
||||||
|
[params, 'params', set, 'fxParams'],
|
||||||
|
]);
|
||||||
|
return f;
|
||||||
|
}
|
||||||
|
|
||||||
|
it('hands the values to the one voice its Play sound gave out, not to another of the same sound', () => {
|
||||||
|
const audio = fakeAudio();
|
||||||
|
const { input, frame } = startWorld(tuning(), { behaviors: true, audio });
|
||||||
|
input.press('x');
|
||||||
|
frame();
|
||||||
|
expect(audio.played).toHaveLength(2);
|
||||||
|
expect(audio.tuned).toEqual([{ voice: 1, params: { depth: 3 } }]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('reaches nothing where its Play sound played nothing', () => {
|
||||||
|
const audio = fakeAudio();
|
||||||
|
const { input, frame } = startWorld(tuning('no-such-sound'), { behaviors: true, audio });
|
||||||
|
input.press('x');
|
||||||
|
frame();
|
||||||
|
expect(audio.played).toHaveLength(1);
|
||||||
|
expect(audio.tuned).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -42,6 +42,7 @@ function fakeAudio(): AudioSink & {
|
|||||||
stoppedOwners: [] as string[],
|
stoppedOwners: [] as string[],
|
||||||
play: (file: string, volume: number, loop: boolean, owner: string) =>
|
play: (file: string, volume: number, loop: boolean, owner: string) =>
|
||||||
sink.played.push({ file, volume, loop, owner }),
|
sink.played.push({ file, volume, loop, owner }),
|
||||||
|
tune: () => {},
|
||||||
stop: (file: string) => sink.stopped.push(file),
|
stop: (file: string) => sink.stopped.push(file),
|
||||||
stopOwner: (owner: string) => sink.stoppedOwners.push(owner),
|
stopOwner: (owner: string) => sink.stoppedOwners.push(owner),
|
||||||
};
|
};
|
||||||
|
|||||||
@@ -214,6 +214,7 @@ export type FakeContext = NodeContext & {
|
|||||||
fxParams: Readonly<Record<string, number>>;
|
fxParams: Readonly<Record<string, number>>;
|
||||||
}[];
|
}[];
|
||||||
silenced: string[];
|
silenced: string[];
|
||||||
|
tuned: { voice: number; fxParams: Readonly<Record<string, number>> }[];
|
||||||
/** The scenes the context was asked to load, in order. */
|
/** The scenes the context was asked to load, in order. */
|
||||||
loaded: string[];
|
loaded: string[];
|
||||||
/** The time scale and the pause the context holds, as the game nodes left them. */
|
/** The time scale and the pause the context holds, as the game nodes left them. */
|
||||||
@@ -265,9 +266,12 @@ export function fakeContext(overrides: Partial<NodeContext> = {}): FakeContext {
|
|||||||
played: [],
|
played: [],
|
||||||
silenced: [],
|
silenced: [],
|
||||||
loaded: [],
|
loaded: [],
|
||||||
|
// The number of a voice is how many have been played: unique, and never 0.
|
||||||
playSound: (assetId, volume, loop, detached, fx, fxParams) =>
|
playSound: (assetId, volume, loop, detached, fx, fxParams) =>
|
||||||
ctx.played.push({ assetId, volume, loop, detached, fx, fxParams }),
|
ctx.played.push({ assetId, volume, loop, detached, fx, fxParams }),
|
||||||
stopSound: (assetId) => ctx.silenced.push(assetId),
|
stopSound: (assetId) => ctx.silenced.push(assetId),
|
||||||
|
tuned: [],
|
||||||
|
tuneSound: (voice, fxParams) => ctx.tuned.push({ voice, fxParams }),
|
||||||
loadScene: (sceneId) => ctx.loaded.push(sceneId),
|
loadScene: (sceneId) => ctx.loaded.push(sceneId),
|
||||||
spawn: () => null,
|
spawn: () => null,
|
||||||
addJoint: () => null,
|
addJoint: () => null,
|
||||||
|
|||||||
@@ -6,16 +6,28 @@
|
|||||||
* The device is stubbed – nothing makes a noise in a unit test, and Web Audio does not exist here.
|
* The device is stubbed – nothing makes a noise in a unit test, and Web Audio does not exist here.
|
||||||
*/
|
*/
|
||||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||||
import type { FxChain } from '../../src/core/soundfx/nodes';
|
import type { FxChain, FxPlan, FxVoice } from '../../src/core/soundfx/nodes';
|
||||||
import { SoundLibrary } from '../../src/viewport/SoundLibrary';
|
import { SoundLibrary } from '../../src/viewport/SoundLibrary';
|
||||||
|
|
||||||
const FILE = 'a.wav';
|
const FILE = 'a.wav';
|
||||||
const OTHER = 'b.wav';
|
const OTHER = 'b.wav';
|
||||||
|
|
||||||
|
/** An AudioParam that remembers where it was sent to glide, and how often. */
|
||||||
|
interface FakeParam {
|
||||||
|
value: number;
|
||||||
|
glides: number[];
|
||||||
|
setTargetAtTime: (value: number) => void;
|
||||||
|
}
|
||||||
|
|
||||||
|
function param(value: number): FakeParam {
|
||||||
|
const made: FakeParam = { value, glides: [], setTargetAtTime: (to) => made.glides.push(to) };
|
||||||
|
return made;
|
||||||
|
}
|
||||||
|
|
||||||
interface FakeSource {
|
interface FakeSource {
|
||||||
buffer: unknown;
|
buffer: unknown;
|
||||||
loop: boolean;
|
loop: boolean;
|
||||||
playbackRate: { value: number };
|
playbackRate: FakeParam;
|
||||||
onended: (() => void) | null;
|
onended: (() => void) | null;
|
||||||
started: boolean;
|
started: boolean;
|
||||||
stopped: boolean;
|
stopped: boolean;
|
||||||
@@ -28,11 +40,11 @@ const sources: FakeSource[] = [];
|
|||||||
/** Every `connect` made, as "from > to" by the kind of node – the order a voice is wired in. */
|
/** Every `connect` made, as "from > to" by the kind of node – the order a voice is wired in. */
|
||||||
const wires: string[] = [];
|
const wires: string[] = [];
|
||||||
/** The level of every gain node made, in the order they were made. */
|
/** The level of every gain node made, in the order they were made. */
|
||||||
const gains: { value: number }[] = [];
|
const gains: FakeParam[] = [];
|
||||||
const filters: {
|
const filters: {
|
||||||
type: string;
|
type: string;
|
||||||
frequency: { value: number };
|
frequency: { value: number };
|
||||||
gain: { value: number };
|
gain: FakeParam;
|
||||||
Q: { value: number };
|
Q: { value: number };
|
||||||
}[] = [];
|
}[] = [];
|
||||||
|
|
||||||
@@ -69,6 +81,7 @@ function fakeBuffer(_channels: number, length: number, sampleRate: number) {
|
|||||||
function fakeContext() {
|
function fakeContext() {
|
||||||
return {
|
return {
|
||||||
state: 'running',
|
state: 'running',
|
||||||
|
currentTime: 0,
|
||||||
destination: { kind: 'destination' },
|
destination: { kind: 'destination' },
|
||||||
resume: () => Promise.resolve(),
|
resume: () => Promise.resolve(),
|
||||||
suspend: () => Promise.resolve(),
|
suspend: () => Promise.resolve(),
|
||||||
@@ -78,7 +91,7 @@ function fakeContext() {
|
|||||||
const source: FakeSource = wired('source', {
|
const source: FakeSource = wired('source', {
|
||||||
buffer: null,
|
buffer: null,
|
||||||
loop: false,
|
loop: false,
|
||||||
playbackRate: { value: 1 },
|
playbackRate: param(1),
|
||||||
onended: null,
|
onended: null,
|
||||||
started: false,
|
started: false,
|
||||||
stopped: false,
|
stopped: false,
|
||||||
@@ -93,7 +106,7 @@ function fakeContext() {
|
|||||||
return source;
|
return source;
|
||||||
},
|
},
|
||||||
createGain: () => {
|
createGain: () => {
|
||||||
const gain = wired('gain', { gain: { value: 1 } });
|
const gain = wired('gain', { gain: param(1) });
|
||||||
gains.push(gain.gain);
|
gains.push(gain.gain);
|
||||||
return gain;
|
return gain;
|
||||||
},
|
},
|
||||||
@@ -101,7 +114,7 @@ function fakeContext() {
|
|||||||
const filter = wired('filter', {
|
const filter = wired('filter', {
|
||||||
type: '',
|
type: '',
|
||||||
frequency: { value: 0 },
|
frequency: { value: 0 },
|
||||||
gain: { value: 0 },
|
gain: param(0),
|
||||||
Q: { value: 0 },
|
Q: { value: 0 },
|
||||||
});
|
});
|
||||||
filters.push(filter);
|
filters.push(filter);
|
||||||
@@ -136,11 +149,37 @@ afterEach(() => {
|
|||||||
vi.unstubAllGlobals();
|
vi.unstubAllGlobals();
|
||||||
});
|
});
|
||||||
|
|
||||||
/** Plays `file` and lets its bytes arrive, unless `hold` keeps the decode pending. */
|
/** A chain as the plan of an FX with nothing left open, so a test can say what it wants to hear. */
|
||||||
async function play(library: SoundLibrary, file = FILE, { hold = false } = {}, fx?: FxChain): Promise<void> {
|
function planOf(chain: FxChain): FxPlan {
|
||||||
library.play(file, 1, false, '', fx);
|
const stages = chain.stages.map((stage) =>
|
||||||
|
stage.kind === 'gain'
|
||||||
|
? { kind: 'gain' as const, db: { value: 20 * Math.log10(stage.gain) } }
|
||||||
|
: { ...stage, gain: { value: stage.gain } },
|
||||||
|
);
|
||||||
|
if ('mix' in chain) return { mix: { value: chain.mix }, a: planOf(chain.a), b: planOf(chain.b), stages };
|
||||||
|
return { pitch: chain.pitch === 0 ? [] : [{ value: chain.pitch }], stages };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Plays `file` and lets its bytes arrive, unless `hold` keeps the decode pending; gives the voice. */
|
||||||
|
async function play(
|
||||||
|
library: SoundLibrary,
|
||||||
|
file = FILE,
|
||||||
|
{ hold = false, owner = '' } = {},
|
||||||
|
fx?: FxChain | FxVoice,
|
||||||
|
): Promise<number> {
|
||||||
|
const voice = library.play(
|
||||||
|
file,
|
||||||
|
1,
|
||||||
|
false,
|
||||||
|
owner,
|
||||||
|
fx && ('plan' in fx ? fx : { plan: planOf(fx), params: {} }),
|
||||||
|
);
|
||||||
await settle();
|
await settle();
|
||||||
if (hold) return;
|
if (!hold) await arrive();
|
||||||
|
return voice;
|
||||||
|
}
|
||||||
|
|
||||||
|
async function arrive(): Promise<void> {
|
||||||
for (const resolve of decodes.splice(0)) resolve();
|
for (const resolve of decodes.splice(0)) resolve();
|
||||||
await settle();
|
await settle();
|
||||||
}
|
}
|
||||||
@@ -208,7 +247,7 @@ describe('the sound library', () => {
|
|||||||
expect([up!.duration, down!.duration]).toEqual([dry!.duration, dry!.duration]);
|
expect([up!.duration, down!.duration]).toEqual([dry!.duration, dry!.duration]);
|
||||||
});
|
});
|
||||||
|
|
||||||
it('mixes two sounds by their shares, a source per pitch, started and stopped together (#316)', async () => {
|
it('mixes two sounds by their shares, a source per way, started and stopped together (#316)', async () => {
|
||||||
const library = new SoundLibrary();
|
const library = new SoundLibrary();
|
||||||
await play(
|
await play(
|
||||||
library,
|
library,
|
||||||
@@ -231,7 +270,7 @@ describe('the sound library', () => {
|
|||||||
'filter > gain',
|
'filter > gain',
|
||||||
'gain > destination',
|
'gain > destination',
|
||||||
]);
|
]);
|
||||||
expect(gains.map((g) => g.value)).toEqual([1, 1, 0.75, 0.5, 0.25]);
|
expect(gains.map((g) => g.value)).toEqual([1, 1, 0.75, expect.closeTo(0.5), 0.25]);
|
||||||
expect(sources.map((s) => s.started)).toEqual([true, true]);
|
expect(sources.map((s) => s.started)).toEqual([true, true]);
|
||||||
expect(library.stats()).toMatchObject({ playing: 1, started: 1, last: { pitches: [12, 0], stages: 2 } });
|
expect(library.stats()).toMatchObject({ playing: 1, started: 1, last: { pitches: [12, 0], stages: 2 } });
|
||||||
|
|
||||||
@@ -242,7 +281,7 @@ describe('the sound library', () => {
|
|||||||
expect(library.stats()).toMatchObject({ playing: 0, stopped: 1 });
|
expect(library.stats()).toMatchObject({ playing: 0, stopped: 1 });
|
||||||
});
|
});
|
||||||
|
|
||||||
it('starts one source for two ways at the same pitch', async () => {
|
it('gives two ways at the same pitch a source each, so each can glide on its own (#315)', async () => {
|
||||||
const library = new SoundLibrary();
|
const library = new SoundLibrary();
|
||||||
await play(
|
await play(
|
||||||
library,
|
library,
|
||||||
@@ -255,10 +294,82 @@ describe('the sound library', () => {
|
|||||||
stages: [],
|
stages: [],
|
||||||
},
|
},
|
||||||
);
|
);
|
||||||
expect(sources).toHaveLength(1);
|
expect(sources).toHaveLength(2);
|
||||||
|
expect(sources[0]!.buffer).toBe(sources[1]!.buffer);
|
||||||
expect(wires.filter((w) => w.startsWith('source'))).toEqual(['source > gain', 'source > gain']);
|
expect(wires.filter((w) => w.startsWith('source'))).toEqual(['source > gain', 'source > gain']);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('a parameter changed while the voice plays (#315)', () => {
|
||||||
|
/** Pitch on a, a gain on b, the share between them – all three open – and a band after the Mix. */
|
||||||
|
const open: FxPlan = {
|
||||||
|
mix: { value: 0.25, param: 'blend', min: 0, max: 1 },
|
||||||
|
a: { pitch: [{ value: 0, param: 'depth', min: -64, max: 64 }], stages: [] },
|
||||||
|
b: { pitch: [], stages: [{ kind: 'gain', db: { value: 0, param: 'loud', min: -120, max: 24 } }] },
|
||||||
|
stages: [{ kind: 'filter', filter: 'lowshelf', frequency: 80, gain: { value: 6 }, q: 1 }],
|
||||||
|
};
|
||||||
|
|
||||||
|
it('glides every knob it moves, a pitch by speed, and leaves the rest alone', async () => {
|
||||||
|
const library = new SoundLibrary();
|
||||||
|
const voice = await play(library, FILE, { owner: 'me' }, { plan: open, params: {} });
|
||||||
|
// Made in the order: the voice, the Mix, share of a, the gain on b, share of b.
|
||||||
|
const [, , shareA, loud, shareB] = gains;
|
||||||
|
expect([shareA!.value, loud!.value, shareB!.value]).toEqual([0.75, 1, 0.25]);
|
||||||
|
|
||||||
|
expect(library.tune(voice, { depth: 12, loud: -6, blend: 0.5, nothing: 3 })).toBe(true);
|
||||||
|
expect(sources[0]!.playbackRate.glides).toEqual([2]);
|
||||||
|
expect(sources[1]!.playbackRate.glides).toEqual([]);
|
||||||
|
expect(loud!.glides).toEqual([expect.closeTo(10 ** (-6 / 20))]);
|
||||||
|
expect([shareA!.glides, shareB!.glides]).toEqual([[0.5], [0.5]]);
|
||||||
|
expect(filters[0]!.gain.glides).toEqual([]);
|
||||||
|
expect(library.stats().tuned).toBe(1);
|
||||||
|
|
||||||
|
// Only what changed is sent again; one not given keeps its value rather than its default.
|
||||||
|
library.tune(voice, { depth: 12, blend: 1 });
|
||||||
|
expect(sources[0]!.playbackRate.glides).toEqual([2]);
|
||||||
|
expect(loud!.glides).toHaveLength(1);
|
||||||
|
expect([shareA!.glides, shareB!.glides]).toEqual([
|
||||||
|
[0.5, 0],
|
||||||
|
[0.5, 1],
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('reaches the one voice it is given – a detached one too – and none that has ended', async () => {
|
||||||
|
const library = new SoundLibrary();
|
||||||
|
const voices = [
|
||||||
|
await play(library, FILE, { owner: 'me' }, { plan: open, params: {} }),
|
||||||
|
await play(library, FILE, { owner: 'me' }, { plan: open, params: {} }),
|
||||||
|
await play(library, OTHER, { owner: 'me' }, { plan: open, params: {} }),
|
||||||
|
await play(library, FILE, {}, { plan: open, params: {} }),
|
||||||
|
];
|
||||||
|
expect(new Set(voices).size).toBe(4);
|
||||||
|
expect(voices).not.toContain(0);
|
||||||
|
library.tune(voices[1]!, { depth: 12 });
|
||||||
|
library.tune(voices[3]!, { depth: 12 });
|
||||||
|
// Two sources a voice, the first of each the pitched way.
|
||||||
|
expect(sources.map((s) => s.playbackRate.glides.length)).toEqual([0, 0, 1, 0, 0, 0, 1, 0]);
|
||||||
|
|
||||||
|
library.stop(FILE);
|
||||||
|
expect(library.tune(voices[1]!, { depth: 3 })).toBe(false);
|
||||||
|
expect(library.tune(0, { depth: 3 })).toBe(false);
|
||||||
|
expect(library.stats().tuned).toBe(2);
|
||||||
|
});
|
||||||
|
|
||||||
|
it('starts a voice still waiting on its bytes with the values handed in meanwhile', async () => {
|
||||||
|
const library = new SoundLibrary();
|
||||||
|
const voice = await play(
|
||||||
|
library,
|
||||||
|
FILE,
|
||||||
|
{ hold: true, owner: 'me' },
|
||||||
|
{ plan: open, params: { blend: 0.5 } },
|
||||||
|
);
|
||||||
|
expect(library.tune(voice, { depth: 7 })).toBe(true);
|
||||||
|
await arrive();
|
||||||
|
expect(library.stats().last?.pitches).toEqual([7, 0]);
|
||||||
|
expect(gains.map((g) => g.value)).toContain(0.5);
|
||||||
|
expect(sources[0]!.playbackRate.glides).toEqual([]);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
it('a stop while the bytes are in flight keeps the sound from starting afterwards', async () => {
|
it('a stop while the bytes are in flight keeps the sound from starting afterwards', async () => {
|
||||||
const library = new SoundLibrary();
|
const library = new SoundLibrary();
|
||||||
await play(library, FILE, { hold: true });
|
await play(library, FILE, { hold: true });
|
||||||
|
|||||||
+67
-1
@@ -1,4 +1,4 @@
|
|||||||
import { edge, node } from '../lib/builders.mjs';
|
import { edge, node, port, variable, written } from '../lib/builders.mjs';
|
||||||
import { LANES, MAKE_OBJECT, pickAssetType } from './lib/scaffold.mjs';
|
import { LANES, MAKE_OBJECT, pickAssetType } from './lib/scaffold.mjs';
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -414,6 +414,72 @@ export async function sound(t) {
|
|||||||
JSON.stringify(gameParam),
|
JSON.stringify(gameParam),
|
||||||
);
|
);
|
||||||
await stop();
|
await stop();
|
||||||
|
// #315: two looping voices of the same sound start at depth 7, the first keeps its number in a
|
||||||
|
// variable, and `Set sound FX parameters` hands that one voice depth 3 every frame. In a
|
||||||
|
// real AudioContext the first one's speed glides there, the second one's stays put.
|
||||||
|
const tuneIds = ['sn-keep', 'sn-play2', 'sn-tick', 'sn-read', 'sn-tune', 'sn-fxp2'];
|
||||||
|
const tuneEdges = [
|
||||||
|
'sn-e-keep',
|
||||||
|
'sn-e-voice',
|
||||||
|
'sn-e-play2',
|
||||||
|
'sn-e-fxp3',
|
||||||
|
'sn-e-tick',
|
||||||
|
'sn-e-read',
|
||||||
|
'sn-e-fxp2',
|
||||||
|
];
|
||||||
|
await mutate(`
|
||||||
|
const logic = __gm.state.project.assets.find((a) => a.id === 'sn-logic');
|
||||||
|
logic.variables = [${JSON.stringify(variable('voice', 'number', 0))}];
|
||||||
|
const graph = logic.graph;
|
||||||
|
const play = graph.nodes.find((n) => n.id === 'sn-play');
|
||||||
|
play.params.loop = 'true';
|
||||||
|
graph.nodes.push(
|
||||||
|
${JSON.stringify(node('sn-keep', 'Variables', 520, 0, {}, [written('voice', 'number')]))},
|
||||||
|
{ ...structuredClone(play), id: 'sn-play2', x: 780 },
|
||||||
|
${JSON.stringify(node('sn-tick', 'OnUpdate', 0, 300))},
|
||||||
|
${JSON.stringify(node('sn-read', 'Variables', 0, 420, {}, [port('voice', 'number')]))},
|
||||||
|
${JSON.stringify(node('sn-tune', 'SetSoundFxParameters', 260, 300))},
|
||||||
|
${JSON.stringify(
|
||||||
|
node('sn-fxp2', 'SoundFxParameters', 0, 540, { soundFxAsset: fxId, depth: 3 }, [
|
||||||
|
{ key: 'depth', kind: 'number' },
|
||||||
|
]),
|
||||||
|
)},
|
||||||
|
);
|
||||||
|
graph.edges.push(
|
||||||
|
${JSON.stringify(edge('sn-e-keep', 'sn-play', 'exec', 'sn-keep', 'exec'))},
|
||||||
|
${JSON.stringify(edge('sn-e-voice', 'sn-play', 'voice', 'sn-keep', 'voice'))},
|
||||||
|
${JSON.stringify(edge('sn-e-play2', 'sn-keep', 'exec', 'sn-play2', 'exec'))},
|
||||||
|
${JSON.stringify(edge('sn-e-fxp3', 'sn-fxp', 'params', 'sn-play2', 'fxParams'))},
|
||||||
|
${JSON.stringify(edge('sn-e-tick', 'sn-tick', 'exec', 'sn-tune', 'exec'))},
|
||||||
|
${JSON.stringify(edge('sn-e-read', 'sn-read', 'voice', 'sn-tune', 'voice'))},
|
||||||
|
${JSON.stringify(edge('sn-e-fxp2', 'sn-fxp2', 'params', 'sn-tune', 'fxParams'))},
|
||||||
|
);`);
|
||||||
|
const beforeTune = await stats();
|
||||||
|
await play();
|
||||||
|
const tunedStats = await waitFor(
|
||||||
|
stats,
|
||||||
|
(s) => s.tuned > beforeTune.tuned && s.started >= beforeTune.started + 2,
|
||||||
|
);
|
||||||
|
const rate = 2 ** ((3 - 7) / 12);
|
||||||
|
const glided = (levels) =>
|
||||||
|
Array.isArray(levels) && levels.length === 2 && Math.abs(levels[0][0] - rate) < 0.01;
|
||||||
|
const levels = await waitFor(() => json(`__gm.sound.levels(${JSON.stringify(blip.file)})`), glided);
|
||||||
|
check(
|
||||||
|
'Set sound FX parameters moves the one voice whose number it holds from depth 7 to 3, not the other',
|
||||||
|
tunedStats.last?.pitches.join() === '7' &&
|
||||||
|
tunedStats.last.loop === true &&
|
||||||
|
glided(levels) &&
|
||||||
|
Math.abs(levels[0][1] - 10 ** (-6 / 20)) < 0.001 &&
|
||||||
|
levels[1][0] === 1,
|
||||||
|
JSON.stringify({ last: tunedStats.last, tuned: tunedStats.tuned, levels, rate }),
|
||||||
|
);
|
||||||
|
await stop();
|
||||||
|
await mutate(`
|
||||||
|
const logic = __gm.state.project.assets.find((a) => a.id === 'sn-logic');
|
||||||
|
logic.variables = [];
|
||||||
|
const graph = logic.graph;
|
||||||
|
graph.nodes = graph.nodes.filter((n) => !${JSON.stringify(tuneIds)}.includes(n.id));
|
||||||
|
graph.edges = graph.edges.filter((e) => !${JSON.stringify(tuneEdges)}.includes(e.id));`);
|
||||||
await mutate(`
|
await mutate(`
|
||||||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||||||
graph.nodes = graph.nodes.filter((n) => n.id !== 'sn-fxp');
|
graph.nodes = graph.nodes.filter((n) => n.id !== 'sn-fxp');
|
||||||
|
|||||||
Reference in new issue
Block a user