Mix sounds through buses and a master (#325)
Until now every sound went straight to the speakers. There was no point where all of them came together, so nothing could turn the music down against the effects, and a compressor or a shared echo over everything had nowhere to sit. The bus technique of #346 was there already, only the end of the chain was missing. - Bus asset (Audio): Volume, Sound FX and Into. Into is empty for the master and never offers the bus itself nor a bus that already goes into it. The list is worked out when the menu opens, so a change made meanwhile counts. A loop that is in a project anyway sends the buses on it into the master. - The master is no asset. Its volume and sound FX are two new rows under Master in Project settings, one undo step each. A project written before has 1 and none, which is how it sounded. - Sound, instrument, pattern and song have a row Bus, empty for the master. Play sound, Play note, Play pattern and Play song have an input bus: From the asset, Master or a bus. Inside a song or a pattern only the bus of the song or pattern counts. - Set bus volume (bus, volume, time, easing) and Bus volume. An empty bus is the master. The volume set holds for the whole run, across scenes, over that of the asset. GameRun resets it at start and end. Set sound FX parameters without a voice now reaches the sound FX of the bus picked in its new bus input. - SoundLibrary: the master is the bus "mix", each bus "mix:<id>". Every bus, the playing masters of #346 included, ends there, and only the master is heard. setMix sets them up once a context exists, so an editor nobody listened to opens no audio device. The same mix again changes nothing, which lets the editor hand it over on every change of the project. - The tabs play as the game does: Sound, Sound FX, Instrument, Pattern and Song tab through the bus of their asset and the master. - Pause game does nothing to the buses (the user's answer 4b). A game that wants quiet in its pause menu sets a bus volume itself. - Streaming long music files is #349, split off as asked. Finding on the way: the Into menu first worked from a list made when the inspector row was built. After another bus was sent into the open one, e2e 34 saw it still offered. It is now asked on open. The SoundLibrary tests expected voices to end at the destination. They now end in the master, which adds its input, level and pan to every wiring and two gains to every list. Those expectations are updated, nothing else in them changed. Tests: - Unit buses (new): 15 green. Ten counter-checks, each red on its own: loop check, bus picked, reset at run start, bus of Set sound FX parameters, master volume in the settings, and in SoundLibrary default into the master, voices into the master, run volume over the mix, no context before a play, a playing on its bus. - Unit sound-library: 90 green (5 new). Node catalog: 416 green (9 new cases). - e2e 34 (melektaus-atx): ALL PASSED, 75 checks (3 new). The voice of the Sound tab was heard on mix:<bus>, that bus into the other, that one at 0.5 into the master. Master volume 0.6 via the dialog, undo back to 1. - Gate (melektaus-atx): CHECK EXIT 0, 262 files, 4072 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
a848cf8469
commit
3a52f2290f
53 files changed
+1431
-147
No files matched your search
+142
-22
@@ -23,6 +23,7 @@ import {
|
||||
import { envelopeLevel, modCurve, releaseTime, SETTLE } from '../core/instrument/envelope';
|
||||
import type { EaseShare } from '../core/model/animation';
|
||||
import { type BusLevel, Playbacks, type SequenceNeeds, type SequencePlay } from '../core/music/playbacks';
|
||||
import { MASTER_MIX, type MixBus, sameMix } from '../core/music/mixer';
|
||||
import type { ScheduledClip, ScheduledNote } from '../core/music/Scheduler';
|
||||
import { dbToGain, SOUND_ITEM_MAX_DB } from '../core/music/song';
|
||||
import { DEFAULT_TEMPO } from '../core/music/time';
|
||||
@@ -299,6 +300,13 @@ export class SoundLibrary implements AudioSink {
|
||||
private held = false;
|
||||
/** Counts the times buses were let go, so a later use of one can keep it. */
|
||||
private leavings = 0;
|
||||
/**
|
||||
* The master and the buses of the project (#325) as last set, made when the context is: setting
|
||||
* them must not open an audio device for an editor nobody has listened to yet.
|
||||
*/
|
||||
private mix: readonly MixBus[] = [];
|
||||
/** The levels a running game set its buses to, over those of their assets; until `resetMix`. */
|
||||
private readonly mixVolumes = new Map<string, number>();
|
||||
|
||||
/** Where the files are served from. Set once at start-up, before anything plays. */
|
||||
configure(base: string): void {
|
||||
@@ -309,10 +317,21 @@ export class SoundLibrary implements AudioSink {
|
||||
* The number is handed out at once, before the bytes are there: a graph can tune a voice in the
|
||||
* same frame it asked for it, and the voice then starts with those values.
|
||||
*/
|
||||
play(file: string, volume: number, loop: boolean, owner = '', fx: FxVoice = DRY_VOICE): number {
|
||||
play(file: string, volume: number, loop: boolean, owner = '', fx: FxVoice = DRY_VOICE, bus = ''): number {
|
||||
if (file === '' || this.hasFailed(file)) return 0;
|
||||
const layered = { layers: [fx.plan], params: fx.params, tempo: fx.tempo ?? DEFAULT_TEMPO };
|
||||
return this.begin({ id: 0, file, files: [file], volume, loop, owner, fx: layered, duration: 0, at: 0 });
|
||||
return this.begin({
|
||||
id: 0,
|
||||
file,
|
||||
files: [file],
|
||||
volume,
|
||||
loop,
|
||||
owner,
|
||||
fx: layered,
|
||||
duration: 0,
|
||||
at: 0,
|
||||
bus,
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -385,7 +404,7 @@ export class SoundLibrary implements AudioSink {
|
||||
const waits: Promise<void>[] = [];
|
||||
for (const { name, gain, pan, into, fx } of levels) {
|
||||
if (into !== undefined) this.bus(into);
|
||||
this.bus(name, into ?? null);
|
||||
this.route(name, this.bus(name, into ?? null), into ?? null);
|
||||
this.setBus(name, gain, pan);
|
||||
waits.push(this.setFx(this.bus(name), fx ?? null));
|
||||
}
|
||||
@@ -417,7 +436,11 @@ export class SoundLibrary implements AudioSink {
|
||||
busLatency(name: string): number {
|
||||
let late = 0;
|
||||
const seen = new Set<Bus>();
|
||||
for (let bus = this.buses.get(name); bus && !seen.has(bus); bus = this.buses.get(bus.into ?? '')) {
|
||||
for (
|
||||
let bus = this.buses.get(name || MASTER_MIX);
|
||||
bus && !seen.has(bus);
|
||||
bus = this.buses.get(bus.into ?? '')
|
||||
) {
|
||||
seen.add(bus);
|
||||
late += bus.chain?.latency ?? 0;
|
||||
}
|
||||
@@ -442,12 +465,15 @@ export class SoundLibrary implements AudioSink {
|
||||
}));
|
||||
}
|
||||
|
||||
/** Where the voices of a track go in; straight out for `''`. */
|
||||
/** Where the voices of a bus go in; the master for `''` (#325). */
|
||||
private busInput(name: string): AudioNode {
|
||||
return name === '' ? this.audio().destination : this.bus(name).input;
|
||||
return this.bus(name || MASTER_MIX).input;
|
||||
}
|
||||
|
||||
/** A bus by its name, made on its first use into `into`, or straight out; it is not to go any more. */
|
||||
/**
|
||||
* A bus by its name, made on its first use into `into`, or into the master; only the master is
|
||||
* heard straight out (#325). It is not to go any more.
|
||||
*/
|
||||
private bus(name: string, into: string | null = null): Bus {
|
||||
const known = this.buses.get(name);
|
||||
if (known) {
|
||||
@@ -455,24 +481,108 @@ export class SoundLibrary implements AudioSink {
|
||||
return known;
|
||||
}
|
||||
const context = this.audio();
|
||||
const target = name === MASTER_MIX ? null : (into ?? MASTER_MIX);
|
||||
const made: Bus = {
|
||||
input: context.createGain(),
|
||||
gain: context.createGain(),
|
||||
pan: context.createStereoPanner(),
|
||||
set: { gain: 1, pan: 0 },
|
||||
into,
|
||||
into: target,
|
||||
fx: null,
|
||||
chain: null,
|
||||
leaving: 0,
|
||||
};
|
||||
made.input
|
||||
.connect(made.gain)
|
||||
.connect(made.pan)
|
||||
.connect(into ? this.bus(into).input : context.destination);
|
||||
made.input.connect(made.gain).connect(made.pan).connect(this.outOf(target));
|
||||
this.buses.set(name, made);
|
||||
return made;
|
||||
}
|
||||
|
||||
private outOf(into: string | null): AudioNode {
|
||||
return into ? this.bus(into).input : this.audio().destination;
|
||||
}
|
||||
|
||||
/** Sends a bus on into another one where it went somewhere else; the master stays heard. */
|
||||
private route(name: string, bus: Bus, into: string | null): void {
|
||||
const target = name === MASTER_MIX ? null : (into ?? MASTER_MIX);
|
||||
if (bus.into === target) return;
|
||||
bus.pan.disconnect();
|
||||
bus.pan.connect(this.outOf(target));
|
||||
bus.into = target;
|
||||
}
|
||||
|
||||
/**
|
||||
* The master and the buses of the project (#325): each into the bus it names, at its level unless
|
||||
* a running game set another, through its sound FX. A bus whose asset is gone is let go. Applied
|
||||
* once there is a context; done once the chains are there. The same mix again changes nothing, so
|
||||
* the editor can hand it over on every change of the project.
|
||||
*/
|
||||
async setMix(buses: readonly MixBus[]): Promise<void> {
|
||||
if (sameMix(this.mix, buses)) return;
|
||||
this.mix = buses;
|
||||
if (this.context) await this.applyMix();
|
||||
}
|
||||
|
||||
private async applyMix(): Promise<void> {
|
||||
const names = new Set(this.mix.map((bus) => bus.name));
|
||||
const waits: Promise<void>[] = [];
|
||||
for (const { name, volume, into, fx } of this.mix) {
|
||||
const bus = this.bus(name, into);
|
||||
this.route(name, bus, into);
|
||||
this.glideLevel(bus, this.mixVolumes.get(name) ?? volume);
|
||||
waits.push(this.setFx(bus, fx));
|
||||
}
|
||||
const gone = [...this.buses.keys()].filter(
|
||||
(name) => name.startsWith(`${MASTER_MIX}:`) && !names.has(name),
|
||||
);
|
||||
for (const name of gone) this.mixVolumes.delete(name);
|
||||
this.releaseBuses(gone);
|
||||
await Promise.all(waits);
|
||||
}
|
||||
|
||||
/** A quick glide of the level of a bus to `gain`. */
|
||||
private glideLevel(bus: Bus, gain: number, time = 0, ease: EaseShare = 'linear'): void {
|
||||
const to = Math.max(0, gain);
|
||||
move(bus.gain.gain, to, this.audio().currentTime, time, false, ease);
|
||||
bus.set = { ...bus.set, gain: to };
|
||||
}
|
||||
|
||||
/**
|
||||
* The level a running game sets a bus of the mix to (#325), over `time` seconds along `ease`; it
|
||||
* holds until `resetMix`, whatever the mix is set to meanwhile.
|
||||
*/
|
||||
setMixVolume(name: string, volume: number, time = 0, ease: EaseShare = 'linear'): void {
|
||||
const level = Number.isFinite(volume) ? Math.max(0, Math.min(1, volume)) : 1;
|
||||
this.mixVolumes.set(name, level);
|
||||
if (this.context || this.buses.has(name)) this.glideLevel(this.bus(name), level, Math.max(0, time), ease);
|
||||
}
|
||||
|
||||
/** The level a bus of the mix is at: what a game set it to, else that of its asset. */
|
||||
mixVolume(name: string): number {
|
||||
return this.mixVolumes.get(name) ?? this.mix.find((bus) => bus.name === name)?.volume ?? 1;
|
||||
}
|
||||
|
||||
/** Back to the levels of the assets, as a new run starts and an old one ends. */
|
||||
resetMix(): void {
|
||||
const set = [...this.mixVolumes.keys()];
|
||||
this.mixVolumes.clear();
|
||||
if (!this.context) return;
|
||||
for (const name of set) {
|
||||
const bus = this.buses.get(name);
|
||||
if (bus) this.glideLevel(bus, this.mixVolume(name));
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The sound FX of a bus of the mix (#325), as `tune` changes those of a voice. Says whether there
|
||||
* is such a bus.
|
||||
*/
|
||||
tuneMix(name: string, params: FxParams, time = 0, ease: EaseShare = 'linear'): boolean {
|
||||
const bus = this.buses.get(name);
|
||||
if (!bus) return this.mix.some((known) => known.name === name);
|
||||
this.tuneBus(bus, params, time, ease);
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Sets the sound FX of a bus (#346), its chain made again only where the FX is another or counts by
|
||||
* another tempo. Done once the chain is there, after the worklet it needs.
|
||||
@@ -631,7 +741,7 @@ export class SoundLibrary implements AudioSink {
|
||||
if (play.sequence.notes.length === 0 && !play.sequence.clips?.length) return 0;
|
||||
const id = this.nextId++;
|
||||
const buses = play.buses ?? [];
|
||||
this.open(id, buses, play.fx ?? null);
|
||||
this.open(id, buses, play.fx ?? null, play.bus ?? MASTER_MIX);
|
||||
const tracks = new Set(buses.map((bus) => bus.name));
|
||||
const into = (bus: string | undefined) => groupBus(id, bus !== undefined && tracks.has(bus) ? bus : '');
|
||||
const { strike, strikeClip } = play;
|
||||
@@ -682,9 +792,9 @@ export class SoundLibrary implements AudioSink {
|
||||
* The buses of a playing, its tracks into its master and a pattern item's into its track (#346),
|
||||
* each with the sound FX it was given. Their chains are made once the worklet is there (`ready`).
|
||||
*/
|
||||
private open(id: number, tracks: readonly BusLevel[], fx: FxVoice | null): void {
|
||||
private open(id: number, tracks: readonly BusLevel[], fx: FxVoice | null, into: string): void {
|
||||
const masterName = groupBus(id, '');
|
||||
const master = this.bus(masterName);
|
||||
const master = this.bus(masterName, into);
|
||||
master.gain.gain.value = 0;
|
||||
master.set = { gain: 0, pan: 0 };
|
||||
master.fx = fx;
|
||||
@@ -875,15 +985,19 @@ export class SoundLibrary implements AudioSink {
|
||||
*/
|
||||
private tuneMaster(id: number, params: FxParams, time: number, ease: EaseShare): boolean {
|
||||
const master = this.buses.get(groupBus(id, ''));
|
||||
const fx = master?.fx;
|
||||
if (!master || !fx) return true;
|
||||
master.fx = { ...fx, params: { ...fx.params, ...params } };
|
||||
if (master) this.tuneBus(master, params, time, ease);
|
||||
return true;
|
||||
}
|
||||
|
||||
private tuneBus(bus: Bus, params: FxParams, time: number, ease: EaseShare): void {
|
||||
const fx = bus.fx;
|
||||
if (!fx) return;
|
||||
bus.fx = { ...fx, params: { ...fx.params, ...params } };
|
||||
// The chain stays: `setFx` would see the same plan, and the knobs move it.
|
||||
const levels = levelsOf(fxChainOf(fx.plan, master.fx.params, fx.tempo));
|
||||
if (master.chain && glideKnobs(master.chain.knobs, levels, this.audio().currentTime, time, ease))
|
||||
const levels = levelsOf(fxChainOf(fx.plan, bus.fx.params, fx.tempo));
|
||||
if (bus.chain && glideKnobs(bus.chain.knobs, levels, this.audio().currentTime, time, ease))
|
||||
this.tunedAt = this.audio().currentTime;
|
||||
this.tuned += 1;
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1308,6 +1422,7 @@ export class SoundLibrary implements AudioSink {
|
||||
duration: Math.max(0, ...files.map((name) => this.duration(name) ?? 0)),
|
||||
at: when,
|
||||
envelopes: wiring.envelopes.size,
|
||||
bus,
|
||||
};
|
||||
const voice: Voice = {
|
||||
id,
|
||||
@@ -1577,7 +1692,10 @@ export class SoundLibrary implements AudioSink {
|
||||
}
|
||||
|
||||
private audio(): AudioContext {
|
||||
return (this.context ??= new AudioContext());
|
||||
if (this.context) return this.context;
|
||||
this.context = new AudioContext();
|
||||
if (this.mix.length > 0) void this.applyMix();
|
||||
return this.context;
|
||||
}
|
||||
|
||||
/** What a name plays, a cut made from its file the first time; null while the file is not loaded. */
|
||||
@@ -1707,6 +1825,8 @@ interface LastVoice {
|
||||
at: number;
|
||||
/** How many Envelopes shape it (#335). */
|
||||
envelopes: number;
|
||||
/** The bus it went into, `''` for the master (#325). */
|
||||
bus: string;
|
||||
}
|
||||
|
||||
/** The stages that run on the worklet. */
|
||||
|
||||
Reference in new issue
Block a user