Route a video's sound through its own Master routing node

An editor recording had no sound while the video tab played a video.
The tab played its <video> straight to the speakers, past the Web
Audio graph, so the recording tap behind the limiter never heard it.
Fixing only that would have left videos as the one sound source the
Master routing cannot reach, so they get their own node, like sounds.

- A video asset with sound has a routing node labelled "Video". One
  without sound has none, it would have nothing to send.
- Its links take the sound after volume and FX. Several links or
  another level give the video its own bus, as with a sound.
- A new video follows the bus most other videos use. With none linked
  yet it goes to the Master, not to where the sounds go. Songs and
  sounds no longer follow videos either.
- Players only know their file, so videoBusesOf maps each file to the
  bus of its first asset. setMix takes that map and moves the gains of
  running videos at once when the routing changes.
- In the game, played in the editor or released, a video's sound
  follows its links. The tabs (video, composition, animation preview)
  play straight into the Master, as the sound tabs have since #368.
- The video tab's sound now goes through the Master, so its volume, FX
  and limiter apply there and the editor recording hears it.
- The recording badge beside the menu is an HTML element over the
  canvas now. The second render that drew it on the desynchronized
  canvas let the screen show a frame without it, so it flickered. The
  element is never in the copy, and clicks go through it.

Tests:
- Unit master-routing and sound-library: 3 new, green. Red without
  video in the routing kinds, without moving the gains, and with the
  old rule for the links of a new video.
- e2e 66e-iii: the tab plays a video linked to a bus, the level is
  0.09 on the Master and 0 on the bus. 66g: in the game the bus has
  0.09. Both red with the old ways.
- e2e 67e reads the badge from a real screenshot, still green.
- The README names 2043 e2e checks.
- Gate on melektaus-atx: 304 files, 4694 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-12 16:15:59 +02:00
1 parent 846a8b70b3
commit 93190e185c
25 files changed
+376 -99

No files matched your search

+50 -7
View File
@@ -376,6 +376,10 @@ export class SoundLibrary implements AudioSink {
private mix: readonly MixBus[] = [];
/** The levels a running game set its buses to, over those of their assets; until `resetMix`. */
private readonly mixVolumes = new Map<string, number>();
/** The bus of each video file, by the routing of its asset; the master for one not named. */
private videoBuses: ReadonlyMap<string, string> = new Map();
/** The sound of every video heard (#402), by its gain: the file it plays, and where it goes. */
private readonly media = new Map<GainNode, { file: string; routed: boolean; bus: string }>();
/** The names streamed (#349) and the copies they stream from, by name and by copy. */
private streams: ReadonlyMap<string, StreamedFile> = new Map();
/** The cut of each clip, with the copy it is decoded from rather than its whole file (#349). */
@@ -826,14 +830,35 @@ export class SoundLibrary implements AudioSink {
* The master and the buses of the project (#325): each into the bus it names, at its level unless
* a running game set another, through its sound FX. A bus whose asset is gone is let go. Applied
* once there is a context; done once the chains are there. The same mix again changes nothing, so
* the editor can hand it over on every change of the project.
* the editor can hand it over on every change of the project. The sound of a video goes into the
* bus `videos` names for its file, at once also for one heard already; left out, they stay.
*/
async setMix(buses: readonly MixBus[]): Promise<void> {
if (sameMix(this.mix, buses)) return;
async setMix(buses: readonly MixBus[], videos = this.videoBuses): Promise<void> {
const rerouted = !sameEntries(this.videoBuses, videos);
this.videoBuses = videos;
if (sameMix(this.mix, buses)) {
if (rerouted) this.rerouteMedia();
return;
}
this.mix = buses;
if (rerouted) this.rerouteMedia();
if (this.context) await this.applyMix();
}
/** Every video heard along the routing into the bus of its file. */
private rerouteMedia(): void {
for (const [gain, heard] of this.media) {
if (!heard.routed) continue;
gain.disconnect();
heard.bus = this.videoBus(heard.file);
gain.connect(this.busInput(heard.bus));
}
}
private videoBus(file: string): string {
return this.videoBuses.get(file) ?? MASTER_MIX;
}
private async applyMix(): Promise<void> {
const names = new Set(this.mix.map((bus) => bus.name));
const waits: Promise<void>[] = [];
@@ -1678,19 +1703,33 @@ export class SoundLibrary implements AudioSink {
}
/**
* The sound of a video (#402) into the master, as a voice goes: from now on `element` is heard
* through the mix and only through it. The gain is its volume. Null while a bake has the context.
* The sound of a video (#402) into the mix, as a voice goes: from now on `element` is heard
* through it and only through it. `routed`, into the bus the routing has for its `file`, else
* straight into the master as a tab's preview (#368). The gain is its volume; hand it to
* `releaseMedia` once the element is let go. Null while a bake has the context.
*/
mediaInput(element: HTMLMediaElement): GainNode | null {
mediaInput(element: HTMLMediaElement, file: string, routed: boolean): GainNode | null {
this.audio();
const context = this.live();
if (!context) return null;
if (context.state === 'suspended' && !this.held) void context.resume();
const gain = context.createGain();
context.createMediaElementSource(element).connect(gain).connect(this.busInput(''));
const bus = routed ? this.videoBus(file) : MASTER_MIX;
context.createMediaElementSource(element).connect(gain).connect(this.busInput(bus));
this.media.set(gain, { file, routed, bus });
return gain;
}
releaseMedia(gain: GainNode): void {
this.media.delete(gain);
gain.disconnect();
}
/** The bus the sound of each video heard goes into (test hook). */
mediaRoutes(): { file: string; routed: boolean; bus: string }[] {
return [...this.media.values()].map(({ file, routed, bus }) => ({ file, routed, bus }));
}
/** The context it plays to the speakers on; null before there is one, and for a bake (#340). */
private live(): AudioContext | null {
return this.render ? null : (this.context as AudioContext | null);
@@ -2639,6 +2678,10 @@ export interface OfflineRender {
/** Seconds a bake fades out over where its notes would have rung on (#340). */
const BAKE_LET_RING = 0.3;
function sameEntries(a: ReadonlyMap<string, string>, b: ReadonlyMap<string, string>): boolean {
return a === b || (a.size === b.size && [...a].every(([key, value]) => b.get(key) === value));
}
/** The bus of a game's playing (#336): its master for `''`, else the track of that name inside it. */
function groupBus(id: number, track: string): string {
return track === '' ? `play:${id}` : `play:${id}:${track}`;