Keep composition sound in step and give compositions a bus of their own
A composition's sound ran on the audio clock while its picture ran on the frames. A frame counts at most 0.1 s, so after a hitch the sound stayed ahead of the picture until the next stop. Songs in a trailer drifted off the cut. The items of a composition also had no bus of their own, and a video sprite could not pick where its sound goes. - CompositionSound.follow compares the audio clock with the picture each frame. More than 40 ms apart, it starts the sound again where the picture is. This holds in the tab and in a run. - A time scale other than 1 now restarts the sound again and again. It stays in time but comes in pieces. - The sprite of a video item shows "Driven (by composition)" as a fixed Video clock row. Video time is hidden there, because the composition sets it every frame. - Every composition has a bus in the mix and a node in the master routing. A new one goes to the master unless other compositions are linked elsewhere, apart from where the sounds go. - Audio items pick From asset, To composition, Master or a bus. New items start with To composition. Items saved before keep From asset, so music already sent through a bus does not bypass it. - Video sprites have a new row Video bus: From asset by default, the master or a bus. In a composition the default is the composition's bus, also for video sprites inside a container item. - The chosen bus is part of the key of the video player, so a sprite with another bus gets a player of its own. Found along the way: - Six tests with full lists of sprite defaults expected the old rows and now list Video bus too. Tests: - tsc, eslint, Prettier clean - Unit: 971 green in the 28 files touched - npm run check on melektaus-atx: green, 309 files, 4804 tests - e2e 68 (melektaus-atx): 13 checks green, 68i new. The sound item and the video item go into mix:cp68-cut, which goes to the master. - e2e 69 (melektaus-atx): ALL PASSED, 23 checks, 69q and 69r new. After a hitch of 500 ms the sound restarted twice and then lay 0.5 to 5 ms off the playhead. - Counter-checks in unit tests: without the comparison the hitch test fails, without the bus in the preload the run test fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
875c76026a
commit
4d0a696327
37 files changed
+657
-83
No files matched your search
@@ -401,8 +401,14 @@ export class SoundLibrary implements AudioSink {
|
||||
private readonly mixVolumes = new Map<string, number>();
|
||||
/** The bus of each video file, by the routing of its asset; the master for one not named. */
|
||||
private videoBuses: ReadonlyMap<string, string> = new Map();
|
||||
/** The sound of every video heard (#402), by its gain: the file it plays, and where it goes. */
|
||||
private readonly media = new Map<GainNode, { file: string; routed: boolean; bus: string }>();
|
||||
/**
|
||||
* The sound of every video heard (#402), by its gain: the file it plays, the bus its sprite picked
|
||||
* (#418) and where it goes.
|
||||
*/
|
||||
private readonly media = new Map<
|
||||
GainNode,
|
||||
{ file: string; routed: boolean; picked?: string; bus: string }
|
||||
>();
|
||||
/** The names streamed (#349) and the copies they stream from, by name and by copy. */
|
||||
private streams: ReadonlyMap<string, StreamedFile> = new Map();
|
||||
/** The cut of each clip, with the copy it is decoded from rather than its whole file (#349). */
|
||||
@@ -869,17 +875,18 @@ export class SoundLibrary implements AudioSink {
|
||||
return;
|
||||
}
|
||||
this.mix = buses;
|
||||
if (rerouted) this.rerouteMedia();
|
||||
this.rerouteMedia();
|
||||
if (this.context) await this.applyMix();
|
||||
}
|
||||
|
||||
/** Every video heard along the routing into the bus of its file. */
|
||||
/** Every video heard into the bus it goes to now: along the routing of its file, or its pick. */
|
||||
private rerouteMedia(): void {
|
||||
for (const [gain, heard] of this.media) {
|
||||
if (!heard.routed) continue;
|
||||
const bus = this.mediaBus(heard.file, heard.routed, heard.picked);
|
||||
if (bus === heard.bus) continue;
|
||||
gain.disconnect();
|
||||
heard.bus = this.videoBus(heard.file);
|
||||
gain.connect(this.busInput(heard.bus));
|
||||
heard.bus = bus;
|
||||
gain.connect(this.busInput(bus));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -887,6 +894,13 @@ export class SoundLibrary implements AudioSink {
|
||||
return this.videoBuses.get(file) ?? MASTER_MIX;
|
||||
}
|
||||
|
||||
/** Where a video is heard: its pick where the mix has that bus – else the master –, else along the routing of its file. */
|
||||
private mediaBus(file: string, routed: boolean, picked: string | undefined): string {
|
||||
if (!routed) return MASTER_MIX;
|
||||
if (picked === undefined) return this.videoBus(file);
|
||||
return this.mix.some((bus) => bus.name === picked) ? picked : MASTER_MIX;
|
||||
}
|
||||
|
||||
private async applyMix(): Promise<void> {
|
||||
const names = new Set(this.mix.map((bus) => bus.name));
|
||||
const waits: Promise<void>[] = [];
|
||||
@@ -1764,19 +1778,20 @@ export class SoundLibrary implements AudioSink {
|
||||
|
||||
/**
|
||||
* The sound of a video (#402) into the mix, as a voice goes: from now on `element` is heard
|
||||
* through it and only through it. `routed`, into the bus the routing has for its `file`, else
|
||||
* straight into the master as a tab's preview (#368). The gain is its volume; hand it to
|
||||
* `releaseMedia` once the element is let go. Null while a bake has the context.
|
||||
* through it and only through it. `routed`, into the bus its sprite `picked` (#418) or else the one
|
||||
* the routing has for its `file`, otherwise straight into the master as a tab's preview (#368). The
|
||||
* gain is its volume; hand it to `releaseMedia` once the element is let go. Null while a bake has
|
||||
* the context.
|
||||
*/
|
||||
mediaInput(element: HTMLMediaElement, file: string, routed: boolean): GainNode | null {
|
||||
mediaInput(element: HTMLMediaElement, file: string, routed: boolean, picked?: string): GainNode | null {
|
||||
this.audio();
|
||||
const context = this.live();
|
||||
if (!context) return null;
|
||||
if (context.state === 'suspended' && !this.held) void context.resume();
|
||||
const gain = context.createGain();
|
||||
const bus = routed ? this.videoBus(file) : MASTER_MIX;
|
||||
const bus = this.mediaBus(file, routed, picked);
|
||||
context.createMediaElementSource(element).connect(gain).connect(this.busInput(bus));
|
||||
this.media.set(gain, { file, routed, bus });
|
||||
this.media.set(gain, { file, routed, ...(picked !== undefined && { picked }), bus });
|
||||
return gain;
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user