Keep composition sound in step and give compositions a bus of their own

A composition's sound ran on the audio clock while its picture ran on
the frames. A frame counts at most 0.1 s, so after a hitch the sound
stayed ahead of the picture until the next stop. Songs in a trailer
drifted off the cut. The items of a composition also had no bus of
their own, and a video sprite could not pick where its sound goes.

- CompositionSound.follow compares the audio clock with the picture
  each frame. More than 40 ms apart, it starts the sound again where
  the picture is. This holds in the tab and in a run.
- A time scale other than 1 now restarts the sound again and again.
  It stays in time but comes in pieces.
- The sprite of a video item shows "Driven (by composition)" as a
  fixed Video clock row. Video time is hidden there, because the
  composition sets it every frame.
- Every composition has a bus in the mix and a node in the master
  routing. A new one goes to the master unless other compositions are
  linked elsewhere, apart from where the sounds go.
- Audio items pick From asset, To composition, Master or a bus. New
  items start with To composition. Items saved before keep From asset,
  so music already sent through a bus does not bypass it.
- Video sprites have a new row Video bus: From asset by default, the
  master or a bus. In a composition the default is the composition's
  bus, also for video sprites inside a container item.
- The chosen bus is part of the key of the video player, so a sprite
  with another bus gets a player of its own.

Found along the way:

- Six tests with full lists of sprite defaults expected the old rows
  and now list Video bus too.

Tests:
- tsc, eslint, Prettier clean
- Unit: 971 green in the 28 files touched
- npm run check on melektaus-atx: green, 309 files, 4804 tests
- e2e 68 (melektaus-atx): 13 checks green, 68i new. The sound item
  and the video item go into mix:cp68-cut, which goes to the master.
- e2e 69 (melektaus-atx): ALL PASSED, 23 checks, 69q and 69r new.
  After a hitch of 500 ms the sound restarted twice and then lay
  0.5 to 5 ms off the playhead.
- Counter-checks in unit tests: without the comparison the hitch test
  fails, without the bus in the preload the run test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-14 10:31:16 +02:00
1 parent 875c76026a
commit 4d0a696327
37 files changed
+657 -83

No files matched your search

+1
View File
@@ -305,6 +305,7 @@ export interface CompositionAudio {
kind: CompositionAudioKind;
asset: string;
volume: number;
/** '' along the routing of its asset, `TO_COMPOSITION_BUS` into the composition's bus, else as a Play node picks. */
bus: string;
}
+15 -2
View File
@@ -4,7 +4,17 @@ import { findPixelFrame, pixelFrameRect } from './pixelart';
import { type PropertyBag, textOf, type Vec2 } from './Properties';
import { findAsset, type Project } from './Project';
import { storedFilePattern } from './storedFile';
import { findVideo, isVideoFile, videoKey, videoPlayOf, type VideoRef } from './video';
import { videoBusOf } from '../properties/videos';
import {
findVideo,
isVideoFile,
isVideoKey,
routedVideoKey,
videoKey,
videoMixBus,
videoPlayOf,
type VideoRef,
} from './video';
const FILE_PATTERN = storedFilePattern('png|jpg|webp|gif');
@@ -93,7 +103,10 @@ export function textureRefIds(asset: TextureAsset): string[] {
/** What a sprite wearing `props.texture` shows. */
export function resolveTexture(project: Project, props: PropertyBag): TextureRef | null {
return resolvePicture(project, textOf(props.texture ?? ''));
const ref = resolvePicture(project, textOf(props.texture ?? ''));
// A video heard on a bus the sprite picked plays in a player of its own (#418).
const bus = ref && isVideoKey(ref.file) ? videoMixBus(project, videoBusOf(props)) : null;
return bus ? { ...ref!, file: routedVideoKey(ref!.file, bus) } : ref;
}
/** A texture, a frame, a video or a clip of one, or a generating asset; any other asset's id is no picture. */
+33 -6
View File
@@ -1,6 +1,7 @@
import { FROM_ASSET_BUS, MASTER_MIX, mixBusName, pickedComposition } from '../properties/buses';
import { videoLoopsOf, videoSpanOf, videoVolumeOf } from '../properties/videos';
import type { VideoAsset, VideoClip } from './Asset';
import { findAsset, type Project } from './Project';
import { findAsset, findAssetOf, type Project } from './Project';
import { clipsOf } from './sound';
import { storedFilePattern } from './storedFile';
@@ -64,6 +65,8 @@ export interface VideoPlay {
end: number;
loop: boolean;
volume: number;
/** The bus of the mix its sound goes into, where a sprite picked one; along its routing where left out. */
bus?: string;
}
export function videoPlayOf(ref: VideoRef): VideoPlay {
@@ -107,12 +110,33 @@ const PREFIX = 'video:';
/**
* The name a playing video goes by where a picture is named by `TextureRef.file` (see
* `generated.ts`): `video:<file>#<start>~<end>~<loop>~<volume>`. No uploaded image looks like it,
* and one key is one player: two sprites wearing the same clip show the same picture.
* `generated.ts`): `video:<file>#<start>~<end>~<loop>~<volume>`, and `!<bus>` where its sound goes
* into a bus a sprite picked. No uploaded image looks like it, and one key is one player: two sprites
* wearing the same clip show the same picture.
*/
export function videoKey(play: VideoPlay): string {
const end = Number.isFinite(play.end) ? play.end : 0;
return `${PREFIX}${play.file}#${play.start}~${end}~${play.loop ? 1 : 0}~${play.volume}`;
const bus = play.bus ? `!${play.bus}` : '';
return `${PREFIX}${play.file}#${play.start}~${end}~${play.loop ? 1 : 0}~${play.volume}${bus}`;
}
/**
* The bus of the mix the Video bus of a sprite names: null along the routing of its video, the
* composition's for one it picked for a sprite of its items. A bus that is gone is the master.
*/
export function videoMixBus(project: Project, picked: string): string | null {
if (picked === '' || picked === FROM_ASSET_BUS) return null;
const comp = pickedComposition(picked);
if (comp !== null) return findAssetOf(project, comp, 'composition') ? mixBusName(comp) : null;
return findAssetOf(project, picked, 'bus') ? mixBusName(picked) : MASTER_MIX;
}
/** The key of the same video heard on `bus` (#418); along its routing for null. */
export function routedVideoKey(key: string, bus: string | null): string {
const play = parseVideoKey(key);
if (!play || isDrivenVideoKey(key)) return key;
const { bus: _, ...rest } = play;
return videoKey(bus ? { ...rest, bus } : rest);
}
/**
@@ -138,11 +162,14 @@ export function parseVideoKey(key: string): VideoPlay | null {
if (hash < 0) return null;
const file = key.slice(PREFIX.length, hash);
const at = key.indexOf('@', hash);
const parts = key.slice(hash + 1, at < 0 ? undefined : at).split('~');
const body = key.slice(hash + 1, at < 0 ? undefined : at);
const bang = body.indexOf('!');
const parts = (bang < 0 ? body : body.slice(0, bang)).split('~');
const [start, end, loop, volume] = parts.map(Number);
if (!isVideoFile(file) || parts.length !== 4 || ![start, end, loop, volume].every(Number.isFinite))
return null;
return { file, start: start!, end: end! > 0 ? end! : Infinity, loop: loop === 1, volume: volume! };
const play = { file, start: start!, end: end! > 0 ? end! : Infinity, loop: loop === 1, volume: volume! };
return bang < 0 ? play : { ...play, bus: body.slice(bang + 1) };
}
/** How far before its end a held clip shows its last picture: the end itself is the next one's. */