Play sounds, patterns and songs on the tracks of a composition (#418)

A trailer needs its music and effects on the same timeline as its
pictures. Until now a composition could only place objects, so sound
had to be started by hand from a behavior and drifted against the
picture.

- Audio items: a sound, a clip of a sound, a pattern or a song dragged
  from the asset panel onto a track becomes an item as long as one
  pass of it. It may share a track with object and video items and
  overlap them.
- Editing works as for other items: move, resize, trim on the left
  (moves the offset), fit to media, delete, copy and paste, box
  selection. Sounds draw their waveform, the colour tells the kind.
- The inspector shows what it plays, start, length, offset, volume
  (-60 to +12 dB) and bus. The bus is "From asset", the master or one
  of the buses, as asked for in the issue.
- An item longer than its material plays in rounds. Sounds and
  patterns start over, a song runs on in its loop. Loop marks on the
  item follow the same rule, a song's first mark sits at its loop end.
- compositionAudio.ts turns the items into sequences, each with its
  own tempo. CompositionSound plays them, in the tab and in a game.
- In the tab the items are heard from the playhead on, straight into
  the master like every tab. A scrub is silent. Playing a song or
  pattern in its own tab stops the composition, and the other way
  round.
- In a game they are planned on the audio clock for the moment the
  composition clock reaches them, with the scheduler's LEAD made up.
  While the clock stands (waiting for a video, paused, placement off)
  they are silent and resume at the right spot.

Found along the way:

- The asset panel never let a song out of its list, so a song could
  not be dragged anywhere. Song and scene tab reject it as they reject
  a pattern.
- The asset panel scrolled to the selected asset on every rebuild of
  its list, so folding a group far below the selection jumped back up.
  Only a new selection scrolls now.
- At the end of the list, folding a group pulled every row down by the
  rows that went away. ScrollContainer has a new opt-in holdOffset,
  set only in the asset panel, that keeps the offset until the user
  scrolls up.

Limits: a time scale other than 1 or a frame longer than 0.1 s moves
the picture and not the sound. A baked song plays its notes here, not
the bake.

Tests:
- tsc, eslint, Prettier clean. Gate on melektaus-atx green,
  309 files, 4788 tests (4772 plus 16 new).
- Unit: composition-audio.test.ts 13 new, scroll-container 3 new.
  Counter-checks: without catching up on load time or without the
  silence on pause the unit tests go red.
- e2e 68h: the sound is planned 1.2 ms after the composition reaches
  0.5 s. Without the LEAD make-up it is 54 ms late and red.
- e2e 04 and 69 ALL PASSED, 86 checks. 69o-ii drags a pattern and a
  song onto a track, 20 frames each.
- e2e 04: folding Sound at the end of the list keeps its header at
  y 234. With the old panel it jumps to 984 and the check is red.
- Not run in a browser: the loop marks of a song item, copy and paste
  of audio items, the bus choice.
- README and wiki count 2072 checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

src/core/commands/compositionCommands.ts

@@ -102,9 +102,9 @@ export function pasteCompositionItems(
   const comp = requireComposition(project, assetId);
   const paste = new PasteObjectsCommand(assetId, clip, {}, { x: 0, y: 0 });
   const ids = new Map(clip.roots.map((r, n) => [r.id, paste.rootIds[n]!]));
-  for (const c of clip.clips ?? []) ids.set(c.id, newId());
+  for (const c of [...(clip.clips ?? []), ...(clip.audio ?? [])]) ids.set(c.id, newId());
   const { items, clips } = pastedItemsWithClips(
-    clip.roots.map((r) => copied.get(r.id)!),
+    [...clip.roots.map((r) => copied.get(r.id)!), ...(clip.audio ?? [])],
     clip.clips ?? [],
     comp.clips,
     ids,

src/core/model/Asset.ts

@@ -274,7 +274,8 @@ export interface CompositionTrack {
  * `offset` is how many seconds into its video the item begins, which a trim of the left edge moves;
  * below 0 the item begins before the video and shows its first picture until then. `fitMedia` says
  * its length still follows that video, or the default while there is none. Items of a track may
- * overlap.
+ * overlap. An item with `audio` spawns no object: it plays a sound, a pattern or a song (#418), and
+ * `offset` is how far into that it begins.
  */
 export interface CompositionItem {
   id: string;
@@ -283,6 +284,21 @@ export interface CompositionItem {
   length: number;
   offset: number;
   fitMedia?: boolean;
+  audio?: CompositionAudio;
+}
+
+export type CompositionAudioKind = 'sound' | 'pattern' | 'song';
+
+/**
+ * What an audio item plays (#418), round and round for as long as it lasts – a song round its loop.
+ * `asset` is a sound, a clip of one, a pattern or a song; `volume` is in decibels. `bus` is the bus
+ * of the mix it is heard on: `''` where the routing sends its asset, else a bus or the master.
+ */
+export interface CompositionAudio {
+  kind: CompositionAudioKind;
+  asset: string;
+  volume: number;
+  bus: string;
 }
 
 /**

src/core/model/clipboard.ts

@@ -43,6 +43,8 @@ export interface ObjectClip {
   clips?: CompositionClip[];
   /** The sprites of a composition's own the objects wear (#419), back for a paste after a cut. */
   sprites?: SpriteAsset[];
+  /** Audio items copied with them (#418): they have no object. */
+  audio?: CompositionItem[];
 }
 
 export interface GraphClip {
@@ -53,10 +55,11 @@ export interface GraphClip {
   annotations: { kind: AnnotationKind; annotation: GraphAnnotation }[];
 }
 
-/** Animation clips of a composition (#417), for a paste into the same one or another. */
+/** Animation clips or audio items of a composition (#417, #418), for a paste into the same one or another. */
 export interface CompositionClipsClip {
   kind: 'compositionClips';
   clips: CompositionClip[];
+  items?: CompositionItem[];
 }
 
 export type Clip = ObjectClip | GraphClip | CompositionClipsClip;

src/core/model/composition.ts

@@ -4,11 +4,15 @@
  * game read the same cut; `compositionHolder` writes it out as objects for a view to draw.
  */
 import { newId } from '../ids';
+import { patternLength, patternTempo } from '../music/pattern';
+import { ITEM_MIN_DB, songLoop, songTempo, SOUND_ITEM_MAX_DB } from '../music/song';
+import { secondsOfTicks } from '../music/time';
 import {
   COMPOSITION_FPS_KEY,
   COMPOSITION_LOOP_KEY,
   DEFAULT_COMPOSITION_FPS,
 } from '../properties/compositions';
+import { playedSoundById } from '../properties/playedSound';
 import { resolveSceneObjectProps } from '../properties/resolve';
 import { VIDEO_CLOCK_KEY, VIDEO_TIME_KEY, videoDrivenOf } from '../properties/videos';
 import { animationDuration, fpsOf, frameCount, targetNamePath } from './animation';
@@ -18,6 +22,8 @@ import type {
   AnimationTrack,
   Asset,
   CompositionAsset,
+  CompositionAudio,
+  CompositionAudioKind,
   CompositionClip,
   CompositionItem,
   CompositionTrack,
@@ -62,9 +68,47 @@ export function itemEnd(item: CompositionItem): number {
   return item.start + item.length;
 }
 
-/** The object an item spawns; undefined once it is gone, and then the item does nothing. */
+/** The object an item spawns; undefined once it is gone, and then the item does nothing – or for an audio item. */
 export function itemObject(comp: CompositionAsset, item: CompositionItem): SceneObject | undefined {
-  return comp.objects.find((o) => o.id === item.id);
+  return item.audio ? undefined : comp.objects.find((o) => o.id === item.id);
+}
+
+/** Whether an item stands on its track: an audio item always (#418), any other while it has its object. */
+export function itemStands(comp: CompositionAsset, item: CompositionItem): boolean {
+  return item.audio !== undefined || itemObject(comp, item) !== undefined;
+}
+
+/** Seconds a sound file lasts once its bytes are there, as the editor's sound library knows it (#418). */
+let soundSeconds: (file: string) => number | null = () => null;
+
+export function setSoundMeasure(measure: (file: string) => number | null): void {
+  soundSeconds = measure;
+}
+
+/**
+ * Seconds the first pass of what an audio item plays lasts: its sound once loaded, its pattern, its
+ * song to the end of its loop – it goes round there, what lies behind is never heard –; 0 while that
+ * is not known or the asset is gone.
+ */
+export function audioSeconds(project: Project, audio: CompositionAudio): number {
+  if (audio.kind === 'sound') {
+    const file = playedSoundById(project, audio.asset);
+    return (file && soundSeconds(file)) || 0;
+  }
+  if (audio.kind === 'pattern') {
+    const pattern = findAssetOf(project, audio.asset, 'pattern');
+    return pattern ? secondsOfTicks(patternLength(pattern), patternTempo(pattern.props)) : 0;
+  }
+  const song = findAssetOf(project, audio.asset, 'song');
+  return song ? secondsOfTicks(songLoop(song).end, songTempo(song.props)) : 0;
+}
+
+/** Seconds every pass after the first lasts: a song's loop, else the same as the first. */
+export function audioRoundSeconds(project: Project, audio: CompositionAudio): number {
+  const song = audio.kind === 'song' ? findAssetOf(project, audio.asset, 'song') : undefined;
+  if (!song) return audioSeconds(project, audio);
+  const loop = songLoop(song);
+  return secondsOfTicks(loop.end - loop.start, songTempo(song.props));
 }
 
 /**
@@ -86,7 +130,7 @@ export function clipLength(project: Project, comp: CompositionAsset, clip: Compo
 /** The frame after the last item or clip ends: how long one run of the composition is. */
 export function compositionLength(project: Project, comp: CompositionAsset): number {
   let end = 0;
-  for (const item of comp.items) if (itemObject(comp, item)) end = Math.max(end, itemEnd(item));
+  for (const item of comp.items) if (itemStands(comp, item)) end = Math.max(end, itemEnd(item));
   for (const clip of comp.clips) end = Math.max(end, clip.start + clipLength(project, comp, clip));
   return end;
 }
@@ -108,13 +152,17 @@ function videoWorn(props: PropertyBag, project: Project): VideoRef | null {
 }
 
 /**
- * The length an item with `fitMedia` takes: from its start to the end of its video, or the default
- * while it wears none or the video's length is not known yet.
+ * The length an item with `fitMedia` takes: from its start to the end of its video, or of what an
+ * audio item plays; the default while it has none or its length is not known yet.
  */
 export function fittedLength(project: Project, comp: CompositionAsset, item: CompositionItem): number {
   const fps = compositionFps(comp);
-  const video = itemVideo(project, comp, item);
-  const seconds = video ? videoLengthOf(video) - item.offset : 0;
+  const video = item.audio ? null : itemVideo(project, comp, item);
+  const seconds = item.audio
+    ? audioSeconds(project, item.audio) - item.offset
+    : video
+      ? videoLengthOf(video) - item.offset
+      : 0;
   return seconds > 0 ? Math.max(1, Math.round(seconds * fps)) : Math.round(DEFAULT_ITEM_SECONDS * fps);
 }
 
@@ -314,8 +362,9 @@ export function heldCompositionTracks(raw: unknown): CompositionTrack[] {
 }
 
 /**
- * Held to what an item may be: the id of an object it spawns and no other item does, a track of
- * the composition, a start in whole frames from 0, at least a frame long, a finite offset.
+ * Held to what an item may be: the id of an object it spawns and no other item does – an audio item
+ * an id of its own and what it plays –, a track of the composition, a start in whole frames from 0,
+ * at least a frame long, a finite offset.
  */
 export function heldCompositionItems(
   raw: unknown,
@@ -329,7 +378,9 @@ export function heldCompositionItems(
     const id = text(i.id);
     const start = Math.round(finite(i.start));
     const length = Math.round(finite(i.length));
-    if (!placed.has(id) || items.some((other) => other.id === id) || !known.has(text(i.track))) continue;
+    const audio = heldAudio(i.audio);
+    if (!id || (audio ? placed.has(id) : !placed.has(id))) continue;
+    if (items.some((other) => other.id === id) || !known.has(text(i.track))) continue;
     if (!(start >= 0) || !(length >= 1)) continue;
     const offset = finite(i.offset);
     const item: CompositionItem = {
@@ -340,11 +391,29 @@ export function heldCompositionItems(
       offset: Number.isFinite(offset) ? offset : 0,
     };
     if (i.fitMedia === true) item.fitMedia = true;
+    if (audio) item.audio = audio;
     items.push(item);
   }
   return items;
 }
 
+const AUDIO_KINDS: readonly CompositionAudioKind[] = ['sound', 'pattern', 'song'];
+
+/** Held to what an audio item plays: a kind, an asset id kept even once the asset is gone, a volume. */
+function heldAudio(raw: unknown): CompositionAudio | null {
+  if (typeof raw !== 'object' || raw === null) return null;
+  const a = raw as Record<string, unknown>;
+  const kind = AUDIO_KINDS.find((k) => k === a.kind);
+  if (!kind || !text(a.asset)) return null;
+  const volume = finite(a.volume);
+  return {
+    kind,
+    asset: text(a.asset),
+    volume: Number.isFinite(volume) ? Math.max(ITEM_MIN_DB, Math.min(SOUND_ITEM_MAX_DB, volume)) : 0,
+    bus: text(a.bus),
+  };
+}
+
 /** Held like an item; an animation that is gone is kept, as an animation keeps a root that is gone. */
 export function heldCompositionClips(raw: unknown, tracks: readonly CompositionTrack[]): CompositionClip[] {
   const known = new Set(tracks.map((t) => t.id));

src/core/model/compositionEdit.ts

@@ -5,9 +5,24 @@
  * into.
  */
 import { videoLoopsOf } from '../properties/videos';
-import type { Asset, CompositionAsset, CompositionClip, CompositionItem } from './Asset';
-import { compositionFps, DEFAULT_ITEM_SECONDS, itemEnd, itemVideo } from './composition';
-import type { Project } from './Project';
+import type {
+  Asset,
+  CompositionAsset,
+  CompositionAudioKind,
+  CompositionClip,
+  CompositionItem,
+} from './Asset';
+import {
+  audioRoundSeconds,
+  audioSeconds,
+  compositionFps,
+  DEFAULT_ITEM_SECONDS,
+  fittedLength,
+  itemEnd,
+  itemVideo,
+} from './composition';
+import { findAsset, type Project } from './Project';
+import { findSound } from './sound';
 import { findVideo, videoLengthOf } from './video';
 
 /** A stretch of a track, in frames. */
@@ -114,7 +129,8 @@ export function movedItems(
 /**
  * The clips that belong to the picked items: a clip belongs to the item lying on its track at its
  * first frame, of two there the one in front – the later start, at the same start the later in the
- * list. A clip that begins before an item and only reaches into it belongs to the one before.
+ * list. A clip that begins before an item and only reaches into it belongs to the one before. An
+ * audio item (#418) has nothing a clip could move, and so has no clips.
  */
 export function carriedClips(
   items: readonly CompositionItem[],
@@ -126,7 +142,7 @@ export function carriedClips(
   for (const clip of clips) {
     let owner: CompositionItem | undefined;
     for (const item of items)
-      if (item.track === clip.track && item.start <= clip.start && clip.start < itemEnd(item))
+      if (!item.audio && item.track === clip.track && item.start <= clip.start && clip.start < itemEnd(item))
         if (!owner || item.start >= owner.start) owner = item;
     if (owner && picked.has(owner.id)) carried.add(clip.id);
   }
@@ -308,32 +324,44 @@ export function trimmedItem(
   return { ...rest, start, length: item.length - by, offset };
 }
 
-/** Where an item's video lies in the composition, in frames: where it begins, how long a pass is, whether it goes round. */
+/**
+ * Where an item's video lies in the composition, in frames: where it begins, how long its first pass
+ * is, whether it goes round, and how long every pass after the first is where that differs – a song
+ * goes round its loop only.
+ */
 export interface ItemMedia {
   begins: number;
   length: number;
   loops: boolean;
+  round?: number;
 }
 
-/** The video of an item as it lies on its track; null without one, or while its length is not known. */
+/**
+ * The video of an item as it lies on its track, or what an audio item plays (#418) – a sound and a
+ * pattern go round, a song to the end of its loop and then round it –; null without one, or while
+ * its length is not known.
+ */
 export function itemMedia(project: Project, comp: CompositionAsset, item: CompositionItem): ItemMedia | null {
-  const video = itemVideo(project, comp, item);
-  const seconds = video ? videoLengthOf(video) : 0;
-  if (!video || !(seconds > 0)) return null;
+  const video = item.audio ? null : itemVideo(project, comp, item);
+  const seconds = item.audio ? audioSeconds(project, item.audio) : video ? videoLengthOf(video) : 0;
+  if (!(seconds > 0)) return null;
   const fps = compositionFps(comp);
-  return {
+  const media = {
     begins: item.start - item.offset * fps,
     length: seconds * fps,
-    loops: videoLoopsOf(video.asset.props),
+    loops: item.audio ? true : videoLoopsOf(video!.asset.props),
   };
+  const round = item.audio?.kind === 'song' ? audioRoundSeconds(project, item.audio) * fps : 0;
+  return round > 0 ? { ...media, round } : media;
 }
 
 /** The frames from `from` to `to` where the video begins or a pass of it ends. */
 export function mediaEdges(media: ItemMedia, from: number, to: number): number[] {
   const edges = media.begins >= from && media.begins <= to ? [media.begins] : [];
-  const first = Math.max(1, Math.ceil((from - media.begins) / media.length));
-  for (let pass = first; pass === 1 || media.loops; pass++) {
-    const end = media.begins + pass * media.length;
+  const round = media.round ?? media.length;
+  const first = media.begins + media.length;
+  for (let pass = Math.max(0, Math.ceil((from - first) / round)); pass === 0 || media.loops; pass++) {
+    const end = first + pass * round;
     if (end > to) break;
     if (end >= from) edges.push(end);
   }
@@ -368,11 +396,28 @@ export function snappedToMedia(frame: number, media: ItemMedia | null, reach: nu
   );
 }
 
+/** What an asset is laid on a track as when it is to be heard (#418): a sound, a clip of one, a pattern or a song. */
+export function audioKindOf(project: Project, assetId: string): CompositionAudioKind | null {
+  const asset = findAsset(project, assetId);
+  if (asset?.type === 'pattern' || asset?.type === 'song') return asset.type;
+  return findSound(project, assetId) ? 'sound' : null;
+}
+
 /**
  * How long an item of `asset` is when it is dropped in: as long as the video it is (#419) or its
- * sprite wears, else 3 s.
+ * sprite wears, or what it plays as an audio item (#418); else 3 s.
  */
 export function droppedLength(project: Project, comp: CompositionAsset, asset: Asset): number {
+  const kind = audioKindOf(project, asset.id);
+  if (kind)
+    return fittedLength(project, comp, {
+      id: '',
+      track: '',
+      start: 0,
+      length: 1,
+      offset: 0,
+      audio: { kind, asset: asset.id, volume: 0, bus: '' },
+    });
   const texture =
     asset.type === 'sprite' ? asset.props.texture : asset.type === 'video' ? asset.id : undefined;
   const video = typeof texture === 'string' ? findVideo(project, texture) : null;
@@ -381,8 +426,9 @@ export function droppedLength(project: Project, comp: CompositionAsset, asset: A
   return seconds > 0 ? Math.max(1, Math.round(seconds * fps)) : Math.round(DEFAULT_ITEM_SECONDS * fps);
 }
 
-/** What an item wears that its length follows: the video or clip's id, '' for none. */
+/** What an item wears that its length follows: the video or clip's id, what an audio item plays, '' for none. */
 export function mediaKey(project: Project, comp: CompositionAsset, item: CompositionItem): string {
+  if (item.audio) return item.audio.asset;
   const video = itemVideo(project, comp, item);
   return video ? (video.clip?.id ?? video.asset.id) : '';
 }

src/core/music/Scheduler.ts

@@ -196,7 +196,7 @@ export class Scheduler {
         note: due.note,
         velocity: due.velocity,
         instrument: due.instrument,
-        tempo: this.tempo,
+        tempo: due.tempo ?? this.tempo,
         ...(due.volume !== undefined && { volume: due.volume }),
         ...(due.bus !== undefined && { bus: due.bus }),
       });
@@ -212,7 +212,7 @@ export class Scheduler {
           fx: due.fx,
           volume: due.volume,
           bus: due.bus,
-          tempo: this.tempo,
+          tempo: due.tempo ?? this.tempo,
         });
         if (voice !== 0) this.voices.push(voice);
       }

src/core/music/compositionAudio.ts

@@ -0,0 +1,198 @@
+/**
+ * What a composition plays of its audio items (#418), as sequences the scheduler plays on the audio
+ * clock. Time is counted in milliseconds – ticks at `COMPOSITION_TEMPO` –, so a frame of any `fps`
+ * lands within a millisecond. A pattern or a song keeps its own tempo inside its item: its notes are
+ * laid out here in the composition's time, each with the tempo a Delay synced to beats counts by.
+ *
+ * Every item is heard on a bus of its own at its volume, through the sound FX of its pattern or song;
+ * a song's tracks are buses of that item. One playing goes to one place of the mix, so the items are
+ * split into one sequence for each place they are sent to.
+ */
+import type { CompositionAsset, CompositionItem } from '../model/Asset';
+import { compositionFps, compositionLength, compositionLoops } from '../model/composition';
+import { findAssetOf, type Project } from '../model/Project';
+import { findSound } from '../model/sound';
+import { pickedFx } from '../properties/assetFx';
+import { MASTER_MIX, type MixRoute, playedRoute } from './mixer';
+import { fxVoiceOf } from './needs';
+import { patternInstrument, patternLength, patternSequence, patternTempo } from './pattern';
+import type { BusLevel } from './playbacks';
+import {
+  clipsIn,
+  type LoopRange,
+  notesIn,
+  type Sequence,
+  type SequenceClip,
+  type SequenceNote,
+} from './sequence';
+import { dbToGain, songLoop, songSequence, songTempo } from './song';
+import { songBuses } from './songBuses';
+import { secondsOfTicks, ticksOfSeconds } from './time';
+
+/** The tempo a composition's sequences count at: 1000 ticks to the second. */
+export const COMPOSITION_TEMPO = 125;
+
+/** Ticks of a composition's sequence: milliseconds. */
+export function compositionTicks(seconds: number): number {
+  return Math.round(ticksOfSeconds(seconds, COMPOSITION_TEMPO));
+}
+
+/** One sequence of a composition and where it goes. */
+export interface CompositionPlaying {
+  route: MixRoute;
+  sequence: Sequence;
+  buses: BusLevel[];
+}
+
+/** The bus an audio item is heard on, at its volume; a song's tracks go into it. */
+export function audioItemBus(compId: string, itemId: string): string {
+  return `composition:${compId}:${itemId}`;
+}
+
+/**
+ * Every audio item of the composition on a track there is, as sequences – one for each place of the
+ * mix they go to, in the order the first item for it lies in the list. With `routed` off all go to
+ * the master, as one.
+ */
+export function compositionPlayings(
+  project: Project,
+  comp: CompositionAsset,
+  routed = true,
+): CompositionPlaying[] {
+  const fps = compositionFps(comp);
+  const tracks = new Set(comp.tracks.map((t) => t.id));
+  const byRoute = new Map<string, CompositionPlaying>();
+  for (const item of comp.items) {
+    if (!item.audio || !tracks.has(item.track)) continue;
+    const laid = laidOut(project, comp.id, item, fps);
+    if (!laid) continue;
+    const route = routed ? playedRoute(project, item.audio.bus, laid.routedAs) : MASTER_MIX;
+    const key = JSON.stringify(route);
+    let playing = byRoute.get(key);
+    if (!playing) byRoute.set(key, (playing = { route, sequence: { notes: [], clips: [] }, buses: [] }));
+    (playing.sequence.notes as SequenceNote[]).push(...laid.notes);
+    (playing.sequence.clips as SequenceClip[]).push(...laid.clips);
+    playing.buses.push(...laid.buses);
+  }
+  const playings = [...byRoute.values()];
+  for (const { sequence } of playings) {
+    (sequence.notes as SequenceNote[]).sort((a, b) => a.tick - b.tick || a.note - b.note);
+    (sequence.clips as SequenceClip[]).sort((a, b) => a.tick - b.tick);
+  }
+  return playings;
+}
+
+/** How long one run of the composition is in ticks, and the loop it goes round where it loops. */
+export function compositionSpan(
+  project: Project,
+  comp: CompositionAsset,
+): { length: number; loop: LoopRange | null } {
+  const length = compositionTicks(compositionLength(project, comp) / compositionFps(comp));
+  return { length, loop: compositionLoops(comp) && length > 0 ? { start: 0, end: length } : null };
+}
+
+interface Laid {
+  notes: SequenceNote[];
+  clips: SequenceClip[];
+  buses: BusLevel[];
+  /** The asset its routing is read from: the sound of a clip, else what it plays. */
+  routedAs: string;
+}
+
+/**
+ * An audio item in the composition's ticks: where it begins and ends, and from how far into what it
+ * plays. An offset below 0 starts that much later in the item.
+ */
+function laidOut(project: Project, compId: string, item: CompositionItem, fps: number): Laid | null {
+  const audio = item.audio!;
+  const late = Math.max(0, -item.offset);
+  const begin = compositionTicks(item.start / fps + late);
+  const end = compositionTicks((item.start + item.length) / fps);
+  if (end <= begin) return null;
+  const into = Math.max(0, item.offset);
+  const seconds = secondsOfTicks(end - begin, COMPOSITION_TEMPO);
+  const bus = audioItemBus(compId, item.id);
+  const own: BusLevel = { name: bus, gain: dbToGain(audio.volume), pan: 0 };
+  if (audio.kind === 'sound') {
+    const ref = findSound(project, audio.asset);
+    if (!ref?.asset.file) return null;
+    const clip = {
+      tick: begin,
+      length: end - begin,
+      sound: audio.asset,
+      fx: pickedFx('', ref.asset.props),
+      offset: into,
+      volume: 1,
+      bus,
+    };
+    return { notes: [], clips: [clip], buses: [own], routedAs: ref.asset.id };
+  }
+  const withFx = (fxId: string, tempo: number): BusLevel => {
+    const fx = fxVoiceOf(project, fxId, tempo);
+    return fx ? { ...own, fx } : own;
+  };
+  if (audio.kind === 'pattern') {
+    const pattern = findAssetOf(project, audio.asset, 'pattern');
+    if (!pattern || !patternInstrument(pattern.props)) return null;
+    const tempo = patternTempo(pattern.props);
+    const length = patternLength(pattern);
+    const heard = played(patternSequence(pattern), { start: 0, end: length }, tempo, into, seconds);
+    return {
+      notes: heard.notes.map((n) => ({ ...n, tick: begin + n.tick, bus })),
+      clips: [],
+      buses: [withFx(pickedFx('', pattern.props), tempo)],
+      routedAs: pattern.id,
+    };
+  }
+  const song = findAssetOf(project, audio.asset, 'song');
+  if (!song) return null;
+  const tempo = songTempo(song.props);
+  const heard = played(songSequence(project, song), songLoop(song), tempo, into, seconds);
+  // Its tracks are the item's: two items of one song do not share them.
+  const prefix = `song:${song.id}`;
+  const renamed = (name: string) => (name.startsWith(prefix) ? `${bus}${name.slice(prefix.length)}` : name);
+  const tracks = songBuses(project, song).map(({ name, into: to, sends, ...level }) => ({
+    ...level,
+    name: renamed(name),
+    ...(to !== undefined && { into: renamed(to) }),
+    ...(sends && { sends: sends.map((send) => ({ ...send, to: send.to ? renamed(send.to) : bus })) }),
+  }));
+  return {
+    notes: heard.notes.map((n) => ({ ...n, tick: begin + n.tick, ...(n.bus && { bus: renamed(n.bus) }) })),
+    clips: heard.clips.map((c) => ({ ...c, tick: begin + c.tick, bus: renamed(c.bus) })),
+    buses: [withFx(pickedFx('', song.props), tempo), ...tracks],
+    routedAs: song.id,
+  };
+}
+
+/**
+ * What a pattern or a song plays from `into` seconds on for `seconds`, round its loop, in ticks of
+ * the composition from 0: each note and clip with its own tempo, none ringing past the end.
+ */
+function played(
+  sequence: Sequence,
+  loop: LoopRange,
+  tempo: number,
+  into: number,
+  seconds: number,
+): { notes: SequenceNote[]; clips: SequenceClip[] } {
+  const from = Math.round(ticksOfSeconds(into, tempo));
+  const to = from + Math.ceil(ticksOfSeconds(seconds, tempo));
+  const span = compositionTicks(seconds);
+  const at = (ticks: number) => compositionTicks(secondsOfTicks(ticks - from, tempo));
+  const notes = notesIn(sequence, 0, loop, from, to).flatMap((n) => {
+    const tick = at(n.at);
+    const length = Math.min(compositionTicks(secondsOfTicks(n.length, tempo)), span - tick);
+    if (tick >= span || length <= 0) return [];
+    const { at: _, ...note } = n;
+    return [{ ...note, tick, length: Math.max(1, length), tempo }];
+  });
+  const clips = clipsIn(sequence, 0, loop, from, to, true).flatMap((c) => {
+    const tick = at(c.at);
+    const length = Math.min(compositionTicks(secondsOfTicks(c.length, tempo)), span - tick);
+    if (tick >= span || length <= 0) return [];
+    const { at: _, skip, ...clip } = c;
+    return [{ ...clip, tick, length, offset: c.offset + secondsOfTicks(skip, tempo), tempo }];
+  });
+  return { notes, clips };
+}

src/core/music/playbacks.ts

@@ -4,6 +4,7 @@ import type { BakePlayback } from './bake';
 import type { MixSend } from './mixer';
 import { LEAD, type ScheduledClip, type ScheduledNote, Scheduler, type SchedulerHost } from './Scheduler';
 import type { LoopRange, Sequence } from './sequence';
+import { ticksOfSeconds } from './time';
 
 /** What a sequence is made of: loaded before the first of its notes and clips is planned. */
 export interface SequenceNeeds {
@@ -56,6 +57,11 @@ export interface SequencePlay {
   strikeClip?(clip: ScheduledClip): number;
   /** Its bake (#340), played in place of its notes and clips where the host can. */
   bake?: BakePlay;
+  /**
+   * Where it starts, in ticks, as heard `LEAD` from now: a composition's sound (#418). Where it first
+   * waits for its files, it starts as much further in, to keep to the clock it was asked for by.
+   */
+  from?: number;
 }
 
 /** A bake as it is played: its file, the level it is heard at, and which stretches of it. */
@@ -158,9 +164,11 @@ export class Playbacks {
       baked: runner instanceof BakedRun,
     };
     this.list.push(playback);
+    const asked = this.host.now();
     void this.host.prepare(play.needs, id).then(() => {
       if (!this.list.includes(playback) || playback.fading !== null) return;
-      runner.start(0);
+      const from = play.from;
+      runner.start(from === undefined ? 0 : from + ticksOfSeconds(this.host.now() - asked, play.tempo));
       // From the first note heard, which is later where the buses are (#346).
       const at = runner.startedAt ?? this.host.now() + LEAD;
       this.host.level(id, play.volume ?? 1, at, Math.max(0, play.fadeIn ?? 0));

src/core/music/sequence.ts

@@ -19,6 +19,8 @@ export interface SequenceNote {
   volume?: number;
   /** The track of a song it is heard on (#334); straight out where left out. */
   bus?: string;
+  /** What a Delay synced to beats counts by, where it is not the sequence's: a pattern in a composition (#418). */
+  tempo?: number;
 }
 
 /**
@@ -34,6 +36,8 @@ export interface SequenceClip {
   offset: number;
   volume: number;
   bus: string;
+  /** As a note's. */
+  tempo?: number;
 }
 
 export interface Sequence {