Play sounds, patterns and songs on the tracks of a composition (#418)
A trailer needs its music and effects on the same timeline as its pictures. Until now a composition could only place objects, so sound had to be started by hand from a behavior and drifted against the picture. - Audio items: a sound, a clip of a sound, a pattern or a song dragged from the asset panel onto a track becomes an item as long as one pass of it. It may share a track with object and video items and overlap them. - Editing works as for other items: move, resize, trim on the left (moves the offset), fit to media, delete, copy and paste, box selection. Sounds draw their waveform, the colour tells the kind. - The inspector shows what it plays, start, length, offset, volume (-60 to +12 dB) and bus. The bus is "From asset", the master or one of the buses, as asked for in the issue. - An item longer than its material plays in rounds. Sounds and patterns start over, a song runs on in its loop. Loop marks on the item follow the same rule, a song's first mark sits at its loop end. - compositionAudio.ts turns the items into sequences, each with its own tempo. CompositionSound plays them, in the tab and in a game. - In the tab the items are heard from the playhead on, straight into the master like every tab. A scrub is silent. Playing a song or pattern in its own tab stops the composition, and the other way round. - In a game they are planned on the audio clock for the moment the composition clock reaches them, with the scheduler's LEAD made up. While the clock stands (waiting for a video, paused, placement off) they are silent and resume at the right spot. Found along the way: - The asset panel never let a song out of its list, so a song could not be dragged anywhere. Song and scene tab reject it as they reject a pattern. - The asset panel scrolled to the selected asset on every rebuild of its list, so folding a group far below the selection jumped back up. Only a new selection scrolls now. - At the end of the list, folding a group pulled every row down by the rows that went away. ScrollContainer has a new opt-in holdOffset, set only in the asset panel, that keeps the offset until the user scrolls up. Limits: a time scale other than 1 or a frame longer than 0.1 s moves the picture and not the sound. A baked song plays its notes here, not the bake. Tests: - tsc, eslint, Prettier clean. Gate on melektaus-atx green, 309 files, 4788 tests (4772 plus 16 new). - Unit: composition-audio.test.ts 13 new, scroll-container 3 new. Counter-checks: without catching up on load time or without the silence on pause the unit tests go red. - e2e 68h: the sound is planned 1.2 ms after the composition reaches 0.5 s. Without the LEAD make-up it is 54 ms late and red. - e2e 04 and 69 ALL PASSED, 86 checks. 69o-ii drags a pattern and a song onto a track, 20 frames each. - e2e 04: folding Sound at the end of the list keeps its header at y 234. With the old panel it jumps to 984 and the check is red. - Not run in a browser: the loop marks of a song item, copy and paste of audio items, the bus choice. - README and wiki count 2072 checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
30b618d9a9
commit
77ac9d1aaf
30 files changed
+1571
-101
No files matched your search
@@ -47,7 +47,7 @@ npm run build # writes dist/ and regenerates index.html; the folder is wh
|
||||
The browser suite and the three example games need a machine with a GPU:
|
||||
|
||||
```bash
|
||||
node tools/seed-e2e.mjs && node tools/e2e.mjs # the editor, 2068 checks (GM_STEPS=20 for one step)
|
||||
node tools/seed-e2e.mjs && node tools/e2e.mjs # the editor, 2072 checks (GM_STEPS=20 for one step)
|
||||
# every tool signs in first – see the Accounts page
|
||||
node tools/seed-jumprun.mjs && node tools/smoke-jumprun.mjs
|
||||
node tools/seed-tetris.mjs && node tools/smoke-tetris.mjs
|
||||
|
||||
@@ -102,9 +102,9 @@ export function pasteCompositionItems(
|
||||
const comp = requireComposition(project, assetId);
|
||||
const paste = new PasteObjectsCommand(assetId, clip, {}, { x: 0, y: 0 });
|
||||
const ids = new Map(clip.roots.map((r, n) => [r.id, paste.rootIds[n]!]));
|
||||
for (const c of clip.clips ?? []) ids.set(c.id, newId());
|
||||
for (const c of [...(clip.clips ?? []), ...(clip.audio ?? [])]) ids.set(c.id, newId());
|
||||
const { items, clips } = pastedItemsWithClips(
|
||||
clip.roots.map((r) => copied.get(r.id)!),
|
||||
[...clip.roots.map((r) => copied.get(r.id)!), ...(clip.audio ?? [])],
|
||||
clip.clips ?? [],
|
||||
comp.clips,
|
||||
ids,
|
||||
|
||||
+17
-1
@@ -274,7 +274,8 @@ export interface CompositionTrack {
|
||||
* `offset` is how many seconds into its video the item begins, which a trim of the left edge moves;
|
||||
* below 0 the item begins before the video and shows its first picture until then. `fitMedia` says
|
||||
* its length still follows that video, or the default while there is none. Items of a track may
|
||||
* overlap.
|
||||
* overlap. An item with `audio` spawns no object: it plays a sound, a pattern or a song (#418), and
|
||||
* `offset` is how far into that it begins.
|
||||
*/
|
||||
export interface CompositionItem {
|
||||
id: string;
|
||||
@@ -283,6 +284,21 @@ export interface CompositionItem {
|
||||
length: number;
|
||||
offset: number;
|
||||
fitMedia?: boolean;
|
||||
audio?: CompositionAudio;
|
||||
}
|
||||
|
||||
export type CompositionAudioKind = 'sound' | 'pattern' | 'song';
|
||||
|
||||
/**
|
||||
* What an audio item plays (#418), round and round for as long as it lasts – a song round its loop.
|
||||
* `asset` is a sound, a clip of one, a pattern or a song; `volume` is in decibels. `bus` is the bus
|
||||
* of the mix it is heard on: `''` where the routing sends its asset, else a bus or the master.
|
||||
*/
|
||||
export interface CompositionAudio {
|
||||
kind: CompositionAudioKind;
|
||||
asset: string;
|
||||
volume: number;
|
||||
bus: string;
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -43,6 +43,8 @@ export interface ObjectClip {
|
||||
clips?: CompositionClip[];
|
||||
/** The sprites of a composition's own the objects wear (#419), back for a paste after a cut. */
|
||||
sprites?: SpriteAsset[];
|
||||
/** Audio items copied with them (#418): they have no object. */
|
||||
audio?: CompositionItem[];
|
||||
}
|
||||
|
||||
export interface GraphClip {
|
||||
@@ -53,10 +55,11 @@ export interface GraphClip {
|
||||
annotations: { kind: AnnotationKind; annotation: GraphAnnotation }[];
|
||||
}
|
||||
|
||||
/** Animation clips of a composition (#417), for a paste into the same one or another. */
|
||||
/** Animation clips or audio items of a composition (#417, #418), for a paste into the same one or another. */
|
||||
export interface CompositionClipsClip {
|
||||
kind: 'compositionClips';
|
||||
clips: CompositionClip[];
|
||||
items?: CompositionItem[];
|
||||
}
|
||||
|
||||
export type Clip = ObjectClip | GraphClip | CompositionClipsClip;
|
||||
|
||||
@@ -4,11 +4,15 @@
|
||||
* game read the same cut; `compositionHolder` writes it out as objects for a view to draw.
|
||||
*/
|
||||
import { newId } from '../ids';
|
||||
import { patternLength, patternTempo } from '../music/pattern';
|
||||
import { ITEM_MIN_DB, songLoop, songTempo, SOUND_ITEM_MAX_DB } from '../music/song';
|
||||
import { secondsOfTicks } from '../music/time';
|
||||
import {
|
||||
COMPOSITION_FPS_KEY,
|
||||
COMPOSITION_LOOP_KEY,
|
||||
DEFAULT_COMPOSITION_FPS,
|
||||
} from '../properties/compositions';
|
||||
import { playedSoundById } from '../properties/playedSound';
|
||||
import { resolveSceneObjectProps } from '../properties/resolve';
|
||||
import { VIDEO_CLOCK_KEY, VIDEO_TIME_KEY, videoDrivenOf } from '../properties/videos';
|
||||
import { animationDuration, fpsOf, frameCount, targetNamePath } from './animation';
|
||||
@@ -18,6 +22,8 @@ import type {
|
||||
AnimationTrack,
|
||||
Asset,
|
||||
CompositionAsset,
|
||||
CompositionAudio,
|
||||
CompositionAudioKind,
|
||||
CompositionClip,
|
||||
CompositionItem,
|
||||
CompositionTrack,
|
||||
@@ -62,9 +68,47 @@ export function itemEnd(item: CompositionItem): number {
|
||||
return item.start + item.length;
|
||||
}
|
||||
|
||||
/** The object an item spawns; undefined once it is gone, and then the item does nothing. */
|
||||
/** The object an item spawns; undefined once it is gone, and then the item does nothing – or for an audio item. */
|
||||
export function itemObject(comp: CompositionAsset, item: CompositionItem): SceneObject | undefined {
|
||||
return comp.objects.find((o) => o.id === item.id);
|
||||
return item.audio ? undefined : comp.objects.find((o) => o.id === item.id);
|
||||
}
|
||||
|
||||
/** Whether an item stands on its track: an audio item always (#418), any other while it has its object. */
|
||||
export function itemStands(comp: CompositionAsset, item: CompositionItem): boolean {
|
||||
return item.audio !== undefined || itemObject(comp, item) !== undefined;
|
||||
}
|
||||
|
||||
/** Seconds a sound file lasts once its bytes are there, as the editor's sound library knows it (#418). */
|
||||
let soundSeconds: (file: string) => number | null = () => null;
|
||||
|
||||
export function setSoundMeasure(measure: (file: string) => number | null): void {
|
||||
soundSeconds = measure;
|
||||
}
|
||||
|
||||
/**
|
||||
* Seconds the first pass of what an audio item plays lasts: its sound once loaded, its pattern, its
|
||||
* song to the end of its loop – it goes round there, what lies behind is never heard –; 0 while that
|
||||
* is not known or the asset is gone.
|
||||
*/
|
||||
export function audioSeconds(project: Project, audio: CompositionAudio): number {
|
||||
if (audio.kind === 'sound') {
|
||||
const file = playedSoundById(project, audio.asset);
|
||||
return (file && soundSeconds(file)) || 0;
|
||||
}
|
||||
if (audio.kind === 'pattern') {
|
||||
const pattern = findAssetOf(project, audio.asset, 'pattern');
|
||||
return pattern ? secondsOfTicks(patternLength(pattern), patternTempo(pattern.props)) : 0;
|
||||
}
|
||||
const song = findAssetOf(project, audio.asset, 'song');
|
||||
return song ? secondsOfTicks(songLoop(song).end, songTempo(song.props)) : 0;
|
||||
}
|
||||
|
||||
/** Seconds every pass after the first lasts: a song's loop, else the same as the first. */
|
||||
export function audioRoundSeconds(project: Project, audio: CompositionAudio): number {
|
||||
const song = audio.kind === 'song' ? findAssetOf(project, audio.asset, 'song') : undefined;
|
||||
if (!song) return audioSeconds(project, audio);
|
||||
const loop = songLoop(song);
|
||||
return secondsOfTicks(loop.end - loop.start, songTempo(song.props));
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -86,7 +130,7 @@ export function clipLength(project: Project, comp: CompositionAsset, clip: Compo
|
||||
/** The frame after the last item or clip ends: how long one run of the composition is. */
|
||||
export function compositionLength(project: Project, comp: CompositionAsset): number {
|
||||
let end = 0;
|
||||
for (const item of comp.items) if (itemObject(comp, item)) end = Math.max(end, itemEnd(item));
|
||||
for (const item of comp.items) if (itemStands(comp, item)) end = Math.max(end, itemEnd(item));
|
||||
for (const clip of comp.clips) end = Math.max(end, clip.start + clipLength(project, comp, clip));
|
||||
return end;
|
||||
}
|
||||
@@ -108,13 +152,17 @@ function videoWorn(props: PropertyBag, project: Project): VideoRef | null {
|
||||
}
|
||||
|
||||
/**
|
||||
* The length an item with `fitMedia` takes: from its start to the end of its video, or the default
|
||||
* while it wears none or the video's length is not known yet.
|
||||
* The length an item with `fitMedia` takes: from its start to the end of its video, or of what an
|
||||
* audio item plays; the default while it has none or its length is not known yet.
|
||||
*/
|
||||
export function fittedLength(project: Project, comp: CompositionAsset, item: CompositionItem): number {
|
||||
const fps = compositionFps(comp);
|
||||
const video = itemVideo(project, comp, item);
|
||||
const seconds = video ? videoLengthOf(video) - item.offset : 0;
|
||||
const video = item.audio ? null : itemVideo(project, comp, item);
|
||||
const seconds = item.audio
|
||||
? audioSeconds(project, item.audio) - item.offset
|
||||
: video
|
||||
? videoLengthOf(video) - item.offset
|
||||
: 0;
|
||||
return seconds > 0 ? Math.max(1, Math.round(seconds * fps)) : Math.round(DEFAULT_ITEM_SECONDS * fps);
|
||||
}
|
||||
|
||||
@@ -314,8 +362,9 @@ export function heldCompositionTracks(raw: unknown): CompositionTrack[] {
|
||||
}
|
||||
|
||||
/**
|
||||
* Held to what an item may be: the id of an object it spawns and no other item does, a track of
|
||||
* the composition, a start in whole frames from 0, at least a frame long, a finite offset.
|
||||
* Held to what an item may be: the id of an object it spawns and no other item does – an audio item
|
||||
* an id of its own and what it plays –, a track of the composition, a start in whole frames from 0,
|
||||
* at least a frame long, a finite offset.
|
||||
*/
|
||||
export function heldCompositionItems(
|
||||
raw: unknown,
|
||||
@@ -329,7 +378,9 @@ export function heldCompositionItems(
|
||||
const id = text(i.id);
|
||||
const start = Math.round(finite(i.start));
|
||||
const length = Math.round(finite(i.length));
|
||||
if (!placed.has(id) || items.some((other) => other.id === id) || !known.has(text(i.track))) continue;
|
||||
const audio = heldAudio(i.audio);
|
||||
if (!id || (audio ? placed.has(id) : !placed.has(id))) continue;
|
||||
if (items.some((other) => other.id === id) || !known.has(text(i.track))) continue;
|
||||
if (!(start >= 0) || !(length >= 1)) continue;
|
||||
const offset = finite(i.offset);
|
||||
const item: CompositionItem = {
|
||||
@@ -340,11 +391,29 @@ export function heldCompositionItems(
|
||||
offset: Number.isFinite(offset) ? offset : 0,
|
||||
};
|
||||
if (i.fitMedia === true) item.fitMedia = true;
|
||||
if (audio) item.audio = audio;
|
||||
items.push(item);
|
||||
}
|
||||
return items;
|
||||
}
|
||||
|
||||
const AUDIO_KINDS: readonly CompositionAudioKind[] = ['sound', 'pattern', 'song'];
|
||||
|
||||
/** Held to what an audio item plays: a kind, an asset id kept even once the asset is gone, a volume. */
|
||||
function heldAudio(raw: unknown): CompositionAudio | null {
|
||||
if (typeof raw !== 'object' || raw === null) return null;
|
||||
const a = raw as Record<string, unknown>;
|
||||
const kind = AUDIO_KINDS.find((k) => k === a.kind);
|
||||
if (!kind || !text(a.asset)) return null;
|
||||
const volume = finite(a.volume);
|
||||
return {
|
||||
kind,
|
||||
asset: text(a.asset),
|
||||
volume: Number.isFinite(volume) ? Math.max(ITEM_MIN_DB, Math.min(SOUND_ITEM_MAX_DB, volume)) : 0,
|
||||
bus: text(a.bus),
|
||||
};
|
||||
}
|
||||
|
||||
/** Held like an item; an animation that is gone is kept, as an animation keeps a root that is gone. */
|
||||
export function heldCompositionClips(raw: unknown, tracks: readonly CompositionTrack[]): CompositionClip[] {
|
||||
const known = new Set(tracks.map((t) => t.id));
|
||||
|
||||
@@ -5,9 +5,24 @@
|
||||
* into.
|
||||
*/
|
||||
import { videoLoopsOf } from '../properties/videos';
|
||||
import type { Asset, CompositionAsset, CompositionClip, CompositionItem } from './Asset';
|
||||
import { compositionFps, DEFAULT_ITEM_SECONDS, itemEnd, itemVideo } from './composition';
|
||||
import type { Project } from './Project';
|
||||
import type {
|
||||
Asset,
|
||||
CompositionAsset,
|
||||
CompositionAudioKind,
|
||||
CompositionClip,
|
||||
CompositionItem,
|
||||
} from './Asset';
|
||||
import {
|
||||
audioRoundSeconds,
|
||||
audioSeconds,
|
||||
compositionFps,
|
||||
DEFAULT_ITEM_SECONDS,
|
||||
fittedLength,
|
||||
itemEnd,
|
||||
itemVideo,
|
||||
} from './composition';
|
||||
import { findAsset, type Project } from './Project';
|
||||
import { findSound } from './sound';
|
||||
import { findVideo, videoLengthOf } from './video';
|
||||
|
||||
/** A stretch of a track, in frames. */
|
||||
@@ -114,7 +129,8 @@ export function movedItems(
|
||||
/**
|
||||
* The clips that belong to the picked items: a clip belongs to the item lying on its track at its
|
||||
* first frame, of two there the one in front – the later start, at the same start the later in the
|
||||
* list. A clip that begins before an item and only reaches into it belongs to the one before.
|
||||
* list. A clip that begins before an item and only reaches into it belongs to the one before. An
|
||||
* audio item (#418) has nothing a clip could move, and so has no clips.
|
||||
*/
|
||||
export function carriedClips(
|
||||
items: readonly CompositionItem[],
|
||||
@@ -126,7 +142,7 @@ export function carriedClips(
|
||||
for (const clip of clips) {
|
||||
let owner: CompositionItem | undefined;
|
||||
for (const item of items)
|
||||
if (item.track === clip.track && item.start <= clip.start && clip.start < itemEnd(item))
|
||||
if (!item.audio && item.track === clip.track && item.start <= clip.start && clip.start < itemEnd(item))
|
||||
if (!owner || item.start >= owner.start) owner = item;
|
||||
if (owner && picked.has(owner.id)) carried.add(clip.id);
|
||||
}
|
||||
@@ -308,32 +324,44 @@ export function trimmedItem(
|
||||
return { ...rest, start, length: item.length - by, offset };
|
||||
}
|
||||
|
||||
/** Where an item's video lies in the composition, in frames: where it begins, how long a pass is, whether it goes round. */
|
||||
/**
|
||||
* Where an item's video lies in the composition, in frames: where it begins, how long its first pass
|
||||
* is, whether it goes round, and how long every pass after the first is where that differs – a song
|
||||
* goes round its loop only.
|
||||
*/
|
||||
export interface ItemMedia {
|
||||
begins: number;
|
||||
length: number;
|
||||
loops: boolean;
|
||||
round?: number;
|
||||
}
|
||||
|
||||
/** The video of an item as it lies on its track; null without one, or while its length is not known. */
|
||||
/**
|
||||
* The video of an item as it lies on its track, or what an audio item plays (#418) – a sound and a
|
||||
* pattern go round, a song to the end of its loop and then round it –; null without one, or while
|
||||
* its length is not known.
|
||||
*/
|
||||
export function itemMedia(project: Project, comp: CompositionAsset, item: CompositionItem): ItemMedia | null {
|
||||
const video = itemVideo(project, comp, item);
|
||||
const seconds = video ? videoLengthOf(video) : 0;
|
||||
if (!video || !(seconds > 0)) return null;
|
||||
const video = item.audio ? null : itemVideo(project, comp, item);
|
||||
const seconds = item.audio ? audioSeconds(project, item.audio) : video ? videoLengthOf(video) : 0;
|
||||
if (!(seconds > 0)) return null;
|
||||
const fps = compositionFps(comp);
|
||||
return {
|
||||
const media = {
|
||||
begins: item.start - item.offset * fps,
|
||||
length: seconds * fps,
|
||||
loops: videoLoopsOf(video.asset.props),
|
||||
loops: item.audio ? true : videoLoopsOf(video!.asset.props),
|
||||
};
|
||||
const round = item.audio?.kind === 'song' ? audioRoundSeconds(project, item.audio) * fps : 0;
|
||||
return round > 0 ? { ...media, round } : media;
|
||||
}
|
||||
|
||||
/** The frames from `from` to `to` where the video begins or a pass of it ends. */
|
||||
export function mediaEdges(media: ItemMedia, from: number, to: number): number[] {
|
||||
const edges = media.begins >= from && media.begins <= to ? [media.begins] : [];
|
||||
const first = Math.max(1, Math.ceil((from - media.begins) / media.length));
|
||||
for (let pass = first; pass === 1 || media.loops; pass++) {
|
||||
const end = media.begins + pass * media.length;
|
||||
const round = media.round ?? media.length;
|
||||
const first = media.begins + media.length;
|
||||
for (let pass = Math.max(0, Math.ceil((from - first) / round)); pass === 0 || media.loops; pass++) {
|
||||
const end = first + pass * round;
|
||||
if (end > to) break;
|
||||
if (end >= from) edges.push(end);
|
||||
}
|
||||
@@ -368,11 +396,28 @@ export function snappedToMedia(frame: number, media: ItemMedia | null, reach: nu
|
||||
);
|
||||
}
|
||||
|
||||
/** What an asset is laid on a track as when it is to be heard (#418): a sound, a clip of one, a pattern or a song. */
|
||||
export function audioKindOf(project: Project, assetId: string): CompositionAudioKind | null {
|
||||
const asset = findAsset(project, assetId);
|
||||
if (asset?.type === 'pattern' || asset?.type === 'song') return asset.type;
|
||||
return findSound(project, assetId) ? 'sound' : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* How long an item of `asset` is when it is dropped in: as long as the video it is (#419) or its
|
||||
* sprite wears, else 3 s.
|
||||
* sprite wears, or what it plays as an audio item (#418); else 3 s.
|
||||
*/
|
||||
export function droppedLength(project: Project, comp: CompositionAsset, asset: Asset): number {
|
||||
const kind = audioKindOf(project, asset.id);
|
||||
if (kind)
|
||||
return fittedLength(project, comp, {
|
||||
id: '',
|
||||
track: '',
|
||||
start: 0,
|
||||
length: 1,
|
||||
offset: 0,
|
||||
audio: { kind, asset: asset.id, volume: 0, bus: '' },
|
||||
});
|
||||
const texture =
|
||||
asset.type === 'sprite' ? asset.props.texture : asset.type === 'video' ? asset.id : undefined;
|
||||
const video = typeof texture === 'string' ? findVideo(project, texture) : null;
|
||||
@@ -381,8 +426,9 @@ export function droppedLength(project: Project, comp: CompositionAsset, asset: A
|
||||
return seconds > 0 ? Math.max(1, Math.round(seconds * fps)) : Math.round(DEFAULT_ITEM_SECONDS * fps);
|
||||
}
|
||||
|
||||
/** What an item wears that its length follows: the video or clip's id, '' for none. */
|
||||
/** What an item wears that its length follows: the video or clip's id, what an audio item plays, '' for none. */
|
||||
export function mediaKey(project: Project, comp: CompositionAsset, item: CompositionItem): string {
|
||||
if (item.audio) return item.audio.asset;
|
||||
const video = itemVideo(project, comp, item);
|
||||
return video ? (video.clip?.id ?? video.asset.id) : '';
|
||||
}
|
||||
@@ -196,7 +196,7 @@ export class Scheduler {
|
||||
note: due.note,
|
||||
velocity: due.velocity,
|
||||
instrument: due.instrument,
|
||||
tempo: this.tempo,
|
||||
tempo: due.tempo ?? this.tempo,
|
||||
...(due.volume !== undefined && { volume: due.volume }),
|
||||
...(due.bus !== undefined && { bus: due.bus }),
|
||||
});
|
||||
@@ -212,7 +212,7 @@ export class Scheduler {
|
||||
fx: due.fx,
|
||||
volume: due.volume,
|
||||
bus: due.bus,
|
||||
tempo: this.tempo,
|
||||
tempo: due.tempo ?? this.tempo,
|
||||
});
|
||||
if (voice !== 0) this.voices.push(voice);
|
||||
}
|
||||
|
||||
@@ -0,0 +1,198 @@
|
||||
/**
|
||||
* What a composition plays of its audio items (#418), as sequences the scheduler plays on the audio
|
||||
* clock. Time is counted in milliseconds – ticks at `COMPOSITION_TEMPO` –, so a frame of any `fps`
|
||||
* lands within a millisecond. A pattern or a song keeps its own tempo inside its item: its notes are
|
||||
* laid out here in the composition's time, each with the tempo a Delay synced to beats counts by.
|
||||
*
|
||||
* Every item is heard on a bus of its own at its volume, through the sound FX of its pattern or song;
|
||||
* a song's tracks are buses of that item. One playing goes to one place of the mix, so the items are
|
||||
* split into one sequence for each place they are sent to.
|
||||
*/
|
||||
import type { CompositionAsset, CompositionItem } from '../model/Asset';
|
||||
import { compositionFps, compositionLength, compositionLoops } from '../model/composition';
|
||||
import { findAssetOf, type Project } from '../model/Project';
|
||||
import { findSound } from '../model/sound';
|
||||
import { pickedFx } from '../properties/assetFx';
|
||||
import { MASTER_MIX, type MixRoute, playedRoute } from './mixer';
|
||||
import { fxVoiceOf } from './needs';
|
||||
import { patternInstrument, patternLength, patternSequence, patternTempo } from './pattern';
|
||||
import type { BusLevel } from './playbacks';
|
||||
import {
|
||||
clipsIn,
|
||||
type LoopRange,
|
||||
notesIn,
|
||||
type Sequence,
|
||||
type SequenceClip,
|
||||
type SequenceNote,
|
||||
} from './sequence';
|
||||
import { dbToGain, songLoop, songSequence, songTempo } from './song';
|
||||
import { songBuses } from './songBuses';
|
||||
import { secondsOfTicks, ticksOfSeconds } from './time';
|
||||
|
||||
/** The tempo a composition's sequences count at: 1000 ticks to the second. */
|
||||
export const COMPOSITION_TEMPO = 125;
|
||||
|
||||
/** Ticks of a composition's sequence: milliseconds. */
|
||||
export function compositionTicks(seconds: number): number {
|
||||
return Math.round(ticksOfSeconds(seconds, COMPOSITION_TEMPO));
|
||||
}
|
||||
|
||||
/** One sequence of a composition and where it goes. */
|
||||
export interface CompositionPlaying {
|
||||
route: MixRoute;
|
||||
sequence: Sequence;
|
||||
buses: BusLevel[];
|
||||
}
|
||||
|
||||
/** The bus an audio item is heard on, at its volume; a song's tracks go into it. */
|
||||
export function audioItemBus(compId: string, itemId: string): string {
|
||||
return `composition:${compId}:${itemId}`;
|
||||
}
|
||||
|
||||
/**
|
||||
* Every audio item of the composition on a track there is, as sequences – one for each place of the
|
||||
* mix they go to, in the order the first item for it lies in the list. With `routed` off all go to
|
||||
* the master, as one.
|
||||
*/
|
||||
export function compositionPlayings(
|
||||
project: Project,
|
||||
comp: CompositionAsset,
|
||||
routed = true,
|
||||
): CompositionPlaying[] {
|
||||
const fps = compositionFps(comp);
|
||||
const tracks = new Set(comp.tracks.map((t) => t.id));
|
||||
const byRoute = new Map<string, CompositionPlaying>();
|
||||
for (const item of comp.items) {
|
||||
if (!item.audio || !tracks.has(item.track)) continue;
|
||||
const laid = laidOut(project, comp.id, item, fps);
|
||||
if (!laid) continue;
|
||||
const route = routed ? playedRoute(project, item.audio.bus, laid.routedAs) : MASTER_MIX;
|
||||
const key = JSON.stringify(route);
|
||||
let playing = byRoute.get(key);
|
||||
if (!playing) byRoute.set(key, (playing = { route, sequence: { notes: [], clips: [] }, buses: [] }));
|
||||
(playing.sequence.notes as SequenceNote[]).push(...laid.notes);
|
||||
(playing.sequence.clips as SequenceClip[]).push(...laid.clips);
|
||||
playing.buses.push(...laid.buses);
|
||||
}
|
||||
const playings = [...byRoute.values()];
|
||||
for (const { sequence } of playings) {
|
||||
(sequence.notes as SequenceNote[]).sort((a, b) => a.tick - b.tick || a.note - b.note);
|
||||
(sequence.clips as SequenceClip[]).sort((a, b) => a.tick - b.tick);
|
||||
}
|
||||
return playings;
|
||||
}
|
||||
|
||||
/** How long one run of the composition is in ticks, and the loop it goes round where it loops. */
|
||||
export function compositionSpan(
|
||||
project: Project,
|
||||
comp: CompositionAsset,
|
||||
): { length: number; loop: LoopRange | null } {
|
||||
const length = compositionTicks(compositionLength(project, comp) / compositionFps(comp));
|
||||
return { length, loop: compositionLoops(comp) && length > 0 ? { start: 0, end: length } : null };
|
||||
}
|
||||
|
||||
interface Laid {
|
||||
notes: SequenceNote[];
|
||||
clips: SequenceClip[];
|
||||
buses: BusLevel[];
|
||||
/** The asset its routing is read from: the sound of a clip, else what it plays. */
|
||||
routedAs: string;
|
||||
}
|
||||
|
||||
/**
|
||||
* An audio item in the composition's ticks: where it begins and ends, and from how far into what it
|
||||
* plays. An offset below 0 starts that much later in the item.
|
||||
*/
|
||||
function laidOut(project: Project, compId: string, item: CompositionItem, fps: number): Laid | null {
|
||||
const audio = item.audio!;
|
||||
const late = Math.max(0, -item.offset);
|
||||
const begin = compositionTicks(item.start / fps + late);
|
||||
const end = compositionTicks((item.start + item.length) / fps);
|
||||
if (end <= begin) return null;
|
||||
const into = Math.max(0, item.offset);
|
||||
const seconds = secondsOfTicks(end - begin, COMPOSITION_TEMPO);
|
||||
const bus = audioItemBus(compId, item.id);
|
||||
const own: BusLevel = { name: bus, gain: dbToGain(audio.volume), pan: 0 };
|
||||
if (audio.kind === 'sound') {
|
||||
const ref = findSound(project, audio.asset);
|
||||
if (!ref?.asset.file) return null;
|
||||
const clip = {
|
||||
tick: begin,
|
||||
length: end - begin,
|
||||
sound: audio.asset,
|
||||
fx: pickedFx('', ref.asset.props),
|
||||
offset: into,
|
||||
volume: 1,
|
||||
bus,
|
||||
};
|
||||
return { notes: [], clips: [clip], buses: [own], routedAs: ref.asset.id };
|
||||
}
|
||||
const withFx = (fxId: string, tempo: number): BusLevel => {
|
||||
const fx = fxVoiceOf(project, fxId, tempo);
|
||||
return fx ? { ...own, fx } : own;
|
||||
};
|
||||
if (audio.kind === 'pattern') {
|
||||
const pattern = findAssetOf(project, audio.asset, 'pattern');
|
||||
if (!pattern || !patternInstrument(pattern.props)) return null;
|
||||
const tempo = patternTempo(pattern.props);
|
||||
const length = patternLength(pattern);
|
||||
const heard = played(patternSequence(pattern), { start: 0, end: length }, tempo, into, seconds);
|
||||
return {
|
||||
notes: heard.notes.map((n) => ({ ...n, tick: begin + n.tick, bus })),
|
||||
clips: [],
|
||||
buses: [withFx(pickedFx('', pattern.props), tempo)],
|
||||
routedAs: pattern.id,
|
||||
};
|
||||
}
|
||||
const song = findAssetOf(project, audio.asset, 'song');
|
||||
if (!song) return null;
|
||||
const tempo = songTempo(song.props);
|
||||
const heard = played(songSequence(project, song), songLoop(song), tempo, into, seconds);
|
||||
// Its tracks are the item's: two items of one song do not share them.
|
||||
const prefix = `song:${song.id}`;
|
||||
const renamed = (name: string) => (name.startsWith(prefix) ? `${bus}${name.slice(prefix.length)}` : name);
|
||||
const tracks = songBuses(project, song).map(({ name, into: to, sends, ...level }) => ({
|
||||
...level,
|
||||
name: renamed(name),
|
||||
...(to !== undefined && { into: renamed(to) }),
|
||||
...(sends && { sends: sends.map((send) => ({ ...send, to: send.to ? renamed(send.to) : bus })) }),
|
||||
}));
|
||||
return {
|
||||
notes: heard.notes.map((n) => ({ ...n, tick: begin + n.tick, ...(n.bus && { bus: renamed(n.bus) }) })),
|
||||
clips: heard.clips.map((c) => ({ ...c, tick: begin + c.tick, bus: renamed(c.bus) })),
|
||||
buses: [withFx(pickedFx('', song.props), tempo), ...tracks],
|
||||
routedAs: song.id,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* What a pattern or a song plays from `into` seconds on for `seconds`, round its loop, in ticks of
|
||||
* the composition from 0: each note and clip with its own tempo, none ringing past the end.
|
||||
*/
|
||||
function played(
|
||||
sequence: Sequence,
|
||||
loop: LoopRange,
|
||||
tempo: number,
|
||||
into: number,
|
||||
seconds: number,
|
||||
): { notes: SequenceNote[]; clips: SequenceClip[] } {
|
||||
const from = Math.round(ticksOfSeconds(into, tempo));
|
||||
const to = from + Math.ceil(ticksOfSeconds(seconds, tempo));
|
||||
const span = compositionTicks(seconds);
|
||||
const at = (ticks: number) => compositionTicks(secondsOfTicks(ticks - from, tempo));
|
||||
const notes = notesIn(sequence, 0, loop, from, to).flatMap((n) => {
|
||||
const tick = at(n.at);
|
||||
const length = Math.min(compositionTicks(secondsOfTicks(n.length, tempo)), span - tick);
|
||||
if (tick >= span || length <= 0) return [];
|
||||
const { at: _, ...note } = n;
|
||||
return [{ ...note, tick, length: Math.max(1, length), tempo }];
|
||||
});
|
||||
const clips = clipsIn(sequence, 0, loop, from, to, true).flatMap((c) => {
|
||||
const tick = at(c.at);
|
||||
const length = Math.min(compositionTicks(secondsOfTicks(c.length, tempo)), span - tick);
|
||||
if (tick >= span || length <= 0) return [];
|
||||
const { at: _, skip, ...clip } = c;
|
||||
return [{ ...clip, tick, length, offset: c.offset + secondsOfTicks(skip, tempo), tempo }];
|
||||
});
|
||||
return { notes, clips };
|
||||
}
|
||||
@@ -4,6 +4,7 @@ import type { BakePlayback } from './bake';
|
||||
import type { MixSend } from './mixer';
|
||||
import { LEAD, type ScheduledClip, type ScheduledNote, Scheduler, type SchedulerHost } from './Scheduler';
|
||||
import type { LoopRange, Sequence } from './sequence';
|
||||
import { ticksOfSeconds } from './time';
|
||||
|
||||
/** What a sequence is made of: loaded before the first of its notes and clips is planned. */
|
||||
export interface SequenceNeeds {
|
||||
@@ -56,6 +57,11 @@ export interface SequencePlay {
|
||||
strikeClip?(clip: ScheduledClip): number;
|
||||
/** Its bake (#340), played in place of its notes and clips where the host can. */
|
||||
bake?: BakePlay;
|
||||
/**
|
||||
* Where it starts, in ticks, as heard `LEAD` from now: a composition's sound (#418). Where it first
|
||||
* waits for its files, it starts as much further in, to keep to the clock it was asked for by.
|
||||
*/
|
||||
from?: number;
|
||||
}
|
||||
|
||||
/** A bake as it is played: its file, the level it is heard at, and which stretches of it. */
|
||||
@@ -158,9 +164,11 @@ export class Playbacks {
|
||||
baked: runner instanceof BakedRun,
|
||||
};
|
||||
this.list.push(playback);
|
||||
const asked = this.host.now();
|
||||
void this.host.prepare(play.needs, id).then(() => {
|
||||
if (!this.list.includes(playback) || playback.fading !== null) return;
|
||||
runner.start(0);
|
||||
const from = play.from;
|
||||
runner.start(from === undefined ? 0 : from + ticksOfSeconds(this.host.now() - asked, play.tempo));
|
||||
// From the first note heard, which is later where the buses are (#346).
|
||||
const at = runner.startedAt ?? this.host.now() + LEAD;
|
||||
this.host.level(id, play.volume ?? 1, at, Math.max(0, play.fadeIn ?? 0));
|
||||
|
||||
@@ -19,6 +19,8 @@ export interface SequenceNote {
|
||||
volume?: number;
|
||||
/** The track of a song it is heard on (#334); straight out where left out. */
|
||||
bus?: string;
|
||||
/** What a Delay synced to beats counts by, where it is not the sequence's: a pattern in a composition (#418). */
|
||||
tempo?: number;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -34,6 +36,8 @@ export interface SequenceClip {
|
||||
offset: number;
|
||||
volume: number;
|
||||
bus: string;
|
||||
/** As a note's. */
|
||||
tempo?: number;
|
||||
}
|
||||
|
||||
export interface Sequence {
|
||||
|
||||
@@ -1420,6 +1420,7 @@ export class Editor implements PlayHost {
|
||||
this.console.refresh();
|
||||
if (this.shows('animation')) this.animationEditor.tick(step);
|
||||
if (this.shows('composition')) this.compositionEditor.tick(dt);
|
||||
else this.compositionEditor.hidden();
|
||||
if (this.shows('tile')) this.tileEditor.tick(step);
|
||||
// The preview is the editor's own, so it runs on while the game is paused.
|
||||
if (this.shows('curve')) this.curveEditor.tick(dt);
|
||||
|
||||
@@ -238,11 +238,11 @@ export function removeSelection(state: EditorState): boolean {
|
||||
else removeTrack(state, s.assetId, s.trackId);
|
||||
return true;
|
||||
case 'composition':
|
||||
// The clips when some are picked, else the track and what lies on it.
|
||||
// The items or clips when some are picked, else the track and what lies on it.
|
||||
if (
|
||||
state.execute(
|
||||
s.clipIds.length > 0
|
||||
? removeFromComposition(state.project, s.assetId, [], s.clipIds)
|
||||
s.clipIds.length > 0 || (s.itemIds ?? []).length > 0
|
||||
? removeFromComposition(state.project, s.assetId, s.itemIds ?? [], s.clipIds)
|
||||
: removeCompositionTrack(state.project, s.assetId, s.trackId),
|
||||
)
|
||||
)
|
||||
|
||||
+25
-6
@@ -60,12 +60,31 @@ export function copyObjectIds(state: EditorState, holderId: string, ids: readonl
|
||||
return put(state, clip, roots, counted(roots.length, clip.items ? 'item' : 'object'));
|
||||
}
|
||||
|
||||
/** Copies animation clips of a composition; false when none of them is there. */
|
||||
export function copyCompositionClips(state: EditorState, assetId: string, ids: readonly string[]): boolean {
|
||||
/**
|
||||
* Copies animation clips or items of a composition: with an item that is an object among them as
|
||||
* objects, the audio items (#418) going along; false when none of them is there.
|
||||
*/
|
||||
export function copyCompositionClips(
|
||||
state: EditorState,
|
||||
assetId: string,
|
||||
ids: readonly string[],
|
||||
itemIds: readonly string[] = [],
|
||||
): boolean {
|
||||
const asset = findAssetOf(state.project, assetId, 'composition');
|
||||
const clips = asset?.clips.filter((c) => ids.includes(c.id)).map((c) => ({ ...c })) ?? [];
|
||||
if (clips.length === 0) return false;
|
||||
return put(state, { kind: 'compositionClips', clips }, [], counted(clips.length, 'clip'));
|
||||
if (!asset) return false;
|
||||
const picked = asset.items.filter((i) => itemIds.includes(i.id));
|
||||
const audio = picked.filter((i) => i.audio).map((i) => structuredClone(i));
|
||||
const objects = picked.filter((i) => !i.audio).map((i) => i.id);
|
||||
if (objects.length > 0) {
|
||||
const clip = copyObjects(state.project, asset, objects);
|
||||
if (!clip) return false;
|
||||
const roots = clip.roots.map((r) => r.id);
|
||||
return put(state, { ...clip, audio }, roots, counted(roots.length + audio.length, 'item'));
|
||||
}
|
||||
const clips = asset.clips.filter((c) => ids.includes(c.id)).map((c) => ({ ...c }));
|
||||
if (clips.length === 0 && audio.length === 0) return false;
|
||||
const noun = audio.length > 0 ? counted(audio.length, 'item') : counted(clips.length, 'clip');
|
||||
return put(state, { kind: 'compositionClips', clips, items: audio }, [], noun);
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -75,7 +94,7 @@ export function copyCompositionClips(state: EditorState, assetId: string, ids: r
|
||||
export function copySelection(state: EditorState): boolean {
|
||||
const s = state.selection;
|
||||
if (s?.kind === 'object') return copyObjectIds(state, s.holderId, s.objectIds);
|
||||
if (s?.kind === 'composition') return copyCompositionClips(state, s.assetId, s.clipIds);
|
||||
if (s?.kind === 'composition') return copyCompositionClips(state, s.assetId, s.clipIds, s.itemIds);
|
||||
if (s?.kind === 'node' || s?.kind === 'annotation') {
|
||||
const graph = findGraphAsset(state.project, s.assetId)?.graph;
|
||||
const nodeIds = s.kind === 'node' ? s.nodeIds : [];
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
import type { Asset, CompositionAsset } from '../../core/model/Asset';
|
||||
import { compositionFps, compositionLength } from '../../core/model/composition';
|
||||
import { VIDEO_WAIT_MS } from '../../core/model/video';
|
||||
import { CompositionSound } from '../../runtime/compositionSound';
|
||||
import { Button, formatNumber, VStack, type Widget } from '../../ui';
|
||||
import { soundLibrary } from '../../viewport/SoundLibrary';
|
||||
import { AssetTabEditor } from '../AssetTabEditor';
|
||||
import { claimTransport, releaseTransport } from '../music/transport';
|
||||
import type { EditorState } from '../state';
|
||||
import { CompositionCanvas } from './CompositionCanvas';
|
||||
import { CompositionTimeline } from './CompositionTimeline';
|
||||
@@ -21,6 +24,12 @@ export class CompositionEditor extends AssetTabEditor<CompositionAsset, Composit
|
||||
readonly playButton: Button;
|
||||
readonly preview: CompositionCanvas;
|
||||
|
||||
/** Its audio items (#418), heard while the playhead runs, never while it is scrubbed. Public for the check scripts. */
|
||||
readonly sound = new CompositionSound(soundLibrary);
|
||||
/** Set by an edit: what the composition plays may have changed under the sound. */
|
||||
private edited = false;
|
||||
private readonly halt = () => this.canvas.pause();
|
||||
|
||||
constructor(state: EditorState) {
|
||||
super(state, { infoWidth: INFO_WIDTH });
|
||||
this.addTrackButton = new Button('+ Track', { onClick: () => this.canvas.addTrack() });
|
||||
@@ -31,7 +40,15 @@ export class CompositionEditor extends AssetTabEditor<CompositionAsset, Composit
|
||||
this.preview.redraw();
|
||||
this.followLiveItems();
|
||||
};
|
||||
timeline.onPlaying = () => this.syncPlayButton();
|
||||
timeline.onPlaying = () => {
|
||||
this.syncPlayButton();
|
||||
if (timeline.isPlaying) claimTransport(this.halt);
|
||||
else {
|
||||
releaseTransport(this.halt);
|
||||
this.sound.stop();
|
||||
}
|
||||
};
|
||||
state.on('project', () => (this.edited = true));
|
||||
this.mount(timeline, [this.addTrackButton, this.playButton]);
|
||||
}
|
||||
|
||||
@@ -88,10 +105,33 @@ export class CompositionEditor extends AssetTabEditor<CompositionAsset, Composit
|
||||
/** Only called while the tab is visible. */
|
||||
tick(dt: number): void {
|
||||
const drawn = this.preview.redrawCount;
|
||||
if (this.canvas.isPlaying && !this.waitsForVideo()) this.canvas.tick(dt);
|
||||
const runs = this.canvas.isPlaying && !this.waitsForVideo();
|
||||
if (runs) this.canvas.tick(dt);
|
||||
this.hear(runs);
|
||||
this.preview.animate(drawn);
|
||||
}
|
||||
|
||||
/** Called instead of `tick` while the tab is not seen: its playhead stands, and so does its sound. */
|
||||
hidden(): void {
|
||||
this.sound.stop();
|
||||
}
|
||||
|
||||
/**
|
||||
* The sound follows the playhead while it runs, straight into the master as every tab plays; an
|
||||
* edit that changes what it plays starts it again there.
|
||||
*/
|
||||
private hear(runs: boolean): void {
|
||||
const asset = this.canvas.isPlaying ? this.asset() : null;
|
||||
if (asset && runs && this.edited && this.sound.changed(this.state.project, asset, false))
|
||||
this.sound.stop();
|
||||
this.edited = false;
|
||||
if (!asset || !runs) {
|
||||
this.sound.stop();
|
||||
return;
|
||||
}
|
||||
this.sound.start(this.state.project, asset, this.canvas.playhead / compositionFps(asset), false);
|
||||
}
|
||||
|
||||
/** Until when the playhead waits for a video it shows, see `waitsForVideo`. */
|
||||
private videoWait: number | null = null;
|
||||
|
||||
|
||||
@@ -16,16 +16,24 @@ import {
|
||||
import { addPictureItem, wornPicture } from '../../core/commands/compositionSprites';
|
||||
import { newId } from '../../core/ids';
|
||||
import { isPlaceableType } from '../../core/model/AssetTypes';
|
||||
import type { CompositionAsset, CompositionClip, CompositionItem } from '../../core/model/Asset';
|
||||
import type {
|
||||
CompositionAsset,
|
||||
CompositionAudioKind,
|
||||
CompositionClip,
|
||||
CompositionItem,
|
||||
} from '../../core/model/Asset';
|
||||
import {
|
||||
clipLength,
|
||||
compositionFps,
|
||||
compositionLength,
|
||||
compositionLoops,
|
||||
fittedLength,
|
||||
itemObject,
|
||||
itemStands,
|
||||
itemVideo,
|
||||
} from '../../core/model/composition';
|
||||
import {
|
||||
audioKindOf,
|
||||
clampedMove,
|
||||
clipSpans,
|
||||
droppedLength,
|
||||
@@ -38,6 +46,7 @@ import {
|
||||
movedItems,
|
||||
movedWithClips,
|
||||
pastedClips,
|
||||
pastedItemsWithClips,
|
||||
resizedItem,
|
||||
snappedToMedia,
|
||||
trimmedItem,
|
||||
@@ -46,6 +55,9 @@ import { placementRejection } from '../../core/model/placement';
|
||||
import type { Vec2 } from '../../core/model/Properties';
|
||||
import { findAsset, findAssetOf } from '../../core/model/Project';
|
||||
import { objectName } from '../../core/model/SceneObject';
|
||||
import { findSound } from '../../core/model/sound';
|
||||
import { assetRefLabel } from '../../core/properties/assetRef';
|
||||
import { playedSoundById } from '../../core/properties/playedSound';
|
||||
import {
|
||||
activeDrag,
|
||||
crispText,
|
||||
@@ -62,6 +74,7 @@ import {
|
||||
zoomFactor,
|
||||
} from '../../ui';
|
||||
import { fitToMedia, makeProjectAssetEntry } from '../inspector/compositionSection';
|
||||
import { soundLibrary } from '../../viewport/SoundLibrary';
|
||||
import { clampTimePan, tickToX, xToTick, zoomTime } from '../music/timeView';
|
||||
import {
|
||||
type Box,
|
||||
@@ -69,6 +82,7 @@ import {
|
||||
EDGE_REACH,
|
||||
fitSong,
|
||||
nearestTrack,
|
||||
peakOver,
|
||||
trackAtY,
|
||||
trackTop,
|
||||
zoomTracks,
|
||||
@@ -100,6 +114,12 @@ const ITEM_COLOR = 0x35598f;
|
||||
const VIDEO_COLOR = 0x2f6f52;
|
||||
const CLIP_COLOR = 0x8a5a2b;
|
||||
const REJECTED_COLOR = 0x6b3a3a;
|
||||
/** Audio items (#418), by what they play. */
|
||||
const AUDIO_COLORS: Record<CompositionAudioKind, number> = {
|
||||
sound: 0x2f6a78,
|
||||
pattern: 0x5b4a8f,
|
||||
song: 0x7a3f6a,
|
||||
};
|
||||
const DEFAULT_VIEW: CompositionView = { panX: 0, pxPerTick: 2, panY: 0, trackHeight: TRACK_HEIGHT };
|
||||
/** Room at the top of an item for its name; marks of its video keep below it. */
|
||||
const LABEL_STRIP = 15;
|
||||
@@ -182,6 +202,8 @@ export class CompositionTimeline extends Widget {
|
||||
private hover: { id: string; part: Part } | null = null;
|
||||
/** Where the pointer was last, in local coordinates: a paste lands there while it is over the lanes. */
|
||||
private pointer: Vec2 | null = null;
|
||||
/** Sound files whose waveform is on its way. */
|
||||
private readonly loading = new Set<string>();
|
||||
private needsFit = true;
|
||||
/** Where the playhead stands, in frames – fractional while playing. */
|
||||
playhead = 0;
|
||||
@@ -338,10 +360,38 @@ export class CompositionTimeline extends Widget {
|
||||
const picked = this.pickedTrack(asset);
|
||||
const row = spot?.row ?? (picked >= 0 ? picked : undefined);
|
||||
const at = spot?.frame ?? Math.floor(this.playhead + 1e-6);
|
||||
if (clip.kind === 'compositionClips' && (clip.items ?? []).length > 0)
|
||||
return this.pasteAudio(asset, clip.items!, clip.clips, at, row);
|
||||
if (clip.kind === 'compositionClips') return this.pasteClips(asset, clip.clips, at, row);
|
||||
const pasted = pasteCompositionItems(this.state.project, asset.id, clip, at, row);
|
||||
if (!pasted || !this.state.execute(pasted.command)) return false;
|
||||
this.state.select(objectSelection(asset.id, pasted.itemIds));
|
||||
this.pickItems(pasted.itemIds);
|
||||
return true;
|
||||
}
|
||||
|
||||
/** Copied audio items (#418), and clips copied with them, put in as items are. */
|
||||
private pasteAudio(
|
||||
asset: CompositionAsset,
|
||||
copied: readonly CompositionItem[],
|
||||
copiedClips: readonly CompositionClip[],
|
||||
at: number,
|
||||
row: number | undefined,
|
||||
): boolean {
|
||||
const ids = new Map([...copied, ...copiedClips].map((c) => [c.id, newId()]));
|
||||
const { items, clips } = pastedItemsWithClips(
|
||||
copied,
|
||||
copiedClips,
|
||||
asset.clips,
|
||||
ids,
|
||||
at,
|
||||
asset.tracks.map((t) => t.id),
|
||||
this.lengthOf(asset),
|
||||
row,
|
||||
);
|
||||
const label = items.length === 1 ? 'Paste item' : 'Paste items';
|
||||
const parts = { items: [...asset.items, ...items], clips: [...asset.clips, ...clips] };
|
||||
if (!this.state.execute(new SetCompositionCommand(asset.id, parts, label))) return false;
|
||||
this.pickItems(items.map((i) => i.id));
|
||||
return true;
|
||||
}
|
||||
|
||||
@@ -463,18 +513,39 @@ export class CompositionTimeline extends Widget {
|
||||
return (clip) => Math.max(1, clipLength(this.state.project, asset, clip));
|
||||
}
|
||||
|
||||
/** The items of the composition there is an object for: one whose object went is not on the tracks. */
|
||||
/** The items of the composition on its tracks: one whose object went is not. */
|
||||
private liveItems(asset: CompositionAsset): CompositionItem[] {
|
||||
return asset.items.filter((i) => itemObject(asset, i));
|
||||
return asset.items.filter((i) => itemStands(asset, i));
|
||||
}
|
||||
|
||||
private pickedItems(asset: CompositionAsset): string[] {
|
||||
const s = this.state.selection;
|
||||
if (s?.kind === 'composition' && s.assetId === asset.id) return s.itemIds ?? [];
|
||||
return s?.kind === 'object' && s.holderId === asset.id
|
||||
? s.objectIds.filter((id) => asset.items.some((i) => i.id === id))
|
||||
: [];
|
||||
}
|
||||
|
||||
/**
|
||||
* Picks items: as objects while they all are – the hierarchy and the inspector show them –, on the
|
||||
* composition once an audio item is among them (#418); the composition itself for none.
|
||||
*/
|
||||
private pickItems(ids: readonly string[]): void {
|
||||
const asset = this.asset();
|
||||
if (!asset) return;
|
||||
const items = ids.flatMap((id) => asset.items.find((i) => i.id === id) ?? []);
|
||||
if (items.length === 0) this.state.select({ kind: 'asset', assetId: asset.id });
|
||||
else if (!items.some((i) => i.audio)) this.state.select(objectSelection(asset.id, [...ids]));
|
||||
else
|
||||
this.state.select({
|
||||
kind: 'composition',
|
||||
assetId: asset.id,
|
||||
trackId: items[0]!.track,
|
||||
clipIds: [],
|
||||
itemIds: items.map((i) => i.id),
|
||||
});
|
||||
}
|
||||
|
||||
private pickedClips(asset: CompositionAsset): string[] {
|
||||
const s = this.state.selection;
|
||||
return s?.kind === 'composition' && s.assetId === asset.id ? s.clipIds : [];
|
||||
@@ -551,11 +622,10 @@ export class CompositionTimeline extends Widget {
|
||||
const item = asset.items.find((i) => i.id === hit.id)!;
|
||||
const picked = this.pickedItems(asset);
|
||||
if (toggle) {
|
||||
this.state.toggleObject(asset.id, item.id);
|
||||
this.pickItems(picked.includes(item.id) ? picked.filter((id) => id !== item.id) : [...picked, item.id]);
|
||||
return;
|
||||
}
|
||||
if (!picked.includes(item.id) || hit.part !== 'body')
|
||||
this.state.select(objectSelection(asset.id, [item.id]));
|
||||
if (!picked.includes(item.id) || hit.part !== 'body') this.pickItems([item.id]);
|
||||
const media = itemMedia(this.state.project, asset, item);
|
||||
if (hit.part === 'end') this.gesture = { kind: 'resize', anchor: item, item, media };
|
||||
else if (hit.part === 'start') this.gesture = { kind: 'trim', anchor: item, item, media };
|
||||
@@ -653,7 +723,9 @@ export class CompositionTimeline extends Widget {
|
||||
gesture.anchor,
|
||||
this.snapped(frame, gesture.media),
|
||||
compositionFps(asset),
|
||||
gesture.media !== null || itemVideo(this.state.project, asset, gesture.anchor) !== null,
|
||||
gesture.media !== null ||
|
||||
gesture.anchor.audio !== undefined ||
|
||||
itemVideo(this.state.project, asset, gesture.anchor) !== null,
|
||||
);
|
||||
break;
|
||||
}
|
||||
@@ -743,7 +815,7 @@ export class CompositionTimeline extends Widget {
|
||||
break;
|
||||
case 'move': {
|
||||
if (!gesture.moved) {
|
||||
if (gesture.what === 'item') this.state.select(objectSelection(asset.id, [gesture.anchor.id]));
|
||||
if (gesture.what === 'item') this.pickItems([gesture.anchor.id]);
|
||||
break;
|
||||
}
|
||||
const parts = this.movedParts(asset, gesture, gesture.frames, gesture.tracks);
|
||||
@@ -788,9 +860,7 @@ export class CompositionTimeline extends Widget {
|
||||
const caught = band.active
|
||||
? itemsInBand(this.view, asset.tracks, this.liveItems(asset), band.start, band.current)
|
||||
: [];
|
||||
const ids = [...new Set([...(band.add ? this.pickedItems(asset) : []), ...caught])];
|
||||
if (ids.length > 0) this.state.select(objectSelection(asset.id, ids));
|
||||
else this.state.select({ kind: 'asset', assetId: asset.id });
|
||||
this.pickItems([...new Set([...(band.add ? this.pickedItems(asset) : []), ...caught])]);
|
||||
}
|
||||
|
||||
private commit(
|
||||
@@ -804,17 +874,21 @@ export class CompositionTimeline extends Widget {
|
||||
|
||||
/**
|
||||
* Follows an asset dragged in from the panel: a placeable one, a picture or a video (#419) lands as
|
||||
* an item under the pointer, as long as its media or 3 s, over whatever lies there; an animation as
|
||||
* a clip one pass long, where it fits whole.
|
||||
* an item under the pointer, as long as its media or 3 s, over whatever lies there; a sound, a clip
|
||||
* of one, a pattern or a song as an audio item as long as one pass of it (#418); an animation as a
|
||||
* clip one pass long, where it fits whole.
|
||||
*/
|
||||
private followDrop(asset: CompositionAsset, assetId: string, local: Vec2, g: Vec2): void {
|
||||
const dragged = findAsset(this.state.project, assetId);
|
||||
const project = this.state.project;
|
||||
const dragged = findAsset(project, assetId);
|
||||
const audio = audioKindOf(project, assetId);
|
||||
const row = trackAtY(this.view, asset.tracks.length, g.y);
|
||||
const inside = local.x >= HEAD_WIDTH && local.x <= this.w && local.y >= RULER_HEIGHT && local.y <= this.h;
|
||||
const picture = dragged ? wornPicture(dragged) : null;
|
||||
const usable =
|
||||
dragged && (dragged.type === 'animation' || isPlaceableType(dragged.type) || picture !== null);
|
||||
if (!inside || row === null || !dragged || !usable) {
|
||||
audio !== null ||
|
||||
(dragged && (dragged.type === 'animation' || isPlaceableType(dragged.type) || picture !== null));
|
||||
if (!inside || row === null || !usable) {
|
||||
if (this.drop) {
|
||||
this.drop = null;
|
||||
this.redraw();
|
||||
@@ -823,7 +897,19 @@ export class CompositionTimeline extends Widget {
|
||||
}
|
||||
const track = asset.tracks[row]!.id;
|
||||
const at = Math.max(0, Math.round(xToTick(this.view, g.x)));
|
||||
if (dragged.type === 'animation') {
|
||||
if (audio) {
|
||||
const file = audio === 'sound' ? playedSoundById(project, assetId) : '';
|
||||
if (file) this.ensureLoaded(file);
|
||||
const item = { id: '', track, start: at, length: 1, offset: 0, audio: audioOf(audio, assetId) };
|
||||
this.drop = {
|
||||
what: 'item',
|
||||
assetId,
|
||||
track,
|
||||
start: at,
|
||||
length: fittedLength(project, asset, item),
|
||||
rejection: '',
|
||||
};
|
||||
} else if (dragged!.type === 'animation') {
|
||||
const length = clipLength(this.state.project, asset, {
|
||||
id: '',
|
||||
track,
|
||||
@@ -852,8 +938,8 @@ export class CompositionTimeline extends Widget {
|
||||
assetId,
|
||||
track,
|
||||
start: at,
|
||||
length: droppedLength(this.state.project, asset, dragged),
|
||||
rejection: picture === null ? placementRejection(this.state.project, asset, dragged) : '',
|
||||
length: droppedLength(this.state.project, asset, dragged!),
|
||||
rejection: picture === null ? placementRejection(this.state.project, asset, dragged!) : '',
|
||||
};
|
||||
this.redraw();
|
||||
}
|
||||
@@ -894,6 +980,13 @@ export class CompositionTimeline extends Widget {
|
||||
return;
|
||||
}
|
||||
const item = { track: drop.track, start: drop.start, length: drop.length, offset: 0, fitMedia: true };
|
||||
const audio = audioKindOf(this.state.project, drop.assetId);
|
||||
if (audio) {
|
||||
const added = { ...item, id: newId(), audio: audioOf(audio, drop.assetId) };
|
||||
const command = new SetCompositionCommand(asset.id, { items: [...asset.items, added] }, 'Add item');
|
||||
if (this.state.execute(command)) this.pickItems([added.id]);
|
||||
return;
|
||||
}
|
||||
const dragged = findAsset(this.state.project, drop.assetId);
|
||||
const picture = dragged ? wornPicture(dragged) : null;
|
||||
const { command, objectId } =
|
||||
@@ -927,15 +1020,30 @@ export class CompositionTimeline extends Widget {
|
||||
const row = local.y >= RULER_HEIGHT ? trackAtY(this.view, asset.tracks.length, g.y) : null;
|
||||
const hit = local.x >= HEAD_WIDTH && local.y >= RULER_HEIGHT ? this.hitAt(asset, g) : null;
|
||||
if (hit?.kind === 'item') {
|
||||
if (!this.pickedItems(asset).includes(hit.id)) this.state.select(objectSelection(asset.id, [hit.id]));
|
||||
if (!this.pickedItems(asset).includes(hit.id)) this.pickItems([hit.id]);
|
||||
const ids = this.pickedItems(asset);
|
||||
const audio = asset.items.find((i) => i.id === hit.id)?.audio;
|
||||
const played =
|
||||
audio &&
|
||||
(audio.kind === 'sound'
|
||||
? findSound(this.state.project, audio.asset)?.asset
|
||||
: findAsset(this.state.project, audio.asset));
|
||||
new Menu([
|
||||
...(audio
|
||||
? [
|
||||
{
|
||||
label: `Open ${audio.kind}`,
|
||||
disabled: !played,
|
||||
onSelect: () => played && this.state.openAsset(played.id),
|
||||
},
|
||||
]
|
||||
: []),
|
||||
{
|
||||
label: 'Fit to media',
|
||||
disabled: ids.length !== 1,
|
||||
onSelect: () => fitToMedia({ state: this.state }, asset.id, hit.id),
|
||||
},
|
||||
makeProjectAssetEntry({ state: this.state }, asset.id, hit.id),
|
||||
...(audio ? [] : [makeProjectAssetEntry({ state: this.state }, asset.id, hit.id)]),
|
||||
{ separator: true },
|
||||
{
|
||||
label: ids.length > 1 ? `Remove ${ids.length} items` : 'Remove item',
|
||||
@@ -1087,7 +1195,7 @@ export class CompositionTimeline extends Widget {
|
||||
const hint = !asset
|
||||
? ''
|
||||
: asset.items.length === 0 && asset.clips.length === 0
|
||||
? 'Drag a sprite, text, camera or container onto a track, an animation to play on it.'
|
||||
? 'Drag a sprite, text, camera or container onto a track, an animation to play on it, a sound, pattern or song to hear.'
|
||||
: '';
|
||||
this.hint.text = hint;
|
||||
this.hint.visible = hint !== '';
|
||||
@@ -1098,7 +1206,12 @@ export class CompositionTimeline extends Widget {
|
||||
const c = Theme.colors;
|
||||
const s = this.state.selection;
|
||||
const lit =
|
||||
s?.kind === 'composition' && s.assetId === asset.id && s.clipIds.length === 0 ? s.trackId : '';
|
||||
s?.kind === 'composition' &&
|
||||
s.assetId === asset.id &&
|
||||
s.clipIds.length === 0 &&
|
||||
(s.itemIds ?? []).length === 0
|
||||
? s.trackId
|
||||
: '';
|
||||
const end = HEAD_WIDTH + tickToX(this.view, compositionLength(this.state.project, asset));
|
||||
const zero = HEAD_WIDTH + tickToX(this.view, 0);
|
||||
asset.tracks.forEach((track, row) => {
|
||||
@@ -1137,7 +1250,13 @@ export class CompositionTimeline extends Widget {
|
||||
if (box.x > this.w || box.x + box.w < HEAD_WIDTH) continue;
|
||||
const lit = picked.has(item.id);
|
||||
const media = itemMedia(this.state.project, asset, item);
|
||||
const color = itemVideo(this.state.project, asset, item) ? VIDEO_COLOR : ITEM_COLOR;
|
||||
const color = item.audio
|
||||
? this.audioName(item) === null
|
||||
? REJECTED_COLOR
|
||||
: AUDIO_COLORS[item.audio.kind]
|
||||
: itemVideo(this.state.project, asset, item)
|
||||
? VIDEO_COLOR
|
||||
: ITEM_COLOR;
|
||||
this.lanes
|
||||
.roundRect(box.x, box.y, box.w, box.h, 3)
|
||||
.fill({ color, alpha: lit ? 1 : 0.85 })
|
||||
@@ -1147,10 +1266,15 @@ export class CompositionTimeline extends Widget {
|
||||
alpha: lit ? 1 : 0.45,
|
||||
alignment: 1,
|
||||
});
|
||||
if (item.audio?.kind === 'sound') this.drawWave(item, box);
|
||||
if (media) this.drawMediaMarks(item, media, box);
|
||||
const object = itemObject(asset, item);
|
||||
const placed = object && findAsset(this.state.project, object.assetId);
|
||||
const name = object ? objectName(object, placed) : '';
|
||||
const name = item.audio
|
||||
? (this.audioName(item) ?? '(missing)')
|
||||
: object
|
||||
? objectName(object, placed)
|
||||
: '';
|
||||
const seen = HEAD_WIDTH + tickToX(this.view, uncoveredEnd(shown, item)) - box.x;
|
||||
if (seen > 18)
|
||||
labels.push({ text: name, x: Math.max(box.x, HEAD_WIDTH) + 4, y: box.y + 2, w: seen - 8 });
|
||||
@@ -1213,6 +1337,62 @@ export class CompositionTimeline extends Widget {
|
||||
}
|
||||
}
|
||||
|
||||
/** What an audio item plays, by name – "Sound / Clip" for a clip –; null once it is gone. */
|
||||
private audioName(item: CompositionItem): string | null {
|
||||
const audio = item.audio!;
|
||||
const project = this.state.project;
|
||||
const there =
|
||||
audio.kind === 'sound'
|
||||
? findSound(project, audio.asset) !== null
|
||||
: findAssetOf(project, audio.asset, audio.kind) !== undefined;
|
||||
return there ? assetRefLabel(project, audio.asset) : null;
|
||||
}
|
||||
|
||||
/**
|
||||
* The waveform of a sound item, round and round as it plays, from where it starts in it, below its
|
||||
* name – the Song tab's.
|
||||
*/
|
||||
private drawWave(item: CompositionItem, box: Box): void {
|
||||
const asset = this.asset();
|
||||
const played = asset && playedSoundById(this.state.project, item.audio!.asset);
|
||||
if (!asset || !played) return;
|
||||
const peaks = soundLibrary.peaks(played);
|
||||
const seconds = soundLibrary.duration(played);
|
||||
if (!peaks || !seconds) {
|
||||
this.ensureLoaded(played);
|
||||
return;
|
||||
}
|
||||
const fps = compositionFps(asset);
|
||||
const width = tickToX(this.view, seconds * fps) - tickToX(this.view, 0);
|
||||
if (!(width > 0)) return;
|
||||
const mid = box.y + LABEL_STRIP + (box.h - LABEL_STRIP) / 2;
|
||||
const half = (box.h - LABEL_STRIP - 4) / 2;
|
||||
const from = Math.max(box.x, HEAD_WIDTH);
|
||||
const to = Math.min(box.x + box.w, this.w);
|
||||
const begins = HEAD_WIDTH + tickToX(this.view, item.start - item.offset * fps);
|
||||
for (let round = Math.floor((from - begins) / width); ; round++) {
|
||||
const start = Math.round(begins + round * width);
|
||||
if (start >= to) break;
|
||||
for (let dx = Math.max(0, Math.ceil((from - start) / 2) * 2); dx < width; dx += 2) {
|
||||
const x = start + dx;
|
||||
if (x >= to) break;
|
||||
// Before a sound that begins inside the item, it is silent.
|
||||
if (x < begins) continue;
|
||||
const reach = Math.max(0.5, peakOver(peaks, dx / width, (dx + 2) / width) * half);
|
||||
this.lanes.rect(x, mid - reach, 1.5, reach * 2).fill({ color: 0xffffff, alpha: 0.6 });
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private ensureLoaded(file: string): void {
|
||||
if (this.loading.has(file) || soundLibrary.peaks(file) !== null) return;
|
||||
this.loading.add(file);
|
||||
void soundLibrary.loadPicture(file).then(() => {
|
||||
this.loading.delete(file);
|
||||
this.redraw();
|
||||
});
|
||||
}
|
||||
|
||||
/** The end of an item the pointer would drag, lit – over an item that covers it, too. */
|
||||
private drawGrip(box: Box, part: 'start' | 'end'): void {
|
||||
const reach = Math.min(EDGE_REACH, box.w / 3);
|
||||
@@ -1307,6 +1487,11 @@ export class CompositionTimeline extends Widget {
|
||||
}
|
||||
}
|
||||
|
||||
/** A new audio item's, as its asset says: unity, on the bus its routing sends it to. */
|
||||
function audioOf(kind: CompositionAudioKind, asset: string): NonNullable<CompositionItem['audio']> {
|
||||
return { kind, asset, volume: 0, bus: '' };
|
||||
}
|
||||
|
||||
/** Puts the texts where they are wanted, reusing the ones there are and hiding the rest. */
|
||||
function syncTexts(pool: Text[], layer: Container, wanted: readonly Placed[], style: TextStyle): void {
|
||||
wanted.forEach((place, i) => {
|
||||
|
||||
@@ -96,6 +96,11 @@ export function documentHooks({ state, editor }: HookContext) {
|
||||
objectScreen: (objectId: string) => editor.compositionEditor.preview.objectScreen(objectId),
|
||||
/** Where the picture draws a sprite right now; null while it is left out, a video without its picture. */
|
||||
drawn: (objectId: string) => editor.compositionEditor.preview.drawnRect(objectId),
|
||||
/** Whether its audio items are heard (#418), and the numbers of their playings. */
|
||||
sound: () => ({
|
||||
playing: editor.compositionEditor.sound.playing,
|
||||
playings: [...editor.compositionEditor.sound.playings],
|
||||
}),
|
||||
},
|
||||
tile: {
|
||||
current: () => editor.tileEditor.currentAssetId,
|
||||
|
||||
@@ -146,7 +146,8 @@ export class Inspector extends Panel {
|
||||
else if (s.kind === 'track') buildTrackSection(this.host, s.assetId, s.trackId, s.keys);
|
||||
else if (s.kind === 'edge') buildEdgeSection(this.host, s.assetId, s.edgeId, s.edgeIds);
|
||||
else if (s.kind === 'song') buildSongSection(this.host, s.assetId, s.trackId, s.itemIds);
|
||||
else if (s.kind === 'composition') buildCompositionSection(this.host, s.assetId, s.trackId, s.clipIds);
|
||||
else if (s.kind === 'composition')
|
||||
buildCompositionSection(this.host, s.assetId, s.trackId, s.clipIds, s.itemIds);
|
||||
else if (s.objectIds.length > 1) buildObjectsSection(this.host, s.holderId, s.objectIds);
|
||||
else buildObjectSection(this.host, s.holderId, s.objectId);
|
||||
this.scroll.setContent(this.host.body);
|
||||
@@ -167,7 +168,7 @@ export class Inspector extends Panel {
|
||||
if (s.kind === 'edge') return edgeSignature(this.state.project, s.assetId, s.edgeId, s.edgeIds);
|
||||
if (s.kind === 'song') return songSignature(this.state.project, s.assetId, s.trackId, s.itemIds);
|
||||
if (s.kind === 'composition')
|
||||
return compositionSignature(this.state.project, s.assetId, s.trackId, s.clipIds);
|
||||
return compositionSignature(this.state.project, s.assetId, s.trackId, s.clipIds, s.itemIds);
|
||||
return objectSignature(this.state, s.holderId, s.objectIds, s.objectId);
|
||||
}
|
||||
|
||||
@@ -208,7 +209,7 @@ export class Inspector extends Panel {
|
||||
: s.kind === 'song'
|
||||
? refreshSongSection(this.host, s.assetId, s.trackId, s.itemIds)
|
||||
: s.kind === 'composition'
|
||||
? refreshCompositionSection(this.host, s.assetId, s.trackId, s.clipIds)
|
||||
? refreshCompositionSection(this.host, s.assetId, s.trackId, s.clipIds, s.itemIds)
|
||||
: s.objectIds.length > 1 ||
|
||||
refreshObjectSection(this.host, s.holderId, s.objectId);
|
||||
if (!ok) this.rebuild();
|
||||
|
||||
@@ -1,6 +1,12 @@
|
||||
import { MakeProjectAssetCommand } from '../../core/commands/assetCommands';
|
||||
import { SetCompositionCommand } from '../../core/commands/compositionCommands';
|
||||
import type { CompositionAsset, CompositionItem } from '../../core/model/Asset';
|
||||
import type {
|
||||
Asset,
|
||||
CompositionAsset,
|
||||
CompositionAudio,
|
||||
CompositionAudioKind,
|
||||
CompositionItem,
|
||||
} from '../../core/model/Asset';
|
||||
import {
|
||||
clipLength,
|
||||
compositionFps,
|
||||
@@ -10,6 +16,8 @@ import {
|
||||
} from '../../core/model/composition';
|
||||
import { heldMove, movedWithClips, resizedItem } from '../../core/model/compositionEdit';
|
||||
import { findAsset, findAssetOf, type Project } from '../../core/model/Project';
|
||||
import { findSound } from '../../core/model/sound';
|
||||
import { ITEM_MIN_DB, SOUND_ITEM_MAX_DB } from '../../core/music/song';
|
||||
import type { PropertyDef } from '../../core/properties/PropertySchema';
|
||||
import { Button, formatNumber, type MenuEntry } from '../../ui';
|
||||
import { createFieldRow } from './fieldFactory';
|
||||
@@ -21,6 +29,36 @@ const LENGTH_ROW = 'composition:length';
|
||||
const OFFSET_ROW = 'composition:offset';
|
||||
const CLIP_START_ROW = 'composition:clipStart';
|
||||
const ANIMATION_ROW = 'composition:animation';
|
||||
const PLAYS_ROW = 'composition:plays';
|
||||
const VOLUME_ROW = 'composition:volume';
|
||||
const BUS_ROW = 'composition:bus';
|
||||
|
||||
const AUDIO_TITLES: Record<CompositionAudioKind, string> = {
|
||||
sound: 'SOUND ITEM',
|
||||
pattern: 'PATTERN ITEM',
|
||||
song: 'SONG ITEM',
|
||||
};
|
||||
|
||||
const VOLUME_DEF: PropertyDef = {
|
||||
key: VOLUME_ROW,
|
||||
label: 'Volume',
|
||||
kind: 'number',
|
||||
default: 0,
|
||||
min: ITEM_MIN_DB,
|
||||
max: SOUND_ITEM_MAX_DB,
|
||||
step: 0.5,
|
||||
decimals: 1,
|
||||
};
|
||||
|
||||
const BUS_DEF: PropertyDef = {
|
||||
key: BUS_ROW,
|
||||
label: 'Bus',
|
||||
kind: 'asset',
|
||||
assetTypes: ['bus'],
|
||||
none: 'From asset',
|
||||
master: true,
|
||||
default: '',
|
||||
};
|
||||
|
||||
const seconds = (key: string, label: string, negative = false): PropertyDef => ({
|
||||
key,
|
||||
@@ -32,15 +70,21 @@ const seconds = (key: string, label: string, negative = false): PropertyDef => (
|
||||
decimals: 3,
|
||||
});
|
||||
|
||||
/** Structure-defining facts of a track or clip selection: which it is, and whether it is still there. */
|
||||
/** Structure-defining facts of a track, clip or item selection: which it is, and whether it is still there. */
|
||||
export function compositionSignature(
|
||||
project: Project,
|
||||
assetId: string,
|
||||
trackId: string,
|
||||
clipIds: string[],
|
||||
itemIds: string[] = [],
|
||||
): string {
|
||||
const comp = findAssetOf(project, assetId, 'composition');
|
||||
if (!comp?.tracks.some((t) => t.id === trackId)) return 'missing';
|
||||
if (itemIds.length === 1) {
|
||||
const audio = comp.items.find((i) => i.id === itemIds[0])?.audio;
|
||||
return `item:${itemIds[0]}:${audio ? playedAsset(project, audio)?.id : ''}${timingSignature(project, assetId, itemIds[0]!)}`;
|
||||
}
|
||||
if (itemIds.length > 0) return `items:${itemIds.join(',')}`;
|
||||
return clipIds.length > 0 ? `clips:${clipIds.join(',')}` : 'track';
|
||||
}
|
||||
|
||||
@@ -50,10 +94,17 @@ export function buildCompositionSection(
|
||||
assetId: string,
|
||||
trackId: string,
|
||||
clipIds: string[],
|
||||
itemIds: string[] = [],
|
||||
): void {
|
||||
const comp = findAssetOf(host.state.project, assetId, 'composition');
|
||||
const track = comp?.tracks.find((t) => t.id === trackId);
|
||||
if (!comp || !track) return;
|
||||
if (itemIds.length > 0) {
|
||||
const item = itemIds.length === 1 ? comp.items.find((i) => i.id === itemIds[0]) : undefined;
|
||||
if (item?.audio) buildAudioItem(host, assetId, item, item.audio);
|
||||
else addTitle(host, `${itemIds.length} ITEMS`);
|
||||
return;
|
||||
}
|
||||
if (clipIds.length === 0) {
|
||||
addTitle(host, 'TRACK');
|
||||
addNameField(host, track.name, {
|
||||
@@ -99,10 +150,20 @@ export function refreshCompositionSection(
|
||||
assetId: string,
|
||||
trackId: string,
|
||||
clipIds: string[],
|
||||
itemIds: string[] = [],
|
||||
): boolean {
|
||||
const comp = findAssetOf(host.state.project, assetId, 'composition');
|
||||
const track = comp?.tracks.find((t) => t.id === trackId);
|
||||
if (!comp || !track) return false;
|
||||
if (itemIds.length > 0) {
|
||||
const audio = itemIds.length === 1 ? comp.items.find((i) => i.id === itemIds[0])?.audio : undefined;
|
||||
if (audio) {
|
||||
refreshTimingRows(host, assetId, itemIds[0]!);
|
||||
host.fields.get(VOLUME_ROW)?.setValue(audio.volume);
|
||||
host.fields.get(BUS_ROW)?.setValue(audio.bus);
|
||||
}
|
||||
return true;
|
||||
}
|
||||
if (clipIds.length === 0) host.nameField?.setValue(track.name);
|
||||
const clip = clipIds.length === 1 ? comp.clips.find((c) => c.id === clipIds[0]) : undefined;
|
||||
if (clip) host.fields.get(CLIP_START_ROW)?.setValue(clip.start / compositionFps(comp));
|
||||
@@ -129,7 +190,7 @@ export function addTimingRows(host: InspectorHost, holderId: string, objectId: s
|
||||
if (!found) return;
|
||||
const { comp, item } = found;
|
||||
const fps = compositionFps(comp);
|
||||
const video = itemVideo(host.state.project, comp, item) !== null;
|
||||
const video = item.audio !== undefined || itemVideo(host.state.project, comp, item) !== null;
|
||||
addTitle(host, 'TIMING');
|
||||
const write = (label: string, edit: (current: CompositionItem) => CompositionItem) => {
|
||||
const current = compositionItemOf(host.state.project, holderId, objectId);
|
||||
@@ -190,9 +251,11 @@ export function addTimingRows(host: InspectorHost, holderId: string, objectId: s
|
||||
addNote(
|
||||
host,
|
||||
item.fitMedia
|
||||
? video
|
||||
? 'Its length follows its video.'
|
||||
: 'Its length follows a video it is given, 3 s while it has none.'
|
||||
? item.audio
|
||||
? 'Its length follows what it plays.'
|
||||
: video
|
||||
? 'Its length follows its video.'
|
||||
: 'Its length follows a video it is given, 3 s while it has none.'
|
||||
: 'Its length is its own.',
|
||||
'small',
|
||||
);
|
||||
@@ -214,7 +277,7 @@ export function refreshTimingRows(host: InspectorHost, holderId: string, objectI
|
||||
export function timingSignature(project: Project, holderId: string, objectId: string): string {
|
||||
const found = compositionItemOf(project, holderId, objectId);
|
||||
if (!found) return '';
|
||||
const video = itemVideo(project, found.comp, found.item) !== null;
|
||||
const video = found.item.audio !== undefined || itemVideo(project, found.comp, found.item) !== null;
|
||||
const fitted = found.item.length === fittedLength(project, found.comp, found.item);
|
||||
return `|timing:${video}:${found.item.fitMedia === true}:${fitted}`;
|
||||
}
|
||||
@@ -248,6 +311,52 @@ export function makeProjectAssetEntry(
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* An audio item (#418): what it plays, when, how loud and on which bus. It plays round and round for
|
||||
* as long as it lasts; its offset is how far into what it plays it begins.
|
||||
*/
|
||||
function buildAudioItem(
|
||||
host: InspectorHost,
|
||||
assetId: string,
|
||||
item: CompositionItem,
|
||||
audio: CompositionAudio,
|
||||
): void {
|
||||
addTitle(host, AUDIO_TITLES[audio.kind]);
|
||||
const played = playedAsset(host.state.project, audio);
|
||||
if (played) assetLink(host, PLAYS_ROW, 'Plays', played);
|
||||
else addNote(host, `Its ${audio.kind} is gone.`, 'warning');
|
||||
addTimingRows(host, assetId, item.id);
|
||||
const edit = (label: string, change: Partial<CompositionAudio>) => {
|
||||
const comp = findAssetOf(host.state.project, assetId, 'composition');
|
||||
if (!comp) return;
|
||||
const items = comp.items.map((i) =>
|
||||
i.id === item.id && i.audio ? { ...i, audio: { ...i.audio, ...change } } : i,
|
||||
);
|
||||
if (SetCompositionCommand.changes(host.state.project, assetId, { items }))
|
||||
host.state.execute(new SetCompositionCommand(assetId, { items }, label));
|
||||
};
|
||||
const volume = createFieldRow(host, VOLUME_DEF, audio.volume, {
|
||||
onInput: () => undefined,
|
||||
onChange: (value) => edit('Set item volume', { volume: Number(value) || 0 }),
|
||||
});
|
||||
host.fields.set(VOLUME_ROW, volume);
|
||||
host.body.add(volume.row);
|
||||
const bus = createFieldRow(host, BUS_DEF, audio.bus, {
|
||||
onInput: () => undefined,
|
||||
onChange: (value) => edit('Set item bus', { bus: typeof value === 'string' ? value : '' }),
|
||||
});
|
||||
host.fields.set(BUS_ROW, bus);
|
||||
host.body.add(bus.row);
|
||||
addNote(host, 'Volume in dB. It plays round and round for as long as it lasts.', 'small');
|
||||
}
|
||||
|
||||
/** The asset an audio item plays, for a clip of a sound the sound; undefined once it is gone. */
|
||||
function playedAsset(project: Project, audio: CompositionAudio): Asset | undefined {
|
||||
return audio.kind === 'sound'
|
||||
? findSound(project, audio.asset)?.asset
|
||||
: findAssetOf(project, audio.asset, audio.kind);
|
||||
}
|
||||
|
||||
function withOffset(item: CompositionItem, offset: number): CompositionItem {
|
||||
const { fitMedia: _, ...rest } = item;
|
||||
return { ...rest, offset: Number.isFinite(offset) ? offset : 0 };
|
||||
|
||||
@@ -78,6 +78,11 @@ export class AssetPanel extends Panel {
|
||||
private readonly foldedCategories = new Set<AssetCategory>();
|
||||
/** Ids of the rows as shown, headings included. */
|
||||
private rows: string[] = [];
|
||||
/**
|
||||
* The selected asset the list last scrolled to. Only a new selection scrolls: a rebuild – folding
|
||||
* a group, any edit – leaves the list where the user put it.
|
||||
*/
|
||||
private scrolledTo: string | null = null;
|
||||
private readonly stopListening: () => void;
|
||||
|
||||
constructor(
|
||||
@@ -112,6 +117,8 @@ export class AssetPanel extends Panel {
|
||||
};
|
||||
// Room to scroll the last group up off the panel's bottom edge, a few rows' worth.
|
||||
this.scroll.tailSpace = 6 * this.list.rowHeight;
|
||||
// Folding a group leaves the rows above it, and the heading under the pointer, where they are.
|
||||
this.scroll.holdOffset = true;
|
||||
scrollTween(this.scroll);
|
||||
this.scroll.setContent(this.list);
|
||||
this.content.add(this.scroll, 1);
|
||||
@@ -121,6 +128,7 @@ export class AssetPanel extends Panel {
|
||||
state.on('load', () => {
|
||||
this.unfolded.clear();
|
||||
this.foldedCategories.clear();
|
||||
this.scrolledTo = null;
|
||||
this.refresh();
|
||||
});
|
||||
state.on('selection', () => this.syncSelection());
|
||||
@@ -274,7 +282,8 @@ export class AssetPanel extends Panel {
|
||||
return;
|
||||
}
|
||||
this.list.select(id, true);
|
||||
if (id) this.scrollTo(id);
|
||||
if (id && id !== this.scrolledTo) this.scrollTo(id);
|
||||
this.scrolledTo = id;
|
||||
}
|
||||
|
||||
private scrollTo(id: string): void {
|
||||
@@ -445,12 +454,16 @@ export class AssetPanel extends Panel {
|
||||
|
||||
/**
|
||||
* Hands the asset to the view under the pointer: a scene places it, a song lays out a pattern or a
|
||||
* sound (#334), a composition an animation as a clip (#417) and a picture or video as an item
|
||||
* (#419). A scene has nowhere to go.
|
||||
* sound (#334), a composition an animation as a clip (#417), a picture or video as an item (#419)
|
||||
* and a sound, pattern or song as an audio item (#418). A scene has nowhere to go.
|
||||
*/
|
||||
private startPlacementDrag(assetId: string): void {
|
||||
const asset = findAsset(this.state.project, assetId);
|
||||
const laid = asset?.type === 'pattern' || asset?.type === 'sound' || asset?.type === 'animation';
|
||||
const laid =
|
||||
asset?.type === 'pattern' ||
|
||||
asset?.type === 'song' ||
|
||||
asset?.type === 'sound' ||
|
||||
asset?.type === 'animation';
|
||||
if (!asset || !(isPlaceableType(asset.type) || laid || wornPicture(asset) !== null)) return;
|
||||
startDrag({ kind: 'asset', id: assetId }, asset.name);
|
||||
}
|
||||
|
||||
+14
-5
@@ -83,9 +83,10 @@ export type Selection =
|
||||
| { kind: 'song'; assetId: string; trackId: string; itemIds: string[] }
|
||||
/**
|
||||
* A track of a composition (#417) and the animation clips picked on it or on others: `clipIds` is
|
||||
* empty when the track itself was clicked. Its items are objects, picked as an object selection.
|
||||
* empty when the track itself was clicked. Items that are objects only are picked as an object
|
||||
* selection; `itemIds` holds picked items once an audio item is among them (#418).
|
||||
*/
|
||||
| { kind: 'composition'; assetId: string; trackId: string; clipIds: string[] }
|
||||
| { kind: 'composition'; assetId: string; trackId: string; clipIds: string[]; itemIds?: string[] }
|
||||
| null;
|
||||
|
||||
/**
|
||||
@@ -857,11 +858,14 @@ export class EditorState {
|
||||
const asset = findAssetOf(this.project, s.assetId, 'composition');
|
||||
if (!asset) this.select(null);
|
||||
else {
|
||||
// The same as a song's: a clip that died leaves, a track that died hands over to the composition.
|
||||
// The same as a song's: a clip or item that died leaves, a track that died hands over to the composition.
|
||||
const clips = s.clipIds.filter((id) => asset.clips.some((clip) => clip.id === id));
|
||||
const picked = s.itemIds ?? [];
|
||||
const items = picked.filter((id) => asset.items.some((item) => item.id === id));
|
||||
const trackAlive = asset.tracks.some((track) => track.id === s.trackId);
|
||||
if (!trackAlive) this.select({ kind: 'asset', assetId: s.assetId });
|
||||
else if (clips.length !== s.clipIds.length) this.select({ ...s, clipIds: clips });
|
||||
else if (clips.length !== s.clipIds.length || items.length !== picked.length)
|
||||
this.select({ ...s, clipIds: clips, itemIds: items });
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1089,7 +1093,12 @@ export function sameSelection(a: Selection, b: Selection): boolean {
|
||||
if (a.kind === 'song' && b.kind === 'song')
|
||||
return a.assetId === b.assetId && a.trackId === b.trackId && sameIds(a.itemIds, b.itemIds);
|
||||
if (a.kind === 'composition' && b.kind === 'composition')
|
||||
return a.assetId === b.assetId && a.trackId === b.trackId && sameIds(a.clipIds, b.clipIds);
|
||||
return (
|
||||
a.assetId === b.assetId &&
|
||||
a.trackId === b.trackId &&
|
||||
sameIds(a.clipIds, b.clipIds) &&
|
||||
sameIds(a.itemIds ?? [], b.itemIds ?? [])
|
||||
);
|
||||
return false;
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,92 @@
|
||||
import type { CompositionAsset } from '../core/model/Asset';
|
||||
import type { Project } from '../core/model/Project';
|
||||
import {
|
||||
COMPOSITION_TEMPO,
|
||||
compositionPlayings,
|
||||
compositionSpan,
|
||||
compositionTicks,
|
||||
} from '../core/music/compositionAudio';
|
||||
import { sequenceNeeds } from '../core/music/needs';
|
||||
import { LEAD } from '../core/music/Scheduler';
|
||||
import type { AudioSink } from './systems/Audio';
|
||||
import { playClip, strikeNote } from './systems/behaviorContext';
|
||||
|
||||
/** How long the sound of a composition takes to fall silent when it stops. */
|
||||
const HUSH = 0.02;
|
||||
|
||||
/**
|
||||
* The sound of one composition as it plays (#418): its audio items as playings of the sink, started
|
||||
* from a time of the composition and stopped as a whole. Whoever plays the composition starts it
|
||||
* while its clock runs and stops it while the clock stands, so the two keep together.
|
||||
*/
|
||||
export class CompositionSound {
|
||||
private ids: number[] = [];
|
||||
private started = false;
|
||||
/** What it plays, to tell an edit that changes it from one that does not. */
|
||||
private heard = '';
|
||||
|
||||
constructor(
|
||||
private readonly audio: AudioSink,
|
||||
private readonly owner = '',
|
||||
) {}
|
||||
|
||||
get playing(): boolean {
|
||||
return this.started;
|
||||
}
|
||||
|
||||
/** The numbers of its playings (test hook). */
|
||||
get playings(): readonly number[] {
|
||||
return this.ids;
|
||||
}
|
||||
|
||||
/**
|
||||
* Plays the composition from `seconds` into it, as heard `LEAD` from now – unless it plays already.
|
||||
* `routed` off sends it all to the master, as the tabs preview a piece.
|
||||
*/
|
||||
start(project: Project, comp: CompositionAsset, seconds: number, routed = true): void {
|
||||
if (this.started) return;
|
||||
this.started = true;
|
||||
const playings = compositionPlayings(project, comp, routed);
|
||||
this.heard = keyOf(playings);
|
||||
const { length, loop } = compositionSpan(project, comp);
|
||||
const sources = { project, audio: this.audio };
|
||||
for (const { route, sequence, buses } of playings) {
|
||||
const id = this.audio.playSequence(
|
||||
`composition:${comp.id}`,
|
||||
{
|
||||
sequence,
|
||||
tempo: COMPOSITION_TEMPO,
|
||||
length,
|
||||
loop,
|
||||
needs: sequenceNeeds(project, sequence),
|
||||
buses,
|
||||
...(typeof route === 'string' ? { bus: route } : { sends: route }),
|
||||
from: compositionTicks(seconds + LEAD),
|
||||
strike: (note) => strikeNote(sources, note, this.owner),
|
||||
strikeClip: (clip) => playClip(sources, clip),
|
||||
},
|
||||
this.owner,
|
||||
);
|
||||
if (id !== 0) this.ids.push(id);
|
||||
}
|
||||
}
|
||||
|
||||
/** Whether the composition plays something else now than when it was started. */
|
||||
changed(project: Project, comp: CompositionAsset, routed = true): boolean {
|
||||
return this.started && keyOf(compositionPlayings(project, comp, routed)) !== this.heard;
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
for (const id of this.ids) this.audio.stopPlayback(id, HUSH);
|
||||
this.ids = [];
|
||||
this.started = false;
|
||||
this.heard = '';
|
||||
}
|
||||
}
|
||||
|
||||
/** The playings as text, a sound FX by its plan's presence: a plan is no plain data. */
|
||||
function keyOf(playings: ReturnType<typeof compositionPlayings>): string {
|
||||
return JSON.stringify(playings, (key, value: unknown) =>
|
||||
key === 'fx' && typeof value === 'object' ? 1 : value,
|
||||
);
|
||||
}
|
||||
@@ -148,7 +148,7 @@ export function addGameSystems(
|
||||
}: GameSystemOptions = {},
|
||||
): BehaviorSystem {
|
||||
const collisions = physics ? new PhysicsSystem(physics) : new CollisionSystem();
|
||||
const compositions = new CompositionSystem(playsCompositions, pictures);
|
||||
const compositions = new CompositionSystem(playsCompositions, pictures, audio);
|
||||
const animations = new AnimationSystem();
|
||||
const sources = { collisions, animations, compositions, audio, clock, saves, recorder, breakpoint };
|
||||
const behaviors = new BehaviorSystem(project, input, sources, shouldRun);
|
||||
|
||||
@@ -15,9 +15,11 @@ import { findAssetOf } from '../../core/model/Project';
|
||||
import type { PropertyValue } from '../../core/model/Properties';
|
||||
import { findVideo, VIDEO_WAIT_MS } from '../../core/model/video';
|
||||
import { VIDEO_CLOCK_KEY, VIDEO_TIME_KEY, videoDrivenOf } from '../../core/properties/videos';
|
||||
import { CompositionSound } from '../compositionSound';
|
||||
import type { Entity } from '../Entity';
|
||||
import type { World } from '../World';
|
||||
import { bind } from './AnimationSystem';
|
||||
import { type AudioSink, SILENT_AUDIO } from './Audio';
|
||||
import type { System } from './System';
|
||||
|
||||
export interface CompositionFinish {
|
||||
@@ -40,6 +42,8 @@ interface Run {
|
||||
live: Map<string, Spawned>;
|
||||
/** Seconds the clock has stood so far for a video without its picture, see `waits`. */
|
||||
waited: number;
|
||||
/** Its audio items (#418), heard while its clock runs. */
|
||||
sound: CompositionSound;
|
||||
}
|
||||
|
||||
/** What the view that shows a run knows of its driven videos (#404). */
|
||||
@@ -57,26 +61,39 @@ export interface VideoPictures {
|
||||
*
|
||||
* While the view still waits for the picture of a video an item brings in, the composition's
|
||||
* clock stands, so the video starts at its first frame and not where the clock has got to meanwhile.
|
||||
* Its audio items are heard only while the clock runs: they stop while it stands – for a video, a
|
||||
* placement switched off, a paused game – and start again where it stands.
|
||||
*/
|
||||
export class CompositionSystem implements System {
|
||||
/** A paused frame stops the sound of every composition; nothing else moves. */
|
||||
readonly runsWhilePaused = true;
|
||||
finished: CompositionFinish[] = [];
|
||||
private readonly runs = new Map<Entity, Run>();
|
||||
|
||||
constructor(
|
||||
private readonly plays = true,
|
||||
private readonly pictures: VideoPictures | null = null,
|
||||
private readonly audio: AudioSink = SILENT_AUDIO,
|
||||
) {}
|
||||
|
||||
update(world: World, dt: number): void {
|
||||
this.finished = [];
|
||||
for (const placement of this.runs.keys())
|
||||
if (!world.entities.includes(placement)) this.runs.delete(placement);
|
||||
for (const [placement, run] of this.runs)
|
||||
if (!world.entities.includes(placement)) {
|
||||
run.sound.stop();
|
||||
this.runs.delete(placement);
|
||||
}
|
||||
if (world.frozen) {
|
||||
for (const run of this.runs.values()) run.sound.stop();
|
||||
return;
|
||||
}
|
||||
const started = world.entities.filter((e) => e.type === 'composition' && e.composition);
|
||||
for (const placement of started) this.advance(world, placement, dt);
|
||||
}
|
||||
|
||||
stop(): void {
|
||||
this.finished = [];
|
||||
for (const run of this.runs.values()) run.sound.stop();
|
||||
this.runs.clear();
|
||||
}
|
||||
|
||||
@@ -96,8 +113,13 @@ export class CompositionSystem implements System {
|
||||
return;
|
||||
}
|
||||
run = this.runs.get(placement);
|
||||
if (!run) this.runs.set(placement, (run = { id: player.run, live: new Map(), waited: 0 }));
|
||||
if (player.shown && isHandleEnabled(placement) && !this.waits(run, dt)) player.time += dt;
|
||||
if (!run)
|
||||
this.runs.set(
|
||||
placement,
|
||||
(run = { id: player.run, live: new Map(), waited: 0, sound: new CompositionSound(this.audio) }),
|
||||
);
|
||||
const runs = player.shown && isHandleEnabled(placement) && !this.waits(run, dt);
|
||||
if (runs) player.time += dt;
|
||||
player.shown = true;
|
||||
const length = compositionLength(world.project, comp);
|
||||
let frame = compositionFrameAt(comp, player.time);
|
||||
@@ -111,6 +133,8 @@ export class CompositionSystem implements System {
|
||||
return;
|
||||
}
|
||||
this.show(world, comp, placement, run, frame);
|
||||
if (runs) run.sound.start(world.project, comp, player.time);
|
||||
else run.sound.stop();
|
||||
}
|
||||
|
||||
/** Whether a video of a live item still has no picture; at most `VIDEO_WAIT_MS` at a stretch. */
|
||||
@@ -131,6 +155,7 @@ export class CompositionSystem implements System {
|
||||
private end(world: World, placement: Entity): void {
|
||||
const run = this.runs.get(placement);
|
||||
if (!run) return;
|
||||
run.sound.stop();
|
||||
for (const { entity } of run.live.values()) if (world.entities.includes(entity)) world.remove(entity);
|
||||
this.runs.delete(placement);
|
||||
}
|
||||
|
||||
@@ -24,6 +24,14 @@ export class ScrollContainer extends Widget {
|
||||
* is made this much taller, so a list lays its empty area (clicks, context menu) over it as well.
|
||||
*/
|
||||
tailSpace = 0;
|
||||
/**
|
||||
* Content that shrinks keeps its offset: the rows above stay where they were, and the room the
|
||||
* gone ones leave stays below until the view is scrolled up off it. Without it the offset is
|
||||
* clamped to the shorter content, and every row slides down by what went away. Opt-in.
|
||||
*/
|
||||
holdOffset = false;
|
||||
/** The height the content asks for, tail included – without the room held for gone rows. */
|
||||
private needed = 0;
|
||||
/**
|
||||
* Carries out a wheel step, by default at once. A panel can hand it to a transition instead,
|
||||
* which makes the step and then shows the way to where it led.
|
||||
@@ -84,7 +92,9 @@ export class ScrollContainer extends Widget {
|
||||
const clamped = clamp(y, 0, this.maxScroll);
|
||||
if (clamped === this.scrollY) return;
|
||||
this.scrollY = clamped;
|
||||
this.applyScroll();
|
||||
// Room held for gone rows is let go as the view leaves it.
|
||||
if (this.holdOffset && this.contentHeight > Math.max(this.needed, this.h)) this.relayoutContent();
|
||||
else this.applyScroll();
|
||||
}
|
||||
|
||||
/** Scrolls just far enough for the content band [top, top + height) to be inside the view. */
|
||||
@@ -96,11 +106,13 @@ export class ScrollContainer extends Widget {
|
||||
/** Re-measure the content after it changed (rows added/removed). */
|
||||
relayoutContent(): void {
|
||||
if (!this.content) return;
|
||||
const held = this.holdOffset ? this.scrollY + this.h : 0;
|
||||
// Wrapped text grows taller when the scrollbar takes its width, so measure again at that width.
|
||||
let needed = this.content.heightFor(this.w) + this.tailSpace;
|
||||
const width = Math.max(0, this.w - (needed > this.h ? Theme.scrollbarWidth : 0));
|
||||
const width = Math.max(0, this.w - (Math.max(needed, held) > this.h ? Theme.scrollbarWidth : 0));
|
||||
if (width !== this.w) needed = this.content.heightFor(width) + this.tailSpace;
|
||||
const height = Math.max(needed, this.h);
|
||||
this.needed = needed;
|
||||
const height = Math.max(needed, held, this.h);
|
||||
// Content shorter than the viewport is pinned to the viewport height either way, so `resize`
|
||||
// would see no change and skip the layout -- leaving rows added since stacked at the origin.
|
||||
const unchanged = this.content.w === width && this.content.h === height;
|
||||
|
||||
@@ -63,6 +63,7 @@ import {
|
||||
import { type Levels, type Spectrum, SpectrumTaps } from './spectrum';
|
||||
import { decodeSpan, StreamFeed, StreamFile } from './streaming';
|
||||
import { wake } from './wake';
|
||||
import { setSoundMeasure } from '../core/model/composition';
|
||||
|
||||
/** A voice is no louder than a sound item of a song turned all the way up. */
|
||||
const MAX_VOLUME = dbToGain(SOUND_ITEM_MAX_DB);
|
||||
@@ -2964,3 +2965,6 @@ function readMeter(meter: BusMeter): [number, number] {
|
||||
}
|
||||
|
||||
export const soundLibrary = new SoundLibrary();
|
||||
|
||||
/** A composition fits an audio item to the length of its sound once the bytes are here (#418). */
|
||||
setSoundMeasure((file) => soundLibrary.duration(file));
|
||||
@@ -0,0 +1,360 @@
|
||||
/**
|
||||
* Sounds, patterns and songs as items of a composition (#418): kept without an object, laid out as
|
||||
* sequences in milliseconds on buses of their own, and heard in a game while the composition's
|
||||
* clock runs.
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
CreateAssetCommand,
|
||||
CreateSoundCommand,
|
||||
SetAssetPropertyCommand,
|
||||
} from '../../src/core/commands/assetCommands';
|
||||
import { SetCompositionCommand } from '../../src/core/commands/compositionCommands';
|
||||
import { SetLoopCommand, SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
|
||||
import { PlaceAssetCommand } from '../../src/core/commands/sceneObjectCommands';
|
||||
import { SetSongCommand } from '../../src/core/commands/songCommands';
|
||||
import type {
|
||||
CompositionAsset,
|
||||
CompositionItem,
|
||||
InstrumentAsset,
|
||||
SongAsset,
|
||||
} from '../../src/core/model/Asset';
|
||||
import { compositionLength, fittedLength, setSoundMeasure } from '../../src/core/model/composition';
|
||||
import { carriedClips, itemMedia, mediaMarks } from '../../src/core/model/compositionEdit';
|
||||
import { findAssetOf } from '../../src/core/model/Project';
|
||||
import { audioItemBus, compositionPlayings, compositionSpan } from '../../src/core/music/compositionAudio';
|
||||
import { MASTER_MIX, mixBusName } from '../../src/core/music/mixer';
|
||||
import { Playbacks, type SequencePlay } from '../../src/core/music/playbacks';
|
||||
import { LEAD } from '../../src/core/music/Scheduler';
|
||||
import { newItem } from '../../src/core/music/song';
|
||||
import { MASTER_BUS } from '../../src/core/properties/buses';
|
||||
import { SILENT_AUDIO } from '../../src/runtime/systems/Audio';
|
||||
import { addGameSystems } from '../../src/runtime/systems/BehaviorSystem';
|
||||
import { Input } from '../../src/runtime/systems/Input';
|
||||
import { World } from '../../src/runtime/World';
|
||||
import { graphEditor, makeBehavior, makeProject } from '../helpers';
|
||||
|
||||
const FILE = `${'e5'.repeat(32)}.wav`;
|
||||
const Q = 480;
|
||||
|
||||
/**
|
||||
* A composition "Cut" at 10 fps placed in a scene, a sound "Blip", a one-bar pattern "Riff" at
|
||||
* 120 BPM (two seconds) with notes on beats 1 and 3, and a song "Theme" playing it from its start.
|
||||
*/
|
||||
function makeCut() {
|
||||
const f = makeProject();
|
||||
const scene = new CreateAssetCommand('scene', 'Level');
|
||||
const comp = new CreateAssetCommand('composition', 'Cut', { fps: 10 });
|
||||
const sound = new CreateSoundCommand('Blip', FILE);
|
||||
const ins = new CreateAssetCommand('instrument', 'Lead');
|
||||
const pattern = new CreateAssetCommand('pattern', 'Riff');
|
||||
const song = new CreateAssetCommand('song', 'Theme');
|
||||
const bus = new CreateAssetCommand('bus', 'Music');
|
||||
for (const c of [scene, comp, sound, ins, pattern, song, bus]) f.stack.execute(c);
|
||||
(ins.asset as InstrumentAsset).graph.nodes.find((n) => n.type === 'InstrumentSample')!.params.sound =
|
||||
sound.asset.id;
|
||||
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'instrument', ins.asset.id));
|
||||
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'bars', 1));
|
||||
f.stack.execute(
|
||||
new SetPatternNotesCommand(pattern.asset.id, [
|
||||
{ id: 'n1', tick: 0, length: Q, note: 60, velocity: 1 },
|
||||
{ id: 'n2', tick: 2 * Q, length: Q, note: 64, velocity: 1 },
|
||||
]),
|
||||
);
|
||||
const theme = () => findAssetOf(f.project, song.asset.id, 'song') as SongAsset;
|
||||
const track = theme().tracks[0]!;
|
||||
f.stack.execute(
|
||||
new SetSongCommand(song.asset.id, {
|
||||
items: [newItem({ track: track.id, kind: 'pattern', asset: pattern.asset.id, tick: 0, length: 4 * Q })],
|
||||
}),
|
||||
);
|
||||
const place = new PlaceAssetCommand(scene.asset.id, comp.asset.id, { x: 0, y: 0 });
|
||||
f.stack.execute(place);
|
||||
const cut = () => findAssetOf(f.project, comp.asset.id, 'composition') as CompositionAsset;
|
||||
const lay = (...items: CompositionItem[]) =>
|
||||
f.stack.execute(new SetCompositionCommand(cut().id, { items }));
|
||||
const audioItem = (
|
||||
id: string,
|
||||
kind: 'sound' | 'pattern' | 'song',
|
||||
start: number,
|
||||
length: number,
|
||||
more: Partial<CompositionItem> & { volume?: number; bus?: string } = {},
|
||||
): CompositionItem => ({
|
||||
id,
|
||||
track: cut().tracks[0]!.id,
|
||||
start,
|
||||
length,
|
||||
offset: more.offset ?? 0,
|
||||
audio: {
|
||||
kind,
|
||||
asset: { sound: sound.asset.id, pattern: pattern.asset.id, song: song.asset.id }[kind],
|
||||
volume: more.volume ?? 0,
|
||||
bus: more.bus ?? '',
|
||||
},
|
||||
});
|
||||
return {
|
||||
...f,
|
||||
sceneId: scene.asset.id,
|
||||
placement: place.objectId,
|
||||
soundId: sound.asset.id,
|
||||
patternId: pattern.asset.id,
|
||||
songId: song.asset.id,
|
||||
busId: bus.asset.id,
|
||||
cut,
|
||||
lay,
|
||||
audioItem,
|
||||
};
|
||||
}
|
||||
|
||||
describe('audio items of a composition', () => {
|
||||
it('are kept without an object, and count to its length', () => {
|
||||
const f = makeCut();
|
||||
f.lay(f.audioItem('a', 'sound', 5, 20, { volume: 40 }), f.audioItem('b', 'pattern', 0, 10));
|
||||
expect(f.cut().items.map((i) => i.id)).toEqual(['a', 'b']);
|
||||
// Held to the most a sound item may be turned up by.
|
||||
expect(f.cut().items[0]!.audio).toEqual({ kind: 'sound', asset: f.soundId, volume: 12, bus: '' });
|
||||
expect(f.cut().objects).toEqual([]);
|
||||
expect(compositionLength(f.project, f.cut())).toBe(25);
|
||||
});
|
||||
|
||||
it('drops one that plays nothing or an unknown kind', () => {
|
||||
const f = makeCut();
|
||||
const bad = {
|
||||
...f.audioItem('a', 'sound', 0, 10),
|
||||
audio: { kind: 'video', asset: 'x', volume: 0, bus: '' },
|
||||
};
|
||||
const empty = {
|
||||
...f.audioItem('b', 'sound', 0, 10),
|
||||
audio: { kind: 'sound', asset: '', volume: 0, bus: '' },
|
||||
};
|
||||
f.lay(bad as CompositionItem, empty as CompositionItem);
|
||||
expect(f.cut().items).toEqual([]);
|
||||
});
|
||||
|
||||
it('take the length of a pattern or song, of a sound once it is measured', () => {
|
||||
const f = makeCut();
|
||||
f.lay(f.audioItem('p', 'pattern', 0, 1, { offset: 0.5 }), f.audioItem('s', 'sound', 0, 1));
|
||||
const [pattern, sound] = f.cut().items;
|
||||
expect(fittedLength(f.project, f.cut(), pattern!)).toBe(15);
|
||||
expect(fittedLength(f.project, f.cut(), sound!)).toBe(30);
|
||||
setSoundMeasure((file) => (file === FILE ? 1.25 : null));
|
||||
try {
|
||||
expect(fittedLength(f.project, f.cut(), sound!)).toBe(13);
|
||||
expect(itemMedia(f.project, f.cut(), sound!)).toEqual({ begins: 0, length: 12.5, loops: true });
|
||||
} finally {
|
||||
setSoundMeasure(() => null);
|
||||
}
|
||||
});
|
||||
|
||||
it('mark a song to the end of its loop, then every time round it', () => {
|
||||
const f = makeCut();
|
||||
f.stack.execute(new SetLoopCommand('song', f.songId, { start: Q, end: 3 * Q }));
|
||||
f.lay(f.audioItem('s', 'song', 0, 40));
|
||||
const song = f.cut().items[0]!;
|
||||
expect(fittedLength(f.project, f.cut(), song)).toBe(15);
|
||||
const media = itemMedia(f.project, f.cut(), song)!;
|
||||
expect(media).toEqual({ begins: 0, length: 15, loops: true, round: 10 });
|
||||
expect(mediaMarks(song, media).seams).toEqual([15, 25, 35]);
|
||||
});
|
||||
|
||||
it('own no clips: a clip on one belongs to the object item it starts on', () => {
|
||||
const f = makeCut();
|
||||
const sound = f.audioItem('a', 'sound', 0, 20);
|
||||
const object: CompositionItem = { id: 'o', track: sound.track, start: 0, length: 20, offset: 0 };
|
||||
const clip = { id: 'c', track: sound.track, animation: 'x', start: 5 };
|
||||
expect([...carriedClips([object, sound], [clip], new Set(['a']))]).toEqual([]);
|
||||
expect([...carriedClips([object, sound], [clip], new Set(['o']))]).toEqual(['c']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the sequences of a composition', () => {
|
||||
it('lay a sound in milliseconds on a bus of its own at its volume, late by an offset below 0', () => {
|
||||
const f = makeCut();
|
||||
f.lay(
|
||||
f.audioItem('a', 'sound', 5, 20, { volume: -6, offset: 0.25 }),
|
||||
f.audioItem('b', 'sound', 30, 10, { offset: -0.3 }),
|
||||
);
|
||||
const [playing] = compositionPlayings(f.project, f.cut());
|
||||
expect(playing!.route).toBe(MASTER_MIX);
|
||||
expect(playing!.sequence.clips).toEqual([
|
||||
{
|
||||
tick: 500,
|
||||
length: 2000,
|
||||
sound: f.soundId,
|
||||
fx: '',
|
||||
offset: 0.25,
|
||||
volume: 1,
|
||||
bus: audioItemBus(f.cut().id, 'a'),
|
||||
},
|
||||
{
|
||||
tick: 3300,
|
||||
length: 700,
|
||||
sound: f.soundId,
|
||||
fx: '',
|
||||
offset: 0,
|
||||
volume: 1,
|
||||
bus: audioItemBus(f.cut().id, 'b'),
|
||||
},
|
||||
]);
|
||||
expect(playing!.buses.map((b) => [b.name, Number(b.gain.toFixed(3))])).toEqual([
|
||||
[audioItemBus(f.cut().id, 'a'), 0.501],
|
||||
[audioItemBus(f.cut().id, 'b'), 1],
|
||||
]);
|
||||
});
|
||||
|
||||
it('play a pattern round and round at its own tempo, cut where the item ends', () => {
|
||||
const f = makeCut();
|
||||
// 3.5 s from half a second in: beat 3 (1 s), then round two (1.5 s, 2.5 s), then the next (3.5 s) is cut.
|
||||
f.lay(f.audioItem('p', 'pattern', 10, 35, { offset: 0.5 }));
|
||||
const notes = compositionPlayings(f.project, f.cut())[0]!.sequence.notes;
|
||||
expect(notes.map((n) => [n.tick, n.length, n.note, n.tempo])).toEqual([
|
||||
[1500, 500, 64, 120],
|
||||
[2500, 500, 60, 120],
|
||||
[3500, 500, 64, 120],
|
||||
]);
|
||||
});
|
||||
|
||||
it('give a song item its own copy of the song’s buses, sending into the item’s', () => {
|
||||
const f = makeCut();
|
||||
f.lay(f.audioItem('s', 'song', 0, 20), f.audioItem('t', 'song', 0, 20));
|
||||
const [playing] = compositionPlayings(f.project, f.cut());
|
||||
const bus = audioItemBus(f.cut().id, 's');
|
||||
const track =
|
||||
f.cut().tracks.length && (findAssetOf(f.project, f.songId, 'song') as SongAsset).tracks[0]!.id;
|
||||
const names = playing!.buses.map((b) => b.name);
|
||||
expect(names).toContain(`${bus}:${track}`);
|
||||
expect(names).toContain(`${audioItemBus(f.cut().id, 't')}:${track}`);
|
||||
expect(playing!.buses.find((b) => b.name === `${bus}:${track}`)!.sends).toEqual([
|
||||
expect.objectContaining({ to: bus }),
|
||||
]);
|
||||
const own = playing!.sequence.notes.filter((n) => n.bus?.startsWith(`${bus}:`));
|
||||
expect(own.map((n) => [n.tick, n.note])).toEqual([
|
||||
[0, 60],
|
||||
[1000, 64],
|
||||
]);
|
||||
});
|
||||
|
||||
it('go one to each place of the mix their items are sent to, or all to the master unrouted', () => {
|
||||
const f = makeCut();
|
||||
f.lay(
|
||||
f.audioItem('a', 'sound', 0, 10),
|
||||
f.audioItem('b', 'sound', 0, 10, { bus: f.busId }),
|
||||
f.audioItem('c', 'pattern', 0, 10, { bus: MASTER_BUS }),
|
||||
);
|
||||
const routes = compositionPlayings(f.project, f.cut()).map((p) => [p.route, p.buses.length]);
|
||||
expect(routes).toEqual([
|
||||
[MASTER_MIX, 2],
|
||||
[mixBusName(f.busId), 1],
|
||||
]);
|
||||
expect(compositionPlayings(f.project, f.cut(), false).map((p) => p.route)).toEqual([MASTER_MIX]);
|
||||
});
|
||||
|
||||
it('run as long as the composition, round it where it loops', () => {
|
||||
const f = makeCut();
|
||||
f.lay(f.audioItem('a', 'sound', 5, 20));
|
||||
expect(compositionSpan(f.project, f.cut())).toEqual({ length: 2500, loop: null });
|
||||
f.stack.execute(new SetAssetPropertyCommand(f.cut().id, 'loop', true));
|
||||
expect(compositionSpan(f.project, f.cut())).toEqual({ length: 2500, loop: { start: 0, end: 2500 } });
|
||||
});
|
||||
});
|
||||
|
||||
describe('a playing started from a tick (#418)', () => {
|
||||
it('starts as much further in as it waited for its files', async () => {
|
||||
let time = 0;
|
||||
let loaded = () => undefined as void;
|
||||
const started: number[] = [];
|
||||
const playbacks = new Playbacks({
|
||||
now: () => time,
|
||||
latency: () => 0,
|
||||
playing: () => false,
|
||||
stop: () => undefined,
|
||||
wake: () => () => undefined,
|
||||
prepare: () => new Promise<void>((resolve) => (loaded = resolve)),
|
||||
level: () => undefined,
|
||||
after: () => undefined,
|
||||
});
|
||||
const play: SequencePlay = {
|
||||
sequence: { notes: [{ tick: 0, length: 1, note: 60, velocity: 1, instrument: 'x' }] },
|
||||
tempo: 125,
|
||||
length: 10000,
|
||||
loop: null,
|
||||
needs: { layers: [], files: [] },
|
||||
from: 1000,
|
||||
strike: (note) => (started.push(note.at), 1),
|
||||
};
|
||||
playbacks.start(1, 'composition:x', play, '');
|
||||
time = 0.25;
|
||||
loaded();
|
||||
await Promise.resolve();
|
||||
const runner = playbacks.current()[0]!.runner as unknown as { position(): { ticks: number } };
|
||||
expect(runner.position().ticks).toBe(1250);
|
||||
});
|
||||
});
|
||||
|
||||
/** A sink that writes down what it is asked to play and stop. */
|
||||
function recordingAudio() {
|
||||
const sink = {
|
||||
...SILENT_AUDIO,
|
||||
started: [] as { id: number; name: string; play: SequencePlay }[],
|
||||
stopped: [] as number[],
|
||||
playSequence: (name: string, play: SequencePlay) => {
|
||||
const id = 100 + sink.started.length;
|
||||
sink.started.push({ id, name, play });
|
||||
return id;
|
||||
},
|
||||
stopPlayback: (id: number) => {
|
||||
sink.stopped.push(id);
|
||||
},
|
||||
};
|
||||
return sink;
|
||||
}
|
||||
|
||||
describe('a composition played in a game', () => {
|
||||
function run(f: ReturnType<typeof makeCut>) {
|
||||
const behavior = makeBehavior(f, 'Director');
|
||||
f.stack.execute(new PlaceAssetCommand(f.sceneId, behavior.id, { x: 0, y: 0 }, f.placement));
|
||||
const edit = graphEditor(f, behavior);
|
||||
const start = edit.addNode('OnStart');
|
||||
const owner = edit.addNode('Owner');
|
||||
const play = edit.addNode('PlayComposition');
|
||||
edit.connect(start, 'exec', play, 'exec');
|
||||
edit.connect(owner, 'object', play, 'object');
|
||||
const audio = recordingAudio();
|
||||
const world = World.instantiate(f.project, f.sceneId);
|
||||
addGameSystems(world, f.project, new Input(), { audio });
|
||||
return { audio, world, frame: (times = 1) => Array.from({ length: times }, () => world.tick(0.1)) };
|
||||
}
|
||||
|
||||
it('is heard from where its clock stands once it runs, and stops with its end', () => {
|
||||
const f = makeCut();
|
||||
f.lay(f.audioItem('a', 'sound', 0, 5));
|
||||
const game = run(f);
|
||||
game.frame();
|
||||
// The first tick shows frame 0; the clock and the sound run from the next.
|
||||
expect(game.audio.started).toEqual([]);
|
||||
game.frame();
|
||||
expect(game.audio.started).toHaveLength(1);
|
||||
expect(game.audio.started[0]!.name).toBe(`composition:${f.cut().id}`);
|
||||
expect(game.audio.started[0]!.play.from).toBe(Math.round((0.1 + LEAD) * 1000));
|
||||
game.frame(2);
|
||||
expect(game.audio.started).toHaveLength(1);
|
||||
game.frame(3);
|
||||
expect(game.audio.stopped).toEqual([100]);
|
||||
});
|
||||
|
||||
it('stands silent while the game is paused, and goes on from where its clock stood', () => {
|
||||
const f = makeCut();
|
||||
f.lay(f.audioItem('a', 'sound', 0, 50));
|
||||
const game = run(f);
|
||||
game.frame(3);
|
||||
expect(game.audio.started).toHaveLength(1);
|
||||
game.world.paused = true;
|
||||
game.frame(2);
|
||||
expect(game.audio.stopped).toEqual([100]);
|
||||
expect(game.audio.started).toHaveLength(1);
|
||||
game.world.paused = false;
|
||||
game.frame();
|
||||
expect(game.audio.started).toHaveLength(2);
|
||||
expect(game.audio.started[1]!.play.from).toBe(Math.round((0.3 + LEAD) * 1000));
|
||||
});
|
||||
});
|
||||
@@ -67,6 +67,44 @@ describe('a scroll container with tail space', () => {
|
||||
});
|
||||
});
|
||||
|
||||
describe('a scroll container holding its offset', () => {
|
||||
const scrolledToEnd = (holdOffset: boolean) => {
|
||||
const scroll = new ScrollContainer();
|
||||
scroll.holdOffset = holdOffset;
|
||||
scroll.resize(200, 100);
|
||||
const rows = new VStack();
|
||||
for (let i = 0; i < 10; i++) rows.add(new Spacer(20));
|
||||
scroll.setContent(rows);
|
||||
scroll.scrollTo(100);
|
||||
// Three rows go, as a folded group's would.
|
||||
rows.removeChildren(7);
|
||||
scroll.relayoutContent();
|
||||
return { scroll, rows };
|
||||
};
|
||||
|
||||
it('keeps the offset when the content shrinks, the room left below', () => {
|
||||
const { scroll, rows } = scrolledToEnd(true);
|
||||
expect(scroll.offset).toBe(100);
|
||||
expect(rows.h).toBe(200);
|
||||
});
|
||||
|
||||
it('lets the room go as the view is scrolled up off it', () => {
|
||||
const { scroll, rows } = scrolledToEnd(true);
|
||||
scroll.scrollTo(70);
|
||||
expect(rows.h).toBe(170);
|
||||
expect(scroll.maxScroll).toBe(70);
|
||||
scroll.scrollTo(20);
|
||||
expect(rows.h).toBe(140);
|
||||
expect(scroll.maxScroll).toBe(40);
|
||||
});
|
||||
|
||||
it('without it clamps the offset, sliding every row down', () => {
|
||||
const { scroll, rows } = scrolledToEnd(false);
|
||||
expect(scroll.offset).toBe(40);
|
||||
expect(rows.h).toBe(140);
|
||||
});
|
||||
});
|
||||
|
||||
describe('a scroll container and its scrollbar', () => {
|
||||
const filled = (viewportH: number, rowCount: number) => {
|
||||
const scroll = new ScrollContainer();
|
||||
|
||||
@@ -57,6 +57,7 @@ export async function assets(t) {
|
||||
|
||||
await addMenuOnRightClick(t);
|
||||
await emptyGroupsOnSwitch(t);
|
||||
await foldKeepsPlace(t);
|
||||
|
||||
// 4b. Drag the Crate asset from the panel into the scene view. Driven step by step rather than
|
||||
// with `drag()`, because the preview only exists while the button is still down.
|
||||
@@ -681,3 +682,56 @@ async function emptyGroupsOnSwitch(t) {
|
||||
JSON.stringify({ before: shape(before), off: shape(off) }),
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Folding a group leaves the list where it stood: it does not jump back to the selected asset, and
|
||||
* at the end of the list, where the room below lets it scroll, the rows do not slide down by the
|
||||
* ones that went away. The heading clicked stays under the pointer, folded and unfolded.
|
||||
*/
|
||||
async function foldKeepsPlace(t) {
|
||||
const { check, sleep, json, mutate, mouse, click, select, showTab } = t;
|
||||
const before = await json('__gm.state.selection');
|
||||
await mutate(`
|
||||
const p = __gm.state.project;
|
||||
for (let i = 1; i <= 30; i++)
|
||||
p.assets.push({ id: 'fold04-' + i, type: 'sprite', name: 'Fold ' + String(i).padStart(2, '0'),
|
||||
props: { width: 10, height: 10, color: 0xffffff, texture: '' } });
|
||||
p.assets.push({ id: 'fold04-sound', type: 'sound', name: 'Fold Sound', props: {}, file: '' });`);
|
||||
// The Crate is selected far above; the Sounds group, unfolded, sits at the end of the list.
|
||||
await select({ kind: 'asset', assetId: 'crate' });
|
||||
await json(`__gm.assetRow('fold04-sound')`);
|
||||
const rect = await json('__gm.assetsRect()');
|
||||
await mouse('mouseWheel', rect.x + rect.w / 2, rect.y + rect.h / 2, 'none', { deltaX: 0, deltaY: 5000 });
|
||||
await sleep(500);
|
||||
const at = async () => ({
|
||||
heading: (await json(`__gm.assetRowShown('group:sound')`))?.y,
|
||||
crate: (await json(`__gm.assetRowShown('crate')`))?.y,
|
||||
});
|
||||
const start = await at();
|
||||
const heading = await json(`__gm.assetRowShown('group:sound')`);
|
||||
await click(heading.x, heading.y);
|
||||
await sleep(200);
|
||||
const folded = await at();
|
||||
const groupFolded = !(await json('__gm.assetGroups()')).find((g) => g.type === 'sound')?.expanded;
|
||||
await sleep(450);
|
||||
await click(heading.x, heading.y);
|
||||
await sleep(200);
|
||||
const unfolded = await at();
|
||||
check(
|
||||
'folding a group at the end of the list leaves its heading under the pointer, the selection out of view',
|
||||
start.heading > rect.y &&
|
||||
start.heading < rect.y + rect.h &&
|
||||
start.crate < rect.y &&
|
||||
groupFolded &&
|
||||
folded.heading === start.heading &&
|
||||
unfolded.heading === start.heading &&
|
||||
folded.crate === start.crate &&
|
||||
unfolded.crate === start.crate,
|
||||
JSON.stringify({ rect, start, folded, groupFolded, unfolded }),
|
||||
);
|
||||
await mutate(`
|
||||
const p = __gm.state.project;
|
||||
p.assets = p.assets.filter((a) => !a.id.startsWith('fold04-'));`);
|
||||
await select(before ?? null);
|
||||
await showTab('scene');
|
||||
}
|
||||
@@ -10,7 +10,8 @@ import { MAKE_OBJECT } from './lib/scaffold.mjs';
|
||||
* item is left. Nothing of the run stays in the scene. An item's video is waited for: the clock
|
||||
* stands until the Game tab has its first picture, for a clip that starts later the picture there,
|
||||
* and then the video plays at once. A recording that runs meanwhile takes no frame. When the item
|
||||
* ends, its video stops and goes quiet with it.
|
||||
* ends, its video stops and goes quiet with it. A sound item is planned once the clock runs, timed
|
||||
* to the cut.
|
||||
*
|
||||
* Everything it adds carries a `cp68` id and goes again at the end.
|
||||
*/
|
||||
@@ -368,6 +369,59 @@ export async function composition(t) {
|
||||
}),
|
||||
);
|
||||
|
||||
// 68h (#418). A sound item half a second in, beside the video item from 68d: the cut waits for the
|
||||
// video, and its sound with it – planned once the clock runs, for the second of the audio
|
||||
// clock the cut reaches the item at, and to its end.
|
||||
const sounds = await evaluate(`fetch('/feuerwerk/api/index.php?r=sounds').then((r) => r.json())`);
|
||||
const blip = sounds.find((item) => item.bytes === 4044);
|
||||
await mutate(`
|
||||
const p = __gm.state.project;
|
||||
p.assets.push({ id: 'cp68-blip', type: 'sound', name: 'Blip 68', props: {}, file: ${JSON.stringify(blip?.file ?? '')} });
|
||||
p.assets.find((a) => a.id === 'cp68-screen').props.texture = 'cp68-video';
|
||||
const c = p.assets.find((a) => a.id === 'cp68-cut');
|
||||
c.items = [
|
||||
{ id: 'cp68-v', track: 'cp68-front', start: 0, length: 120, offset: 0 },
|
||||
{ id: 'cp68-tone', track: 'cp68-back', start: 30, length: 30, offset: 0,
|
||||
audio: { kind: 'sound', asset: 'cp68-blip', volume: 0, bus: '' } },
|
||||
];`);
|
||||
await evaluate(`{
|
||||
window.cp68Sound = new Promise((done) => {
|
||||
const rows = [];
|
||||
const until = performance.now() + 20000;
|
||||
const step = () => {
|
||||
if (__gm.state.mode === 'play') {
|
||||
const time = ${clock};
|
||||
const tone = __gm.sound.playbacks().find((p) => p.name === 'composition:cp68-cut');
|
||||
rows.push({ time, clock: __gm.sound.clock(), planned: tone?.clips.length ?? 0, clip: tone?.clips[0] ?? null });
|
||||
if (time !== null && time >= 0.6) return done(JSON.stringify(rows));
|
||||
}
|
||||
if (performance.now() > until) return done(JSON.stringify(rows));
|
||||
requestAnimationFrame(step);
|
||||
};
|
||||
step();
|
||||
});
|
||||
null;
|
||||
}`);
|
||||
await play();
|
||||
const toneRows = JSON.parse(await evaluate('window.cp68Sound'));
|
||||
await stop();
|
||||
const ran = toneRows.findIndex((row) => row.time > 0);
|
||||
const reached = toneRows.find((row) => row.time >= 0.5);
|
||||
const toneClip = toneRows.at(-1)?.clip;
|
||||
const early = reached && toneClip ? reached.clock - toneClip.at : null;
|
||||
check(
|
||||
'a sound item is planned only once the cut’s clock runs, for when the cut reaches it, to its end',
|
||||
ran > 0 &&
|
||||
toneRows.slice(0, ran).every((row) => row.planned === 0) &&
|
||||
toneClip?.sound === 'cp68-blip' &&
|
||||
toneClip.offset === 0 &&
|
||||
Math.abs(toneClip.duration - 0.5) < 0.002 &&
|
||||
early !== null &&
|
||||
early > -0.02 &&
|
||||
early < 0.05,
|
||||
JSON.stringify({ waited: ran, early, reached, toneClip }),
|
||||
);
|
||||
|
||||
await mutate(`
|
||||
const p = __gm.state.project;
|
||||
p.assets = p.assets.filter((a) => !a.id.startsWith('cp68-') && a.id !== ${JSON.stringify(madeId)});
|
||||
|
||||
@@ -8,7 +8,7 @@
|
||||
* whose player is still loading. A copied item is pasted under the pointer or at the playhead. A
|
||||
* video dragged in wears a sprite the composition keeps to itself. An item dragged in the picture
|
||||
* follows the pointer while the button is down. A band picks items, with Alt clips, and copied clips
|
||||
* are pasted where they fit.
|
||||
* are pasted where they fit. A sound dragged in is an audio item, heard while the playhead runs.
|
||||
*
|
||||
* Everything it adds carries a `ct69` id and goes again at the end.
|
||||
*/
|
||||
@@ -486,6 +486,111 @@ export async function compositionTab(t) {
|
||||
JSON.stringify({ frameWidth, along, alone }),
|
||||
);
|
||||
|
||||
// 69o (#418). A sound dragged onto a track is an audio item as long as one pass of it, picked on
|
||||
// the composition, the inspector naming it. Played from the middle, the composition plans it as a
|
||||
// clip that far into the sound, on the item's own bus and to the item's end; Pause ends it, and a
|
||||
// scrub plays nothing.
|
||||
const sounds = await evaluate(`fetch('/feuerwerk/api/index.php?r=sounds').then((r) => r.json())`);
|
||||
const blip = sounds.find((item) => item.bytes === 4044);
|
||||
await mutate(`
|
||||
const p = __gm.state.project;
|
||||
p.assets.push({ id: 'ct69-sound', type: 'sound', name: 'Blip 69', props: {}, file: ${JSON.stringify(blip?.file ?? '')} });
|
||||
const c = p.assets.find((a) => a.id === '${CUT}');
|
||||
c.objects = [];
|
||||
c.items = [];
|
||||
c.clips = [];`);
|
||||
const seconds = await evaluate(`__gm.sound.load(${JSON.stringify(blip?.file ?? '')})`);
|
||||
const soundDrop = await dragAsset('ct69-sound', await json(`__gm.composition.laneAt('ct69-t2', 10)`));
|
||||
m = await model();
|
||||
const tone = m.items[0];
|
||||
const tonePick = await json('__gm.state.selection');
|
||||
const toneSection = await json(`__gm.section('SOUND ITEM')`);
|
||||
await mutate(`
|
||||
const c = __gm.state.project.assets.find((a) => a.id === '${CUT}');
|
||||
c.items[0].length = 20;`);
|
||||
await evaluate('__gm.composition.scrub(15)');
|
||||
await sleep(200);
|
||||
const scrubbed = (await json('__gm.sound.playbacks()')).filter((p) => p.name === `composition:${CUT}`);
|
||||
const toneAt = await json('__gm.composition.playAt()');
|
||||
await mouse('mouseMoved', toneAt.x, toneAt.y);
|
||||
await t.click(toneAt.x, toneAt.y);
|
||||
let heard = [];
|
||||
for (let i = 0; i < 40 && !heard.some((p) => p.name === `composition:${CUT}` && p.clips.length > 0); i++) {
|
||||
await sleep(50);
|
||||
heard = await json('__gm.sound.playbacks()');
|
||||
}
|
||||
const toneView = await json('__gm.composition.view()');
|
||||
const toneSound = await json('__gm.composition.sound()');
|
||||
const playing = heard.find((p) => p.name === `composition:${CUT}`);
|
||||
const toneClip = playing?.clips[0];
|
||||
await t.click(toneAt.x, toneAt.y);
|
||||
await sleep(300);
|
||||
const paused = (await json('__gm.sound.playbacks()')).filter((p) => p.name === `composition:${CUT}`);
|
||||
check(
|
||||
'a sound dragged onto a track is an audio item, heard from the playhead on its own bus until the item ends',
|
||||
seconds > 0 &&
|
||||
soundDrop?.what === 'item' &&
|
||||
tone?.audio?.kind === 'sound' &&
|
||||
tone.audio.asset === 'ct69-sound' &&
|
||||
tone.start === 10 &&
|
||||
tone.length === Math.max(1, Math.round(seconds * 10)) &&
|
||||
m.objects.length === 0 &&
|
||||
tonePick?.kind === 'composition' &&
|
||||
JSON.stringify(tonePick.itemIds) === JSON.stringify([tone.id]) &&
|
||||
toneSection !== null &&
|
||||
scrubbed.length === 0 &&
|
||||
toneClip?.sound === 'ct69-sound' &&
|
||||
toneClip.bus === `composition:${CUT}:${tone.id}` &&
|
||||
toneSound.playing &&
|
||||
toneSound.playings.includes(playing.id) &&
|
||||
toneClip.offset > 0.5 &&
|
||||
toneClip.offset < 0.8 &&
|
||||
Math.abs(toneClip.offset + toneClip.duration - 2) < 0.002 &&
|
||||
paused.length === 0,
|
||||
JSON.stringify({
|
||||
seconds,
|
||||
soundDrop,
|
||||
tone,
|
||||
tonePick,
|
||||
scrubbed,
|
||||
toneClip,
|
||||
paused: paused.length,
|
||||
toneView,
|
||||
heard,
|
||||
toneSection,
|
||||
toneSound,
|
||||
}),
|
||||
);
|
||||
|
||||
// 69o-ii. A pattern and a song leave the Assets panel as well and land as audio items as long as
|
||||
// their preview promised – the panel once let only sounds go.
|
||||
await mutate(`
|
||||
const p = __gm.state.project;
|
||||
p.assets.push(
|
||||
{ id: 'ct69-pattern', type: 'pattern', name: 'Riff 69', props: { bars: 1 }, notes: [] },
|
||||
{ id: 'ct69-song', type: 'song', name: 'Song 69', props: {}, tracks: [], items: [], graph: { nodes: [], edges: [] } },
|
||||
);
|
||||
p.assets.find((a) => a.id === '${CUT}').items = [];`);
|
||||
const patternDrop = await dragAsset('ct69-pattern', await json(`__gm.composition.laneAt('ct69-t1', 10)`));
|
||||
const songDrop = await dragAsset('ct69-song', await json(`__gm.composition.laneAt('ct69-t2', 10)`));
|
||||
m = await model();
|
||||
const kinds = m.items.map((i) => ({
|
||||
kind: i.audio?.kind,
|
||||
asset: i.audio?.asset,
|
||||
start: i.start,
|
||||
length: i.length,
|
||||
}));
|
||||
const landed = (drop, kind) =>
|
||||
drop?.what === 'item' &&
|
||||
kinds.some(
|
||||
(i) => i.kind === kind && i.asset === drop.assetId && i.start === 10 && i.length === drop.length,
|
||||
);
|
||||
check(
|
||||
'a pattern and a song dragged onto a track are audio items as long as their preview',
|
||||
landed(patternDrop, 'pattern') && landed(songDrop, 'song') && m.items.length === 2,
|
||||
JSON.stringify({ patternDrop, songDrop, kinds }),
|
||||
);
|
||||
|
||||
await showTab('scene');
|
||||
await mutate(`
|
||||
const p = __gm.state.project;
|
||||
|
||||
Reference in new issue
Block a user