Bake a pattern or song to Opus, played in its place by a game (#340)
A pattern or song played by a game runs every note through the scheduler, the instruments and their FX, live. Bake works that out once, keeps it as a file on the asset, and Play pattern and Play song play the file instead, as one voice, for as long as nothing it is made of has changed. The tabs always play live. - Bake is an icon in the header of the Pattern and the Song tab, lit while the bake holds. A click bakes, cancels a bake that runs, or offers Bake again and Remove bake. The toolbar shows the progress. The info of the tab says "baked" or "bake outdated". - The bake holds everything up to the output of the pattern or song. The sound FX of the node, fx params, volume, fade in, bus, Master routing and stopping with a fade stay live. - A print of the worked-out plan decides whether it holds: notes and clips, tempo, length, loop, tracks, the build of each instrument for each note, sound FX, files, cuts and levels. A rename keeps it, an undo back to the baked state makes it hold again. - The file holds the straight run and then one more round of the loop as heard the second time, tails included. Looped, it plays to the loop end and then round that round, to the sample. Once, it fades out over its last 10 ms. Let ring fades it out over 0.3 s. - It is worked out by a second SoundLibrary on an OfflineAudioContext, its scheduler woken by suspends of the offline clock, and what reaches the master is taken. A worklet probe tells whether that was mono, so a mono bake goes on exactly as the notes did. - Opus in Ogg, 160 kbit/s stereo or 96 kbit/s mono, encoded with WebCodecs, the Ogg pages written by hand. Normalized to full scale, the peak kept and played back. - The file goes up into the sounds of the account as "[bake] name", which the sound picker leaves out. An old bake file is deleted once the project is saved without it. Only a bake that holds is loaded. - Ogg files are decoded at 48 kHz: resampled to 44.1 kHz, the loop of a bake lost a sample a round. - Baking and removing a bake are no undo steps, like the EQ's zoom. Findings: - Chrome's AudioEncoder refuses opus.format 'ogg' (NotSupportedError) but hands out the OpusHead with a pre-skip of 312 for 'opus'. - The Song tab's header is full at 1280 px: a captioned Bake pushed Fit off the tab. Hence the icon. Tests: - Unit: tests/core/ogg.test.ts (6), tests/core/bake.test.ts (16), four new in sound-library.test.ts, one in song-play.test.ts. One test cuts, reverses and fades a clip on a song's track: each change outdates the bake, each undo makes it hold again. - e2e 64 (melektaus-atx): ALL PASSED, 36 checks (6 new). Live and baked rendered offline differ by at most 0.06 dB over 114 windows of 20 ms, two loop ends among them. e2e 65 had no red check with the icon in its header. - Gate (melektaus-atx): CHECK EXIT 0, 277 files, 4281 tests. - Counter-checks: print without instrument builds, print taking the sound's file instead of the clip's cut, round taken from the first pass, bake voice without its looping source, Play pattern or Play song ignoring the bake: one unit test red each. Pre-skip set to 0: the e2e comparison 4.8 dB off, only that check red. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed
1 parent
3377524958
commit
84cd36f902
28 files changed
+1979
-78
No files matched your search
@@ -0,0 +1,336 @@
|
||||
/**
|
||||
* A pattern or song baked (#340): where the parts of its file lie, how it is put together from the
|
||||
* playing looped and the one played once, raised to full scale, how it is played back – and what it
|
||||
* is made from, so a change to anything it sounds by outdates it, and an undo brings it back.
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
CreateAssetCommand,
|
||||
CreateSoundCommand,
|
||||
RenameAssetCommand,
|
||||
SetAssetPropertyCommand,
|
||||
} from '../../src/core/commands/assetCommands';
|
||||
import { AddClipCommand, SetClipPropsCommand } from '../../src/core/commands/clipCommands';
|
||||
import { SetNodeParamCommand } from '../../src/core/commands/graphCommands';
|
||||
import { SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
|
||||
import { SetSongCommand } from '../../src/core/commands/songCommands';
|
||||
import type { InstrumentAsset, PatternAsset, SongAsset } from '../../src/core/model/Asset';
|
||||
import type { PropertyBag } from '../../src/core/model/Properties';
|
||||
import { projectFiles } from '../../src/core/model/projectFiles';
|
||||
import {
|
||||
assembleBake,
|
||||
type Bake,
|
||||
bakePlayback,
|
||||
bakePrint,
|
||||
bakeRuns,
|
||||
bakeSpans,
|
||||
bakeUploadName,
|
||||
heldBake,
|
||||
isBakeUpload,
|
||||
normalizeBake,
|
||||
validBake,
|
||||
} from '../../src/core/music/bake';
|
||||
import { type BakePlay, Playbacks, type SequencePlay } from '../../src/core/music/playbacks';
|
||||
import { LEAD } from '../../src/core/music/Scheduler';
|
||||
import { newItem } from '../../src/core/music/song';
|
||||
import { deserializeProject, serializeProject } from '../../src/core/store/serialize';
|
||||
import { SILENT_AUDIO } from '../../src/runtime/systems/Audio';
|
||||
import { graphEditor, makeBehavior, makeSpriteScene, place, startWorld } from '../helpers';
|
||||
|
||||
const Q = 480;
|
||||
const FILE = `${'a1'.repeat(32)}.wav`;
|
||||
const OTHER = `${'b2'.repeat(32)}.wav`;
|
||||
const BAKED = `${'c3'.repeat(6)}/${'d4'.repeat(32)}.ogg`;
|
||||
|
||||
/** A pattern of two notes over two bars at 90 BPM, through an instrument of one sample. */
|
||||
function baked() {
|
||||
const f = makeSpriteScene('Box');
|
||||
const sound = new CreateSoundCommand('Blip', FILE);
|
||||
f.stack.execute(sound);
|
||||
const other = new CreateSoundCommand('Bell', OTHER);
|
||||
f.stack.execute(other);
|
||||
const ins = new CreateAssetCommand('instrument', 'Lead');
|
||||
f.stack.execute(ins);
|
||||
const lead = ins.asset as InstrumentAsset;
|
||||
const sample = lead.graph.nodes.find((n) => n.type === 'InstrumentSample')!;
|
||||
sample.params.sound = sound.asset.id;
|
||||
const made = new CreateAssetCommand('pattern', 'Tune');
|
||||
f.stack.execute(made);
|
||||
const id = made.asset.id;
|
||||
f.stack.execute(new SetAssetPropertyCommand(id, 'instrument', lead.id));
|
||||
f.stack.execute(new SetAssetPropertyCommand(id, 'tempo', 90));
|
||||
f.stack.execute(new SetAssetPropertyCommand(id, 'bars', 2));
|
||||
f.stack.execute(
|
||||
new SetPatternNotesCommand(id, [
|
||||
{ id: 'n1', tick: 0, length: Q, note: 60, velocity: 0.5 },
|
||||
{ id: 'n2', tick: Q, length: Q, note: 64, velocity: 1 },
|
||||
]),
|
||||
);
|
||||
const pattern = () => f.project.assets.find((a) => a.id === id) as PatternAsset;
|
||||
return { ...f, id, lead, sample, sound: sound.asset, other: other.asset, pattern };
|
||||
}
|
||||
|
||||
function bakeOf(print: string, fields: Partial<Bake> = {}): Bake {
|
||||
return {
|
||||
file: BAKED,
|
||||
print,
|
||||
peak: 0.5,
|
||||
length: 100,
|
||||
loopEnd: 100,
|
||||
head: 100,
|
||||
round: 100,
|
||||
bytes: 1234,
|
||||
...fields,
|
||||
};
|
||||
}
|
||||
|
||||
describe('the parts of a bake', () => {
|
||||
it('lies in frames at 48 kHz: the straight run as far as the end or the loop end, then one round', () => {
|
||||
// Two bars of 4/4 at 120 BPM are 4 s; a loop of the second bar.
|
||||
const source = { tempo: 120, length: 8 * Q, loop: { start: 4 * Q, end: 8 * Q } };
|
||||
expect(bakeSpans(source)).toEqual({ length: 192000, loopEnd: 192000, head: 192000, round: 96000 });
|
||||
const short = { ...source, loop: { start: 0, end: 4 * Q } };
|
||||
expect(bakeSpans(short)).toEqual({ length: 192000, loopEnd: 96000, head: 192000, round: 96000 });
|
||||
});
|
||||
|
||||
it('is played looped once past the loop end; once only where that goes further', () => {
|
||||
expect(bakeRuns({ length: 100, loopEnd: 100, round: 100 })).toEqual({ looped: 200, once: null });
|
||||
expect(bakeRuns({ length: 300, loopEnd: 100, round: 50 })).toEqual({ looped: 150, once: 300 });
|
||||
});
|
||||
|
||||
it('takes the head from the playing once where there is one, the round always from the looped one', () => {
|
||||
const ramp = (length: number, from: number) => Float32Array.from({ length }, (_, i) => from + i);
|
||||
const looped = [ramp(150, 0), ramp(150, 1000)];
|
||||
const out = assembleBake({ loopEnd: 100, head: 100, round: 50 }, looped, null);
|
||||
expect([out[0]!.length, out[0]![99], out[0]![100], out[1]![149]]).toEqual([150, 99, 100, 1149]);
|
||||
const once = [ramp(300, 5000), ramp(300, 6000)];
|
||||
const both = assembleBake({ loopEnd: 100, head: 300, round: 50 }, looped, once);
|
||||
// The round follows the head and starts where the looped playing passed its loop end.
|
||||
expect([both[0]!.length, both[0]![299], both[0]![300], both[0]![349]]).toEqual([350, 5299, 100, 149]);
|
||||
});
|
||||
|
||||
it('is raised to full scale, and played back at the peak it had', () => {
|
||||
const sides = [Float32Array.of(0.25, -0.5), Float32Array.of(0.125, 0)];
|
||||
expect(normalizeBake(sides)).toBe(0.5);
|
||||
expect([...sides[0]!, ...sides[1]!]).toEqual([0.5, -1, 0.25, 0]);
|
||||
expect(normalizeBake([new Float32Array(3)])).toBe(0);
|
||||
});
|
||||
|
||||
it('plays once to its end, or to the loop end and then round its round', () => {
|
||||
const spans = { length: 96000, loopEnd: 96000, head: 96000, round: 48000 };
|
||||
expect(bakePlayback(spans, false)).toEqual({ end: 2, until: 2, round: null });
|
||||
expect(bakePlayback(spans, true)).toEqual({ end: null, until: 2, round: { start: 2, end: 3 } });
|
||||
expect(bakePlayback({ ...spans, length: 144000, head: 144000 }, true)).toEqual({
|
||||
end: null,
|
||||
until: 2,
|
||||
round: { start: 3, end: 4 },
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('what a bake is made from', () => {
|
||||
it('stays the same through a rename, changes with a note and comes back with the undo', () => {
|
||||
const f = baked();
|
||||
const print = bakePrint(f.project, f.pattern());
|
||||
expect(print).not.toBe('');
|
||||
f.stack.execute(new RenameAssetCommand(f.id, 'Other tune'));
|
||||
expect(bakePrint(f.project, f.pattern())).toBe(print);
|
||||
f.stack.execute(
|
||||
new SetPatternNotesCommand(f.id, [{ id: 'n1', tick: 0, length: Q, note: 60, velocity: 0.75 }]),
|
||||
);
|
||||
expect(bakePrint(f.project, f.pattern())).not.toBe(print);
|
||||
f.stack.undo();
|
||||
expect(bakePrint(f.project, f.pattern())).toBe(print);
|
||||
});
|
||||
|
||||
it('changes with what the pattern only names: the instrument and the level of its sound', () => {
|
||||
const f = baked();
|
||||
const print = bakePrint(f.project, f.pattern());
|
||||
f.stack.execute(new SetNodeParamCommand(f.lead.id, f.sample.id, 'sound', f.other.id));
|
||||
expect(bakePrint(f.project, f.pattern())).not.toBe(print);
|
||||
f.stack.undo();
|
||||
expect(bakePrint(f.project, f.pattern())).toBe(print);
|
||||
f.stack.execute(new SetAssetPropertyCommand(f.sound.id, 'volume', 0.5));
|
||||
expect(bakePrint(f.project, f.pattern())).not.toBe(print);
|
||||
});
|
||||
|
||||
it('changes with the cut of a clip on a track of a song, and comes back with the undo', () => {
|
||||
const f = baked();
|
||||
const clip = new AddClipCommand(f.sound.id, { start: 0.25, end: 1 });
|
||||
f.stack.execute(clip);
|
||||
const made = new CreateAssetCommand('song', 'Theme');
|
||||
f.stack.execute(made);
|
||||
const song = () => f.project.assets.find((a) => a.id === made.asset.id) as SongAsset;
|
||||
const track = song().tracks[0]!.id;
|
||||
const item = newItem({ track, kind: 'sound', asset: clip.clipId, tick: 0, length: 4 * Q });
|
||||
f.stack.execute(new SetSongCommand(song().id, { items: [item] }));
|
||||
const print = bakePrint(f.project, song());
|
||||
expect(print).not.toBe('');
|
||||
const changes: PropertyBag[] = [
|
||||
{ end: 0.75 },
|
||||
{ reverse: true },
|
||||
{ fadeOut: 50 },
|
||||
{ fadeInCurve: 'sCurve' },
|
||||
];
|
||||
for (const change of changes) {
|
||||
f.stack.execute(new SetClipPropsCommand(f.sound.id, clip.clipId, change));
|
||||
expect(bakePrint(f.project, song()), JSON.stringify(change)).not.toBe(print);
|
||||
f.stack.undo();
|
||||
expect(bakePrint(f.project, song())).toBe(print);
|
||||
}
|
||||
});
|
||||
|
||||
it('is nothing for a pattern that plays nothing', () => {
|
||||
const f = baked();
|
||||
f.stack.execute(new SetAssetPropertyCommand(f.id, 'instrument', ''));
|
||||
expect(bakePrint(f.project, f.pattern())).toBe('');
|
||||
});
|
||||
|
||||
it('holds while the print is the same, and only then is its file loaded with the project', () => {
|
||||
const f = baked();
|
||||
f.pattern().bake = bakeOf(bakePrint(f.project, f.pattern()));
|
||||
expect(validBake(f.project, f.pattern())?.file).toBe(BAKED);
|
||||
expect(projectFiles(f.project).sounds).toContain(BAKED);
|
||||
f.stack.execute(new SetAssetPropertyCommand(f.id, 'tempo', 100));
|
||||
expect(validBake(f.project, f.pattern())).toBeNull();
|
||||
expect(projectFiles(f.project).sounds).not.toContain(BAKED);
|
||||
});
|
||||
});
|
||||
|
||||
describe('a bake kept with the project', () => {
|
||||
it('is saved and loaded with its pattern; one that does not fit together is dropped', () => {
|
||||
const f = baked();
|
||||
f.pattern().bake = bakeOf('x');
|
||||
const back = deserializeProject(serializeProject(f.project));
|
||||
expect((back.assets.find((a) => a.id === f.id) as PatternAsset).bake).toEqual(bakeOf('x'));
|
||||
expect(heldBake(bakeOf('x', { head: 50 }))).toBeNull();
|
||||
expect(heldBake(bakeOf('x', { round: 101 }))).toBeNull();
|
||||
expect(heldBake(bakeOf('x', { file: 'x.ogg' }))).toBeNull();
|
||||
expect(heldBake(bakeOf(''))).toBeNull();
|
||||
});
|
||||
|
||||
it('goes up under its name behind a mark the sound picker leaves out by, a slash no folder', () => {
|
||||
expect(bakeUploadName('Boss / theme')).toBe('[bake] Boss - theme.ogg');
|
||||
expect([isBakeUpload('[bake] Boss - theme.ogg'), isBakeUpload('Boss.ogg')]).toEqual([true, false]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Play pattern with a bake', () => {
|
||||
function played(loop: boolean) {
|
||||
const f = baked();
|
||||
const behavior = makeBehavior(f, 'Logic');
|
||||
const box = place(f, f.spriteId, { x: 0, y: 0 });
|
||||
place(f, behavior.id, { x: 0, y: 0 }, box);
|
||||
const edit = graphEditor(f, behavior);
|
||||
edit.connect(
|
||||
edit.addNode('OnStart'),
|
||||
'exec',
|
||||
edit.addNode('PlayPattern', { pattern: f.id, volume: 0.5, loop }),
|
||||
'exec',
|
||||
);
|
||||
const asked: SequencePlay[] = [];
|
||||
const audio = {
|
||||
...SILENT_AUDIO,
|
||||
playSequence: (_name: string, play: SequencePlay) => (asked.push(play), 1),
|
||||
};
|
||||
return { ...f, run: () => startWorld(f, { behaviors: true, audio }).frame(), asked };
|
||||
}
|
||||
|
||||
it('hands the bake over while it holds, at its peak times the volume of the node', () => {
|
||||
const f = played(true);
|
||||
const length = 2 * 4 * Q;
|
||||
const spans = bakeSpans({ tempo: 90, length, loop: { start: 0, end: length } });
|
||||
f.pattern().bake = bakeOf(bakePrint(f.project, f.pattern()), spans);
|
||||
f.run();
|
||||
expect(f.asked[0]?.bake).toEqual({ file: BAKED, gain: 0.25, playback: bakePlayback(spans, true) });
|
||||
expect(f.asked[0]?.sequence.notes).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('hands over none once it is outdated, and the notes are played', () => {
|
||||
const f = played(false);
|
||||
f.pattern().bake = bakeOf('outdated');
|
||||
f.run();
|
||||
expect(f.asked[0]?.bake).toBeUndefined();
|
||||
expect(f.asked[0]?.loop).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
describe('a playing that plays its bake', () => {
|
||||
function host() {
|
||||
const made = {
|
||||
time: 10,
|
||||
sounding: new Set<number>(),
|
||||
baked: [] as { id: number; bake: BakePlay; at: number }[],
|
||||
stopped: [] as { voices: number[]; letRing: boolean }[],
|
||||
pumps: 0,
|
||||
loading: [] as (() => void)[],
|
||||
};
|
||||
const playbacks = new Playbacks({
|
||||
now: () => made.time,
|
||||
latency: () => 0,
|
||||
playing: (voice) => made.sounding.has(voice),
|
||||
stop: (voices, letRing = false) => {
|
||||
made.stopped.push({ voices: [...voices], letRing });
|
||||
for (const voice of voices) made.sounding.delete(voice);
|
||||
},
|
||||
wake: () => {
|
||||
made.pumps += 1;
|
||||
return () => {};
|
||||
},
|
||||
prepare: () => new Promise<void>((resolve) => made.loading.push(resolve)),
|
||||
ahead: () => 0.25,
|
||||
level: () => {},
|
||||
after: () => {},
|
||||
playBaked: (id, bake, at) => {
|
||||
made.baked.push({ id, bake, at });
|
||||
made.sounding.add(500);
|
||||
return 500;
|
||||
},
|
||||
});
|
||||
const loaded = async () => {
|
||||
for (const resolve of made.loading.splice(0)) resolve();
|
||||
await Promise.resolve();
|
||||
};
|
||||
return { made, playbacks, loaded };
|
||||
}
|
||||
|
||||
const play = (end: number | null): SequencePlay => ({
|
||||
sequence: { notes: [{ tick: 0, length: Q, note: 60, velocity: 1, instrument: 'lead' }] },
|
||||
tempo: 120,
|
||||
length: 4 * Q,
|
||||
loop: end === null ? { start: 0, end: 4 * Q } : null,
|
||||
needs: { layers: [], files: [BAKED] },
|
||||
strike: () => {
|
||||
throw new Error('a baked playing strikes no note');
|
||||
},
|
||||
bake: {
|
||||
file: BAKED,
|
||||
gain: 1,
|
||||
playback: { end, until: 2, round: end === null ? { start: 2, end: 4 } : null },
|
||||
},
|
||||
});
|
||||
|
||||
it('starts one voice of it once its file is there, as late as its first note would be, waking nothing', async () => {
|
||||
const { made, playbacks, loaded } = host();
|
||||
playbacks.start(7, 'pattern:a', play(null), '');
|
||||
expect(made.baked).toEqual([]);
|
||||
await loaded();
|
||||
expect(made.baked.map(({ id, at }) => ({ id, at }))).toEqual([{ id: 7, at: 10 + LEAD + 0.25 }]);
|
||||
expect(made.pumps).toBe(0);
|
||||
expect(playbacks.current().map((p) => [p.id, p.baked, p.runner.startedAt])).toEqual([
|
||||
[7, true, 10 + LEAD + 0.25],
|
||||
]);
|
||||
playbacks.stopId(7, true);
|
||||
expect(made.stopped).toEqual([{ voices: [500], letRing: true }]);
|
||||
});
|
||||
|
||||
it('played once, it is gone with its voice; looped it stays', async () => {
|
||||
const { made, playbacks, loaded } = host();
|
||||
playbacks.start(7, 'pattern:a', play(2), '');
|
||||
playbacks.start(8, 'pattern:b', play(null), '');
|
||||
await loaded();
|
||||
expect(playbacks.has(7)).toBe(true);
|
||||
made.sounding.clear();
|
||||
expect(playbacks.current().map((p) => p.id)).toEqual([8]);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user