adminandClaude Opus 5.5
2764919bb8
A sound was decoded whole as soon as the project opened, in the editor and in a released game: five minutes of stereo held some 115 MB for as long as the page was open. A sound with Stream on now plays from a copy as Opus in Ogg, fetched a piece at a time and decoded a few seconds ahead by WebCodecs, so a game holds a few megabytes for it. - The Stream switch is a row of the sound, off by default. The Sound tab suggests it from a minute on and says how far the copy has got. - StreamKeeper makes the copies in the browser, a moment after the last change, one at a time: the sound's own and one per clip of its cut, fades and Reverse baked in. A copy that is no longer wanted is taken off and deleted once the project is saved without it. - streamTable says which played names stream from which copy. A clip without a fitting copy, a Sample and a Reverb impulse stay decoded. A sound with Stream on is never decoded ahead. - The page fetches by Range and takes the whole file from a server that ignores it, as the CI's PHP server does. It keeps the bytes and the first two seconds decoded, the rest goes four seconds ahead. - A jump decodes 32 packets before the place first. Measured in Chrome, what follows is then the same to the bit as decoding from the start. With 4 packets it was 0.06 off. - The stream-player on the FX worklet reads the samples at the rate of the context along a Catmull-Rom curve. What comes late is silent and skipped once it is there, so a clip keeps its beat. - The opening stretch goes with the player as it is made. Sent by message it came a few milliseconds late, and 557 samples of the attack were lost. - Pitch on a streamed sound is a shifter after the player, as on a bus: the length stays and it is heard 42.7 ms late. - Bakes of patterns and songs (#340) always stream now. - The song timeline draws a streamed sound from its copy decoded at 8 kHz, keeping only the peaks. Tests: - Gate: CHECK EXIT 0, 280 files, 4312 tests (31 new: ogg +3, stream-queue 6, streams 11, streaming 6, sound-library +5). - e2e 34: ALL PASSED, 106 checks (7 new). Five minutes of mono, 4.8 MB as WAV, 3.6 MB as its copy: streamed the renderer grows by 5.8 MB, decoded by 69.6 MB. Offline at 48 kHz, start, loop seam and a reversed clip are the copy decoded whole to the sample (0). - Counter-checks red as they should be: no pre-roll, no catch-up of late samples at the end, no opening stretch, a streamed sound decoded ahead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
84 lines
3.3 KiB
TypeScript
84 lines
3.3 KiB
TypeScript
/**
|
|
* What a streamed sound (#349) is played from on the audio thread: the samples in order, read at the
|
|
* rate of the context, silent where they come late and skipped once they are there.
|
|
*/
|
|
import { describe, expect, it } from 'vitest';
|
|
import { STREAM_RATE, StreamQueue } from '../../src/core/music/streamQueue';
|
|
|
|
const ramp = (from: number, count: number) => Float32Array.from({ length: count }, (_, i) => from + i);
|
|
const block = (size = 128) => [new Float32Array(size), new Float32Array(size)];
|
|
|
|
describe('the queue of a streamed sound', () => {
|
|
it('plays what it is sent sample for sample at the rate of the stream, across its chunks', () => {
|
|
const queue = new StreamQueue(2, 1);
|
|
queue.push([ramp(0, 100), ramp(1000, 100)]);
|
|
queue.push([ramp(100, 100), ramp(1100, 100)]);
|
|
queue.end();
|
|
const out = block();
|
|
expect(queue.render(out)).toBe(true);
|
|
expect(Array.from(out[0]!.subarray(98, 102))).toEqual([98, 99, 100, 101]);
|
|
expect(Array.from(out[1]!.subarray(0, 2))).toEqual([1000, 1001]);
|
|
expect(queue.render(out)).toBe(false);
|
|
expect(Array.from(out[0]!.subarray(70, 74))).toEqual([198, 199, 0, 0]);
|
|
expect(queue.used).toBe(200);
|
|
});
|
|
|
|
it('plays a mono stream on both sides', () => {
|
|
const queue = new StreamQueue(1, 1);
|
|
queue.push([ramp(5, 300)]);
|
|
const out = block();
|
|
queue.render(out);
|
|
expect(out[1]![3]).toBe(8);
|
|
});
|
|
|
|
it('reads it at the rate of a context running at 44.1 kHz along a smooth curve', () => {
|
|
const step = STREAM_RATE / 44100;
|
|
const queue = new StreamQueue(1, step);
|
|
const sine = (i: number) => Math.sin((2 * Math.PI * 440 * i) / STREAM_RATE);
|
|
for (let at = 0; at < 2 * STREAM_RATE; at += 4800)
|
|
queue.push([Float32Array.from({ length: 4800 }, (_, i) => sine(at + i))]);
|
|
const out = [new Float32Array(44100)];
|
|
queue.render(out);
|
|
let worst = 0;
|
|
for (let i = 4; i < 44000; i++) worst = Math.max(worst, Math.abs(out[0]![i]! - sine(i * step)));
|
|
expect(worst).toBeLessThan(1e-4);
|
|
expect(queue.used).toBe(Math.floor(44100 * step));
|
|
});
|
|
|
|
it('is silent while what is due has not come, and skips as much once it has, so it keeps time', () => {
|
|
const queue = new StreamQueue(1, 1);
|
|
queue.push([ramp(0, 64)]);
|
|
const out = [new Float32Array(128)];
|
|
queue.render(out);
|
|
// The last two it had are held for the curve and come late with the rest.
|
|
expect(out[0]![61]).toBe(61);
|
|
expect(out[0]![100]).toBe(0);
|
|
queue.push([ramp(64, 1000)]);
|
|
expect(queue.used).toBe(128);
|
|
expect(queue.skipped).toBe(66);
|
|
queue.render(out);
|
|
expect(out[0]![0]).toBe(128);
|
|
});
|
|
|
|
it('says it has played out once it has ended and all of it has been read', () => {
|
|
const queue = new StreamQueue(2, 1);
|
|
queue.push([ramp(0, 300), ramp(0, 300)]);
|
|
queue.end();
|
|
expect(queue.render(block())).toBe(true);
|
|
expect(queue.render(block())).toBe(true);
|
|
const out = block();
|
|
expect(queue.render(out)).toBe(false);
|
|
expect(Array.from(out[0]!.subarray(42, 46))).toEqual([298, 299, 0, 0]);
|
|
expect(queue.done).toBe(true);
|
|
});
|
|
|
|
it('skips what came too late even when nothing more comes after it', () => {
|
|
const queue = new StreamQueue(1, 1);
|
|
queue.push([ramp(0, 10)]);
|
|
queue.render(block());
|
|
queue.end();
|
|
expect(queue.done).toBe(true);
|
|
expect(queue.render(block())).toBe(false);
|
|
});
|
|
});
|