Stream long sounds from an Opus copy instead of decoding them (#349)
A sound was decoded whole as soon as the project opened, in the editor and in a released game: five minutes of stereo held some 115 MB for as long as the page was open. A sound with Stream on now plays from a copy as Opus in Ogg, fetched a piece at a time and decoded a few seconds ahead by WebCodecs, so a game holds a few megabytes for it. - The Stream switch is a row of the sound, off by default. The Sound tab suggests it from a minute on and says how far the copy has got. - StreamKeeper makes the copies in the browser, a moment after the last change, one at a time: the sound's own and one per clip of its cut, fades and Reverse baked in. A copy that is no longer wanted is taken off and deleted once the project is saved without it. - streamTable says which played names stream from which copy. A clip without a fitting copy, a Sample and a Reverb impulse stay decoded. A sound with Stream on is never decoded ahead. - The page fetches by Range and takes the whole file from a server that ignores it, as the CI's PHP server does. It keeps the bytes and the first two seconds decoded, the rest goes four seconds ahead. - A jump decodes 32 packets before the place first. Measured in Chrome, what follows is then the same to the bit as decoding from the start. With 4 packets it was 0.06 off. - The stream-player on the FX worklet reads the samples at the rate of the context along a Catmull-Rom curve. What comes late is silent and skipped once it is there, so a clip keeps its beat. - The opening stretch goes with the player as it is made. Sent by message it came a few milliseconds late, and 557 samples of the attack were lost. - Pitch on a streamed sound is a shifter after the player, as on a bus: the length stays and it is heard 42.7 ms late. - Bakes of patterns and songs (#340) always stream now. - The song timeline draws a streamed sound from its copy decoded at 8 kHz, keeping only the peaks. Tests: - Gate: CHECK EXIT 0, 280 files, 4312 tests (31 new: ogg +3, stream-queue 6, streams 11, streaming 6, sound-library +5). - e2e 34: ALL PASSED, 106 checks (7 new). Five minutes of mono, 4.8 MB as WAV, 3.6 MB as its copy: streamed the renderer grows by 5.8 MB, decoded by 69.6 MB. Offline at 48 kHz, start, loop seam and a reversed clip are the copy decoded whole to the sample (0). - Counter-checks red as they should be: no pre-roll, no catch-up of late samples at the end, no opening stretch, a streamed sound decoded ahead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
84cd36f902
commit
2764919bb8
30 files changed
+2443
-146
No files matched your search
@@ -22,6 +22,7 @@ import { isShaderNodeDef } from '../shader/nodes';
|
||||
import { isSoundFxNodeDef } from '../soundfx/nodes';
|
||||
import { INSTRUMENT_SAMPLE, isInstrumentNodeDef, withSamplePlay } from '../instrument/nodes';
|
||||
import { heldBake } from '../music/bake';
|
||||
import { heldStream } from '../music/streams';
|
||||
import { heldPatternNotes } from '../music/pattern';
|
||||
import { isEmbeddable } from '../music/embedded';
|
||||
import { heldSongItems, heldSongTracks } from '../music/song';
|
||||
@@ -263,7 +264,16 @@ export function deserializeProject(raw: unknown, fallbackId = ''): Project {
|
||||
} else if (type === 'sound') {
|
||||
const file = typeof a.file === 'string' && isSoundFile(a.file) ? a.file : '';
|
||||
const clips = normalizeClips(a.clips, takenIds);
|
||||
assets.push({ id, type, name, props, file, ...(clips.length > 0 && { clips }) });
|
||||
const stream = heldStream(a.stream);
|
||||
assets.push({
|
||||
id,
|
||||
type,
|
||||
name,
|
||||
props,
|
||||
file,
|
||||
...(clips.length > 0 && { clips }),
|
||||
...(stream && { stream }),
|
||||
});
|
||||
} else if (type === 'tile') {
|
||||
// A picture naming nothing is kept, like an animation's dangling root: it reads as missing
|
||||
// in the tab and comes back when the texture it named is restored by an undo.
|
||||
@@ -763,10 +773,12 @@ function normalizeClips(raw: unknown, takenIds: Set<string>): SoundClip[] {
|
||||
const id = typeof c.id === 'string' ? c.id : '';
|
||||
if (!id || takenIds.has(id)) continue;
|
||||
takenIds.add(id);
|
||||
const stream = heldStream(c.stream);
|
||||
clips.push({
|
||||
id,
|
||||
name: typeof c.name === 'string' ? c.name : '',
|
||||
props: normalizeBag(c.props, CLIP_ROWS, true),
|
||||
...(stream && { stream }),
|
||||
});
|
||||
}
|
||||
return clips;
|
||||
|
||||
Reference in new issue
Block a user