Describe clips decoded from a file of their own (#349)

- Sounds: every clip has an Opus copy of its cut, with Stream on or
  off. What is still decoded whole, and the measured difference.
- Architecture: streams and StreamKeeper cover every clip.
- Development: 1937 e2e checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
adminandClaude Opus 5.5 committed 2026-10-09 20:53:53 +02:00
1 parent 5515c67008
commit 8a4dde0e04
3 files changed
+30 -9

No files matched your search

+3 -3
@@ -174,8 +174,8 @@ src/core/music/ time (ticks, 480 to the quarter, tempo and meter, a posit
bake (#340: what a bake is made from as a print, where the parts of its file lie, how they
are put together from the playing looped and the one played once, and played back) +
ogg (#340: Opus in Ogg, the pages around an encoder's packets and back, also read as
the bytes come in, #349), streams (#349: which names stream from which copy, what
copies are missing, how a played sound walks its copy) + streamQueue (#349: the
the bytes come in, #349), streams (#349: which names stream from which copy, which
clips decode from their own, what copies are missing, how a played sound walks its copy) + streamQueue (#349: the
samples a streamed sound plays from on the worklet, read at the rate of the context)
src/core/soundfx/ the sound FX graph (#311): nodes (every effect and what it adds to a plan), compile (the
graph walked back from the Output to a plan; a parameter stays open until a voice
@@ -412,7 +412,7 @@ src/editor/song/ SongEditor (header: Play, Grid, Bake; plays through the s
src/editor/sound/ SoundEditor (#343: header with Play, the clip's direction and fades), WaveCanvas (the
waveform, the clips as bands, the cut) + waveView (its time axis), SoundPreview (play
and length in the inspector), SoundPicker (the "+ → Sound" rows with their play mark),
StreamKeeper (#349: the copies streamed sounds and their clips play from, made in the
StreamKeeper (#349: the copies streamed sounds and every clip play from, made in the
browser and let go once no longer wanted)
src/editor/soundfx/ SoundFxEditor (#311: the graph, a sound to try it on, Play and Stop), PreviewParams (the
header's fields for the parameters a tab plays with, shared with the Instrument tab),
+1 -1
@@ -19,7 +19,7 @@ php tests/php/accounts.test.php # the API's accoun
php tests/php/snapshots.test.php # the snapshot store (#231), the same way
php tests/php/releases.test.php # the release store (#266), the same way
php tests/php/files.test.php # uploaded files in a folder per account (#344), the same way
node tools/seed-e2e.mjs && node tools/e2e.mjs # 1935 editor checks, screenshots in /tmp/gm-e2e-*.png
node tools/seed-e2e.mjs && node tools/e2e.mjs # 1937 editor checks, screenshots in /tmp/gm-e2e-*.png
GM_STEPS=20 node tools/e2e.mjs # or 1-12, or 6,20 — a subset while working on one step
./e2e.sh start [GM_STEPS] # the same run detached, see below
./e2e.sh status # live progress, or the last run's result
+26 -5
@@ -127,6 +127,23 @@ library lets the oldest go once they hold more than 64 Mi samples together — a
in its tab leaves no pile of copies behind. The sound itself, and a clip over the whole file, is no
cut at all and plays the file as it is.
**Each clip has a file of its own** (#349). A moment after a clip was made or last changed, the
editor cuts it from the file, fades and Reverse worked in, encodes that as Opus in Ogg (as a
[streamed](#streaming-long-sounds) sound is) and uploads it as *[stream] Sound - Clip*. The sound
picker leaves it out. This happens for every clip of every sound, with Stream on or off.
- **A game plays the clip from that file.** A sound that is only played through its clips is never
decoded whole: a long recording with a few short clips in it costs the clips, not the recording.
- **The whole file is still decoded** where something may play the sound itself: a sound field,
a song item or the Sample of an instrument that names the sound and not one of its clips, or a
clip whose own file is not there yet. Then the clips are cut from the whole file as before.
- Something that plays the sound itself only while the game runs, by an id the project does not
hold, starts late once: the whole file is fetched and decoded then.
- **The copy is lossy** and at 48 kHz, as Opus always is: 160 kbit/s in stereo, 96 kbit/s in mono.
The Sound tab plays and draws the original.
- A changed clip plays from the whole file until its new copy is there; the old one is deleted
once the project is saved without it.
**Song items** keep their place and their length when their clip is cut anew: they play from the
new start, and one longer than the new stretch plays it round and round.
@@ -205,7 +222,8 @@ them at once.
**It is loaded ahead.** Every sound of the project is fetched and decoded as soon as the project is
open, the way images and fonts are, so the first play starts in the frame its node ran. A sound
that [streams](#streaming-long-sounds) has only the start of its copy fetched. Decoding
that [streams](#streaming-long-sounds) has only the start of its copy fetched, and one only played
through its clips has [the clips' own files](#clips) decoded instead of its own. Decoding
ahead opens no audio device: that only happens on the first play. A sound that is still loading
when it is played — a run started the moment the project opened — starts a moment late instead.
`Play sound` never waits on the network, and a `Stop sound` that happens while the bytes are still
@@ -245,9 +263,9 @@ as the page is open. **Stream** (#349) plays such a sound from a copy, a few sec
in stereo, 96 kbit/s in mono, as a [bake](Songs#baking) is) and uploads it as *[stream] name*.
The sound picker leaves it out. The original stays as it is: the Sound tab shows and cuts that one.
Switched off, the copy is taken off the sound and deleted once the project is saved without it.
- **Each clip gets a copy of its own cut**, its fades and Reverse in it, so a clip never needs the
whole file. A changed clip gets a new copy a moment after the last change. A clip that is the whole
file plays the sound's copy.
- **Each clip streams from the copy of its own cut** that every clip has
([with Stream or without](#clips)), so a clip never needs the whole file. A clip that is the
whole file plays the sound's copy.
- **Until a copy is there**, the sound or clip plays decoded, as without Stream. A sound with Stream
on is never decoded ahead when the project opens, though: only once it is played without a copy.
- **What a game holds** is the compressed copy as it is fetched, and its first two seconds decoded,
@@ -273,7 +291,10 @@ Measured (2026-10-09, e2e 34 on the CI's Windows machine): five minutes of a sin
with Stream off the same sound decoded grows it by 69.6 MB. Streamed offline at 48 kHz, every
sample is the one the copy decoded whole has there, by the feed's decoder and by the browser's
`decodeAudioData` alike: from a start at 0.25 s two seconds in, round the seam of its loop, and for
a reversed clip of it round its loop (largest difference 0, no sample late).
a reversed clip of it round its loop (largest difference 0, no sample late). With Stream off, a
game that plays the sound only through that clip decodes the clip's own copy and not the file: its
ten seconds are the cut made from the original to within 0.0091 a sample, and to 0.013 dB in any
20 ms.
## Buses and the master