Decode a clip from a file of its own, not its whole sound (#349)
A clip was cut from its sound's file, so a game decoded the whole file for it and held it for as long as the page was open. A long recording played through a few short clips cost the recording. Every clip now has a copy of its cut as Opus, with Stream on or off, and a game decodes that instead. - StreamKeeper makes the copy of each clip whether its sound streams or not. Stream only decides whether a sound gets a copy of its own. - clipCopyTable says which cut decodes from which copy. The sound library loads the copy for a clip unless the whole file is decoded already. Then the clip is cut from that as before. - projectFiles decodes the whole file ahead only where something may play the sound itself: no clips, a clip without a fitting copy, or the sound's id anywhere in an asset that can name one. That last test walks the data, not the fields, so no place is missed. - A copy that cannot be had falls back to the whole file. A library that bakes keeps to the whole files, as the bake's fingerprint does. Tests: - Gate: CHECK EXIT 0, 280 files, 4319 tests (7 new: streams +4, sound-library +3). - e2e 34: ALL PASSED, 108 checks (2 new). With Stream off a fresh library decodes the ten-second reversed clip's copy and not the five-minute file. It is the cut made from the original to within 0.0091 a sample and 0.013 dB in any 20 ms. Once a node plays the sound itself, the whole file is decoded ahead again. - e2e 65: all passed with this change. - Counter-checks red as they should be: the library ignoring copies, the whole file always decoded ahead, clip copies only with Stream. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
2764919bb8
commit
82e96e680f
10 files changed
+332
-42
No files matched your search
@@ -1,7 +1,8 @@
|
||||
import { findAssetOf } from '../../core/model/Project';
|
||||
import { clipsOf, findSound } from '../../core/model/sound';
|
||||
import { projectFiles } from '../../core/model/projectFiles';
|
||||
import { clipsOf, cutChannels, cutRange, findSound, parseCutName } from '../../core/model/sound';
|
||||
import { STREAM_RATE } from '../../core/music/streamQueue';
|
||||
import { playedPlan, streamTable } from '../../core/music/streams';
|
||||
import { clipCopyTable, playedPlan, streamTable } from '../../core/music/streams';
|
||||
import { playedSoundOf } from '../../core/properties/playedSound';
|
||||
import { soundLibrary } from '../../viewport/SoundLibrary';
|
||||
import { centreOf, type HookContext } from './hookHelpers';
|
||||
@@ -121,6 +122,49 @@ export function soundHooks({ state, editor }: HookContext) {
|
||||
skipped: played.skipped,
|
||||
};
|
||||
},
|
||||
/**
|
||||
* What a game decodes that has just opened the project, in a library of its own so nothing this
|
||||
* page holds counts: the names it decodes ahead, whether that decoded the whole file of a name's
|
||||
* sound, and what the name then plays held against its cut made from that file – samples, rate,
|
||||
* the loudest difference of a sample and of the level of any 20 ms.
|
||||
*/
|
||||
fresh: async (name: string) => {
|
||||
const library = soundLibrary.aside();
|
||||
library.setStreams(streamTable(state.project));
|
||||
library.setCopies(clipCopyTable(state.project));
|
||||
const ahead = projectFiles(state.project).sounds;
|
||||
await Promise.all(ahead.map((file) => library.load(file)));
|
||||
const { file, cut } = parseCutName(name);
|
||||
const whole = library.decoded(file) !== null;
|
||||
const played = await library.load(name);
|
||||
const original = await soundLibrary.decodeApart(file, STREAM_RATE);
|
||||
if (!played || !cut || played.sampleRate !== STREAM_RATE) return { ahead, whole, played: null };
|
||||
const [reference] = cutChannels(
|
||||
[original.getChannelData(0)],
|
||||
cutRange(original.length, STREAM_RATE, cut),
|
||||
);
|
||||
const heard = played.getChannelData(0);
|
||||
let worst = 0;
|
||||
let level = 0;
|
||||
const span = Math.round(0.02 * STREAM_RATE);
|
||||
for (let from = 0; from + span <= heard.length; from += span) {
|
||||
let own = 0;
|
||||
let wanted = 0;
|
||||
for (let i = from; i < from + span; i++) {
|
||||
worst = Math.max(worst, Math.abs(heard[i]! - (reference![i] ?? 0)));
|
||||
own += heard[i]! ** 2;
|
||||
wanted += (reference![i] ?? 0) ** 2;
|
||||
}
|
||||
// Only where it is heard: the fades start from silence.
|
||||
if (wanted / span > 1e-6) level = Math.max(level, Math.abs(10 * Math.log10(own / wanted)));
|
||||
}
|
||||
return {
|
||||
ahead,
|
||||
whole,
|
||||
played: { length: played.length, wanted: reference!.length, worst, level },
|
||||
decoded: library.stats().decoded,
|
||||
};
|
||||
},
|
||||
/** What a sound and each of its clips keep of their copies. */
|
||||
copies: (id: string) => {
|
||||
const asset = findAssetOf(state.project, id, 'sound');
|
||||
|
||||
Reference in new issue
Block a user