Load the sounds of a project ahead of their first play

A sound was fetched and decoded on its first Play sound. That play
started late, its beginning was not heard, and a graph that tuned the
voice in the meantime had moved on by the time it started. Images and
fonts were already loaded when a project opens. Sounds now are too.

- projectFiles lists the sounds of a project, and preloadProjectFiles
  hands them to the new soundLibrary.preload. That covers the editor
  and the published game.
- Preloading opens no audio device. It decodes with an
  OfflineAudioContext at 48 kHz. The real AudioContext is still made
  on the first play, and a source resamples where the device runs at
  another rate.
- A run started right after opening a project can still find a sound
  loading. It then starts late as before.

Tests:
- sound-library: a preload makes only the offline context, and a
  play after it starts at once, opening the device only then.
- sound-library + project-files: 18 tests, green.
- e2e 34 on melektaus-atx: ALL PASSED, 30 checks.
- Counter-check: decoding with the real AudioContext again fails the
  new test.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-05 21:47:00 +02:00
1 parent ded0022903
commit ed2b58b3c0
5 files changed
+72 -8

No files matched your search

+5 -2
View File
@@ -1,13 +1,15 @@
import { isFontFile } from './font';
import { celFilesOf } from './pixelart';
import type { Project } from './Project';
import { isSoundFile } from './sound';
import { isTextureFile } from './texture';
/** The stored files a project draws with, per library – what is loaded before anything asks. */
/** The stored files a project draws and plays, per library – what is loaded before anything asks. */
export interface ProjectFiles {
textures: string[];
fonts: string[];
cels: string[];
sounds: string[];
}
/**
@@ -16,11 +18,12 @@ export interface ProjectFiles {
* an upload that failed – is left out, as the libraries would never get it anyway.
*/
export function projectFiles(project: Project): ProjectFiles {
const files: ProjectFiles = { textures: [], fonts: [], cels: [] };
const files: ProjectFiles = { textures: [], fonts: [], cels: [], sounds: [] };
for (const asset of project.assets) {
if (asset.type === 'texture' && isTextureFile(asset.file)) files.textures.push(asset.file);
else if (asset.type === 'font' && isFontFile(asset.file)) files.fonts.push(asset.file);
else if (asset.type === 'pixelart') files.cels.push(...celFilesOf(asset));
else if (asset.type === 'sound' && isSoundFile(asset.file)) files.sounds.push(asset.file);
}
return files;
}
+20 -2
View File
@@ -14,6 +14,8 @@ const DEFAULT_BASE = '/game-maker/data/sounds';
const SHIFTED_KEPT = 32;
/** Seconds a changed parameter takes to get most of the way there: quick, but no click (#315). */
const GLIDE = 0.03;
/** The rate a sound loaded ahead of its first play is decoded at; a source resamples it on the way out. */
const DECODE_RATE = 48000;
/**
* One number of a voice that can still change while it plays (#315): a level, a share of a Mix, the
@@ -68,11 +70,13 @@ interface Pending {
*
* The context is created on the first play, not at start-up: a browser refuses one before the page
* has been interacted with, and an editor that was only ever looked at should not be holding an
* audio device open.
* audio device open. What `preload` fetches before that is decoded by an offline context, which
* holds no device.
*/
export class SoundLibrary implements AudioSink {
private base = DEFAULT_BASE;
private context: AudioContext | null = null;
private offline: OfflineAudioContext | null = null;
private readonly buffers = new Map<string, AudioBuffer>();
/** Pitched copies of decoded sounds, oldest use first (see `pitched`). */
private readonly shifted = new Map<string, AudioBuffer>();
@@ -212,6 +216,14 @@ export class SoundLibrary implements AudioSink {
return this.buffers.get(file)?.duration ?? null;
}
/**
* Starts loading every file not held yet. Waiting for the first play left that play late: its
* start was not heard, and a graph tuning it in the meantime had moved on by the time it started.
*/
preload(files: Iterable<string>): void {
for (const file of files) if (file) void this.load(file);
}
/** Fetches and decodes once per file; a second play while it is in flight waits on the first. */
async load(file: string): Promise<AudioBuffer | null> {
const held = this.buffers.get(file);
@@ -224,7 +236,7 @@ export class SoundLibrary implements AudioSink {
const response = await fetch(`${this.base}/${file}`);
if (!response.ok) throw new Error(`HTTP ${response.status}`);
const bytes = await response.arrayBuffer();
const buffer = await this.audio().decodeAudioData(bytes);
const buffer = await this.decoder().decodeAudioData(bytes);
this.buffers.set(file, buffer);
return buffer;
} catch (error: unknown) {
@@ -371,6 +383,12 @@ export class SoundLibrary implements AudioSink {
return (this.context ??= new AudioContext());
}
/** The context that is already playing, or else one that decodes without opening a device. */
private decoder(): BaseAudioContext {
if (this.context || typeof OfflineAudioContext === 'undefined') return this.audio();
return (this.offline ??= new OfflineAudioContext(1, 1, DECODE_RATE));
}
/** Everything that would have silenced this file for this owner, counted rather than flagged. */
private cancelsFor(file: string, owner: string): number {
return (this.stops.get(file) ?? 0) + this.allStops + (this.ownerStops.get(owner) ?? 0);
+1
View File
@@ -46,4 +46,5 @@ export function preloadProjectFiles(project: Project): void {
textureLibrary.preload(files.textures);
fontLibrary.preload(files.fonts);
pixelCels.preload(files.cels);
soundLibrary.preload(files.sounds);
}