adminandClaude Opus 5.5
fb4d65a2ab
A Mix crossfaded: at 0.5 each sound was at half. Layering two sounds in an instrument now goes through a Mix (076c564), so the user asked for a second curve where each side stays full up to the middle and only fades on the other half. At 0.25 a is full and b half, at 0.5 both are full, at 0.75 a is half and b full. - The Mix has a setting Curve with Crossfade (default) and Both at center. A plan and a chain only say `curve` where it is `both`, so old projects and their compiled plans stay as they were. - A share is not a straight line of `mix` with that curve. Two ramps of their own would dip to half each when `mix` glides across the middle. So a ConstantSourceNode carries `mix` on the audio thread, and two WaveShaperNodes make the shares from it. The knob of such a Mix is that one source. - The source is started with the voice and stopped when it is cut or ends by itself, so it does not run on after the voice. - A sound FX node in an instrument keeps the curve of a Mix inside it. - At 0.5 both sounds are added in full, up to +6 dB where both are loud. That is what several edges into the Output did before. The wiki says so. Found on the way: the suite of23bdc3f(task 529) failed the Step check in e2e 34. It timed the jump from the start of the voice, but a timed change runs from when it arrives. In CI that was 80 ms later, and the pitch rightly held to 2.08 s. - stats() says `tunedAt`, the second of the audio clock the last change moved a voice. A change that moves nothing, as On update sends every frame, leaves it. - The Step check times its readings from there. Tests: - sound-library.test.ts: both shares shaped from one source, a glide moves only the source, the source stops with a cut and with an end by itself (50 tests, 2 new) - soundfx.test.ts: curve only where set, one sound at 0 (41 tests) - instrument.test.ts: the curve through a sound FX node (34 tests) - Counter-checks: dropping the curve in throughFx and not stopping the source in done() each turn their new test red - e2e 34 on melektaus-atx: ALL PASSED, 48 checks (+1). Rendered offline in Chrome, a and b come to 1/0, 1/0.5, 1/1, 0.5/1 and 0/1 at mix 0 to 1, and a glide from 0 to 1 reads the same at 0.25, 0.5 and 0.75 s. Step jumped 2.04 s after the change. - Gate on melektaus-atx: CHECK EXIT 0, 244 test files, 3793 tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1100 lines
51 KiB
JavaScript
1100 lines
51 KiB
JavaScript
import { edge, node, port, variable, written } from '../lib/builders.mjs';
|
||
import { LANES, MAKE_OBJECT, pickAssetType } from './lib/scaffold.mjs';
|
||
|
||
/**
|
||
* Sound assets (#45): uploaded like a typeface – picked from what the server holds through
|
||
* "+ → Sound", or dropped onto the editor – and played by a graph through `Play sound`. Nothing
|
||
* wears a sound and nothing places it; the asset says how loud it is and whether it loops, and the
|
||
* node scales the one and can overrule the other in either direction.
|
||
*
|
||
* Nothing is ever *heard* here: CI runs the browser muted (see tools/lib/browser.mjs). The witness
|
||
* is `__gm.sound.stats()` – voices started, stopped and in flight – the way `renderStats` is the
|
||
* witness for a repaint that did not happen. No pixel check could see any of this.
|
||
*
|
||
* The seed puts two generated WAVs on the server without making assets of them: a 0.25 s blip and
|
||
* a 0.75 s drone, told apart by their exact size rather than by their name.
|
||
*/
|
||
const LANE = LANES.sound;
|
||
const BLIP_BYTES = 4044;
|
||
const DRONE_BYTES = 12044;
|
||
|
||
export async function sound(t) {
|
||
const { check, sleep, json, evaluate, mutate, click, clickText, press, play, stop, shot, showTab } = t;
|
||
const { select } = t;
|
||
const assetsOfType = (type) =>
|
||
json(`__gm.state.project.assets.filter((a) => a.type === ${JSON.stringify(type)})
|
||
.map((a) => ({ id: a.id, name: a.name, file: a.file, props: a.props }))`);
|
||
const stats = () => json('__gm.sound.stats()');
|
||
/** A voice starts only once its bytes have been decoded, so what it changes is awaited. */
|
||
const waitFor = async (read, done, tries = 25) => {
|
||
let value = await read();
|
||
for (let i = 0; i < tries && !done(value); i++) {
|
||
await sleep(150);
|
||
value = await read();
|
||
}
|
||
return value;
|
||
};
|
||
// Every reading up to the one `done` takes: a check holds each against its own time on the audio
|
||
// clock, however late the first one comes in.
|
||
const trace = async (read, done, tries = 40) => {
|
||
const seen = [await read()];
|
||
for (let i = 0; i < tries && !done(seen.at(-1)); i++) {
|
||
await sleep(150);
|
||
seen.push(await read());
|
||
}
|
||
return seen;
|
||
};
|
||
|
||
await press('Escape');
|
||
await evaluate('document.activeElement instanceof HTMLElement && document.activeElement.blur()');
|
||
await evaluate('__gm.state.select(null)');
|
||
await showTab('scene');
|
||
|
||
// 34a. What the server holds, straight from the API: the two files the seed put there.
|
||
const stored = await evaluate(`fetch('/game-maker/api/index.php?r=sounds').then((r) => r.json())`);
|
||
const blip = stored.find((item) => item.bytes === BLIP_BYTES);
|
||
const drone = stored.find((item) => item.bytes === DRONE_BYTES);
|
||
check(
|
||
'the server holds the two seeded sounds, named by their content',
|
||
Boolean(blip && drone) && /^[a-f0-9]{64}\.wav$/.test(blip.file),
|
||
JSON.stringify({ blip: blip?.file, drone: drone?.file, count: stored.length }),
|
||
);
|
||
if (!blip || !drone) return;
|
||
check(
|
||
'the listing carries the name each file was uploaded under',
|
||
blip.name === 'blip.wav' && drone.name === 'drone.wav',
|
||
JSON.stringify({ blip: blip.name, drone: drone.name }),
|
||
);
|
||
|
||
// 34b. "+ → Sound" offers them; a sound is picked rather than created empty, like a typeface.
|
||
await clickText('+');
|
||
await pickAssetType(t, 'Sound');
|
||
await sleep(500);
|
||
const picker = await json('__gm.soundPicker()');
|
||
check(
|
||
'the + menu opens the sound picker on what the server holds, none of it in the project yet',
|
||
Boolean(picker) && picker.files.includes(blip.file) && picker.used.length === 0,
|
||
JSON.stringify(picker),
|
||
);
|
||
check(
|
||
'a row nobody uses yet is captioned with the name the file was uploaded under',
|
||
picker.captions[picker.files.indexOf(blip.file)] === 'blip.wav',
|
||
JSON.stringify(picker.captions),
|
||
);
|
||
await shot('34-sound-picker');
|
||
|
||
// 34c. The ▶ at a row's left is a preview: it plays that file without choosing it.
|
||
const playAt = await json(`__gm.soundPickerPlay(${JSON.stringify(drone.file)})`);
|
||
await click(playAt.x, playAt.y);
|
||
const previewed = await waitFor(stats, (s) => s.started > 0);
|
||
const afterPreview = await json('__gm.soundPicker()');
|
||
check(
|
||
'pressing a row’s ▶ plays that file and picks nothing',
|
||
previewed.started === 1 &&
|
||
previewed.last?.file === drone.file &&
|
||
afterPreview.played === drone.file &&
|
||
afterPreview.selected === null,
|
||
JSON.stringify({ previewed, selected: afterPreview.selected, played: afterPreview.played }),
|
||
);
|
||
|
||
const row = await json(`__gm.soundPickerRow(${JSON.stringify(blip.file)})`);
|
||
await click(row.x, row.y);
|
||
await clickText('Use sound');
|
||
const sounds = await assetsOfType('sound');
|
||
check(
|
||
'picking a row makes a sound asset of that file, named after the file rather than "Sound"',
|
||
sounds.length === 1 && sounds[0].file === blip.file && sounds[0].name === 'blip',
|
||
JSON.stringify(sounds),
|
||
);
|
||
if (sounds.length !== 1) return;
|
||
const soundId = sounds[0].id;
|
||
check(
|
||
'a fresh sound asset is full volume and one-shot',
|
||
sounds[0].props.volume === 1 && sounds[0].props.loop === false,
|
||
JSON.stringify(sounds[0].props),
|
||
);
|
||
// Closing the picker silences its preview; nothing it started may outlive the dialog.
|
||
check(
|
||
'closing the picker leaves nothing playing',
|
||
(await stats()).playing === 0,
|
||
JSON.stringify(await stats()),
|
||
);
|
||
|
||
// 34d. Its inspector: the two rows, and the preview that is the only way to tell which sound it is.
|
||
await select({ kind: 'asset', assetId: soundId });
|
||
const volumeRow = await json("__gm.inspectorField('Volume', true)");
|
||
const loopRow = await json("__gm.inspectorField('Loop', true)");
|
||
check(
|
||
'a sound asset has a Volume row and a Loop row',
|
||
Boolean(volumeRow) && Boolean(loopRow),
|
||
JSON.stringify({ volumeRow, loopRow }),
|
||
);
|
||
await shot('34-sound-asset');
|
||
|
||
// 34e. A graph plays it. The behavior sits in its own lane; the node's volume is a factor on the
|
||
// asset's, so 0.5 × 0.4 has to reach the library as 0.2 rather than as either of the two.
|
||
await mutate(`
|
||
const p = __gm.state.project;
|
||
const scene = __gm.state.activeScene;
|
||
p.assets.find((a) => a.id === ${JSON.stringify(soundId)}).props.volume = 0.4;
|
||
const asset = structuredClone(p.assets.find((a) => a.id === 'mover'));
|
||
Object.assign(asset, {
|
||
id: 'sn-logic',
|
||
name: 'Noisy',
|
||
props: { enabled: true, runInEditor: false },
|
||
variables: [],
|
||
graph: ${JSON.stringify({
|
||
nodes: [
|
||
node('sn-start', 'OnStart', 0, 0),
|
||
node('sn-play', 'PlaySound', 260, 0, { sound: soundId, volume: 0.5, loop: '' }),
|
||
],
|
||
edges: [edge('sn-e1', 'sn-start', 'exec', 'sn-play', 'exec')],
|
||
})},
|
||
});
|
||
p.assets.unshift(asset);
|
||
${MAKE_OBJECT}
|
||
scene.objects.push(
|
||
make('sn-box', 'crate', '', ${LANE}, 'Noisy box'),
|
||
make('sn-brain', 'sn-logic', 'sn-box', 0, 'Noisy'),
|
||
);
|
||
__gm.lookAt(0, ${LANE});`);
|
||
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const before = await stats();
|
||
await play();
|
||
const played = await waitFor(stats, (s) => s.started > before.started);
|
||
check(
|
||
'a graph playing a sound reaches the library, at the asset volume scaled by the node’s',
|
||
played.started === before.started + 1 &&
|
||
played.last?.file === blip.file &&
|
||
Math.abs(played.last.volume - 0.2) < 0.001 &&
|
||
played.last.loop === false,
|
||
JSON.stringify(played),
|
||
);
|
||
await stop();
|
||
|
||
// 34f. A looping sound is music: it plays on, and ending the run ends it – nothing may outlive
|
||
// the world that started it.
|
||
await mutate(`
|
||
const p = __gm.state.project;
|
||
p.assets.find((a) => a.id === ${JSON.stringify(soundId)}).props.loop = true;`);
|
||
await play();
|
||
const looping = await waitFor(stats, (s) => s.playing > 0 && s.last?.loop === true);
|
||
check(
|
||
'the asset’s own loop flag reaches the library even though the node never asked for one',
|
||
looping.playing >= 1 && looping.last?.loop === true,
|
||
JSON.stringify(looping),
|
||
);
|
||
// Well past the blip's own 0.25 s, so a one-shot would have ended by itself here.
|
||
await sleep(700);
|
||
const stillLooping = await stats();
|
||
await stop();
|
||
const afterStop = await stats();
|
||
check(
|
||
'a loop keeps playing while the run is on and ends with it',
|
||
stillLooping.playing >= 1 && afterStop.playing === 0,
|
||
JSON.stringify({ stillLooping, afterStop }),
|
||
);
|
||
|
||
// 34f-2. The node overrules the asset in both directions – the reason `loop` is a choice of three
|
||
// and not a checkbox: music has to be playable as a one-shot.
|
||
const setNodeLoop = (value) =>
|
||
mutate(`
|
||
const p = __gm.state.project;
|
||
const graph = p.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.find((n) => n.id === 'sn-play').params.loop = ${JSON.stringify(value)};`);
|
||
await setNodeLoop('false');
|
||
const beforeOnce = await stats();
|
||
await play();
|
||
const once = await waitFor(stats, (s) => s.started > beforeOnce.started);
|
||
check('a node saying "off" plays a looping asset once', once.last?.loop === false, JSON.stringify(once));
|
||
await stop();
|
||
await setNodeLoop('');
|
||
|
||
// 34g. `Stop sound` silences it from inside the graph, without ending the run.
|
||
await mutate(`
|
||
const p = __gm.state.project;
|
||
const graph = p.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.push(${JSON.stringify(node('sn-stop', 'StopSound', 520, 0, { sound: soundId }))});
|
||
graph.nodes.push(${JSON.stringify(node('sn-key', 'OnKeyDown', 260, 220, { key: 's' }))});
|
||
graph.edges.push(${JSON.stringify(edge('sn-e2', 'sn-key', 'exec', 'sn-stop', 'exec'))});`);
|
||
await play();
|
||
await waitFor(stats, (s) => s.playing > 0);
|
||
await press('s');
|
||
const silenced = await waitFor(stats, (s) => s.playing === 0);
|
||
check(
|
||
'Stop sound silences a looping sound while the run carries on',
|
||
silenced.playing === 0 && (await json('__gm.playInfo()')).playing === true,
|
||
JSON.stringify(silenced),
|
||
);
|
||
await stop();
|
||
|
||
// 34g-2. A voice belongs to the behavior playing it: destroy the object and its sounds go with it,
|
||
// without any `Stop sound` in the graph. `detached` is the one way out, for the death
|
||
// sound of an object that destroys itself.
|
||
await mutate(`
|
||
const p = __gm.state.project;
|
||
const graph = p.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.push(
|
||
${JSON.stringify(node('sn-key2', 'OnKeyDown', 260, 440, { key: 'd' }))},
|
||
${JSON.stringify(node('sn-owner', 'Owner', 260, 620))},
|
||
${JSON.stringify(node('sn-destroy', 'Destroy', 520, 440))},
|
||
);
|
||
graph.edges.push(
|
||
${JSON.stringify(edge('sn-e3', 'sn-key2', 'exec', 'sn-destroy', 'exec'))},
|
||
${JSON.stringify(edge('sn-e4', 'sn-owner', 'object', 'sn-destroy', 'object'))},
|
||
);`);
|
||
await play();
|
||
await waitFor(stats, (s) => s.playing > 0);
|
||
await press('d');
|
||
const destroyed = await waitFor(stats, (s) => s.playing === 0);
|
||
check(
|
||
'destroying the object silences the looping sound its behavior was playing',
|
||
destroyed.playing === 0 && (await json('__gm.playInfo()')).playing === true,
|
||
JSON.stringify(destroyed),
|
||
);
|
||
await stop();
|
||
|
||
const setDetached = (value) =>
|
||
mutate(`
|
||
const p = __gm.state.project;
|
||
const graph = p.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.find((n) => n.id === 'sn-play').params.detached = ${JSON.stringify(value)};`);
|
||
await setDetached(true);
|
||
await play();
|
||
await waitFor(stats, (s) => s.playing > 0);
|
||
await press('d');
|
||
// Long enough that the reaping frame has been and gone – a voice still here now is here on purpose.
|
||
await sleep(600);
|
||
const detached = await stats();
|
||
check(
|
||
'a detached voice is nobody’s and plays on after the object that started it is gone',
|
||
detached.playing >= 1,
|
||
JSON.stringify(detached),
|
||
);
|
||
await stop();
|
||
check(
|
||
'ending the run silences a detached voice too',
|
||
(await stats()).playing === 0,
|
||
JSON.stringify(await stats()),
|
||
);
|
||
await setDetached(false);
|
||
|
||
// 34g-3. A sound FX asset (#311): "+ → Sound FX" opens its tab, which says what keeps the graph
|
||
// from compiling; built into Input → Pitch → Gain → Output, the header plays the sound
|
||
// through it, and a `Play sound` naming it sends the game's voice the same way. The pitch
|
||
// comes from a parameter `depth` of the Input: the header has a field for it, and a
|
||
// `Sound FX parameters` node hands the game's value in.
|
||
await showTab('scene');
|
||
await clickText('+');
|
||
await pickAssetType(t, 'Sound FX');
|
||
const fxCreated = await json(`{
|
||
tab: __gm.tab(),
|
||
current: __gm.soundfx.current(),
|
||
info: __gm.soundfx.info(),
|
||
preview: __gm.soundfx.preview(),
|
||
assets: __gm.state.project.assets.filter((a) => a.type === 'soundfx').map((a) => a.id),
|
||
}`);
|
||
check(
|
||
'the + menu creates a sound FX asset and opens its tab, with no Output to compile and no Play',
|
||
fxCreated.tab === 'soundfx' &&
|
||
fxCreated.assets.length === 1 &&
|
||
fxCreated.current === fxCreated.assets[0] &&
|
||
fxCreated.info === 'No Output node' &&
|
||
fxCreated.preview.sound === soundId &&
|
||
fxCreated.preview.play === false,
|
||
JSON.stringify(fxCreated),
|
||
);
|
||
const fxId = fxCreated.assets[0];
|
||
const fxCanvas = await json('__gm.soundfx.canvasRect()');
|
||
await click(fxCanvas.x + 80, fxCanvas.y + 60, 'right');
|
||
const fxPalette = await json('__gm.overlayTexts()');
|
||
check(
|
||
'its palette offers the sound effects and no behavior node',
|
||
['Input', 'Output', 'Gain', 'Pitch', '5-Band-EQ', 'Mix', 'Delay'].every((label) =>
|
||
fxPalette.includes(label),
|
||
) && !fxPalette.includes('Branch'),
|
||
JSON.stringify(fxPalette.slice(0, 12)),
|
||
);
|
||
await press('Escape');
|
||
await mutate(`
|
||
const fx = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)});
|
||
fx.name = 'Low';
|
||
fx.graph = ${JSON.stringify({
|
||
nodes: [
|
||
node('fx-in', 'SoundFxInput', 0, 0, {}, [{ key: 'depth', kind: 'number', default: -12 }]),
|
||
node('fx-pitch', 'SoundFxPitch', 220, 0),
|
||
node('fx-gain', 'SoundFxGain', 440, 0, { gain: -6 }),
|
||
node('fx-out', 'SoundFxOutput', 660, 0),
|
||
],
|
||
edges: [
|
||
edge('fx-e1', 'fx-in', 'out', 'fx-pitch', 'in'),
|
||
edge('fx-e2', 'fx-pitch', 'out', 'fx-gain', 'in'),
|
||
edge('fx-e3', 'fx-gain', 'out', 'fx-out', 'in'),
|
||
edge('fx-e4', 'fx-in', 'depth', 'fx-pitch', 'semitones'),
|
||
],
|
||
})};`);
|
||
const fxBuilt = await json(
|
||
'{ info: __gm.soundfx.info(), preview: __gm.soundfx.preview(), params: __gm.soundfx.params() }',
|
||
);
|
||
check(
|
||
'Input → Pitch → Gain → Output compiles to two effects, Play is offered, depth starts at its default',
|
||
fxBuilt.info === '2 effects' && fxBuilt.preview.play === true && fxBuilt.params.depth === -12,
|
||
JSON.stringify(fxBuilt),
|
||
);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeFx = await stats();
|
||
const fxPlayAt = await json('__gm.soundfx.playAt()');
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
const heardFx = await waitFor(stats, (s) => s.started > beforeFx.started);
|
||
const blipSeconds = await json(`__gm.sound.duration(${JSON.stringify(blip.file)})`);
|
||
check(
|
||
'the tab’s Play sends the sound through the graph: an octave down, as long as before, one gain stage',
|
||
heardFx.last?.file === blip.file &&
|
||
heardFx.last.pitches.join() === '-12' &&
|
||
heardFx.last.duration === blipSeconds &&
|
||
heardFx.last.stages === 1,
|
||
JSON.stringify({ ...heardFx, blipSeconds }),
|
||
);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
await evaluate('__gm.soundfx.setParam("depth", 5)');
|
||
const beforeParam = await stats();
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
const heardParam = await waitFor(stats, (s) => s.started > beforeParam.started);
|
||
check(
|
||
'the header plays with the value its depth field is set to',
|
||
heardParam.last?.pitches.join() === '5' && heardParam.last.stages === 1,
|
||
JSON.stringify(heardParam),
|
||
);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
// #316: the Pitch → Gain way and the dry Input meet in a Mix – two sources, one per pitch.
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)}).graph;
|
||
graph.nodes.push(${JSON.stringify(node('fx-mix', 'SoundFxMix', 550, 0, { mix: 0.5 }))});
|
||
graph.edges = graph.edges.filter((e) => e.id !== 'fx-e3');
|
||
graph.edges.push(
|
||
${JSON.stringify(edge('fx-e5', 'fx-gain', 'out', 'fx-mix', 'a'))},
|
||
${JSON.stringify(edge('fx-e6', 'fx-in', 'out', 'fx-mix', 'b'))},
|
||
${JSON.stringify(edge('fx-e7', 'fx-mix', 'out', 'fx-out', 'in'))},
|
||
);`);
|
||
const mixInfo = await json('__gm.soundfx.info()');
|
||
const beforeMix = await stats();
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
const heardMix = await waitFor(stats, (s) => s.started > beforeMix.started);
|
||
check(
|
||
'a Mix of the pitched way and the dry Input plays one voice from two sources of the same length',
|
||
mixInfo === '3 effects' &&
|
||
heardMix.last?.pitches.join() === '5,0' &&
|
||
heardMix.last.stages === 1 &&
|
||
heardMix.last.duration === blipSeconds,
|
||
JSON.stringify({ mixInfo, ...heardMix }),
|
||
);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)}).graph;
|
||
graph.nodes = graph.nodes.filter((n) => n.id !== 'fx-mix');
|
||
graph.edges = graph.edges.filter((e) => !['fx-e5', 'fx-e6', 'fx-e7'].includes(e.id));
|
||
graph.edges.push(${JSON.stringify(edge('fx-e3', 'fx-gain', 'out', 'fx-out', 'in'))});`);
|
||
// #324, #327: a ping-pong Delay behind the gain. The blip is a quarter of a second, its echoes come
|
||
// 0.3 s apart, each half as loud as the one before: nine are heard, 2.7 s – the voice
|
||
// stays that long after its sound and goes then. Asked to let ring, only the sound
|
||
// stops and the voice stays; a stop after that cuts it.
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)}).graph;
|
||
graph.nodes.push(${JSON.stringify(
|
||
node('fx-delay', 'SoundFxDelay', 550, 0, { time: 0.3, feedback: 0.5, mix: 0.5, pingPong: true }),
|
||
)});
|
||
graph.edges = graph.edges.filter((e) => e.id !== 'fx-e3');
|
||
graph.edges.push(
|
||
${JSON.stringify(edge('fx-e8', 'fx-gain', 'out', 'fx-delay', 'in'))},
|
||
${JSON.stringify(edge('fx-e9', 'fx-delay', 'out', 'fx-out', 'in'))},
|
||
);`);
|
||
const delayInfo = await json('__gm.soundfx.info()');
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeEcho = await stats();
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
const echoStarted = await waitFor(stats, (s) => s.started > beforeEcho.started);
|
||
// Timed on the audio clock: the page's can fall behind it (task 523 saw a 2.95 s tail end 1 s in).
|
||
const audioNow = () => json('({ playing: __gm.sound.stats().playing, clock: __gm.sound.clock() })');
|
||
const startedAt = (await audioNow()).clock;
|
||
const echoLevels = await json(`__gm.sound.levels(${JSON.stringify(blip.file)})`);
|
||
const ringing = await waitFor(audioNow, (s) => s.clock - startedAt >= 1);
|
||
const quiet = await waitFor(audioNow, (s) => s.playing === 0, 40);
|
||
const rangFor = quiet.clock - startedAt;
|
||
const echoKnobs = echoLevels?.[0]?.slice(2) ?? [];
|
||
check(
|
||
'a Delay keeps the voice after its sound has ended, until its echoes have faded',
|
||
delayInfo === '3 effects' &&
|
||
echoStarted.last?.stages === 2 &&
|
||
echoKnobs.length === 4 &&
|
||
Math.abs(echoKnobs[0] - 0.3) < 1e-6 &&
|
||
Math.abs(echoKnobs[1] - 0.5) < 1e-6 &&
|
||
Math.abs(echoKnobs[2] / (20000 * 0.025 ** 0.3) - 1) < 1e-3 &&
|
||
Math.abs(echoKnobs[3] - 0.5) < 1e-6 &&
|
||
ringing.playing === 1 &&
|
||
ringing.clock - startedAt < 2.4 &&
|
||
quiet.playing === 0 &&
|
||
rangFor > 2.4 &&
|
||
rangFor < 4.5,
|
||
JSON.stringify({ delayInfo, last: echoStarted.last, echoLevels, ringing, startedAt, rangFor }),
|
||
);
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
await waitFor(stats, (s) => s.playing === 1);
|
||
const beforeLetRing = await stats();
|
||
await evaluate(`__gm.sound.stop(${JSON.stringify(blip.file)}, true)`);
|
||
await sleep(500);
|
||
const letRing = await stats();
|
||
await evaluate(`__gm.sound.stop(${JSON.stringify(blip.file)})`);
|
||
const cut = await stats();
|
||
check(
|
||
'Let ring stops the sound and keeps the voice for its echoes; a stop after that cuts it',
|
||
letRing.playing === 1 &&
|
||
letRing.stopped === beforeLetRing.stopped + 1 &&
|
||
cut.playing === 0 &&
|
||
cut.stopped === beforeLetRing.stopped + 2,
|
||
JSON.stringify({ before: beforeLetRing.stopped, letRing, cut }),
|
||
);
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)}).graph;
|
||
graph.nodes = graph.nodes.filter((n) => n.id !== 'fx-delay');
|
||
graph.edges = graph.edges.filter((e) => !['fx-e8', 'fx-e9'].includes(e.id));
|
||
graph.edges.push(${JSON.stringify(edge('fx-e3', 'fx-gain', 'out', 'fx-out', 'in'))});`);
|
||
await showTab('scene');
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
Object.assign(graph.nodes.find((n) => n.id === 'sn-play').params, { loop: 'false', fx: ${JSON.stringify(fxId)} });`);
|
||
const beforeGameFx = await stats();
|
||
await play();
|
||
const gameFx = await waitFor(stats, (s) => s.started > beforeGameFx.started);
|
||
check(
|
||
'a Play sound naming the asset plays the game’s voice through it',
|
||
gameFx.last?.pitches.join() === '-12' && gameFx.last.stages === 1,
|
||
JSON.stringify(gameFx),
|
||
);
|
||
await stop();
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.push(${JSON.stringify(
|
||
node('sn-fxp', 'SoundFxParameters', 0, 120, { soundFxAsset: fxId, depth: 7 }, [
|
||
{ key: 'depth', kind: 'number' },
|
||
]),
|
||
)});
|
||
graph.edges.push(${JSON.stringify(edge('sn-e-fxp', 'sn-fxp', 'params', 'sn-play', 'fxParams'))});`);
|
||
const beforeGameParam = await stats();
|
||
await play();
|
||
const gameParam = await waitFor(stats, (s) => s.started > beforeGameParam.started);
|
||
check(
|
||
'a Sound FX parameters node hands depth 7 to the game’s voice',
|
||
gameParam.last?.pitches.join() === '7' && gameParam.last.stages === 1,
|
||
JSON.stringify(gameParam),
|
||
);
|
||
await stop();
|
||
// #315: two looping voices of the same sound start at depth 7, the first keeps its number in a
|
||
// variable, and `Set sound FX parameters` hands that one voice depth 3 every frame. In a
|
||
// real AudioContext the first one's pitch – the AudioParam of its worklet (#317), in
|
||
// semitones – glides there, the second one's stays put.
|
||
const tuneIds = ['sn-keep', 'sn-play2', 'sn-tick', 'sn-read', 'sn-tune', 'sn-fxp2'];
|
||
const tuneEdges = [
|
||
'sn-e-keep',
|
||
'sn-e-voice',
|
||
'sn-e-play2',
|
||
'sn-e-fxp3',
|
||
'sn-e-tick',
|
||
'sn-e-read',
|
||
'sn-e-fxp2',
|
||
];
|
||
await mutate(`
|
||
const logic = __gm.state.project.assets.find((a) => a.id === 'sn-logic');
|
||
logic.variables = [${JSON.stringify(variable('voice', 'number', 0))}];
|
||
const graph = logic.graph;
|
||
const play = graph.nodes.find((n) => n.id === 'sn-play');
|
||
play.params.loop = 'true';
|
||
graph.nodes.push(
|
||
${JSON.stringify(node('sn-keep', 'Variables', 520, 0, {}, [written('voice', 'number')]))},
|
||
{ ...structuredClone(play), id: 'sn-play2', x: 780 },
|
||
${JSON.stringify(node('sn-tick', 'OnUpdate', 0, 300))},
|
||
${JSON.stringify(node('sn-read', 'Variables', 0, 420, {}, [port('voice', 'number')]))},
|
||
${JSON.stringify(node('sn-tune', 'SetSoundFxParameters', 260, 300))},
|
||
${JSON.stringify(
|
||
node('sn-fxp2', 'SoundFxParameters', 0, 540, { soundFxAsset: fxId, depth: 3 }, [
|
||
{ key: 'depth', kind: 'number' },
|
||
]),
|
||
)},
|
||
);
|
||
graph.edges.push(
|
||
${JSON.stringify(edge('sn-e-keep', 'sn-play', 'exec', 'sn-keep', 'exec'))},
|
||
${JSON.stringify(edge('sn-e-voice', 'sn-play', 'voice', 'sn-keep', 'voice'))},
|
||
${JSON.stringify(edge('sn-e-play2', 'sn-keep', 'exec', 'sn-play2', 'exec'))},
|
||
${JSON.stringify(edge('sn-e-fxp3', 'sn-fxp', 'params', 'sn-play2', 'fxParams'))},
|
||
${JSON.stringify(edge('sn-e-tick', 'sn-tick', 'exec', 'sn-tune', 'exec'))},
|
||
${JSON.stringify(edge('sn-e-read', 'sn-read', 'voice', 'sn-tune', 'voice'))},
|
||
${JSON.stringify(edge('sn-e-fxp2', 'sn-fxp2', 'params', 'sn-tune', 'fxParams'))},
|
||
);`);
|
||
const beforeTune = await stats();
|
||
await play();
|
||
const tunedStats = await waitFor(
|
||
stats,
|
||
(s) => s.tuned > beforeTune.tuned && s.started >= beforeTune.started + 2,
|
||
);
|
||
const glided = (levels) =>
|
||
Array.isArray(levels) && levels.length === 2 && Math.abs(levels[0][0] - 3) < 0.05;
|
||
const levels = await waitFor(() => json(`__gm.sound.levels(${JSON.stringify(blip.file)})`), glided);
|
||
check(
|
||
'Set sound FX parameters moves the one voice whose number it holds from depth 7 to 3, not the other',
|
||
tunedStats.last?.pitches.join() === '7' &&
|
||
tunedStats.last.loop === true &&
|
||
tunedStats.worklet === true &&
|
||
glided(levels) &&
|
||
Math.abs(levels[0][1] - 10 ** (-6 / 20)) < 0.001 &&
|
||
levels[1][0] === 7,
|
||
JSON.stringify({ last: tunedStats.last, tuned: tunedStats.tuned, worklet: tunedStats.worklet, levels }),
|
||
);
|
||
await stop();
|
||
// A `time` of 2 s takes the same voice from depth 7 to -5 evenly on the audio clock, 6 semitones
|
||
// a second: half a second in it is still well above the end, where a quick glide would be there.
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.find((n) => n.id === 'sn-tune').params.time = 2;
|
||
graph.nodes.find((n) => n.id === 'sn-fxp2').params.depth = -5;`);
|
||
await play();
|
||
const pitch = async () => (await json(`__gm.sound.levels(${JSON.stringify(blip.file)})`))?.[0]?.[0];
|
||
const early = await waitFor(pitch, (s) => typeof s === 'number' && s < 6.9);
|
||
const later = await waitFor(pitch, (s) => typeof s === 'number' && s < early - 0.6);
|
||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||
const afterHalf = await pitch();
|
||
check(
|
||
'Set sound FX parameters with a time ramps the pitch over that time instead of jumping',
|
||
early > -1.8 && later > -3.3 && afterHalf > -4.3 && afterHalf < later,
|
||
JSON.stringify({ early, later, afterHalf }),
|
||
);
|
||
await stop();
|
||
// The easings of an animation key: Quad in hands the clock a curve, 7 - 3 * t^2 towards -5 in 2 s,
|
||
// where linear is 7 - 6 * t. Each reading is timed on the audio clock from the voice's start, so a
|
||
// first one that comes in late is held against where the curve is then. Between 0.3 and 1.7 s the
|
||
// curve is more than a semitone above linear, and it never falls below it.
|
||
await mutate(`
|
||
const tune = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph.nodes.find((n) => n.id === 'sn-tune');
|
||
Object.assign(tune.params, { ease: 'quad', direction: 'in' });`);
|
||
await play();
|
||
const pitchNow = () =>
|
||
json(`({ pitch: __gm.sound.levels(${JSON.stringify(blip.file)})?.[0]?.[0], clock: __gm.sound.clock() })`);
|
||
const sinceStart = async (seen, since = 'start') => {
|
||
const now = await stats();
|
||
const from = since === 'tune' ? now.tunedAt : now.last.at;
|
||
return seen
|
||
.filter((s) => typeof s.pitch === 'number')
|
||
.map((s) => ({ t: +(s.clock - from).toFixed(3), pitch: +s.pitch.toFixed(3) }));
|
||
};
|
||
const sweep = await sinceStart(await trace(pitchNow, (s) => typeof s.pitch === 'number' && s.pitch < -4.3));
|
||
const midway = sweep.filter(({ t }) => t > 0.3 && t < 1.7);
|
||
check(
|
||
'Set sound FX parameters with Quad in ramps slowly first, then on to the end',
|
||
midway.length > 0 &&
|
||
midway.every(({ t, pitch }) => pitch > 7 - 6 * t + 1) &&
|
||
sweep.every(({ t, pitch }) => pitch > 7 - 6 * t - 0.1) &&
|
||
sweep.at(-1).pitch < -4.3,
|
||
JSON.stringify(sweep),
|
||
);
|
||
await stop();
|
||
// Step holds the pitch for the whole time and then jumps there: every reading before 2 s on the
|
||
// audio clock is still 7, the first one past it is the end. Timed from the change, not from the
|
||
// start of the voice: task 529 saw it arrive 80 ms later and the 7 held, rightly, to 2.08 s.
|
||
await mutate(`
|
||
const tune = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph.nodes.find((n) => n.id === 'sn-tune');
|
||
Object.assign(tune.params, { ease: 'step', time: 2 });`);
|
||
await play();
|
||
const steps = await sinceStart(
|
||
await trace(pitchNow, (s) => typeof s.pitch === 'number' && s.pitch < 6),
|
||
'tune',
|
||
);
|
||
const held = steps.filter(({ pitch }) => pitch === 7);
|
||
const jumped = steps.find(({ pitch }) => pitch < 6);
|
||
check(
|
||
'Set sound FX parameters with Step holds the pitch for the time, then jumps to the end',
|
||
held.length > 0 &&
|
||
held.every(({ t }) => t < 2.05) &&
|
||
jumped !== undefined &&
|
||
jumped.t > 1.95 &&
|
||
Math.abs(jumped.pitch + 5) < 0.001,
|
||
JSON.stringify(steps),
|
||
);
|
||
await stop();
|
||
// #317: the worklet itself, rendered in an offline context of this browser – what a muted check
|
||
// can hear. A second of 440 Hz an octave up is 880 Hz and still a second long; a sweep from
|
||
// 0 to -12 over it keeps that second too. Then 16 stereo voices at +5 for 2 s, timed, for
|
||
// how many this machine plays at once.
|
||
const offline = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const render = async (voices, seconds, channels, shape) => {
|
||
const context = new OfflineAudioContext(channels, RATE * seconds, RATE);
|
||
await context.audioWorklet.addModule(__gm.sound.pitchWorkletUrl());
|
||
const tone = new Float32Array(RATE * seconds);
|
||
for (let i = 0; i < tone.length; i++) tone[i] = Math.sin((2 * Math.PI * 440 * i) / RATE) * 0.5;
|
||
for (let v = 0; v < voices; v++) {
|
||
const node = new AudioWorkletNode(context, 'pitch-player', {
|
||
numberOfInputs: 0,
|
||
outputChannelCount: [channels],
|
||
parameterData: { pitch: shape.from },
|
||
processorOptions: { file: 'probe', channels: v === 0 ? Array(channels).fill(tone) : undefined, step: 1, loop: false },
|
||
});
|
||
if (shape.to !== shape.from)
|
||
node.parameters.get('pitch').setValueAtTime(shape.from, 0).linearRampToValueAtTime(shape.to, seconds);
|
||
node.connect(context.destination);
|
||
}
|
||
const started = performance.now();
|
||
const out = (await context.startRendering()).getChannelData(0);
|
||
return { out, ms: performance.now() - started };
|
||
};
|
||
const rises = (out, from, to) => {
|
||
let count = 0;
|
||
for (let i = from + 1; i < to; i++) if (out[i - 1] < 0 && out[i] >= 0) count++;
|
||
return count / ((to - from) / RATE);
|
||
};
|
||
const heard = (out) => {
|
||
let last = out.length;
|
||
while (last > 0 && Math.abs(out[last - 1]) <= 1e-4) last--;
|
||
return last;
|
||
};
|
||
const up = await render(1, 1, 1, { from: 12, to: 12 });
|
||
const sweep = await render(1, 1, 1, { from: 0, to: -12 });
|
||
const load = await render(16, 2, 2, { from: 5, to: 5 });
|
||
return {
|
||
upHz: rises(up.out, 12000, 36000),
|
||
upLength: heard(up.out),
|
||
sweepEarlyHz: rises(sweep.out, 6000, 12000),
|
||
sweepLateHz: rises(sweep.out, 36000, 42000),
|
||
sweepLength: heard(sweep.out),
|
||
voices: 16,
|
||
renderMs: Math.round(load.ms),
|
||
realTimeVoices: Math.round((16 * 2000) / load.ms),
|
||
};
|
||
})()`);
|
||
check(
|
||
'the pitch worklet plays an octave up at the length of the sound, and a sweep down keeps it too',
|
||
Math.abs(offline.upHz / 880 - 1) < 0.02 &&
|
||
offline.upLength > 48000 - 100 &&
|
||
offline.sweepEarlyHz > offline.sweepLateHz * 1.3 &&
|
||
offline.sweepLength > 48000 - 100,
|
||
JSON.stringify(offline),
|
||
);
|
||
// #327: the player's own echo, rendered offline. A 10 ms burst into a ping-pong Delay (0.3 s,
|
||
// feedback 0.5, mix 0.5) comes back left at 0.3 s at half its level, right at 0.6 s at
|
||
// half of that, left again at 0.9 s – in Chrome each round 128 samples later than that. Then
|
||
// 64 stereo voices of it for 2 s, timed, for how many this machine plays at once (#324).
|
||
const echo = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const echoOf = (pingPong) => ({ kind: 'delay', time: 0.3, feedback: 0.5, damping: 20000, mix: 0.5, pingPong });
|
||
const render = async (voices, seconds, stage = echoOf(true)) => {
|
||
const context = new OfflineAudioContext(2, RATE * seconds, RATE);
|
||
const burst = context.createBuffer(1, RATE * 0.01, RATE);
|
||
burst.getChannelData(0).fill(0.5);
|
||
for (let v = 0; v < voices; v++) {
|
||
const source = context.createBufferSource();
|
||
source.buffer = burst;
|
||
const nodes = __gm.sound.stageNodes(context, stage);
|
||
source.connect(nodes.input);
|
||
nodes.output.connect(context.destination);
|
||
source.start();
|
||
}
|
||
const started = performance.now();
|
||
const out = await context.startRendering();
|
||
return { left: out.getChannelData(0), right: out.getChannelData(1), ms: performance.now() - started };
|
||
};
|
||
// The middle of the burst: its edges ring a little through the damping.
|
||
const peak = (data, at) => {
|
||
let sum = 0;
|
||
const from = Math.round((at + 0.003) * RATE);
|
||
const to = Math.round((at + 0.007) * RATE);
|
||
for (let i = from; i < to; i++) sum += Math.abs(data[i]);
|
||
return Math.round((sum / (to - from)) * 1000) / 1000;
|
||
};
|
||
// Samples from where each echo is due to where it is heard, in both kinds of echo.
|
||
const late = ({ left, right }) => {
|
||
const found = [];
|
||
for (let k = 1; k <= 4; k++) {
|
||
const from = Math.round((k * 0.3 - 0.05) * RATE);
|
||
let at = from;
|
||
while (at < from + RATE * 0.1 && Math.abs(left[at]) <= 0.002 && Math.abs(right[at]) <= 0.002) at++;
|
||
found.push(at - Math.round(k * 0.3 * RATE));
|
||
}
|
||
return found;
|
||
};
|
||
const one = await render(1, 1.3);
|
||
const plain = await render(1, 1.3, echoOf(false));
|
||
const load = await render(64, 2);
|
||
const lateBy = late(one);
|
||
const at = (k) => k * 0.3 + lateBy[k - 1] / RATE;
|
||
return {
|
||
late: lateBy,
|
||
latePlain: late(plain),
|
||
dry: [peak(one.left, 0), peak(one.right, 0)],
|
||
first: [peak(one.left, at(1)), peak(one.right, at(1))],
|
||
second: [peak(one.left, at(2)), peak(one.right, at(2))],
|
||
third: [peak(one.left, at(3)), peak(one.right, at(3))],
|
||
between: [peak(one.left, 0.45), peak(one.right, 0.45)],
|
||
voices: 64,
|
||
renderMs: Math.round(load.ms),
|
||
realTimeVoices: Math.round((64 * 2000) / load.ms),
|
||
};
|
||
})()`);
|
||
const near = (value, to) => Math.abs(value - to) < 0.02;
|
||
check(
|
||
'the player’s ping-pong echo comes back left, then right, then left, each round half as loud',
|
||
near(echo.dry[0], 0.5) &&
|
||
near(echo.dry[1], 0.5) &&
|
||
near(echo.first[0], 0.25) &&
|
||
echo.first[1] < 0.005 &&
|
||
echo.second[0] < 0.005 &&
|
||
near(echo.second[1], 0.125) &&
|
||
near(echo.third[0], 0.0625) &&
|
||
echo.third[1] < 0.005 &&
|
||
echo.between.every((v) => v < 0.005) &&
|
||
// Chrome takes a render quantum more for every round of a feedback loop.
|
||
[echo.late, echo.latePlain].every((late) => late.every((n, k) => n === 0 || n === k * 128)),
|
||
JSON.stringify(echo),
|
||
);
|
||
// A Mix "Both at center", rendered offline: a steady 1 on a and 0.5 on b in two channels, so each
|
||
// share reads off as it is. Then mix glides 0 → 1 over a second: at its middle both stay full.
|
||
const atCenter = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const render = async (mix, glide) => {
|
||
const context = new OfflineAudioContext(2, RATE, RATE);
|
||
const merger = context.createChannelMerger(2);
|
||
merger.connect(context.destination);
|
||
const shareA = context.createGain();
|
||
const shareB = context.createGain();
|
||
shareA.gain.value = 0;
|
||
shareB.gain.value = 0;
|
||
const driver = __gm.sound.bothAtCenter(context, mix, shareA, shareB);
|
||
for (const [share, level, side] of [[shareA, 1, 0], [shareB, 0.5, 1]]) {
|
||
const steady = context.createConstantSource();
|
||
steady.offset.value = level;
|
||
steady.connect(share).connect(merger, 0, side);
|
||
steady.start();
|
||
}
|
||
if (glide) driver.offset.setValueAtTime(0, 0).linearRampToValueAtTime(1, 1);
|
||
driver.start();
|
||
const out = await context.startRendering();
|
||
const at = (s) => [out.getChannelData(0)[Math.round(s * RATE)], out.getChannelData(1)[Math.round(s * RATE)] * 2];
|
||
return glide ? [0.25, 0.5, 0.75].map(at) : at(0.5);
|
||
};
|
||
const round = (pair) => pair.map((v) => Math.round(v * 1000) / 1000);
|
||
const fixed = {};
|
||
for (const mix of [0, 0.25, 0.5, 0.75, 1]) fixed[mix] = round(await render(mix, false));
|
||
return { fixed, glide: (await render(0, true)).map(round) };
|
||
})()`);
|
||
const pairNear = (pair, a, b) => near(pair[0], a) && near(pair[1], b);
|
||
check(
|
||
'a Mix "Both at center" keeps a full up to the middle and b full from it, also while mix glides',
|
||
pairNear(atCenter.fixed[0], 1, 0) &&
|
||
pairNear(atCenter.fixed[0.25], 1, 0.5) &&
|
||
pairNear(atCenter.fixed[0.5], 1, 1) &&
|
||
pairNear(atCenter.fixed[0.75], 0.5, 1) &&
|
||
pairNear(atCenter.fixed[1], 0, 1) &&
|
||
pairNear(atCenter.glide[0], 1, 0.5) &&
|
||
pairNear(atCenter.glide[1], 1, 1) &&
|
||
pairNear(atCenter.glide[2], 0.5, 1),
|
||
JSON.stringify(atCenter),
|
||
);
|
||
await mutate(`
|
||
const logic = __gm.state.project.assets.find((a) => a.id === 'sn-logic');
|
||
logic.variables = [];
|
||
const graph = logic.graph;
|
||
graph.nodes = graph.nodes.filter((n) => !${JSON.stringify(tuneIds)}.includes(n.id));
|
||
graph.edges = graph.edges.filter((e) => !${JSON.stringify(tuneEdges)}.includes(e.id));`);
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes = graph.nodes.filter((n) => n.id !== 'sn-fxp');
|
||
graph.edges = graph.edges.filter((e) => e.id !== 'sn-e-fxp');
|
||
Object.assign(graph.nodes.find((n) => n.id === 'sn-play').params, { loop: '', fx: '' });
|
||
const project = __gm.state.project;
|
||
project.assets = project.assets.filter((a) => a.id !== ${JSON.stringify(fxId)});`);
|
||
|
||
// 34g-5. An instrument (#331): "+ → Instrument" opens its tab on Notes → Sample → Output, which
|
||
// plays nothing until the Sample has a sound. With the blip, looped, every key of C4–B4
|
||
// sounds; a key held by the pointer plays until it is let go, the computer keyboard plays
|
||
// as a tracker does, and a game's `Play note` releases its note after its duration.
|
||
await showTab('scene');
|
||
await clickText('+');
|
||
await pickAssetType(t, 'Instrument');
|
||
const insCreated = await json(`{
|
||
tab: __gm.tab(),
|
||
current: __gm.instrument.current(),
|
||
info: __gm.instrument.info(),
|
||
keys: __gm.instrument.keys(),
|
||
assets: __gm.state.project.assets.filter((a) => a.type === 'instrument')
|
||
.map((a) => ({ id: a.id, nodes: a.graph.nodes.map((n) => n.type) })),
|
||
}`);
|
||
check(
|
||
'the + menu creates an instrument, Notes → Sample → Output, and opens its tab; no key sounds yet',
|
||
insCreated.tab === 'instrument' &&
|
||
insCreated.assets.length === 1 &&
|
||
insCreated.current === insCreated.assets[0].id &&
|
||
insCreated.assets[0].nodes.join() === 'InstrumentNotes,InstrumentSample,InstrumentOutput' &&
|
||
insCreated.info === 'No sound picked' &&
|
||
insCreated.keys.sounding.length === 0,
|
||
JSON.stringify(insCreated),
|
||
);
|
||
const insId = insCreated.assets[0]?.id;
|
||
// Bottom left: the fitted graph is one row of three nodes, its links among them.
|
||
const insCanvas = await json('__gm.instrument.canvasRect()');
|
||
await click(insCanvas.x + 30, insCanvas.y + insCanvas.h - 30, 'right');
|
||
const insPalette = await json('__gm.overlayTexts()');
|
||
check(
|
||
'its palette offers its own nodes and the effects, but no Input and no behavior node',
|
||
['Notes', 'Sample', 'Parameters', 'Sound FX', 'Gain', 'Delay'].every((label) =>
|
||
insPalette.includes(label),
|
||
) &&
|
||
!insPalette.includes('Input') &&
|
||
!insPalette.includes('Branch'),
|
||
JSON.stringify(insPalette.slice(0, 14)),
|
||
);
|
||
await press('Escape');
|
||
await mutate(`
|
||
const ins = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(insId)});
|
||
Object.assign(ins.graph.nodes.find((n) => n.type === 'InstrumentSample').params, {
|
||
sound: ${JSON.stringify(soundId)},
|
||
play: 'loop',
|
||
});`);
|
||
const insReady = await json('{ info: __gm.instrument.info(), keys: __gm.instrument.keys() }');
|
||
check(
|
||
'with a sound picked the twelve keys from C4 to B4 sound, the others stay grey',
|
||
insReady.info === '12 notes' &&
|
||
insReady.keys.sounding.join() === [60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71].join(),
|
||
JSON.stringify(insReady),
|
||
);
|
||
await shot('34-instrument');
|
||
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeKey = await stats();
|
||
const e4 = await json('__gm.instrument.keyAt(64)');
|
||
await t.mouse('mouseMoved', e4.x, e4.y);
|
||
await t.mouse('mousePressed', e4.x, e4.y);
|
||
const heldKey = await waitFor(stats, (s) => s.started > beforeKey.started);
|
||
// Longer than the blip: a looped note that ended with its sound would be gone by now.
|
||
await sleep(600);
|
||
const stillHeld = await json('{ stats: __gm.sound.stats(), keys: __gm.instrument.keys() }');
|
||
check(
|
||
'holding E4 plays the blip four semitones up, looped, as one layer of the instrument, and lights the key',
|
||
heldKey.last?.file === `instrument:${insId}` &&
|
||
heldKey.last.pitches.join() === '4' &&
|
||
heldKey.last.layers === 1 &&
|
||
stillHeld.stats.playing === 1 &&
|
||
stillHeld.keys.held.join() === '64',
|
||
JSON.stringify({ heldKey, stillHeld }),
|
||
);
|
||
await t.mouse('mouseReleased', e4.x, e4.y);
|
||
const letGo = await waitFor(stats, (s) => s.playing === 0);
|
||
check(
|
||
'letting go of the key ends the note: its sample fades out and the voice goes',
|
||
letGo.playing === 0 && (await json('__gm.instrument.keys()')).held.length === 0,
|
||
JSON.stringify(letGo),
|
||
);
|
||
|
||
// "Loop, then release": held, it loops 0.05 to 0.1 s; let go, it plays the 0.15 s after the loop.
|
||
const setPlay = (fields) =>
|
||
mutate(`
|
||
const ins = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(insId)});
|
||
Object.assign(ins.graph.nodes.find((n) => n.type === 'InstrumentSample').params, ${JSON.stringify(fields)});`);
|
||
await setPlay({ play: 'release', loopStart: 0.05, loopEnd: 0.1 });
|
||
const beforeRelease = await stats();
|
||
await t.mouse('mousePressed', e4.x, e4.y);
|
||
await waitFor(stats, (s) => s.started > beforeRelease.started);
|
||
await sleep(600);
|
||
const heldLoop = await stats();
|
||
await t.mouse('mouseReleased', e4.x, e4.y);
|
||
const leftAt = await audioNow();
|
||
const ringingKeys = await json('__gm.instrument.keys()');
|
||
const playedOut = await waitFor(audioNow, (s) => s.playing === 0);
|
||
const afterKeys = await waitFor(
|
||
() => json('__gm.instrument.keys()'),
|
||
(keys) => keys.ringing.length === 0,
|
||
);
|
||
// 0.15 s after the loop, up to two rounds more (the one it is in, the one the worklet read into)
|
||
// and the 150 ms waitFor polls at: a fade would be gone in 0.01 s.
|
||
const outFor = playedOut.clock - leftAt.clock;
|
||
check(
|
||
'"Loop, then release" loops while held, and let go plays on past the loop with its key half lit',
|
||
heldLoop.playing === 1 &&
|
||
ringingKeys.held.length === 0 &&
|
||
ringingKeys.ringing.join() === '64' &&
|
||
outFor > 0.12 &&
|
||
outFor < 0.5 &&
|
||
afterKeys.ringing.length === 0,
|
||
JSON.stringify({ heldLoop: heldLoop.playing, ringingKeys, outFor, afterKeys }),
|
||
);
|
||
await setPlay({ play: 'loop', loopStart: 0, loopEnd: 0 });
|
||
|
||
// The click left the keys focused. Z is C3 there – no sound in this instrument –, one octave up C4.
|
||
const beforeZ = await stats();
|
||
await press('z');
|
||
await sleep(200);
|
||
const lowZ = await stats();
|
||
await press('ArrowRight');
|
||
await t.keyDown('z');
|
||
const typed = await waitFor(stats, (s) => s.started > lowZ.started);
|
||
const typedKeys = await json('__gm.instrument.keys()');
|
||
await t.keyUp('z');
|
||
const typedOff = await waitFor(stats, (s) => s.playing === 0);
|
||
check(
|
||
'the computer keyboard plays as a tracker: Z is C3 and silent here, one octave up it is C4 until let go',
|
||
lowZ.started === beforeZ.started &&
|
||
typedKeys.octave === 4 &&
|
||
typedKeys.held.join() === '60' &&
|
||
typed.last?.pitches.join() === '0' &&
|
||
typedOff.playing === 0,
|
||
JSON.stringify({ beforeZ: beforeZ.started, lowZ: lowZ.started, typed, typedKeys, typedOff }),
|
||
);
|
||
await press('ArrowLeft');
|
||
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.push(${JSON.stringify(
|
||
node('sn-note', 'PlayNote', 260, 220, { instrument: insId, note: 67, duration: 1.5 }),
|
||
)});
|
||
graph.edges.find((e) => e.id === 'sn-e1').to = { node: 'sn-note', port: 'exec' };`);
|
||
await showTab('scene');
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeNote = await stats();
|
||
await play();
|
||
// The sample loops, so only the release after 1.5 s on the audio clock ends the note: it is heard
|
||
// well past the 0.25 s of the blip, never past its duration, and is gone soon after.
|
||
const noteNow = () => json('({ stats: __gm.sound.stats(), clock: __gm.sound.clock() })');
|
||
const noted = await waitFor(noteNow, (s) => s.stats.started > beforeNote.started);
|
||
const heard = await trace(noteNow, (s) => s.stats.playing === 0);
|
||
const noteAt = noted.stats.last?.at ?? 0;
|
||
const heardAt = heard.filter((s) => s.stats.playing === 1).map((s) => +(s.clock - noteAt).toFixed(3));
|
||
const goneAt = +(heard.at(-1).clock - noteAt).toFixed(3);
|
||
check(
|
||
'a game’s Play note plays G4 – the blip seven semitones up – and lets it go after its duration',
|
||
noted.stats.last?.file === `instrument:${insId}` &&
|
||
noted.stats.last.pitches.join() === '7' &&
|
||
heardAt.some((t) => t > 0.5) &&
|
||
heardAt.every((t) => t < 1.6) &&
|
||
heard.at(-1).stats.playing === 0 &&
|
||
goneAt < 3,
|
||
JSON.stringify({ last: noted.stats.last, heardAt, goneAt }),
|
||
);
|
||
await stop();
|
||
|
||
// #332: the scheduler, driven by the hook – nothing in the editor plays a sequence yet. A bar of
|
||
// quarters at 150 BPM in a loop: every note is planned before its second comes, each 0.4 s
|
||
// after the one before, also across the seam into the next round. Where the music stands
|
||
// is the audio clock less the output latency, counted from the first note.
|
||
const bar = [60, 62, 64, 65].map((pitch, i) => ({
|
||
tick: i * 480,
|
||
length: 240,
|
||
note: pitch,
|
||
velocity: 1,
|
||
instrument: insId,
|
||
}));
|
||
await evaluate(
|
||
`__gm.music.play(${JSON.stringify({ notes: bar })}, { tempo: 150, loop: { start: 0, end: 1920 } })`,
|
||
);
|
||
const planned = await waitFor(
|
||
() => json('__gm.music.planned()'),
|
||
(p) => p.length >= 9,
|
||
40,
|
||
);
|
||
const where = await json(
|
||
'({ position: __gm.music.position(), clock: __gm.sound.clock(), latency: __gm.music.latency() })',
|
||
);
|
||
await evaluate('__gm.music.stop()');
|
||
const musicOff = await waitFor(stats, (s) => s.playing === 0);
|
||
const apart = planned.slice(1, 9).map((p, i) => +(p.at - planned[i].at).toFixed(6));
|
||
const ahead = planned.map((p) => +(p.at - p.planned).toFixed(3));
|
||
const heardTicks = (((where.clock - where.latency - planned[0].at) * 150 * 480) / 60) % 1920;
|
||
check(
|
||
'the scheduler plans a looped bar on the audio clock: every note ahead of its second, 0.4 s apart across the seam',
|
||
planned.length >= 9 &&
|
||
planned
|
||
.slice(0, 9)
|
||
.map((p) => p.note)
|
||
.join() === '60,62,64,65,60,62,64,65,60' &&
|
||
planned.every((p) => p.voice > 0 && Math.abs(p.duration - 0.2) < 1e-9 && p.tempo === 150) &&
|
||
apart.every((gap) => Math.abs(gap - 0.4) < 1e-6) &&
|
||
ahead.every((s) => s > 0 && s <= 0.16) &&
|
||
Math.abs(where.position.ticks - heardTicks) < 1 &&
|
||
where.position.bar === 1 &&
|
||
musicOff.playing === 0 &&
|
||
(await json('__gm.music.playing()')) === false,
|
||
JSON.stringify({ apart, ahead, where, heardTicks, playing: musicOff.playing }),
|
||
);
|
||
|
||
// A note planned ahead through the pitch worklet starts on its sample, also inside a render block:
|
||
// rendered offline at pitch 0, which plays the sound as it is, so the first sample heard is it.
|
||
const onsets = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const onset = async (startAt) => {
|
||
const context = new OfflineAudioContext(1, RATE / 2, RATE);
|
||
await context.audioWorklet.addModule(__gm.sound.pitchWorkletUrl());
|
||
const node = new AudioWorkletNode(context, 'pitch-player', {
|
||
numberOfInputs: 0,
|
||
outputChannelCount: [1],
|
||
parameterData: { pitch: 0 },
|
||
processorOptions: { file: 'onset', channels: [new Float32Array(RATE).fill(0.5)], step: 1, loop: false, startAt },
|
||
});
|
||
node.connect(context.destination);
|
||
const out = (await context.startRendering()).getChannelData(0);
|
||
return out.findIndex((v) => Math.abs(v) > 1e-6);
|
||
};
|
||
const at = [0.0123, 0.25, 0.05 + 0.4 * 2 / 3];
|
||
return { expected: at.map((s) => Math.round(s * RATE)), heard: await Promise.all(at.map(onset)) };
|
||
})()`);
|
||
check(
|
||
'a pitched note planned ahead starts on its very sample, inside a render block too',
|
||
onsets.heard.join() === onsets.expected.join() && onsets.expected.some((i) => i % 128 !== 0),
|
||
JSON.stringify(onsets),
|
||
);
|
||
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes = graph.nodes.filter((n) => n.id !== 'sn-note');
|
||
graph.edges.find((e) => e.id === 'sn-e1').to = { node: 'sn-play', port: 'exec' };
|
||
const project = __gm.state.project;
|
||
project.assets = project.assets.filter((a) => a.id !== ${JSON.stringify(insId)});`);
|
||
|
||
// 34h. Dropping a sound file onto the editor uploads it and makes an asset named after the file.
|
||
const dropped = await evaluate(`
|
||
(async () => {
|
||
const bytes = await fetch('/game-maker/data/sounds/${drone.file}').then((r) => r.arrayBuffer());
|
||
const file = new File([bytes], 'drone.wav', { type: 'audio/wav' });
|
||
const data = new DataTransfer();
|
||
data.items.add(file);
|
||
const canvas = document.querySelector('canvas');
|
||
canvas.dispatchEvent(new DragEvent('drop', { dataTransfer: data, bubbles: true, cancelable: true }));
|
||
return file.size;
|
||
})()`);
|
||
await waitFor(
|
||
() => assetsOfType('sound'),
|
||
(list) => list.length === 2,
|
||
);
|
||
const both = await assetsOfType('sound');
|
||
check(
|
||
'a dropped sound is uploaded and becomes an asset named after the file',
|
||
dropped === DRONE_BYTES &&
|
||
both.length === 2 &&
|
||
both.some((a) => a.file === drone.file && a.name === 'drone'),
|
||
JSON.stringify({ dropped, both, notice: await json('__gm.state.notice') }),
|
||
);
|
||
|
||
// 34i. The step takes back everything it made, so the steps after it see the fixture again.
|
||
await mutate(`
|
||
const gone = new Set(['sn-logic', ...${JSON.stringify(both.map((a) => a.id))}]);
|
||
const project = __gm.state.project;
|
||
project.assets = project.assets.filter((a) => !gone.has(a.id));
|
||
__gm.state.activeScene.objects = __gm.state.activeScene.objects.filter(
|
||
(o) => !['sn-box', 'sn-brain'].includes(o.id));
|
||
__gm.state.select(null);
|
||
__gm.lookAt(0, 0);`);
|
||
check(
|
||
'the step leaves nothing of its own behind',
|
||
(await assetsOfType('sound')).length === 0 && (await stats()).playing === 0,
|
||
JSON.stringify(await assetsOfType('sound')),
|
||
);
|
||
}
|