Without an envelope a note set in hard and broke off 10 ms after its end. An instrument now has an Envelope node with attack, decay, sustain and release. On the way of the sound it is the volume. Its output "value" moves a number of the note with the same curve, from "From" to "To", for a filter sweep among others. A new Filter node gives that sweep something to move. - Envelope (instrument only): attack a straight line, decay and release ease out and are 60 dB of the way at their time. A release takes 10 ms at least. Defaults 0.005 / 0.1 / 1 / 0.1. - The end of a note goes to the Envelope instead of the 10 ms fade. A Gate or Loop Sample under it plays on through the release, "Loop, then release" leaves its loop too, and the voice ends with the release. A One-shot under an Envelope of its own plays through. - A note ending before attack and decay releases from the level it has reached. Chrome holds it with cancelAndHoldAtTime, Firefox gets the level worked out from the shape. - New port type "envelope" (coral). It fits only a number input that says it can follow one (PortDef.followsEnvelope): Filter cutoff and resonance, Pitch, Gain, the EQ bands and Mix. Not the Delay, not the Envelope's own numbers. portsMeet() decides it for ConnectCommand, the drag and the drawn edges. - On the audio thread every Envelope of a note is one ConstantSourceNode. A number it moves gets the curve through a WaveShaper, in equal ratios for a cutoff and a gain. A number an Envelope moves is no longer reached by a parameter. - Filter (sound FX and instrument): low-, high- or band-pass, cutoff in Hz, resonance 0 to 1 as a Q. A low- or high-pass is flat at 0 and rises 20 dB at 1, a band-pass goes from Q 0.7 to 20. - Stop sound with "let ring" lets a note with an Envelope fall away in its release. Without it, and with Stop in the tab, it is cut. - The same note again during a release is a voice of its own. - The Envelope's numbers are set when the note starts. They can come from velocity or a parameter, later changes do not bend the note. Tests: - envelope.test.ts (new, 12): curve, ratios, Filter Q, compiling an Envelope, its value into each number, a Delay refused - sound-library +7: attack and decay on the clock, release from the level reached, One-shot held, Stop sound with and without let ring, a cutoff and a pitch moved by the curve - e2e 34: ALL PASSED, 50 checks (+2). Offline in Chrome the curve rendered 0.5 / 1 / 0.5005 at 0.05 / 0.1 / 0.3 s, released at 0.5 s it was 0.0158 at 0.65 s and 0.0005 at 0.8 s, released mid-attack it fell from 0.5 to 0.05 in 0.1 s. A Gain moved 0.25 to 1 was 0.5 halfway. A looped note of 0.2 s with a release of 1 s was gone after 1.32 s. - Five counter-checks (release end, One-shot hold, the follows check, ratios, pitch sum) failed 3, 1, 1, 2 and 2 tests. - Gate on melektaus-atx: CHECK EXIT 0, 245 files, 3812 tests Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1170 lines
54 KiB
JavaScript
1170 lines
54 KiB
JavaScript
import { edge, node, port, variable, written } from '../lib/builders.mjs';
|
||
import { LANES, MAKE_OBJECT, pickAssetType } from './lib/scaffold.mjs';
|
||
|
||
/**
|
||
* Sound assets (#45): uploaded like a typeface – picked from what the server holds through
|
||
* "+ → Sound", or dropped onto the editor – and played by a graph through `Play sound`. Nothing
|
||
* wears a sound and nothing places it; the asset says how loud it is and whether it loops, and the
|
||
* node scales the one and can overrule the other in either direction.
|
||
*
|
||
* Nothing is ever *heard* here: CI runs the browser muted (see tools/lib/browser.mjs). The witness
|
||
* is `__gm.sound.stats()` – voices started, stopped and in flight – the way `renderStats` is the
|
||
* witness for a repaint that did not happen. No pixel check could see any of this.
|
||
*
|
||
* The seed puts two generated WAVs on the server without making assets of them: a 0.25 s blip and
|
||
* a 0.75 s drone, told apart by their exact size rather than by their name.
|
||
*/
|
||
const LANE = LANES.sound;
|
||
const BLIP_BYTES = 4044;
|
||
const DRONE_BYTES = 12044;
|
||
|
||
export async function sound(t) {
|
||
const { check, sleep, json, evaluate, mutate, click, clickText, press, play, stop, shot, showTab } = t;
|
||
const { select } = t;
|
||
const assetsOfType = (type) =>
|
||
json(`__gm.state.project.assets.filter((a) => a.type === ${JSON.stringify(type)})
|
||
.map((a) => ({ id: a.id, name: a.name, file: a.file, props: a.props }))`);
|
||
const stats = () => json('__gm.sound.stats()');
|
||
/** A voice starts only once its bytes have been decoded, so what it changes is awaited. */
|
||
const waitFor = async (read, done, tries = 25) => {
|
||
let value = await read();
|
||
for (let i = 0; i < tries && !done(value); i++) {
|
||
await sleep(150);
|
||
value = await read();
|
||
}
|
||
return value;
|
||
};
|
||
// Every reading up to the one `done` takes: a check holds each against its own time on the audio
|
||
// clock, however late the first one comes in.
|
||
const trace = async (read, done, tries = 40) => {
|
||
const seen = [await read()];
|
||
for (let i = 0; i < tries && !done(seen.at(-1)); i++) {
|
||
await sleep(150);
|
||
seen.push(await read());
|
||
}
|
||
return seen;
|
||
};
|
||
|
||
await press('Escape');
|
||
await evaluate('document.activeElement instanceof HTMLElement && document.activeElement.blur()');
|
||
await evaluate('__gm.state.select(null)');
|
||
await showTab('scene');
|
||
|
||
// 34a. What the server holds, straight from the API: the two files the seed put there.
|
||
const stored = await evaluate(`fetch('/game-maker/api/index.php?r=sounds').then((r) => r.json())`);
|
||
const blip = stored.find((item) => item.bytes === BLIP_BYTES);
|
||
const drone = stored.find((item) => item.bytes === DRONE_BYTES);
|
||
check(
|
||
'the server holds the two seeded sounds, named by their content',
|
||
Boolean(blip && drone) && /^[a-f0-9]{64}\.wav$/.test(blip.file),
|
||
JSON.stringify({ blip: blip?.file, drone: drone?.file, count: stored.length }),
|
||
);
|
||
if (!blip || !drone) return;
|
||
check(
|
||
'the listing carries the name each file was uploaded under',
|
||
blip.name === 'blip.wav' && drone.name === 'drone.wav',
|
||
JSON.stringify({ blip: blip.name, drone: drone.name }),
|
||
);
|
||
|
||
// 34b. "+ → Sound" offers them; a sound is picked rather than created empty, like a typeface.
|
||
await clickText('+');
|
||
await pickAssetType(t, 'Sound');
|
||
await sleep(500);
|
||
const picker = await json('__gm.soundPicker()');
|
||
check(
|
||
'the + menu opens the sound picker on what the server holds, none of it in the project yet',
|
||
Boolean(picker) && picker.files.includes(blip.file) && picker.used.length === 0,
|
||
JSON.stringify(picker),
|
||
);
|
||
check(
|
||
'a row nobody uses yet is captioned with the name the file was uploaded under',
|
||
picker.captions[picker.files.indexOf(blip.file)] === 'blip.wav',
|
||
JSON.stringify(picker.captions),
|
||
);
|
||
await shot('34-sound-picker');
|
||
|
||
// 34c. The ▶ at a row's left is a preview: it plays that file without choosing it.
|
||
const playAt = await json(`__gm.soundPickerPlay(${JSON.stringify(drone.file)})`);
|
||
await click(playAt.x, playAt.y);
|
||
const previewed = await waitFor(stats, (s) => s.started > 0);
|
||
const afterPreview = await json('__gm.soundPicker()');
|
||
check(
|
||
'pressing a row’s ▶ plays that file and picks nothing',
|
||
previewed.started === 1 &&
|
||
previewed.last?.file === drone.file &&
|
||
afterPreview.played === drone.file &&
|
||
afterPreview.selected === null,
|
||
JSON.stringify({ previewed, selected: afterPreview.selected, played: afterPreview.played }),
|
||
);
|
||
|
||
const row = await json(`__gm.soundPickerRow(${JSON.stringify(blip.file)})`);
|
||
await click(row.x, row.y);
|
||
await clickText('Use sound');
|
||
const sounds = await assetsOfType('sound');
|
||
check(
|
||
'picking a row makes a sound asset of that file, named after the file rather than "Sound"',
|
||
sounds.length === 1 && sounds[0].file === blip.file && sounds[0].name === 'blip',
|
||
JSON.stringify(sounds),
|
||
);
|
||
if (sounds.length !== 1) return;
|
||
const soundId = sounds[0].id;
|
||
check(
|
||
'a fresh sound asset is full volume and one-shot',
|
||
sounds[0].props.volume === 1 && sounds[0].props.loop === false,
|
||
JSON.stringify(sounds[0].props),
|
||
);
|
||
// Closing the picker silences its preview; nothing it started may outlive the dialog.
|
||
check(
|
||
'closing the picker leaves nothing playing',
|
||
(await stats()).playing === 0,
|
||
JSON.stringify(await stats()),
|
||
);
|
||
|
||
// 34d. Its inspector: the two rows, and the preview that is the only way to tell which sound it is.
|
||
await select({ kind: 'asset', assetId: soundId });
|
||
const volumeRow = await json("__gm.inspectorField('Volume', true)");
|
||
const loopRow = await json("__gm.inspectorField('Loop', true)");
|
||
check(
|
||
'a sound asset has a Volume row and a Loop row',
|
||
Boolean(volumeRow) && Boolean(loopRow),
|
||
JSON.stringify({ volumeRow, loopRow }),
|
||
);
|
||
await shot('34-sound-asset');
|
||
|
||
// 34e. A graph plays it. The behavior sits in its own lane; the node's volume is a factor on the
|
||
// asset's, so 0.5 × 0.4 has to reach the library as 0.2 rather than as either of the two.
|
||
await mutate(`
|
||
const p = __gm.state.project;
|
||
const scene = __gm.state.activeScene;
|
||
p.assets.find((a) => a.id === ${JSON.stringify(soundId)}).props.volume = 0.4;
|
||
const asset = structuredClone(p.assets.find((a) => a.id === 'mover'));
|
||
Object.assign(asset, {
|
||
id: 'sn-logic',
|
||
name: 'Noisy',
|
||
props: { enabled: true, runInEditor: false },
|
||
variables: [],
|
||
graph: ${JSON.stringify({
|
||
nodes: [
|
||
node('sn-start', 'OnStart', 0, 0),
|
||
node('sn-play', 'PlaySound', 260, 0, { sound: soundId, volume: 0.5, loop: '' }),
|
||
],
|
||
edges: [edge('sn-e1', 'sn-start', 'exec', 'sn-play', 'exec')],
|
||
})},
|
||
});
|
||
p.assets.unshift(asset);
|
||
${MAKE_OBJECT}
|
||
scene.objects.push(
|
||
make('sn-box', 'crate', '', ${LANE}, 'Noisy box'),
|
||
make('sn-brain', 'sn-logic', 'sn-box', 0, 'Noisy'),
|
||
);
|
||
__gm.lookAt(0, ${LANE});`);
|
||
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const before = await stats();
|
||
await play();
|
||
const played = await waitFor(stats, (s) => s.started > before.started);
|
||
check(
|
||
'a graph playing a sound reaches the library, at the asset volume scaled by the node’s',
|
||
played.started === before.started + 1 &&
|
||
played.last?.file === blip.file &&
|
||
Math.abs(played.last.volume - 0.2) < 0.001 &&
|
||
played.last.loop === false,
|
||
JSON.stringify(played),
|
||
);
|
||
await stop();
|
||
|
||
// 34f. A looping sound is music: it plays on, and ending the run ends it – nothing may outlive
|
||
// the world that started it.
|
||
await mutate(`
|
||
const p = __gm.state.project;
|
||
p.assets.find((a) => a.id === ${JSON.stringify(soundId)}).props.loop = true;`);
|
||
await play();
|
||
const looping = await waitFor(stats, (s) => s.playing > 0 && s.last?.loop === true);
|
||
check(
|
||
'the asset’s own loop flag reaches the library even though the node never asked for one',
|
||
looping.playing >= 1 && looping.last?.loop === true,
|
||
JSON.stringify(looping),
|
||
);
|
||
// Well past the blip's own 0.25 s, so a one-shot would have ended by itself here.
|
||
await sleep(700);
|
||
const stillLooping = await stats();
|
||
await stop();
|
||
const afterStop = await stats();
|
||
check(
|
||
'a loop keeps playing while the run is on and ends with it',
|
||
stillLooping.playing >= 1 && afterStop.playing === 0,
|
||
JSON.stringify({ stillLooping, afterStop }),
|
||
);
|
||
|
||
// 34f-2. The node overrules the asset in both directions – the reason `loop` is a choice of three
|
||
// and not a checkbox: music has to be playable as a one-shot.
|
||
const setNodeLoop = (value) =>
|
||
mutate(`
|
||
const p = __gm.state.project;
|
||
const graph = p.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.find((n) => n.id === 'sn-play').params.loop = ${JSON.stringify(value)};`);
|
||
await setNodeLoop('false');
|
||
const beforeOnce = await stats();
|
||
await play();
|
||
const once = await waitFor(stats, (s) => s.started > beforeOnce.started);
|
||
check('a node saying "off" plays a looping asset once', once.last?.loop === false, JSON.stringify(once));
|
||
await stop();
|
||
await setNodeLoop('');
|
||
|
||
// 34g. `Stop sound` silences it from inside the graph, without ending the run.
|
||
await mutate(`
|
||
const p = __gm.state.project;
|
||
const graph = p.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.push(${JSON.stringify(node('sn-stop', 'StopSound', 520, 0, { sound: soundId }))});
|
||
graph.nodes.push(${JSON.stringify(node('sn-key', 'OnKeyDown', 260, 220, { key: 's' }))});
|
||
graph.edges.push(${JSON.stringify(edge('sn-e2', 'sn-key', 'exec', 'sn-stop', 'exec'))});`);
|
||
await play();
|
||
await waitFor(stats, (s) => s.playing > 0);
|
||
await press('s');
|
||
const silenced = await waitFor(stats, (s) => s.playing === 0);
|
||
check(
|
||
'Stop sound silences a looping sound while the run carries on',
|
||
silenced.playing === 0 && (await json('__gm.playInfo()')).playing === true,
|
||
JSON.stringify(silenced),
|
||
);
|
||
await stop();
|
||
|
||
// 34g-2. A voice belongs to the behavior playing it: destroy the object and its sounds go with it,
|
||
// without any `Stop sound` in the graph. `detached` is the one way out, for the death
|
||
// sound of an object that destroys itself.
|
||
await mutate(`
|
||
const p = __gm.state.project;
|
||
const graph = p.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.push(
|
||
${JSON.stringify(node('sn-key2', 'OnKeyDown', 260, 440, { key: 'd' }))},
|
||
${JSON.stringify(node('sn-owner', 'Owner', 260, 620))},
|
||
${JSON.stringify(node('sn-destroy', 'Destroy', 520, 440))},
|
||
);
|
||
graph.edges.push(
|
||
${JSON.stringify(edge('sn-e3', 'sn-key2', 'exec', 'sn-destroy', 'exec'))},
|
||
${JSON.stringify(edge('sn-e4', 'sn-owner', 'object', 'sn-destroy', 'object'))},
|
||
);`);
|
||
await play();
|
||
await waitFor(stats, (s) => s.playing > 0);
|
||
await press('d');
|
||
const destroyed = await waitFor(stats, (s) => s.playing === 0);
|
||
check(
|
||
'destroying the object silences the looping sound its behavior was playing',
|
||
destroyed.playing === 0 && (await json('__gm.playInfo()')).playing === true,
|
||
JSON.stringify(destroyed),
|
||
);
|
||
await stop();
|
||
|
||
const setDetached = (value) =>
|
||
mutate(`
|
||
const p = __gm.state.project;
|
||
const graph = p.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.find((n) => n.id === 'sn-play').params.detached = ${JSON.stringify(value)};`);
|
||
await setDetached(true);
|
||
await play();
|
||
await waitFor(stats, (s) => s.playing > 0);
|
||
await press('d');
|
||
// Long enough that the reaping frame has been and gone – a voice still here now is here on purpose.
|
||
await sleep(600);
|
||
const detached = await stats();
|
||
check(
|
||
'a detached voice is nobody’s and plays on after the object that started it is gone',
|
||
detached.playing >= 1,
|
||
JSON.stringify(detached),
|
||
);
|
||
await stop();
|
||
check(
|
||
'ending the run silences a detached voice too',
|
||
(await stats()).playing === 0,
|
||
JSON.stringify(await stats()),
|
||
);
|
||
await setDetached(false);
|
||
|
||
// 34g-3. A sound FX asset (#311): "+ → Sound FX" opens its tab, which says what keeps the graph
|
||
// from compiling; built into Input → Pitch → Gain → Output, the header plays the sound
|
||
// through it, and a `Play sound` naming it sends the game's voice the same way. The pitch
|
||
// comes from a parameter `depth` of the Input: the header has a field for it, and a
|
||
// `Sound FX parameters` node hands the game's value in.
|
||
await showTab('scene');
|
||
await clickText('+');
|
||
await pickAssetType(t, 'Sound FX');
|
||
const fxCreated = await json(`{
|
||
tab: __gm.tab(),
|
||
current: __gm.soundfx.current(),
|
||
info: __gm.soundfx.info(),
|
||
preview: __gm.soundfx.preview(),
|
||
assets: __gm.state.project.assets.filter((a) => a.type === 'soundfx').map((a) => a.id),
|
||
}`);
|
||
check(
|
||
'the + menu creates a sound FX asset and opens its tab, with no Output to compile and no Play',
|
||
fxCreated.tab === 'soundfx' &&
|
||
fxCreated.assets.length === 1 &&
|
||
fxCreated.current === fxCreated.assets[0] &&
|
||
fxCreated.info === 'No Output node' &&
|
||
fxCreated.preview.sound === soundId &&
|
||
fxCreated.preview.play === false,
|
||
JSON.stringify(fxCreated),
|
||
);
|
||
const fxId = fxCreated.assets[0];
|
||
const fxCanvas = await json('__gm.soundfx.canvasRect()');
|
||
await click(fxCanvas.x + 80, fxCanvas.y + 60, 'right');
|
||
const fxPalette = await json('__gm.overlayTexts()');
|
||
check(
|
||
'its palette offers the sound effects and no behavior node',
|
||
['Input', 'Output', 'Gain', 'Pitch', '5-Band-EQ', 'Mix', 'Delay'].every((label) =>
|
||
fxPalette.includes(label),
|
||
) && !fxPalette.includes('Branch'),
|
||
JSON.stringify(fxPalette.slice(0, 12)),
|
||
);
|
||
await press('Escape');
|
||
await mutate(`
|
||
const fx = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)});
|
||
fx.name = 'Low';
|
||
fx.graph = ${JSON.stringify({
|
||
nodes: [
|
||
node('fx-in', 'SoundFxInput', 0, 0, {}, [{ key: 'depth', kind: 'number', default: -12 }]),
|
||
node('fx-pitch', 'SoundFxPitch', 220, 0),
|
||
node('fx-gain', 'SoundFxGain', 440, 0, { gain: -6 }),
|
||
node('fx-out', 'SoundFxOutput', 660, 0),
|
||
],
|
||
edges: [
|
||
edge('fx-e1', 'fx-in', 'out', 'fx-pitch', 'in'),
|
||
edge('fx-e2', 'fx-pitch', 'out', 'fx-gain', 'in'),
|
||
edge('fx-e3', 'fx-gain', 'out', 'fx-out', 'in'),
|
||
edge('fx-e4', 'fx-in', 'depth', 'fx-pitch', 'semitones'),
|
||
],
|
||
})};`);
|
||
const fxBuilt = await json(
|
||
'{ info: __gm.soundfx.info(), preview: __gm.soundfx.preview(), params: __gm.soundfx.params() }',
|
||
);
|
||
check(
|
||
'Input → Pitch → Gain → Output compiles to two effects, Play is offered, depth starts at its default',
|
||
fxBuilt.info === '2 effects' && fxBuilt.preview.play === true && fxBuilt.params.depth === -12,
|
||
JSON.stringify(fxBuilt),
|
||
);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeFx = await stats();
|
||
const fxPlayAt = await json('__gm.soundfx.playAt()');
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
const heardFx = await waitFor(stats, (s) => s.started > beforeFx.started);
|
||
const blipSeconds = await json(`__gm.sound.duration(${JSON.stringify(blip.file)})`);
|
||
check(
|
||
'the tab’s Play sends the sound through the graph: an octave down, as long as before, one gain stage',
|
||
heardFx.last?.file === blip.file &&
|
||
heardFx.last.pitches.join() === '-12' &&
|
||
heardFx.last.duration === blipSeconds &&
|
||
heardFx.last.stages === 1,
|
||
JSON.stringify({ ...heardFx, blipSeconds }),
|
||
);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
await evaluate('__gm.soundfx.setParam("depth", 5)');
|
||
const beforeParam = await stats();
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
const heardParam = await waitFor(stats, (s) => s.started > beforeParam.started);
|
||
check(
|
||
'the header plays with the value its depth field is set to',
|
||
heardParam.last?.pitches.join() === '5' && heardParam.last.stages === 1,
|
||
JSON.stringify(heardParam),
|
||
);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
// #316: the Pitch → Gain way and the dry Input meet in a Mix – two sources, one per pitch.
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)}).graph;
|
||
graph.nodes.push(${JSON.stringify(node('fx-mix', 'SoundFxMix', 550, 0, { mix: 0.5 }))});
|
||
graph.edges = graph.edges.filter((e) => e.id !== 'fx-e3');
|
||
graph.edges.push(
|
||
${JSON.stringify(edge('fx-e5', 'fx-gain', 'out', 'fx-mix', 'a'))},
|
||
${JSON.stringify(edge('fx-e6', 'fx-in', 'out', 'fx-mix', 'b'))},
|
||
${JSON.stringify(edge('fx-e7', 'fx-mix', 'out', 'fx-out', 'in'))},
|
||
);`);
|
||
const mixInfo = await json('__gm.soundfx.info()');
|
||
const beforeMix = await stats();
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
const heardMix = await waitFor(stats, (s) => s.started > beforeMix.started);
|
||
check(
|
||
'a Mix of the pitched way and the dry Input plays one voice from two sources of the same length',
|
||
mixInfo === '3 effects' &&
|
||
heardMix.last?.pitches.join() === '5,0' &&
|
||
heardMix.last.stages === 1 &&
|
||
heardMix.last.duration === blipSeconds,
|
||
JSON.stringify({ mixInfo, ...heardMix }),
|
||
);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)}).graph;
|
||
graph.nodes = graph.nodes.filter((n) => n.id !== 'fx-mix');
|
||
graph.edges = graph.edges.filter((e) => !['fx-e5', 'fx-e6', 'fx-e7'].includes(e.id));
|
||
graph.edges.push(${JSON.stringify(edge('fx-e3', 'fx-gain', 'out', 'fx-out', 'in'))});`);
|
||
// #324, #327: a ping-pong Delay behind the gain. The blip is a quarter of a second, its echoes come
|
||
// 0.3 s apart, each half as loud as the one before: nine are heard, 2.7 s – the voice
|
||
// stays that long after its sound and goes then. Asked to let ring, only the sound
|
||
// stops and the voice stays; a stop after that cuts it.
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)}).graph;
|
||
graph.nodes.push(${JSON.stringify(
|
||
node('fx-delay', 'SoundFxDelay', 550, 0, { time: 0.3, feedback: 0.5, mix: 0.5, pingPong: true }),
|
||
)});
|
||
graph.edges = graph.edges.filter((e) => e.id !== 'fx-e3');
|
||
graph.edges.push(
|
||
${JSON.stringify(edge('fx-e8', 'fx-gain', 'out', 'fx-delay', 'in'))},
|
||
${JSON.stringify(edge('fx-e9', 'fx-delay', 'out', 'fx-out', 'in'))},
|
||
);`);
|
||
const delayInfo = await json('__gm.soundfx.info()');
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeEcho = await stats();
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
const echoStarted = await waitFor(stats, (s) => s.started > beforeEcho.started);
|
||
// Timed on the audio clock: the page's can fall behind it (task 523 saw a 2.95 s tail end 1 s in).
|
||
const audioNow = () => json('({ playing: __gm.sound.stats().playing, clock: __gm.sound.clock() })');
|
||
const startedAt = (await audioNow()).clock;
|
||
const echoLevels = await json(`__gm.sound.levels(${JSON.stringify(blip.file)})`);
|
||
const ringing = await waitFor(audioNow, (s) => s.clock - startedAt >= 1);
|
||
const quiet = await waitFor(audioNow, (s) => s.playing === 0, 40);
|
||
const rangFor = quiet.clock - startedAt;
|
||
const echoKnobs = echoLevels?.[0]?.slice(2) ?? [];
|
||
check(
|
||
'a Delay keeps the voice after its sound has ended, until its echoes have faded',
|
||
delayInfo === '3 effects' &&
|
||
echoStarted.last?.stages === 2 &&
|
||
echoKnobs.length === 4 &&
|
||
Math.abs(echoKnobs[0] - 0.3) < 1e-6 &&
|
||
Math.abs(echoKnobs[1] - 0.5) < 1e-6 &&
|
||
Math.abs(echoKnobs[2] / (20000 * 0.025 ** 0.3) - 1) < 1e-3 &&
|
||
Math.abs(echoKnobs[3] - 0.5) < 1e-6 &&
|
||
ringing.playing === 1 &&
|
||
ringing.clock - startedAt < 2.4 &&
|
||
quiet.playing === 0 &&
|
||
rangFor > 2.4 &&
|
||
rangFor < 4.5,
|
||
JSON.stringify({ delayInfo, last: echoStarted.last, echoLevels, ringing, startedAt, rangFor }),
|
||
);
|
||
await click(fxPlayAt.x, fxPlayAt.y);
|
||
await waitFor(stats, (s) => s.playing === 1);
|
||
const beforeLetRing = await stats();
|
||
await evaluate(`__gm.sound.stop(${JSON.stringify(blip.file)}, true)`);
|
||
await sleep(500);
|
||
const letRing = await stats();
|
||
await evaluate(`__gm.sound.stop(${JSON.stringify(blip.file)})`);
|
||
const cut = await stats();
|
||
check(
|
||
'Let ring stops the sound and keeps the voice for its echoes; a stop after that cuts it',
|
||
letRing.playing === 1 &&
|
||
letRing.stopped === beforeLetRing.stopped + 1 &&
|
||
cut.playing === 0 &&
|
||
cut.stopped === beforeLetRing.stopped + 2,
|
||
JSON.stringify({ before: beforeLetRing.stopped, letRing, cut }),
|
||
);
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(fxId)}).graph;
|
||
graph.nodes = graph.nodes.filter((n) => n.id !== 'fx-delay');
|
||
graph.edges = graph.edges.filter((e) => !['fx-e8', 'fx-e9'].includes(e.id));
|
||
graph.edges.push(${JSON.stringify(edge('fx-e3', 'fx-gain', 'out', 'fx-out', 'in'))});`);
|
||
await showTab('scene');
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
Object.assign(graph.nodes.find((n) => n.id === 'sn-play').params, { loop: 'false', fx: ${JSON.stringify(fxId)} });`);
|
||
const beforeGameFx = await stats();
|
||
await play();
|
||
const gameFx = await waitFor(stats, (s) => s.started > beforeGameFx.started);
|
||
check(
|
||
'a Play sound naming the asset plays the game’s voice through it',
|
||
gameFx.last?.pitches.join() === '-12' && gameFx.last.stages === 1,
|
||
JSON.stringify(gameFx),
|
||
);
|
||
await stop();
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.push(${JSON.stringify(
|
||
node('sn-fxp', 'SoundFxParameters', 0, 120, { soundFxAsset: fxId, depth: 7 }, [
|
||
{ key: 'depth', kind: 'number' },
|
||
]),
|
||
)});
|
||
graph.edges.push(${JSON.stringify(edge('sn-e-fxp', 'sn-fxp', 'params', 'sn-play', 'fxParams'))});`);
|
||
const beforeGameParam = await stats();
|
||
await play();
|
||
const gameParam = await waitFor(stats, (s) => s.started > beforeGameParam.started);
|
||
check(
|
||
'a Sound FX parameters node hands depth 7 to the game’s voice',
|
||
gameParam.last?.pitches.join() === '7' && gameParam.last.stages === 1,
|
||
JSON.stringify(gameParam),
|
||
);
|
||
await stop();
|
||
// #315: two looping voices of the same sound start at depth 7, the first keeps its number in a
|
||
// variable, and `Set sound FX parameters` hands that one voice depth 3 every frame. In a
|
||
// real AudioContext the first one's pitch – the AudioParam of its worklet (#317), in
|
||
// semitones – glides there, the second one's stays put.
|
||
const tuneIds = ['sn-keep', 'sn-play2', 'sn-tick', 'sn-read', 'sn-tune', 'sn-fxp2'];
|
||
const tuneEdges = [
|
||
'sn-e-keep',
|
||
'sn-e-voice',
|
||
'sn-e-play2',
|
||
'sn-e-fxp3',
|
||
'sn-e-tick',
|
||
'sn-e-read',
|
||
'sn-e-fxp2',
|
||
];
|
||
await mutate(`
|
||
const logic = __gm.state.project.assets.find((a) => a.id === 'sn-logic');
|
||
logic.variables = [${JSON.stringify(variable('voice', 'number', 0))}];
|
||
const graph = logic.graph;
|
||
const play = graph.nodes.find((n) => n.id === 'sn-play');
|
||
play.params.loop = 'true';
|
||
graph.nodes.push(
|
||
${JSON.stringify(node('sn-keep', 'Variables', 520, 0, {}, [written('voice', 'number')]))},
|
||
{ ...structuredClone(play), id: 'sn-play2', x: 780 },
|
||
${JSON.stringify(node('sn-tick', 'OnUpdate', 0, 300))},
|
||
${JSON.stringify(node('sn-read', 'Variables', 0, 420, {}, [port('voice', 'number')]))},
|
||
${JSON.stringify(node('sn-tune', 'SetSoundFxParameters', 260, 300))},
|
||
${JSON.stringify(
|
||
node('sn-fxp2', 'SoundFxParameters', 0, 540, { soundFxAsset: fxId, depth: 3 }, [
|
||
{ key: 'depth', kind: 'number' },
|
||
]),
|
||
)},
|
||
);
|
||
graph.edges.push(
|
||
${JSON.stringify(edge('sn-e-keep', 'sn-play', 'exec', 'sn-keep', 'exec'))},
|
||
${JSON.stringify(edge('sn-e-voice', 'sn-play', 'voice', 'sn-keep', 'voice'))},
|
||
${JSON.stringify(edge('sn-e-play2', 'sn-keep', 'exec', 'sn-play2', 'exec'))},
|
||
${JSON.stringify(edge('sn-e-fxp3', 'sn-fxp', 'params', 'sn-play2', 'fxParams'))},
|
||
${JSON.stringify(edge('sn-e-tick', 'sn-tick', 'exec', 'sn-tune', 'exec'))},
|
||
${JSON.stringify(edge('sn-e-read', 'sn-read', 'voice', 'sn-tune', 'voice'))},
|
||
${JSON.stringify(edge('sn-e-fxp2', 'sn-fxp2', 'params', 'sn-tune', 'fxParams'))},
|
||
);`);
|
||
const beforeTune = await stats();
|
||
await play();
|
||
const tunedStats = await waitFor(
|
||
stats,
|
||
(s) => s.tuned > beforeTune.tuned && s.started >= beforeTune.started + 2,
|
||
);
|
||
const glided = (levels) =>
|
||
Array.isArray(levels) && levels.length === 2 && Math.abs(levels[0][0] - 3) < 0.05;
|
||
const levels = await waitFor(() => json(`__gm.sound.levels(${JSON.stringify(blip.file)})`), glided);
|
||
check(
|
||
'Set sound FX parameters moves the one voice whose number it holds from depth 7 to 3, not the other',
|
||
tunedStats.last?.pitches.join() === '7' &&
|
||
tunedStats.last.loop === true &&
|
||
tunedStats.worklet === true &&
|
||
glided(levels) &&
|
||
Math.abs(levels[0][1] - 10 ** (-6 / 20)) < 0.001 &&
|
||
levels[1][0] === 7,
|
||
JSON.stringify({ last: tunedStats.last, tuned: tunedStats.tuned, worklet: tunedStats.worklet, levels }),
|
||
);
|
||
await stop();
|
||
// A `time` of 2 s takes the same voice from depth 7 to -5 evenly on the audio clock, 6 semitones
|
||
// a second: half a second in it is still well above the end, where a quick glide would be there.
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.find((n) => n.id === 'sn-tune').params.time = 2;
|
||
graph.nodes.find((n) => n.id === 'sn-fxp2').params.depth = -5;`);
|
||
await play();
|
||
const pitch = async () => (await json(`__gm.sound.levels(${JSON.stringify(blip.file)})`))?.[0]?.[0];
|
||
const early = await waitFor(pitch, (s) => typeof s === 'number' && s < 6.9);
|
||
const later = await waitFor(pitch, (s) => typeof s === 'number' && s < early - 0.6);
|
||
await new Promise((resolve) => setTimeout(resolve, 500));
|
||
const afterHalf = await pitch();
|
||
check(
|
||
'Set sound FX parameters with a time ramps the pitch over that time instead of jumping',
|
||
early > -1.8 && later > -3.3 && afterHalf > -4.3 && afterHalf < later,
|
||
JSON.stringify({ early, later, afterHalf }),
|
||
);
|
||
await stop();
|
||
// The easings of an animation key: Quad in hands the clock a curve, 7 - 3 * t^2 towards -5 in 2 s,
|
||
// where linear is 7 - 6 * t. Each reading is timed on the audio clock from the voice's start, so a
|
||
// first one that comes in late is held against where the curve is then. Between 0.3 and 1.7 s the
|
||
// curve is more than a semitone above linear, and it never falls below it.
|
||
await mutate(`
|
||
const tune = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph.nodes.find((n) => n.id === 'sn-tune');
|
||
Object.assign(tune.params, { ease: 'quad', direction: 'in' });`);
|
||
await play();
|
||
const pitchNow = () =>
|
||
json(`({ pitch: __gm.sound.levels(${JSON.stringify(blip.file)})?.[0]?.[0], clock: __gm.sound.clock() })`);
|
||
const sinceStart = async (seen, since = 'start') => {
|
||
const now = await stats();
|
||
const from = since === 'tune' ? now.tunedAt : now.last.at;
|
||
return seen
|
||
.filter((s) => typeof s.pitch === 'number')
|
||
.map((s) => ({ t: +(s.clock - from).toFixed(3), pitch: +s.pitch.toFixed(3) }));
|
||
};
|
||
const sweep = await sinceStart(await trace(pitchNow, (s) => typeof s.pitch === 'number' && s.pitch < -4.3));
|
||
const midway = sweep.filter(({ t }) => t > 0.3 && t < 1.7);
|
||
check(
|
||
'Set sound FX parameters with Quad in ramps slowly first, then on to the end',
|
||
midway.length > 0 &&
|
||
midway.every(({ t, pitch }) => pitch > 7 - 6 * t + 1) &&
|
||
sweep.every(({ t, pitch }) => pitch > 7 - 6 * t - 0.1) &&
|
||
sweep.at(-1).pitch < -4.3,
|
||
JSON.stringify(sweep),
|
||
);
|
||
await stop();
|
||
// Step holds the pitch for the whole time and then jumps there: every reading before 2 s on the
|
||
// audio clock is still 7, the first one past it is the end. Timed from the change, not from the
|
||
// start of the voice: task 529 saw it arrive 80 ms later and the 7 held, rightly, to 2.08 s.
|
||
await mutate(`
|
||
const tune = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph.nodes.find((n) => n.id === 'sn-tune');
|
||
Object.assign(tune.params, { ease: 'step', time: 2 });`);
|
||
await play();
|
||
const steps = await sinceStart(
|
||
await trace(pitchNow, (s) => typeof s.pitch === 'number' && s.pitch < 6),
|
||
'tune',
|
||
);
|
||
const held = steps.filter(({ pitch }) => pitch === 7);
|
||
const jumped = steps.find(({ pitch }) => pitch < 6);
|
||
check(
|
||
'Set sound FX parameters with Step holds the pitch for the time, then jumps to the end',
|
||
held.length > 0 &&
|
||
held.every(({ t }) => t < 2.05) &&
|
||
jumped !== undefined &&
|
||
jumped.t > 1.95 &&
|
||
Math.abs(jumped.pitch + 5) < 0.001,
|
||
JSON.stringify(steps),
|
||
);
|
||
await stop();
|
||
// #317: the worklet itself, rendered in an offline context of this browser – what a muted check
|
||
// can hear. A second of 440 Hz an octave up is 880 Hz and still a second long; a sweep from
|
||
// 0 to -12 over it keeps that second too. Then 16 stereo voices at +5 for 2 s, timed, for
|
||
// how many this machine plays at once.
|
||
const offline = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const render = async (voices, seconds, channels, shape) => {
|
||
const context = new OfflineAudioContext(channels, RATE * seconds, RATE);
|
||
await context.audioWorklet.addModule(__gm.sound.pitchWorkletUrl());
|
||
const tone = new Float32Array(RATE * seconds);
|
||
for (let i = 0; i < tone.length; i++) tone[i] = Math.sin((2 * Math.PI * 440 * i) / RATE) * 0.5;
|
||
for (let v = 0; v < voices; v++) {
|
||
const node = new AudioWorkletNode(context, 'pitch-player', {
|
||
numberOfInputs: 0,
|
||
outputChannelCount: [channels],
|
||
parameterData: { pitch: shape.from },
|
||
processorOptions: { file: 'probe', channels: v === 0 ? Array(channels).fill(tone) : undefined, step: 1, loop: false },
|
||
});
|
||
if (shape.to !== shape.from)
|
||
node.parameters.get('pitch').setValueAtTime(shape.from, 0).linearRampToValueAtTime(shape.to, seconds);
|
||
node.connect(context.destination);
|
||
}
|
||
const started = performance.now();
|
||
const out = (await context.startRendering()).getChannelData(0);
|
||
return { out, ms: performance.now() - started };
|
||
};
|
||
const rises = (out, from, to) => {
|
||
let count = 0;
|
||
for (let i = from + 1; i < to; i++) if (out[i - 1] < 0 && out[i] >= 0) count++;
|
||
return count / ((to - from) / RATE);
|
||
};
|
||
const heard = (out) => {
|
||
let last = out.length;
|
||
while (last > 0 && Math.abs(out[last - 1]) <= 1e-4) last--;
|
||
return last;
|
||
};
|
||
const up = await render(1, 1, 1, { from: 12, to: 12 });
|
||
const sweep = await render(1, 1, 1, { from: 0, to: -12 });
|
||
const load = await render(16, 2, 2, { from: 5, to: 5 });
|
||
return {
|
||
upHz: rises(up.out, 12000, 36000),
|
||
upLength: heard(up.out),
|
||
sweepEarlyHz: rises(sweep.out, 6000, 12000),
|
||
sweepLateHz: rises(sweep.out, 36000, 42000),
|
||
sweepLength: heard(sweep.out),
|
||
voices: 16,
|
||
renderMs: Math.round(load.ms),
|
||
realTimeVoices: Math.round((16 * 2000) / load.ms),
|
||
};
|
||
})()`);
|
||
check(
|
||
'the pitch worklet plays an octave up at the length of the sound, and a sweep down keeps it too',
|
||
Math.abs(offline.upHz / 880 - 1) < 0.02 &&
|
||
offline.upLength > 48000 - 100 &&
|
||
offline.sweepEarlyHz > offline.sweepLateHz * 1.3 &&
|
||
offline.sweepLength > 48000 - 100,
|
||
JSON.stringify(offline),
|
||
);
|
||
// #327: the player's own echo, rendered offline. A 10 ms burst into a ping-pong Delay (0.3 s,
|
||
// feedback 0.5, mix 0.5) comes back left at 0.3 s at half its level, right at 0.6 s at
|
||
// half of that, left again at 0.9 s – in Chrome each round 128 samples later than that. Then
|
||
// 64 stereo voices of it for 2 s, timed, for how many this machine plays at once (#324).
|
||
const echo = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const echoOf = (pingPong) => ({ kind: 'delay', time: 0.3, feedback: 0.5, damping: 20000, mix: 0.5, pingPong });
|
||
const render = async (voices, seconds, stage = echoOf(true)) => {
|
||
const context = new OfflineAudioContext(2, RATE * seconds, RATE);
|
||
const burst = context.createBuffer(1, RATE * 0.01, RATE);
|
||
burst.getChannelData(0).fill(0.5);
|
||
for (let v = 0; v < voices; v++) {
|
||
const source = context.createBufferSource();
|
||
source.buffer = burst;
|
||
const nodes = __gm.sound.stageNodes(context, stage);
|
||
source.connect(nodes.input);
|
||
nodes.output.connect(context.destination);
|
||
source.start();
|
||
}
|
||
const started = performance.now();
|
||
const out = await context.startRendering();
|
||
return { left: out.getChannelData(0), right: out.getChannelData(1), ms: performance.now() - started };
|
||
};
|
||
// The middle of the burst: its edges ring a little through the damping.
|
||
const peak = (data, at) => {
|
||
let sum = 0;
|
||
const from = Math.round((at + 0.003) * RATE);
|
||
const to = Math.round((at + 0.007) * RATE);
|
||
for (let i = from; i < to; i++) sum += Math.abs(data[i]);
|
||
return Math.round((sum / (to - from)) * 1000) / 1000;
|
||
};
|
||
// Samples from where each echo is due to where it is heard, in both kinds of echo.
|
||
const late = ({ left, right }) => {
|
||
const found = [];
|
||
for (let k = 1; k <= 4; k++) {
|
||
const from = Math.round((k * 0.3 - 0.05) * RATE);
|
||
let at = from;
|
||
while (at < from + RATE * 0.1 && Math.abs(left[at]) <= 0.002 && Math.abs(right[at]) <= 0.002) at++;
|
||
found.push(at - Math.round(k * 0.3 * RATE));
|
||
}
|
||
return found;
|
||
};
|
||
const one = await render(1, 1.3);
|
||
const plain = await render(1, 1.3, echoOf(false));
|
||
const load = await render(64, 2);
|
||
const lateBy = late(one);
|
||
const at = (k) => k * 0.3 + lateBy[k - 1] / RATE;
|
||
return {
|
||
late: lateBy,
|
||
latePlain: late(plain),
|
||
dry: [peak(one.left, 0), peak(one.right, 0)],
|
||
first: [peak(one.left, at(1)), peak(one.right, at(1))],
|
||
second: [peak(one.left, at(2)), peak(one.right, at(2))],
|
||
third: [peak(one.left, at(3)), peak(one.right, at(3))],
|
||
between: [peak(one.left, 0.45), peak(one.right, 0.45)],
|
||
voices: 64,
|
||
renderMs: Math.round(load.ms),
|
||
realTimeVoices: Math.round((64 * 2000) / load.ms),
|
||
};
|
||
})()`);
|
||
const near = (value, to) => Math.abs(value - to) < 0.02;
|
||
check(
|
||
'the player’s ping-pong echo comes back left, then right, then left, each round half as loud',
|
||
near(echo.dry[0], 0.5) &&
|
||
near(echo.dry[1], 0.5) &&
|
||
near(echo.first[0], 0.25) &&
|
||
echo.first[1] < 0.005 &&
|
||
echo.second[0] < 0.005 &&
|
||
near(echo.second[1], 0.125) &&
|
||
near(echo.third[0], 0.0625) &&
|
||
echo.third[1] < 0.005 &&
|
||
echo.between.every((v) => v < 0.005) &&
|
||
// Chrome takes a render quantum more for every round of a feedback loop.
|
||
[echo.late, echo.latePlain].every((late) => late.every((n, k) => n === 0 || n === k * 128)),
|
||
JSON.stringify(echo),
|
||
);
|
||
// A Mix "Both at center", rendered offline: a steady 1 on a and 0.5 on b in two channels, so each
|
||
// share reads off as it is. Then mix glides 0 → 1 over a second: at its middle both stay full.
|
||
const atCenter = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const render = async (mix, glide) => {
|
||
const context = new OfflineAudioContext(2, RATE, RATE);
|
||
const merger = context.createChannelMerger(2);
|
||
merger.connect(context.destination);
|
||
const shareA = context.createGain();
|
||
const shareB = context.createGain();
|
||
shareA.gain.value = 0;
|
||
shareB.gain.value = 0;
|
||
const driver = __gm.sound.bothAtCenter(context, mix, shareA, shareB);
|
||
for (const [share, level, side] of [[shareA, 1, 0], [shareB, 0.5, 1]]) {
|
||
const steady = context.createConstantSource();
|
||
steady.offset.value = level;
|
||
steady.connect(share).connect(merger, 0, side);
|
||
steady.start();
|
||
}
|
||
if (glide) driver.offset.setValueAtTime(0, 0).linearRampToValueAtTime(1, 1);
|
||
driver.start();
|
||
const out = await context.startRendering();
|
||
const at = (s) => [out.getChannelData(0)[Math.round(s * RATE)], out.getChannelData(1)[Math.round(s * RATE)] * 2];
|
||
return glide ? [0.25, 0.5, 0.75].map(at) : at(0.5);
|
||
};
|
||
const round = (pair) => pair.map((v) => Math.round(v * 1000) / 1000);
|
||
const fixed = {};
|
||
for (const mix of [0, 0.25, 0.5, 0.75, 1]) fixed[mix] = round(await render(mix, false));
|
||
return { fixed, glide: (await render(0, true)).map(round) };
|
||
})()`);
|
||
const pairNear = (pair, a, b) => near(pair[0], a) && near(pair[1], b);
|
||
check(
|
||
'a Mix "Both at center" keeps a full up to the middle and b full from it, also while mix glides',
|
||
pairNear(atCenter.fixed[0], 1, 0) &&
|
||
pairNear(atCenter.fixed[0.25], 1, 0.5) &&
|
||
pairNear(atCenter.fixed[0.5], 1, 1) &&
|
||
pairNear(atCenter.fixed[0.75], 0.5, 1) &&
|
||
pairNear(atCenter.fixed[1], 0, 1) &&
|
||
pairNear(atCenter.glide[0], 1, 0.5) &&
|
||
pairNear(atCenter.glide[1], 1, 1) &&
|
||
pairNear(atCenter.glide[2], 0.5, 1),
|
||
JSON.stringify(atCenter),
|
||
);
|
||
await mutate(`
|
||
const logic = __gm.state.project.assets.find((a) => a.id === 'sn-logic');
|
||
logic.variables = [];
|
||
const graph = logic.graph;
|
||
graph.nodes = graph.nodes.filter((n) => !${JSON.stringify(tuneIds)}.includes(n.id));
|
||
graph.edges = graph.edges.filter((e) => !${JSON.stringify(tuneEdges)}.includes(e.id));`);
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes = graph.nodes.filter((n) => n.id !== 'sn-fxp');
|
||
graph.edges = graph.edges.filter((e) => e.id !== 'sn-e-fxp');
|
||
Object.assign(graph.nodes.find((n) => n.id === 'sn-play').params, { loop: '', fx: '' });
|
||
const project = __gm.state.project;
|
||
project.assets = project.assets.filter((a) => a.id !== ${JSON.stringify(fxId)});`);
|
||
|
||
// 34g-5. An instrument (#331): "+ → Instrument" opens its tab on Notes → Sample → Output, which
|
||
// plays nothing until the Sample has a sound. With the blip, looped, every key of C4–B4
|
||
// sounds; a key held by the pointer plays until it is let go, the computer keyboard plays
|
||
// as a tracker does, and a game's `Play note` releases its note after its duration.
|
||
await showTab('scene');
|
||
await clickText('+');
|
||
await pickAssetType(t, 'Instrument');
|
||
const insCreated = await json(`{
|
||
tab: __gm.tab(),
|
||
current: __gm.instrument.current(),
|
||
info: __gm.instrument.info(),
|
||
keys: __gm.instrument.keys(),
|
||
assets: __gm.state.project.assets.filter((a) => a.type === 'instrument')
|
||
.map((a) => ({ id: a.id, nodes: a.graph.nodes.map((n) => n.type) })),
|
||
}`);
|
||
check(
|
||
'the + menu creates an instrument, Notes → Sample → Output, and opens its tab; no key sounds yet',
|
||
insCreated.tab === 'instrument' &&
|
||
insCreated.assets.length === 1 &&
|
||
insCreated.current === insCreated.assets[0].id &&
|
||
insCreated.assets[0].nodes.join() === 'InstrumentNotes,InstrumentSample,InstrumentOutput' &&
|
||
insCreated.info === 'No sound picked' &&
|
||
insCreated.keys.sounding.length === 0,
|
||
JSON.stringify(insCreated),
|
||
);
|
||
const insId = insCreated.assets[0]?.id;
|
||
// Bottom left: the fitted graph is one row of three nodes, its links among them.
|
||
const insCanvas = await json('__gm.instrument.canvasRect()');
|
||
await click(insCanvas.x + 30, insCanvas.y + insCanvas.h - 30, 'right');
|
||
const insPalette = await json('__gm.overlayTexts()');
|
||
check(
|
||
'its palette offers its own nodes and the effects, but no Input and no behavior node',
|
||
['Notes', 'Sample', 'Parameters', 'Sound FX', 'Envelope', 'Gain', 'Filter', 'Delay'].every((label) =>
|
||
insPalette.includes(label),
|
||
) &&
|
||
!insPalette.includes('Input') &&
|
||
!insPalette.includes('Branch'),
|
||
JSON.stringify(insPalette.slice(0, 14)),
|
||
);
|
||
await press('Escape');
|
||
await mutate(`
|
||
const ins = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(insId)});
|
||
Object.assign(ins.graph.nodes.find((n) => n.type === 'InstrumentSample').params, {
|
||
sound: ${JSON.stringify(soundId)},
|
||
play: 'loop',
|
||
});`);
|
||
const insReady = await json('{ info: __gm.instrument.info(), keys: __gm.instrument.keys() }');
|
||
check(
|
||
'with a sound picked the twelve keys from C4 to B4 sound, the others stay grey',
|
||
insReady.info === '12 notes' &&
|
||
insReady.keys.sounding.join() === [60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71].join(),
|
||
JSON.stringify(insReady),
|
||
);
|
||
await shot('34-instrument');
|
||
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeKey = await stats();
|
||
const e4 = await json('__gm.instrument.keyAt(64)');
|
||
await t.mouse('mouseMoved', e4.x, e4.y);
|
||
await t.mouse('mousePressed', e4.x, e4.y);
|
||
const heldKey = await waitFor(stats, (s) => s.started > beforeKey.started);
|
||
// Longer than the blip: a looped note that ended with its sound would be gone by now.
|
||
await sleep(600);
|
||
const stillHeld = await json('{ stats: __gm.sound.stats(), keys: __gm.instrument.keys() }');
|
||
check(
|
||
'holding E4 plays the blip four semitones up, looped, as one layer of the instrument, and lights the key',
|
||
heldKey.last?.file === `instrument:${insId}` &&
|
||
heldKey.last.pitches.join() === '4' &&
|
||
heldKey.last.layers === 1 &&
|
||
stillHeld.stats.playing === 1 &&
|
||
stillHeld.keys.held.join() === '64',
|
||
JSON.stringify({ heldKey, stillHeld }),
|
||
);
|
||
await t.mouse('mouseReleased', e4.x, e4.y);
|
||
const letGo = await waitFor(stats, (s) => s.playing === 0);
|
||
check(
|
||
'letting go of the key ends the note: its sample fades out and the voice goes',
|
||
letGo.playing === 0 && (await json('__gm.instrument.keys()')).held.length === 0,
|
||
JSON.stringify(letGo),
|
||
);
|
||
|
||
// "Loop, then release": held, it loops 0.05 to 0.1 s; let go, it plays the 0.15 s after the loop.
|
||
const setPlay = (fields) =>
|
||
mutate(`
|
||
const ins = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(insId)});
|
||
Object.assign(ins.graph.nodes.find((n) => n.type === 'InstrumentSample').params, ${JSON.stringify(fields)});`);
|
||
await setPlay({ play: 'release', loopStart: 0.05, loopEnd: 0.1 });
|
||
const beforeRelease = await stats();
|
||
await t.mouse('mousePressed', e4.x, e4.y);
|
||
await waitFor(stats, (s) => s.started > beforeRelease.started);
|
||
await sleep(600);
|
||
const heldLoop = await stats();
|
||
await t.mouse('mouseReleased', e4.x, e4.y);
|
||
const leftAt = await audioNow();
|
||
const ringingKeys = await json('__gm.instrument.keys()');
|
||
const playedOut = await waitFor(audioNow, (s) => s.playing === 0);
|
||
const afterKeys = await waitFor(
|
||
() => json('__gm.instrument.keys()'),
|
||
(keys) => keys.ringing.length === 0,
|
||
);
|
||
// 0.15 s after the loop, up to two rounds more (the one it is in, the one the worklet read into)
|
||
// and the 150 ms waitFor polls at: a fade would be gone in 0.01 s.
|
||
const outFor = playedOut.clock - leftAt.clock;
|
||
check(
|
||
'"Loop, then release" loops while held, and let go plays on past the loop with its key half lit',
|
||
heldLoop.playing === 1 &&
|
||
ringingKeys.held.length === 0 &&
|
||
ringingKeys.ringing.join() === '64' &&
|
||
outFor > 0.12 &&
|
||
outFor < 0.5 &&
|
||
afterKeys.ringing.length === 0,
|
||
JSON.stringify({ heldLoop: heldLoop.playing, ringingKeys, outFor, afterKeys }),
|
||
);
|
||
await setPlay({ play: 'loop', loopStart: 0, loopEnd: 0 });
|
||
|
||
// The click left the keys focused. Z is C3 there – no sound in this instrument –, one octave up C4.
|
||
const beforeZ = await stats();
|
||
await press('z');
|
||
await sleep(200);
|
||
const lowZ = await stats();
|
||
await press('ArrowRight');
|
||
await t.keyDown('z');
|
||
const typed = await waitFor(stats, (s) => s.started > lowZ.started);
|
||
const typedKeys = await json('__gm.instrument.keys()');
|
||
await t.keyUp('z');
|
||
const typedOff = await waitFor(stats, (s) => s.playing === 0);
|
||
check(
|
||
'the computer keyboard plays as a tracker: Z is C3 and silent here, one octave up it is C4 until let go',
|
||
lowZ.started === beforeZ.started &&
|
||
typedKeys.octave === 4 &&
|
||
typedKeys.held.join() === '60' &&
|
||
typed.last?.pitches.join() === '0' &&
|
||
typedOff.playing === 0,
|
||
JSON.stringify({ beforeZ: beforeZ.started, lowZ: lowZ.started, typed, typedKeys, typedOff }),
|
||
);
|
||
await press('ArrowLeft');
|
||
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes.push(${JSON.stringify(
|
||
node('sn-note', 'PlayNote', 260, 220, { instrument: insId, note: 67, duration: 1.5 }),
|
||
)});
|
||
graph.edges.find((e) => e.id === 'sn-e1').to = { node: 'sn-note', port: 'exec' };`);
|
||
await showTab('scene');
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeNote = await stats();
|
||
await play();
|
||
// The sample loops, so only the release after 1.5 s on the audio clock ends the note: it is heard
|
||
// well past the 0.25 s of the blip, never past its duration, and is gone soon after.
|
||
const noteNow = () => json('({ stats: __gm.sound.stats(), clock: __gm.sound.clock() })');
|
||
const noted = await waitFor(noteNow, (s) => s.stats.started > beforeNote.started);
|
||
const heard = await trace(noteNow, (s) => s.stats.playing === 0);
|
||
const noteAt = noted.stats.last?.at ?? 0;
|
||
const heardAt = heard.filter((s) => s.stats.playing === 1).map((s) => +(s.clock - noteAt).toFixed(3));
|
||
const goneAt = +(heard.at(-1).clock - noteAt).toFixed(3);
|
||
check(
|
||
'a game’s Play note plays G4 – the blip seven semitones up – and lets it go after its duration',
|
||
noted.stats.last?.file === `instrument:${insId}` &&
|
||
noted.stats.last.pitches.join() === '7' &&
|
||
heardAt.some((t) => t > 0.5) &&
|
||
heardAt.every((t) => t < 1.6) &&
|
||
heard.at(-1).stats.playing === 0 &&
|
||
goneAt < 3,
|
||
JSON.stringify({ last: noted.stats.last, heardAt, goneAt }),
|
||
);
|
||
await stop();
|
||
|
||
// #332: the scheduler, driven by the hook – nothing in the editor plays a sequence yet. A bar of
|
||
// quarters at 150 BPM in a loop: every note is planned before its second comes, each 0.4 s
|
||
// after the one before, also across the seam into the next round. Where the music stands
|
||
// is the audio clock less the output latency, counted from the first note.
|
||
const bar = [60, 62, 64, 65].map((pitch, i) => ({
|
||
tick: i * 480,
|
||
length: 240,
|
||
note: pitch,
|
||
velocity: 1,
|
||
instrument: insId,
|
||
}));
|
||
await evaluate(
|
||
`__gm.music.play(${JSON.stringify({ notes: bar })}, { tempo: 150, loop: { start: 0, end: 1920 } })`,
|
||
);
|
||
const planned = await waitFor(
|
||
() => json('__gm.music.planned()'),
|
||
(p) => p.length >= 9,
|
||
40,
|
||
);
|
||
const where = await json(
|
||
'({ position: __gm.music.position(), clock: __gm.sound.clock(), latency: __gm.music.latency() })',
|
||
);
|
||
await evaluate('__gm.music.stop()');
|
||
const musicOff = await waitFor(stats, (s) => s.playing === 0);
|
||
const apart = planned.slice(1, 9).map((p, i) => +(p.at - planned[i].at).toFixed(6));
|
||
const ahead = planned.map((p) => +(p.at - p.planned).toFixed(3));
|
||
const heardTicks = (((where.clock - where.latency - planned[0].at) * 150 * 480) / 60) % 1920;
|
||
check(
|
||
'the scheduler plans a looped bar on the audio clock: every note ahead of its second, 0.4 s apart across the seam',
|
||
planned.length >= 9 &&
|
||
planned
|
||
.slice(0, 9)
|
||
.map((p) => p.note)
|
||
.join() === '60,62,64,65,60,62,64,65,60' &&
|
||
planned.every((p) => p.voice > 0 && Math.abs(p.duration - 0.2) < 1e-9 && p.tempo === 150) &&
|
||
apart.every((gap) => Math.abs(gap - 0.4) < 1e-6) &&
|
||
ahead.every((s) => s > 0 && s <= 0.16) &&
|
||
Math.abs(where.position.ticks - heardTicks) < 1 &&
|
||
where.position.bar === 1 &&
|
||
musicOff.playing === 0 &&
|
||
(await json('__gm.music.playing()')) === false,
|
||
JSON.stringify({ apart, ahead, where, heardTicks, playing: musicOff.playing }),
|
||
);
|
||
|
||
// A note planned ahead through the pitch worklet starts on its sample, also inside a render block:
|
||
// rendered offline at pitch 0, which plays the sound as it is, so the first sample heard is it.
|
||
const onsets = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const onset = async (startAt) => {
|
||
const context = new OfflineAudioContext(1, RATE / 2, RATE);
|
||
await context.audioWorklet.addModule(__gm.sound.pitchWorkletUrl());
|
||
const node = new AudioWorkletNode(context, 'pitch-player', {
|
||
numberOfInputs: 0,
|
||
outputChannelCount: [1],
|
||
parameterData: { pitch: 0 },
|
||
processorOptions: { file: 'onset', channels: [new Float32Array(RATE).fill(0.5)], step: 1, loop: false, startAt },
|
||
});
|
||
node.connect(context.destination);
|
||
const out = (await context.startRendering()).getChannelData(0);
|
||
return out.findIndex((v) => Math.abs(v) > 1e-6);
|
||
};
|
||
const at = [0.0123, 0.25, 0.05 + 0.4 * 2 / 3];
|
||
return { expected: at.map((s) => Math.round(s * RATE)), heard: await Promise.all(at.map(onset)) };
|
||
})()`);
|
||
check(
|
||
'a pitched note planned ahead starts on its very sample, inside a render block too',
|
||
onsets.heard.join() === onsets.expected.join() && onsets.expected.some((i) => i % 128 !== 0),
|
||
JSON.stringify(onsets),
|
||
);
|
||
|
||
// #335: an Envelope's curve as Chrome renders it – up in the attack, easing towards the sustain,
|
||
// and from wherever it is when the note ends down to silence in the release, mid-attack too –
|
||
// and a Gain it moves through a WaveShaper, from a quarter to full in equal ratios.
|
||
const curves = await evaluate(`
|
||
(async () => {
|
||
const RATE = 48000;
|
||
const envelope = { id: 'e', attack: 0.1, decay: 0.2, sustain: 0.5, release: 0.3 };
|
||
const render = async (releaseAt, moved) => {
|
||
const context = new OfflineAudioContext(1, RATE, RATE);
|
||
const driver = __gm.sound.envelopeDriver(context, envelope, 0);
|
||
if (releaseAt !== null) __gm.sound.releaseEnvelope(driver.offset, envelope, 0, releaseAt);
|
||
if (moved) {
|
||
const steady = context.createConstantSource();
|
||
const stage = { kind: 'gain', gain: 1, mod: { envelope, from: 0.25, to: 1 } };
|
||
const made = __gm.sound.stageWith(context, stage, driver);
|
||
steady.connect(made.input);
|
||
made.output.connect(context.destination);
|
||
steady.start();
|
||
} else driver.connect(context.destination);
|
||
driver.start();
|
||
const out = (await context.startRendering()).getChannelData(0);
|
||
return (s) => Math.round(out[Math.round(s * RATE)] * 10000) / 10000;
|
||
};
|
||
const held = await render(0.5, false);
|
||
const early = await render(0.05, false);
|
||
const gain = await render(null, true);
|
||
return {
|
||
held: [0.05, 0.1, 0.3, 0.5, 0.65, 0.8].map(held),
|
||
early: [0.05, 0.15, 0.35].map(early),
|
||
gain: [0.05, 0.1, 0.9].map(gain),
|
||
};
|
||
})()`);
|
||
const closeTo = (values, expected) => values.every((v, i) => Math.abs(v - expected[i]) < 0.002);
|
||
check(
|
||
'an Envelope rises, eases to its sustain and releases from where it is, and moves a Gain with that curve',
|
||
closeTo(curves.held, [0.5, 1, 0.5005, 0.5, 0.0158, 0.0005]) &&
|
||
closeTo(curves.early, [0.5, 0.05, 0.0005]) &&
|
||
closeTo(curves.gain, [0.5, 1, 0.5]),
|
||
JSON.stringify(curves),
|
||
);
|
||
|
||
// A note of 0.2 s through an Envelope with a release of 1 s: the looped blip plays on through the
|
||
// release, and the voice goes with it – without one it would end 10 ms after the note.
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(insId)}).graph;
|
||
const into = graph.edges.find((e) => e.to.node === graph.nodes.find((n) => n.type === 'InstrumentOutput').id);
|
||
graph.nodes.push(${JSON.stringify(node('ins-env', 'InstrumentEnvelope', 300, 120, { release: 1 }))});
|
||
graph.edges.push({ id: 'ins-e-env', from: { node: 'ins-env', port: 'out' }, to: { ...into.to } });
|
||
into.to = { node: 'ins-env', port: 'in' };`);
|
||
await evaluate('__gm.sound.stopAll()');
|
||
const beforeShaped = await stats();
|
||
await evaluate(
|
||
`__gm.music.play(${JSON.stringify({ notes: [{ tick: 0, length: 240, note: 60, velocity: 1, instrument: insId }] })}, { tempo: 150 })`,
|
||
);
|
||
const shaped = await waitFor(noteNow, (s) => s.stats.started > beforeShaped.started);
|
||
const ringOut = await trace(noteNow, (s) => s.stats.playing === 0);
|
||
await evaluate('__gm.music.stop()');
|
||
const shapedAt = shaped.stats.last?.at ?? 0;
|
||
const shapedGone = +(ringOut.at(-1).clock - shapedAt).toFixed(3);
|
||
// 0.2 s of note and 1 s of release; up to the 150 ms waitFor polls at on top.
|
||
check(
|
||
'a note through an Envelope plays on through its release and goes with it',
|
||
shaped.stats.last?.envelopes === 1 &&
|
||
ringOut.some((s) => s.stats.playing === 1 && s.clock - shapedAt > 0.8) &&
|
||
ringOut.at(-1).stats.playing === 0 &&
|
||
shapedGone > 1.15 &&
|
||
shapedGone < 1.6,
|
||
JSON.stringify({ last: shaped.stats.last, shapedGone }),
|
||
);
|
||
|
||
await mutate(`
|
||
const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
|
||
graph.nodes = graph.nodes.filter((n) => n.id !== 'sn-note');
|
||
graph.edges.find((e) => e.id === 'sn-e1').to = { node: 'sn-play', port: 'exec' };
|
||
const project = __gm.state.project;
|
||
project.assets = project.assets.filter((a) => a.id !== ${JSON.stringify(insId)});`);
|
||
|
||
// 34h. Dropping a sound file onto the editor uploads it and makes an asset named after the file.
|
||
const dropped = await evaluate(`
|
||
(async () => {
|
||
const bytes = await fetch('/game-maker/data/sounds/${drone.file}').then((r) => r.arrayBuffer());
|
||
const file = new File([bytes], 'drone.wav', { type: 'audio/wav' });
|
||
const data = new DataTransfer();
|
||
data.items.add(file);
|
||
const canvas = document.querySelector('canvas');
|
||
canvas.dispatchEvent(new DragEvent('drop', { dataTransfer: data, bubbles: true, cancelable: true }));
|
||
return file.size;
|
||
})()`);
|
||
await waitFor(
|
||
() => assetsOfType('sound'),
|
||
(list) => list.length === 2,
|
||
);
|
||
const both = await assetsOfType('sound');
|
||
check(
|
||
'a dropped sound is uploaded and becomes an asset named after the file',
|
||
dropped === DRONE_BYTES &&
|
||
both.length === 2 &&
|
||
both.some((a) => a.file === drone.file && a.name === 'drone'),
|
||
JSON.stringify({ dropped, both, notice: await json('__gm.state.notice') }),
|
||
);
|
||
|
||
// 34i. The step takes back everything it made, so the steps after it see the fixture again.
|
||
await mutate(`
|
||
const gone = new Set(['sn-logic', ...${JSON.stringify(both.map((a) => a.id))}]);
|
||
const project = __gm.state.project;
|
||
project.assets = project.assets.filter((a) => !gone.has(a.id));
|
||
__gm.state.activeScene.objects = __gm.state.activeScene.objects.filter(
|
||
(o) => !['sn-box', 'sn-brain'].includes(o.id));
|
||
__gm.state.select(null);
|
||
__gm.lookAt(0, 0);`);
|
||
check(
|
||
'the step leaves nothing of its own behind',
|
||
(await assetsOfType('sound')).length === 0 && (await stats()).playing === 0,
|
||
JSON.stringify(await assetsOfType('sound')),
|
||
);
|
||
}
|