Play notes in time on the audio clock with a scheduler #332

Patterns and songs (#333, #334) have to play sample-accurate in time,
whatever the frame rate does. What is counted in game time runs away
from the sound. This lays the ground: musical time, a scheduler that
plans notes ahead on the audio clock, and a player that starts and ends
a note at a given second. Nothing in the editor plays a sequence yet,
the check scripts drive it through `__gm.music`.

- Ticks, 480 to the quarter as in MIDI files (decided in #332).
  Triplets and quintuplets land on whole ticks. Any meter, a beat being
  the note value under the line. The tempo always counts quarters.
- `core/music/sequence.ts` is a pure rule: the notes starting in a
  window of playing time. Across the end of a loop it looks into the
  next round, so the seam has no gap. A note past the loop end ends
  there. Windows are whole ticks, and the seam is stepped exactly, so
  no rounding can drop the first note of a round.
- `core/music/Scheduler.ts` plans the next 100 ms every 25 ms through a
  host, so it is tested with a clock of its own. The first note sounds
  50 ms after the start. Changes to the sequence are heard from the
  next window on. Stop and seek fade what it started. A new tempo or
  loop counts from what is planned, and the playhead keeps its old
  pace until the clock gets there.
- The position is the audio clock less `outputLatency`, as bar, beat
  and tick.
- The beat comes from a Web Worker (`viewport/musicTimer.ts`), which
  keeps its pace in a hidden tab.
- The sound library starts a note at `timing.at`: a buffer source by
  `start(when)`, the pitch worklet by counting down inside the render
  block it falls in. A note's end is set on the clock at once, its
  fade to the sample. Leaving a loop still waits for the page.
- Delay synced to beats (decided in #332, noted from #327): a setting
  `Sync` (Seconds / Beats), `time` then in quarter notes, held to the
  2 s of the line. Play sound and Play note have an input `tempo`,
  120 BPM where nothing is connected. A scheduled note carries the
  tempo of its sequence.
- Swing is left for #333/#334, as decided.

Also fixes the red suite of 6f1f315 (task 526). Three checks in e2e 34
waited on the page's clock for things that run on the audio clock. In
CI the voice was already 0.7 s old when first seen, and the 0.3 s note
of Play note was over. `stats().last` now says when a voice started on
the audio clock, and the Quad, Step and Play note checks hold every
reading against it.

Tests:
- music.test.ts (new): 17 tests, time, windows, loop seam, scheduler
  with a jittery timer, stop, seek, tempo change
- sound-library +4, soundfx +1, node-catalog +2 (tempo inputs)
- Counter-checks: an unclamped note length, a window that never moves
  on, a buffer source ignoring its start time and a Delay ignoring
  Beats each turn 1, 6, 1 and 2 tests red
- e2e 34 on melektaus-atx: ALL PASSED, 47 checks (+2). Nine notes of a
  looped bar at 150 BPM exactly 0.4 s apart, each planned 89 to 91 ms
  ahead. The position matched the clock less 32 ms latency to the
  tick. Offline the worklet started on samples 590, 12000 and 15200.
- Gate on melektaus-atx: CHECK EXIT 0, 244 test files, 3786 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

tools/e2e/34-sound.mjs

@@ -34,6 +34,16 @@ export async function sound(t) {
     }
     return value;
   };
+  // Every reading up to the one `done` takes: a check holds each against its own time on the audio
+  // clock, however late the first one comes in.
+  const trace = async (read, done, tries = 40) => {
+    const seen = [await read()];
+    for (let i = 0; i < tries && !done(seen.at(-1)); i++) {
+      await sleep(150);
+      seen.push(await read());
+    }
+    return seen;
+  };
 
   await press('Escape');
   await evaluate('document.activeElement instanceof HTMLElement && document.activeElement.blur()');
@@ -558,39 +568,50 @@ export async function sound(t) {
     JSON.stringify({ early, later, afterHalf }),
   );
   await stop();
-  // The easings of an animation key: Quad in hands the clock a curve, which is still near the start
-  // after 0.8 s and only then falls to the end (-5): 7 - 12 * (t / 2)^2, where linear is 7 - 6 * t.
-  // Timed on the audio clock from the moment the voice is first seen, which is no earlier than its
-  // start: the curve is then further along than `into` says, and still well above linear.
+  // The easings of an animation key: Quad in hands the clock a curve, 7 - 3 * t^2 towards -5 in 2 s,
+  // where linear is 7 - 6 * t. Each reading is timed on the audio clock from the voice's start, so a
+  // first one that comes in late is held against where the curve is then. Between 0.3 and 1.7 s the
+  // curve is more than a semitone above linear, and it never falls below it.
   await mutate(`
     const tune = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph.nodes.find((n) => n.id === 'sn-tune');
     Object.assign(tune.params, { ease: 'quad', direction: 'in' });`);
   await play();
   const pitchNow = () =>
     json(`({ pitch: __gm.sound.levels(${JSON.stringify(blip.file)})?.[0]?.[0], clock: __gm.sound.clock() })`);
-  const sweepSeen = await waitFor(pitchNow, (s) => typeof s.pitch === 'number');
-  const eased = await waitFor(pitchNow, (s) => s.clock - sweepSeen.clock >= 0.8);
-  const into = eased.clock - sweepSeen.clock;
-  const easedEnd = await waitFor(pitch, (s) => typeof s === 'number' && s < -4.3);
+  const sinceStart = async (seen) => {
+    const from = (await stats()).last.at;
+    return seen
+      .filter((s) => typeof s.pitch === 'number')
+      .map((s) => ({ t: +(s.clock - from).toFixed(3), pitch: +s.pitch.toFixed(3) }));
+  };
+  const sweep = await sinceStart(await trace(pitchNow, (s) => typeof s.pitch === 'number' && s.pitch < -4.3));
+  const midway = sweep.filter(({ t }) => t > 0.3 && t < 1.7);
   check(
     'Set sound FX parameters with Quad in ramps slowly first, then on to the end',
-    eased.pitch < 6.95 && eased.pitch > 7 - 6 * into + 1 && easedEnd < -4.3,
-    JSON.stringify({ into, eased: eased.pitch, linearThere: 7 - 6 * into, easedEnd }),
+    midway.length > 0 &&
+      midway.every(({ t, pitch }) => pitch > 7 - 6 * t + 1) &&
+      sweep.every(({ t, pitch }) => pitch > 7 - 6 * t - 0.1) &&
+      sweep.at(-1).pitch < -4.3,
+    JSON.stringify(sweep),
   );
   await stop();
-  // Step holds the pitch for the whole time and then jumps there.
+  // Step holds the pitch for the whole time and then jumps there: every reading before 2 s on the
+  // audio clock is still 7, the first one past it is the end.
   await mutate(`
     const tune = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph.nodes.find((n) => n.id === 'sn-tune');
-    Object.assign(tune.params, { ease: 'step', time: 1 });`);
+    Object.assign(tune.params, { ease: 'step', time: 2 });`);
   await play();
-  await waitFor(pitch, (s) => typeof s === 'number');
-  await new Promise((resolve) => setTimeout(resolve, 500));
-  const stepHeld = await pitch();
-  const stepEnd = await waitFor(pitch, (s) => typeof s === 'number' && s < 6);
+  const steps = await sinceStart(await trace(pitchNow, (s) => typeof s.pitch === 'number' && s.pitch < 6));
+  const held = steps.filter(({ pitch }) => pitch === 7);
+  const jumped = steps.find(({ pitch }) => pitch < 6);
   check(
     'Set sound FX parameters with Step holds the pitch for the time, then jumps to the end',
-    stepHeld === 7 && Math.abs(stepEnd + 5) < 0.001,
-    JSON.stringify({ stepHeld, stepEnd }),
+    held.length > 0 &&
+      held.every(({ t }) => t < 2.05) &&
+      jumped !== undefined &&
+      jumped.t > 1.95 &&
+      Math.abs(jumped.pitch + 5) < 0.001,
+    JSON.stringify(steps),
   );
   await stop();
   // #317: the worklet itself, rendered in an offline context of this browser – what a muted check
@@ -883,25 +904,104 @@ export async function sound(t) {
   await mutate(`
     const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
     graph.nodes.push(${JSON.stringify(
-      node('sn-note', 'PlayNote', 260, 220, { instrument: insId, note: 67, duration: 0.3 }),
+      node('sn-note', 'PlayNote', 260, 220, { instrument: insId, note: 67, duration: 1.5 }),
     )});
     graph.edges.find((e) => e.id === 'sn-e1').to = { node: 'sn-note', port: 'exec' };`);
   await showTab('scene');
   await evaluate('__gm.sound.stopAll()');
   const beforeNote = await stats();
   await play();
-  const noted = await waitFor(stats, (s) => s.started > beforeNote.started);
-  await sleep(900);
-  const released = await stats();
+  // The sample loops, so only the release after 1.5 s on the audio clock ends the note: it is heard
+  // well past the 0.25 s of the blip, never past its duration, and is gone soon after.
+  const noteNow = () => json('({ stats: __gm.sound.stats(), clock: __gm.sound.clock() })');
+  const noted = await waitFor(noteNow, (s) => s.stats.started > beforeNote.started);
+  const heard = await trace(noteNow, (s) => s.stats.playing === 0);
+  const noteAt = noted.stats.last?.at ?? 0;
+  const heardAt = heard.filter((s) => s.stats.playing === 1).map((s) => +(s.clock - noteAt).toFixed(3));
+  const goneAt = +(heard.at(-1).clock - noteAt).toFixed(3);
   check(
     'a game’s Play note plays G4 – the blip seven semitones up – and lets it go after its duration',
-    noted.last?.file === `instrument:${insId}` &&
-      noted.last.pitches.join() === '7' &&
-      noted.playing === 1 &&
-      released.playing === 0,
-    JSON.stringify({ noted, released }),
+    noted.stats.last?.file === `instrument:${insId}` &&
+      noted.stats.last.pitches.join() === '7' &&
+      heardAt.some((t) => t > 0.5) &&
+      heardAt.every((t) => t < 1.6) &&
+      heard.at(-1).stats.playing === 0 &&
+      goneAt < 3,
+    JSON.stringify({ last: noted.stats.last, heardAt, goneAt }),
   );
   await stop();
+
+  // #332: the scheduler, driven by the hook – nothing in the editor plays a sequence yet. A bar of
+  //       quarters at 150 BPM in a loop: every note is planned before its second comes, each 0.4 s
+  //       after the one before, also across the seam into the next round. Where the music stands
+  //       is the audio clock less the output latency, counted from the first note.
+  const bar = [60, 62, 64, 65].map((pitch, i) => ({
+    tick: i * 480,
+    length: 240,
+    note: pitch,
+    velocity: 1,
+    instrument: insId,
+  }));
+  await evaluate(
+    `__gm.music.play(${JSON.stringify({ notes: bar })}, { tempo: 150, loop: { start: 0, end: 1920 } })`,
+  );
+  const planned = await waitFor(
+    () => json('__gm.music.planned()'),
+    (p) => p.length >= 9,
+    40,
+  );
+  const where = await json(
+    '({ position: __gm.music.position(), clock: __gm.sound.clock(), latency: __gm.music.latency() })',
+  );
+  await evaluate('__gm.music.stop()');
+  const musicOff = await waitFor(stats, (s) => s.playing === 0);
+  const apart = planned.slice(1, 9).map((p, i) => +(p.at - planned[i].at).toFixed(6));
+  const ahead = planned.map((p) => +(p.at - p.planned).toFixed(3));
+  const heardTicks = (((where.clock - where.latency - planned[0].at) * 150 * 480) / 60) % 1920;
+  check(
+    'the scheduler plans a looped bar on the audio clock: every note ahead of its second, 0.4 s apart across the seam',
+    planned.length >= 9 &&
+      planned
+        .slice(0, 9)
+        .map((p) => p.note)
+        .join() === '60,62,64,65,60,62,64,65,60' &&
+      planned.every((p) => p.voice > 0 && Math.abs(p.duration - 0.2) < 1e-9 && p.tempo === 150) &&
+      apart.every((gap) => Math.abs(gap - 0.4) < 1e-6) &&
+      ahead.every((s) => s > 0 && s <= 0.16) &&
+      Math.abs(where.position.ticks - heardTicks) < 1 &&
+      where.position.bar === 1 &&
+      musicOff.playing === 0 &&
+      (await json('__gm.music.playing()')) === false,
+    JSON.stringify({ apart, ahead, where, heardTicks, playing: musicOff.playing }),
+  );
+
+  // A note planned ahead through the pitch worklet starts on its sample, also inside a render block:
+  // rendered offline at pitch 0, which plays the sound as it is, so the first sample heard is it.
+  const onsets = await evaluate(`
+    (async () => {
+      const RATE = 48000;
+      const onset = async (startAt) => {
+        const context = new OfflineAudioContext(1, RATE / 2, RATE);
+        await context.audioWorklet.addModule(__gm.sound.pitchWorkletUrl());
+        const node = new AudioWorkletNode(context, 'pitch-player', {
+          numberOfInputs: 0,
+          outputChannelCount: [1],
+          parameterData: { pitch: 0 },
+          processorOptions: { file: 'onset', channels: [new Float32Array(RATE).fill(0.5)], step: 1, loop: false, startAt },
+        });
+        node.connect(context.destination);
+        const out = (await context.startRendering()).getChannelData(0);
+        return out.findIndex((v) => Math.abs(v) > 1e-6);
+      };
+      const at = [0.0123, 0.25, 0.05 + 0.4 * 2 / 3];
+      return { expected: at.map((s) => Math.round(s * RATE)), heard: await Promise.all(at.map(onset)) };
+    })()`);
+  check(
+    'a pitched note planned ahead starts on its very sample, inside a render block too',
+    onsets.heard.join() === onsets.expected.join() && onsets.expected.some((i) => i % 128 !== 0),
+    JSON.stringify(onsets),
+  );
+
   await mutate(`
     const graph = __gm.state.project.assets.find((a) => a.id === 'sn-logic').graph;
     graph.nodes = graph.nodes.filter((n) => n.id !== 'sn-note');