From fd1bb5b9a588e90f55b2804a12928edbc8575d38 Mon Sep 17 00:00:00 2001 From: meleki Date: Tue, 6 Oct 2026 12:29:58 +0200 Subject: [PATCH] Describe the scheduler and Delay synced to beats #332 Instruments gets a section Playing in time and the tempo input of Play note. Sounds describes the Sync setting of the Delay and the tempo input of Play sound. Architecture names core/music and the timer worker. Co-Authored-By: Claude Opus 5.5 --- Architecture.md | 9 ++++++++- Instruments.md | 39 +++++++++++++++++++++++++++++++++++++-- Sounds.md | 16 ++++++++++++---- 3 files changed, 57 insertions(+), 7 deletions(-) diff --git a/Architecture.md b/Architecture.md index 29ab9d1..b8af203 100644 --- a/Architecture.md +++ b/Architecture.md @@ -145,6 +145,10 @@ src/core/store/ serialize (untrusted JSON → Project), projectPlugins (w generic client over the three file collections) — TextureStore, FontStore and SoundStore are its type parameters, nothing else src/core/time.ts the global clock: one scaled step a frame, shared by the world, the particles, the shaders and the animation preview +src/core/music/ time (ticks, 480 to the quarter, tempo and meter, a position as bar, beat and tick), sequence + (the notes of a window of playing time, across the seam of a loop — a pure rule), Scheduler + (#332: plans the notes ahead on the audio clock through a host, so it is measured with a clock + of the test's own) src/runtime/ GameRun (one run: start, tick, `Load scene` — shared by Play and a released game, #266), World, Entity, System interface, Input tracker, CollisionSystem (the editor run's detection), solids (what `Move` collides with and how it slides — one-way solids let it through unless @@ -200,7 +204,10 @@ src/viewport/ WorldViewport (the drawing of a world or a holder, the ba CPU and laid side by side onto one canvas, so a frame is cut out like a texture frame), SoundLibrary + pitchWorklet (the AudioWorklet that plays a way with a Pitch through core/soundfx/pitchPlayer, a phase vocoder, #317; a note of an instrument is one voice - of several layers, each way with its own file, #331), ShaderLibrary, ParticleLibrary, + of several layers, each way with its own file, #331; a voice can start and be released + at a second of the audio clock, #332), music (the scheduler on the library's clock, its + notes played by the project's instruments, driven by `__gm.music` until #333) + musicTimer + (the Web Worker whose interval is the scheduler's beat, #332), ShaderLibrary, ParticleLibrary, particles (one ParticleContainer per source), tilemap (one container per layer), shaderPicture (the picture a shader is laid over when it shades a camera's frame, #299, or a particle field, #170: drawn anew each time into a render texture kept while its size diff --git a/Instruments.md b/Instruments.md index cda222f..92458b3 100644 --- a/Instruments.md +++ b/Instruments.md @@ -116,9 +116,10 @@ heard at once on what is playing. Those values are the tab's own and never writt | `note` | Its number, 60 is C4 (the default). | | `velocity` | 0 to 1, how hard it is struck: quieter, and handed to what the instrument does with it. Default 1. | | `volume` | A factor like the volume of `Play sound`. Default 1. | -| `duration` | Seconds until the note is released, on the clock of the sound. **0 holds it** until a `Stop sound` ends it. | +| `duration` | Seconds until the note is released, on the clock of the sound: its fade is set there to the sample when the note starts. **0 holds it** until a `Stop sound` ends it. | | `detached` | As at `Play sound`: the note outlives the object that played it. | | `params` | Values for the instrument's parameters, from an **Instrument parameters** node. | +| `tempo` | BPM, what a [Delay synced to beats](Sounds#delay-and-the-tail-of-a-voice) in the instrument counts by. Default 120. | Its output `voice` is the number of the note, like `Play sound`'s: `Set sound FX parameters` moves its parameters while it plays (the bundle of an Instrument parameters node goes into its @@ -130,8 +131,42 @@ parameters to set, each a number input starting at the default its Parameters no Renaming a parameter there renames it on every such node; where `Play note` and the node name two different instruments, `Play note` warns. +## Playing in time + +The **scheduler** (#332) plays notes of instruments in time, on the clock of the sound rather than +the game's, so a slow frame or a busy page never moves a note. Patterns and songs (#333, #334) will +be made of it; nothing in the editor plays a sequence yet, only the check scripts do (`__gm.music`). + +- **Ticks.** Musical time is counted in ticks, **480 to the quarter note** as in MIDI files: a + 1/32 triplet is 40, a 1/16 quintuplet 96, all whole numbers. The tempo always counts quarter + notes a minute (20 to 400 BPM), in 6/8 as in 4/4. +- **Meter.** Any meter: a beat is the note value under the line, so 6/8 has six beats of 240 + ticks and 7/8 a bar of 3.5 quarters. A position reads as bar and beat from 1 and the tick inside + the beat from 0. +- **Lookahead.** Every 25 ms it plans the notes of the next 100 ms with their exact start. The beat + comes from a timer in a Web Worker, which keeps its pace in a hidden tab, where the page's own + timers are held to once a second. The first note sounds 50 ms after the start, the time the page + needs to get it to the audio thread. +- **To the sample.** A note planned ahead starts on its sample: a buffer source by Web Audio's + `start(when)`, the pitch worklet by counting down inside the render block it falls in. Its end is + set on the clock to the sample as well: the fade of a *Gate* or *Loop* Sample. Leaving the loop of + a *Loop, then release* Sample waits for the page, a few milliseconds. +- **Loops.** At the end of a loop the window looks on into the next round, so the seam has no gap. + A note reaching past the end of the loop ends there, where the playhead jumps back. Started + behind the loop, it plays on. +- **Changes.** The sequence is read again for every window, so a note drawn while it plays is heard + from the next window on; what is planned stays. A new tempo or loop counts from what is planned. + **Stop** fades every note it started, those still to come too, and the next start goes on from + there; a **jump** does the same and starts at the new place. +- **Where it stands** is the audio clock less the output latency (`AudioContext.outputLatency`), + so a playhead drawn from it is not ahead of the sound. +- **Measured** (2026-10-06, e2e 34, headless Chrome on the CI's Windows machine): a looped bar at + 150 BPM, nine notes exactly 0.4 s apart across the seam, each planned 89 to 91 ms before its + second; the position matched the clock less 32 ms latency to the tick. Offline, the worklet + started on samples 590, 12000 and 15200 as planned. + ## Not yet No envelope with a release after the note (#335), no oscillators or 8-bit sources (#329), no -patterns, songs or scheduler (#332-#334). Notes can not be played from an `.mid` file or a MIDI +patterns or songs (#333, #334), no swing. Notes can not be played from an `.mid` file or a MIDI keyboard. diff --git a/Sounds.md b/Sounds.md index 4e9e04b..9c5bf65 100644 --- a/Sounds.md +++ b/Sounds.md @@ -36,7 +36,7 @@ so what the inspector plays is what the graph will play. | Node | What it does | | --- | --- | -| **Play sound** | Starts the sound named by its `sound` input. `volume` **multiplies** the asset's own volume (1 leaves it alone), `loop` is a choice of three (*From the asset*, *On*, *Off*), `detached` says the voice belongs to nobody, `fx` names a [sound FX](#sound-fx) to play it through, and `fx params` takes values for that FX's [parameters](#parameters). Its output `voice` is the number of this one play, for changing it [while it plays](#changing-a-playing-sound), `length` how many seconds one run of the sound lasts, and `tail` that length with the [echoes](#delay-and-the-tail-of-a-voice) of its FX after it, at the values it starts with: how long until the voice is quiet. All three are 0 where nothing was played. | +| **Play sound** | Starts the sound named by its `sound` input. `volume` **multiplies** the asset's own volume (1 leaves it alone), `loop` is a choice of three (*From the asset*, *On*, *Off*), `detached` says the voice belongs to nobody, `fx` names a [sound FX](#sound-fx) to play it through, and `fx params` takes values for that FX's [parameters](#parameters). Its output `voice` is the number of this one play, for changing it [while it plays](#changing-a-playing-sound), `length` how many seconds one run of the sound lasts, and `tail` that length with the [echoes](#delay-and-the-tail-of-a-voice) of its FX after it, at the values it starts with: how long until the voice is quiet. All three are 0 where nothing was played. `tempo` (BPM, 120 where nothing is connected) is what a [Delay synced to beats](#delay-and-the-tail-of-a-voice) in its FX counts by. | | **Stop sound** | Silences every voice of the sound it names, however many plays started it — a loop included. Naming no sound silences everything the run is playing. With `voice` connected it silences only that one play (the number `Play sound` gave out), and nothing where that is 0. A voice is faded out in 10 ms rather than cut, so it does not click, its echoes with it; **Let ring** stops only the sound and lets the [echoes](#delay-and-the-tail-of-a-voice) fade as they would. | | **Set sound FX parameters** | Hands new values to the [parameters](#changing-a-playing-sound) of one voice that is already playing, by the `voice` number its `Play sound` gave out, at once or over `time` seconds. | | **Sound** | A constant sound, for either of the two to work on through an edge. | @@ -105,12 +105,12 @@ Input and the Output once each; their pins carry sound and fit nothing but each | **Pitch** | Higher or lower, in semitones (12 is an octave, 0.5 a quarter tone). The sound **keeps its length**, also when the pitch changes while it plays. The pitch is made where the sound is read, so an effect before the Pitch acts as if it came after it. | | **5-Band-EQ** | Five bands in dB: 80 Hz and 12 kHz as shelves, 250 Hz, 1 kHz and 4 kHz as peaks. A band at 0 does nothing. | | **Mix** | Blends the sounds at `a` and `b`: at 0 only `a` is heard, at 1 only `b`, at 0.5 half of each (0.5 is the default). | -| **Delay** | An echo: the sound comes back after `time` seconds, again and again, each time `feedback` times as loud. `damping` makes every echo duller than the last, `mix` is how loud the echoes are next to the sound, which stays as it is. **Ping-pong** sends the echoes left and right in turn. See [below](#delay-and-the-tail-of-a-voice). | +| **Delay** | An echo: the sound comes back after `time`, again and again, each time `feedback` times as loud. **Sync** says what `time` counts: *Seconds*, or *Beats* in quarter notes at the tempo of the voice. `damping` makes every echo duller than the last, `mix` is how loud the echoes are next to the sound, which stays as it is. **Ping-pong** sends the echoes left and right in turn. See [below](#delay-and-the-tail-of-a-voice). | Every number of these nodes is an **input**: typed in while nothing is connected, or fed from a parameter of the Input. Each is held to its range (Gain -120 to +24 dB, Pitch -64 to +64, the EQ -bands -60 to +24 dB, Mix 0 to 1, Delay `time` 0.01 to 2 s, `feedback` 0 to 0.95, `damping` and -`mix` 0 to 1), whichever way the value arrives. +bands -60 to +24 dB, Mix 0 to 1, Delay `time` 0.01 to 8 and never more than 2 s, `feedback` 0 to +0.95, `damping` and `mix` 0 to 1), whichever way the value arrives. **How Pitch keeps the length** (#317). A way with a Pitch is played by an `AudioWorklet` on the audio thread, a **phase vocoder**: the sound is cut into frames of about 43 ms (2048 samples at @@ -167,6 +167,14 @@ never makes the sound quieter: `mix` 0.5 is echoes at half its level. Defaults: swaps the sides every round: the first echo is left, the second right. It is a setting of the node, not an input, because it changes how the voice is wired. - A `mix` typed in at 0 leaves the Delay out. Fed from a parameter it stays, at whatever value. +- **Sync** (#332): with *Seconds* (the default) `time` is seconds. With *Beats* it is quarter + notes at the tempo of the voice: 1 is a quarter, 0.5 an eighth, 0.75 a dotted eighth, 0.333 an + eighth triplet. A voice started by `Play sound` or `Play note` takes its tempo from their input + `tempo` (120 BPM where nothing is connected), a note the scheduler plays the tempo of its + sequence. It is a number like any other, so it can be fed from a parameter and moved while the + voice plays. Either way an echo comes back after **2 s at most**, the length of the line: a whole + note (4) at 60 BPM would be 4 s and is held to 2. The tempo is taken when the voice starts; a + later change of tempo does not reach a voice already playing. - **In Chrome every round comes 128 samples (2.7 ms at 48 kHz) later** than `time`: a feedback loop costs one render quantum each time round (measured in e2e 34: echoes 1 to 4 at 0, 128, 256 and 384 samples late). After ten echoes at 0.3 s that is 27 ms. It is left so, because a browser