Let a behavior follow the beat, bar and markers of the music (#338)
feuerwerk / gate (push) Failing after 15m41s
feuerwerk / suite (push) Canceled after 18m24s

Stage 2 of #338. A game that syncs to its music has to know where the
music is heard. This stage reads that on the audio clock and hands it
to behaviors as events and as a position.

- New event nodes On beat (playback, bar, beat, late), On bar
  (playback, bar, late) and On song marker (playback, marker, name,
  late).
- New data node Song position with bar, beat, fraction, seconds and
  playing.
- The nodes have the fields Song and Pattern. With neither picked,
  every playback counts. A linked playback input picks that one.
  Compositions do not count.
- The Marker field offers the markers of all songs as Song / Marker,
  as a sound field offers its clips.
- Everything follows what is heard: the audio clock minus the output
  latency. That holds for baked songs and for Play pattern too.
- Only the last beat since the frame before fires, and only when it is
  at most half a beat late, 0.15 s at most. Bars and markers alike.
- A copied song gets markers with ids of its own, so a Marker field
  never names two songs.
- The audio clock is only asked while something plays. Asked in every
  frame, it made the AudioContext without a user gesture (#436).

Tests:
- beats 14 and music-events 6 new, 4 in song-play, 2 in song-markers,
  2 catalog cases, the touched groups 756/756
- 16 counter-probes, all compile, all red
- e2e 65 on atx: ALL PASSED, 71 checks, 65k2 new: 11 beats 1.1 to 2.4
  and 2.1 again, none missing or twice, across the pause too. late
  0.015 to 0.027 s. Counter-probe without the last position: red
- gate on atx: CHECK EXIT 0, 334 files, 5121 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-29 19:48:51 +01:00
1 parent 5cc1cc9ef6
commit a6a41b6b69
25 files changed
+1171 -12

No files matched your search

+34
View File
@@ -0,0 +1,34 @@
import type { NodeDef } from '../../NodeRegistry';
import { asNumber, asString } from '../../values';
import { patternPort } from '../action/PlayPattern';
import { songPort } from '../action/PlaySong';
import { PLAYBACK_INPUT, playbackPort } from '../events/musicEvents';
import { bool, num } from '../shared';
export const SongPosition: NodeDef = {
type: 'SongPosition',
label: 'Song position',
category: 'data',
description:
'Where a playing song or pattern stands, as it is heard: the output latency of the device taken ' +
'off, a baked song too, so what is drawn by it keeps to the sound. "bar" and "beat" count from 1 ' +
'as the ruler does, "fraction" is how far into the beat (0 to below 1), "seconds" how far from ' +
'the start of the song, back with the loop. Connect "playback" for that playing; else it is the ' +
'one started last of the Song or the Pattern picked (both picked, either), of every song and ' +
'pattern where none is. "playing" is off before its first note is heard and after its end; with ' +
'no such playing everything is 0.',
inputs: [songPort(), patternPort(), playbackPort()],
outputs: [num('bar'), num('beat'), num('fraction'), num('seconds'), bool('playing')],
params: [],
read: (ctx, _params, input) => {
const playback = input(PLAYBACK_INPUT);
const at = ctx.songPosition(
playback === null ? null : asNumber(playback),
asString(input('song')),
asString(input('pattern')),
);
return at
? { bar: at.bar, beat: at.beat, fraction: at.fraction, seconds: at.seconds, playing: at.playing }
: { bar: 0, beat: 0, fraction: 0, seconds: 0, playing: false };
},
};
@@ -0,0 +1,94 @@
import { textOf } from '../../../model/Properties';
import type { PropertyDef } from '../../../properties/PropertySchema';
import type { NodeDef, PortDef } from '../../NodeRegistry';
import { hiddenOrder, orderParam } from '../../eventOrder';
import { exec, num, str } from '../shared';
/** The input that picks one playing (#338): not typed in, so unconnected it is null and hears every one. */
export const PLAYBACK_INPUT = 'playback';
export const playbackPort = (): PortDef => ({ ...num(PLAYBACK_INPUT), inline: false });
const songParam: PropertyDef = {
key: 'song',
label: 'Song',
kind: 'asset',
default: '',
assetTypes: ['song'],
};
const patternParam: PropertyDef = {
key: 'pattern',
label: 'Pattern',
kind: 'asset',
default: '',
assetTypes: ['pattern'],
};
const markerParam: PropertyDef = {
key: 'marker',
label: 'Marker',
kind: 'asset',
default: '',
assetTypes: ['song'],
markers: true,
};
/** A pick left at None is not shown on the node: it hears every one. */
const hiddenEmpty =
(...keys: string[]) =>
(params: Parameters<NonNullable<NodeDef['hiddenParams']>>[0]) => [
...keys.filter((key) => textOf(params[key] ?? '') === ''),
...hiddenOrder(params),
];
const HEARS =
'It hears every song a Play song plays and every pattern a Play pattern plays, wherever it was ' +
'played; pick a Song or a Pattern to hear only that one (both picked, either), or connect ' +
'"playback" to hear only that playing. A composition plays none of them. It goes by what is heard, ' +
'the output latency of the device taken off, a baked song too. A frame comes now and then, so ' +
'"late" says how many seconds ago it was heard; one more than half a beat (at most 0.15 s) ago is ' +
'not told, and of several passed in one frame only the last.';
export const OnBeat: NodeDef = {
type: 'OnBeat',
label: 'On beat',
category: 'event',
description:
'Runs on every beat of a playing song or pattern, the first beat of a bar too. A beat is a note ' +
'of the value under the line of the time signature: a quarter in 4/4, an eighth in 6/8. "bar" ' +
'and "beat" count from 1, as the ruler does, and start over with the loop. ' +
HEARS,
inputs: [playbackPort()],
outputs: [exec(), num('playback'), num('bar'), num('beat'), num('late')],
params: [songParam, patternParam, orderParam],
hiddenParams: hiddenEmpty('song', 'pattern'),
};
export const OnBar: NodeDef = {
type: 'OnBar',
label: 'On bar',
category: 'event',
description:
'Runs at the start of every bar of a playing song or pattern, "bar" counting from 1 as the ruler ' +
'does; with the loop it starts over at the bar the loop starts in. ' +
HEARS,
inputs: [playbackPort()],
outputs: [exec(), num('playback'), num('bar'), num('late')],
params: [songParam, patternParam, orderParam],
hiddenParams: hiddenEmpty('song', 'pattern'),
};
export const OnSongMarker: NodeDef = {
type: 'OnSongMarker',
label: 'On song marker',
category: 'event',
description:
'Runs when a playing song passes a marker of its ruler, every time round the loop. Pick a Marker ' +
'to hear only that one, or a Song to hear only its markers; markers on one tick each run it. ' +
'"marker" is the id a marker is known by, "name" what it is called. Connect "playback" to hear ' +
'only that playing. It goes by what is heard, a baked song too. "late" says how many seconds ago ' +
'it was heard; one more than half a beat (at most 0.15 s) ago is not told, and of several passed ' +
'in one frame only the last.',
inputs: [playbackPort()],
outputs: [exec(), num('playback'), str('marker'), str('name'), num('late')],
params: [songParam, markerParam, orderParam],
hiddenParams: hiddenEmpty('song', 'marker'),
};
+6
View File
@@ -125,6 +125,8 @@ import { SubList } from './data/SubList';
import { TextureSize } from './data/TextureSize';
import { OnAnimationFinished } from './events/OnAnimationFinished';
import { OnCompositionFinished } from './events/OnCompositionFinished';
import { OnBar, OnBeat, OnSongMarker } from './events/musicEvents';
import { SongPosition } from './data/SongPosition';
import { OnBehavior } from './events/OnBehavior';
import { OnClick } from './events/OnClick';
import { OnMouseWheel } from './events/OnMouseWheel';
@@ -186,6 +188,9 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
OnMouseWheel,
OnAnimationFinished,
OnCompositionFinished,
OnBeat,
OnBar,
OnSongMarker,
OnBehavior,
OnStateEntered,
OnStateExited,
@@ -323,6 +328,7 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
StopPattern,
PlaySong,
StopSong,
SongPosition,
SetBusVolume,
BusVolume,
LoadScene,