Compare commits

..
10 Commits
Author SHA1 Message Date
adminandClaude Opus 5.5 53ec1094f4 Let a game switch songs and turn their tracks on the beat (#338)
A game could start and stop a song, but not steer one while it plays.
The battle theme could not take over on the next bar, a song could not
jump to a marker, and a track could not fall away when the player
hides. This is stage 3 of 5 in #338.

- Quantize: now, next beat, next bar (the default) or next marker. The
  spot is the next one as the song is heard, and the change is planned
  on the audio clock to the sample, round the loop as the ruler counts.
- Set song track turns one track in dB on top of the song's level, over
  a fade, on the bus input before its FX and sends.
- Switch song plays another song, or the same one from a marker, in
  place of the playing one. It keeps the playback number, so Stop song,
  Set song track and On beat go on reaching it.
- A playing now has a key for its buses apart from its public number.
  During a switch the old and the new song each have buses of their own.
- Play song has a "live tracks" switch. With it off a baked song plays
  its bake, and Set song track on it writes a line to the log.
- A Switch song to a marker plays the notes, as a bake only plays from
  its start.

Found on the way:

- A cut on a bar line faded over seconds in Chrome. cancelAndHoldAtTime
  holds nothing where nothing is planned later, so the ramp ran from the
  bus's last change. The turn now sets its start value on the spot. It
  had been measured at 0.52 of full level 0.2 s past the bar line.

Tests:
- Unit: 23 new cases in beats, song-play, sound-library and the node
  catalog. Counter-checks: 17 bends, all compile, all red.
- e2e 65 (atx): ALL PASSED, 73 checks. 65k3 is new. The switch landed
  three bars after the start under the same number, and the track cut
  on the next bar line with nothing in between. With the old ramp 65k3
  is red (0.53 and 0.51 between).
- Gate (atx): CHECK EXIT 0, 334 files, 5144 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 21:17:52 +01:00
adminandClaude Opus 5.5 a6a41b6b69 Let a behavior follow the beat, bar and markers of the music (#338)
feuerwerk / gate (push) Failing after 15m41s
feuerwerk / suite (push) Canceled after 18m24s
Stage 2 of #338. A game that syncs to its music has to know where the
music is heard. This stage reads that on the audio clock and hands it
to behaviors as events and as a position.

- New event nodes On beat (playback, bar, beat, late), On bar
  (playback, bar, late) and On song marker (playback, marker, name,
  late).
- New data node Song position with bar, beat, fraction, seconds and
  playing.
- The nodes have the fields Song and Pattern. With neither picked,
  every playback counts. A linked playback input picks that one.
  Compositions do not count.
- The Marker field offers the markers of all songs as Song / Marker,
  as a sound field offers its clips.
- Everything follows what is heard: the audio clock minus the output
  latency. That holds for baked songs and for Play pattern too.
- Only the last beat since the frame before fires, and only when it is
  at most half a beat late, 0.15 s at most. Bars and markers alike.
- A copied song gets markers with ids of its own, so a Marker field
  never names two songs.
- The audio clock is only asked while something plays. Asked in every
  frame, it made the AudioContext without a user gesture (#436).

Tests:
- beats 14 and music-events 6 new, 4 in song-play, 2 in song-markers,
  2 catalog cases, the touched groups 756/756
- 16 counter-probes, all compile, all red
- e2e 65 on atx: ALL PASSED, 71 checks, 65k2 new: 11 beats 1.1 to 2.4
  and 2.1 again, none missing or twice, across the pause too. late
  0.015 to 0.027 s. Counter-probe without the last position: red
- gate on atx: CHECK EXIT 0, 334 files, 5121 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 19:48:51 +01:00
adminandClaude Opus 5.5 5cc1cc9ef6 Tell a stream that fell behind apart from a pause that lost samples
feuerwerk / gate (push) Failing after 15m58s
feuerwerk / suite (push) Canceled after 19m19s
CI task 750 failed the pause check in e2e 34 although the pause lost
nothing. The stream on the runner never got a second ahead. It had
skipped 1.1 s before the pause was clicked, with 0.04 s left ahead.
The wait for the second ran out without a word, so the check blamed
the pause for samples the stream had lost from the start.

- A check of its own before the pause asks for a second ahead and not
  a sample skipped. A runner too slow to decode fails there now.
- When the stream was behind, the pause check judges only the clock,
  the resume and the stop, and says in its name that the samples were
  not judged.
- Nothing in the product changes.

Tests:
- e2e 34 on melektaus-atx: ALL PASSED, 149 checks. The new check saw
  1.3 MB ahead and 0 samples skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-29 19:48:01 +01:00
adminandClaude Opus 5.5 56425fca6a Let a song carry named markers on a strip in the song tab (#338)
feuerwerk / gate (push) Successful in 5m22s
feuerwerk / suite (push) Failing after 19m54s
Stage 1 of #338. Game sync needs named places in a song: an event when
the playhead passes one, and a switch that jumps to one. This stage
adds the markers themselves. Games do nothing with them yet.

- A song has markers, each with id, name and tick. A song without
  markers stores nothing for them. Two markers may share tick and
  name, since nodes will name them by id.
- Markers change neither the sound nor the bake fingerprint.
- The song tab draws a strip under the ruler. A marker has a green
  flag with its name and a faint line through the tracks. Tracks and
  lanes sit 18 px lower. The ruler of the pattern tab is unchanged.
- Double click on an empty place adds a marker on the grid, with Alt
  or Shift on the exact tick. Dragging moves it in one undo step.
- A click on a marker sets the start of playback there. A double click
  renames it in the name dialog of the hierarchy groups.
- The context menu offers Play from here, Rename and Delete marker,
  on an empty place Add marker. A tooltip shows the full name. The
  wheel over the strip zooms time like the ruler.
- New agent commands add_song_marker, change_song_marker and
  remove_song_marker. get_asset lists them under markers.

Tests:
- song-markers in core 6 and editor 4, 1 agent test, with the touched
  song, agent and MCP tests 175/175
- 9 counter-probes, all compile, all red
- e2e 65 on atx: ALL PASSED, 70 checks, 3 new in 65g3. Counter-probe
  where a dragged marker is not taken over: 65g3 red
- gate on atx: CHECK EXIT 0, 332 files, 5093 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:44:58 +02:00
adminandClaude Opus 5.5 2099511695 Let the agent set automation lanes of a song track (#337)
feuerwerk / gate (push) Successful in 6m3s
feuerwerk / suite (push) Successful in 20m14s
Stage 4 of #337. Lanes could only be drawn in the song tab. Now the
agent can set them in one step and sees which parameters a lane of a
track can move.

- New command set_song_lane. The target is named as it is stored:
  volume, pan or param:<name>. All points come at once, each with
  tick, value and ease, from the same eases as animation keys,
  curve assets included.
- Points on one tick keep the order they come in, which makes a jump.
  points [] removes a volume or pan lane, remove a parameter lane.
- It refuses with a clear message: a value outside the range, an
  unknown ease, a missing curve asset, a range for volume or pan, and
  a new lane for a parameter nothing on the track has. The message
  then names the parameters there are.
- Without min and max a new parameter lane gets the range the dialog
  in the song tab offers.
- get_asset names under params of each song track the parameters a
  lane can move, with their default. The lanes were already in it.

Live in the user's project: get_asset named both parameters of their
two lanes (frequency, env) with defaults 20000 and 0.5. Playing to
3 s and 4.2 s, both tracks were heard, the audio clock running.

Tests:
- agent-music 13/13 (2 new), agent and MCP tests 129/129
- 6 counter-probes, all compile, all red
- gate on atx: CHECK EXIT 0, 330 files, 5082 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 10:54:51 +02:00
adminandClaude Opus 5.5 b50c7dbc35 Let a song track automate parameters of its sounds and FX (#337)
Stage 3 of #337. Lanes moved volume and pan only. Now a lane can
steer a parameter of anything on its track, and a voice already
planned or still waiting for its file follows it too.

- A lane may aim at param:<name>. A parameter has no range of its
  own, so the lane keeps min and max. A parameter lane stays without
  points, so its range is not lost. Remove lane takes it away.
- It reaches the track FX, the FX of an item, the instrument of a
  note with its own FX, and the FX of a clip. When several have the
  name, all of them change, as with Set sound FX parameters.
- The scheduler hands the lane value to a note or clip at its start.
  Values a voice only reads at its start act from the next note.
- While a voice sounds, the knobs of buses and voices of the track
  follow the lane on the audio clock, read every 10 ms, because a
  knob is not always the parameter itself (trackParams.ts).
- A voice waiting for its file takes the marks when it starts.
- When a lane is removed, the parameter goes back to its default.
  An edit while playing does not reset a parameter a lane rules.
- This holds in the song tab, for Play song, in compositions and in
  the bake.
- Song tab: Add parameter lane in the track head menu opens a dialog
  for parameter and range (ParamLaneDialog.ts). The lane head menu
  has Range and Remove lane.
- Inspector: while a lane rules a value, a hint at the track names
  the lanes and says the fields act once the lane is gone.

Known limit: a knob that depends on two automated parameters reckons
each lane with the last value of the other, so it may lag one window.

Tests:
- automation-params.test.ts (9, new), 3 new in sound-library and 3
  in song-lanes. Affected files 407/407.
- 17 counter checks, all compile, all red.
- e2e 65 on melektaus-atx: ALL PASSED, 67 checks. 65h3 is new: a
  clip with FX parameter cut starts at the lane value and its knob
  stays within 0.04 dB of the lane. It goes red when automateBus
  does not pass parameters on.
- Gate on melektaus-atx: CHECK EXIT 0, 330 files, 5080 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 10:30:42 +02:00
adminandClaude Opus 5.5 bd99f9e79d Let automation lanes of a song track be drawn in the song tab (#337)
Stage 2 of #337. The lanes of stage 1 played, but nobody could make
one. Now a track folds out a lane for volume and one for pan, and
points are set, dragged and eased right on the timeline.

- A lane is a row of its own, as high as a track, below its track.
  The view functions stay row based, so zoom, fit, item drags and
  the composition (which shares them) are unchanged.
- itemAt and itemsWithin take a map of track id to row.
- A triangle in the track head folds the lanes out. The head menu
  shows or hides each lane. A lane with points is open by itself.
- Which lanes are open belongs to the view of the tab and is saved
  with it. It is not part of the song and makes no undo step.
- A click sets a point on the grid, Alt or Shift on the exact tick.
  Press and drag sets one and drags it in a single undo step.
- A dragged point stops at its neighbour. Two points on one tick
  make a jump. Double click or Delete removes a point.
- The value shows next to a dragged point and as a tooltip.
- Right click on a point opens "Ease to next point" with the same
  choice as animation keys, curve assets included.
- The lane head shows the value at the playhead. Its menu clears
  or hides the lane.
- While a lane rules a value, the fader shows it in the lane colour
  and cannot be dragged (decision 4). A hint says why.
- Geometry in laneView.ts, edit rules in laneEdit.ts, the easing
  menu in easingMenu.ts. New test hooks automationAt,
  automationHeadAt and automation under __gm.song.

Open: the Volume (dB) field of a track in the inspector can still be
changed while a lane rules it. It has no effect while playing.

Tests:
- song-lanes.test.ts (12, new) and song-edit adapted: 27/27.
  Editor tests of song and composition 62/62.
- 9 counter checks, all compile, all red.
- e2e 65 on melektaus-atx: ALL PASSED, 66 checks. 65g2 is new with
  6 checks. It goes red when a set point is not taken on release.
- Gate on melektaus-atx: CHECK EXIT 0, 329 files, 5065 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 09:31:48 +02:00
adminandClaude Opus 5.5 389a7c0fd1 Let volume and pan of a song track follow automation lanes (#337)
A song track can now change its volume and pan over time, played on
the audio clock wherever the song is heard. This is stage 1 of #337:
the model and the playback. Lanes cannot be drawn in the song tab yet.

- A track holds automation lanes for volume (dB) and pan. Each point
  has a tick, a value and an easing from the shared list, curve assets
  included. Two points on one tick make a jump. Before the first point
  its value holds, after the last point the last value holds.
- The rules are pure functions in core/music/automation.ts: the value
  at a tick and the pieces of a lane inside one scheduling window.
- The scheduler hands those pieces to the host in every window. The
  host sets them as ramps or jumps on the gain and panner of the
  track's bus. Shapes the clock cannot ramp directly (dB as gain,
  curved easings) are sampled every 10 ms. At the loop end the value
  jumps back with the playhead.
- This works in the song tab, for Play song, inside a composition
  (across the song loop, on the item's own bus) and in a bake.
- While a lane has points the fader no longer sets that value. When
  the lane goes, the fader takes over from the current value.
- Automation enters the bake fingerprint only when there is some, so
  existing bakes stay valid.
- The level helpers moved to core/music/level.ts, so song.ts and
  automation.ts do not import each other.

Known gap: a delaying sound FX on a bus after the track (a pitch
shifter) makes the automation late by that delay.

Tests:
- automation.test.ts (13, new), 3 new in sound-library.test.ts.
  Song, bake and composition tests together 286/286.
- 12 counter-checks, all compile, all red.
- e2e 65 on atx: ALL PASSED, 60 checks. New 65h2: in the song tab the
  bus level follows a volume lane, 0.09 dB off, also with the fader at
  -60 dB. Red when the song tab does not pass the automation on.
- Gate on atx: 328 files, 5053 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 06:45:55 +02:00
adminandClaude Opus 5.5 b5e071ca68 Let the agent read what is heard in the running game (#436)
The agent has no ears. It could start music and sounds, but not tell
whether anything reached the speakers: a song without an instrument, a
bus at volume 0 or a browser that holds the audio clock all looked the
same from outside.

- get_runtime has a section audio. Under levels it gives the peak and
  RMS in dB of the master, each bus of the mix, each pattern, song or
  composition playing and the tracks of a song. Below -90 dB it says
  silent.
- The levels come from a tap of their own after the level and pan of
  each bus, an AnalyserNode per side as for the meters of the Song tab.
  It looks back about 0.7 s, so the gap between two notes is not
  silence. A pause holds the values, which fits play with until.
- Under voices it names what sounds now: a sound by its asset, a note
  of an instrument with its note name, each with its bus and object.
  Next to them planned, playing, waiting (bytes not here yet) and
  failed.
- Under clock it gives the state of the audio clock, with a hint when
  it stands still while the game runs. The state is told as it was
  before a pause, else the pause would hide that case.
- The buses are tapped only while the agent is switched on.
- The note now travels with a voice from the Start of a pattern or song
  to the SoundLibrary, so a voice can be named by it.
- tools.json and the relay's instructions tell the agent to check under
  audio that music and sounds are really heard.

Found on the user's project: started right after F5, before a click in
the page, the browser kept the audio clock suspended for the whole run.
Nothing was heard and the notes stayed planned. That is what clock now
shows.

Tests:
- Unit: sound-library (taps, levels, voices, clock), agent-play
  (audioView). Pattern and song tests expect the note in their timing.
- 9 counter-checks, 8 red. Not dropping a tap with its bus stays green,
  as the bus is cut off and gone anyway.
- e2e 71 (melektaus-atx): 28/28. New 71h4: a sound on the master and
  one on a bus at volume 0, both voices named, the master loud, the bus
  silent, the values held in the pause.
- Gate (melektaus-atx): CHECK EXIT 0, 327 files, 5037 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 22:14:07 +02:00
adminandClaude Opus 5.5 dd2096e0df Make a bus of the mix at its level, so a quiet bus starts quiet
A bus of the project's mix was made at unity and then glided to its
level. Where the audio context comes with the first sound of a run,
that sound already plays on the new bus. On a bus at volume 0 it was
heard for a moment, loud, while the level glided down.

- setMix now sets the level of a bus it makes at once, without a
  glide. A bus that is there already glides to a new level as before.
- Found while building #436: the e2e check with a sound on a bus at
  volume 0 measured -7.8 dB on that bus. With the fix it is silent.

Tests:
- New unit test in sound-library: buses made at their level with no
  glide, a later change glides. Red without the fix.
- sound-library 130/130, tsc clean on this change alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 22:13:48 +02:00
71 changed files with 7484 additions and 265 deletions

No files matched your search

+2
View File
@@ -231,6 +231,8 @@ function copyOf(original: Asset, id: string, name: string): Asset {
if (copy.type === 'texture' || copy.type === 'pixelart')
for (const frame of copy.frames) frame.id = newId();
renewClipIds(copy, new Map());
// A node names a marker by its id alone (#338), so a copy's markers are told apart.
if (copy.type === 'song') for (const marker of copy.markers ?? []) marker.id = newId();
if (copy.type === 'scene' || copy.type === 'container' || copy.type === 'composition') {
const ids = new Map(copy.objects.map((o) => [o.id, newId()]));
for (const object of copy.objects) {
+31
View File
@@ -8,6 +8,7 @@ import type { EdgeSend, Graph } from '../graph/Graph';
import type { Asset, SongAsset } from '../model/Asset';
import { findGraphAsset, type Project } from '../model/Project';
import { settleEmbedded } from '../music/embedded';
import { heldSongMarkers, type SongMarker } from '../music/markers';
import { routedKind, sendsTapped } from '../music/projectRouting';
import { heldSend, heldSongGraph } from '../music/routing';
import { heldSongItems, heldSongTracks, type SongItem, type SongTrack } from '../music/song';
@@ -75,6 +76,36 @@ export class SetSongCommand implements Command {
}
}
/** Writes the markers of a song whole (#338): one gesture, one step to undo. */
export class SetSongMarkersCommand implements Command {
private previous: SongMarker[] | undefined;
constructor(
private readonly assetId: string,
private readonly markers: readonly SongMarker[],
readonly label = 'Edit markers',
) {}
/** Nothing to do when the song already has exactly these markers – no empty undo step. */
static changes(asset: SongAsset, markers: readonly SongMarker[]): boolean {
return JSON.stringify(asset.markers ?? []) !== JSON.stringify(heldSongMarkers(markers));
}
execute(project: Project): void {
const asset = requireSong(project, this.assetId);
this.previous = asset.markers;
const held = heldSongMarkers(this.markers);
if (held.length > 0) asset.markers = held;
else delete asset.markers;
}
undo(project: Project): void {
const asset = requireSong(project, this.assetId);
if (this.previous) asset.markers = this.previous;
else delete asset.markers;
}
}
/**
* How loud a link of a song's routing sends, and where it taps its track (#350); the same for a link
* of the master routing (#351), where only a bus and a song may tap anywhere but after their level.
+20 -2
View File
@@ -3,6 +3,7 @@ import type { CompositionPlayer } from '../model/composition';
import type { Mat } from '../model/matrix';
import type { Position, PropertyBag } from '../model/Properties';
import type { FxParams } from '../soundfx/nodes';
import type { HeardPosition, Quantize } from '../music/beats';
import type { RecordingSettings } from '../recording/recording';
import { type ChainTransform, composeAlongChain } from '../model/walk';
import { enabledAlongChain } from '../properties/enabled';
@@ -304,8 +305,9 @@ export interface NodeContext {
* Plays a song asset (#336) at its tempo, its tracks as they are set: from its start, then round
* the loop of its ruler with `loop`, else once to its end. `volume` (0 to 1) is the level of the
* whole of it, rising there from silence over `fadeIn` seconds. Owned like a `playSound` voice.
* `fx`, `fxParams` and `bus` as for `playPattern`, after its tracks. Returns its number; 0 where
* there is nothing to play.
* `fx`, `fxParams` and `bus` as for `playPattern`, after its tracks. With `live` it plays its notes
* even where it has a bake, so its tracks can be turned (#338). Returns its number; 0 where there is
* nothing to play.
*/
playSong(
songId: string,
@@ -316,7 +318,18 @@ export interface NodeContext {
fx: string,
fxParams: FxParams,
bus: string,
live: boolean,
): number;
/**
* Turns one track of a playing song (#338) to `volumeDb` on top of its own level, over `fade`
* seconds from where `quantize` lands; a playing of its bake has no tracks, which is logged.
*/
setSongTrack(playback: number, track: string, volumeDb: number, fade: number, quantize: Quantize): void;
/**
* Plays `songId` in place of a playing song (#338), `''` the same one, from the marker `marker`
* (`''` its start), where `quantize` lands: over `fade` seconds, else in a cut. It keeps its number.
*/
switchSong(playback: number, songId: string, marker: string, quantize: Quantize, fade: number): void;
/** Stops every playing of that song, `''` every song, falling silent over `fadeOut` seconds. */
stopSong(songId: string, fadeOut: number): void;
/** Stops one playing of a song or pattern by its number, falling silent over `fadeOut` seconds. */
@@ -354,6 +367,11 @@ export interface NodeContext {
setBusVolume(bus: string, volume: number, time: number, ease: EaseShare): void;
/** The volume a bus is at, `''` the master: what a game set, else that of its asset; 0 for none. */
busVolume(bus: string): number;
/**
* Where a playing song or pattern stands as it is heard (#338): the one numbered `playback`, else
* the one started last of `song` or `pattern`, of every one where both are `''`; null for none.
*/
songPosition(playback: number | null, song: string, pattern: string): HeardPosition | null;
/**
* Asks for another scene, by asset id. The swap happens once the frame is over – a graph must be
* able to finish the flow it is in, and half a frame in the old world and half in the new one is
+6 -2
View File
@@ -33,8 +33,10 @@ export const PlaySong: NodeDef = {
'is on. "fx" sends the whole song through one sound FX, after its tracks and before "volume"; ' +
'left empty, the one the song names, and None is none. ' +
'"fx params" takes values for its parameters from a Sound FX parameters node. "playback" is the ' +
'number of this playing, for Stop song and for Set sound FX parameters; 0 where the song has ' +
'nothing to play. "bus" is the bus all of it is heard on; left empty, the one the song names.',
'number of this playing, for Stop song, Set song track, Switch song and Set sound FX parameters; ' +
'0 where the song has nothing to play. "bus" is the bus all of it is heard on; left empty, the ' +
'one the song names. A song with a bake plays the bake, one sound for all its tracks; with "live ' +
'tracks" it plays its notes instead, as the Song tab does, so Set song track can turn its tracks.',
inputs: [
exec(),
songPort(),
@@ -45,6 +47,7 @@ export const PlaySong: NodeDef = {
soundFxPort(),
fxParamsPort(),
playedBusPort(),
checkPort('live', 'live tracks', 'Live tracks', false),
],
outputs: [exec(), num('playback')],
params: [],
@@ -60,6 +63,7 @@ export const PlaySong: NodeDef = {
asString(inputs.fx),
fxParamsOf(inputs.fxParams),
asString(inputs.bus),
asBoolean(inputs.live),
)
: 0;
return { next: 'exec', outputs: { playback } };
+110
View File
@@ -0,0 +1,110 @@
import { heldQuantize, QUANTIZE_OPTIONS } from '../../../music/beats';
import { TRACK_MAX_DB, TRACK_MIN_DB } from '../../../music/level';
import type { NodeDef, PortDef } from '../../NodeRegistry';
import { asNumber, asString } from '../../values';
import { exec, num, withDefault } from '../shared';
import { songPort } from './PlaySong';
/** Where a change to the music lands (#338): at once, or on the next beat, bar or marker of the playing. */
const quantizePort = (): PortDef => ({
key: 'quantize',
label: 'quantize',
type: 'string',
default: 'bar',
field: { key: 'quantize', label: 'Quantize', kind: 'enum', default: 'bar', options: [...QUANTIZE_OPTIONS] },
});
/** The playing a node changes: not typed in, as it is a number Play song hands out. */
const playbackPort = (): PortDef => ({ ...num('playback'), inline: false });
const trackPort = (): PortDef => ({
key: 'track',
label: 'track',
type: 'string',
field: { key: 'track', label: 'Track', kind: 'asset', default: '', assetTypes: ['song'], tracks: true },
});
const markerPort = (): PortDef => ({
key: 'marker',
label: 'marker',
type: 'string',
field: { key: 'marker', label: 'Marker', kind: 'asset', default: '', assetTypes: ['song'], markers: true },
});
const QUANTIZE =
'"quantize" says where it lands: Now, or on the next beat, bar or marker of the song as it is ' +
'heard, planned on the audio clock to the sample; Next marker in a song without markers is the ' +
'next bar.';
export const SetSongTrack: NodeDef = {
type: 'SetSongTrack',
label: 'Set song track',
category: 'action',
description:
'Turns one track of a playing song up or down, by the "playback" number Play song handed out: ' +
'"volume" in decibels on top of the level the song gives it, 0 as the song has it, -60 silent – ' +
'what goes into the track, its sound FX and its sends alike. It gets there over "fade" seconds. ' +
`${QUANTIZE} The track is picked as "Song / Track"; a track the playing song does not have is ` +
'left alone. A song played from its bake has no tracks to turn: switch on "live tracks" in Play ' +
'song, else the log says so. A Switch song to the same song keeps what was turned.',
inputs: [
exec(),
playbackPort(),
trackPort(),
withDefault(
{
...num('volume', 'volume (dB)'),
field: {
key: 'volume',
label: 'Volume (dB)',
kind: 'number',
default: 0,
min: TRACK_MIN_DB,
max: TRACK_MAX_DB,
},
},
0,
),
withDefault(num('fade'), 1),
quantizePort(),
],
outputs: [exec()],
params: [],
run: (ctx, inputs) => {
ctx.setSongTrack(
asNumber(inputs.playback),
asString(inputs.track),
asNumber(inputs.volume),
asNumber(inputs.fade),
heldQuantize(inputs.quantize),
);
return { next: 'exec' };
},
};
export const SwitchSong: NodeDef = {
type: 'SwitchSong',
label: 'Switch song',
category: 'action',
description:
'Plays another song, or the same one from a marker, in place of a playing song, by the ' +
'"playback" number Play song handed out; the playing keeps its number, so Stop song and Set ' +
'song track go on reaching it. "song" left empty is the same song, "marker" left empty its ' +
`start. ${QUANTIZE} With "fade" 0 it is a cut on that spot; else the old song falls silent over ` +
'that many seconds as the new one rises from it. The new song plays with the volume, loop, ' +
'bus, sound FX and "live tracks" of its Play song. A marker the song does not have is logged, ' +
'and nothing changes.',
inputs: [exec(), playbackPort(), songPort(), markerPort(), quantizePort(), withDefault(num('fade'), 0)],
outputs: [exec()],
params: [],
run: (ctx, inputs) => {
ctx.switchSong(
asNumber(inputs.playback),
asString(inputs.song),
asString(inputs.marker),
heldQuantize(inputs.quantize),
asNumber(inputs.fade),
);
return { next: 'exec' };
},
};
+34
View File
@@ -0,0 +1,34 @@
import type { NodeDef } from '../../NodeRegistry';
import { asNumber, asString } from '../../values';
import { patternPort } from '../action/PlayPattern';
import { songPort } from '../action/PlaySong';
import { PLAYBACK_INPUT, playbackPort } from '../events/musicEvents';
import { bool, num } from '../shared';
export const SongPosition: NodeDef = {
type: 'SongPosition',
label: 'Song position',
category: 'data',
description:
'Where a playing song or pattern stands, as it is heard: the output latency of the device taken ' +
'off, a baked song too, so what is drawn by it keeps to the sound. "bar" and "beat" count from 1 ' +
'as the ruler does, "fraction" is how far into the beat (0 to below 1), "seconds" how far from ' +
'the start of the song, back with the loop. Connect "playback" for that playing; else it is the ' +
'one started last of the Song or the Pattern picked (both picked, either), of every song and ' +
'pattern where none is. "playing" is off before its first note is heard and after its end; with ' +
'no such playing everything is 0.',
inputs: [songPort(), patternPort(), playbackPort()],
outputs: [num('bar'), num('beat'), num('fraction'), num('seconds'), bool('playing')],
params: [],
read: (ctx, _params, input) => {
const playback = input(PLAYBACK_INPUT);
const at = ctx.songPosition(
playback === null ? null : asNumber(playback),
asString(input('song')),
asString(input('pattern')),
);
return at
? { bar: at.bar, beat: at.beat, fraction: at.fraction, seconds: at.seconds, playing: at.playing }
: { bar: 0, beat: 0, fraction: 0, seconds: 0, playing: false };
},
};
@@ -0,0 +1,94 @@
import { textOf } from '../../../model/Properties';
import type { PropertyDef } from '../../../properties/PropertySchema';
import type { NodeDef, PortDef } from '../../NodeRegistry';
import { hiddenOrder, orderParam } from '../../eventOrder';
import { exec, num, str } from '../shared';
/** The input that picks one playing (#338): not typed in, so unconnected it is null and hears every one. */
export const PLAYBACK_INPUT = 'playback';
export const playbackPort = (): PortDef => ({ ...num(PLAYBACK_INPUT), inline: false });
const songParam: PropertyDef = {
key: 'song',
label: 'Song',
kind: 'asset',
default: '',
assetTypes: ['song'],
};
const patternParam: PropertyDef = {
key: 'pattern',
label: 'Pattern',
kind: 'asset',
default: '',
assetTypes: ['pattern'],
};
const markerParam: PropertyDef = {
key: 'marker',
label: 'Marker',
kind: 'asset',
default: '',
assetTypes: ['song'],
markers: true,
};
/** A pick left at None is not shown on the node: it hears every one. */
const hiddenEmpty =
(...keys: string[]) =>
(params: Parameters<NonNullable<NodeDef['hiddenParams']>>[0]) => [
...keys.filter((key) => textOf(params[key] ?? '') === ''),
...hiddenOrder(params),
];
const HEARS =
'It hears every song a Play song plays and every pattern a Play pattern plays, wherever it was ' +
'played; pick a Song or a Pattern to hear only that one (both picked, either), or connect ' +
'"playback" to hear only that playing. A composition plays none of them. It goes by what is heard, ' +
'the output latency of the device taken off, a baked song too. A frame comes now and then, so ' +
'"late" says how many seconds ago it was heard; one more than half a beat (at most 0.15 s) ago is ' +
'not told, and of several passed in one frame only the last.';
export const OnBeat: NodeDef = {
type: 'OnBeat',
label: 'On beat',
category: 'event',
description:
'Runs on every beat of a playing song or pattern, the first beat of a bar too. A beat is a note ' +
'of the value under the line of the time signature: a quarter in 4/4, an eighth in 6/8. "bar" ' +
'and "beat" count from 1, as the ruler does, and start over with the loop. ' +
HEARS,
inputs: [playbackPort()],
outputs: [exec(), num('playback'), num('bar'), num('beat'), num('late')],
params: [songParam, patternParam, orderParam],
hiddenParams: hiddenEmpty('song', 'pattern'),
};
export const OnBar: NodeDef = {
type: 'OnBar',
label: 'On bar',
category: 'event',
description:
'Runs at the start of every bar of a playing song or pattern, "bar" counting from 1 as the ruler ' +
'does; with the loop it starts over at the bar the loop starts in. ' +
HEARS,
inputs: [playbackPort()],
outputs: [exec(), num('playback'), num('bar'), num('late')],
params: [songParam, patternParam, orderParam],
hiddenParams: hiddenEmpty('song', 'pattern'),
};
export const OnSongMarker: NodeDef = {
type: 'OnSongMarker',
label: 'On song marker',
category: 'event',
description:
'Runs when a playing song passes a marker of its ruler, every time round the loop. Pick a Marker ' +
'to hear only that one, or a Song to hear only its markers; markers on one tick each run it. ' +
'"marker" is the id a marker is known by, "name" what it is called. Connect "playback" to hear ' +
'only that playing. It goes by what is heard, a baked song too. "late" says how many seconds ago ' +
'it was heard; one more than half a beat (at most 0.15 s) ago is not told, and of several passed ' +
'in one frame only the last.',
inputs: [playbackPort()],
outputs: [exec(), num('playback'), str('marker'), str('name'), num('late')],
params: [songParam, markerParam, orderParam],
hiddenParams: hiddenEmpty('song', 'marker'),
};
+9
View File
@@ -31,6 +31,7 @@ import { PlayPattern } from './action/PlayPattern';
import { StopPattern } from './action/StopPattern';
import { PlaySong } from './action/PlaySong';
import { StopSong } from './action/StopSong';
import { SetSongTrack, SwitchSong } from './action/songSync';
import { SetZ } from './action/SetZ';
import { Spawn } from './action/Spawn';
import { StopGame } from './action/StopGame';
@@ -125,6 +126,8 @@ import { SubList } from './data/SubList';
import { TextureSize } from './data/TextureSize';
import { OnAnimationFinished } from './events/OnAnimationFinished';
import { OnCompositionFinished } from './events/OnCompositionFinished';
import { OnBar, OnBeat, OnSongMarker } from './events/musicEvents';
import { SongPosition } from './data/SongPosition';
import { OnBehavior } from './events/OnBehavior';
import { OnClick } from './events/OnClick';
import { OnMouseWheel } from './events/OnMouseWheel';
@@ -186,6 +189,9 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
OnMouseWheel,
OnAnimationFinished,
OnCompositionFinished,
OnBeat,
OnBar,
OnSongMarker,
OnBehavior,
OnStateEntered,
OnStateExited,
@@ -323,6 +329,9 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
StopPattern,
PlaySong,
StopSong,
SetSongTrack,
SwitchSong,
SongPosition,
SetBusVolume,
BusVolume,
LoadScene,
+3
View File
@@ -1,5 +1,6 @@
import type { Graph } from '../graph/Graph';
import type { Bake } from '../music/bake';
import type { SongMarker } from '../music/markers';
import type { PatternNote } from '../music/pattern';
import type { SongItem, SongTrack } from '../music/song';
import type { LoopRange } from '../music/sequence';
@@ -490,6 +491,8 @@ export interface SongAsset extends AssetBase {
tracks: SongTrack[];
items: SongItem[];
loop?: LoopRange;
/** Named spots on its time (#338); left out without one. */
markers?: SongMarker[];
graph: Graph;
bake?: Bake;
}
+44 -3
View File
@@ -1,4 +1,6 @@
import type { FadeCurve } from '../model/sound';
import type { FxParams } from '../soundfx/nodes';
import { AUTOMATION_STEP, type AutomationTarget, type LaneMark, laneMarks, paramsAt } from './automation';
import { clipsIn, heldLoop, levelsIn, type LoopRange, notesIn, playheadAt, type Sequence } from './sequence';
import {
COMMON_TIME,
@@ -30,6 +32,8 @@ export interface ScheduledNote {
/** A factor on its loudness, and the track of a song it is heard on (#334). */
volume?: number;
bus?: string;
/** The values the parameter lanes of its track have where it starts (#337). */
params?: FxParams;
}
/** One clip handed to the player (#334): when it starts, how long it plays and from where in the sound. */
@@ -43,6 +47,8 @@ export interface ScheduledClip {
volume: number;
bus: string;
tempo: number;
/** As a note's. */
params?: FxParams;
}
/**
@@ -59,6 +65,18 @@ export interface ScheduledRamp {
done: number;
}
/**
* Where a lane of automation (#337) takes a bus over one window: each mark a second of the audio
* clock with the value the bus is ramped to, or jumps to there. With `fresh` the playing has just
* started, and what was planned for the bus from the first mark on is to go.
*/
export interface ScheduledAutomation {
bus: string;
target: AutomationTarget;
marks: LaneMark[];
fresh: boolean;
}
/** What the scheduler needs of the audio side; a test hands in a clock of its own. */
export interface SchedulerHost {
/** Seconds on the audio clock. */
@@ -77,6 +95,8 @@ export interface SchedulerHost {
playClip?(clip: ScheduledClip): number;
/** Ramps the level of a bus; a host without levels leaves it out. */
ramp?(ramp: ScheduledRamp): void;
/** Takes a bus along its automation; a host without it leaves it out. */
automate?(automation: ScheduledAutomation): void;
playing(voice: number): boolean;
/** Ends these voices; with `letRing` a note falls away in the release of its Envelope. */
stop(voices: readonly number[], letRing?: boolean): void;
@@ -143,11 +163,14 @@ export class Scheduler {
return this.ended && this.voices.every((voice) => !this.host.playing(voice));
}
/** Plays from `from` ticks of the sequence, or from where it was last started or sent. */
start(from = this.from): void {
/**
* Plays from `from` ticks of the sequence, or from where it was last started or sent – heard from
* `at` on the audio clock where that is later than it can be (#338).
*/
start(from = this.from, at = -Infinity): void {
this.stop();
this.from = Math.max(0, Math.round(from));
this.anchor = this.host.now() + LEAD + this.ahead();
this.anchor = Math.max(this.host.now() + LEAD + this.ahead(), at);
this.until = 0;
this.before = null;
this.ended = false;
@@ -206,7 +229,9 @@ export class Scheduler {
if (to <= this.until) return;
this.voices = this.voices.filter((voice) => this.host.playing(voice));
const sequence = this.sequence();
const lanes = sequence.lanes ?? [];
for (const due of notesIn(sequence, this.from, this.loop, this.until, to)) {
const params = paramsAt(lanes, due.bus, due.tick);
const voice = this.host.play({
at: this.anchor + secondsOfTicks(due.at, this.tempo),
duration: secondsOfTicks(due.length, this.tempo),
@@ -216,11 +241,13 @@ export class Scheduler {
tempo: due.tempo ?? this.tempo,
...(due.volume !== undefined && { volume: due.volume }),
...(due.bus !== undefined && { bus: due.bus }),
...(params && { params }),
});
if (voice !== 0) this.voices.push(voice);
}
if (this.host.playClip)
for (const due of clipsIn(sequence, this.from, this.loop, this.until, to, this.fresh)) {
const params = paramsAt(lanes, due.bus, due.tick + due.skip);
const voice = this.host.playClip({
at: this.anchor + secondsOfTicks(due.at, this.tempo),
duration: secondsOfTicks(due.length, this.tempo),
@@ -230,6 +257,7 @@ export class Scheduler {
volume: due.volume,
bus: due.bus,
tempo: due.tempo ?? this.tempo,
...(params && { params }),
});
if (voice !== 0) this.voices.push(voice);
}
@@ -246,6 +274,19 @@ export class Scheduler {
done: whole > 0 ? due.skip / whole : 0,
});
}
if (this.host.automate) {
const anchor = this.anchor;
const step = ticksOfSeconds(AUTOMATION_STEP, this.tempo);
for (const lane of lanes) {
const marks = laneMarks(lane, this.from, this.loop, this.until, to, this.fresh, step);
this.host.automate({
bus: lane.bus,
target: lane.target,
marks: marks.map((mark) => ({ ...mark, at: anchor + secondsOfTicks(mark.at, this.tempo) })),
fresh: this.fresh,
});
}
}
this.fresh = false;
this.until = to;
if (!this.loop && playheadAt(to, this.from, null) >= this.end) {
+414
View File
@@ -0,0 +1,414 @@
/**
* Automation of a song (#337): the volume and the pan of a track changing over the song, as points
* on a lane. From each point the value runs to the next along that point's easing, as from one
* animation key to the next. While a lane has points it rules its value in place of the fader: before
* its first point it is that point's value, after its last the last one's. Two points on one tick
* are a jump there.
*
* The scheduler reads a lane window by window as marks on the audio clock, each ramped or jumped to.
* Where the value moves along a shape the clock cannot ramp, it is read every `AUTOMATION_STEP`.
*
* A lane can also move a parameter (`param:<name>`): of the track's sound FX, of the sound FX of its
* items, of the instruments of its notes and the sound FX of its clips – wherever one of them has a
* parameter of that name, as `Set sound FX parameters` reaches them. A note or clip starts at the
* value its lane has there, so a parameter a voice only reads when it starts is heard from the next one.
*/
import type { KeyInterpolation, VectorPoint } from '../model/Asset';
import { easedShare, easingOf, isKeyInterpolation } from '../model/animation';
import type { CurveLookup } from '../model/curve';
import type { FxParams } from '../soundfx/nodes';
import { dbToGain, TRACK_MAX_DB, TRACK_MIN_DB } from './level';
import { type LoopRange, playheadAt } from './sequence';
/** What a lane always moves on a track: its fader and its pan. */
export type TrackTarget = 'volume' | 'pan';
/** A parameter by its name (#337, stage 3); no name of a parameter has a colon. */
export type ParamTarget = `param:${string}`;
export type AutomationTarget = TrackTarget | ParamTarget;
export const AUTOMATION_TARGETS: readonly TrackTarget[] = ['volume', 'pan'];
const PARAM_PREFIX = 'param:';
export function paramTarget(name: string): ParamTarget {
return `${PARAM_PREFIX}${name}`;
}
/** The parameter a target moves; null for the volume and the pan. */
export function paramOf(target: AutomationTarget): string | null {
return target.startsWith(PARAM_PREFIX) ? target.slice(PARAM_PREFIX.length) : null;
}
/** Whether a text is a target a lane can have: the volume, the pan or a parameter with a name. */
export function isAutomationTarget(value: unknown): value is AutomationTarget {
if (value === 'volume' || value === 'pan') return true;
if (typeof value !== 'string' || !value.startsWith(PARAM_PREFIX)) return false;
const name = value.slice(PARAM_PREFIX.length);
return name !== '' && !name.includes(':');
}
export interface AutomationPoint {
tick: number;
/** In the units of its target: decibels for the volume, -1 (left) to 1 (right) for the pan. */
value: number;
/** How it runs to the next point; linear where left out. */
ease?: KeyInterpolation;
}
export interface AutomationLane {
target: AutomationTarget;
/** In the order of their ticks. */
points: AutomationPoint[];
/**
* Only on a parameter's lane: the range its values are drawn and held in, `PARAM_RANGE` where left
* out. A parameter says nothing of its own range, so the lane does. Such a lane is kept without a
* point too, for its range; it rules its parameter once it has one.
*/
min?: number;
max?: number;
}
/** What says how a lane's values are held and drawn: its target, and a parameter's range. */
export type LaneKind = Pick<AutomationLane, 'target' | 'min' | 'max'>;
/** A lane's kind, or the volume or the pan by name: neither has a range of its own to name. */
export type LaneSpec = LaneKind | TrackTarget;
export function laneKind(spec: LaneSpec): LaneKind {
return typeof spec === 'string' ? { target: spec } : spec;
}
/** The range of a parameter's lane that sets none. */
export const PARAM_RANGE = { min: 0, max: 1 } as const;
/** Seconds between two readings of a value that moves along a shape the audio clock cannot ramp. */
export const AUTOMATION_STEP = 0.01;
/** The lowest and the highest value a lane of that kind holds. */
export function laneRange(spec: LaneSpec): { min: number; max: number } {
const { target, min, max } = laneKind(spec);
if (target === 'volume') return { min: TRACK_MIN_DB, max: TRACK_MAX_DB };
if (target === 'pan') return { min: -1, max: 1 };
return heldRange(min, max) ?? { ...PARAM_RANGE };
}
/** A range a parameter's lane may have: both ends finite, the top above the bottom; null otherwise. */
export function heldRange(min: unknown, max: unknown): { min: number; max: number } | null {
if (typeof min !== 'number' || typeof max !== 'number') return null;
if (!Number.isFinite(min) || !Number.isFinite(max) || !(max > min)) return null;
return { min, max };
}
/** A lane's value as its bus takes it: the volume as a gain, the pan and a parameter as they are. */
export function audioValueOf(lane: LaneKind, value: number): number {
const { min, max } = laneRange(lane);
const held = Math.max(min, Math.min(max, value));
return lane.target === 'volume' ? dbToGain(held) : held;
}
/**
* Held to what the lanes of a track may be: a target it knows, one lane for each, points in whole
* ticks from 0 with their values in range, in the order of their ticks. A lane of the volume or the
* pan without a point is dropped; a parameter's is kept for its range.
*/
export function heldLanes(raw: unknown): AutomationLane[] {
const lanes: AutomationLane[] = [];
for (const entry of Array.isArray(raw) ? (raw as unknown[]) : []) {
if (typeof entry !== 'object' || entry === null) continue;
const lane = entry as Record<string, unknown>;
const target = lane.target;
if (!isAutomationTarget(target) || lanes.some((l) => l.target === target)) continue;
const range = paramOf(target) !== null ? heldRange(lane.min, lane.max) : null;
const { min, max } = range ?? laneRange({ target });
const points = (Array.isArray(lane.points) ? (lane.points as unknown[]) : []).flatMap(
(point): AutomationPoint[] => {
if (typeof point !== 'object' || point === null) return [];
const { tick, value, ease } = point as Record<string, unknown>;
if (typeof tick !== 'number' || !Number.isFinite(tick) || !(tick >= 0)) return [];
if (typeof value !== 'number' || !Number.isFinite(value)) return [];
return [
{
tick: Math.round(tick),
value: Math.max(min, Math.min(max, value)),
...(isKeyInterpolation(ease) && ease !== 'linear' && { ease }),
},
];
},
);
points.sort((a, b) => a.tick - b.tick);
if (range) lanes.push({ target, points, ...range });
else if (points.length > 0 || paramOf(target) !== null) lanes.push({ target, points });
}
return lanes;
}
/** The parameters a track's lanes move, in their order, those without a point among them. */
export function laneParams(track: { automation?: readonly AutomationLane[] }): string[] {
return (track.automation ?? []).flatMap((lane) => paramOf(lane.target) ?? []);
}
/** The lane of a track for that target, while it has a point. */
export function laneOf(
track: { automation?: readonly AutomationLane[] },
target: AutomationTarget,
): AutomationLane | undefined {
return track.automation?.find((lane) => lane.target === target && lane.points.length > 0);
}
/** A lane on the bus it acts on, the Curve assets its points ease along looked up; one gone is linear. */
export function sequenceLaneOf(lane: AutomationLane, bus: string, curves: CurveLookup): SequenceLane {
return {
bus,
target: lane.target,
...(lane.min !== undefined && lane.max !== undefined && { min: lane.min, max: lane.max }),
points: lane.points.map((point) => {
const { shape, curve } = easingOf(point.ease ?? 'linear');
const found = shape === 'curve' ? curves(curve ?? '') : undefined;
return found ? { ...point, curve: found } : point;
}),
};
}
/** A point as a sequence plays it: a Curve asset it eases along already looked up. */
export interface SequenceLanePoint extends AutomationPoint {
curve?: readonly VectorPoint[];
}
/**
* A lane as a sequence plays it: on the bus it acts on, never without a point. A parameter's acts on
* that bus's sound FX and on everything heard on a bus inside it (`reaches`).
*/
export interface SequenceLane extends LaneKind {
bus: string;
points: readonly SequenceLanePoint[];
}
/** Whether a voice or a bus by that name is on the track of `bus` – it, or a bus inside it (`<bus>:…`). */
export function reaches(bus: string, name: string): boolean {
return name === bus || name.startsWith(`${bus}:`);
}
/**
* The values the parameter lanes of a sequence have at `tick` for what is heard on `bus`, a note's or
* a clip's; undefined where none reaches it.
*/
export function paramsAt(
lanes: readonly SequenceLane[],
bus: string | undefined,
tick: number,
): FxParams | undefined {
if (bus === undefined) return undefined;
let params: Record<string, number> | undefined;
for (const lane of lanes) {
const name = paramOf(lane.target);
if (name === null || !reaches(lane.bus, bus)) continue;
const { min, max } = laneRange(lane);
(params ??= {})[name] = Math.max(min, Math.min(max, laneValueAt(lane.points, tick)));
}
return params;
}
function between(a: SequenceLanePoint, b: SequenceLanePoint, t: number): number {
return a.value + (b.value - a.value) * easedShare(a.ease ?? 'linear', t, () => a.curve);
}
/** The index of the last point on or before `tick`, -1 for none. */
function lastAt(points: readonly SequenceLanePoint[], tick: number): number {
let i = -1;
while (i + 1 < points.length && points[i + 1]!.tick <= tick) i++;
return i;
}
/** The value at `tick`, a jump there already taken. */
export function laneValueAt(points: readonly SequenceLanePoint[], tick: number): number {
const i = lastAt(points, tick);
if (i < 0) return points[0]!.value;
const a = points[i]!;
const b = points[i + 1];
return b ? between(a, b, (tick - a.tick) / (b.tick - a.tick)) : a.value;
}
/** The value just before `tick`: what a jump there jumps from. */
export function laneValueBefore(points: readonly SequenceLanePoint[], tick: number): number {
let i = -1;
while (i + 1 < points.length && points[i + 1]!.tick < tick) i++;
if (i < 0) return points[0]!.value;
const a = points[i]!;
const b = points[i + 1];
if (!b) return a.value;
if (b.tick > tick) return between(a, b, (tick - a.tick) / (b.tick - a.tick));
return a.ease === 'step' ? a.value : between(a, b, 1);
}
/** Whether the way from `a` to `b` holds or is a straight line in lane units. */
function straight(a: SequenceLanePoint, b: SequenceLanePoint): boolean {
const ease = a.ease ?? 'linear';
return a.value === b.value || ease === 'step' || ease === 'linear';
}
/**
* Whether the bus takes it as one ramp: held, or a straight line also in its own units. A parameter
* becomes its knobs along ways of their own, so only one held is a ramp of them.
*/
function rampable(target: AutomationTarget, a: SequenceLanePoint, b: SequenceLanePoint): boolean {
const ease = a.ease ?? 'linear';
return a.value === b.value || ease === 'step' || (ease === 'linear' && target === 'pan');
}
/** One place on the way of a lane: `at` in ticks, the value as its bus takes it, ramped to or jumped to. */
export interface LaneMark {
at: number;
value: number;
jump: boolean;
}
/**
* Where a lane is over the playing time from `start` up to `end`, as `notesIn` lays out notes: a mark
* where the window starts, at every point and every reading in between, and where it ends. Where the
* playhead lands – at the start with `fresh`, after a jump back – the first mark of that stretch is a
* jump. `step` is `AUTOMATION_STEP` in ticks.
*/
export function laneMarks(
lane: SequenceLane,
from: number,
loop: LoopRange | null,
start: number,
end: number,
fresh: boolean,
step: number,
): LaneMark[] {
const { points } = lane;
const audio = (value: number) => audioValueOf(lane, value);
const marks: LaneMark[] = [];
const stretchEnd = loop !== null && from < loop.end ? loop.end : Infinity;
let elapsed = start;
let head = playheadAt(start, from, loop);
let landed = fresh;
while (elapsed < end) {
const span = Math.min(end - elapsed, stretchEnd - head);
if (!(span > 0)) break;
const to = head + span;
const first = laneValueAt(points, head);
marks.push({
at: elapsed,
value: audio(first),
jump: landed || laneValueBefore(points, head) !== first,
});
for (const tick of readings(lane, head, to, step)) {
const before = laneValueBefore(points, tick);
const value = laneValueAt(points, tick);
const at = elapsed + (tick - head);
marks.push({ at, value: audio(before), jump: false });
if (value !== before) marks.push({ at, value: audio(value), jump: true });
}
marks.push({ at: elapsed + span, value: audio(laneValueBefore(points, to)), jump: false });
elapsed += span;
landed = to >= stretchEnd;
head = landed ? loop!.start : to;
}
return thinned(marks);
}
/** The ticks strictly between `head` and `to` to mark: every point, and readings where the bus cannot ramp. */
function readings(lane: SequenceLane, head: number, to: number, step: number): number[] {
const { points, target } = lane;
const ticks: number[] = [];
points.forEach((a, i) => {
if (a.tick > head && a.tick < to) ticks.push(a.tick);
const b = points[i + 1];
if (b && b.tick > head && a.tick < to && !rampable(target, a, b))
ticks.push(...stepsOf(a.tick, head, Math.min(b.tick, to), step));
});
ticks.sort((x, y) => x - y);
return ticks.filter((tick, i) => i === 0 || tick !== ticks[i - 1]);
}
/** The ticks `step` apart from `origin` on, strictly between `after` and `before`; none without a step. */
function stepsOf(origin: number, after: number, before: number, step: number): number[] {
const ticks: number[] = [];
if (!(step > 0)) return ticks;
for (let k = Math.max(1, Math.floor((after - origin) / step) + 1); origin + k * step < before; k++)
if (origin + k * step > after) ticks.push(origin + k * step);
return ticks;
}
/** Without the marks a ramp would pass anyway: a value held from the mark before on. */
function thinned(marks: readonly LaneMark[]): LaneMark[] {
const kept: LaneMark[] = [];
marks.forEach((mark, i) => {
const last = kept.at(-1);
const next = marks[i + 1];
const held =
last !== undefined &&
next !== undefined &&
!mark.jump &&
last.value === mark.value &&
(next.jump || next.value === mark.value);
if (!held) kept.push(mark);
});
return kept;
}
/**
* A lane as it is heard over the playing time from `start` up to `end`, played from 0 round `loop` –
* a song in an item of a composition (#418) –, as a lane of its own in ticks of that playing time:
* each round with its points, cut at the loop end and jumping back there. A way cut in the middle
* that is no straight line is read every `step` ticks.
*/
export function unrolledLane(
points: readonly SequenceLanePoint[],
loop: LoopRange | null,
start: number,
end: number,
step: number,
): SequenceLanePoint[] {
const out: SequenceLanePoint[] = [];
const stretchEnd = loop !== null && loop.end > 0 ? loop.end : Infinity;
let elapsed = start;
let head = playheadAt(start, 0, loop);
while (elapsed < end) {
const span = Math.min(end - elapsed, stretchEnd - head);
if (!(span > 0)) break;
const shift = elapsed - head;
for (const point of stretchOf(points, head, head + span, step))
out.push({ ...point, tick: point.tick + shift });
elapsed += span;
head = head + span >= stretchEnd ? loop!.start : head + span;
}
return out;
}
/** The points of a lane from `head` to `to` as a lane by itself, ending on the value it reaches there. */
function stretchOf(
points: readonly SequenceLanePoint[],
head: number,
to: number,
step: number,
): SequenceLanePoint[] {
const out: SequenceLanePoint[] = [];
const at = lastAt(points, head);
const starts: { tick: number; own: number; whole: boolean }[] = [
{ tick: head, own: Math.max(0, at), whole: at >= 0 && points[at]!.tick === head },
];
points.forEach((point, i) => {
if (point.tick > head && point.tick < to) starts.push({ tick: point.tick, own: i, whole: true });
});
starts.forEach((s, k) => {
const next = starts[k + 1]?.tick ?? to;
const a = points[s.own]!;
const b = points[s.own + 1];
const value = s.whole ? a.value : laneValueAt(points, s.tick);
if (!b || (k === 0 && at < 0)) out.push({ tick: s.tick, value });
else if (s.whole && b.tick <= to) out.push({ ...a, tick: s.tick });
else if (straight(a, b)) out.push({ tick: s.tick, value, ...(a.ease === 'step' && { ease: 'step' }) });
else {
out.push({ tick: s.tick, value });
for (const tick of stepsOf(a.tick, s.tick, next, step))
out.push({ tick, value: laneValueAt(points, tick) });
}
});
out.push({ tick: to, value: laneValueBefore(points, to) });
return out;
}
+2
View File
@@ -233,6 +233,8 @@ export function bakePrint(project: Project, asset: BakedAsset): string {
clips,
buses,
instruments,
// Only where there is any, so a bake made before automation (#337) still holds.
...(sequence.lanes?.length ? [sequence.lanes] : []),
]);
return hashText(text);
}
+275
View File
@@ -0,0 +1,275 @@
/**
* Where a playing of a song or a pattern is heard, and the beats, bars and markers it has passed
* (#338). The audio side keeps each playing's time as an anchor on its clock; a game looks once a
* frame, so whatever it passed since the last look is told late, and too late is not told at all.
*/
import type { SongMarker } from './markers';
import { playheadAt, type LoopRange } from './sequence';
import { type Meter, positionOf, secondsOfTicks, ticksOfSeconds, ticksPerBar, ticksPerBeat } from './time';
/** The time of one playing: how fast, in what meter, from where, round what, to where. */
export interface Timeline {
tempo: number;
meter: Meter;
/** Where it started, in ticks. */
from: number;
/** What it runs round once it gets there; null plays it once to `length`. */
loop: LoopRange | null;
/** Where it ends when it does not loop, in ticks. */
length: number;
}
/** The most a spot passed is told late, and less where half a beat is shorter. */
export const LATE_MAX = 0.15;
/**
* How late a spot may be told: half a beat, at most `LATE_MAX`. A frame that slips loses no beat;
* after a long stall the old ones are gone, and no burst of them follows.
*/
export function lateLimit(timeline: Pick<Timeline, 'tempo' | 'meter'>): number {
return Math.min(LATE_MAX, secondsOfTicks(ticksPerBeat(timeline.meter), timeline.tempo) / 2);
}
/** What `Song position` hands on. */
export interface HeardPosition {
bar: number;
beat: number;
/** How far into the beat, 0 to below 1. */
fraction: number;
/** Seconds from the start of the song to where it stands; back to the loop's start with it. */
seconds: number;
tick: number;
/** Heard now: started, and not played to its end. */
playing: boolean;
}
/**
* Where a playing stands at `heard` seconds of its playing time, as heard: before it starts (null, or
* less than 0) where it starts, after its end at the end.
*/
export function heardPosition(timeline: Timeline, heard: number | null): HeardPosition {
const { tempo, meter, from, loop, length } = timeline;
let tick = from;
let playing = false;
if (heard !== null && heard >= 0) {
tick = playheadAt(ticksOfSeconds(heard, tempo), from, loop);
playing = !ends(timeline) || tick < length;
if (!playing) tick = length;
}
const beat = ticksPerBeat(meter);
const at = positionOf(tick, meter);
const inBeat = tick - Math.floor(tick / beat) * beat;
return {
bar: at.bar,
beat: at.beat,
fraction: Math.min(Math.max(inBeat / beat, 0), 1 - Number.EPSILON),
seconds: secondsOfTicks(tick, tempo),
tick,
playing,
};
}
/** A spot the playhead passed: the tick it is at, and how many seconds ago it was heard. */
export interface Passed {
tick: number;
late: number;
}
/**
* The ticks among the playhead's run from `low` to `high` that are spots of a kind. `lowOpen` leaves
* `low` itself out (it was told before), `highOpen` leaves `high` out (the loop's end is its start).
*/
export type SpotsIn = (low: number, high: number, lowOpen: boolean, highOpen: boolean) => number[];
/** Every multiple of `step` in the run: the beats or the bars. */
export function gridSpots(step: number): SpotsIn {
return (low, high, lowOpen, highOpen) => {
const spots: number[] = [];
let n = Math.ceil(low / step);
if (lowOpen && n * step <= low) n++;
for (; n * step < high || (!highOpen && n * step === high); n++) spots.push(n * step);
return spots;
};
}
/** Those of `ticks` in the run: the markers. */
export function tickSpots(ticks: readonly number[]): SpotsIn {
return (low, high, lowOpen, highOpen) =>
[...new Set(ticks)].filter((t) => (lowOpen ? t > low : t >= low) && (highOpen ? t < high : t <= high));
}
/** Whether it ends rather than going round. */
function ends(timeline: Timeline): boolean {
return timeline.loop === null;
}
/** One stretch of playing time the playhead runs straight through: from `start` on, at `head`, to `end`. */
interface Stretch {
start: number;
head: number;
end: number;
}
/** The stretch playing time `elapsed` (in ticks) lies in. */
function stretchAt({ from, loop }: Timeline, elapsed: number): Stretch {
if (!loop || from >= loop.end) return { start: 0, head: from, end: Infinity };
const toEnd = loop.end - from;
if (elapsed < toEnd) return { start: 0, head: from, end: toEnd };
const round = loop.end - loop.start;
const start = toEnd + Math.floor((elapsed - toEnd) / round) * round;
return { start, head: loop.start, end: start + round };
}
/** The stretch that runs up to where `stretch` begins; null for the first. */
function stretchBefore({ from, loop }: Timeline, stretch: Stretch): Stretch | null {
if (!loop || stretch.start <= 0) return null;
const toEnd = loop.end - from;
const round = loop.end - loop.start;
const start = stretch.start - round;
return start >= toEnd
? { start, head: loop.start, end: stretch.start }
: { start: 0, head: from, end: toEnd };
}
/** No more stretches than this are looked back through: a loop shorter than a frame is no tune. */
const MOST_STRETCHES = 16;
/**
* The spots the playhead passed after `before` up to `now` (seconds of playing time heard, `before`
* null where nothing was heard of it yet), newest first – only those no more than `lateLimit` ago.
* A loop's start is passed each time it comes round; nothing at or after the end of one that ends.
*/
export function passed(timeline: Timeline, before: number | null, now: number, spots: SpotsIn): Passed[] {
const low = Math.max(before ?? -Infinity, now - lateLimit(timeline));
if (!(now >= 0) || !(now > low)) return [];
const { tempo, length } = timeline;
const elapsedLow = ticksOfSeconds(low, tempo);
const high = ticksOfSeconds(now, tempo);
const found: Passed[] = [];
let stretch: Stretch | null = stretchAt(timeline, high);
let highOpen = false;
for (let n = 0; stretch && n < MOST_STRETCHES; n++) {
// What was heard at `low` was told before; where the run reaches back past a stretch's start,
// that start is among it – the very start of the playing too, before which nothing was heard.
const lowInside = elapsedLow >= stretch.start;
const headLow = stretch.head + ((lowInside ? elapsedLow : stretch.start) - stretch.start);
let headHigh = stretch.head + ((highOpen ? stretch.end : high) - stretch.start);
let open = highOpen;
if (ends(timeline) && headHigh >= length) {
headHigh = length;
open = true;
}
const ticks = spots(headLow, headHigh, lowInside, open).sort((a, b) => b - a);
for (const tick of ticks) {
const at = stretch.start + (tick - stretch.head);
found.push({ tick, late: Math.max(0, now - secondsOfTicks(at, tempo)) });
}
if (lowInside) break;
stretch = stretchBefore(timeline, stretch);
highOpen = true;
}
return found;
}
/** A beat passed, as `On beat` tells it. */
export interface PassedBeat extends Passed {
bar: number;
beat: number;
}
/** The newest beat passed, if one is still to be told. */
export function passedBeat(timeline: Timeline, before: number | null, now: number): PassedBeat | null {
const [newest] = passed(timeline, before, now, gridSpots(ticksPerBeat(timeline.meter)));
if (!newest) return null;
const at = positionOf(newest.tick, timeline.meter);
return { ...newest, bar: at.bar, beat: at.beat };
}
/** A marker passed, as `On song marker` tells it. */
export interface PassedMarker extends Passed {
id: string;
name: string;
}
/** Every marker on a tick passed, newest first; markers sharing a tick in their own order. */
export function passedMarkers(
timeline: Timeline,
before: number | null,
now: number,
markers: readonly SongMarker[],
): PassedMarker[] {
if (markers.length === 0) return [];
return passed(timeline, before, now, tickSpots(markers.map((m) => m.tick))).flatMap((spot) =>
markers.filter((m) => m.tick === spot.tick).map((m) => ({ ...spot, id: m.id, name: m.name })),
);
}
/** Of what passed, those passed last: one spot, but every marker on it. */
export function newestPassed<T extends Passed>(list: readonly T[]): T[] {
const [first] = list;
return first ? list.filter((p) => p.late === first.late) : [];
}
/** Where a change a game asks of its music lands (#338): at once, or on the next of these. */
export type Quantize = 'now' | 'beat' | 'bar' | 'marker';
export const QUANTIZE_OPTIONS: readonly { value: Quantize; label: string }[] = [
{ value: 'now', label: 'Now' },
{ value: 'beat', label: 'Next beat' },
{ value: 'bar', label: 'Next bar' },
{ value: 'marker', label: 'Next marker' },
];
export function heldQuantize(value: unknown): Quantize {
return QUANTIZE_OPTIONS.some((option) => option.value === value) ? (value as Quantize) : 'bar';
}
/** The spots a change waits for; a song without markers, or a pattern, waits for the next bar. */
export function quantizeSpots(
quantize: Exclude<Quantize, 'now'>,
meter: Meter,
markers: readonly SongMarker[],
): SpotsIn {
if (quantize === 'beat') return gridSpots(ticksPerBeat(meter));
if (quantize === 'marker' && markers.length > 0) return tickSpots(markers.map((m) => m.tick));
return gridSpots(ticksPerBar(meter));
}
/** The furthest ahead a spot is looked for on a stretch without end. */
const LOOK_AHEAD = 600;
/** The stretch that follows `stretch`; null after the last one of a playing that ends. */
function stretchAfter(timeline: Timeline, stretch: Stretch): Stretch | null {
return stretch.end === Infinity ? null : stretchAt(timeline, stretch.end);
}
/**
* The first spot at or after `elapsed` seconds of playing time, in seconds of playing time; null where
* none comes before a playing that ends does – a change then lands at once.
*/
export function nextSpot(timeline: Timeline, elapsed: number, spots: SpotsIn): number | null {
const { tempo, length } = timeline;
const low = ticksOfSeconds(Math.max(0, elapsed), tempo);
let stretch: Stretch | null = stretchAt(timeline, low);
for (let n = 0; stretch && n < MOST_STRETCHES; n++) {
const headLow = stretch.head + (Math.max(low, stretch.start) - stretch.start);
// One that runs on without end is looked through for `LOOK_AHEAD` seconds, not for ever.
let headHigh =
stretch.head + (Math.min(stretch.end, low + ticksOfSeconds(LOOK_AHEAD, tempo)) - stretch.start);
if (ends(timeline)) headHigh = Math.min(headHigh, length);
if (headHigh > headLow) {
const [first] = spots(headLow, headHigh, false, true).sort((a, b) => a - b);
if (first !== undefined) return secondsOfTicks(stretch.start + (first - stretch.head), tempo);
}
if (ends(timeline) && headHigh >= length) return null;
stretch = stretchAfter(timeline, stretch);
}
return null;
}
/** The newest bar line passed, if one is still to be told. */
export function passedBar(timeline: Timeline, before: number | null, now: number): PassedBeat | null {
const [newest] = passed(timeline, before, now, gridSpots(ticksPerBar(timeline.meter)));
if (!newest) return null;
return { ...newest, bar: positionOf(newest.tick, timeline.meter).bar, beat: 1 };
}
+16 -2
View File
@@ -13,6 +13,7 @@ import { compositionFps, compositionLength, compositionLoops, itemFades } from '
import { findAssetOf, type Project } from '../model/Project';
import { findSound } from '../model/sound';
import { pickedFx } from '../properties/assetFx';
import { AUTOMATION_STEP, type SequenceLane, unrolledLane } from './automation';
import { itemRoute, MASTER_MIX, type MixRoute } from './mixer';
import { fxVoiceOf } from './needs';
import { patternInstrument, patternLength, patternSequence, patternTempo } from './pattern';
@@ -75,6 +76,7 @@ export function compositionPlayings(
(playing.sequence.notes as SequenceNote[]).push(...laid.notes);
(playing.sequence.clips as SequenceClip[]).push(...laid.clips);
(playing.sequence.levels as SequenceLevel[]).push(...laid.levels);
if (laid.lanes?.length) playing.sequence.lanes = [...(playing.sequence.lanes ?? []), ...laid.lanes];
playing.buses.push(...laid.buses);
}
const playings = [...byRoute.values()];
@@ -100,6 +102,8 @@ interface Laid {
clips: SequenceClip[];
/** Its fades, on its bus. */
levels: SequenceLevel[];
/** A song's automation (#337), round its loop as the song plays inside the item. */
lanes?: SequenceLane[];
buses: BusLevel[];
/** The asset its routing is read from: the sound of a clip, else what it plays. */
routedAs: string;
@@ -168,6 +172,11 @@ function laidOut(project: Project, compId: string, item: CompositionItem, fps: n
return {
notes: heard.notes.map((n) => ({ ...n, tick: begin + n.tick, ...(n.bus && { bus: renamed(n.bus) }) })),
clips: heard.clips.map((c) => ({ ...c, tick: begin + c.tick, bus: renamed(c.bus) })),
lanes: heard.lanes.map((lane) => ({
...lane,
bus: renamed(lane.bus),
points: lane.points.map((p) => ({ ...p, tick: begin + p.tick })),
})),
levels,
buses: [withFx(pickedFx('', song.props), tempo), ...tracks],
routedAs: song.id,
@@ -207,7 +216,7 @@ function played(
tempo: number,
into: number,
seconds: number,
): { notes: SequenceNote[]; clips: SequenceClip[] } {
): { notes: SequenceNote[]; clips: SequenceClip[]; lanes: SequenceLane[] } {
const from = Math.round(ticksOfSeconds(into, tempo));
const to = from + Math.ceil(ticksOfSeconds(seconds, tempo));
const span = compositionTicks(seconds);
@@ -226,5 +235,10 @@ function played(
const { at: _, skip, ...clip } = c;
return [{ ...clip, tick, length, offset: c.offset + secondsOfTicks(skip, tempo), tempo }];
});
return { notes, clips };
const step = ticksOfSeconds(AUTOMATION_STEP, tempo);
const lanes = (sequence.lanes ?? []).map((lane) => ({
...lane,
points: unrolledLane(lane.points, loop, from, to, step).map((p) => ({ ...p, tick: at(p.tick) })),
}));
return { notes, clips, lanes };
}
+7
View File
@@ -0,0 +1,7 @@
/** The volume of a track of a song (#334), in decibels; at the lowest it is silent. */
export const TRACK_MIN_DB = -60;
export const TRACK_MAX_DB = 12;
export function dbToGain(db: number): number {
return db <= TRACK_MIN_DB ? 0 : 10 ** (db / 20);
}
+57
View File
@@ -0,0 +1,57 @@
/**
* Markers of a song (#338): named spots on its time, where an event fires in the game and where a
* switch to the song can start. A marker sits on a whole tick; two may share one, and so may their
* names – a node names a marker by its id.
*/
import { newId } from '../ids';
export interface SongMarker {
id: string;
name: string;
tick: number;
}
const finite = (value: unknown) => (typeof value === 'number' && Number.isFinite(value) ? value : NaN);
/** Held to what a marker may be: a whole tick from 0, a name, an id no other has; in the order of their ticks. */
export function heldSongMarkers(raw: unknown): SongMarker[] {
const ids = new Set<string>();
const markers: SongMarker[] = [];
for (const entry of Array.isArray(raw) ? (raw as unknown[]) : []) {
if (typeof entry !== 'object' || entry === null) continue;
const m = entry as Record<string, unknown>;
const tick = Math.round(finite(m.tick));
if (!(tick >= 0)) continue;
let id = typeof m.id === 'string' && m.id ? m.id : newId();
while (ids.has(id)) id = newId();
ids.add(id);
const name = typeof m.name === 'string' ? m.name.trim() : '';
markers.push({ id, name: name || `Marker ${markers.length + 1}`, tick });
}
return markers.sort((a, b) => a.tick - b.tick);
}
/** "Marker 3": the first number no marker is called by yet. */
export function nextMarkerName(markers: readonly SongMarker[]): string {
const taken = new Set(markers.map((m) => m.name));
let n = 1;
while (taken.has(`Marker ${n}`)) n++;
return `Marker ${n}`;
}
export function newMarker(markers: readonly SongMarker[], tick: number, name?: string): SongMarker {
return { id: newId(), name: name?.trim() || nextMarkerName(markers), tick: Math.max(0, Math.round(tick)) };
}
/** The markers with one of them changed; the others as they are. */
export function changedMarker(
markers: readonly SongMarker[],
id: string,
change: Partial<Pick<SongMarker, 'name' | 'tick'>>,
): SongMarker[] {
return markers.map((m) => (m.id === id ? { ...m, ...change } : m));
}
export function withoutMarker(markers: readonly SongMarker[], id: string): SongMarker[] {
return markers.filter((m) => m.id !== id);
}
+3 -3
View File
@@ -22,9 +22,9 @@ export function fxVoiceOf(
return plan ? { plan, params, tempo } : null;
}
/** The sound FX a clip goes through. */
export function clipFx(project: Project, fxId: string, tempo: number): FxVoice {
return fxVoiceOf(project, fxId, tempo) ?? DRY_VOICE;
/** The sound FX a clip goes through, with the values its parameters start at. */
export function clipFx(project: Project, fxId: string, tempo: number, params: FxParams = {}): FxVoice {
return fxVoiceOf(project, fxId, tempo, params) ?? DRY_VOICE;
}
/** Every layer and FX of every note and every file and FX of every clip a sequence plays. */
+209 -22
View File
@@ -1,9 +1,11 @@
import type { SendTap } from '../graph/Graph';
import type { FxPlan, FxVoice } from '../soundfx/nodes';
import type { AutomationTarget } from './automation';
import type { BakePlayback } from './bake';
import type { MixSend } from './mixer';
import {
LEAD,
type ScheduledAutomation,
type ScheduledClip,
type ScheduledNote,
type ScheduledRamp,
@@ -30,6 +32,11 @@ export interface BusLevel {
sends?: readonly BusSend[];
/** The sound FX all of it goes through as one chain (#346), before its level and pan. */
fx?: FxVoice;
/**
* What its automation sets while it plays (#337): `gain`, `pan` and a parameter of `fx` are then
* only where it starts.
*/
automated?: readonly AutomationTarget[];
}
/** One send of a track (#350): into another track's bus, or the output of the song where `to` is left out. */
@@ -64,6 +71,8 @@ export interface SequencePlay {
strikeClip?(clip: ScheduledClip): number;
/** Ramps the level of one of its buses (#418). */
ramp?(ramp: ScheduledRamp): void;
/** Takes one of its buses along its automation (#337). */
automate?(automation: ScheduledAutomation): void;
/** Its bake (#340), played in place of its notes and clips where the host can. */
bake?: BakePlay;
/**
@@ -87,27 +96,59 @@ export interface Runner {
readonly startedAt: number | null;
/** Played to its end, and nothing of it is heard any more. */
readonly done: boolean;
start(from?: number): void;
/** Plays from `from` ticks, heard from `at` on the audio clock where that is still to come (#338). */
start(from?: number, at?: number): void;
stop(letRing?: boolean): void;
}
export interface PlaybackHost extends Omit<SchedulerHost, 'play' | 'playClip' | 'ahead' | 'ramp'> {
/** What a change a game asks of a playing goes by (#338): its time, where it started, the clock now. */
export interface PlayingClock {
time: PlayingTime;
/** The second of the audio clock its playing time 0 is heard at; null before it starts. */
startedAt: number | null;
now: number;
/** The soonest second of the audio clock the change can be heard from. */
soonest: number;
}
/** The second of the audio clock a change to a playing lands at, from its clock (#338). */
export type LandingAt = (clock: PlayingClock) => number;
/** Seconds a hard cut between two playings takes, so it does not click. */
export const CUT = 0.005;
/**
* The audio side of the playings. Each is reached here by its `key`, the number its buses go by: the
* same as its `id` but where it took over another one's (#338).
*/
export interface PlaybackHost extends Omit<
SchedulerHost,
'play' | 'playClip' | 'ahead' | 'ramp' | 'automate'
> {
/** Loads what that playing needs and makes its buses ready, their sound FX among them (#346). */
prepare(needs: SequenceNeeds, id: number): Promise<void>;
prepare(needs: SequenceNeeds, key: number): Promise<void>;
/** How far ahead of its time a note of that playing may be started (see `SchedulerHost.ahead`). */
ahead?(id: number): number;
ahead?(key: number): number;
/** The level of a playing, from where it is at `at` to `gain`, reached `seconds` later. */
level(id: number, gain: number, at: number, seconds: number): void;
level(key: number, gain: number, at: number, seconds: number): void;
/** Calls `then` once the audio clock reaches `at`. */
after(at: number, then: () => void): void;
/** A playing is gone: stopped, or played to its end. */
closed?(id: number): void;
closed?(key: number): void;
/** Starts the bake of that playing to be heard from `at`, as one voice; 0 where nothing plays. */
playBaked?(id: number, bake: BakePlay, at: number): number;
playBaked?(key: number, bake: BakePlay, at: number): number;
}
export interface Playback {
id: number;
/** What its buses go by; its `id` but where it took over from another playing (#338). */
key: number;
/** The level of the whole of it, as it was asked for. */
volume: number;
/** The second of the audio clock another playing takes over from it at (#338); null while it is the one. */
until: number | null;
/** It takes over from another playing of its number, once it is heard (#338). */
successor: boolean;
/** What stands for the pattern or song, as a file stands for a sound: `pattern:<id>`, `song:<id>`. */
name: string;
owner: string;
@@ -119,6 +160,26 @@ export interface Playback {
fading: number | null;
/** Played from its bake (#340) rather than its notes. */
baked: boolean;
/** How fast, round what and to where it plays, and from where it started (#338). */
time: PlayingTime;
}
/** The time of a playing (#338): its tempo, where it started, its loop and its length, in ticks. */
export interface PlayingTime {
tempo: number;
from: number;
loop: LoopRange | null;
length: number;
}
/** A playing and how far into its playing time it is heard now, null before it starts (#338). */
export interface HeardTime extends PlayingTime {
id: number;
/** Told apart from the playing of its number it took over from (`Playback.key`). */
key: number;
name: string;
heard: number | null;
baked: boolean;
}
/** Whether a playing goes by `name`: its own, every one of a kind for `kind:`, every one for `''`. */
@@ -139,6 +200,112 @@ export class Playbacks {
/** Starts once its files are there; a stop that came first wins. */
start(id: number, name: string, play: SequencePlay, owner: string): void {
const playback = this.make(id, id, name, play, owner, false);
const asked = this.host.now();
void this.host.prepare(play.needs, id).then(() => {
if (!this.list.includes(playback) || playback.fading !== null) return;
const from = play.from;
playback.time.from =
from === undefined ? 0 : Math.round(from + ticksOfSeconds(this.host.now() - asked, play.tempo));
playback.runner.start(playback.time.from);
// From the first note heard, which is later where the buses are (#346).
const at = playback.runner.startedAt ?? this.host.now() + LEAD;
this.host.level(id, playback.volume, at, Math.max(0, play.fadeIn ?? 0));
});
}
/**
* Plays `play` in place of the playing numbered `id` (#338), on its buses `key`, from `play.from`
* ticks: once its files are there, it is heard from the second `at` names for the playing it takes
* over from, which falls silent there – over `fade` seconds as this one rises over them, else in a
* cut. It keeps the number, so whatever knows the playing goes on reaching it. Asked again before
* that, the one still waiting is dropped and the playing heard now is the one to go. Where nothing
* of that number plays any more, it starts as soon as it can.
*/
switchTo(
id: number,
key: number,
name: string,
play: SequencePlay,
owner: string,
at: LandingAt,
fade: number,
): void {
const now = this.host.now();
const waiting = this.list.filter((p) => p.id === id && this.waits(p, now));
this.close(waiting, false);
for (const p of this.list)
if (p.id === id && p.until !== null && p.until > now) {
p.until = null;
p.fading = null;
this.host.level(p.key, p.volume, now, 0);
}
const playback = this.make(id, key, name, play, owner, true);
void this.host.prepare(play.needs, key).then(() => {
if (!this.list.includes(playback) || playback.fading !== null) return;
const from = this.heardOne(id, playback);
const now = this.host.now();
const ahead = this.host.ahead?.(key) ?? 0;
const soonest = now + LEAD + (Number.isFinite(ahead) && ahead > 0 ? ahead : 0);
const clock = from && { time: from.time, startedAt: from.runner.startedAt, now, soonest };
playback.time.from = Math.max(0, Math.round(play.from ?? 0));
playback.runner.start(playback.time.from, clock ? at(clock) : undefined);
const start = playback.runner.startedAt ?? soonest;
const over = fade > 0 ? fade : 0;
if (from) this.handOver(from, start, Math.max(over, CUT));
this.host.level(key, playback.volume, start, over);
});
}
/** The playing of that number heard now, but `besides`: the one a switch takes over from. */
private heardOne(id: number, besides: Playback): Playback | null {
return this.current().find((p) => p.id === id && p !== besides && p.until === null) ?? null;
}
/** Falls silent from `at` over `fade` seconds and goes then: the one taken over from (#338). */
private handOver(playback: Playback, at: number, fade: number): void {
const end = at + fade;
playback.until = at;
playback.fading = end;
this.host.level(playback.key, 0, at, fade);
this.host.after(end, () => {
if (this.list.includes(playback) && playback.fading === end) this.close([playback], false);
});
}
/** The clock of the playing of that number heard now (#338), and whether it is a bake; null where none is. */
clockOf(id: number): (PlayingClock & { baked: boolean }) | null {
const playing = this.current().find((p) => p.id === id && p.until === null && !this.waits(p));
if (!playing) return null;
const now = this.host.now();
const { time, baked } = playing;
return { time, startedAt: playing.runner.startedAt, now, soonest: now + CUT, baked };
}
/** The playings of that number, the one heard now and one waiting to take over from it (#338). */
named(id: number): readonly Readonly<Playback>[] {
return this.current().filter((p) => p.id === id);
}
/** Whether the buses `key` belong to a playing still to be heard. */
holds(key: number): boolean {
return this.current().some((playback) => playback.key === key);
}
/** A successor not heard yet: waiting on its files, or on the second it takes over at. */
private waits(playback: Playback, now = this.host.now()): boolean {
const at = playback.runner.startedAt;
return playback.successor && (at === null || at > now);
}
private make(
id: number,
key: number,
name: string,
play: SequencePlay,
owner: string,
successor: boolean,
): Playback {
const planned: ScheduledNote[] = [];
const clips: ScheduledClip[] = [];
const keep = <T>(list: T[], item: T) => {
@@ -146,12 +313,12 @@ export class Playbacks {
if (list.length > KEPT) list.shift();
};
const strikeClip = play.strikeClip;
const ahead = () => this.host.ahead?.(id) ?? 0;
const ahead = () => this.host.ahead?.(key) ?? 0;
const { bake } = play;
const playBaked = this.host.playBaked?.bind(this.host);
const runner: Runner =
bake && playBaked
? new BakedRun(this.host, ahead, (at) => playBaked(id, bake, at), bake.playback.end !== null)
? new BakedRun(this.host, ahead, (at) => playBaked(key, bake, at), bake.playback.end !== null)
: new Scheduler(
{
...this.host,
@@ -159,12 +326,14 @@ export class Playbacks {
play: (note) => (keep(planned, note), play.strike(note)),
...(strikeClip && { playClip: (clip: ScheduledClip) => (keep(clips, clip), strikeClip(clip)) }),
...(play.ramp && { ramp: play.ramp }),
...(play.automate && { automate: play.automate }),
},
() => play.sequence,
{ tempo: play.tempo, loop: play.loop, end: play.length },
);
const playback: Playback = {
id,
key,
name,
owner,
runner,
@@ -172,17 +341,13 @@ export class Playbacks {
clips,
fading: null,
baked: runner instanceof BakedRun,
volume: play.volume ?? 1,
until: null,
successor,
time: { tempo: play.tempo, from: 0, loop: play.loop, length: play.length },
};
this.list.push(playback);
const asked = this.host.now();
void this.host.prepare(play.needs, id).then(() => {
if (!this.list.includes(playback) || playback.fading !== null) return;
const from = play.from;
runner.start(from === undefined ? 0 : from + ticksOfSeconds(this.host.now() - asked, play.tempo));
// From the first note heard, which is later where the buses are (#346).
const at = runner.startedAt ?? this.host.now() + LEAD;
this.host.level(id, play.volume ?? 1, at, Math.max(0, play.fadeIn ?? 0));
});
return playback;
}
/** Whether that one is still to be heard: waiting on its files, playing, fading, or its last notes ringing. */
@@ -190,6 +355,27 @@ export class Playbacks {
return this.current().some((playback) => playback.id === id);
}
/**
* Every playing with how far into its playing time it is heard now: the audio clock less the
* output latency, from the moment its first note is heard (#338). With none playing the clock is
* not asked: asking for it makes the audio context, which a page may make only after a gesture.
*/
times(): HeardTime[] {
const list = this.current();
if (list.length === 0) return [];
const heard = this.host.now() - this.host.latency();
// Of a switch (#338) one is heard at a time: the old one up to where the new one is.
const told = list.filter(
({ until, successor, runner }) =>
(until === null || heard < until) &&
(!successor || (runner.startedAt !== null && heard >= runner.startedAt)),
);
return told.map(({ id, key, name, runner, time, baked }) => {
const at = runner.startedAt;
return { id, key, name, ...time, heard: at === null ? null : heard - at, baked };
});
}
/** Those that go by `name` (see `goesBy`); with `fade` they fall silent over that many seconds first. */
stopNamed(name: string, letRing = false, fade = 0): void {
this.stopWhere((playback) => goesBy(playback.name, name), letRing, fade);
@@ -219,7 +405,7 @@ export class Playbacks {
for (const playback of stopping) {
if (playback.fading !== null && playback.fading <= end) continue;
playback.fading = end;
this.host.level(playback.id, 0, now, fade);
this.host.level(playback.key, 0, now, fade);
this.host.after(end, () => {
if (this.list.includes(playback) && playback.fading === end) this.close([playback], false);
});
@@ -232,7 +418,7 @@ export class Playbacks {
this.list = this.list.filter((playback) => !going.includes(playback));
for (const playback of going) {
playback.runner.stop(letRing);
this.host.closed?.(playback.id);
this.host.closed?.(playback.key);
}
}
}
@@ -264,10 +450,11 @@ class BakedRun implements Runner {
return this.anchor !== null && this.ends && !this.host.playing(this.voice);
}
start(): void {
/** From its start: a bake plays from nowhere else. Heard from `at` where that is still to come. */
start(_from?: number, at = -Infinity): void {
this.stop();
const ahead = this.ahead();
this.anchor = this.host.now() + LEAD + (Number.isFinite(ahead) && ahead > 0 ? ahead : 0);
this.anchor = Math.max(this.host.now() + LEAD + (Number.isFinite(ahead) && ahead > 0 ? ahead : 0), at);
this.voice = this.play(this.anchor);
}
+3
View File
@@ -5,6 +5,7 @@
* where the playhead stands, which jumps back at the end of a loop.
*/
import type { FadeCurve } from '../model/sound';
import type { SequenceLane } from './automation';
export interface SequenceNote {
/** Where it starts in the sequence, and how long it is held, in ticks. */
@@ -59,6 +60,8 @@ export interface Sequence {
notes: readonly SequenceNote[];
clips?: readonly SequenceClip[];
levels?: readonly SequenceLevel[];
/** The automation of a song's tracks (#337), each lane on the bus it acts on. */
lanes?: readonly SequenceLane[];
}
/** From `start` up to `end`, in ticks of the sequence; `end` is where the playhead jumps back. */
+15 -7
View File
@@ -8,10 +8,13 @@
import { newId } from '../ids';
import { HIGHEST_NOTE, LOWEST_NOTE } from '../instrument/notes';
import type { PatternAsset, SongAsset, SoundAsset } from '../model/Asset';
import { curveLookup } from '../model/curve';
import { type PropertyBag, textOf } from '../model/Properties';
import { findAssetOf, type Project } from '../model/Project';
import { findSound } from '../model/sound';
import { pickedFx } from '../properties/assetFx';
import { type AutomationLane, heldLanes, sequenceLaneOf } from './automation';
import { dbToGain, TRACK_MAX_DB, TRACK_MIN_DB } from './level';
import { gridTicks, GRID_OPTIONS, patternInstrument, patternLength } from './pattern';
import { heldLoop, type LoopRange, type Sequence, type SequenceClip, type SequenceNote } from './sequence';
import { DEFAULT_SIGN, heldTempo, type Meter, secondsOfTicks, signMeter, ticksPerBar } from './time';
@@ -27,6 +30,8 @@ export interface SongTrack {
solo: boolean;
/** The sound FX everything on the track goes through, before its volume and pan (#346); `''` for none. */
fx: string;
/** Its volume, pan and parameters over the song (#337); left out without a lane. */
automation?: AutomationLane[];
}
export type SongItemKind = 'pattern' | 'sound';
@@ -66,8 +71,7 @@ const DEFAULT_TRACKS = 1;
/** A song is never shorter than this many bars, on the ruler or when it plays. */
const MIN_BARS = 1;
export const TRACK_MIN_DB = -60;
export const TRACK_MAX_DB = 12;
export { dbToGain, TRACK_MAX_DB, TRACK_MIN_DB };
export const ITEM_MIN_DB = -60;
/** A sound item can be turned up; a pattern item's notes are held to unity, as a note always is. */
export const SOUND_ITEM_MAX_DB = 12;
@@ -138,10 +142,6 @@ export function itemBus(songId: string, item: Pick<SongItem, 'id' | 'track'>): s
return `${trackBus(songId, item.track)}:${item.id}`;
}
export function dbToGain(db: number): number {
return db <= TRACK_MIN_DB ? 0 : 10 ** (db / 20);
}
/** The sound FX an item goes through (#352): its own pick, else the one of what it plays; '' for none. */
export function itemFx(project: Project, item: SongItem): string {
return pickedFx(item.fx, itemAsset(project, item)?.props ?? {});
@@ -228,7 +228,13 @@ export function songSequence(project: Project, song: SongAsset): Sequence {
}
notes.sort((a, b) => a.tick - b.tick || a.note - b.note);
clips.sort((a, b) => a.tick - b.tick);
return { notes, clips };
const curves = curveLookup(project);
const lanes = song.tracks.flatMap((track) =>
(track.automation ?? [])
.filter((lane) => lane.points.length > 0)
.map((lane) => sequenceLaneOf(lane, trackBus(song.id, track.id), curves)),
);
return { notes, clips, ...(lanes.length > 0 && { lanes }) };
}
const finite = (value: unknown, fallback: number) =>
@@ -255,6 +261,8 @@ export function heldSongTracks(raw: unknown): SongTrack[] {
solo: t.solo === true,
fx: text(t.fx),
});
const automation = heldLanes(t.automation);
if (automation.length > 0) tracks.at(-1)!.automation = automation;
}
return tracks.length > 0 ? tracks : [newTrack(1)];
}
+38 -8
View File
@@ -1,6 +1,7 @@
import type { Project } from '../model/Project';
import type { PropertyBag } from '../model/Properties';
import type { Graph } from '../graph/Graph';
import { audioValueOf, laneOf, laneValueAt, paramOf, type TrackTarget } from './automation';
import { fxVoiceOf } from './needs';
import type { BusLevel } from './playbacks';
import { songRoutes, trackGates } from './routing';
@@ -18,8 +19,10 @@ import {
/**
* The buses of a song: each track's at its volume and pan, through its sound FX (#346), on to where
* its routing sends it (#350); a bus for its own items into it; and one for each pattern item with a
* sound FX into that, at unity – its volume is its notes'. Mute and solo close sends and items
* its routing sends it (#350), a level, pan or parameter its lane rules (#337) from where the lane
* starts – a parameter on the sound FX of its items too; a bus
* for its own items into it; and one for each pattern item with a sound FX into that, at unity – its
* volume is its notes'. Mute and solo close sends and items
* (`trackGates`). A sound FX that does not compile is none, and the bus then only passes it on.
*/
export function songBuses(
@@ -33,18 +36,36 @@ export function songBuses(
},
): BusLevel[] {
const tempo = songTempo(song.props);
const fxOf = (id: string) => {
const fx = fxVoiceOf(project, id, tempo);
const byId = new Map(song.tracks.map((track) => [track.id, track]));
// A parameter a lane rules starts where the lane does (#337).
const fxOf = (id: string, track: SongTrack | undefined) => {
const lanes = (track?.automation ?? []).filter((lane) => paramOf(lane.target) !== null);
const params = Object.fromEntries(
lanes.flatMap((lane) =>
lane.points.length > 0
? [[paramOf(lane.target)!, audioValueOf(lane, laneValueAt(lane.points, 0))]]
: [],
),
);
const fx = fxVoiceOf(project, id, tempo, params);
return fx ? { fx } : {};
};
const routes = songRoutes(song);
const gates = trackGates(song.tracks, routes);
const tracks = song.tracks.map((track) => {
const gate = gates.get(track.id);
const automated = (track.automation ?? []).flatMap((lane) =>
lane.points.length > 0 ? [lane.target] : [],
);
const atStart = (target: TrackTarget, fader: number) => {
const lane = laneOf(track, target);
return lane ? audioValueOf(lane, laneValueAt(lane.points, 0)) : fader;
};
return {
name: trackBus(song.id, track.id),
gain: dbToGain(track.volume),
pan: track.pan,
gain: atStart('volume', dbToGain(track.volume)),
pan: atStart('pan', track.pan),
...(automated.length > 0 && { automated }),
sends: routes
.filter((route) => route.from === track.id)
.map((route) => ({
@@ -52,7 +73,7 @@ export function songBuses(
gain: gate?.passes(route.to) ? dbToGain(route.volume) : 0,
tap: route.tap,
})),
...fxOf(track.fx),
...fxOf(track.fx, track),
};
});
const own = song.tracks.map((track) => ({
@@ -69,7 +90,16 @@ export function songBuses(
gain: 1,
pan: 0,
into: trackItemsBus(song.id, item.track),
...fxOf(itemFx(project, item)),
...fxOf(itemFx(project, item), byId.get(item.track)),
...automatedParams(byId.get(item.track)),
}));
return [...tracks, ...own, ...items];
}
/** The parameters a track's lanes rule, for the bus of one of its items: its sound FX leaves them alone. */
function automatedParams(track: SongTrack | undefined): Pick<BusLevel, 'automated'> {
const automated = (track?.automation ?? []).flatMap((lane) =>
paramOf(lane.target) !== null && lane.points.length > 0 ? [lane.target] : [],
);
return automated.length > 0 ? { automated } : {};
}
+63
View File
@@ -0,0 +1,63 @@
import { VELOCITY_PARAM } from '../instrument/nodes';
import { instrumentParams } from '../instrument/compile';
import { findAssetOf, type Project } from '../model/Project';
import { assetFxOf } from '../properties/assetFx';
import { soundFxParams } from '../soundfx/compile';
import { patternInstrument } from './pattern';
import { itemFx, type SongItem, type SongTrack } from './song';
/** A parameter a lane of a track can move, with the default it has where it is first declared. */
export interface TrackParam {
name: string;
default: number;
}
/**
* The parameters a lane of a track can move (#337): those of its sound FX, of the sound FX of its
* items, of the instruments its pattern items play and of their own sound FX, each name once, in
* that order. A name several of them have moves all of them.
*/
export function trackParams(
project: Project,
song: { items: readonly SongItem[] },
track: SongTrack,
): TrackParam[] {
const found = new Map<string, number>();
const add = (ports: readonly { key: string; default?: number }[]) => {
for (const port of ports)
if (!found.has(port.key) && port.key !== VELOCITY_PARAM) found.set(port.key, port.default ?? 0);
};
const addFx = (fxId: string) => {
const fx = fxId ? findAssetOf(project, fxId, 'soundfx') : undefined;
if (fx) add(soundFxParams(fx.graph));
};
addFx(track.fx);
for (const item of song.items) {
if (item.track !== track.id) continue;
addFx(itemFx(project, item));
if (item.kind !== 'pattern') continue;
const pattern = findAssetOf(project, item.asset, 'pattern');
const id = item.instrument || (pattern ? patternInstrument(pattern.props) : '');
const instrument = id ? findAssetOf(project, id, 'instrument') : undefined;
if (!instrument) continue;
add(instrumentParams(instrument.graph));
addFx(assetFxOf(instrument.props));
}
return [...found].map(([name, value]) => ({ name, default: value }));
}
/**
* The range a new lane of a parameter gets: 0 to 1 for a default in there, else from 0 – or as far
* below – to twice the default, on a round number.
*/
export function defaultParamRange(value: number): { min: number; max: number } {
if (!Number.isFinite(value) || (value >= 0 && value <= 1)) return { min: 0, max: 1 };
const top = roundUp(Math.abs(value) * 2);
return value > 0 ? { min: 0, max: top } : { min: -top, max: top };
}
/** The next of 1, 2 and 5 times a power of ten from `value` up. */
function roundUp(value: number): number {
const power = 10 ** Math.floor(Math.log10(value));
return ([1, 2, 5, 10].find((step) => step * power >= value) ?? 10) * power;
}
+4
View File
@@ -159,6 +159,10 @@ type Extras<K extends FlatKind> = K extends 'number'
/** Offers `TO_COMPOSITION_BUS`, where empty means the routing of the asset. */
toComposition?: boolean;
noFx?: boolean;
/** Offers the markers of the songs, each as "Song / Marker", instead of the songs (#338). */
markers?: boolean;
/** Offers the tracks of the songs, each as "Song / Track", instead of the songs (#338). */
tracks?: boolean;
}
: K extends 'texture'
? { uploadedOnly?: boolean; moving?: boolean }
+49 -2
View File
@@ -1,7 +1,8 @@
import type { Asset, SoundAsset } from '../model/Asset';
import { type AssetType, isPlaceableType } from '../model/AssetTypes';
import { findAsset, type Project } from '../model/Project';
import { clipsOf, findSound, soundRefName } from '../model/sound';
import { clipRefName, clipsOf, findSound, soundRefName } from '../model/sound';
import { heldSongMarkers } from '../music/markers';
import { listedAssets, markedName } from '../music/embedded';
import { NO_FX } from './assetFx';
import {
@@ -29,7 +30,53 @@ export function assetRefLabel(project: Project | null, id: string): string {
if (asset) return markedName(asset);
// A clip of a sound (#357) reads "Sound / Clip", as a frame does.
const clip = project ? findSound(project, id) : null;
return clip ? soundRefName(clip, markedName(clip.asset)) : `${id}?`;
if (clip) return soundRefName(clip, markedName(clip.asset));
// A marker of a song (#338) reads "Song / Marker", as a clip does.
const marker = project ? findSongMarker(project, id) : null;
if (marker) return clipRefName(markedName(marker.song), marker.name);
// So does a track of a song (#338).
const track = project ? findSongTrack(project, id) : null;
return track ? clipRefName(markedName(track.song), track.name) : `${id}?`;
}
/** The first song with a track of that id (#338), with the track's name; null where none has one. */
export function findSongTrack(project: Project, id: string): { song: Asset; name: string } | null {
if (id === '') return null;
for (const song of project.assets) {
if (song.type !== 'song') continue;
const track = song.tracks.find((t) => t.id === id);
if (track) return { song, name: track.name };
}
return null;
}
/** What a track field offers (#338): the tracks of every song, "Theme / Drums", in their order. */
export function trackChoices(project: Project): { id: string; label: string }[] {
return assetRefChoices(project, ['song']).flatMap((song) =>
song.type === 'song'
? song.tracks.map((t) => ({ id: t.id, label: clipRefName(markedName(song), t.name) }))
: [],
);
}
/** The marker of a song an id names (#338), with its song; null where none does. */
export function findSongMarker(project: Project, id: string): { song: Asset; name: string } | null {
if (id === '') return null;
for (const song of project.assets) {
if (song.type !== 'song') continue;
const marker = heldSongMarkers(song.markers).find((m) => m.id === id);
if (marker) return { song, name: marker.name };
}
return null;
}
/** What a marker field offers (#338): the markers of every song, "Theme / Chorus", in the order of their ticks. */
export function markerChoices(project: Project): { id: string; label: string }[] {
return assetRefChoices(project, ['song']).flatMap((song) =>
song.type === 'song'
? heldSongMarkers(song.markers).map((m) => ({ id: m.id, label: clipRefName(markedName(song), m.name) }))
: [],
);
}
/** What a sound field offers (#357): each sound, followed by its clips. */
+3
View File
@@ -25,6 +25,7 @@ import { heldBake } from '../music/bake';
import { heldStream } from '../music/streams';
import { heldPatternNotes } from '../music/pattern';
import { isEmbeddable } from '../music/embedded';
import { heldSongMarkers } from '../music/markers';
import { heldSongItems, heldSongTracks } from '../music/song';
import { heldSend, heldSongGraph } from '../music/routing';
import { isMasterRoutingDef, isSongRoutingDef } from '../music/routingNodes';
@@ -348,6 +349,7 @@ export function deserializeProject(raw: unknown, fallbackId = ''): Project {
const loop = readLoop(a.loop);
const bake = heldBake(a.bake);
const tracks = heldSongTracks(a.tracks);
const markers = heldSongMarkers(a.markers);
assets.push({
id,
type,
@@ -356,6 +358,7 @@ export function deserializeProject(raw: unknown, fallbackId = ''): Project {
tracks,
items: heldSongItems(a.items, tracks),
...(loop && { loop }),
...(markers.length > 0 && { markers }),
graph: heldSongGraph(normalizeGraph(a.graph, isSongRoutingDef), tracks),
...(bake && { bake }),
});
+6 -1
View File
@@ -984,6 +984,7 @@ export class Editor implements PlayHost {
play: this.play,
follows: followsAgent,
stores,
heard: () => soundLibrary.heard(),
},
wait,
),
@@ -993,7 +994,11 @@ export class Editor implements PlayHost {
this.state.setNotice('Agent switched off: another tab of your account has it now');
},
});
link.onChange = () => this.syncAgentBadge();
link.onChange = () => {
this.syncAgentBadge();
// The buses are tapped for what the agent hears only while it is switched on (#436).
soundLibrary.watch(link.on);
};
return link;
}
+171
View File
@@ -0,0 +1,171 @@
import { noteName } from '../../core/instrument/notes';
import { findAsset, type Project } from '../../core/model/Project';
import { clipsOf } from '../../core/model/sound';
import { MASTER_MIX } from '../../core/music/mixer';
import { playedSoundOf } from '../../core/properties/playedSound';
import type { World } from '../../runtime/World';
import type { HeardSound, HeardVoice } from '../../viewport/SoundLibrary';
/** Below this a bus counts as silent: far under anything a speaker gives out. */
export const SILENT_DB = -90;
/** The most voices an answer lists; the rest are counted. */
export const MAX_VOICES = 40;
export const STALLED_HINT =
'The browser holds the audio clock while the game runs, so nothing is heard and notes stay planned. ' +
'The browser starts it only once the user has clicked or pressed a key in the editor page, and not ' +
'without a sound output: ask the user.';
const INSTRUMENT = 'instrument:';
const PLAYING_BUS = /^play:(\d+)(?::(.*))?$/;
/** The bus of a track of a song played on its own: `play:<n>:song:<song>:<track>`. */
const SONG_TRACK = /^song:([^:]+):([^:#]+)$/;
/**
* What `get_runtime` says of the sound (#436): the level of the master and of every bus that is
* heard by itself, the voices sounding now with what they play and who asked, the patterns, songs
* and compositions playing, and what is not heard for lack of its bytes. Names stand in for ids.
*/
export function audioView(heard: HeardSound, project: Project, world: World): object {
const names = new SoundNames(project, heard, world);
const sounding = heard.voices.filter((voice) => voice.startsIn === 0);
const planned = heard.voices.length - sounding.length;
// A pause holds the clock; its state is the one it had while the game ran.
const { state, time } = heard.clock;
return {
clock: { state, time: Math.round(time * 1000) / 1000 },
...(state === 'suspended' && { hint: STALLED_HINT }),
window: Math.round(heard.window * 100) / 100,
levels: heard.buses
.filter((bus) => shownBus(bus.name))
.sort((a, b) => Number(b.name === MASTER_MIX) - Number(a.name === MASTER_MIX))
.map(({ name, peak, rms }) => levelRow(names.bus(name), peak, rms)),
voices: sounding.slice(0, MAX_VOICES).map((voice) => ({
...names.voice(voice),
...(voice.released && { released: true }),
...(voice.ringing && { ringing: true }),
})),
...(sounding.length > MAX_VOICES && { moreVoices: sounding.length - MAX_VOICES }),
...(planned > 0 && { planned }),
playing: heard.playings.map((playing) => ({
...names.playing(playing.name),
...(names.by(playing.owner) && { by: names.by(playing.owner) }),
...(!playing.playing && { ended: true }),
...(playing.fading !== null && { fading: true }),
...(playing.baked && { baked: true }),
})),
...(heard.waiting.length > 0 && { waiting: heard.waiting.map((voice) => names.voice(voice)) }),
...(heard.failed.length > 0 && { failed: heard.failed.map((file) => names.file(file)) }),
};
}
/**
* The buses worth a level of their own: the master, those of the mix, the master of each playing
* and the tracks of a song. What lies inside a track – its items, its own bus for them – is heard on
* the track.
*/
function shownBus(name: string): boolean {
if (name === MASTER_MIX || name.startsWith('mix:')) return true;
const playing = PLAYING_BUS.exec(name);
return playing !== null && (playing[2] === undefined || SONG_TRACK.test(playing[2]));
}
function levelRow(bus: string, peak: readonly number[], rms: readonly number[]): object {
const loudest = decibels(Math.max(...peak));
if (loudest < SILENT_DB) return { bus, silent: true };
const power = rms.reduce((sum, side) => sum + side * side, 0) / rms.length;
return { bus, peak: round(loudest), rms: round(decibels(Math.sqrt(power))) };
}
function decibels(gain: number): number {
return gain > 0 ? 20 * Math.log10(gain) : -Infinity;
}
function round(db: number): number {
return Number.isFinite(db) ? Math.round(db * 10) / 10 : SILENT_DB;
}
/** Names for what the sink knows by file, by bus and by the id of an entity. */
class SoundNames {
private readonly files = new Map<string, Record<string, string>>();
private readonly playings: ReadonlyMap<number, string>;
constructor(
private readonly project: Project,
heard: HeardSound,
private readonly world: World,
) {
for (const asset of project.assets) {
if (asset.type === 'sound') {
this.files.set(asset.file, { sound: asset.name });
for (const clip of clipsOf(asset))
this.files.set(playedSoundOf({ asset, clip }), { sound: `${asset.name} / ${clip.name}` });
}
if ((asset.type === 'pattern' || asset.type === 'song') && asset.bake)
this.files.set(asset.bake.file, { bake: `${asset.type} ${asset.name}` });
}
this.playings = new Map(heard.playings.map(({ id, name }) => [id, this.label(name)]));
}
voice(voice: HeardVoice): object {
const by = this.by(voice.owner);
return {
...this.file(voice.file, voice.note),
bus: this.bus(voice.bus),
...(by && { by }),
};
}
/** What a file plays: a sound or a clip of one, a note of an instrument, the bake of a pattern or song. */
file(file: string, note?: number): Record<string, string> {
if (file.startsWith(INSTRUMENT)) {
const id = file.slice(INSTRUMENT.length);
return {
instrument: findAsset(this.project, id)?.name ?? id,
...(note !== undefined && { note: noteName(note) }),
};
}
return this.files.get(file) ?? { file };
}
bus(name: string): string {
if (name === '' || name === MASTER_MIX) return 'master';
if (name.startsWith('mix:')) {
const id = name.slice('mix:'.length);
return findAsset(this.project, id)?.name ?? id;
}
const playing = PLAYING_BUS.exec(name);
if (!playing) return name;
const of = this.playings.get(Number(playing[1])) ?? `playing ${playing[1]}`;
const track = SONG_TRACK.exec(playing[2] ?? '');
if (!track) return of;
const song = findAsset(this.project, track[1]!);
const named = song?.type === 'song' ? song.tracks.find((t) => t.id === track[2]) : undefined;
return `${of}, track ${named?.name ?? track[2]}`;
}
playing(name: string): Record<string, string> {
const [kind, id] = splitName(name);
return kind ? { [kind]: findAsset(this.project, id)?.name ?? id } : { name };
}
/** The object a voice's behavior sits on, or the behavior itself; '' for no owner. */
by(owner: string): string {
if (owner === '') return '';
const entity = this.world.entities.find((e) => e.id === owner);
if (!entity) return owner;
return entity.parent?.name ?? entity.name;
}
private label(name: string): string {
const [kind, id] = splitName(name);
return kind ? `${kind} ${findAsset(this.project, id)?.name ?? id}` : name;
}
}
/** `pattern:<id>`, `song:<id>` or `composition:<id>` as kind and id; no kind for anything else. */
function splitName(name: string): [string, string] {
const at = name.indexOf(':');
const kind = at > 0 ? name.slice(0, at) : '';
return ['pattern', 'song', 'composition'].includes(kind) ? [kind, name.slice(at + 1)] : ['', name];
}
+1 -1
View File
@@ -65,7 +65,7 @@ const keyOf = (args: Args, name: string): AnimationKey => {
};
/** Every easing a key can name, but the curve ones: those take the id of a Curve asset. */
const EASINGS = EASING_SHAPES.flatMap(({ value }) =>
export const EASINGS = EASING_SHAPES.flatMap(({ value }) =>
value === 'linear' || value === 'step'
? [value]
: value === 'curve'
+207 -1
View File
@@ -9,7 +9,7 @@ import { oneStep } from '../../../core/commands/CompoundCommand';
import { ReplaceGraphCommand } from '../../../core/commands/graphCommands';
import { SetLoopCommand, SetPatternNotesCommand } from '../../../core/commands/patternCommands';
import { requireAsset, requireEdge, requirePattern, requireSong } from '../../../core/commands/require';
import { SetSendCommand, SetSongCommand } from '../../../core/commands/songCommands';
import { SetSendCommand, SetSongCommand, SetSongMarkersCommand } from '../../../core/commands/songCommands';
import type { SendTap } from '../../../core/graph/Graph';
import { newId } from '../../../core/ids';
import {
@@ -20,6 +20,7 @@ import {
TONE_PRESETS,
} from '../../../core/instrument/chipPresets';
import { HIGHEST_NOTE, LOWEST_NOTE, noteName } from '../../../core/instrument/notes';
import { easingOf, isKeyInterpolation } from '../../../core/model/animation';
import type { PatternAsset, SongAsset } from '../../../core/model/Asset';
import { audioSeconds } from '../../../core/model/composition';
import type { PropertyBag } from '../../../core/model/Properties';
@@ -31,6 +32,16 @@ import {
ROUTING_ID,
} from '../../../core/model/Project';
import { clipsOf, findSound, hasClips } from '../../../core/model/sound';
import {
type AutomationLane,
type AutomationPoint,
heldRange,
isAutomationTarget,
type LaneKind,
laneRange,
paramOf,
} from '../../../core/music/automation';
import { changedMarker, newMarker, type SongMarker, withoutMarker } from '../../../core/music/markers';
import { DEFAULT_VELOCITY, patternLength, type PatternNote } from '../../../core/music/pattern';
import { SEND_TAPS } from '../../../core/music/routing';
import {
@@ -47,8 +58,11 @@ import {
TRACK_MIN_DB,
} from '../../../core/music/song';
import { TICKS_PER_QUARTER, ticksOfSeconds, ticksPerBar } from '../../../core/music/time';
import { defaultParamRange, trackParams } from '../../../core/music/trackParams';
import { NO_FX } from '../../../core/properties/assetFx';
import { withLanePoints, withoutLane, withParamLane } from '../../song/laneEdit';
import type { ArgSchema } from '../argCheck';
import { EASINGS } from './animation';
import {
type Args,
type CatalogCommand,
@@ -68,6 +82,7 @@ import {
const PATTERN = id('Id of the pattern asset.');
const SONG = id('Id of the song asset.');
const TRACK = id('Id of the track, as get_asset lists it under tracks.');
const MARKER = id('Id of the marker, as get_asset lists it under markers.');
const ITEM = id('Id of the item, as get_asset lists it under items.');
const INDEX = whole('Place in the list of tracks, from 0 (the top).', 0);
const TICK = whole(`Ticks from the start; ${TICKS_PER_QUARTER} to a quarter note.`, 0);
@@ -280,6 +295,81 @@ export const MUSIC_COMMANDS: readonly CatalogCommand[] = [
},
shows: theTab,
},
{
name: 'set_song_lane',
description:
'Sets an automation lane of a song track: its volume (in dB), its pan (-1 left to 1 right) or a parameter ' +
'changing over the song along points; the points given replace those the lane had. From each point the ' +
'value runs to the next along its ease; before the first point it holds the first value, after the last ' +
'the last one; two points on one tick jump there. While the lane of the volume or the pan has a point, it ' +
"rules that value in place of the track's own; with no point that lane goes and the track's value rules " +
'again. A parameter lane moves every parameter of that name on the track: of its sound FX, of the sound FX ' +
'of its items, of the instruments its patterns play and of their sound FX (get_asset lists them under ' +
"params of the track). It keeps a range its points lie in, round the parameter's default where none is " +
'given, and stays without a point until remove. A note or clip starts at the value there, so what a ' +
'voice reads only when it starts changes from the next one.',
args: object(
{
asset: SONG,
track: TRACK,
target: id(
'"volume", "pan", or "param:<name>" for a parameter, as get_asset lists lanes under automation.',
),
points: {
type: 'array',
items: object(
{
tick: TICK,
value: number(
'In the units of the lane: dB, -1 to 1 for the pan, within the range of a parameter.',
),
ease: id(
`How it runs to the next point: ${EASINGS.join(', ')}, or "curve:<id of a Curve asset>"; ` +
'linear where left out.',
),
},
['tick', 'value'],
),
description: 'All points of the lane, in any order; those on one tick in the order given.',
},
min: number('A parameter lane: the lowest value it holds.'),
max: number('A parameter lane: the highest value it holds.'),
remove: flag("Take the lane away, a parameter's range with it."),
},
['asset', 'track', 'target'],
),
build: (args, { project }) => {
const song = requireSong(project, text(args, 'asset'));
const track = requireTrack(song, text(args, 'track'));
const target = args.target;
if (!isAutomationTarget(target))
throw new CommandError(
`${JSON.stringify(target)} is no lane – give "volume", "pan" or "param:<name>"`,
);
const had = track.automation?.find((lane) => lane.target === target);
const given = [args.points, args.min, args.max].some((value) => value !== undefined);
let changed: SongTrack;
let label: string;
if (args.remove === true) {
if (given) throw new CommandError('remove takes no points and no range');
if (!had) throw new CommandError(`${track.name} has no ${target} lane`);
changed = withoutLane(track, target);
label = 'Remove automation lane';
} else {
const name = paramOf(target);
const range = name === null ? noRange(args) : paramRange(project, song, track, name, had, args);
const kind: LaneKind = { target, ...range };
const points = args.points === undefined ? null : lanePointsOf(project, kind, args.points as Args[]);
if (name === null && !points) throw new CommandError('Give the points, [] to take the lane away');
changed = range ? withParamLane(track, name!, range) : track;
if (points) changed = withLanePoints(changed, target, points);
label = had ? 'Set automation lane' : 'Add automation lane';
}
const tracks = song.tracks.map((t) => (t.id === track.id ? changed : t));
return { command: new SetSongCommand(song.id, { tracks }, label) };
},
shows: theTab,
},
{
name: 'move_song_track',
description: 'Moves a track of a song to another place in its list.',
@@ -309,6 +399,57 @@ export const MUSIC_COMMANDS: readonly CatalogCommand[] = [
},
shows: theTab,
},
{
name: 'add_song_marker',
description:
'Adds a marker to a song: a named spot on its time, drawn on the strip under the ruler of the song tab. ' +
'Markers may share a tick and a name; a marker is known by its id.',
args: object({ asset: SONG, tick: TICK, name: id('Its name; "Marker <n>" where left out.') }, [
'asset',
'tick',
]),
build: (args, { project }) => {
const song = requireSong(project, text(args, 'asset'));
const markers = song.markers ?? [];
const name = typeof args.name === 'string' ? args.name : undefined;
const marker = newMarker(markers, numberOf(args, 'tick'), name);
return {
command: new SetSongMarkersCommand(song.id, [...markers, marker], 'Add marker'),
made: { id: marker.id },
};
},
shows: theTab,
},
{
name: 'change_song_marker',
description: 'Renames a marker of a song or moves it to another tick. Give only what changes.',
args: object({ asset: SONG, marker: MARKER, name: id('Its name.'), tick: TICK }, ['asset', 'marker']),
build: (args, { project }) => {
const song = requireSong(project, text(args, 'asset'));
const marker = requireMarker(song, text(args, 'marker'));
const change: { name?: string; tick?: number } = {};
if (typeof args.name === 'string') {
if (!args.name.trim()) throw new CommandError('A marker needs a name');
change.name = args.name.trim();
}
if (typeof args.tick === 'number') change.tick = numberOf(args, 'tick');
const markers = changedMarker(song.markers ?? [], marker.id, change);
return { command: new SetSongMarkersCommand(song.id, markers, 'Change marker') };
},
shows: theTab,
},
{
name: 'remove_song_marker',
description: 'Removes a marker of a song.',
args: object({ asset: SONG, marker: MARKER }, ['asset', 'marker']),
build: (args, { project }) => {
const song = requireSong(project, text(args, 'asset'));
const marker = requireMarker(song, text(args, 'marker'));
const markers = withoutMarker(song.markers ?? [], marker.id);
return { command: new SetSongMarkersCommand(song.id, markers, 'Delete marker') };
},
shows: theTab,
},
{
name: 'add_song_item',
description:
@@ -537,12 +678,77 @@ function itemLength(project: Project, song: SongAsset, kind: SongItem['kind'], p
return Math.max(1, Math.round(ticksOfSeconds(seconds, songTempo(song.props))));
}
/** Refuses a range given to the lane of the volume or the pan: theirs is fixed. */
function noRange(args: Args): null {
if (args.min !== undefined || args.max !== undefined)
throw new CommandError('Only a parameter lane has a range – the volume and the pan have theirs');
return null;
}
/**
* The range of a parameter's lane: the one given, else the one it has, else one round the default of
* the parameter. A new lane is refused for a parameter nothing on the track has.
*/
function paramRange(
project: Project,
song: SongAsset,
track: SongTrack,
name: string,
had: AutomationLane | undefined,
args: Args,
): { min: number; max: number } {
const offered = trackParams(project, song, track);
const param = offered.find((p) => p.name === name);
if (!had && !param)
throw new CommandError(
offered.length === 0
? `Nothing on ${track.name} has a parameter to move`
: `Nothing on ${track.name} has a parameter ${name} – it has ${offered.map((p) => p.name).join(', ')}`,
);
const given = [args.min, args.max].filter((value) => value !== undefined).length;
if (given === 1) throw new CommandError('Give min and max, or neither');
if (given === 2) {
const range = heldRange(args.min, args.max);
if (!range) throw new CommandError('max has to be above min');
return range;
}
return had ? laneRange(had) : defaultParamRange(param!.default);
}
/**
* The points the agent wrote, refused where a value or an easing is wrong. The song puts them in the
* order of their ticks, those on one tick as they came.
*/
function lanePointsOf(project: Project, kind: LaneKind, raw: readonly Args[]): AutomationPoint[] {
const { min, max } = laneRange(kind);
return raw.map((point): AutomationPoint => {
const tick = point.tick as number;
const value = within(point.value as number, min, max, `value at tick ${tick}`);
const ease = point.ease;
if (ease === undefined || ease === 'linear') return { tick, value };
if (!isKeyInterpolation(ease))
throw new CommandError(
`${JSON.stringify(ease)} is no easing – give ${EASINGS.join(', ')} or "curve:<id>"`,
);
const { curve } = easingOf(ease);
if (curve !== undefined && !findAssetOf(project, curve, 'curve'))
throw new CommandError(`${curve} is no Curve asset`);
return { tick, value, ease };
});
}
function requireTrack(song: SongAsset, trackId: string): SongTrack {
const track = song.tracks.find((t) => t.id === trackId);
if (!track) throw new CommandError(`${song.name} has no track ${trackId}`);
return track;
}
function requireMarker(song: SongAsset, markerId: string): SongMarker {
const marker = song.markers?.find((m) => m.id === markerId);
if (!marker) throw new CommandError(`${song.name} has no marker ${markerId}`);
return marker;
}
/** A pitch the agent wrote as a MIDI number or a name like "C#4". */
export function pitchOf(value: unknown): number {
const number =
+11 -1
View File
@@ -13,6 +13,7 @@ import type { StepTarget } from './commandCatalog';
import { screenshot } from './screenshot';
import type { FileAssetType } from './storedFiles';
import type { FileStore } from '../../core/store/FileStore';
import type { HeardSound } from '../../viewport/SoundLibrary';
/** What the agent's tools reach of the editor beyond its state. */
export interface EditorParts {
@@ -32,6 +33,8 @@ export interface EditorParts {
follows(): boolean;
/** Where the files of each kind are kept (#431). */
stores: Record<FileAssetType, FileStore>;
/** What is heard now (#436). */
heard(): HeardSound;
}
/** How long the user has to answer an agent's question before it counts as a no. */
@@ -92,7 +95,14 @@ export function editorToolHost(parts: EditorParts, wait: () => void): ToolHost {
const run = parts.play.agentRun();
if (!run) return null;
const { world, input, shared } = run;
return { world, shared, project: state.project, heldKeys: input.heldKeys, paused: state.paused };
return {
world,
shared,
project: state.project,
heldKeys: input.heldKeys,
paused: state.paused,
audio: parts.heard(),
};
},
takesLong: wait,
files: (type) => parts.stores[type].list(),
+16 -3
View File
@@ -17,6 +17,7 @@ import { instrumentSourcesOf, soundingNotes } from '../../core/instrument/compil
import { noteName } from '../../core/instrument/notes';
import { patternInstrument, patternLength, patternSequence } from '../../core/music/pattern';
import { itemNotes, songLength } from '../../core/music/song';
import { trackParams } from '../../core/music/trackParams';
import { TICKS_PER_QUARTER } from '../../core/music/time';
import { CLIP_ROWS } from '../../core/properties/sounds';
import { VIDEO_CLIP_ROWS } from '../../core/properties/videos';
@@ -100,7 +101,15 @@ export function getAsset(project: Project, assetId: string, readCel?: CelLookup)
plain.notes = asset.notes.map((note) => ({ ...note, name: noteName(note.note) }));
plain.ticks = { perQuarter: TICKS_PER_QUARTER, length: patternLength(asset) };
}
if (asset.type === 'song') plain.ticks = { perQuarter: TICKS_PER_QUARTER, length: songLength(asset) };
if (asset.type === 'song') {
plain.ticks = { perQuarter: TICKS_PER_QUARTER, length: songLength(asset) };
// What a lane of each track can move (#337): set_song_lane takes them as "param:<name>".
plain.tracks = asset.tracks.map((track, i) => {
const params = trackParams(project, asset, track);
const stored = (plain.tracks as object[])[i];
return params.length > 0 ? { ...stored, params } : stored;
});
}
const silent = silentParts(project, asset);
if (silent.length > 0) plain.silent = silent;
if (asset.type === 'sound' || asset.type === 'video') {
@@ -415,11 +424,15 @@ function nodeEntry({ id, def, params }: PaletteEntry): object {
};
}
/** A property's row without what only the inspector's drawing needs. */
/**
* A property's row without what only the inspector's drawing needs. `markers` says the field takes
* the id of a marker of a song (#338), as `get_asset` lists them, not the song's; `tracks` the id of
* a track of one.
*/
function propertyRow(def: PropertyDef): object {
const { key, label, kind } = def;
const row: Record<string, unknown> = { key, label, kind, default: def.default };
for (const extra of ['options', 'min', 'max', 'assetTypes'] as const) {
for (const extra of ['options', 'min', 'max', 'assetTypes', 'markers', 'tracks'] as const) {
if (extra in def) row[extra] = (def as Record<string, unknown>)[extra];
}
return row;
+5
View File
@@ -8,6 +8,8 @@ import type { Entity } from '../../runtime/Entity';
import type { World } from '../../runtime/World';
import { cameraOfItems, cameraOfWorld } from '../../viewport/activeCamera';
import { collectHolderItems } from '../../viewport/items';
import type { HeardSound } from '../../viewport/SoundLibrary';
import { audioView } from './audioView';
import { exact } from './playInput';
/** The running game as `get_runtime` reads it (#429). */
@@ -19,6 +21,8 @@ export interface RuntimeFacts {
heldKeys: ReadonlySet<string>;
/** Paused from the toolbar; the game's own pause is the world's. */
paused: boolean;
/** What is heard (#436); left out where the run plays to nobody. */
audio?: HeardSound;
}
/** How many console lines an answer carries unless the agent asks for more. */
@@ -66,6 +70,7 @@ export function runtimeView(
mouse: { ...world.pointer, x: round(world.pointer.x), y: round(world.pointer.y) },
objects: picked.map((entity) => entityRow(entity, project, named.length > 0)),
variables: sharedRows(facts.shared, project),
...(facts.audio && { audio: audioView(facts.audio, project, world) }),
console: lines > 0 ? world.log.slice(-lines) : [],
consoleLines: world.log.length,
};
+2 -2
View File
@@ -59,7 +59,7 @@
{
"name": "get_asset",
"title": "Read an asset",
"description": "One asset as stored: its property values, the rows of its properties (key, kind, default), the objects of a scene or container with their names and positions (the content of a placed container too, marked fromContainer), what the rows of an animation may aim at (targets), the cells of a small tilemap, the pixels of small pixel art as rows of palette indices (colors as \"#rrggbb\"), the notes of a pattern with their names and how many ticks it is long, the rows a clip of a sound or video has (clipProperties), how many seconds a loaded sound lasts, and under silent what of a pattern, song or composition is not heard: all of it without an instrument, or the notes its instrument does not sound. Under usedBy: every asset naming it or one of its frames or clips, with the objects, nodes, items or keys in there that do (none for a property of the asset itself), and the project's start scene, master FX or Master routing; empty when nothing uses it. A graph is only counted here; read it with get_graph.",
"description": "One asset as stored: its property values, the rows of its properties (key, kind, default), the objects of a scene or container with their names and positions (the content of a placed container too, marked fromContainer), what the rows of an animation may aim at (targets), the cells of a small tilemap, the pixels of small pixel art as rows of palette indices (colors as \"#rrggbb\"), the notes of a pattern with their names and how many ticks it is long, on each track of a song its automation lanes and the parameters a lane can move (params), the markers of a song with their ids, names and ticks, the rows a clip of a sound or video has (clipProperties), how many seconds a loaded sound lasts, and under silent what of a pattern, song or composition is not heard: all of it without an instrument, or the notes its instrument does not sound. Under usedBy: every asset naming it or one of its frames or clips, with the objects, nodes, items or keys in there that do (none for a property of the asset itself), and the project's start scene, master FX or Master routing; empty when nothing uses it. A graph is only counted here; read it with get_graph.",
"inputSchema": {
"type": "object",
"properties": { "asset": { "type": "string", "description": "Id of the asset." } },
@@ -250,7 +250,7 @@
{
"name": "get_runtime",
"title": "Read the running game",
"description": "The game as it runs now: game time, whether it is paused, the camera, the keys held and the mouse, every object with its id, type and position in world units (behaviors with their variables), the values of the Variables assets, and the last console lines. Name objects (ids or names) to get all their properties.",
"description": "The game as it runs now: game time, whether it is paused, the camera, the keys held and the mouse, every object with its id, type and position in world units (behaviors with their variables), the values of the Variables assets, the sound (audio), and the last console lines. Under audio: the audio clock (with a hint when it stands still while the game runs: then nothing is heard); the level of the master, of each bus of the mix, of each pattern, song or composition playing and of the tracks of a song, as peak and RMS in dB over the last window seconds (silent below -90 dB; held while the game is paused); the voices sounding now (a sound, or an instrument with its note), each with its bus and the object it belongs to (by); planned, the notes already set to start; the patterns, songs and compositions playing; waiting, the voices whose bytes have not arrived; and failed, the files that would not load. Name objects (ids or names) to get all their properties.",
"inputSchema": {
"type": "object",
"properties": {
+17 -2
View File
@@ -1,3 +1,4 @@
import type { AutomationTarget } from '../../core/music/automation';
import { findAssetOf } from '../../core/model/Project';
import type { HeadPart } from '../song/songView';
import { soundLibrary } from '../../viewport/SoundLibrary';
@@ -12,12 +13,17 @@ export function songHooks({ state, editor }: HookContext) {
current: () => songEditor.currentAssetId,
info: () => songEditor.shownInfo,
rect: () => songEditor.canvasRect(),
/** The tracks, items and loop as the model holds them. */
/** The tracks, items, loop and markers as the model holds them. */
model: (assetId: string) => {
const asset = findAssetOf(state.project, assetId, 'song');
return asset
? (JSON.parse(
JSON.stringify({ tracks: asset.tracks, items: asset.items, loop: asset.loop ?? null }),
JSON.stringify({
tracks: asset.tracks,
items: asset.items,
loop: asset.loop ?? null,
markers: asset.markers ?? [],
}),
) as unknown)
: null;
},
@@ -29,6 +35,15 @@ export function songHooks({ state, editor }: HookContext) {
headAt: (track: number, part: HeadPart, share?: number) => timeline.headPoint(track, part, share),
rulerAt: (tick: number) => timeline.rulerPoint(tick),
loopAt: (tick: number) => timeline.loopPoint(tick),
/** The markers (#338): where a tick of their strip is on screen, and each as drawn. */
markerAt: (tick: number) => timeline.markerPoint(tick),
markers: () => timeline.markersShown(),
/** The automation lanes (#337): where a value of one is drawn, its head, and the rows as shown. */
automationAt: (track: number, target: AutomationTarget, tick: number, value: number) =>
timeline.automationPoint(track, target, tick, value),
automationHeadAt: (track: number, target: AutomationTarget) =>
timeline.automationHeadPoint(track, target),
automation: () => timeline.automationShown(),
view: () => timeline.viewInfo(),
items: () => timeline.itemsDrawn(),
playAt: () => ({ ...centreOf(songEditor.playButton), text: songEditor.playButton.text }),
+4
View File
@@ -15,9 +15,11 @@ import {
assetRefChoices,
assetRefLabel,
MASTER_LABEL,
markerChoices,
NO_ASSET_LABEL,
soundChoices,
soundEntries,
trackChoices,
} from '../../core/properties/assetRef';
import { NO_FX } from '../../core/properties/assetFx';
import {
@@ -415,6 +417,8 @@ const REFERENCES: Record<ReferenceKind, Reference> = {
label: assetRefLabel,
choices: (project, def) => {
const extras = def.kind === 'asset' ? def : undefined;
if (extras?.markers) return markerChoices(project);
if (extras?.tracks) return trackChoices(project);
return [
...(extras?.fromAsset ? [{ id: FROM_ASSET_BUS, label: FROM_ASSET_LABEL }] : []),
...(extras?.toComposition ? [{ id: TO_COMPOSITION_BUS, label: TO_COMPOSITION_LABEL }] : []),
+23 -6
View File
@@ -16,6 +16,7 @@ import {
TRACK_MIN_DB,
} from '../../core/music/song';
import type { MenuEntry } from '../../ui';
import { laneLabel } from '../song/laneView';
import { ownFxEntries } from '../song/ownFx';
import type { EditorState } from '../state';
import { createFieldRow } from './fieldFactory';
@@ -33,6 +34,9 @@ const FX_ROW = {
/** What an item's pick says while it is empty (#352): what its asset names. */
const FROM_ASSET = 'From the asset';
/** What a row of a track sets: one of its own values, never its lanes (#337). */
type TrackField = Exclude<keyof SongTrack, 'automation'>;
const TRACK_ROWS: readonly PropertyDef[] = [
{
key: 'volume',
@@ -104,7 +108,13 @@ export function songSignature(
const asset = findAssetOf(project, assetId, 'song');
const item = itemOf(asset, itemIds);
if (item) return `item:${item.id}:${item.kind}`;
return asset?.tracks.some((t) => t.id === trackId) ? `track:${trackId}` : 'missing';
const track = asset?.tracks.find((t) => t.id === trackId);
return track ? `track:${trackId}:${ruledLanes(track).join(',')}` : 'missing';
}
/** The lanes with a point on a track (#337): what its fields no longer set while it plays. */
function ruledLanes(track: SongTrack): string[] {
return (track.automation ?? []).flatMap((lane) => (lane.points.length > 0 ? [laneLabel(lane.target)] : []));
}
/** The item picked last (#334), else the track: its rows. */
@@ -137,7 +147,7 @@ export function refreshSongSection(
const track = asset?.tracks.find((t) => t.id === trackId);
if (!track) return false;
host.nameField?.setValue(track.name);
for (const def of TRACK_ROWS) host.fields.get(def.key)?.setValue(track[def.key as keyof SongTrack]);
for (const def of TRACK_ROWS) host.fields.get(def.key)?.setValue(track[def.key as TrackField]);
return true;
}
@@ -160,22 +170,29 @@ function buildTrack(host: InspectorHost, asset: SongAsset, track: SongTrack): vo
const count = asset.items.filter((i) => i.track === track.id).length;
addNote(host, `Track of the song "${asset.name}", ${count} ${count === 1 ? 'item' : 'items'}.`);
addNote(host, 'Its Sound FX takes everything on it together, before its volume and pan.', 'small');
const ruled = ruledLanes(track);
if (ruled.length > 0)
addNote(
host,
`Automation lanes rule ${ruled.join(', ')} while it plays; a field here is only heard once its lane is cleared.`,
'small',
);
for (const def of TRACK_ROWS) {
const commit = (value: SongTrack[keyof SongTrack]) =>
const commit = (value: SongTrack[TrackField]) =>
new SetSongCommand(
asset.id,
{ tracks: withTrack(state.project, asset.id, track.id, { [def.key]: value }) },
`Set track ${def.key}`,
);
const handlers = previewingHandlers<SongTrack, SongTrack[keyof SongTrack]>(host, {
const handlers = previewingHandlers<SongTrack, SongTrack[TrackField]>(host, {
target: () => findAssetOf(state.project, asset.id, 'song')?.tracks.find((t) => t.id === track.id),
read: (target) => target[def.key as keyof SongTrack],
read: (target) => target[def.key as TrackField],
write: (target, value) => Object.assign(target, { [def.key]: value }),
commit,
});
if (def.key === FX_ROW.key)
handlers.moreChoices = (current) => ownFxEntries(state, asset.id, current, commit);
const row = createFieldRow(host, def, track[def.key as keyof SongTrack], handlers);
const row = createFieldRow(host, def, track[def.key as TrackField], handlers);
host.fields.set(def.key, row);
host.body.add(row.row);
}
+120
View File
@@ -0,0 +1,120 @@
import { heldRange } from '../../core/music/automation';
import { defaultParamRange, type TrackParam } from '../../core/music/trackParams';
import { type Button, Dropdown, Label, Menu, NumberField } from '../../ui';
import { dialogSlot, FormDialog } from '../FormDialog';
const PICKER_WIDTH = 200;
const FIELD_WIDTH = 90;
const slot = dialogSlot<ParamLaneDialog>();
/** The open dialog, for the check scripts. */
export const currentParamLaneDialog = slot.current;
/** Opens the dialog centred, closing whichever one was open before it. */
export const openParamLaneDialog = slot.open;
/**
* A lane for a parameter of a track (#337): which parameter, and the range its lane spans – a
* parameter says nothing of its own. A new one starts at a range round its default; `editing` sets
* the range of a lane there is, its parameter fixed.
*/
export class ParamLaneDialog extends FormDialog {
private param: string;
private min: number;
private max: number;
private readonly picker: Dropdown;
private readonly minField: NumberField;
private readonly maxField: NumberField;
private readonly summary: Label;
/** Public for the check scripts. */
readonly confirm: Button;
constructor(
private readonly offered: readonly TrackParam[],
editing: { name: string; min: number; max: number } | null,
onConfirm: (name: string, range: { min: number; max: number }) => void,
) {
super(340, editing ? `Range of the ${editing.name} lane` : 'Add a parameter lane');
this.param = editing?.name ?? offered[0]?.name ?? '';
this.picker = new Dropdown('', { width: PICKER_WIDTH, onClick: () => this.openPicker() });
this.picker.setEnabled(!editing && offered.length > 1);
this.content.add(this.formRow(this.fieldLabel('Parameter'), this.picker));
({ min: this.min, max: this.max } = editing ?? this.rangeOf(this.param));
const field = (value: number, set: (value: number) => void) => {
const made = new NumberField(value, {
onInput: (now) => (set(now), this.refresh()),
onChange: (now) => (set(now), this.refresh()),
});
made.minWidth = FIELD_WIDTH;
return made;
};
this.minField = field(this.min, (value) => (this.min = value));
this.maxField = field(this.max, (value) => (this.max = value));
this.content.add(this.formRow(this.fieldLabel('From'), this.minField));
this.content.add(this.formRow(this.fieldLabel('To'), this.maxField));
this.summary = new Label('', { style: 'muted' });
this.content.add(this.summary);
this.confirm = this.addButtons(editing ? 'Set range' : 'Add lane', 'primary', () => {
const range = this.range();
if (range && this.param) onConfirm(this.param, range);
});
this.refresh();
}
/** What the dialog holds, for the check scripts. */
debugInfo(): { name: string; offered: string[]; min: number; max: number; ok: boolean } {
return {
name: this.param,
offered: this.offered.map((p) => p.name),
min: this.min,
max: this.max,
ok: this.confirm.enabled,
};
}
/** Picks a parameter and a range by value rather than by hand, for the check scripts. */
pick(name: string, range?: { min: number; max: number }): void {
this.param = name;
({ min: this.min, max: this.max } = range ?? this.rangeOf(name));
this.minField.setNumber(this.min);
this.maxField.setNumber(this.max);
this.refresh();
}
private rangeOf(name: string): { min: number; max: number } {
return defaultParamRange(this.offered.find((p) => p.name === name)?.default ?? 0);
}
private range(): { min: number; max: number } | null {
return heldRange(this.min, this.max);
}
private openPicker(): void {
new Menu(
this.offered.map((param) => ({
label: param.name,
current: param.name === this.param,
onSelect: () => this.pick(param.name),
})),
{ searchable: this.offered.length > 9 },
).openBelow(this.picker);
}
private refresh(): void {
this.picker.setText(this.param || '—');
const range = this.range();
const param = this.offered.find((p) => p.name === this.param);
this.summary.setText(
!this.param
? 'Nothing on this track has a parameter to move.'
: !range
? '"To" has to be above "From".'
: param
? `Its default is ${param.default}.`
: 'Nothing on the track has this parameter now.',
);
this.confirm.setEnabled(!!this.param && range !== null);
this.refreshLayout();
}
}
File diff suppressed because it is too large. Load diff
+43
View File
@@ -0,0 +1,43 @@
import { listedAssets } from '../../core/music/embedded';
import type { KeyInterpolation } from '../../core/model/Asset';
import { EASING_DIRECTIONS, EASING_SHAPES, easingOf, easingValue } from '../../core/model/animation';
import type { Project } from '../../core/model/Project';
import type { MenuEntry } from '../../ui';
/**
* The easings as entries of a menu (#337), the same choice the inspector gives an animation key: Linear
* and Step straight away, every curve with its directions in a submenu, and the Curve assets of the
* project. The one in use is marked.
*/
export function easingEntries(
project: Project,
current: KeyInterpolation,
set: (ease: KeyInterpolation) => void,
): MenuEntry[] {
const now = easingOf(current);
const curves = listedAssets(project)
.filter((a) => a.type === 'curve')
.sort((a, b) => a.name.localeCompare(b.name));
return EASING_SHAPES.map(({ value, label }): MenuEntry => {
if (value === 'linear' || value === 'step')
return { label, current: now.shape === value, onSelect: () => set(value) };
if (value === 'curve')
return {
label,
disabled: curves.length === 0,
submenu: curves.map((curve) => ({
label: curve.name,
current: now.shape === 'curve' && now.curve === curve.id,
onSelect: () => set(easingValue('curve', undefined, curve.id)),
})),
};
return {
label,
submenu: EASING_DIRECTIONS.map((direction) => ({
label: direction.label,
current: now.shape === value && now.direction === direction.value,
onSelect: () => set(easingValue(value, direction.value)),
})),
};
});
}
+132
View File
@@ -0,0 +1,132 @@
import {
AUTOMATION_TARGETS,
type AutomationLane,
type AutomationPoint,
type AutomationTarget,
type LaneSpec,
laneRange,
paramOf,
paramTarget,
} from '../../core/music/automation';
import type { SongTrack } from '../../core/music/song';
import type { KeyInterpolation } from '../../core/model/Asset';
/**
* Editing the points of a lane in the Song tab (#337), apart from Pixi so a test can drive it. Points
* stay in the order of their ticks: one that is dragged stops at its neighbours, so two on one tick –
* a jump – keep which comes first.
*/
/** The points of a track's lane for that target; none while it has no lane. */
export function lanePoints(track: SongTrack, target: AutomationTarget): AutomationPoint[] {
return track.automation?.find((lane) => lane.target === target)?.points ?? [];
}
/**
* The track with these points on that lane. With none the lane of the volume or the pan goes; a
* parameter's stays for its range. With the last lane `automation` goes.
*/
export function withLanePoints(
track: SongTrack,
target: AutomationTarget,
points: readonly AutomationPoint[],
): SongTrack {
const old = track.automation?.find((lane) => lane.target === target);
const range = paramOf(target) !== null ? laneRange(old ?? { target }) : null;
const lane: AutomationLane | null =
points.length > 0 || range ? { target, points: [...points], ...range } : null;
return withLane(track, target, lane);
}
/** The track with a lane for that parameter in this range, its points held to it; a new one is empty. */
export function withParamLane(
track: SongTrack,
name: string,
range: { min: number; max: number },
): SongTrack {
const target = paramTarget(name);
const points = lanePoints(track, target).map((point) => ({
...point,
value: Math.max(range.min, Math.min(range.max, point.value)),
}));
return withLane(track, target, { target, points, min: range.min, max: range.max });
}
/** The track without that lane at all, a parameter's range with it. */
export function withoutLane(track: SongTrack, target: AutomationTarget): SongTrack {
return withLane(track, target, null);
}
/** In a fixed order: the volume, the pan, then the parameters as they came; a new one last. */
function withLane(track: SongTrack, target: AutomationTarget, lane: AutomationLane | null): SongTrack {
const others = (track.automation ?? []).filter((l) => l.target !== target);
const fixed = AUTOMATION_TARGETS.flatMap((t) =>
t === target && lane ? [lane] : others.filter((l) => l.target === t),
);
const params = (track.automation ?? []).flatMap((l) =>
paramOf(l.target) === null ? [] : l.target === target ? (lane ? [lane] : []) : [l],
);
const known = (track.automation ?? []).some((l) => l.target === target);
if (lane && paramOf(target) !== null && !known) params.push(lane);
const lanes = [...fixed, ...params];
const { automation: _, ...rest } = track;
return lanes.length > 0 ? { ...rest, automation: lanes } : rest;
}
function held(kind: LaneSpec, value: number): number {
const { min, max } = laneRange(kind);
return Math.max(min, Math.min(max, value));
}
/** A new point, after those already on its tick, and where it went. */
export function addedPoint(
points: readonly AutomationPoint[],
kind: LaneSpec,
tick: number,
value: number,
): { points: AutomationPoint[]; index: number } {
const at = Math.max(0, Math.round(tick));
let index = 0;
while (index < points.length && points[index]!.tick <= at) index++;
const out = [...points];
out.splice(index, 0, { tick: at, value: held(kind, value) });
return { points: out, index };
}
/** The point at `index` dragged to a tick and a value, the tick held between its neighbours'. */
export function draggedPoint(
points: readonly AutomationPoint[],
kind: LaneSpec,
index: number,
tick: number,
value: number,
): AutomationPoint[] {
const point = points[index];
if (!point) return [...points];
const low = points[index - 1]?.tick ?? 0;
const high = points[index + 1]?.tick ?? Infinity;
const out = [...points];
out[index] = {
...point,
tick: Math.min(high, Math.max(low, Math.round(tick))),
value: held(kind, value),
};
return out;
}
export function withoutPoint(points: readonly AutomationPoint[], index: number): AutomationPoint[] {
return points.filter((_, i) => i !== index);
}
/** The point at `index` easing to the next one along `ease`; linear is stored as no easing. */
export function easedPoint(
points: readonly AutomationPoint[],
index: number,
ease: KeyInterpolation,
): AutomationPoint[] {
return points.map((point, i) => {
if (i !== index) return point;
const { ease: _, ...rest } = point;
return ease === 'linear' ? rest : { ...rest, ease };
});
}
+186
View File
@@ -0,0 +1,186 @@
import {
AUTOMATION_TARGETS,
type AutomationPoint,
type AutomationTarget,
type LaneKind,
laneKind,
laneOf,
laneParams,
laneRange,
laneValueAt,
laneValueBefore,
paramOf,
paramTarget,
type LaneSpec,
type SequenceLanePoint,
} from '../../core/music/automation';
import type { SongTrack } from '../../core/music/song';
import { clamp } from '../../ui';
import { tickToX, xToTick } from '../music/timeView';
import type { SongView } from './songView';
/**
* The automation lanes of the Song tab (#337), apart from Pixi so a test can drive them: each lane a
* row of its own right under its track, as high as a track, so the rows stay evenly spaced and the
* geometry of `songView` holds for them as for the tracks.
*/
/** A row of the tracks: a track itself, or one of its lanes. `track` is the track's index. */
export interface SongRow {
track: number;
lane: AutomationTarget | null;
}
/** The lanes a track was opened to or closed on by hand, by its id. */
export type OpenLanes = ReadonlyMap<string, readonly AutomationTarget[]>;
/** What the head of a lane calls it: the volume, the pan, or the name of its parameter. */
export function laneLabel(target: AutomationTarget): string {
return target === 'volume' ? 'Volume' : target === 'pan' ? 'Pan' : (paramOf(target) ?? target);
}
/** Every lane a track can show: the volume, the pan, and each parameter it has a lane of (#337). */
export function trackLanes(track: SongTrack): AutomationTarget[] {
return [...AUTOMATION_TARGETS, ...laneParams(track).map(paramTarget)];
}
/**
* The lanes a track shows, in a fixed order: as opened by hand, else the ones with a point and every
* parameter's – one is only there because it was added.
*/
export function shownLanes(track: SongTrack, open: OpenLanes): AutomationTarget[] {
const named = open.get(track.id);
return trackLanes(track).filter((target) =>
named ? named.includes(target) : paramOf(target) !== null || !!laneOf(track, target),
);
}
/** A track's lane of that target as far as its kind goes: a parameter's range is on its lane. */
export function laneKindOf(track: SongTrack, target: AutomationTarget): LaneKind {
const lane = track.automation?.find((l) => l.target === target);
return lane ? { target, min: lane.min, max: lane.max } : { target };
}
export function songRows(tracks: readonly SongTrack[], open: OpenLanes): SongRow[] {
return tracks.flatMap((track, i) => [
{ track: i, lane: null },
...shownLanes(track, open).map((lane) => ({ track: i, lane })),
]);
}
/** The row of each track, by its id. */
export function trackRows(tracks: readonly SongTrack[], rows: readonly SongRow[]): Map<string, number> {
const out = new Map<string, number>();
rows.forEach((row, i) => {
if (row.lane === null) out.set(tracks[row.track]!.id, i);
});
return out;
}
/** What is left free above and below the way of a lane, so a point at either end can still be taken. */
export const LANE_INSET = 6;
/** How near a point a press takes it. */
export const POINT_REACH = 6;
/** Where a value is drawn in a lane `h` high, from its top; past the range it is held to the edge. */
export function laneY(kind: LaneSpec, value: number, h: number): number {
const { min, max } = laneRange(kind);
const share = clamp((value - min) / (max - min), 0, 1);
return LANE_INSET + (1 - share) * Math.max(1, h - 2 * LANE_INSET);
}
/**
* The step a lane's values are set in: a tenth of a dB, a hundredth of the pan, and for a parameter
* a power of ten that cuts its range into at least a hundred.
*/
export function laneStep(kind: LaneSpec): number {
const { target } = laneKind(kind);
if (target === 'volume') return 0.1;
if (target === 'pan') return 0.01;
const { min, max } = laneRange(kind);
return 10 ** Math.floor(Math.log10((max - min) / 100));
}
/** The value a height of a lane stands for, in its step. */
export function laneValue(kind: LaneSpec, y: number, h: number): number {
const { min, max } = laneRange(kind);
const share = clamp(1 - (y - LANE_INSET) / Math.max(1, h - 2 * LANE_INSET), 0, 1);
const step = laneStep(kind);
return clamp(roundTo(min + share * (max - min), step), min, max);
}
/** Rounded to a step, without the tail a float adds to a tenth. */
function roundTo(value: number, step: number): number {
const digits = Math.max(0, -Math.floor(Math.log10(step)));
return Number((Math.round(value / step) * step).toFixed(Math.min(digits, 12)));
}
/** The point of a lane under a point of its row, the last drawn first; null for none. */
export function pointAt(
view: SongView,
target: LaneSpec,
points: readonly AutomationPoint[],
x: number,
y: number,
): number | null {
for (let i = points.length - 1; i >= 0; i--) {
const point = points[i]!;
const dx = tickToX(view, point.tick) - x;
const dy = laneY(target, point.value, view.trackHeight) - y;
if (Math.abs(dx) <= POINT_REACH && Math.abs(dy) <= POINT_REACH) return i;
}
return null;
}
/**
* The way of a lane across `width` pixels, a corner every `every` pixels and at each point – a jump
* shows as an upright –, in the row's own pixels.
*/
export function laneWay(
view: SongView,
target: LaneSpec,
points: readonly SequenceLanePoint[],
width: number,
every = 2,
): { x: number; y: number }[] {
if (points.length === 0) return [];
const h = view.trackHeight;
const way: { x: number; y: number }[] = [];
const corners = new Set<number>();
for (let x = 0; x <= width; x += every) corners.add(x);
for (const point of points) {
const x = tickToX(view, point.tick);
if (x > 0 && x < width) corners.add(x);
}
for (const x of [...corners].sort((a, b) => a - b)) {
const at = points.findIndex((p) => tickToX(view, p.tick) === x);
if (at < 0) {
way.push({ x, y: laneY(target, laneValueAt(points, xToTick(view, x)), h) });
continue;
}
// On a point the way comes in with the value before it and leaves from the last point on its tick.
const tick = points[at]!.tick;
const before = laneValueBefore(points, tick);
if (before !== points[at]!.value) way.push({ x, y: laneY(target, before, h) });
for (let i = at; i < points.length && points[i]!.tick === tick; i++)
way.push({ x, y: laneY(target, points[i]!.value, h) });
}
return way;
}
/** A value as the head of a lane and the sign over a point say it. */
export function laneValueText(kind: LaneSpec, value: number): string {
const { target } = laneKind(kind);
if (target === 'volume') {
const { min } = laneRange('volume');
if (value <= min) return '-inf dB';
const shown = Math.round(value * 10) / 10;
return `${shown > 0 ? '+' : ''}${shown.toFixed(1)} dB`;
}
if (target === 'pan') {
const share = Math.round(Math.abs(value) * 100);
return share === 0 ? 'Centre' : `${share}% ${value < 0 ? 'left' : 'right'}`;
}
const digits = Math.max(0, -Math.floor(Math.log10(laneStep(kind))));
return roundTo(value, laneStep(kind)).toFixed(Math.min(digits, 6));
}
+51
View File
@@ -0,0 +1,51 @@
import type { SongMarker } from '../../core/music/markers';
import { RULER_HEIGHT, type TimeView, tickToX } from '../music/timeView';
/** The strip under the song's ruler that holds its markers (#338). */
export const MARKER_STRIP = 18;
/** Where the tracks of the Song tab start: below the ruler and its markers. */
export const SONG_TOP = RULER_HEIGHT + MARKER_STRIP;
/** How far either side of its line a press still takes a marker. */
export const MARKER_REACH = 5;
/** The flag at the head of a marker's line, and the room between it and the name. */
export const MARKER_FLAG = 8;
const NAME_GAP = 3;
/** Where a marker is drawn on the strip: its line at `x`, flag and name over `w` pixels right of it. */
export interface MarkerSpan {
id: string;
x: number;
w: number;
}
/**
* Each marker as drawn on the strip: its flag and as much of its name as there is room for before
* the next one. `measure` gives the width of a name in pixels.
*/
export function markerSpans(
view: TimeView,
markers: readonly SongMarker[],
measure: (name: string) => number,
): MarkerSpan[] {
return markers.map((marker, i) => {
const x = tickToX(view, marker.tick);
const next = markers[i + 1];
const room = next ? tickToX(view, next.tick) - x - 2 : Infinity;
const w = Math.max(MARKER_FLAG, Math.min(MARKER_FLAG + NAME_GAP + measure(marker.name), room));
return { id: marker.id, x, w };
});
}
/** Where a marker's name starts, right of its flag. */
export function markerNameX(span: MarkerSpan): number {
return span.x + MARKER_FLAG + NAME_GAP;
}
/** The marker under `x` on the strip, the last drawn first: near its line, or on its flag and name. */
export function markerAt(spans: readonly MarkerSpan[], x: number): string | null {
for (let i = spans.length - 1; i >= 0; i--) {
const span = spans[i]!;
if (x >= span.x - MARKER_REACH && x <= span.x + Math.max(span.w, MARKER_REACH)) return span.id;
}
return null;
}
+16 -11
View File
@@ -99,17 +99,19 @@ export function itemBox(view: SongView, index: number, item: Pick<SongItem, 'tic
export type ItemPart = 'start' | 'body' | 'end';
/** The item under a point, the last drawn first, and which part of it: an end within reach, else the body. */
/**
* The item under a point, the last drawn first, and which part of it: an end within reach, else the
* body. `rows` is the row of each track by its id (`indexOf` while no lane is open, #337).
*/
export function itemAt(
view: SongView,
items: readonly SongItem[],
tracks: readonly SongTrack[],
rows: ReadonlyMap<string, number>,
p: Vec2,
): { id: string; part: ItemPart } | null {
const index = indexOf(tracks);
for (let i = items.length - 1; i >= 0; i--) {
const item = items[i]!;
const row = index.get(item.track);
const row = rows.get(item.track);
if (row === undefined) continue;
const box = itemBox(view, row, item);
if (p.y < box.y || p.y > box.y + box.h || p.x < box.x || p.x > box.x + box.w) continue;
@@ -125,18 +127,17 @@ export function itemAt(
export function itemsWithin(
view: SongView,
items: readonly SongItem[],
tracks: readonly SongTrack[],
rows: ReadonlyMap<string, number>,
a: Vec2,
b: Vec2,
): string[] {
const index = indexOf(tracks);
const x0 = Math.min(a.x, b.x);
const x1 = Math.max(a.x, b.x);
const y0 = Math.min(a.y, b.y);
const y1 = Math.max(a.y, b.y);
return items
.filter((item) => {
const row = index.get(item.track);
const row = rows.get(item.track);
if (row === undefined) return false;
const box = itemBox(view, row, item);
return box.x <= x1 && box.x + box.w >= x0 && box.y <= y1 && box.y + box.h >= y0;
@@ -180,11 +181,15 @@ export function indexOf(tracks: readonly SongTrack[]): Map<string, number> {
return new Map(tracks.map((t, i) => [t.id, i]));
}
/** The parts of a track's head, in its own pixels: name, sound FX (#346), mute, solo, volume and pan. */
export type HeadPart = 'name' | 'fx' | 'mute' | 'solo' | 'volume' | 'pan';
/**
* The parts of a track's head, in its own pixels: the switch that opens its lanes (#337), name, sound
* FX (#346), mute, solo, volume and pan.
*/
export type HeadPart = 'lanes' | 'name' | 'fx' | 'mute' | 'solo' | 'volume' | 'pan';
export const HEAD_BOXES: Readonly<Record<HeadPart, Box>> = {
name: { x: 8, y: 4, w: 88, h: 18 },
lanes: { x: 4, y: 5, w: 14, h: 16 },
name: { x: 22, y: 4, w: 76, h: 18 },
fx: { x: 102, y: 5, w: 22, h: 16 },
mute: { x: 128, y: 5, w: 18, h: 16 },
solo: { x: 150, y: 5, w: 18, h: 16 },
@@ -202,7 +207,7 @@ export function trackMeterBox(trackHeight: number): Box {
export function headPartAt(x: number, yInTrack: number, trackHeight = TRACK_HEIGHT): HeadPart {
const sliders = trackHeight >= SLIDERS_MIN_HEIGHT;
for (const part of ['fx', 'mute', 'solo', 'volume', 'pan'] as const) {
for (const part of ['lanes', 'fx', 'mute', 'solo', 'volume', 'pan'] as const) {
if (!sliders && (part === 'volume' || part === 'pan')) continue;
const box = HEAD_BOXES[part];
if (x >= box.x - 2 && x <= box.x + box.w + 2 && yInTrack >= box.y - 3 && yInTrack <= box.y + box.h + 3)
+20 -1
View File
@@ -1,6 +1,6 @@
import type { EaseShare } from '../core/model/animation';
import type { MixBus } from '../core/music/mixer';
import type { SequencePlay } from '../core/music/playbacks';
import type { HeardTime, LandingAt, SequencePlay } from '../core/music/playbacks';
import type { FxParams, FxPlan, FxVoice } from '../core/soundfx/nodes';
import type { AudioSink, ClipTiming, NoteTiming } from './systems/Audio';
@@ -133,4 +133,23 @@ export class AudioSwitch implements AudioSink {
clock(): number {
return this.now.clock?.() ?? 0;
}
/** Both sinks' playings: each counts its own on its own clock. */
playingTimes(): HeardTime[] {
return this.both.flatMap((sink) => sink.playingTimes?.() ?? []);
}
switchSequence(id: number, name: string, play: SequencePlay, at: LandingAt, fade: number): boolean {
return this.of(id)?.switchSequence?.(id, name, play, at, fade) ?? false;
}
setSongTrack(
id: number,
track: string,
gain: number,
fade: number,
at: LandingAt,
): 'set' | 'baked' | 'none' {
return this.of(id)?.setSongTrack?.(id, track, gain, fade, at) ?? 'none';
}
}
+26 -1
View File
@@ -1,5 +1,5 @@
import type { EaseShare } from '../../core/model/animation';
import type { SequencePlay } from '../../core/music/playbacks';
import type { HeardTime, LandingAt, SequencePlay } from '../../core/music/playbacks';
import type { MixBus } from '../../core/music/mixer';
import type { FxParams, FxPlan, FxVoice } from '../../core/soundfx/nodes';
@@ -108,6 +108,29 @@ export interface AudioSink {
* picture by. A sink without it keeps no time, and nothing is held to it.
*/
clock?(): number;
/**
* The patterns and songs playing, each with how far into its playing time it is heard now (#338):
* what beats, bars and `Song position` go by. A sink without it plays none to be followed.
*/
playingTimes?(): readonly HeardTime[];
/**
* Plays `play` in place of the playing numbered `id` (#338), heard from the second `at` names and
* taking its number; the old one falls silent there, over `fade` seconds or in a cut. Says whether
* there was such a playing. A sink without it switches nothing.
*/
switchSequence?(id: number, name: string, play: SequencePlay, at: LandingAt, fade: number): boolean;
/**
* Turns the track bus `track` of the playing numbered `id` to `gain` (#338), on top of its own
* level, over `fade` seconds from the second `at` names: `baked` where it plays its bake, `none`
* where there is no such playing or track.
*/
setSongTrack?(
id: number,
track: string,
gain: number,
fade: number,
at: LandingAt,
): 'set' | 'baked' | 'none';
}
/**
@@ -119,6 +142,8 @@ export interface NoteTiming {
tempo?: number;
/** The bus it is heard on: a track of a song (#334) or one of the mix (#325); the master where left out. */
bus?: string;
/** The note it is, for whoever asks what is heard (#436). */
note?: number;
}
/** When and how a clip of a song plays (#334); see `playClip`. */
+17
View File
@@ -24,6 +24,7 @@ import {
type Value,
} from '../../core/graph';
import { GraphCache } from '../../core/graph/graphIndex';
import { PLAYBACK_INPUT } from '../../core/graph/nodes/events/musicEvents';
import { PER_CELL_INPUT } from '../../core/graph/nodes/shared';
import { keyName } from '../../core/keys';
import { graphLookup } from '../../core/model/graphLookup';
@@ -53,6 +54,14 @@ import {
timerPulses,
} from './behaviorPulses';
import { CollisionSystem } from './CollisionSystem';
import {
barPulses,
beatPulses,
heardPlayings,
markerPulses,
MusicFollower,
type PlaybackOf,
} from './musicPulses';
import type { CollisionSource } from './solids';
import { AnimationSystem } from './AnimationSystem';
import { CompositionSystem, type VideoPictures } from './CompositionSystem';
@@ -223,6 +232,8 @@ export class BehaviorSystem implements System {
private readonly breakpoint: Breakpoint | null;
/** What was last written to `saves`, so a frame that changed nothing writes nothing. */
private lastSaved = '';
/** Where each playing of a song or pattern was heard last frame (#338). */
private readonly music = new MusicFollower();
constructor(
private readonly project: Project,
@@ -636,6 +647,9 @@ export class BehaviorSystem implements System {
const { input, collisions, animations, compositions } = this;
const perCell: PerCell = (i, node) =>
asBoolean(i.interpreter.eventInput(node, PER_CELL_INPUT, contextOf(i)));
// Every frame, paused or not: the music goes on, and what a paused one passed is not told after.
const passed = this.music.follow(heardPlayings(this.project, this.audio));
const playbackOf: PlaybackOf = (i, node) => i.interpreter.eventInput(node, PLAYBACK_INPUT, contextOf(i));
const kinds: (() => Pulse[])[] = [
() =>
ticking.flatMap((i) =>
@@ -650,6 +664,9 @@ export class BehaviorSystem implements System {
() => (frozen ? [] : ticking.flatMap((i) => collisionExitPulses(i, collisions, perCell))),
() => (frozen ? [] : ticking.flatMap((i) => animationFinishedPulses(i, animations))),
() => (frozen ? [] : ticking.flatMap((i) => compositionFinishedPulses(i, compositions))),
() => ticking.flatMap((i) => beatPulses(i, passed, playbackOf)),
() => ticking.flatMap((i) => barPulses(i, passed, playbackOf)),
() => ticking.flatMap((i) => markerPulses(i, passed, playbackOf)),
() =>
pointer.pressed ? pulsesOf(ticking.filter(pointed), 'OnClick', { x: pointer.x, y: pointer.y }) : [],
() =>
+186 -39
View File
@@ -24,6 +24,8 @@ import { bakePlayback, validBake } from '../../core/music/bake';
import { clipFx, fxVoiceOf, sequenceNeeds } from '../../core/music/needs';
import { patternInstrument, patternLength, patternSequence, patternTempo } from '../../core/music/pattern';
import type { ScheduledClip, ScheduledNote } from '../../core/music/Scheduler';
import { nextSpot, type Quantize, quantizeSpots } from '../../core/music/beats';
import { heldSongMarkers } from '../../core/music/markers';
import {
dbToGain,
songLength,
@@ -31,8 +33,12 @@ import {
songSequence,
songTempo,
SOUND_ITEM_MAX_DB,
TRACK_MAX_DB,
TRACK_MIN_DB,
trackBus,
} from '../../core/music/song';
import type { SequencePlay } from '../../core/music/playbacks';
import type { LandingAt, SequencePlay } from '../../core/music/playbacks';
import { findSongMarker } from '../../core/properties/assetRef';
import { songBuses } from '../../core/music/songBuses';
import { type MixRoute, playedBus, playedRoute, reachedBus } from '../../core/music/mixer';
import { DRY_VOICE, type FxParams, type FxVoice } from '../../core/soundfx/nodes';
@@ -42,6 +48,7 @@ import type { World } from '../World';
import type { AudioSink, NoteTiming } from './Audio';
import type { Instance } from './behaviorPulses';
import type { Input } from './Input';
import { type HeardPlaying, heardPlayings, positionOf } from './musicPulses';
import type { RecordingSink } from '../../core/recording/recording';
import { bodyOf, type CollisionSource, layerHolderOf, NOT_BLOCKED } from './solids';
@@ -206,8 +213,21 @@ export function behaviorContext(
playPattern: (patternId, volume, loop, detached, fx, fxParams, bus) =>
playPattern(run, patternId, volume, loop, detached ? '' : entity.id, fx, fxParams, bus),
stopPattern: (patternId, letRing) => run.audio.stopSequence(patternVoiceName(patternId), letRing),
playSong: (songId, volume, fadeIn, loop, detached, fx, fxParams, bus) =>
playSong(run, songId, volume, fadeIn, loop, detached ? '' : entity.id, fx, fxParams, bus),
playSong: (songId, volume, fadeIn, loop, detached, fx, fxParams, bus, live) =>
playSong(run, songId, {
volume,
fadeIn,
loop,
owner: detached ? '' : entity.id,
fx,
fxParams,
bus,
live,
}),
setSongTrack: (playback, track, volumeDb, fade, quantize) =>
setSongTrack(run, world, playback, track, volumeDb, heldFade(fade), quantize),
switchSong: (playback, songId, marker, quantize, fade) =>
switchSong(run, world, playback, songId, marker, quantize, heldFade(fade)),
stopSong: (songId, fadeOut) => run.audio.stopSequence(songVoiceName(songId), false, heldFade(fadeOut)),
stopPlayback: (playback, fadeOut) => run.audio.stopPlayback(playback, heldFade(fadeOut)),
soundLength: (assetId) => {
@@ -235,6 +255,8 @@ export function behaviorContext(
const reached = reachedBus(project, bus);
return reached ? run.audio.mixVolume(reached) : 0;
},
songPosition: (playback, song, pattern) =>
positionOf(heardPlayings(project, run.audio), playback, song, pattern),
spawn: (assetId, position, parent) => world.spawn(assetId, position, resolve(parent) ?? null),
addJoint: (assetId, owner, connected, at) => {
const on = resolve(owner);
@@ -349,14 +371,18 @@ export function playNote(
): number {
const asset = findAssetOf(project, instrumentId, 'instrument');
if (!asset) return 0;
const built = instrumentBuildFor(asset, heldNote(struck.note), instrumentSourcesOf(project)).layers;
const note = heldNote(struck.note);
const built = instrumentBuildFor(asset, note, instrumentSourcesOf(project)).layers;
if (built.length === 0) return 0;
const layers = fx ? built.map((layer) => throughScopedFx(fx.plan, layer, FX_SCOPE)) : built;
const velocity = Number.isFinite(struck.velocity) ? Math.max(0, Math.min(1, struck.velocity)) : 1;
const level = Math.max(0, Math.min(1, struck.volume * velocity));
const duration = Number.isFinite(struck.duration) ? Math.max(0, struck.duration) : 0;
const values = { ...params, [VELOCITY_PARAM]: velocity, ...scoped(fx?.params ?? {}) };
return audio.playNote(instrumentVoiceName(instrumentId), layers, values, level, owner, duration, timing);
return audio.playNote(instrumentVoiceName(instrumentId), layers, values, level, owner, duration, {
...timing,
note,
});
}
/** Where the parameters of a note's own sound FX go by (#346): no name of a parameter has a colon. */
@@ -427,48 +453,162 @@ function playPattern(
);
}
/** How a Play song asked for a song (#338): what a Switch song plays the next one by. */
interface SongAsk {
volume: number;
fadeIn: number;
loop: boolean;
owner: string;
fx: string;
fxParams: FxParams;
bus: string;
/** Its notes, even where it has a bake: its tracks can be turned then. */
live: boolean;
}
/** What each playing of a song was asked for, by its number, in the sink it plays in. */
const songAsks = new WeakMap<AudioSink, Map<number, SongAsk>>();
/** So many asks are kept before those of playings gone are let go. */
const ASKS_KEPT = 64;
function keepAsk(audio: AudioSink, id: number, ask: SongAsk): void {
let asks = songAsks.get(audio);
if (!asks) songAsks.set(audio, (asks = new Map<number, SongAsk>()));
if (asks.size >= ASKS_KEPT) {
const playing = new Set((audio.playingTimes?.() ?? []).map((time) => time.id));
for (const known of asks.keys()) if (!playing.has(known)) asks.delete(known);
}
asks.set(id, ask);
}
/**
* A song played by a game (#336): its tracks at their level, pan, mute and solo, at its tempo. With
* `loop` it plays from its start and then runs round the loop of its ruler, else once to its end.
* `volume` and `fadeIn` are the level of the whole playing. Its bake (#340) is played instead while
* it holds. Returns the number of the playing, 0 where there is no song or nothing in it to play.
* it holds, unless `live` (#338). Returns the number of the playing, 0 where there is no song or
* nothing in it to play.
*/
function playSong(
function playSong(sources: Pick<ContextSources, 'project' | 'audio'>, songId: string, ask: SongAsk): number {
const play = songPlay(sources, songId, ask, 0);
const id = play ? sources.audio.playSequence(songVoiceName(songId), play, ask.owner) : 0;
if (id !== 0) keepAsk(sources.audio, id, ask);
return id;
}
/**
* What a song plays as, asked for so (see `playSong`), from `from` ticks on (#338): a bake plays only
* from the start, so one asked to start further in plays its notes.
*/
function songPlay(
{ project, audio }: Pick<ContextSources, 'project' | 'audio'>,
songId: string,
volume: number,
fadeIn: number,
loop: boolean,
owner: string,
fxId: string,
fxParams: FxParams,
bus: string,
): number {
ask: SongAsk,
from: number,
): SequencePlay | null {
const song = findAssetOf(project, songId, 'song');
if (!song) return 0;
if (!song) return null;
const sequence = songSequence(project, song);
const tempo = songTempo(song.props);
const fx = fxVoiceOf(project, pickedFx(fxId, song.props), tempo, fxParams);
const bake = validBake(project, song);
return audio.playSequence(
songVoiceName(songId),
{
sequence,
tempo,
length: songLength(song),
loop: loop ? songLoop(song) : null,
needs: sequenceNeeds(project, sequence),
volume: heldLevel(volume),
fadeIn: heldFade(fadeIn),
buses: songBuses(project, song),
...(fx && { fx }),
...heardOn(playedRoute(project, bus, songId)),
strike: (note) => strikeNote({ project, audio }, note, owner),
strikeClip: (clip) => playClip({ project, audio }, clip),
...(bake && { bake: { file: bake.file, gain: bake.peak, playback: bakePlayback(bake, loop) } }),
},
owner,
const fx = fxVoiceOf(project, pickedFx(ask.fx, song.props), tempo, ask.fxParams);
const bake = ask.live || from > 0 ? null : validBake(project, song);
return {
sequence,
tempo,
length: songLength(song),
loop: ask.loop ? songLoop(song) : null,
needs: sequenceNeeds(project, sequence),
volume: heldLevel(ask.volume),
fadeIn: heldFade(ask.fadeIn),
buses: songBuses(project, song),
...(fx && { fx }),
...heardOn(playedRoute(project, ask.bus, songId)),
strike: (note) => strikeNote({ project, audio }, note, ask.owner),
strikeClip: (clip) => playClip({ project, audio }, clip),
...(bake && { bake: { file: bake.file, gain: bake.peak, playback: bakePlayback(bake, ask.loop) } }),
...(from > 0 && { from }),
};
}
/** Where a change to a playing lands (#338): as soon as it can be heard, or on the next spot `quantize` names. */
function landing(quantize: Quantize, playing: HeardPlaying): LandingAt {
return ({ time, startedAt, soonest }) => {
if (quantize === 'now' || startedAt === null) return soonest;
const spots = quantizeSpots(quantize, playing.timeline.meter, playing.markers);
const next = nextSpot({ ...time, meter: playing.timeline.meter }, soonest - startedAt, spots);
return next === null ? soonest : startedAt + next;
};
}
/** The song playing numbered `playback` as it is heard now; null for none, or a pattern. */
function heardSong(project: Project, audio: AudioSink, playback: number): HeardPlaying | null {
if (!(playback > 0)) return null;
return heardPlayings(project, audio).find((p) => p.id === playback && p.kind === 'song') ?? null;
}
/** Turns a track of a playing song (#338); where it plays its bake there is nothing to turn, and the log says so. */
function setSongTrack(
{ project, audio }: Pick<ContextSources, 'project' | 'audio'>,
world: World,
playback: number,
track: string,
volumeDb: number,
fade: number,
quantize: Quantize,
): void {
const playing = heardSong(project, audio, playback);
if (!playing || track === '') return;
const db = Number.isFinite(volumeDb) ? Math.max(TRACK_MIN_DB, Math.min(TRACK_MAX_DB, volumeDb)) : 0;
const done = audio.setSongTrack?.(
playback,
trackBus(playing.asset, track),
dbToGain(db),
fade,
landing(quantize, playing),
);
if (done === 'baked') {
const name = findAssetOf(project, playing.asset, 'song')?.name ?? playing.asset;
world.log.push(`Set song track: song "${name}" plays its bake; switch on "live tracks" in Play song`);
}
}
/**
* Plays another song, or the same one from a marker (#338), in place of a playing song: asked for
* as that one was, where `quantize` lands for the one heard now. A marker the song does not have is
* logged, and nothing changes.
*/
function switchSong(
sources: Pick<ContextSources, 'project' | 'audio'>,
world: World,
playback: number,
songId: string,
markerId: string,
quantize: Quantize,
fade: number,
): void {
const { project, audio } = sources;
const playing = heardSong(project, audio, playback);
if (!playing) return;
const target = songId || playing.asset;
const song = findAssetOf(project, target, 'song');
if (!song) return;
const marker = markerId ? heldSongMarkers(song.markers).find((m) => m.id === markerId) : undefined;
if (markerId && !marker) {
const named = findSongMarker(project, markerId)?.name ?? markerId;
world.log.push(`Switch song: song "${song.name}" has no marker "${named}"`);
return;
}
const ask = songAsks.get(audio)?.get(playback) ?? {
volume: 1,
fadeIn: 0,
loop: true,
owner: '',
fx: '',
fxParams: {},
bus: '',
live: false,
};
const play = songPlay(sources, target, ask, marker?.tick ?? 0);
if (play) audio.switchSequence?.(playback, songVoiceName(target), play, landing(quantize, playing), fade);
}
/**
@@ -487,8 +627,15 @@ export function strikeNote(
duration: note.duration,
};
const timing = { at: note.at, tempo: note.tempo, ...(note.bus && { bus: note.bus }) };
const fx = fxVoiceOf(sources.project, instrumentFx(sources.project, note.instrument, ''), note.tempo);
return playNote(sources, note.instrument, struck, owner, {}, timing, fx);
// What the lanes of its track start it with (#337) reaches its instrument and its sound FX alike.
const params = note.params ?? {};
const fx = fxVoiceOf(
sources.project,
instrumentFx(sources.project, note.instrument, ''),
note.tempo,
params,
);
return playNote(sources, note.instrument, struck, owner, params, timing, fx);
}
/** A clip of a song (#334): its sound at its own volume times the item's, through its sound FX. */
@@ -505,7 +652,7 @@ export function playClip(
return audio.playClip(
playedSoundOf(ref),
{ at: clip.at, offset: clip.offset, duration: clip.duration, volume, bus: clip.bus },
clipFx(project, clip.fx, clip.tempo),
clipFx(project, clip.fx, clip.tempo, clip.params),
);
}
+190
View File
@@ -0,0 +1,190 @@
/**
* What a game hears of its music (#338): the songs and patterns playing, each with its time, and the
* beats, bars and markers each has passed since the frame before – what `On beat`, `On bar`,
* `On song marker` and `Song position` go by. A composition's playings are none of them.
*/
import { asNumber, type GraphNode, type Value } from '../../core/graph';
import { textOf } from '../../core/model/Properties';
import { findAssetOf, type Project } from '../../core/model/Project';
import {
heardPosition,
type HeardPosition,
newestPassed,
passedBar,
type PassedBeat,
type PassedMarker,
passedBeat,
passedMarkers,
type Timeline,
} from '../../core/music/beats';
import { heldSongMarkers, type SongMarker } from '../../core/music/markers';
import { patternMeter } from '../../core/music/pattern';
import { songMeter } from '../../core/music/song';
import { COMMON_TIME } from '../../core/music/time';
import type { AudioSink } from './Audio';
import type { Instance, Pulse } from './behaviorPulses';
/** A playing of a song or a pattern, as a game follows it. */
export interface HeardPlaying {
id: number;
/** Told apart from the playing of its number a Switch song took over from (#338). */
key: number;
kind: 'song' | 'pattern';
/** The id of the song or pattern. */
asset: string;
timeline: Timeline;
/** Seconds into its playing time it is heard now; null before it starts. */
heard: number | null;
markers: readonly SongMarker[];
}
/** The playings of songs and patterns, oldest first. */
export function heardPlayings(project: Project, audio: AudioSink): HeardPlaying[] {
return (audio.playingTimes?.() ?? []).flatMap((time): HeardPlaying[] => {
const split = time.name.indexOf(':');
const kind = time.name.slice(0, split);
const id = time.name.slice(split + 1);
if (kind !== 'song' && kind !== 'pattern') return [];
const song = kind === 'song' ? findAssetOf(project, id, 'song') : undefined;
const pattern = kind === 'pattern' ? findAssetOf(project, id, 'pattern') : undefined;
const meter = song ? songMeter(song.props) : pattern ? patternMeter(pattern.props) : COMMON_TIME;
const { tempo, from, loop, length } = time;
return [
{
id: time.id,
key: time.key,
kind,
asset: id,
timeline: { tempo, meter, from, loop, length },
heard: time.heard,
markers: song ? heldSongMarkers(song.markers) : [],
},
];
});
}
/** What a playing passed since the frame before. */
export interface MusicPassed {
playing: HeardPlaying;
beat: PassedBeat | null;
bar: PassedBeat | null;
markers: PassedMarker[];
}
/**
* Follows the playings from frame to frame: each frame what each passed since the last one. One
* first seen is told what it passed no more than the limit ago, its first beat among it – so is one
* a Switch song started in place of another, under the same number.
*/
export class MusicFollower {
private before = new Map<number, number>();
follow(playings: readonly HeardPlaying[]): MusicPassed[] {
const next = new Map<number, number>();
const passed = playings.map((playing) => {
const { key, heard, timeline } = playing;
const before = this.before.get(key) ?? null;
if (heard === null) return { playing, beat: null, bar: null, markers: [] };
next.set(key, heard);
return {
playing,
beat: passedBeat(timeline, before, heard),
bar: passedBar(timeline, before, heard),
markers: passedMarkers(timeline, before, heard, playing.markers),
};
});
this.before = next;
return passed;
}
}
/** The playing a node's "playback" input names; null where it is not connected. */
export type PlaybackOf = (instance: Instance, node: GraphNode) => Value;
/** Whether a node hears that playing: by its number, else by the song and the pattern picked. */
function hears(node: GraphNode, wanted: Value, playing: HeardPlaying, keys: readonly string[]): boolean {
if (wanted !== null) return asNumber(wanted) === playing.id;
const picked = keys.map((key) => ({ key, id: textOf(node.params[key] ?? '') })).filter((p) => p.id !== '');
return picked.length === 0 || picked.some((p) => p.key === playing.kind && p.id === playing.asset);
}
export function beatPulses(
instance: Instance,
passed: readonly MusicPassed[],
playbackOf: PlaybackOf,
): Pulse[] {
return instance.interpreter.eventNodes('OnBeat').flatMap((node) => {
const wanted = playbackOf(instance, node);
return passed.flatMap(({ playing, beat }) =>
beat && hears(node, wanted, playing, ['song', 'pattern'])
? [
{
instance,
node,
outputs: { playback: playing.id, bar: beat.bar, beat: beat.beat, late: beat.late },
},
]
: [],
);
});
}
export function barPulses(
instance: Instance,
passed: readonly MusicPassed[],
playbackOf: PlaybackOf,
): Pulse[] {
return instance.interpreter.eventNodes('OnBar').flatMap((node) => {
const wanted = playbackOf(instance, node);
return passed.flatMap(({ playing, bar }) =>
bar && hears(node, wanted, playing, ['song', 'pattern'])
? [{ instance, node, outputs: { playback: playing.id, bar: bar.bar, late: bar.late } }]
: [],
);
});
}
/** Of the markers a playing passed, the last spot of those the node hears: every marker on it. */
export function markerPulses(
instance: Instance,
passed: readonly MusicPassed[],
playbackOf: PlaybackOf,
): Pulse[] {
return instance.interpreter.eventNodes('OnSongMarker').flatMap((node) => {
const wanted = playbackOf(instance, node);
const marker = textOf(node.params.marker ?? '');
return passed.flatMap(({ playing, markers }) => {
if (markers.length === 0 || !hears(node, wanted, playing, ['song'])) return [];
const heard = newestPassed(markers.filter((m) => marker === '' || m.id === marker));
return heard.map((m) => ({
instance,
node,
outputs: { playback: playing.id, marker: m.id, name: m.name, late: m.late },
}));
});
});
}
/**
* Where the playing a `Song position` asks for stands: the one numbered `playback`, else the one
* started last of the song or pattern picked, of every one where none is.
*/
export function positionOf(
playings: readonly HeardPlaying[],
playback: number | null,
song: string,
pattern: string,
): HeardPosition | null {
const picked = [
...(song ? [{ kind: 'song', id: song }] : []),
...(pattern ? [{ kind: 'pattern', id: pattern }] : []),
];
const fits = (p: HeardPlaying) =>
playback !== null
? p.id === playback
: picked.length === 0 || picked.some((pick) => pick.kind === p.kind && pick.id === p.asset);
const playing = playings
.filter(fits)
.reduce<HeardPlaying | null>((last, p) => (!last || p.id > last.id ? p : last), null);
return playing ? heardPosition(playing.timeline, playing.heard) : null;
}
+403 -30
View File
@@ -22,6 +22,9 @@ import { BAKE_RATE } from '../core/music/bake';
import {
type BakePlay,
type BusLevel,
CUT,
type HeardTime,
type LandingAt,
Playbacks,
type SequenceNeeds,
type SequencePlay,
@@ -30,7 +33,13 @@ import { arrivalOf, sendDelays, type TimedBus } from '../core/music/routing';
import type { SendTap } from '../core/graph/Graph';
import { MASTER_MIX, type MixBus, type MixSend, sameMix } from '../core/music/mixer';
import { limiterLatency } from '../core/soundfx/dynamics';
import type { ScheduledClip, ScheduledNote, ScheduledRamp } from '../core/music/Scheduler';
import { type AutomationTarget, paramOf, reaches } from '../core/music/automation';
import type {
ScheduledAutomation,
ScheduledClip,
ScheduledNote,
ScheduledRamp,
} from '../core/music/Scheduler';
import { dbToGain, SOUND_ITEM_MAX_DB } from '../core/music/song';
import { DEFAULT_TEMPO } from '../core/music/time';
import { playedPlan, type StreamedFile, type StreamPlan } from '../core/music/streams';
@@ -48,11 +57,13 @@ import {
FADE,
type Follow,
followCurve,
followMarks,
fadeOut,
gainNode,
GLIDE,
glideKnobs,
heldBack,
holdParam,
type Knob,
levelsOf,
move,
@@ -79,6 +90,11 @@ const MAX_SEND_DELAY = 1;
/** Samples a level meter looks back over (#359): a frame of 40 ms at 48 kHz misses no peak. */
const METER_SPAN = 2048;
/**
* Samples what the agent hears looks back over (#436): the most an analyser holds, about 0.7 s, so
* the gap between two notes does not read as silence.
*/
const WATCH_SPAN = 32768;
/** The loudest the limiter at the output lets a sample be (#359), and how fast it lets go again. */
const GUARD_DB = -1;
const GUARD_RELEASE = 0.1;
@@ -157,6 +173,8 @@ interface Voice {
owner: string;
/** The bus it is heard on, `''` straight out. */
bus: string;
/** The note it is, for a note of an instrument (#436). */
note?: number;
/** When it started on the audio clock, what its Envelopes count from. */
at: number;
sources: Source[];
@@ -252,6 +270,8 @@ interface Start {
bus?: string;
/** The copy `file` streams from (#349); its bytes are not loaded then, its head is. */
stream?: StreamFile;
/** The note it is, for a note of an instrument (#436). */
note?: number;
}
/**
@@ -260,9 +280,15 @@ interface Start {
*/
interface Pending {
id: number;
/** What it plays and who asked, as its voice will have them (#436). */
file: string;
owner: string;
note?: number;
bus: string;
fx: Layered;
timed: { params: FxParams; time: number; ease: EaseShare }[];
/** The marks of parameter lanes (#337) planned for it meanwhile, taken up once it starts. */
lanes: { param: string; automation: ScheduledAutomation }[];
/** Stopped by its number before it started. */
stopped: boolean;
/** A note released before it started: it starts and is released at once, so a one-shot is heard. */
@@ -298,6 +324,13 @@ export class SoundLibrary implements AudioSink {
*/
private guard: { input: GainNode; limiter: AudioWorkletNode | null; latency: number } | null = null;
private guardMeter: BusMeter | null = null;
/** Whether every bus is tapped for what the agent hears (#436). */
private watching = false;
/**
* The state of the audio clock as a pause took hold (#436): one the browser had suspended already
* stays so after the pause, and nothing was heard before it either.
*/
private heldFrom = 'none';
/**
* `guarded` false leaves the limiter out: for checks that follow every node of a voice. With
@@ -334,6 +367,8 @@ export class SoundLibrary implements AudioSink {
private readonly failed = new Set<string>();
private voices: Voice[] = [];
private pending: Pending[] = [];
/** The knobs a parameter's lane (#337) has planned for: the next window ramps on from there. */
private readonly followed = new WeakSet<Knob>();
/**
* The last note of each Acid (#421), by who plays it and the Acid: when it starts, until when it
* is held, its frequency and voice, and what cuts it short – each of the notes that started then.
@@ -509,7 +544,20 @@ export class SoundLibrary implements AudioSink {
const fx = { layers, params, tempo: timing.tempo ?? DEFAULT_TEMPO };
const at = timing.at ?? 0;
const bus = timing.bus ?? '';
return this.begin({ id: 0, file: name, files, volume, loop: false, owner, fx, duration, at, bus });
const note = timing.note;
return this.begin({
id: 0,
file: name,
files,
volume,
loop: false,
owner,
fx,
duration,
at,
bus,
...(note !== undefined && { note }),
});
}
/**
@@ -540,14 +588,43 @@ export class SoundLibrary implements AudioSink {
/**
* Sets the level and the pan of a track of a song (#334) in a quick glide, so a change reaches
* what is sounding on it already. A bus is made on its first use and kept.
* what is sounding on it already. A bus is made on its first use and kept. What is `automated`
* (#337) glides there once, and is its lane's from then on.
*/
setBus(name: string, gain: number, pan: number): void {
setBus(name: string, gain: number, pan: number, automated: readonly AutomationTarget[] = []): void {
const bus = this.bus(name);
const now = this.audio().currentTime;
bus.set = { gain: Math.max(0, gain), pan: Math.max(-1, Math.min(1, pan)) };
bus.gain.gain.setTargetAtTime(Math.max(0, gain), now, GLIDE / 3);
bus.pan.pan.setTargetAtTime(Math.max(-1, Math.min(1, pan)), now, GLIDE / 3);
const glide = (target: AutomationTarget, param: AudioParam, value: number) => {
const ruled = automated.includes(target);
// Its lane takes it from here (#337); one let go hands it back from where the lane left it.
if (ruled && bus.automated.has(target)) return;
if (!ruled && bus.automated.has(target)) holdParam(param, now);
param.setTargetAtTime(value, now, GLIDE / 3);
};
glide('volume', bus.gain.gain, bus.set.gain);
glide('pan', bus.pan.pan, bus.set.pan);
bus.automated = new Set(automated);
}
/**
* A bus taken along its automation over one window (#337): ramped to each mark, or jumped there.
* A fresh start drops what was planned for it from its first mark on.
*/
automateBus(name: string, automation: ScheduledAutomation): void {
const named = paramOf(automation.target);
if (named !== null) return this.automateParam(name, named, automation);
const bus = this.buses.get(name);
const first = automation.marks[0];
if (!bus || !first) return;
const param = automation.target === 'volume' ? bus.gain.gain : bus.pan.pan;
const now = this.audio().currentTime;
if (automation.fresh) holdParam(param, Math.max(now, first.at));
for (const mark of automation.marks) {
const at = Math.max(now, mark.at);
if (mark.jump) param.setValueAtTime(mark.value, at);
else param.linearRampToValueAtTime(mark.value, at);
}
}
/**
@@ -557,7 +634,7 @@ export class SoundLibrary implements AudioSink {
*/
async setBuses(levels: readonly BusLevel[]): Promise<void> {
const waits: Promise<void>[] = [];
for (const { name, gain, pan, into, sends, fx } of levels) {
for (const { name, gain, pan, into, sends, fx, automated } of levels) {
if (into !== undefined) this.bus(into);
const bus = this.bus(name, into ?? null);
if (sends)
@@ -566,12 +643,76 @@ export class SoundLibrary implements AudioSink {
sends.map((send) => ({ ...send, to: send.to ?? MASTER_MIX })),
);
else this.route(name, bus, into ?? null);
this.setBus(name, gain, pan);
this.setBus(name, gain, pan, automated);
waits.push(this.setFx(bus, fx ?? null));
this.keepParams(bus, fx ?? null, automated ?? []);
}
await Promise.all(waits);
}
/**
* A parameter taken along its lane over one window (#337): on the sound FX of the track's bus and of
* every bus inside it, and on every voice heard inside it – playing, or still waiting on its bytes.
* Each mark is read through the chain, so a knob that is not the parameter itself follows too; one
* without a parameter of that name stays where it is.
*/
private automateParam(name: string, param: string, automation: ScheduledAutomation): void {
if (automation.marks.length === 0) return;
const now = this.audio().currentTime;
for (const [busName, bus] of this.buses) {
const fx = bus.fx;
if (!fx || !reaches(name, busName)) continue;
const values = automation.marks.map((mark) => ({ ...fx.params, [param]: mark.value }));
if (bus.chain) {
const levels = values.map((params) => levelsOf(fxChainOf(fx.plan, params, fx.tempo)));
followMarks(bus.chain.knobs, levels, automation.marks, now, automation.fresh, this.followed);
}
bus.fx = { ...fx, params: values.at(-1)! };
}
for (const voice of this.voices)
if (voice.bus !== '' && reaches(name, voice.bus)) this.followVoice(voice, param, automation);
for (const waiting of this.pending)
if (waiting.bus !== '' && reaches(name, waiting.bus)) waiting.lanes.push({ param, automation });
}
/**
* A voice along a parameter's lane: a note's instrument and its own sound FX (`fx:`, see
* `strikeNote`) alike, as `Set sound FX parameters` reaches both.
*/
private followVoice(voice: Voice, param: string, automation: ScheduledAutomation): void {
const { layers, tempo } = voice.fx;
const values = automation.marks.map((mark) => ({
...voice.fx.params,
[param]: mark.value,
[`fx:${param}`]: mark.value,
}));
const levels = values.map((params) => {
const out: number[] = [];
for (const plan of layers) levelsOf(fxChainOf(plan, params, tempo), out);
return out;
});
const now = this.audio().currentTime;
followMarks(voice.knobs, levels, automation.marks, now, automation.fresh, this.followed);
voice.fx = { ...voice.fx, params: values.at(-1)! };
}
/**
* A sound FX set again as it was (#337): a parameter goes to what it is set to now – one a lane let
* go back to where it stands without one –, but one a lane rules is the lane's.
*/
private keepParams(bus: Bus, fx: FxVoice | null, automated: readonly AutomationTarget[]): void {
const held = bus.fx;
if (!fx || !held || held === fx || held.plan !== fx.plan) return;
const ruled = new Set(automated.flatMap((target) => paramOf(target) ?? []));
const params: Record<string, number> = { ...fx.params };
for (const key of ruled) if (held.params[key] !== undefined) params[key] = held.params[key];
const keys = new Set([...Object.keys(params), ...Object.keys(held.params)]);
if ([...keys].every((key) => params[key] === held.params[key])) return;
bus.fx = { ...held, params };
const levels = levelsOf(fxChainOf(held.plan, params, held.tempo));
if (bus.chain) glideKnobs(bus.chain.knobs, levels, this.audio().currentTime, 0, 'linear');
}
/**
* Lets these buses go once what is on them now has faded (#346): their chains would otherwise run
* on for nothing. One used again before then stays.
@@ -606,7 +747,8 @@ export class SoundLibrary implements AudioSink {
/**
* What each track's bus was last set to, with whether its sound FX is there and how late, and the
* level it has now, ramps and fades included (test hook).
* level it has now, ramps and fades included – `turned` what its input is at now, where a game turned
* it (#338) (test hook).
*/
busLevels(): {
name: string;
@@ -617,11 +759,13 @@ export class SoundLibrary implements AudioSink {
fx: boolean;
latency: number;
sends?: { to: string; tap: SendTap; gain: number; delay: number }[];
turned?: number;
}[] {
return [...this.buses].map(([name, bus]) => ({
name,
...bus.set,
level: bus.gain.gain.value,
...((bus.steer !== 1 || bus.input.gain.value !== 1) && { turned: bus.input.gain.value }),
into: bus.sends ? null : bus.into,
fx: bus.chain !== null,
latency: bus.chain?.latency ?? 0,
@@ -716,23 +860,23 @@ export class SoundLibrary implements AudioSink {
return [...this.buses.values()].filter((bus) => bus.meter).length;
}
private tapMeter(from: AudioNode): BusMeter {
private tapMeter(from: AudioNode, span = METER_SPAN): BusMeter {
const context = this.audio();
const split = context.createChannelSplitter(2);
const sides = [0, 1].map((channel) => {
const side = context.createAnalyser();
side.fftSize = METER_SPAN;
side.fftSize = span;
split.connect(side, channel);
return side;
});
from.connect(split);
return { split, sides, samples: new Float32Array(METER_SPAN) };
return { split, sides, samples: new Float32Array(span) };
}
private dropMeter(bus: Bus): void {
const meter = bus.meter;
private dropMeter(bus: Bus, which: 'meter' | 'watch' = 'meter'): void {
const meter = bus[which];
if (!meter) return;
bus.meter = null;
bus[which] = null;
try {
bus.pan.disconnect(meter.split);
} catch {
@@ -741,10 +885,63 @@ export class SoundLibrary implements AudioSink {
meter.split.disconnect();
}
/** Cuts the pan of a bus from where it went; its meter stays on. */
/** Cuts the pan of a bus from where it went; its meters stay on. */
private cutPan(bus: Bus): void {
bus.pan.disconnect();
if (bus.meter) bus.pan.connect(bus.meter.split);
if (bus.watch) bus.pan.connect(bus.watch.split);
}
/**
* Taps every bus for what the agent hears (#436), those made from now on as well, or lets the taps
* go: on only while the agent is switched on, so the editor pays nothing for it otherwise.
*/
watch(on: boolean): void {
if (this.watching === on) return;
this.watching = on;
for (const bus of this.buses.values()) {
if (!on) this.dropMeter(bus, 'watch');
else bus.watch ??= this.tapMeter(bus.pan, WATCH_SPAN);
}
}
/**
* What is heard now (#436): every voice and playing, the voices still waiting on their bytes, the
* files that would not load, and the level of each bus tapped by `watch` over the last
* `WATCH_SPAN` samples, after its level and pan – held where a pause holds the clock.
*/
heard(): HeardSound {
const now = this.context?.currentTime ?? 0;
const voiceOf = (voice: Voice | Pending): HeardVoice => ({
id: voice.id,
file: voice.file,
owner: voice.owner,
bus: voice.bus,
...(voice.note !== undefined && { note: voice.note }),
});
return {
window: this.context ? WATCH_SPAN / this.context.sampleRate : 0,
clock: { state: this.held ? this.heldFrom : this.clockState(), time: now, held: this.held },
voices: this.voices.map((voice) => ({
...voiceOf(voice),
startsIn: Math.max(0, voice.at - now),
released: voice.released,
ringing: voice.ringing,
})),
waiting: this.pending.filter((waiting) => !waiting.stopped).map(voiceOf),
failed: [...this.failed],
playings: this.playbacks().map(({ id, name, owner, playing, fading, baked }) => ({
id,
name,
owner,
playing,
fading,
baked,
})),
buses: [...this.buses].flatMap(([name, bus]) =>
bus.watch ? [{ name, ...readLevels(bus.watch) }] : [],
),
};
}
/** Where the voices of a bus go in; the master for `''` (#325). */
@@ -770,14 +967,19 @@ export class SoundLibrary implements AudioSink {
gain: context.createGain(),
pan: context.createStereoPanner(),
set: { gain: 1, pan: 0 },
steer: 1,
steerEnd: 0,
automated: new Set(),
into: target,
sends: null,
fx: null,
chain: null,
leaving: 0,
meter: null,
watch: null,
};
made.input.connect(made.gain).connect(made.pan).connect(this.outOf(target));
if (this.watching) made.watch = this.tapMeter(made.pan, WATCH_SPAN);
this.buses.set(name, made);
return made;
}
@@ -909,10 +1111,15 @@ export class SoundLibrary implements AudioSink {
const names = new Set(this.mix.map((bus) => bus.name));
const waits: Promise<void>[] = [];
for (const { name, volume, into, sends, fx } of this.mix) {
const made = !this.buses.has(name);
const bus = this.bus(name, into);
if (sends) this.setSends(bus, sends);
else this.route(name, bus, into);
this.glideLevel(bus, this.mixVolumes.get(name) ?? volume);
const level = this.mixVolumes.get(name) ?? volume;
// A bus made now is at its level at once: made with the context, it carries the first sound of
// a run, which would otherwise be heard through a glide down from unity (found by #436).
if (made) this.setLevel(bus, level);
else this.glideLevel(bus, level);
waits.push(this.setFx(bus, fx));
}
const gone = [...this.buses.keys()].filter(
@@ -923,6 +1130,13 @@ export class SoundLibrary implements AudioSink {
await Promise.all(waits);
}
/** The level of a bus at `gain` from now on, without a glide. */
private setLevel(bus: Bus, gain: number): void {
const to = Math.max(0, gain);
bus.gain.gain.value = to;
bus.set = { ...bus.set, gain: to };
}
/** A quick glide of the level of a bus to `gain`. */
private glideLevel(bus: Bus, gain: number, time = 0, ease: EaseShare = 'linear'): void {
const to = Math.max(0, gain);
@@ -1122,6 +1336,7 @@ export class SoundLibrary implements AudioSink {
private dropBus(name: string, bus: Bus): void {
this.dropMeter(bus);
this.dropMeter(bus, 'watch');
this.dropSends(bus);
this.unchain(bus);
bus.input.disconnect();
@@ -1132,23 +1347,86 @@ export class SoundLibrary implements AudioSink {
/**
* A sequence without notes or clips plays nothing, and is given no number. Each playing is heard
* through a master of its own, silent until it starts (#336): its level and its fades. The tracks
* of a song go into it, each set as it was given; a note or clip naming no track goes straight in.
* through a master of its own, silent until it starts (#336): its level and its fades.
*/
playSequence(name: string, play: SequencePlay, owner = ''): number {
if (play.sequence.notes.length === 0 && !play.sequence.clips?.length) return 0;
const id = this.nextId++;
this.sequences.start(id, name, this.routed(id, play), owner);
return id;
}
/**
* Plays `play` in place of the playing numbered `id` (#338), from the second `at` names, as
* `Playbacks.switchTo` does: on buses of its own, its tracks turned as a game turned those of the
* same name, and owned by whoever owns the playing. Says whether there was such a playing.
*/
switchSequence(id: number, name: string, play: SequencePlay, at: LandingAt, fade: number): boolean {
const playings = this.sequences.named(id);
const from = playings.find((playing) => playing.until === null) ?? playings[0];
if (!from || (play.sequence.notes.length === 0 && !play.sequence.clips?.length)) return false;
const key = this.nextId++;
const routed = this.routed(key, play);
for (const [busName, bus] of this.buses) {
const track = trackOf(from.key, busName);
const into = track === null ? undefined : this.buses.get(groupBus(key, track));
if (!into) continue;
into.steer = bus.steer;
into.input.gain.value = bus.steer;
}
this.sequences.switchTo(id, key, name, routed, from.owner, at, fade);
return true;
}
/**
* Turns one track of a playing (#338) to `gain` over `fade` seconds from the second `at` names, on
* top of its own level: the one heard now, and one waiting to take over from it alike. A bake has
* no tracks to turn.
*/
setSongTrack(
id: number,
track: string,
gain: number,
fade: number,
at: LandingAt,
): 'set' | 'baked' | 'none' {
const clock = this.sequences.clockOf(id);
if (!clock) return 'none';
if (clock.baked) return 'baked';
const when = Math.max(this.audio().currentTime, at(clock));
let found = false;
for (const playing of this.sequences.named(id)) {
const bus = this.buses.get(groupBus(playing.key, track));
if (!bus) continue;
found = true;
const from = bus.steer;
bus.steer = Math.max(0, gain);
// Where the last turn is through by then, nothing is planned past `when`, and Chrome's
// cancelAndHoldAtTime holds nothing: the ramp would run from that turn on, seconds early.
if (when < bus.steerEnd) holdParam(bus.input.gain, when);
else bus.input.gain.setValueAtTime(from, when);
bus.steerEnd = when + Math.max(CUT, fade);
bus.input.gain.linearRampToValueAtTime(bus.steer, bus.steerEnd);
}
return found ? 'set' : 'none';
}
/**
* The buses a playing is heard through (#336), `key` its master's: its tracks into it, each set as
* it was given; a note or clip naming no track goes straight in. Silent until it starts.
*/
private routed(key: number, play: SequencePlay): SequencePlay {
// A bake whose file could not be had is no bake: the notes are played instead (#340).
const { bake: asked, ...rest } = play;
const bake = asked && !this.hasFailed(asked.file) ? asked : null;
const buses = bake ? [] : (play.buses ?? []);
this.open(id, buses, play.fx ?? null, play.bus ?? MASTER_MIX, play.sends);
this.open(key, buses, play.fx ?? null, play.bus ?? MASTER_MIX, play.sends);
const tracks = new Set(buses.map((bus) => bus.name));
const into = (bus: string | undefined) => groupBus(id, bus !== undefined && tracks.has(bus) ? bus : '');
const into = (bus: string | undefined) => groupBus(key, bus !== undefined && tracks.has(bus) ? bus : '');
const { strike, strikeClip } = play;
const chains = [...buses.flatMap((bus) => bus.fx ?? []), ...(play.fx ? [play.fx] : [])];
const needs = bake ? { layers: [], files: [bake.file] } : play.needs;
const routed: SequencePlay = {
return {
...rest,
...(bake && { bake }),
needs: { ...needs, layers: [...needs.layers, ...chains.map((fx) => fx.plan)] },
@@ -1158,11 +1436,12 @@ export class SoundLibrary implements AudioSink {
}),
// Only a track's: the master of the playing is its level and its fades.
ramp: (ramp) => {
if (tracks.has(ramp.bus)) this.rampBus(groupBus(id, ramp.bus), ramp);
if (tracks.has(ramp.bus)) this.rampBus(groupBus(key, ramp.bus), ramp);
},
automate: (automation) => {
if (tracks.has(automation.bus)) this.automateBus(groupBus(key, automation.bus), automation);
},
};
this.sequences.start(id, name, routed, owner);
return id;
}
stopSequence(name: string, letRing = false, fade = 0): void {
@@ -1202,6 +1481,11 @@ export class SoundLibrary implements AudioSink {
}));
}
/** Every pattern and song playing, with how far into it is heard now (#338). */
playingTimes(): HeardTime[] {
return this.sequences.times();
}
/**
* What goes into the master of a playing, taken off before its level into `to` (#340): what a bake
* keeps. Says whether there was such a playing.
@@ -1455,7 +1739,7 @@ export class SoundLibrary implements AudioSink {
private sweep(): void {
if (this.groups.size === 0) return;
for (const id of [...this.groups]) {
if (!this.groups.has(id) || this.sequences.has(id)) continue;
if (!this.groups.has(id) || this.sequences.holds(id)) continue;
const on = (bus: string) => bus === groupBus(id, '') || bus.startsWith(`${groupBus(id, '')}:`);
if (this.voices.some((voice) => on(voice.bus))) continue;
if (this.pending.some((waiting) => !waiting.stopped && on(waiting.bus))) continue;
@@ -1553,9 +1837,13 @@ export class SoundLibrary implements AudioSink {
const askedAt = this.cancelsFor(start.file, start.owner);
const waiting: Pending = {
id,
file: start.file,
owner: start.owner,
...(start.note !== undefined && { note: start.note }),
bus: start.bus ?? '',
fx: { ...start.fx, params: { ...start.fx.params } },
timed: [],
lanes: [],
stopped: false,
released: false,
};
@@ -1576,6 +1864,9 @@ export class SoundLibrary implements AudioSink {
if (stream?.failed) delete started.stream;
this.start(started);
for (const { params, time, ease } of waiting.timed) this.tune(id, params, time, ease);
const voice = this.voices.find((other) => other.id === id);
if (voice)
for (const { param, automation } of waiting.lanes) this.followVoice(voice, param, automation);
if (waiting.released) this.release(id);
});
return id;
@@ -1592,7 +1883,12 @@ export class SoundLibrary implements AudioSink {
return true;
}
const voice = this.voices.find((other) => other.id === id);
if (!voice) return this.sequences.has(id) && this.tuneMaster(id, params, time, ease);
// Every playing of that number (#338): one taking over from another is tuned alike.
if (!voice) {
const playings = this.sequences.named(id);
for (const playing of playings) this.tuneMaster(playing.key, params, time, ease);
return playings.length > 0;
}
voice.fx = { ...voice.fx, params: { ...voice.fx.params, ...params } };
if (this.glide(voice, time, ease)) this.tunedAt = this.audio().currentTime;
// A longer feedback makes for a longer tail: it is counted again from where the change ends.
@@ -1605,10 +1901,9 @@ export class SoundLibrary implements AudioSink {
* The sound FX of a pattern or song playing (#346), on its master; one still loading starts with
* the values. Says it was reached, whether or not it has a sound FX.
*/
private tuneMaster(id: number, params: FxParams, time: number, ease: EaseShare): boolean {
const master = this.buses.get(groupBus(id, ''));
private tuneMaster(key: number, params: FxParams, time: number, ease: EaseShare): void {
const master = this.buses.get(groupBus(key, ''));
if (master) this.tuneBus(master, params, time, ease);
return true;
}
private tuneBus(bus: Bus, params: FxParams, time: number, ease: EaseShare): void {
@@ -1726,6 +2021,7 @@ export class SoundLibrary implements AudioSink {
* wakes the clock – a pattern or song reading it every few milliseconds would undo the pause.
*/
setSuspended(suspended: boolean): void {
if (suspended && !this.held) this.heldFrom = this.clockState();
this.held = suspended;
const context = this.live();
if (!context) return;
@@ -2200,6 +2496,7 @@ export class SoundLibrary implements AudioSink {
clip: asked,
bus = '',
stream,
note,
}: Start): void {
const context = this.audio();
// A context starts suspended where the page has not been interacted with; without this the
@@ -2267,6 +2564,7 @@ export class SoundLibrary implements AudioSink {
file,
owner,
bus,
...(note !== undefined && { note }),
at: when,
sources,
ending: new Set(),
@@ -2914,6 +3212,12 @@ function groupBus(id: number, track: string): string {
return track === '' ? `play:${id}` : `play:${id}:${track}`;
}
/** The track a bus of the playing `key` is (#338); null for its master and any other bus. */
function trackOf(key: number, bus: string): string | null {
const head = `${groupBus(key, '')}:`;
return bus.startsWith(head) ? bus.slice(head.length) : null;
}
/**
* A track of a song (#334): what goes in passes its sound FX (#346), then its level, then its pan,
* then the bus it goes into or straight out – or each of its sends (#350); `set` is what level and
@@ -2924,6 +3228,15 @@ interface Bus {
gain: GainNode;
pan: StereoPannerNode;
set: { gain: number; pan: number };
/**
* What a game turns a track of its playing to (#338), on its input: on top of its level, before its
* sound FX and every send, what goes into it from other tracks too.
*/
steer: number;
/** The second of the audio clock the last turn of `steer` is through at; 0 for none. */
steerEnd: number;
/** What a lane of automation sets instead (#337), where `setBus` leaves it alone. */
automated: Set<AutomationTarget>;
/** Where it goes as a whole while it has no sends; null straight out. */
into: string | null;
sends: BusSendNodes[] | null;
@@ -2934,6 +3247,8 @@ interface Bus {
leaving: number;
/** Its level meter (#359), while one is asked after. */
meter: BusMeter | null;
/** What the agent hears of it (#436), while the agent is switched on. */
watch: BusMeter | null;
}
/** A tap after the pan of a bus: an analyser for each side, so a peak panned hard is not halved. */
@@ -3061,6 +3376,64 @@ function readMeter(meter: BusMeter): [number, number] {
return [left!, right!];
}
/**
* The loudest sample and the RMS of each side over the stretch a meter looks back over (#436), 1 at
* full scale.
*/
function readLevels(meter: BusMeter): { peak: [number, number]; rms: [number, number] } {
const [left, right] = meter.sides.map((side) => {
side.getFloatTimeDomainData(meter.samples);
let peak = 0;
let squares = 0;
for (const sample of meter.samples) {
peak = Math.max(peak, Math.abs(sample));
squares += sample * sample;
}
return { peak, rms: Math.sqrt(squares / meter.samples.length) };
});
return { peak: [left!.peak, right!.peak], rms: [left!.rms, right!.rms] };
}
/** One voice as `heard` gives it (#436): a sound by its file, a note by `instrument:<id>` and its note. */
export interface HeardVoice {
id: number;
file: string;
/** The behavior it belongs to, `''` for none. */
owner: string;
/** Its bus, `''` the master. */
bus: string;
note?: number;
}
/** What `SoundLibrary.heard` gives (#436). */
export interface HeardSound {
/** Seconds the levels look back over; 0 before there is any sound. */
window: number;
/**
* The audio clock: `running`, `suspended` or `none` before there is any sound – while a pause holds
* it, as it was when the pause took hold –, the second it is at, and whether a pause holds it.
* Suspended, nothing is heard.
*/
clock: { state: string; time: number; held: boolean };
/** Voices started, a planned one `startsIn` seconds from now. */
voices: (HeardVoice & { startsIn: number; released: boolean; ringing: boolean })[];
/** Voices waiting on their bytes. */
waiting: HeardVoice[];
/** Files that would not load. */
failed: string[];
/** Patterns, songs and compositions playing, by `pattern:<id>`, `song:<id>`, `composition:<id>`. */
playings: {
id: number;
name: string;
owner: string;
playing: boolean;
fading: number | null;
baked: boolean;
}[];
/** The tapped buses by their name in the sink: `mix`, `mix:<id>`, `play:<n>`, `play:<n>:<track>`. */
buses: { name: string; peak: [number, number]; rms: [number, number] }[];
}
export const soundLibrary = new SoundLibrary();
/** A composition fits an audio item to the length of its sound once the bytes are here (#418). */
+46 -8
View File
@@ -620,6 +620,18 @@ export function fadeOut(param: AudioParam, now: number, seconds = FADE): void {
param.linearRampToValueAtTime(0, now + seconds);
}
/** Keeps a level where it is at `at` and drops what was planned for it after that. */
export function holdParam(param: AudioParam, at: number): void {
// Firefox has no cancelAndHoldAtTime: there the value it reads out now is held by hand.
const held = param as Omit<AudioParam, 'cancelAndHoldAtTime'> &
Partial<Pick<AudioParam, 'cancelAndHoldAtTime'>>;
if (held.cancelAndHoldAtTime) held.cancelAndHoldAtTime(at);
else {
param.cancelScheduledValues(at);
param.setValueAtTime(param.value, at);
}
}
/**
* Sends one knob from where it is right now to `to`. Whatever was still planned for it is dropped
* first, so a change never queues behind an older one. A timed change runs along `ease` – a `ratio`
@@ -635,14 +647,7 @@ export function move(
ease: EaseShare = 'linear',
): void {
const from = param.value;
// Firefox has no cancelAndHoldAtTime: there the value it reads out now is held by hand.
const held = param as Omit<AudioParam, 'cancelAndHoldAtTime'> &
Partial<Pick<AudioParam, 'cancelAndHoldAtTime'>>;
if (held.cancelAndHoldAtTime) held.cancelAndHoldAtTime(now);
else {
param.cancelScheduledValues(now);
param.setValueAtTime(from, now);
}
holdParam(param, now);
const logarithmic = ratio && from > 0 && to > 0;
if (time <= 0) param.setTargetAtTime(to, now, GLIDE);
else if (ease === 'step') param.setValueAtTime(to, now + time);
@@ -670,6 +675,39 @@ export function glideKnobs(
return moved;
}
/**
* Takes knobs along the marks of a parameter's lane (#337): `levels` are the knobs' levels at each
* mark, in the order of `marks`. A knob that stays where it is all the way is left alone. Each other
* one is ramped to its level at each mark, or jumped there; its first mark is a jump where nothing
* has been planned for it by a lane yet – `followed` says which have –, or where the playing has
* just started (`fresh`), which drops what was planned for it from there on.
*/
export function followMarks(
knobs: readonly Knob[],
levels: readonly (readonly number[])[],
marks: readonly { at: number; jump: boolean }[],
now: number,
fresh: boolean,
followed: WeakSet<Knob>,
): void {
knobs.forEach((knob, i) => {
const param = knob.param;
const way = levels.map((level) => level[i]);
if (!param || way.every((level) => level === undefined || level === knob.target)) return;
const restart = fresh || !followed.has(knob);
followed.add(knob);
marks.forEach((mark, m) => {
const level = way[m];
if (level === undefined) return;
const at = Math.max(now, mark.at);
if (m === 0 && fresh) holdParam(param, at);
if (mark.jump || (m === 0 && restart)) param.setValueAtTime(level, at);
else param.linearRampToValueAtTime(level, at);
});
knob.target = way.at(-1) ?? knob.target;
});
}
/** Samples per second of an eased curve: a bounce keeps its corners, a long sweep stays small. */
const CURVE_RATE = 200;
+1
View File
@@ -83,6 +83,7 @@ export class Music {
),
play: (note) => this.playOne(note),
playClip: (clip) => this.playClip(clip),
automate: (automation) => soundLibrary.automateBus(automation.bus, automation),
playing: (voice) => soundLibrary.playing(voice),
stop: (voices) => {
for (const voice of voices) soundLibrary.stopVoice(voice);
+310
View File
@@ -0,0 +1,310 @@
/**
* Automation of the parameters of a song's track (#337, stage 3): a lane `param:<name>` with a range
* of its own, the value each note and clip starts with, the sound FX of the track and its items set
* where the lane starts, and which parameters a track has to offer.
*/
import { describe, expect, it } from 'vitest';
import { CreateAssetCommand, SetAssetPropertyCommand } from '../../src/core/commands/assetCommands';
import { SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
import { SetSongCommand } from '../../src/core/commands/songCommands';
import type { BehaviorAsset, InstrumentAsset, SongAsset, SoundFxAsset } from '../../src/core/model/Asset';
import { findAssetOf } from '../../src/core/model/Project';
import { chipPresetGraph } from '../../src/core/instrument/chipPresets';
import {
type AutomationLane,
heldLanes,
laneMarks,
laneRange,
paramsAt,
reaches,
type SequenceLane,
} from '../../src/core/music/automation';
import {
type ScheduledClip,
type ScheduledNote,
Scheduler,
type SchedulerHost,
} from '../../src/core/music/Scheduler';
import { newItem, songSequence, trackBus } from '../../src/core/music/song';
import { songBuses } from '../../src/core/music/songBuses';
import { defaultParamRange, trackParams } from '../../src/core/music/trackParams';
import { strikeNote } from '../../src/runtime/systems/behaviorContext';
import { type AudioSink, SILENT_AUDIO } from '../../src/runtime/systems/Audio';
import { graphEditor, makeProject, type ProjectFixture } from '../helpers';
const Q = 480;
const BAR = 4 * Q;
describe('a lane of a parameter (#337)', () => {
it('is held to a name without a colon, its points to its own range, and kept without a point', () => {
expect(
heldLanes([
{
target: 'param:cut',
min: 100,
max: 5000,
points: [
{ tick: 0, value: 20000 },
{ tick: 10, value: 50 },
],
},
{ target: 'param:', points: [{ tick: 0, value: 1 }] },
{ target: 'param:a:b', points: [{ tick: 0, value: 1 }] },
{ target: 'param:mix', min: 1, max: 1, points: [{ tick: 0, value: 3 }] },
{ target: 'param:empty', min: -1, max: 1, points: [] },
{ target: 'param:bare', points: [] },
]),
).toEqual([
{
target: 'param:cut',
min: 100,
max: 5000,
points: [
{ tick: 0, value: 5000 },
{ tick: 10, value: 100 },
],
},
// A range that is none is the one a lane without a range has: 0 to 1.
{ target: 'param:mix', points: [{ tick: 0, value: 1 }] },
{ target: 'param:empty', min: -1, max: 1, points: [] },
{ target: 'param:bare', points: [] },
]);
expect(laneRange({ target: 'param:mix' })).toEqual({ min: 0, max: 1 });
expect(laneRange('volume')).toEqual({ min: -60, max: 12 });
});
it('reaches its track and every bus inside it, and starts what is heard there at its value', () => {
expect(reaches('song:s:t', 'song:s:t')).toBe(true);
expect(reaches('song:s:t', 'song:s:t:#items')).toBe(true);
expect(reaches('song:s:t', 'song:s:t2:#items')).toBe(false);
const lanes: SequenceLane[] = [
{
bus: 'song:s:t',
target: 'param:cut',
min: 0,
max: 10,
points: [
{ tick: 0, value: 0 },
{ tick: 100, value: 20 },
],
},
{ bus: 'song:s:t', target: 'volume', points: [{ tick: 0, value: -6 }] },
{ bus: 'song:s:u', target: 'param:other', points: [{ tick: 0, value: 1 }] },
];
expect(paramsAt(lanes, 'song:s:t:#items', 25)).toEqual({ cut: 5 });
// Past its range the way is held to it.
expect(paramsAt(lanes, 'song:s:t:item', 90)).toEqual({ cut: 10 });
expect(paramsAt(lanes, 'song:s:x:#items', 25)).toBeUndefined();
expect(paramsAt(lanes, undefined, 25)).toBeUndefined();
});
it('is read every step even where it runs straight, as a knob may not be the parameter itself', () => {
const lane: SequenceLane = {
bus: 'b',
target: 'param:cut',
min: 0,
max: 1000,
points: [
{ tick: 0, value: 0 },
{ tick: 960, value: 1000 },
],
};
const marks = laneMarks(lane, 0, null, 0, 480, true, 9.6);
expect(marks).toHaveLength(51);
// In the units of the parameter, not as a gain.
for (const mark of marks) expect(mark.value).toBeCloseTo((1000 * mark.at) / 960, 9);
const held = laneMarks({ ...lane, points: [{ tick: 0, value: 300 }] }, 0, null, 0, 480, true, 9.6);
expect(held).toEqual([
{ at: 0, value: 300, jump: true },
{ at: 480, value: 300, jump: false },
]);
});
});
describe('the scheduler and a parameter lane (#337)', () => {
it('starts each note and clip of the track at the value the lane has where it starts', () => {
const notes: ScheduledNote[] = [];
const clips: ScheduledClip[] = [];
let time = 0;
const host: SchedulerHost = {
now: () => time,
latency: () => 0,
play: (note) => (notes.push(note), 1),
playClip: (clip) => (clips.push(clip), 2),
playing: () => true,
stop: () => {},
wake: () => () => {},
};
const lane: SequenceLane = {
bus: 'song:s:t',
target: 'param:cut',
min: 0,
max: 1000,
points: [
{ tick: 0, value: 0 },
{ tick: 1000, value: 1000 },
],
};
const scheduler = new Scheduler(
host,
() => ({
notes: [
{ tick: 200, length: 10, note: 60, velocity: 1, instrument: 'i', bus: 'song:s:t:#items' },
{ tick: 210, length: 10, note: 60, velocity: 1, instrument: 'i', bus: 'song:s:u:#items' },
],
clips: [{ tick: 100, length: 500, sound: 's', fx: '', offset: 0, volume: 1, bus: 'song:s:t:#items' }],
lanes: [lane],
}),
{ tempo: 120 },
);
scheduler.start(300);
time = 0.5;
scheduler.pump();
expect(notes).toEqual([]);
// Started inside the clip: it is taken up 200 ticks in, at the value there.
expect(clips.map((clip) => clip.params)).toEqual([{ cut: 300 }]);
scheduler.start(150);
time = 1;
scheduler.pump();
expect(notes.map((note) => [note.bus, note.params])).toEqual([
['song:s:t:#items', { cut: 200 }],
['song:s:u:#items', undefined],
]);
});
});
/** A sound FX "Filter" with a parameter `cut` (default 800) into its gain. */
function makeFilter(f: ProjectFixture): SoundFxAsset {
const command = new CreateAssetCommand('soundfx', 'Filter');
f.stack.execute(command);
const fx = command.asset as SoundFxAsset;
const edit = graphEditor(f, fx as unknown as BehaviorAsset);
const input = edit.addNode('SoundFxInput');
const gain = edit.addNode('SoundFxGain');
const output = edit.addNode('SoundFxOutput');
input.ports = [{ key: 'cut', kind: 'number', default: 800 }];
edit.connect(input, 'out', gain, 'in');
edit.connect(input, 'cut', gain, 'gain');
edit.connect(gain, 'out', output, 'in');
return fx;
}
/** A song "Tune" with a pattern of one note on its one track, played by a chip instrument with a parameter `tone`. */
function makeTune() {
const f = makeProject();
const song = new CreateAssetCommand('song', 'Tune');
const pattern = new CreateAssetCommand('pattern', 'Riff');
const chip = new CreateAssetCommand('instrument', 'Chip');
for (const command of [song, pattern, chip]) f.stack.execute(command);
const instrument = chip.asset as InstrumentAsset;
instrument.graph = chipPresetGraph('pickup', () => 0);
instrument.graph.nodes.push({
id: 'params',
type: 'InstrumentParameters',
x: 0,
y: 0,
params: {},
ports: [
{ key: 'tone', kind: 'number', default: 0.25 },
{ key: 'cut', kind: 'number', default: 5 },
],
});
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'bars', 1));
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'instrument', instrument.id));
f.stack.execute(
new SetPatternNotesCommand(pattern.asset.id, [{ id: 'n1', tick: 0, length: Q, note: 60, velocity: 1 }]),
);
const tune = () => findAssetOf(f.project, song.asset.id, 'song') as SongAsset;
const trackId = tune().tracks[0]!.id;
f.stack.execute(
new SetSongCommand(song.asset.id, {
items: [newItem({ track: trackId, kind: 'pattern', asset: pattern.asset.id, tick: 0, length: BAR })],
}),
);
const setTrack = (fields: Partial<SongAsset['tracks'][number]>) =>
f.stack.execute(new SetSongCommand(song.asset.id, { tracks: [{ ...tune().tracks[0]!, ...fields }] }));
return { ...f, tune, trackId, setTrack, instrument, patternId: pattern.asset.id };
}
describe('a song with a parameter lane (#337)', () => {
const cutLane: AutomationLane = {
target: 'param:cut',
min: 0,
max: 2000,
points: [
{ tick: 0, value: 400 },
{ tick: BAR, value: 1600 },
],
};
it('plays it on its track, and sets the sound FX of the track where it starts', () => {
const f = makeTune();
const fx = makeFilter(f);
f.setTrack({ fx: fx.id, automation: [cutLane] });
expect(songSequence(f.project, f.tune()).lanes).toEqual([
{ bus: trackBus(f.tune().id, f.trackId), ...cutLane },
]);
const [track] = songBuses(f.project, f.tune());
expect(track).toMatchObject({ automated: ['param:cut'], fx: { params: { cut: 400 } } });
// Without a point it rules nothing, and is not played.
f.setTrack({ automation: [{ ...cutLane, points: [] }] });
expect(songSequence(f.project, f.tune()).lanes).toBeUndefined();
expect(songBuses(f.project, f.tune())[0]).not.toHaveProperty('automated');
expect(songBuses(f.project, f.tune())[0]!.fx!.params).toEqual({});
});
it('sets the sound FX of a pattern item on the track the same way, which leaves it to the lane', () => {
const f = makeTune();
const fx = makeFilter(f);
const item = f.tune().items[0]!;
f.stack.execute(new SetSongCommand(f.tune().id, { items: [{ ...item, fx: fx.id }] }));
f.setTrack({ automation: [cutLane] });
const own = songBuses(f.project, f.tune()).find((bus) => bus.name.endsWith(item.id));
expect(own).toMatchObject({ automated: ['param:cut'], fx: { params: { cut: 400 } } });
});
it('offers the parameters of the track, its items and their instruments, each name once', () => {
const f = makeTune();
const fx = makeFilter(f);
expect(trackParams(f.project, f.tune(), f.tune().tracks[0]!)).toEqual([
{ name: 'tone', default: 0.25 },
{ name: 'cut', default: 5 },
]);
f.setTrack({ fx: fx.id });
// The track's own first: its `cut` is the one whose default is named.
expect(trackParams(f.project, f.tune(), f.tune().tracks[0]!)).toEqual([
{ name: 'cut', default: 800 },
{ name: 'tone', default: 0.25 },
]);
});
it('starts a new lane at a range round the default, on a round number', () => {
expect(defaultParamRange(0.25)).toEqual({ min: 0, max: 1 });
expect(defaultParamRange(800)).toEqual({ min: 0, max: 2000 });
expect(defaultParamRange(1.5)).toEqual({ min: 0, max: 5 });
expect(defaultParamRange(-3)).toEqual({ min: -10, max: 10 });
expect(defaultParamRange(Number.NaN)).toEqual({ min: 0, max: 1 });
});
it('hands a note its values for its instrument and its own sound FX alike', () => {
const f = makeTune();
const fx = makeFilter(f);
f.stack.execute(new SetAssetPropertyCommand(f.instrument.id, 'fx', fx.id));
const struck: { params: Record<string, number> }[] = [];
const audio: AudioSink = {
...SILENT_AUDIO,
playNote: (_name, _layers, params) => (struck.push({ params: { ...params } }), 1),
};
const note: ScheduledNote = {
at: 0,
duration: 0.1,
note: 60,
velocity: 1,
instrument: f.instrument.id,
tempo: 120,
params: { cut: 1200 },
};
expect(strikeNote({ project: f.project, audio }, note, '')).toBe(1);
expect(struck[0]!.params).toMatchObject({ cut: 1200, 'fx:cut': 1200 });
});
});
+403
View File
@@ -0,0 +1,403 @@
/**
* Automation of a song (#337): lanes of a track's volume and pan, their value along the song, and
* how the scheduler takes a bus along them on the audio clock – in the Song tab, in a game, in a
* composition and in a bake alike.
*/
import { describe, expect, it } from 'vitest';
import { CreateAssetCommand, SetAssetPropertyCommand } from '../../src/core/commands/assetCommands';
import { SetCompositionCommand } from '../../src/core/commands/compositionCommands';
import { SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
import { SetSongCommand } from '../../src/core/commands/songCommands';
import type { CompositionAsset, SongAsset } from '../../src/core/model/Asset';
import { curveLookup } from '../../src/core/model/curve';
import { findAssetOf } from '../../src/core/model/Project';
import {
type AutomationLane,
heldLanes,
laneMarks,
laneValueAt,
laneValueBefore,
type SequenceLane,
type SequenceLanePoint,
sequenceLaneOf,
unrolledLane,
} from '../../src/core/music/automation';
import { bakePrint } from '../../src/core/music/bake';
import { audioItemBus, compositionPlayings } from '../../src/core/music/compositionAudio';
import { dbToGain } from '../../src/core/music/level';
import {
LEAD,
type ScheduledAutomation,
Scheduler,
type SchedulerHost,
} from '../../src/core/music/Scheduler';
import { heldSongTracks, newItem, songSequence, trackBus } from '../../src/core/music/song';
import { songBuses } from '../../src/core/music/songBuses';
import { makeProject, roundTrip } from '../helpers';
const Q = 480;
const BAR = 4 * Q;
const pan = (points: SequenceLanePoint[]): SequenceLane => ({ bus: 'b', target: 'pan', points });
/** The marks with their ticks and values to a millionth, so a sum of floats reads as what it is. */
const rounded = (marks: { at: number; value: number; jump: boolean }[]) =>
marks.map(({ at, value, jump }) => [Math.round(at * 1e6) / 1e6, Math.round(value * 1e6) / 1e6, jump]);
describe('the lanes of a track (#337)', () => {
it('are held to a target each, points in whole ticks from 0 with values in range, in tick order', () => {
expect(
heldLanes([
{
target: 'pan',
points: [
{ tick: 960.4, value: 3 },
{ tick: 0, value: -0.5, ease: 'sine-in' },
{ tick: -1, value: 0 },
{ tick: 10, value: Number.NaN },
],
},
{ target: 'pan', points: [{ tick: 0, value: 1 }] },
{ target: 'volume', points: [] },
{ target: 'speed', points: [{ tick: 0, value: 1 }] },
'volume',
{
target: 'volume',
points: [
{ tick: 5, value: 0, ease: 'wobbly' },
{ tick: 0, value: -100, ease: 'linear' },
],
},
]),
).toEqual([
{
target: 'pan',
points: [
{ tick: 0, value: -0.5, ease: 'sine-in' },
{ tick: 960, value: 1 },
],
},
{
target: 'volume',
points: [
{ tick: 0, value: -60 },
{ tick: 5, value: 0 },
],
},
]);
expect(heldLanes('x')).toEqual([]);
});
it('are kept with their track when the project is saved, and a track without one has none', () => {
const f = makeProject();
const create = new CreateAssetCommand('song', 'Tune');
f.stack.execute(create);
const song = () => findAssetOf(f.project, create.asset.id, 'song') as SongAsset;
const lane: AutomationLane = { target: 'volume', points: [{ tick: 0, value: -6, ease: 'expo-out' }] };
const [first] = song().tracks;
f.stack.execute(
new SetSongCommand(song().id, {
tracks: [
{ ...first!, automation: [lane] },
{ ...first!, id: 'plain', automation: [] },
],
}),
);
const saved = roundTrip(f.project).assets.find((a) => a.id === song().id) as SongAsset;
expect(saved.tracks.map((t) => t.automation)).toEqual([[lane], undefined]);
expect(Object.keys(heldSongTracks([{ id: 'x' }])[0]!)).not.toContain('automation');
});
it('runs from each point to the next along its easing, holds outside them, and jumps where two meet', () => {
const points: SequenceLanePoint[] = [
{ tick: 0, value: 0 },
{ tick: 100, value: 1, ease: 'step' },
{ tick: 200, value: 0, ease: 'sine-in' },
{ tick: 300, value: 1 },
{ tick: 300, value: -1 },
];
const at = (tick: number) => laneValueAt(points, tick);
const before = (tick: number) => laneValueBefore(points, tick);
expect([at(-5), at(50), at(100), before(100)]).toEqual([0, 0.5, 1, 1]);
// A step holds up to the next point, which it jumps to.
expect([at(150), before(200), at(200)]).toEqual([1, 1, 0]);
expect(at(250)).toBeCloseTo(1 - Math.cos(Math.PI / 4), 12);
expect(before(300)).toBeCloseTo(1, 12);
expect([at(300), at(400)]).toEqual([-1, -1]);
});
it('eases along a Curve asset it names, and linear once that is gone', () => {
const f = makeProject();
f.project.assets.push({
id: 'cv',
type: 'curve',
name: 'Knee',
props: {},
points: [
{ x: 0, y: 0 },
{ x: 0.5, y: 0.9 },
{ x: 1, y: 1 },
],
});
const lane: AutomationLane = {
target: 'pan',
points: [
{ tick: 0, value: 0, ease: 'curve:cv' },
{ tick: 100, value: 1 },
],
};
const curves = curveLookup(f.project);
expect(laneValueAt(sequenceLaneOf(lane, 'b', curves).points, 25)).toBeCloseTo(0.45, 9);
f.project.assets.pop();
expect(laneValueAt(sequenceLaneOf(lane, 'b', curveLookup(f.project)).points, 25)).toBe(0.25);
});
});
describe('a lane over a window of the scheduler (#337)', () => {
const sweep = pan([
{ tick: 0, value: -1 },
{ tick: 960, value: 1 },
]);
it('is one ramp where the bus can ramp along it: a straight pan', () => {
expect(rounded(laneMarks(sweep, 0, null, 0, 480, true, 9.6))).toEqual([
[0, -1, true],
[480, 0, false],
]);
// Going on, it starts where the last window ended, as a ramp.
expect(rounded(laneMarks(sweep, 0, null, 480, 600, false, 9.6))).toEqual([
[480, 0, false],
[600, 0.25, false],
]);
});
it('is read every step where the bus cannot: decibels in a gain, or an easing', () => {
const fade: SequenceLane = {
bus: 'b',
target: 'volume',
points: [
{ tick: 0, value: -60 },
{ tick: 960, value: 0 },
],
};
const marks = laneMarks(fade, 0, null, 0, 480, true, 9.6);
// 0 and 480, and the 49 readings between.
expect(marks).toHaveLength(51);
expect(marks[0]).toEqual({ at: 0, value: 0, jump: true });
for (const mark of marks) expect(mark.value).toBeCloseTo(dbToGain(-60 + (60 * mark.at) / 960), 12);
expect(marks.slice(1).every((mark, i) => !mark.jump && mark.at > marks[i]!.at)).toBe(true);
const eased = pan([
{ tick: 0, value: 0, ease: 'sine-in' },
{ tick: 100, value: 1 },
]);
expect(laneMarks(eased, 0, null, 0, 100, true, 25).map((mark) => mark.at)).toEqual([0, 25, 50, 75, 100]);
});
it('jumps back with the playhead at the end of a loop, and where a step lands', () => {
expect(rounded(laneMarks(sweep, 0, { start: 0, end: 960 }, 480, 1440, false, 9.6))).toEqual([
[480, 0, false],
[960, 1, false],
[960, -1, true],
[1440, 0, false],
]);
const step = pan([
{ tick: 0, value: -1, ease: 'step' },
{ tick: 100, value: 1 },
]);
expect(rounded(laneMarks(step, 0, null, 0, 200, true, 9.6))).toEqual([
[0, -1, true],
[100, 1, true],
[200, 1, false],
]);
});
});
describe('a lane round the loop of a song inside a composition (#337)', () => {
it('is cut at the loop end and starts again there, each round on from where it was', () => {
const points: SequenceLanePoint[] = [
{ tick: 0, value: -1 },
{ tick: 960, value: 1 },
];
const unrolled = unrolledLane(points, { start: 0, end: 960 }, 480, 1440, 9.6);
expect(unrolled).toEqual([
{ tick: 480, value: 0 },
{ tick: 960, value: 1 },
{ tick: 960, value: -1 },
{ tick: 1440, value: 0 },
]);
expect(laneValueAt(unrolled, 700)).toBeCloseTo(laneValueAt(points, 700), 12);
expect(laneValueAt(unrolled, 1200)).toBeCloseTo(laneValueAt(points, 240), 12);
});
it('keeps a whole easing as it is, and reads one cut in the middle every step', () => {
const eased: SequenceLanePoint[] = [
{ tick: 0, value: 0, ease: 'sine-in' },
{ tick: 960, value: 1 },
];
expect(unrolledLane(eased, null, 0, 1920, 96)).toEqual([
{ tick: 0, value: 0, ease: 'sine-in' },
{ tick: 960, value: 1 },
{ tick: 1920, value: 1 },
]);
const cut = unrolledLane(eased, null, 480, 960, 96);
expect(cut.map((p) => p.tick)).toEqual([480, 576, 672, 768, 864, 960]);
for (const point of cut) {
expect(point.ease).toBeUndefined();
expect(point.value).toBeCloseTo(laneValueAt(eased, point.tick), 12);
}
});
});
/** A song "Tune" with a one-bar pattern of one note on its track, and a composition "Cut" at 10 fps. */
function makeTune() {
const f = makeProject();
const song = new CreateAssetCommand('song', 'Tune');
const pattern = new CreateAssetCommand('pattern', 'Riff');
const piano = new CreateAssetCommand('instrument', 'Piano');
const comp = new CreateAssetCommand('composition', 'Cut', { fps: 10 });
for (const command of [song, pattern, piano, comp]) f.stack.execute(command);
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'bars', 1));
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'instrument', piano.asset.id));
f.stack.execute(
new SetPatternNotesCommand(pattern.asset.id, [{ id: 'n1', tick: 0, length: Q, note: 60, velocity: 1 }]),
);
const tune = () => findAssetOf(f.project, song.asset.id, 'song') as SongAsset;
const trackId = tune().tracks[0]!.id;
f.stack.execute(
new SetSongCommand(song.asset.id, {
items: [newItem({ track: trackId, kind: 'pattern', asset: pattern.asset.id, tick: 0, length: BAR })],
}),
);
const automate = (...automation: AutomationLane[]) =>
f.stack.execute(new SetSongCommand(song.asset.id, { tracks: [{ ...tune().tracks[0]!, automation }] }));
const cut = () => findAssetOf(f.project, comp.asset.id, 'composition') as CompositionAsset;
return { ...f, tune, trackId, automate, cut };
}
describe('a song with automation (#337)', () => {
it('plays its lanes on the buses of its tracks, which start where the lanes do', () => {
const f = makeTune();
expect(songSequence(f.project, f.tune()).lanes).toBeUndefined();
expect(songBuses(f.project, f.tune())[0]).not.toHaveProperty('automated');
f.automate({
target: 'volume',
points: [
{ tick: 0, value: -12 },
{ tick: BAR, value: 0 },
],
});
expect(songSequence(f.project, f.tune()).lanes).toEqual([
{
bus: trackBus(f.tune().id, f.trackId),
target: 'volume',
points: [
{ tick: 0, value: -12 },
{ tick: BAR, value: 0 },
],
},
]);
// The lane rules the volume, the fader the pan.
expect(songBuses(f.project, f.tune())[0]).toMatchObject({
gain: dbToGain(-12),
pan: 0,
automated: ['volume'],
});
});
it('is a bake of its own, and one without automation keeps the print it had', () => {
const f = makeTune();
const plain = bakePrint(f.project, f.tune());
const sweep = (to: number): AutomationLane => ({
target: 'pan',
points: [
{ tick: 0, value: 0.5 },
{ tick: BAR / 2, value: to },
],
});
f.automate(sweep(1));
const panned = bakePrint(f.project, f.tune());
expect(panned).not.toBe(plain);
// Only later in the song: where its tracks start is the same.
f.automate(sweep(-1));
expect(bakePrint(f.project, f.tune())).not.toBe(panned);
f.automate();
expect(bakePrint(f.project, f.tune())).toBe(plain);
});
it('plays its lanes in a composition on the buses of its item, round its loop, in milliseconds', () => {
const f = makeTune();
f.automate({
target: 'pan',
points: [
{ tick: 0, value: -1 },
{ tick: BAR, value: 1 },
],
});
// From 1 s for 3 s, half a second into the song: it goes round its two-second bar once.
const item = {
id: 'it',
track: f.cut().tracks[0]!.id,
start: 10,
length: 30,
offset: 0.5,
audio: { kind: 'song' as const, asset: f.tune().id, volume: 0, bus: '' },
};
f.stack.execute(new SetCompositionCommand(f.cut().id, { items: [item] }));
const [playing] = compositionPlayings(f.project, f.cut());
expect(playing!.sequence.lanes).toEqual([
{
bus: `${audioItemBus(f.cut().id, 'it')}:${f.trackId}`,
target: 'pan',
points: [
{ tick: 1000, value: -0.5 },
{ tick: 2500, value: 1 },
{ tick: 2500, value: -1 },
{ tick: 4000, value: 0.5 },
],
},
]);
expect(playing!.buses.map((bus) => bus.name)).toContain(`${audioItemBus(f.cut().id, 'it')}:${f.trackId}`);
});
});
describe('the scheduler and a lane (#337)', () => {
it('hands each window to the host on the audio clock, the first fresh, each going on from the last', () => {
const automated: ScheduledAutomation[] = [];
let time = 10;
let pump: (() => void) | null = null;
const host: SchedulerHost = {
now: () => time,
latency: () => 0,
play: () => 0,
automate: (automation) => void automated.push(automation),
playing: () => false,
stop: () => {},
wake: (wake) => ((pump = wake), () => (pump = null)),
};
const lane = {
...pan([
{ tick: 0, value: -1 },
{ tick: BAR, value: 1 },
]),
bus: 'song:s:t',
};
const scheduler = new Scheduler(host, () => ({ notes: [], lanes: [lane] }), { tempo: 120 });
scheduler.start();
for (let i = 0; i < 8; i++) {
time += 0.025;
pump!();
}
expect(automated.map((a) => [a.bus, a.target, a.fresh])).toEqual([
['song:s:t', 'pan', true],
...Array.from({ length: automated.length - 1 }, () => ['song:s:t', 'pan', false]),
]);
expect(automated[0]!.marks[0]).toEqual({ at: 10 + LEAD, value: -1, jump: true });
automated.slice(1).forEach((window, i) => {
expect(window.marks[0]!.at).toBeCloseTo(automated[i]!.marks.at(-1)!.at, 12);
expect(window.marks[0]!.jump).toBe(false);
});
// Two seconds a bar: from -1 by 1 a second.
for (const { marks } of automated)
for (const mark of marks) expect(mark.value).toBeCloseTo(-1 + (mark.at - 10 - LEAD), 9);
});
});
+182
View File
@@ -0,0 +1,182 @@
/**
* Where a playing is heard, and the beats, bars and markers it passed (#338): a frame looks now and
* then, so what passed since is told late – and too late, or more than the last, not at all.
*/
import { describe, expect, it } from 'vitest';
import {
heardPosition,
heldQuantize,
lateLimit,
newestPassed,
nextSpot,
passedBar,
passedBeat,
passedMarkers,
quantizeSpots,
type Timeline,
} from '../../src/core/music/beats';
import { COMMON_TIME } from '../../src/core/music/time';
const Q = 480;
const BAR = 4 * Q;
/** 120 BPM in 4/4: a beat is half a second, a bar two. Intro bar 1, then round bar 2. */
const LOOPED: Timeline = {
tempo: 120,
meter: COMMON_TIME,
from: 0,
loop: { start: BAR, end: 2 * BAR },
length: 2 * BAR,
};
const ONCE: Timeline = { ...LOOPED, loop: null };
const beatAt = (t: Timeline, before: number | null, now: number) => {
const beat = passedBeat(t, before, now);
return beat && [beat.bar, beat.beat, +beat.late.toFixed(6)];
};
describe('where a playing is heard', () => {
it('counts bar and beat from 1, the fraction into the beat, the seconds into the song', () => {
expect(heardPosition(LOOPED, 0)).toMatchObject({
bar: 1,
beat: 1,
fraction: 0,
seconds: 0,
playing: true,
});
expect(heardPosition(LOOPED, 1.25)).toMatchObject({ bar: 1, beat: 3, fraction: 0.5, seconds: 1.25 });
});
it('goes back round the loop with the playhead', () => {
// 5.25 s in: the intro took 2, the loop round 2 more, so a quarter second into the second round.
expect(heardPosition(LOOPED, 5.25)).toMatchObject({ bar: 2, beat: 3, fraction: 0.5, seconds: 3.25 });
});
it('stands where it starts before its first note, at its end after it, playing neither time', () => {
expect(heardPosition(LOOPED, null)).toMatchObject({ bar: 1, beat: 1, playing: false });
expect(heardPosition(LOOPED, -0.03)).toMatchObject({ bar: 1, beat: 1, playing: false });
expect(heardPosition(ONCE, 4.5)).toMatchObject({ bar: 3, beat: 1, seconds: 4, playing: false });
});
it('takes the beat from the line of the meter: an eighth in 6/8', () => {
const six: Timeline = { ...ONCE, meter: { beats: 6, unit: 8 } };
// An eighth is a quarter second at 120 quarters a minute; a bar of six is 1.5 s.
expect(heardPosition(six, 1.625)).toMatchObject({ bar: 2, beat: 1, fraction: 0.5 });
expect(lateLimit(six)).toBe(0.125);
expect(lateLimit(LOOPED)).toBe(0.15);
});
});
describe('the beats it passed', () => {
it('tells a beat passed since the last frame, with how late', () => {
expect(beatAt(LOOPED, 0.49, 0.51)).toEqual([1, 2, 0.01]);
expect(beatAt(LOOPED, 0.51, 0.53)).toBeNull();
});
it('tells the very first beat to a playing seen for the first time', () => {
expect(beatAt(LOOPED, null, 0.02)).toEqual([1, 1, 0.02]);
expect(beatAt(LOOPED, -0.01, 0.02)).toEqual([1, 1, 0.02]);
});
it('still tells a beat a slipped frame passed, but not one more than the limit ago', () => {
expect(beatAt(LOOPED, 0.4, 0.6)).toEqual([1, 2, 0.1]);
expect(beatAt(LOOPED, 0.4, 0.7)).toBeNull();
});
it('tells only the last of several passed in one frame, and nothing after a long stall', () => {
expect(beatAt(LOOPED, 0, 1.6)).toEqual([1, 4, 0.1]);
expect(beatAt(LOOPED, 0, 10.3)).toBeNull();
});
it('tells the loop’s start each time round', () => {
// 4 s in, the intro and one round are over: the head is back at bar 2.
expect(beatAt(LOOPED, 3.99, 4.01)).toEqual([2, 1, 0.01]);
expect(passedBar(LOOPED, 5.99, 6.02)).toMatchObject({ bar: 2, beat: 1 });
});
it('counts beats on the ruler’s grid where the loop starts between them', () => {
const offBeat: Timeline = { ...LOOPED, loop: { start: BAR + Q / 2, end: 2 * BAR } };
// The second round starts 4 s in at a quarter past bar 2's first beat: its next beat is 0.25 s later.
expect(beatAt(offBeat, 3.99, 4.01)).toBeNull();
expect(beatAt(offBeat, 4.2, 4.3)).toEqual([2, 2, 0.05]);
});
it('tells nothing at the end of one that plays once, nor after it', () => {
expect(beatAt(ONCE, 3.95, 4.05)).toBeNull();
expect(passedBar(ONCE, 3.95, 4.05)).toBeNull();
expect(beatAt(ONCE, 3.45, 3.55)).toEqual([2, 4, 0.05]);
});
it('tells a bar only on its first beat', () => {
expect(passedBar(LOOPED, 0.49, 0.51)).toBeNull();
expect(passedBar(LOOPED, 1.99, 2.01)).toMatchObject({ bar: 2, beat: 1 });
});
});
describe('the markers it passed', () => {
const markers = [
{ id: 'a', name: 'Verse', tick: BAR },
{ id: 'b', name: 'Also verse', tick: BAR },
{ id: 'c', name: 'Turn', tick: 2 * BAR - 10 },
];
it('tells every marker on the tick passed, each time round the loop', () => {
const passed = passedMarkers(LOOPED, 1.99, 2.01, markers);
expect(passed.map((m) => [m.id, m.name, +m.late.toFixed(6)])).toEqual([
['a', 'Verse', 0.01],
['b', 'Also verse', 0.01],
]);
expect(passedMarkers(LOOPED, 3.99, 4.01, markers).map((m) => m.id)).toEqual(['a', 'b']);
});
it('of markers passed across the loop’s end, keeps the last spot only', () => {
// "Turn" is 10 ticks before the end, heard 3.9896 s in; the loop's start follows at 4 s.
const passed = passedMarkers(LOOPED, 3.98, 4.02, markers);
expect(passed.map((m) => m.id)).toEqual(['a', 'b', 'c']);
expect(newestPassed(passed).map((m) => m.id)).toEqual(['a', 'b']);
expect(newestPassed(passed.filter((m) => m.id === 'c')).map((m) => m.id)).toEqual(['c']);
});
});
describe('where a change to the music lands (#338)', () => {
const markers = [
{ id: 'a', name: 'Verse', tick: BAR + Q },
{ id: 'b', name: 'Late', tick: 2 * BAR - Q },
];
const at = (t: Timeline, elapsed: number, quantize: 'beat' | 'bar' | 'marker', list = markers) =>
nextSpot(t, elapsed, quantizeSpots(quantize, t.meter, list));
it('on the next beat or bar, one it stands on being the one', () => {
expect(at(LOOPED, 0.2, 'beat')).toBeCloseTo(0.5, 9);
expect(at(LOOPED, 0.5, 'beat')).toBeCloseTo(0.5, 9);
expect(at(LOOPED, 0.2, 'bar')).toBeCloseTo(2, 9);
});
it('round the loop: past its last bar the next one is its start come round again', () => {
// Played time 3.9 s is the end of bar 2; the loop's start follows at 4 s.
expect(at(LOOPED, 3.9, 'bar')).toBeCloseTo(4, 9);
expect(at(LOOPED, 5.9, 'bar')).toBeCloseTo(6, 9);
});
it('on the next marker, round the loop too, and on the next bar in a song without markers', () => {
expect(at(LOOPED, 0, 'marker')).toBeCloseTo(2.5, 9);
expect(at(LOOPED, 2.6, 'marker')).toBeCloseTo(3.5, 9);
expect(at(LOOPED, 3.6, 'marker')).toBeCloseTo(4.5, 9);
expect(at(LOOPED, 0.2, 'marker', [])).toBeCloseTo(2, 9);
});
it('from where a playing started, a marker its start lies past is not come to again', () => {
expect(at({ ...LOOPED, from: BAR + 2 * Q }, 0, 'marker')).toBeCloseTo(0.5, 9);
expect(at({ ...ONCE, from: BAR + 2 * Q }, 0, 'marker', [markers[0]!])).toBeNull();
});
it('nowhere at or past the end of one that ends: the change then lands at once', () => {
expect(at(ONCE, 3.2, 'bar')).toBeNull();
expect(at(ONCE, 3.2, 'beat')).toBeCloseTo(3.5, 9);
});
it('takes what it does not know for the next bar', () => {
expect(heldQuantize('now')).toBe('now');
expect(heldQuantize('soon')).toBe('bar');
expect(heldQuantize(undefined)).toBe('bar');
});
});
+1 -1
View File
@@ -104,7 +104,7 @@ describe('the order rule', () => {
const events = BUILT_IN_NODES.filter(
(def) => def.category === 'event' && def.type !== BEHAVIOR_START && def.type !== ON_BEHAVIOR,
);
expect(events).toHaveLength(19);
expect(events).toHaveLength(22);
for (const def of events) {
expect(
def.params.map((p) => p.key),
+230
View File
@@ -0,0 +1,230 @@
/**
* A game follows its music (#338): On beat, On bar and On song marker run as the songs and patterns
* it plays pass them, as heard, and Song position says where one stands.
*/
import { describe, expect, it } from 'vitest';
import { CreateAssetCommand, SetAssetPropertyCommand } from '../../src/core/commands/assetCommands';
import { SetSongMarkersCommand } from '../../src/core/commands/songCommands';
import type { HeardTime } from '../../src/core/music/playbacks';
import { SILENT_AUDIO } from '../../src/runtime/systems/Audio';
import { heardPlayings, positionOf } from '../../src/runtime/systems/musicPulses';
import { graphEditor, makeBehavior, makeSpriteScene, place, startWorld } from '../helpers';
const Q = 480;
const BAR = 4 * Q;
/** A song with two markers on bar 2 and one in its middle, a pattern, and a behavior to listen. */
function listening() {
const f = makeSpriteScene('Box');
const create = (type: 'song' | 'pattern', name: string) => {
const command = new CreateAssetCommand(type, name);
f.stack.execute(command);
return command.asset.id;
};
const song = create('song', 'Theme');
const pattern = create('pattern', 'Riff');
f.stack.execute(new SetAssetPropertyCommand(pattern, 'sign', '3/4'));
f.stack.execute(
new SetSongMarkersCommand(song, [
{ id: 'verse', name: 'Verse', tick: BAR },
{ id: 'also', name: 'Also', tick: BAR },
{ id: 'half', name: 'Half', tick: BAR / 2 },
]),
);
const behavior = makeBehavior(f, 'Logic');
place(f, behavior.id, { x: 0, y: 0 });
const edit = graphEditor(f, behavior);
/** An event that logs `prefix` and two of its outputs, run together. */
const logs = (type: string, params: Record<string, string>, prefix: string, a: string, b: string) => {
const event = edit.addNode(type, params);
const join = edit.addNode('Join');
const first = edit.addNode('Join', { a: prefix });
edit.connect(event, a, first, 'b');
edit.connect(first, 'text', join, 'a');
edit.connect(event, b, join, 'b');
const log = edit.addNode('Log');
edit.connect(event, 'exec', log, 'exec');
edit.connect(join, 'text', log, 'message');
return event;
};
return { ...f, song, pattern, edit, logs };
}
/** A sink whose playings are heard where a test puts them. */
function heardAudio(song: string, pattern: string) {
const times: HeardTime[] = [
{
id: 7,
key: 7,
name: `song:${song}`,
heard: null,
baked: false,
tempo: 120,
from: 0,
loop: { start: BAR, end: 2 * BAR },
length: 2 * BAR,
},
{
id: 8,
key: 8,
name: `pattern:${pattern}`,
heard: null,
baked: false,
tempo: 120,
from: 0,
loop: { start: 0, end: 3 * Q },
length: 3 * Q,
},
{
id: 9,
key: 9,
name: 'composition:c',
heard: null,
baked: false,
tempo: 120,
from: 0,
loop: null,
length: 2 * BAR,
},
];
return {
audio: { ...SILENT_AUDIO, playingTimes: () => times },
at(heard: number | null, ...ids: number[]) {
for (const time of times) if (ids.length === 0 || ids.includes(time.id)) time.heard = heard;
},
};
}
describe('On beat and On bar', () => {
it('run on every beat and bar a song plays, as heard, counting from 1', () => {
const f = listening();
f.logs('OnBeat', { song: f.song }, 'beat ', 'bar', 'beat');
f.logs('OnBar', { song: f.song }, 'bar ', 'bar', 'late');
const { audio, at } = heardAudio(f.song, f.pattern);
const { world, frame } = startWorld(f, { behaviors: true, audio });
frame();
expect(world.log).toEqual([]);
at(0.01, 7);
frame();
expect(world.log).toEqual(['beat 11', 'bar 10.01']);
// A frame the audio clock did not move in: nothing new was heard.
frame();
expect(world.log).toHaveLength(2);
at(0.3, 7);
frame();
at(0.52, 7);
frame();
expect(world.log.slice(2)).toEqual(['beat 12']);
// Round the loop: the intro and one round are over 4 s in, the head is back on bar 2.
at(3.98, 7);
frame();
at(4, 7);
frame();
expect(world.log.slice(3)).toEqual(['beat 21', 'bar 20']);
});
it('tell nothing a stalled frame passed too long ago, and of several only the last', () => {
const f = listening();
f.logs('OnBeat', {}, 'beat ', 'bar', 'beat');
const { audio, at } = heardAudio(f.song, f.pattern);
const { world, frame } = startWorld(f, { behaviors: true, audio });
at(0.6, 7);
frame();
expect(world.log).toEqual(['beat 12']);
at(1.55, 7);
frame();
at(3.3, 7);
frame();
expect(world.log.slice(1)).toEqual(['beat 14']);
});
it('start over with a song a Switch song plays under the same number (#338)', () => {
const f = listening();
f.logs('OnBeat', {}, 'beat ', 'bar', 'beat');
const { audio, at } = heardAudio(f.song, f.pattern);
const { world, frame } = startWorld(f, { behaviors: true, audio });
at(3.01, 7);
frame();
expect(world.log).toEqual(['beat 23']);
// Switched to: the same number on buses of its own, its first beat heard a moment ago.
const switched = audio.playingTimes().find((time) => time.id === 7)!;
switched.key = 70;
at(0.01, 7);
frame();
expect(world.log.slice(1)).toEqual(['beat 11']);
});
it('hear the song or pattern picked, every one where none is, a composition never', () => {
const f = listening();
f.logs('OnBeat', {}, 'any ', 'playback', 'beat');
f.logs('OnBeat', { pattern: f.pattern }, 'pattern ', 'playback', 'beat');
f.logs('OnBeat', { song: f.song }, 'song ', 'playback', 'beat');
const { audio, at } = heardAudio(f.song, f.pattern);
const { world, frame } = startWorld(f, { behaviors: true, audio });
at(0.01);
frame();
expect(world.log).toEqual(['any 71', 'any 81', 'pattern 81', 'song 71']);
});
it('hear only the playing "playback" names where it is connected', () => {
const f = listening();
const beat = f.logs('OnBeat', {}, 'beat ', 'playback', 'beat');
const number = f.edit.addNode('Number', { value: 8 });
f.edit.connect(number, 'value', beat, 'playback');
const { audio, at } = heardAudio(f.song, f.pattern);
const { world, frame } = startWorld(f, { behaviors: true, audio });
at(0.01);
frame();
expect(world.log).toEqual(['beat 81']);
});
});
describe('On song marker', () => {
it('runs for every marker on the tick passed, or for the one picked', () => {
const f = listening();
f.logs('OnSongMarker', { song: f.song }, 'any ', 'name', 'marker');
f.logs('OnSongMarker', { marker: 'also' }, 'picked ', 'name', 'playback');
const { audio, at } = heardAudio(f.song, f.pattern);
const { world, frame } = startWorld(f, { behaviors: true, audio });
at(0.98, 7);
frame();
at(1.01, 7);
frame();
expect(world.log).toEqual(['any Halfhalf']);
at(2.01, 7);
frame();
expect(world.log.slice(1)).toEqual(['any Verseverse', 'any Alsoalso', 'picked Also7']);
});
});
describe('Song position', () => {
it('answers where the playing picked stands, the one started last where several are', () => {
const f = listening();
const { audio, at } = heardAudio(f.song, f.pattern);
at(1.25, 7);
at(0.625, 8);
const playings = heardPlayings(f.project, audio);
// The pattern is in 3/4 and runs round its one bar of 1.5 s.
expect(playings.map((p) => [p.kind, p.timeline.meter.beats])).toEqual([
['song', 4],
['pattern', 3],
]);
expect(positionOf(playings, null, f.song, '')).toMatchObject({
bar: 1,
beat: 3,
fraction: 0.5,
seconds: 1.25,
playing: true,
});
// None picked: the pattern, started after the song.
expect(positionOf(playings, null, '', '')).toMatchObject({ beat: 2, fraction: 0.25, seconds: 0.625 });
expect(positionOf(playings, 7, '', f.pattern)).toMatchObject({ seconds: 1.25 });
expect(positionOf(playings, 3, '', '')).toBeNull();
at(1.75, 8);
expect(positionOf(heardPlayings(f.project, audio), null, '', f.pattern)).toMatchObject({
bar: 1,
beat: 1,
fraction: 0.5,
});
});
});
+53
View File
@@ -2526,6 +2526,56 @@ const CASES: Record<string, Case[]> = {
outputs: { volume: 0.4 },
},
],
SongPosition: [
{
name: 'answers where the playing stands, by its number where "playback" is connected (#338)',
context: {
songPosition: (playback: number | null, song: string, pattern: string) =>
playback === 4 && song === 's' && pattern === ''
? { bar: 3, beat: 2, fraction: 0.25, seconds: 5.125, tick: 0, playing: true }
: null,
},
inputs: { playback: 4, song: 's', pattern: '' },
outputs: { bar: 3, beat: 2, fraction: 0.25, seconds: 5.125, playing: true },
},
{
name: 'asks by the song and pattern alone where "playback" is not connected, all 0 for none',
context: {
songPosition: (playback: number | null) =>
playback === null ? null : { bar: 1, beat: 1, fraction: 0, seconds: 0, tick: 0, playing: true },
},
inputs: { song: 's' },
outputs: { bar: 0, beat: 0, fraction: 0, seconds: 0, playing: false },
},
],
SetSongTrack: [
{
name: 'turns the track of the playing it is given, on the next bar where nothing else is picked (#338)',
inputs: { playback: 4, track: 't1', volume: -12, fade: 1 },
next: 'exec',
after: (ctx) => expect(ctx.songChanges).toEqual(['track 4 t1 -12 dB fade 1 bar']),
},
{
name: 'a quantize it does not know is the next bar',
inputs: { playback: 4, track: 't1', volume: -60, fade: 0, quantize: 'soon' },
next: 'exec',
after: (ctx) => expect(ctx.songChanges).toEqual(['track 4 t1 -60 dB fade 0 bar']),
},
],
SwitchSong: [
{
name: 'switches the playing it is given, in a cut on the next bar by default (#338)',
inputs: { playback: 4 },
next: 'exec',
after: (ctx) => expect(ctx.songChanges).toEqual(['switch 4 to (same) at (start) bar fade 0']),
},
{
name: 'to the song and marker it picks, where its quantize says',
inputs: { playback: 4, song: 'boss', marker: 'm2', quantize: 'marker', fade: 2 },
next: 'exec',
after: (ctx) => expect(ctx.songChanges).toEqual(['switch 4 to boss at m2 marker fade 2']),
},
],
};
/** Two distinct handles that share a name – `Equals` must still tell them apart. */
@@ -2557,6 +2607,9 @@ const EVENT_NODES: Record<string, string> = {
OnMouseWheel: 'clip-areas.test.ts',
OnAnimationFinished: 'animation.test.ts',
OnCompositionFinished: 'composition-play.test.ts',
OnBeat: 'music-events.test.ts',
OnBar: 'music-events.test.ts',
OnSongMarker: 'music-events.test.ts',
OnStateEntered: 'state-machine.test.ts',
OnStateExited: 'state-machine.test.ts',
};
+9 -1
View File
@@ -236,7 +236,15 @@ describe('Play pattern', () => {
expect([asked?.owner, asked?.play.loop]).toEqual(['', null]);
asked!.play.strike({ at: 3, duration: 0.25, note: 60, velocity: 0.5, instrument: f.lead.id, tempo: 90 });
expect(audio.notes).toEqual([
{ name: `instrument:${f.lead.id}`, volume: 0.25, owner: '', duration: 0.25, at: 3, tempo: 90 },
{
name: `instrument:${f.lead.id}`,
volume: 0.25,
owner: '',
duration: 0.25,
at: 3,
tempo: 90,
note: 60,
},
]);
});
+132
View File
@@ -0,0 +1,132 @@
import { describe, expect, it } from 'vitest';
import { CreateAssetCommand, DuplicateAssetCommand } from '../../src/core/commands/assetCommands';
import { SetSongMarkersCommand } from '../../src/core/commands/songCommands';
import type { SongAsset } from '../../src/core/model/Asset';
import {
changedMarker,
heldSongMarkers,
newMarker,
nextMarkerName,
withoutMarker,
} from '../../src/core/music/markers';
import { assetRefLabel, findSongMarker, markerChoices } from '../../src/core/properties/assetRef';
import { makeProject, roundTrip } from '../helpers';
function makeSong() {
const f = makeProject();
const create = new CreateAssetCommand('song', 'Tune');
f.stack.execute(create);
const song = () => f.project.assets.find((a) => a.id === create.asset.id) as SongAsset;
return { ...f, song };
}
describe('song markers (#338)', () => {
it('holds markers to whole ticks from 0, a name and ids of their own, in the order of their ticks', () => {
const held = heldSongMarkers([
{ id: 'b', name: ' Boss ', tick: 3840.4 },
{ id: 'a', name: 'Intro', tick: 0 },
{ id: 'a', name: '', tick: 1920 },
{ id: 'c', name: 'Before', tick: -1 },
{ id: 'd', name: 'Nowhere' },
'no marker',
null,
]);
expect(held.map((m) => [m.name, m.tick])).toEqual([
['Intro', 0],
['Marker 3', 1920],
['Boss', 3840],
]);
expect(new Set(held.map((m) => m.id)).size).toBe(3);
expect(held.find((m) => m.tick === 1920)!.id).not.toBe('a');
expect(heldSongMarkers(undefined)).toEqual([]);
});
it('keeps markers on one tick in the order they came', () => {
const held = heldSongMarkers([
{ id: 'x', name: 'Second', tick: 960 },
{ id: 'y', name: 'First', tick: 0 },
{ id: 'z', name: 'Third', tick: 960 },
]);
expect(held.map((m) => m.name)).toEqual(['First', 'Second', 'Third']);
});
it('names a new marker by the first number not taken', () => {
expect(nextMarkerName([])).toBe('Marker 1');
const markers = heldSongMarkers([
{ name: 'Marker 1', tick: 0 },
{ name: 'Marker 3', tick: 10 },
]);
expect(nextMarkerName(markers)).toBe('Marker 2');
const made = newMarker(markers, 479.6);
expect(made).toMatchObject({ name: 'Marker 2', tick: 480 });
expect(newMarker(markers, -5, ' Drop ')).toMatchObject({ name: 'Drop', tick: 0 });
});
it('changes or takes away one marker and leaves the others', () => {
const markers = heldSongMarkers([
{ id: 'a', name: 'A', tick: 0 },
{ id: 'b', name: 'B', tick: 960 },
]);
expect(changedMarker(markers, 'b', { tick: 1920 })).toEqual([markers[0], { ...markers[1], tick: 1920 }]);
expect(changedMarker(markers, 'a', { name: 'Start' })[0]).toEqual({ id: 'a', name: 'Start', tick: 0 });
expect(withoutMarker(markers, 'a')).toEqual([markers[1]]);
});
it('writes the markers as one step, leaves the key out without one and takes it back on undo', () => {
const f = makeSong();
expect(f.song().markers).toBeUndefined();
const one = [{ id: 'm', name: 'Drop', tick: 1920.2 }];
expect(SetSongMarkersCommand.changes(f.song(), one)).toBe(true);
f.stack.execute(new SetSongMarkersCommand(f.song().id, one, 'Add marker'));
expect(f.song().markers).toEqual([{ id: 'm', name: 'Drop', tick: 1920 }]);
expect(SetSongMarkersCommand.changes(f.song(), one)).toBe(false);
f.stack.execute(new SetSongMarkersCommand(f.song().id, [], 'Delete marker'));
expect('markers' in f.song()).toBe(false);
f.stack.undo();
expect(f.song().markers).toEqual([{ id: 'm', name: 'Drop', tick: 1920 }]);
f.stack.undo();
expect('markers' in f.song()).toBe(false);
});
it('keeps the markers through saving and loading', () => {
const f = makeSong();
const markers = [
{ id: 'a', name: 'Intro', tick: 0 },
{ id: 'b', name: 'Boss', tick: 7680 },
];
f.stack.execute(new SetSongMarkersCommand(f.song().id, markers));
const back = roundTrip(f.project).assets.find((a) => a.id === f.song().id) as SongAsset;
expect(back.markers).toEqual(markers);
const none = makeSong();
const plain = roundTrip(none.project).assets.find((a) => a.id === none.song().id) as SongAsset;
expect('markers' in plain).toBe(false);
});
it('offers the markers of every song to a marker field, each read "Song / Marker"', () => {
const f = makeSong();
f.stack.execute(
new SetSongMarkersCommand(f.song().id, [
{ id: 'm2', name: 'Boss', tick: 1920 },
{ id: 'm1', name: 'Intro', tick: 0 },
]),
);
expect(markerChoices(f.project)).toEqual([
{ id: 'm1', label: 'Tune / Intro' },
{ id: 'm2', label: 'Tune / Boss' },
]);
expect(assetRefLabel(f.project, 'm2')).toBe('Tune / Boss');
expect(findSongMarker(f.project, 'm1')?.song.id).toBe(f.song().id);
expect(assetRefLabel(f.project, 'gone')).toBe('gone?');
});
it('gives the markers of a copy of a song ids of their own, so a node names one of them', () => {
const f = makeSong();
f.stack.execute(new SetSongMarkersCommand(f.song().id, [{ id: 'm1', name: 'Intro', tick: 0 }]));
const duplicate = new DuplicateAssetCommand(f.song().id);
f.stack.execute(duplicate);
const copy = f.project.assets.find((a) => a.id === duplicate.copyId) as SongAsset;
expect(copy.markers?.map((m) => m.name)).toEqual(['Intro']);
expect(copy.markers?.[0]?.id).not.toBe('m1');
expect(findSongMarker(f.project, 'm1')?.song.id).toBe(f.song().id);
});
});
+283 -6
View File
@@ -9,12 +9,19 @@ import {
SetAssetPropertyCommand,
} from '../../src/core/commands/assetCommands';
import { SetLoopCommand, SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
import { SetSongCommand } from '../../src/core/commands/songCommands';
import { SetSongCommand, SetSongMarkersCommand } from '../../src/core/commands/songCommands';
import type { InstrumentAsset, SongAsset } from '../../src/core/model/Asset';
import { bakePlayback, bakePrint, bakeSpans } from '../../src/core/music/bake';
import { goesBy, Playbacks, type SequencePlay } from '../../src/core/music/playbacks';
import {
CUT,
goesBy,
type LandingAt,
type PlayingClock,
Playbacks,
type SequencePlay,
} from '../../src/core/music/playbacks';
import { INTERVAL, LEAD, type ScheduledClip, type ScheduledNote } from '../../src/core/music/Scheduler';
import { newItem, newTrack, trackBus } from '../../src/core/music/song';
import { dbToGain, newItem, newTrack, trackBus } from '../../src/core/music/song';
import { type AudioSink, type ClipTiming, SILENT_AUDIO } from '../../src/runtime/systems/Audio';
import { graphEditor, makeBehavior, makeSpriteScene, place, startWorld } from '../helpers';
@@ -23,9 +30,10 @@ const Q = 480;
const BAR = 4 * Q;
/** An audio clock moved by hand, its timers run as it passes them. */
function fakeHost() {
function fakeHost(baked = false) {
const host = {
time: 0,
latency: 0,
sounding: new Set<number>(),
pumps: new Set<() => void>(),
loading: [] as (() => void)[],
@@ -35,7 +43,7 @@ function fakeHost() {
};
const playbacks = new Playbacks({
now: () => host.time,
latency: () => 0,
latency: () => host.latency,
playing: (voice) => host.sounding.has(voice),
stop: (voices) => {
for (const voice of voices) host.sounding.delete(voice);
@@ -48,6 +56,7 @@ function fakeHost() {
level: (id, gain, at, seconds) => host.levels.push({ id, gain, at, seconds }),
after: (at, then) => host.timers.push({ at, then }),
closed: (id) => host.closed.push(id),
...(baked && { playBaked: () => 77 }),
});
const run = (seconds: number) => {
const end = host.time + seconds;
@@ -113,6 +122,65 @@ describe('the songs a game plays', () => {
expect(playbacks.current()[0]?.clips[0]).toMatchObject({ sound: 'drum', bus: 'track' });
});
it('tells how far into it is heard: from its first note, the output latency taken off (#338)', async () => {
const { host, playbacks, loaded } = fakeHost();
host.latency = 0.02;
playbacks.start(5, 'song:a', song([]), '');
expect(playbacks.times()).toEqual([
{
id: 5,
key: 5,
name: 'song:a',
heard: null,
baked: false,
tempo: 120,
from: 0,
loop: { start: BAR, end: 2 * BAR },
length: 2 * BAR,
},
]);
host.time = 1;
await loaded();
// Its first note is heard LEAD after it starts, and the speakers are 0.02 s behind the clock.
host.time = 3;
expect(playbacks.times()[0]!.heard).toBeCloseTo(3 - 0.02 - (1 + LEAD), 9);
});
it('asks no clock while nothing plays: that would make the audio context before a gesture (#338)', () => {
let asked = 0;
const playbacks = new Playbacks({
now: () => (asked++, 0),
latency: () => 0,
playing: () => false,
stop: () => {},
wake: () => () => {},
prepare: () => Promise.resolve(),
level: () => {},
after: () => {},
});
expect(playbacks.times()).toEqual([]);
expect(asked).toBe(0);
});
it('tells how far a baked one is heard the same way, from where its bake is heard (#338)', async () => {
const { host, playbacks, loaded } = fakeHost(true);
const bake = { file: 'b.ogg', gain: 1, playback: { end: null, until: 0, round: null } };
playbacks.start(5, 'song:a', song([], { bake }), '');
host.time = 1;
await loaded();
host.time = 2.5;
expect(playbacks.current()[0]?.baked).toBe(true);
expect(playbacks.times()[0]!.heard).toBeCloseTo(2.5 - (1 + LEAD), 9);
});
it('starts where it was asked to, as far further in as it waited on its files (#418)', async () => {
const { host, playbacks, loaded } = fakeHost();
playbacks.start(5, 'composition:c', song([], { from: 960 }), '');
host.time = 0.5;
await loaded();
expect(playbacks.times()[0]!.from).toBe(960 + 480);
});
it('rises from silence to its volume over its fade in, from its first note on', async () => {
const { host, playbacks, loaded } = fakeHost();
playbacks.start(5, 'song:a', song([], { volume: 0.5, fadeIn: 3 }), '');
@@ -193,6 +261,93 @@ describe('the songs a game plays', () => {
});
});
describe('a song switched to in place of another (#338)', () => {
/** Song a playing from 0.05 s on, as heard, and a clock it was asked for at 1 s. */
async function playingA() {
const fake = fakeHost();
const old: (ScheduledNote | ScheduledClip)[] = [];
fake.playbacks.start(5, 'song:a', song(old), '');
await fake.loaded();
fake.run(1);
return { ...fake, old };
}
const levelsOf = (host: ReturnType<typeof fakeHost>['host']) =>
host.levels.map(({ id, gain, at, seconds }) => [id, gain, +at.toFixed(6), seconds]);
it('is heard from where it lands, under the same number; the old one falls silent there in a cut and goes', async () => {
const { host, playbacks, run, loaded } = await playingA();
const fresh: (ScheduledNote | ScheduledClip)[] = [];
const asked: PlayingClock[] = [];
const twoBarsIn: LandingAt = (clock) => (asked.push(clock), clock.startedAt! + 2);
playbacks.switchTo(5, 9, 'song:b', song(fresh, { from: BAR }), '', twoBarsIn, 0);
await loaded();
expect(asked.map((c) => [c.startedAt, +c.now.toFixed(9), +c.soonest.toFixed(9)])).toEqual([
[LEAD, 1, 1 + LEAD],
]);
expect(levelsOf(host).slice(-2)).toEqual([
[5, 0, 2 + LEAD, CUT],
[9, 1, 2 + LEAD, 0],
]);
// Until the clock gets there the old one is the one heard, from there on the new one.
expect(playbacks.times().map((t) => [t.id, t.key])).toEqual([[5, 5]]);
run(1.1);
expect(playbacks.times().map((t) => [t.id, t.key, t.from, +t.heard!.toFixed(6)])).toEqual([
[5, 9, BAR, +(host.time - 2 - LEAD).toFixed(6)],
]);
expect([playbacks.has(5), host.closed]).toEqual([true, [5]]);
const [first] = fresh as ScheduledNote[];
expect([first?.note, first?.at]).toEqual([61, 2 + LEAD]);
});
it('over its fade the old one falls as the new one rises, and goes once it is silent', async () => {
const { host, playbacks, run, loaded } = await playingA();
playbacks.switchTo(5, 9, 'song:a', song([]), '', (clock) => clock.startedAt! + 2, 1.5);
await loaded();
expect(levelsOf(host).slice(-2)).toEqual([
[5, 0, 2 + LEAD, 1.5],
[9, 1, 2 + LEAD, 1.5],
]);
// While the old one fades, only the new one is told: beats and the position go by it.
run(1.2);
expect(playbacks.times().map((t) => [t.id, t.key])).toEqual([[5, 9]]);
run(1.2);
expect(host.closed).toEqual([]);
run(0.3);
expect(host.closed).toEqual([5]);
});
it('asked again before it lands, drops the one waiting, and the old one is heard on at its level', async () => {
const { host, playbacks, loaded } = await playingA();
playbacks.switchTo(5, 9, 'song:b', song([]), '', (clock) => clock.startedAt! + 2, 0);
await loaded();
playbacks.switchTo(5, 10, 'song:c', song([]), '', (clock) => clock.startedAt! + 4, 0);
expect(host.closed).toEqual([9]);
expect(levelsOf(host).at(-1)).toEqual([5, 1, 1, 0]);
await loaded();
expect(levelsOf(host).slice(-2)).toEqual([
[5, 0, 4 + LEAD, CUT],
[10, 1, 4 + LEAD, 0],
]);
expect(playbacks.named(5).map((p) => p.key)).toEqual([5, 10]);
});
it('a stop of its number stops the old one and the one waiting alike', async () => {
const { host, playbacks, loaded } = await playingA();
playbacks.switchTo(5, 9, 'song:b', song([]), '', (clock) => clock.startedAt! + 2, 0);
await loaded();
playbacks.stopId(5);
expect([playbacks.has(5), host.closed.sort()]).toEqual([false, [5, 9]]);
});
it('where nothing of that number plays any more, starts as soon as it can', async () => {
const { host, playbacks, loaded } = fakeHost();
let asked = 0;
playbacks.switchTo(5, 9, 'song:b', song([]), '', () => (asked++, 99), 0);
await loaded();
expect([asked, levelsOf(host)]).toEqual([0, [[9, 1, LEAD, 0]]]);
});
});
/**
* A scene whose behavior plays a song on start: on its first track a two-note pattern at bar 1, on
* its second a sound at bar 2, the second track at -6 dB and panned left, the third soloed.
@@ -260,6 +415,38 @@ function recordingAudio() {
playClip: (file: string, clip: ClipTiming) => (sink.clips.push({ file, clip }), 2),
stopSequence: (name: string, _letRing = false, fade = 0) => sink.stopped.push(`${name} (fade ${fade})`),
stopPlayback: (id: number, fade: number) => sink.stopped.push(`playback ${id} (fade ${fade})`),
/** Every playing heard 0.3 s into a clock that started it at 10 s (#338). */
clockOf: (play: SequencePlay): PlayingClock => ({
time: { tempo: play.tempo, from: 0, loop: play.loop, length: play.length },
startedAt: 10,
now: 10.3,
soonest: 10.31,
}),
playingTimes: () =>
sink.sequences.map(({ name, play }, i) => ({
id: 1001 + i,
key: 1001 + i,
name,
heard: 0.3,
baked: play.bake !== undefined,
tempo: play.tempo,
from: 0,
loop: play.loop,
length: play.length,
})),
switched: [] as { id: number; name: string; play: SequencePlay; at: number; fade: number }[],
switchSequence: (id: number, name: string, play: SequencePlay, at: LandingAt, fade: number) => {
const from = sink.sequences[id - 1001]!.play;
sink.switched.push({ id, name, play, at: at(sink.clockOf(from)), fade });
return true;
},
turned: [] as { id: number; track: string; gain: number; fade: number; at: number }[],
setSongTrack: (id: number, track: string, gain: number, fade: number, at: LandingAt) => {
const playing = sink.sequences[id - 1001]!.play;
if (playing.bake) return 'baked' as const;
sink.turned.push({ id, track, gain, fade, at: at(sink.clockOf(playing)) });
return 'set' as const;
},
};
return sink;
}
@@ -341,7 +528,7 @@ describe('Play song', () => {
tempo: 90,
});
expect(audio.notes).toEqual([
{ name: `instrument:${f.lead.id}`, volume: 1, owner: '', at: 3, tempo: 90, bus },
{ name: `instrument:${f.lead.id}`, volume: 1, owner: '', at: 3, tempo: 90, bus, note: 60 },
]);
expect(audio.clips).toEqual([
{ file: FILE, clip: { at: 4, offset: 0.5, duration: 2, volume: 0.5, bus } },
@@ -368,3 +555,93 @@ describe('Stop song', () => {
]);
});
});
describe('Set song track and Switch song (#338)', () => {
/** 90 BPM: a beat is 2/3 s, a bar 8/3 s, played from 10 s on and heard 0.3 s in. */
const BEAT = 2 / 3;
it('turns a track of the playing on the next beat, in decibels on top of its level', () => {
const f = playingSong();
const turn = f.edit.addNode('SetSongTrack', {
track: f.tracks[1]!,
volume: -6,
fade: 0.5,
quantize: 'beat',
});
f.edit.connect(f.play, 'exec', turn, 'exec');
f.edit.connect(f.play, 'playback', turn, 'playback');
const audio = recordingAudio();
startWorld(f, { behaviors: true, audio }).frame();
expect(audio.turned.map((t) => ({ ...t, at: +t.at.toFixed(9) }))).toEqual([
{
id: 1001,
track: trackBus(f.songId, f.tracks[1]!),
gain: dbToGain(-6),
fade: 0.5,
at: +(10 + BEAT).toFixed(9),
},
]);
});
it('a playing of its bake has no tracks: the log says to switch on "live tracks", which plays its notes', () => {
const baked = (live: boolean) => {
const f = playingSong({ live });
const song = f.project.assets.find((a) => a.id === f.songId) as SongAsset;
const spans = bakeSpans({ tempo: 90, length: 2 * BAR, loop: { start: BAR, end: 2 * BAR } });
const file = `${'c3'.repeat(6)}/${'d4'.repeat(32)}.ogg`;
song.bake = { file, print: bakePrint(f.project, song), peak: 0.5, bytes: 1, ...spans };
const turn = f.edit.addNode('SetSongTrack', { track: f.tracks[0]!, volume: -60, quantize: 'now' });
f.edit.connect(f.play, 'exec', turn, 'exec');
f.edit.connect(f.play, 'playback', turn, 'playback');
const audio = recordingAudio();
const { world, frame } = startWorld(f, { behaviors: true, audio });
frame();
return { audio, log: world.log };
};
const off = baked(false);
expect([off.audio.turned, off.log]).toEqual([
[],
['Set song track: song "Theme" plays its bake; switch on "live tracks" in Play song'],
]);
const on = baked(true);
expect([on.audio.sequences[0]?.play.bake, on.audio.turned[0], on.log]).toEqual([
undefined,
expect.objectContaining({ gain: 0, at: 10.31 }),
[],
]);
});
it('switches to a marker of the same song on the next marker, asked for as its Play song was', () => {
const f = playingSong();
f.stack.execute(new SetSongMarkersCommand(f.songId, [{ id: 'm1', name: 'Chorus', tick: BAR + Q }]));
const go = f.edit.addNode('SwitchSong', { marker: 'm1', quantize: 'marker', fade: 1.5 });
f.edit.connect(f.play, 'exec', go, 'exec');
f.edit.connect(f.play, 'playback', go, 'playback');
const audio = recordingAudio();
startWorld(f, { behaviors: true, audio }).frame();
const [switched] = audio.switched;
// Bar 2 beat 2 is 2400 ticks in, 3 1/3 s at 90 BPM.
expect([switched?.id, switched?.name, switched?.fade, switched?.at]).toEqual([
1001,
`song:${f.songId}`,
1.5,
expect.closeTo(10 + 10 / 3, 9),
]);
expect(switched?.play).toMatchObject({ from: BAR + Q, volume: 0.8, loop: { start: BAR, end: 2 * BAR } });
expect(switched?.play.bake).toBeUndefined();
});
it('logs a marker the song does not have, and switches nothing', () => {
const f = playingSong();
const other = new CreateAssetCommand('song', 'Boss');
f.stack.execute(other);
f.stack.execute(new SetSongMarkersCommand(other.asset.id, [{ id: 'm9', name: 'Rage', tick: 0 }]));
const go = f.edit.addNode('SwitchSong', { marker: 'm9' });
f.edit.connect(f.play, 'exec', go, 'exec');
f.edit.connect(f.play, 'playback', go, 'playback');
const audio = recordingAudio();
const { world, frame } = startWorld(f, { behaviors: true, audio });
frame();
expect([audio.switched, world.log]).toEqual([[], ['Switch song: song "Theme" has no marker "Rage"']]);
});
});
+172 -2
View File
@@ -1,12 +1,19 @@
import { afterEach, describe, expect, it } from 'vitest';
import { CreateSoundCommand } from '../../src/core/commands/assetCommands';
import type { CompositionAsset, PatternAsset, SongAsset, SoundAsset } from '../../src/core/model/Asset';
import type {
BehaviorAsset,
CompositionAsset,
PatternAsset,
SongAsset,
SoundAsset,
SoundFxAsset,
} from '../../src/core/model/Asset';
import { setSoundMeasure } from '../../src/core/model/composition';
import { ROUTING_ID } from '../../src/core/model/Project';
import { settleRouting } from '../../src/core/music/projectRouting';
import { TO_COMPOSITION_BUS } from '../../src/core/properties/buses';
import { runTool } from '../../src/editor/agent/agentTools';
import { makeSpriteScene, type SceneFixture } from '../helpers';
import { graphEditor, makeSpriteScene, type SceneFixture } from '../helpers';
import { hostFor, text } from './agentTestHost';
/** The commands of #429 stage 5d: patterns, songs, sends, clips of sounds and compositions. */
@@ -217,6 +224,169 @@ describe('songs', () => {
expect(noFx.text).toContain('is no Sound FX asset');
});
it('adds, renames, moves and removes markers of a song (#338), and get_asset lists them', async () => {
const fixture = makeSpriteScene();
const { Theme } = await made(fixture, { Theme: 'song' });
const song = () => byId<SongAsset>(fixture, Theme);
const done = await run(fixture, [
{ command: 'add_song_marker', args: { asset: Theme, tick: 7680, name: 'Boss' }, as: 'boss' },
{ command: 'add_song_marker', args: { asset: Theme, tick: 0 } },
{ command: 'change_song_marker', args: { asset: Theme, marker: '$boss', tick: 3840 } },
]);
expect(done.result.isError).toBeUndefined();
expect(song().markers!.map(({ name, tick }) => [name, tick])).toEqual([
['Marker 1', 0],
['Boss', 3840],
]);
const listed = (await read(fixture, Theme)).markers as { id: string; name: string; tick: number }[];
expect(listed.map((m) => m.name)).toEqual(['Marker 1', 'Boss']);
const boss = song().markers![1]!.id;
const blank = await run(fixture, [
{ command: 'change_song_marker', args: { asset: Theme, marker: boss, name: ' ' } },
]);
expect(blank.text).toContain('A marker needs a name');
const renamed = await run(fixture, [
{ command: 'change_song_marker', args: { asset: Theme, marker: boss, name: 'Fight' } },
]);
expect(renamed.result.isError).toBeUndefined();
expect(song().markers![1]).toEqual({ id: boss, name: 'Fight', tick: 3840 });
const gone = await run(fixture, [
{ command: 'remove_song_marker', args: { asset: Theme, marker: boss } },
]);
expect(gone.result.isError).toBeUndefined();
expect(song().markers!.map((m) => m.name)).toEqual(['Marker 1']);
const missing = await run(fixture, [
{ command: 'remove_song_marker', args: { asset: Theme, marker: boss } },
]);
expect(missing.text).toContain(`Theme has no marker ${boss}`);
});
it('sets the volume lane of a track in tick order with its easings, refuses what is out of range, and clears it with []', async () => {
const fixture = makeSpriteScene();
const { Theme, Bend } = await made(fixture, { Theme: 'song', Bend: 'curve' });
const song = () => byId<SongAsset>(fixture, Theme);
const track = song().tracks[0]!.id;
const set = await run(
fixture,
[
{
command: 'set_song_lane',
args: {
asset: Theme,
track,
target: 'volume',
points: [
{ tick: 1920, value: -24, ease: 'step' },
{ tick: 0, value: -6, ease: 'sine-in' },
{ tick: 1920, value: 0, ease: `curve:${Bend}` },
{ tick: 3840, value: -12, ease: 'linear' },
],
},
},
],
true,
);
expect(set.result.isError).toBeUndefined();
expect(set.shown).toEqual([{ kind: 'asset', asset: Theme, open: true }]);
// In tick order, two on one tick as given (a jump); linear is stored as no easing.
expect(song().tracks[0]!.automation).toEqual([
{
target: 'volume',
points: [
{ tick: 0, value: -6, ease: 'sine-in' },
{ tick: 1920, value: -24, ease: 'step' },
{ tick: 1920, value: 0, ease: `curve:${Bend}` },
{ tick: 3840, value: -12 },
],
},
]);
const lanes = (await read(fixture, Theme)).tracks as { automation?: unknown[] }[];
expect(lanes[0]!.automation).toHaveLength(1);
const point = (value: number, ease?: string) => [{ tick: 0, value, ...(ease && { ease }) }];
const refused = async (target: string, points: object[], extra: object = {}) =>
(
await run(fixture, [
{ command: 'set_song_lane', args: { asset: Theme, track, target, points, ...extra } },
])
).text;
expect(await refused('volume', point(13))).toContain('value at tick 0 has to be from -60 to 12');
expect(await refused('pan', point(-2))).toContain('from -1 to 1');
expect(await refused('volume', point(0, 'wobbly'))).toContain('"wobbly" is no easing');
expect(await refused('volume', point(0, 'curve:nope'))).toContain('nope is no Curve asset');
expect(await refused('loudness', point(0))).toContain('is no lane');
expect(await refused('volume', point(0), { min: 0, max: 1 })).toContain('Only a parameter lane');
expect(song().tracks[0]!.automation![0]!.points).toHaveLength(4);
const nothing = await run(fixture, [
{ command: 'set_song_lane', args: { asset: Theme, track, target: 'pan' } },
]);
expect(nothing.text).toContain('Give the points');
const cleared = await run(fixture, [
{ command: 'set_song_lane', args: { asset: Theme, track, target: 'volume', points: [] } },
]);
expect(cleared.result.isError).toBeUndefined();
expect(song().tracks[0]!.automation).toBeUndefined();
});
it('adds a parameter lane only for a parameter on the track, round its default, and get_asset lists what a lane can move', async () => {
const fixture = makeSpriteScene();
const { Theme, Filter } = await made(fixture, { Theme: 'song', Filter: 'soundfx' });
const fx = byId<SoundFxAsset>(fixture, Filter);
const edit = graphEditor(fixture, fx as unknown as BehaviorAsset);
const input = edit.addNode('SoundFxInput');
const gain = edit.addNode('SoundFxGain');
const output = edit.addNode('SoundFxOutput');
input.ports = [{ key: 'cut', kind: 'number', default: 800 }];
edit.connect(input, 'out', gain, 'in');
edit.connect(input, 'cut', gain, 'gain');
edit.connect(gain, 'out', output, 'in');
const song = () => byId<SongAsset>(fixture, Theme);
const track = song().tracks[0]!.id;
const lane = (args: object) => ({
command: 'set_song_lane',
args: { asset: Theme, track, target: 'param:cut', ...args },
});
const none = await run(fixture, [lane({})]);
expect(none.text).toContain('Nothing on Track 1 has a parameter to move');
expect(((await read(fixture, Theme)).tracks as object[])[0]).not.toHaveProperty('params');
await run(fixture, [{ command: 'change_song_track', args: { asset: Theme, track, fx: Filter } }]);
expect(((await read(fixture, Theme)).tracks as { params?: object }[])[0]!.params).toEqual([
{ name: 'cut', default: 800 },
]);
const other = await run(fixture, [{ ...lane({}), args: { asset: Theme, track, target: 'param:tone' } }]);
expect(other.text).toContain('Nothing on Track 1 has a parameter tone – it has cut');
// Without a range it gets one round the default, and keeps it without a point.
const added = await run(fixture, [lane({})]);
expect(added.result.isError).toBeUndefined();
expect(song().tracks[0]!.automation).toEqual([{ target: 'param:cut', points: [], min: 0, max: 2000 }]);
const pointed = await run(fixture, [lane({ points: [{ tick: 0, value: 1500 }] })]);
expect(pointed.result.isError).toBeUndefined();
const over = await run(fixture, [lane({ points: [{ tick: 0, value: 2500 }] })]);
expect(over.text).toContain('from 0 to 2000');
// A narrower range holds the points it had.
await run(fixture, [lane({ min: 100, max: 1000 })]);
expect(song().tracks[0]!.automation).toEqual([
{ target: 'param:cut', points: [{ tick: 0, value: 1000 }], min: 100, max: 1000 },
]);
expect((await run(fixture, [lane({ min: 5 })])).text).toContain('Give min and max');
expect((await run(fixture, [lane({ min: 5, max: 5 })])).text).toContain('max has to be above min');
// A lane there is stays settable when its parameter is gone from the track.
await run(fixture, [{ command: 'change_song_track', args: { asset: Theme, track, fx: '' } }]);
expect((await run(fixture, [lane({ min: 0, max: 10 })])).result.isError).toBeUndefined();
expect((await run(fixture, [lane({ remove: true, min: 0 })])).text).toContain('remove takes no points');
const removed = await run(fixture, [lane({ remove: true })]);
expect(removed.result.isError).toBeUndefined();
expect(song().tracks[0]!.automation).toBeUndefined();
expect((await run(fixture, [lane({ remove: true })])).text).toContain('Track 1 has no param:cut lane');
});
it('sets the send of a song link and of a master routing link, where a sound only sends post', async () => {
const fixture = makeSpriteScene();
const { Theme } = await made(fixture, { Theme: 'song' });
+92
View File
@@ -7,6 +7,8 @@ import type { AgentView } from '../../src/editor/agent/agentView';
import { frameWithin } from '../../src/core/time';
import { type PlayDriver, pointOf, sendInput } from '../../src/editor/agent/playInput';
import { cameraHint, NO_CAMERA_HINT } from '../../src/editor/agent/runtimeView';
import type { HeardSound } from '../../src/viewport/SoundLibrary';
import { STALLED_HINT } from '../../src/editor/agent/audioView';
import {
graphEditor,
makeBehavior,
@@ -352,6 +354,96 @@ describe('get_runtime', () => {
);
expect(text(await runTool('get_runtime', {}, hostFor(f, started, driver, false)))).toMatch(/not running/);
});
it('says what is heard: levels in dB by bus name, voices and playings by asset, the rest left out (#436)', async () => {
const { f, started, driver } = game();
const made = (type: 'sound' | 'instrument' | 'song' | 'bus', name: string) => {
const command = new CreateAssetCommand(type, name);
f.stack.execute(command);
return command.asset;
};
const sound = made('sound', 'Jump');
if (sound.type === 'sound') sound.file = 'jump.wav';
const instrument = made('instrument', 'Lead');
const song = made('song', 'Theme');
const track = song.type === 'song' ? song.tracks[0]! : null;
const bus = made('bus', 'Music');
const owner = started.world.entities.find((entity) => entity.type === 'behavior')!.id;
const trackBus = `play:7:song:${song.id}:${track!.id}`;
const quiet = { released: false, ringing: false, startsIn: 0 };
const level = (peak: number, rms: number): Pick<HeardSound['buses'][number], 'peak' | 'rms'> => ({
peak: [peak, peak / 2],
rms: [rms, rms],
});
const heard: HeardSound = {
window: 32768 / 48000,
clock: { state: 'running', time: 12.34567, held: false },
voices: [
{ id: 1, file: 'jump.wav', owner, bus: '', ...quiet },
{
id: 2,
file: `instrument:${instrument.id}`,
owner: '',
bus: trackBus,
note: 64,
...quiet,
released: true,
},
{
id: 3,
file: `instrument:${instrument.id}`,
owner: '',
bus: trackBus,
note: 65,
...quiet,
startsIn: 0.2,
},
],
waiting: [{ id: 4, file: 'boom.wav', owner: '', bus: `mix:${bus.id}` }],
failed: ['boom.wav'],
playings: [{ id: 7, name: `song:${song.id}`, owner, playing: true, fading: null, baked: false }],
buses: [
{ name: trackBus, ...level(0.5, 0.1) },
{ name: `${trackBus}:#items`, ...level(0.5, 0.1) },
{ name: 'mix', ...level(1, 0.01) },
{ name: `mix:${bus.id}`, ...level(0, 0) },
{ name: 'play:7', ...level(0.5, 0.1) },
],
};
const host = hostFor(f, started, driver);
const facts = host.runtime!()!;
host.runtime = () => ({ ...facts, audio: heard });
const view = (await runTool('get_runtime', {}, host)).structuredContent as { audio: object };
expect(view.audio).toEqual({
clock: { state: 'running', time: 12.346 },
window: 0.68,
levels: [
{ bus: 'master', peak: 0, rms: -40 },
{ bus: `song Theme, track ${track!.name}`, peak: -6, rms: -20 },
{ bus: 'Music', silent: true },
{ bus: 'song Theme', peak: -6, rms: -20 },
],
voices: [
{ sound: 'Jump', bus: 'master', by: 'Box' },
{ instrument: 'Lead', note: 'E4', bus: `song Theme, track ${track!.name}`, released: true },
],
planned: 1,
playing: [{ song: 'Theme', by: 'Box' }],
waiting: [{ file: 'boom.wav', bus: 'Music' }],
failed: [{ file: 'boom.wav' }],
});
// A clock the browser holds while the game runs means nothing is heard, paused since or not.
const audioWith = async (clock: HeardSound['clock']) => {
host.runtime = () => ({ ...facts, audio: { ...heard, clock } });
return ((await runTool('get_runtime', {}, host)).structuredContent as { audio: { hint?: string } })
.audio;
};
expect(await audioWith({ state: 'running', time: 3, held: true })).not.toHaveProperty('hint');
expect((await audioWith({ state: 'suspended', time: 0, held: false })).hint).toBe(STALLED_HINT);
expect((await audioWith({ state: 'suspended', time: 0, held: true })).hint).toBe(STALLED_HINT);
});
});
describe('the camera', () => {
+7 -7
View File
@@ -41,7 +41,6 @@ import { MAX_PX_PER_TICK, tickToX, zoomTime } from '../../src/editor/music/timeV
const BAR = 1920;
const TRACKS = ['t1', 't2', 't3'];
const rows = new Map(TRACKS.map((id, i) => [id, i]));
const tracks = TRACKS.map((id) => ({ id, name: id, volume: 0, pan: 0, mute: false, solo: false, fx: '' }));
function item(
id: string,
@@ -175,14 +174,14 @@ describe('where things are in the Song tab (#334)', () => {
const items = [item('a', 't1', 0, BAR), item('b', 't2', BAR, BAR)];
it('finds an item and which end of it is under the pointer', () => {
expect(itemAt(view, items, tracks, { x: 96, y: TRACK_HEIGHT / 2 })).toEqual({ id: 'a', part: 'body' });
expect(itemAt(view, items, tracks, { x: 190, y: TRACK_HEIGHT / 2 })).toEqual({ id: 'a', part: 'end' });
expect(itemAt(view, items, tracks, { x: 194, y: TRACK_HEIGHT * 1.5 })).toEqual({
expect(itemAt(view, items, rows, { x: 96, y: TRACK_HEIGHT / 2 })).toEqual({ id: 'a', part: 'body' });
expect(itemAt(view, items, rows, { x: 190, y: TRACK_HEIGHT / 2 })).toEqual({ id: 'a', part: 'end' });
expect(itemAt(view, items, rows, { x: 194, y: TRACK_HEIGHT * 1.5 })).toEqual({
id: 'b',
part: 'start',
});
expect(itemAt(view, items, tracks, { x: 96, y: TRACK_HEIGHT * 1.5 })).toBeNull();
expect(itemsWithin(view, items, tracks, { x: 150, y: 10 }, { x: 250, y: TRACK_HEIGHT + 10 })).toEqual([
expect(itemAt(view, items, rows, { x: 96, y: TRACK_HEIGHT * 1.5 })).toBeNull();
expect(itemsWithin(view, items, rows, { x: 150, y: 10 }, { x: 250, y: TRACK_HEIGHT + 10 })).toEqual([
'a',
'b',
]);
@@ -202,7 +201,8 @@ describe('where things are in the Song tab (#334)', () => {
expect(headPartAt(solo.x + 4, solo.y + 4)).toBe('solo');
expect(headPartAt(volume.x + 10, volume.y + 4)).toBe('volume');
expect(headPartAt(pan.x + 10, pan.y + 4)).toBe('pan');
expect(headPartAt(20, 8)).toBe('name');
expect(headPartAt(30, 8)).toBe('name');
expect(headPartAt(10, 12)).toBe('lanes');
expect(sliderValue(volume.x, volume, -60, 6, 0.5)).toBe(-60);
expect(sliderValue(volume.x + volume.w, volume, -60, 6, 0.5)).toBe(6);
expect(sliderValue(pan.x + pan.w / 2, pan, -1, 1, 0.05)).toBe(0);
+276
View File
@@ -0,0 +1,276 @@
import { describe, expect, it, vi } from 'vitest';
import type { AutomationPoint } from '../../src/core/music/automation';
import type { SongTrack } from '../../src/core/music/song';
import type { Project } from '../../src/core/model/Project';
import type { MenuEntry, MenuItem, MenuSubmenu } from '../../src/ui';
import { easingEntries } from '../../src/editor/song/easingMenu';
import {
addedPoint,
draggedPoint,
easedPoint,
lanePoints,
withLanePoints,
withoutLane,
withoutPoint,
withParamLane,
} from '../../src/editor/song/laneEdit';
import {
LANE_INSET,
laneKindOf,
laneLabel,
laneStep,
laneValue,
laneValueText,
laneWay,
laneY,
pointAt,
shownLanes,
songRows,
trackLanes,
trackRows,
} from '../../src/editor/song/laneView';
import { trackAtY, type SongView } from '../../src/editor/song/songView';
function track(id: string, automation?: SongTrack['automation']): SongTrack {
return {
id,
name: id,
volume: -6,
pan: 0,
mute: false,
solo: false,
fx: '',
...(automation && { automation }),
};
}
const VOLUME: AutomationPoint[] = [
{ tick: 0, value: -12 },
{ tick: 960, value: 0, ease: 'step' },
{ tick: 1920, value: -24 },
];
describe('the rows of the Song tab with automation lanes (#337)', () => {
const tracks = [track('a'), track('b', [{ target: 'pan', points: [{ tick: 0, value: 0.5 }] }]), track('c')];
it('shows the lanes with points until a track is opened or closed by hand', () => {
expect(shownLanes(tracks[1]!, new Map())).toEqual(['pan']);
expect(shownLanes(tracks[0]!, new Map())).toEqual([]);
// Opened by hand the lanes come in their own order, closed by hand even one with points goes.
expect(shownLanes(tracks[0]!, new Map([['a', ['pan', 'volume'] as const]]))).toEqual(['volume', 'pan']);
expect(shownLanes(tracks[1]!, new Map([['b', []]]))).toEqual([]);
});
it('puts each lane right under its track, and finds a track by the row of its own', () => {
const open = new Map([['a', ['volume', 'pan'] as const]]);
const rows = songRows(tracks, open);
expect(rows).toEqual([
{ track: 0, lane: null },
{ track: 0, lane: 'volume' },
{ track: 0, lane: 'pan' },
{ track: 1, lane: null },
{ track: 1, lane: 'pan' },
{ track: 2, lane: null },
]);
expect([...trackRows(tracks, rows)]).toEqual([
['a', 0],
['b', 3],
['c', 5],
]);
// A lane is as high as a track, so the rows stay evenly spaced.
const view: SongView = { panX: 0, pxPerTick: 0.1, panY: 0, trackHeight: 40 };
expect(trackAtY(view, rows.length, 3 * 40 + 1)).toBe(3);
});
});
describe('the way of a lane (#337)', () => {
const view: SongView = { panX: 10, pxPerTick: 0.1, panY: 0, trackHeight: 52 };
it('draws a value from the bottom of its range to the top, and reads it back to a step', () => {
expect(laneY('volume', -60, 52)).toBe(52 - LANE_INSET);
expect(laneY('volume', 12, 52)).toBe(LANE_INSET);
expect(laneY('pan', 0, 52)).toBe(26);
// Past the range a value is drawn at the edge.
expect(laneY('pan', 3, 52)).toBe(LANE_INSET);
expect(laneValue('pan', 26, 52)).toBe(0);
expect(laneValue('volume', laneY('volume', -6.3, 52), 52)).toBeCloseTo(-6.3, 6);
expect(laneValue('volume', -100, 52)).toBe(12);
expect(laneValue('pan', laneY('pan', 0.333, 52), 52)).toBeCloseTo(0.33, 6);
});
it('finds the point under the pointer, the last drawn first', () => {
const points = [
{ tick: 0, value: 0 },
{ tick: 0, value: 0 },
{ tick: 960, value: 6 },
];
const y = laneY('volume', 0, 52);
expect(pointAt(view, 'volume', points, 10, y)).toBe(1);
expect(pointAt(view, 'volume', points, 10 + 96 + 5, laneY('volume', 6, 52) - 5)).toBe(2);
expect(pointAt(view, 'volume', points, 60, y)).toBeNull();
});
it('holds before the first point and after the last, and jumps upright on a step', () => {
const way = laneWay(view, 'volume', VOLUME, 300, 2);
const at = (x: number) => way.filter((p) => p.x === x).map((p) => p.y);
// Before the first point (x 10) the lane holds its value.
expect(at(0)).toEqual([laneY('volume', -12, 52)]);
// Halfway to the point at tick 960 (x 106) the linear way is at -6 dB, and reaches 0 dB there.
expect(at(58)).toEqual([laneY('volume', -6, 52)]);
expect(at(106)).toEqual([laneY('volume', 0, 52)]);
// The step holds until tick 1920 (x 202), where it jumps to the last point.
expect(at(200)).toEqual([laneY('volume', 0, 52)]);
expect(at(202)).toEqual([laneY('volume', 0, 52), laneY('volume', -24, 52)]);
expect(at(300)).toEqual([laneY('volume', -24, 52)]);
expect(way.map((p) => p.x)).toEqual([...way.map((p) => p.x)].sort((a, b) => a - b));
});
it('says a value as a head and a sign do', () => {
expect(laneValueText('volume', -6)).toBe('-6.0 dB');
expect(laneValueText('volume', 3.25)).toBe('+3.3 dB');
expect(laneValueText('volume', -60)).toBe('-inf dB');
expect(laneValueText('pan', 0)).toBe('Centre');
expect(laneValueText('pan', -0.25)).toBe('25% left');
expect(laneValueText('pan', 1)).toBe('100% right');
});
});
describe('editing the points of a lane (#337)', () => {
it('puts the points on their lane, the lanes in their order, and drops a lane without points', () => {
const a = withLanePoints(
track('a', [{ target: 'pan', points: [{ tick: 0, value: 1 }] }]),
'volume',
VOLUME,
);
expect(a.automation?.map((l) => l.target)).toEqual(['volume', 'pan']);
expect(lanePoints(a, 'volume')).toEqual(VOLUME);
const b = withLanePoints(a, 'pan', []);
expect(b.automation?.map((l) => l.target)).toEqual(['volume']);
expect('automation' in withLanePoints(b, 'volume', [])).toBe(false);
});
it('adds a point after the ones already on its tick, held to whole ticks and the range', () => {
expect(addedPoint(VOLUME, 'volume', 960.4, 3)).toEqual({
points: [VOLUME[0], VOLUME[1], { tick: 960, value: 3 }, VOLUME[2]],
index: 2,
});
expect(addedPoint([], 'pan', -5, 4)).toEqual({ points: [{ tick: 0, value: 1 }], index: 0 });
});
it('drags a point no further than its neighbours, keeping how it eases', () => {
expect(draggedPoint(VOLUME, 'volume', 1, 5000, -3)[1]).toEqual({ tick: 1920, value: -3, ease: 'step' });
expect(draggedPoint(VOLUME, 'volume', 1, -10, 99)[1]).toEqual({ tick: 0, value: 12, ease: 'step' });
expect(draggedPoint(VOLUME, 'volume', 2, 9999.6, -60)[2]).toEqual({ tick: 10000, value: -60 });
expect(draggedPoint(VOLUME, 'volume', 7, 0, 0)).toEqual(VOLUME);
});
it('takes a point away and sets how one eases, linear stored as none', () => {
expect(withoutPoint(VOLUME, 1)).toEqual([VOLUME[0], VOLUME[2]]);
expect(easedPoint(VOLUME, 0, 'sine-in')[0]).toEqual({ tick: 0, value: -12, ease: 'sine-in' });
expect(easedPoint(VOLUME, 1, 'linear')[1]).toEqual({ tick: 960, value: 0 });
});
});
describe('the lane of a parameter (#337)', () => {
const cut = { target: 'param:cut' as const, min: 0, max: 2000 };
const withCut = track('a', [
{ target: 'volume', points: [{ tick: 0, value: -6 }] },
{ ...cut, points: [] },
]);
it('is shown by its name after the volume and the pan, with no point too: it was added', () => {
expect(laneLabel('param:cut')).toBe('cut');
expect(trackLanes(withCut)).toEqual(['volume', 'pan', 'param:cut']);
expect(shownLanes(withCut, new Map())).toEqual(['volume', 'param:cut']);
expect(laneKindOf(withCut, 'param:cut')).toEqual(cut);
expect(laneKindOf(withCut, 'pan')).toEqual({ target: 'pan' });
});
it('draws and reads its values in its own range, to a step that cuts it into a hundred or more', () => {
expect(laneY(cut, 0, 52)).toBe(52 - LANE_INSET);
expect(laneY(cut, 2000, 52)).toBe(LANE_INSET);
expect(laneStep(cut)).toBe(10);
expect(laneStep({ target: 'param:mix' })).toBe(0.01);
expect(laneValue(cut, laneY(cut, 1234, 52), 52)).toBe(1230);
expect(laneValue({ target: 'param:mix' }, 26, 52)).toBe(0.5);
expect(laneValueText(cut, 1234)).toBe('1230');
expect(laneValueText({ target: 'param:mix' }, 0.123)).toBe('0.12');
expect(addedPoint([], cut, 0, 5000).points).toEqual([{ tick: 0, value: 2000 }]);
});
it('keeps its lane and range without a point, takes its points into a new range, and goes only when removed', () => {
const a = withLanePoints(withCut, 'param:cut', [{ tick: 0, value: 1500 }]);
expect(a.automation?.map((l) => l.target)).toEqual(['volume', 'param:cut']);
const emptied = withLanePoints(a, 'param:cut', []);
expect(emptied.automation?.[1]).toEqual({ ...cut, points: [] });
const narrow = withParamLane(a, 'cut', { min: 0, max: 1000 });
expect(narrow.automation?.[1]).toEqual({ ...cut, max: 1000, points: [{ tick: 0, value: 1000 }] });
// A new one comes last; a later volume lane still goes first.
const more = withLanePoints(withParamLane(narrow, 'tone', { min: 0, max: 1 }), 'pan', [
{ tick: 0, value: 0 },
]);
expect(more.automation?.map((l) => l.target)).toEqual(['volume', 'pan', 'param:cut', 'param:tone']);
expect(withoutLane(more, 'param:cut').automation?.map((l) => l.target)).toEqual([
'volume',
'pan',
'param:tone',
]);
});
});
describe('the easings as a menu (#337)', () => {
const project = {
assets: [
{ id: 'c2', type: 'curve', name: 'Swell' },
{ id: 'c1', type: 'curve', name: 'Dip' },
{ id: 'c3', type: 'curve', name: 'Inside', owner: 'song' },
],
} as unknown as Project;
const byLabel = (entries: MenuEntry[], label: string) =>
entries.find((e) => 'label' in e && e.label === label) as MenuItem & MenuSubmenu;
it('offers every easing, marks the one in use and hands back what is picked', () => {
const set = vi.fn();
const entries = easingEntries(project, 'cubic-out', set);
expect(entries.map((e) => ('label' in e ? e.label : '-'))).toEqual([
'Linear',
'Step',
'Sine',
'Quad',
'Cubic',
'Quart',
'Quint',
'Expo',
'Circ',
'Back',
'Elastic',
'Bounce',
'Curve',
]);
expect(byLabel(entries, 'Linear').current).toBe(false);
const cubic = byLabel(entries, 'Cubic').submenu as MenuItem[];
expect(cubic.map((e) => [e.label, e.current])).toEqual([
['In', false],
['Out', true],
['In-out', false],
]);
(byLabel(entries, 'Quad').submenu as MenuItem[])[0]!.onSelect();
byLabel(entries, 'Step').onSelect();
expect(set.mock.calls).toEqual([['ease-in'], ['step']]);
});
it('lists the Curve assets of the project by name, not those a song keeps to itself', () => {
const set = vi.fn();
const curve = byLabel(easingEntries(project, 'curve:c2', set), 'Curve');
const items = curve.submenu as MenuItem[];
expect(items.map((e) => [e.label, e.current])).toEqual([
['Dip', false],
['Swell', true],
]);
items[0]!.onSelect();
expect(set).toHaveBeenCalledWith('curve:c1');
expect(
byLabel(easingEntries({ assets: [] } as unknown as Project, 'linear', set), 'Curve').disabled,
).toBe(true);
});
});
+59
View File
@@ -0,0 +1,59 @@
import { describe, expect, it } from 'vitest';
import { heldSongMarkers } from '../../src/core/music/markers';
import {
MARKER_FLAG,
MARKER_REACH,
MARKER_STRIP,
markerAt,
markerNameX,
markerSpans,
SONG_TOP,
} from '../../src/editor/song/markerView';
import { RULER_HEIGHT } from '../../src/editor/music/timeView';
/** A tick a pixel, from x 100; every name 6 pixels a letter. */
const VIEW = { panX: 100, pxPerTick: 1 };
const measure = (name: string) => name.length * 6;
describe('the marker strip of the song tab (#338)', () => {
it('lies between the ruler and the tracks', () => {
expect(SONG_TOP).toBe(RULER_HEIGHT + MARKER_STRIP);
});
it('draws a marker at its tick, as wide as its flag and name, but not into the next one', () => {
const markers = heldSongMarkers([
{ id: 'a', name: 'Intro', tick: 0 },
{ id: 'b', name: 'A long name', tick: 40 },
{ id: 'c', name: '', tick: 300 },
]);
const spans = markerSpans(VIEW, markers, measure);
expect(spans.map((s) => s.x)).toEqual([100, 140, 400]);
// Room to the next one is 40 pixels, less 2: the name of "Intro" (30) and the flag fit just so.
expect(spans[0]!.w).toBe(38);
expect(spans[1]!.w).toBe(MARKER_FLAG + 3 + measure('A long name'));
expect(spans[2]!.w).toBe(MARKER_FLAG + 3 + measure('Marker 3'));
expect(markerNameX(spans[0]!)).toBe(100 + MARKER_FLAG + 3);
});
it('keeps the flag of a marker with another one close behind it', () => {
const markers = heldSongMarkers([
{ id: 'a', name: 'Close', tick: 0 },
{ id: 'b', name: 'By', tick: 2 },
]);
expect(markerSpans(VIEW, markers, measure)[0]!.w).toBe(MARKER_FLAG);
});
it('takes the marker near its line or on its flag and name, the later one where they overlap', () => {
const markers = heldSongMarkers([
{ id: 'a', name: 'Intro', tick: 0 },
{ id: 'b', name: 'B', tick: 6 },
]);
const spans = markerSpans(VIEW, markers, measure);
expect(markerAt(spans, 100 - MARKER_REACH)).toBe('a');
expect(markerAt(spans, 100 - MARKER_REACH - 1)).toBeNull();
expect(markerAt(spans, 103)).toBe('b');
expect(markerAt(spans, 106 + MARKER_FLAG + 3 + 6)).toBe('b');
expect(markerAt(spans, 106 + MARKER_FLAG + 3 + 7)).toBeNull();
expect(markerAt([], 100)).toBeNull();
});
});
+23 -2
View File
@@ -276,7 +276,10 @@ export type FakeContext = NodeContext & {
fx: string;
fxParams: Readonly<Record<string, number>>;
bus?: string;
live?: boolean;
}[];
/** What Set song track and Switch song asked of a playing (#338), one line each. */
songChanges: string[];
silenced: string[];
tuned: { voice: number; fxParams: Readonly<Record<string, number>>; time: number; ease: EaseShare }[];
/** What buses the context was told to change (#325): a volume or FX parameters, in order. */
@@ -378,8 +381,25 @@ export function fakeContext(overrides: Partial<NodeContext> = {}): FakeContext {
stopPattern: (patternId, letRing) =>
ctx.silenced.push(`pattern ${patternId}${letRing ? ' (let ring)' : ''}`),
songs: [],
playSong: (songId, volume, fadeIn, loop, detached, fx, fxParams, bus) =>
ctx.songs.push({ songId, volume, fadeIn, loop, detached, fx, fxParams, ...(bus && { bus }) }),
playSong: (songId, volume, fadeIn, loop, detached, fx, fxParams, bus, live) =>
ctx.songs.push({
songId,
volume,
fadeIn,
loop,
detached,
fx,
fxParams,
...(bus && { bus }),
...(live && { live }),
}),
songChanges: [],
setSongTrack: (playback, track, volumeDb, fade, quantize) =>
ctx.songChanges.push(`track ${playback} ${track} ${volumeDb} dB fade ${fade} ${quantize}`),
switchSong: (playback, songId, marker, quantize, fade) =>
ctx.songChanges.push(
`switch ${playback} to ${songId || '(same)'} at ${marker || '(start)'} ${quantize} fade ${fade}`,
),
stopSong: (songId, fadeOut) => ctx.silenced.push(`song ${songId} (fade ${fadeOut})`),
stopPlayback: (playback, fadeOut) => ctx.silenced.push(`playback ${playback} (fade ${fadeOut})`),
// Every sound lasts a quarter of a second, as if it had been loaded.
@@ -398,6 +418,7 @@ export function fakeContext(overrides: Partial<NodeContext> = {}): FakeContext {
busVolume: (bus) =>
[...ctx.buses].reverse().find((change) => change.bus === bus && change.volume !== undefined)?.volume ??
1,
songPosition: () => null,
loadScene: (sceneId) => ctx.loaded.push(sceneId),
spawn: () => null,
addJoint: () => null,
+360 -2
View File
@@ -15,9 +15,11 @@ import {
type FxStagePlan,
type FxVoice,
} from '../../src/core/soundfx/nodes';
import type { AutomationTarget } from '../../src/core/music/automation';
import { dbToGain } from '../../src/core/music/level';
import type { MixBus } from '../../src/core/music/mixer';
import type { BakePlay, SequencePlay } from '../../src/core/music/playbacks';
import type { ScheduledNote } from '../../src/core/music/Scheduler';
import { type BakePlay, CUT, type LandingAt, type SequencePlay } from '../../src/core/music/playbacks';
import type { ScheduledAutomation, ScheduledNote } from '../../src/core/music/Scheduler';
import type { LfoOptions, PitchOptions, StreamOptions } from '../../src/viewport/fxWorklet';
import { saturationCurve } from '../../src/core/soundfx/saturation';
import { stageNodes } from '../../src/viewport/fxNodes';
@@ -1980,6 +1982,272 @@ describe('the patterns and songs a game plays (#336)', () => {
library.time();
expect(resumes).toBe(2);
});
/** A bus of the library by its name, with the params a test reads. */
const busOf = (library: SoundLibrary, name: string) =>
(
library as unknown as { buses: Map<string, { input: { gain: FakeParam }; gain: { gain: FakeParam } }> }
).buses.get(name);
it('turns a track of a playing on its input from where it lands, on top of its level (#338)', async () => {
const library = new SoundLibrary(false);
const id = library.playSequence(
'song:s',
sequence(() => 0),
'',
);
await settle();
const landed: (number | null)[] = [];
const at: LandingAt = (clock) => (landed.push(clock.startedAt), clock.startedAt! + 2);
expect(library.setSongTrack(id, 'song:s:t', 0.25, 1, at)).toBe('set');
const track = busOf(library, `play:${id}:song:s:t`)!;
expect([landed, track.input.gain.ramps]).toEqual([
[0.05],
[
['set', 1, 2.05],
['linear', 0.25, 3.05],
],
]);
// Its own level is the song's, left alone.
expect(track.gain.gain.value).toBe(0.25);
// A turn after that one is through starts where it left the track, not where it is now.
library.setSongTrack(id, 'song:s:t', 0.5, 0, (clock) => clock.startedAt! + 4);
expect(track.input.gain.ramps.slice(2)).toEqual([
['set', 0.25, 4.05],
['linear', 0.5, expect.closeTo(4.055, 9)],
]);
expect(library.setSongTrack(id, 'song:s:gone', 0.5, 0, at)).toBe('none');
expect(library.setSongTrack(id + 99, 'song:s:t', 0.5, 0, at)).toBe('none');
});
it('a playing of its bake has no tracks to turn (#338)', () => {
const library = new SoundLibrary(false);
const bake = { file: 'baked.ogg', gain: 0.5, playback: { end: 2, until: 2, round: null } };
const id = library.playSequence(
'song:s',
sequence(() => 0, { bake }),
'',
);
expect(library.setSongTrack(id, 'song:s:t', 0.5, 0, () => 0)).toBe('baked');
});
it('switches a playing on buses of its own, its tracks turned as they were, the old one gone once silent (#338)', async () => {
const library = new SoundLibrary(false);
const id = library.playSequence(
'song:s',
sequence(() => 0),
'logic',
);
await settle();
library.setSongTrack(id, 'song:s:t', 0.25, 0, () => 0.5);
const at: LandingAt = (clock) => clock.startedAt! + 2;
expect(
library.switchSequence(
id,
'song:s',
sequence(() => 0, { from: 480 }),
at,
0,
),
).toBe(true);
const fresh = library
.busLevels()
.find((bus) => bus.name.endsWith(':song:s:t') && bus.name !== `play:${id}:song:s:t`);
expect(busOf(library, fresh!.name)!.input.gain.value).toBe(0.25);
await settle();
expect(library.playbacks().map((p) => [p.id, p.owner])).toEqual([
[id, 'logic'],
[id, 'logic'],
]);
expect(busOf(library, `play:${id}`)!.gain.gain.ramps.slice(-2)).toEqual([
['set', 0, 2.05],
['linear', 0, 2.05 + CUT],
]);
clockTo();
expect(library.playbacks().map((p) => p.id)).toEqual([id]);
// The old one's buses are let go, though a playing of its number plays on.
expect(library.busLevels().filter((bus) => bus.name.startsWith(`play:${id}`))).toEqual([]);
expect(
library.switchSequence(
id + 99,
'song:s',
sequence(() => 0),
at,
0,
),
).toBe(false);
});
});
describe('the automation of the tracks of a song (#337)', () => {
const automation = (fields: Partial<ScheduledAutomation>): ScheduledAutomation => ({
bus: 'song:s:t',
target: 'volume',
fresh: false,
marks: [],
...fields,
});
it('leaves what a lane sets alone, and hands it back to the fader from where the lane left it', async () => {
const library = new SoundLibrary(false);
// The first gain is where the bus is fed, the second its level.
await library.setBuses([{ name: 'song:s:t', gain: 0.5, pan: 0.25, automated: ['pan'] }]);
await library.setBuses([{ name: 'song:s:t', gain: 0.4, pan: -1, automated: ['pan'] }]);
expect([gains[1]!.glides, panners[0]!.glides, panners[0]!.holds]).toEqual([[0.5, 0.4], [0.25], 0]);
await library.setBuses([{ name: 'song:s:t', gain: 0.4, pan: -1 }]);
expect([panners[0]!.glides, panners[0]!.holds, gains[1]!.holds]).toEqual([[0.25, -1], 1, 0]);
});
it('ramps a bus to each mark or jumps there, a fresh start first dropping what was planned', async () => {
const library = new SoundLibrary(false);
await library.setBuses([{ name: 'song:s:t', gain: 1, pan: 0, automated: ['volume'] }]);
library.automateBus(
'song:s:t',
automation({
fresh: true,
marks: [
{ at: 1, value: 0, jump: true },
{ at: 2, value: 0.5, jump: false },
],
}),
);
library.automateBus('song:s:t', automation({ marks: [{ at: 3, value: 0.25, jump: false }] }));
// The hold by hand of a context without cancelAndHoldAtTime, at the first mark.
expect(gains[1]!.ramps).toEqual([
['set', 1, 1],
['set', 0, 1],
['linear', 0.5, 2],
['linear', 0.25, 3],
]);
library.automateBus(
'song:s:gone',
automation({ bus: 'song:s:gone', marks: [{ at: 1, value: 0, jump: true }] }),
);
expect(library.busLevels().map((bus) => bus.name)).toEqual(['mix', 'song:s:t']);
});
it('takes the tracks of a playing in a game along their lanes, and no bus the song does not have', async () => {
const library = new SoundLibrary(false);
const lane = (bus: string) => ({
bus,
target: 'pan' as const,
points: [
{ tick: 0, value: -1 },
{ tick: 1920, value: 1 },
],
});
const id = library.playSequence(
'song:s',
{
sequence: {
notes: [{ tick: 0, length: 480, note: 60, velocity: 1, instrument: 'i', bus: 'song:s:t' }],
lanes: [lane('song:s:t'), lane('song:s:gone')],
},
tempo: 120,
length: 1920,
loop: null,
needs: { layers: [], files: [] },
buses: [{ name: 'song:s:t', gain: 1, pan: -1, automated: ['pan'] }],
strike: () => 0,
},
'',
);
await settle();
expect(library.busLevels().map((bus) => bus.name)).toContain(`play:${id}:song:s:t`);
// One bus only is taken along: the track's, over its first window – on from the first note, a
// lookahead ahead, 48 ticks at 120 BPM.
const taken = panners.filter((panner) => panner.ramps.length > 0);
expect(taken).toHaveLength(1);
const ramps = taken[0]!.ramps.map(([kind, value, at]) => [
kind,
value,
Math.round(Number(at) * 1e6) / 1e6,
]);
expect(ramps).toEqual([
['set', -1, 0.05],
['set', -1, 0.05],
['linear', -0.95, 0.1],
]);
expect(library.busLevels().some((bus) => bus.name.includes('gone'))).toBe(false);
});
/** A gain whose decibels are the parameter `loud`; with `source` it plays FILE, as a clip's FX does. */
const loud = (fields: Partial<FxPlan> = {}): FxVoice => ({
plan: {
pitch: [],
stages: [{ kind: 'gain', db: { value: 0, param: 'loud', min: -120, max: 24 } }],
...fields,
},
params: {},
tempo: 120,
});
const clip = (library: SoundLibrary, bus: string, file = FILE) =>
library.playClip(file, { at: 0, offset: 0, duration: 10, volume: 1, bus }, loud());
const tracks = (fx: FxVoice, automated: readonly AutomationTarget[] = ['param:loud']) => [
{ name: 'song:s:t', gain: 1, pan: 0, fx, automated },
{ name: 'song:s:t:#items', gain: 1, pan: 0, into: 'song:s:t' },
{ name: 'song:s:u:#items', gain: 1, pan: 0 },
];
const marks = (...values: [number, number][]) =>
automation({
target: 'param:loud',
marks: values.map(([at, value]) => ({ at, value, jump: false })),
});
/** What was planned for each gain before the clips fade out at their end, 10 s on. */
const moved = () =>
gains.map((gain) => gain.ramps.filter(([, , at]) => Number(at) < 5)).filter((ramps) => ramps.length > 0);
it('takes the sound FX of a track and every voice inside it along a parameter, in what their knobs hold', async () => {
const library = new SoundLibrary(false);
await library.setBuses(tracks(loud()));
await settle();
clip(library, 'song:s:t:#items');
clip(library, 'song:s:u:#items');
await settle();
await arrive();
library.automateBus('song:s:t', marks([1, -6], [2, -12]));
// The chain of the track and the voice on it, each first set where nothing was planned yet.
const way = [
['set', expect.closeTo(dbToGain(-6), 9), 1],
['linear', expect.closeTo(dbToGain(-12), 9), 2],
];
expect(moved()).toEqual([way, way]);
// The next window ramps on from there; a mark where a knob stays is left out.
library.automateBus('song:s:t', marks([2, -12], [3, 0]));
expect(moved()[0]).toEqual([...way, ['linear', expect.closeTo(dbToGain(-12), 9), 2], ['linear', 1, 3]]);
library.automateBus('song:s:t', marks([3, 0], [4, 0]));
expect(moved()[0]).toHaveLength(4);
});
it('takes up the marks planned for a voice still waiting on its bytes once it starts', async () => {
const library = new SoundLibrary(false);
await library.setBuses(tracks({ ...loud(), plan: { pitch: [], stages: [] } }, []));
clip(library, 'song:s:t:#items', OTHER);
library.automateBus('song:s:t', marks([1, -6], [2, -12]));
expect(moved()).toEqual([]);
await settle();
await arrive();
expect(moved()).toEqual([
[
['set', expect.closeTo(dbToGain(-6), 9), 1],
['linear', expect.closeTo(dbToGain(-12), 9), 2],
],
]);
});
it('leaves a parameter a lane rules alone when the tracks are set again, and hands one let go back', async () => {
const library = new SoundLibrary(false);
// One plan throughout, as a sound FX compiles to: another would be another chain.
const fx = loud();
await library.setBuses(tracks(fx));
library.automateBus('song:s:t', marks([1, -12]));
const knob = gains.find((gain) => gain.ramps.length > 0)!;
await library.setBuses(tracks({ ...fx, params: { loud: -3 } }));
expect(knob.glides).toEqual([]);
// The lane is gone: back to what the track's FX is set to, its default.
await library.setBuses(tracks(fx, []));
expect(knob.glides).toEqual([1]);
});
});
describe('a bake (#340)', () => {
@@ -2342,6 +2610,80 @@ describe('a sound FX on a bus (#346)', () => {
expect(disconnects).toEqual(['panner', 'splitter']);
});
it('taps every bus for the agent while it watches, a later one too, with its peak and RMS (#436)', async () => {
const library = new SoundLibrary(false);
await library.setBuses([{ name: 'song:s:t', gain: 1, pan: 0 }]);
expect(library.heard().buses).toEqual([]);
library.watch(true);
expect(analysers.map((side) => side.fftSize)).toEqual([32768, 32768, 32768, 32768]);
await library.setBuses([
{ name: 'song:s:t', gain: 1, pan: 0 },
{ name: 'song:s:u', gain: 1, pan: 0 },
]);
expect(library.heard().buses.map((bus) => bus.name)).toEqual(['mix', 'song:s:t', 'song:s:u']);
// The fake holds one sample in the middle of the stretch and silence round it.
analysers[2]!.sample = -0.5;
analysers[3]!.sample = 0.25;
const track = library.heard().buses[1]!;
expect(track).toMatchObject({ name: 'song:s:t', peak: [0.5, 0.25] });
expect(track.rms[0]).toBeCloseTo(0.5 / Math.sqrt(32768), 9);
expect(track.rms[1]).toBeCloseTo(0.25 / Math.sqrt(32768), 9);
expect(library.heard().window).toBeCloseTo(32768 / 48000);
// The Song tab's meter is another tap: letting it go leaves the agent's.
library.busPeaks(['song:s:t']);
library.unmeter();
expect(library.heard().buses).toHaveLength(3);
// Moved elsewhere, a bus keeps its tap.
wires.length = 0;
await library.setBuses([{ name: 'song:s:t', gain: 1, pan: 0, into: 'song:s:u' }]);
expect(wires.filter((wire) => wire === 'panner > splitter')).toHaveLength(1);
disconnects.length = 0;
library.watch(false);
expect(library.heard().buses).toEqual([]);
expect(disconnects.filter((node) => node === 'splitter')).toHaveLength(3);
});
it('says what plays: each voice with its owner, bus and note, those planned, waiting or failed (#436)', async () => {
const library = new SoundLibrary(false);
await play(library, FILE, { owner: 'o1' });
const gate = {
pitch: [],
stages: [],
source: { file: FILE, volume: 1, loop: true, loopStart: 0, loopEnd: 0, gate: true },
} as FxPlan;
library.playNote('instrument:i', [gate], {}, 1, '', 0, { at: 5, bus: 'song:s:t', note: 64 });
library.play(OTHER, 1, false, 'o2');
vi.stubGlobal('fetch', () => Promise.resolve({ ok: false, status: 404 }));
vi.spyOn(console, 'error').mockImplementation(() => {});
await library.load('gone.wav');
const heard = library.heard();
expect(heard.voices).toEqual([
{ id: 1, file: FILE, owner: 'o1', bus: '', startsIn: 0, released: false, ringing: false },
{
id: 2,
file: 'instrument:i',
owner: '',
bus: 'song:s:t',
note: 64,
startsIn: 5,
released: false,
ringing: false,
},
]);
expect(heard.waiting).toEqual([{ id: 3, file: OTHER, owner: 'o2', bus: '' }]);
expect(heard.failed).toEqual(['gone.wav']);
library.release(2);
expect(library.heard().voices[1]).toMatchObject({ released: true });
// Paused, the clock is told as it ran before: held by the pause, or by the browser already.
expect(heard.clock).toEqual({ state: 'running', time: 0, held: false });
library.setSuspended(true);
contextState = 'suspended';
expect(library.heard().clock).toEqual({ state: 'running', time: 0, held: true });
library.setSuspended(false);
library.setSuspended(true);
expect(library.heard().clock).toEqual({ state: 'suspended', time: 0, held: true });
});
it('makes the chain of a track in the Song tab once, and again only for another sound FX', async () => {
const library = new SoundLibrary(false);
const track = { name: 'song:s:t', gain: 1, pan: 0, fx: voice(crush) };
@@ -2539,6 +2881,22 @@ describe('the master and the buses of the project (#325)', () => {
expect(wires.filter((wire) => wire.endsWith('> destination'))).toEqual(['panner > destination']);
});
it('makes a bus at its level, so the first sound of a run on a quiet bus is not heard gliding down (#436)', async () => {
const library = new SoundLibrary(false);
await library.setMix(mix);
await play(library);
expect(library.busLevels().map(({ name, level }) => [name, level])).toEqual([
['mix', 0.5],
['mix:a', 1],
['mix:b', 0.25],
]);
expect(gains.filter((gain) => gain.glides.length > 0)).toEqual([]);
// One there already glides to a new level, as before.
await library.setMix([bus('mix', 0.25, null), mix[1]!, mix[2]!]);
expect(library.busLevels()[0]).toMatchObject({ gain: 0.25, level: 0.5 });
expect(gains.some((gain) => gain.glides.includes(0.25))).toBe(true);
});
it('keeps the mix where the first thing heard is a tab of a song, which names its bus before there is a context', async () => {
const library = new SoundLibrary(false);
await library.setMix(mix);
+20 -3
View File
@@ -3964,9 +3964,23 @@ export async function sound(t) {
// counted once it is there, so the pause is judged by what it adds. The wait after it lets the
// worklet report how far it has read: until then a stretch sent counts as ahead.
const headsOnly = statsBefore.streaming.held;
await waitFor(stats, (s) => s.streaming.held - headsOnly >= STREAM_SECOND_BYTES, 40);
const aheadStreamed = await waitFor(stats, (s) => s.streaming.held - headsOnly >= STREAM_SECOND_BYTES, 40);
await sleep(500);
const heldStreamed = await stats();
// A runner too slow to decode never gets the second ahead and skips from the start (CI task 750,
// 0.04 s ahead, 1.1 s skipped before the pause): that is told here, not blamed on the pause.
const streamAhead =
aheadStreamed.streaming.held - headsOnly >= STREAM_SECOND_BYTES &&
heldStreamed.streaming.skipped === statsBefore.streaming.skipped;
check(
'before the pause the stream runs a second ahead, not a sample skipped',
streamAhead,
JSON.stringify({
before: statsBefore.streaming,
ahead: aheadStreamed.streaming,
held: heldStreamed.streaming,
}),
);
await t.clickToolbar('pause');
await sleep(500);
const paused = { clock: await json('__gm.sound.clock()'), state: await json('__gm.sound.clockState()') };
@@ -3988,12 +4002,15 @@ export async function sound(t) {
Math.max(...(heardLong?.mix ?? [0])) > 0.01,
JSON.stringify({ streaming, heardLong }),
);
// Behind before it, the pause cannot be judged by what it skips: the clock and stopping still are.
check(
'the pause holds the stream where it is, it plays on after it without a sample lost, and stopping ends it',
streamAhead
? 'the pause holds the stream where it is, it plays on after it without a sample lost, and stopping ends it'
: 'the pause holds the clock and stopping ends it (samples not judged: the stream was behind before the pause)',
paused.state === 'suspended' &&
pausedLater === paused.clock &&
resumedLong.streaming.playing === 1 &&
resumedLong.streaming.skipped === heldStreamed.streaming.skipped &&
(!streamAhead || resumedLong.streaming.skipped === heldStreamed.streaming.skipped) &&
stoppedLong.streaming.playing === 0 &&
stoppedLong.playing === 0,
// Held before the pause too: it tells a stream that fell behind from the start from one the
+477 -2
View File
@@ -490,6 +490,215 @@ export async function song(t) {
song.tracks[2].solo = false;
song.tracks[0].mute = false;`);
// 65g2. Automation lanes (#337). The triangle in a head opens a lane for the volume and one for the
// pan right under its track, and what lies below moves down. A click on a lane sets a point on
// the grid, a press dragged on sets one and takes it along, as one step. A dragged point stops
// at its neighbour. A point's menu sets how it eases to the next, it says its value when the
// pointer rests on it, and a double-click takes it away. The fader a lane rules does not move.
// Clear lane leaves the lane open and empty, the triangle closes it.
const lanesShown = () => json('__gm.song.automation()');
const volumeAt = (tick, value) => json(`__gm.song.automationAt(1, 'volume', ${tick}, ${value})`);
const volumePoints = async () =>
(await model()).tracks[1].automation?.find((lane) => lane.target === 'volume')?.points ?? [];
const lowItem = (await model()).items.find((item) => item.track !== tracks[0] && item.track !== tracks[1]);
const lowY = async () => (await json('__gm.song.items()')).find((item) => item.id === lowItem?.id)?.box.y;
const lowBefore = await lowY();
const rowHeight = (await json('__gm.song.view()')).trackHeight;
p = await json("__gm.song.headAt(1, 'lanes')");
await click(p.x, p.y);
await sleep(100);
const lanesOpened = await lanesShown();
const lowAfter = await lowY();
check(
'the triangle of a head opens a volume and a pan lane under its track, and what lies below moves down',
JSON.stringify(lanesOpened.rows.map((row) => row.lane)) ===
JSON.stringify([null, null, 'volume', 'pan', null, null, null]) &&
lanesOpened.rows[2].track === tracks[1] &&
lanesOpened.rows[3].track === tracks[1] &&
lowItem !== undefined &&
Math.abs(lowAfter - lowBefore - 2 * rowHeight) < 0.5,
JSON.stringify({ lanesOpened, lowBefore, lowAfter, rowHeight }),
);
p = await volumeAt(BAR + 100, -12);
await click(p.x, p.y);
await sleep(100);
const onePoint = await volumePoints();
p = await volumeAt(3 * BAR, -30);
to = await volumeAt(4 * BAR + 50, 0);
await drag(p.x, p.y, to.x, to.y);
await sleep(100);
const twoPoints = await volumePoints();
const addedLabel = await json('__gm.state.commands.undoLabel');
await press('z', { ctrl: true });
await sleep(100);
const laneUndone = await volumePoints();
await clickToolbar('redo');
await sleep(100);
const laneRedone = await volumePoints();
check(
'a click on a lane sets a point on the bar, a drag sets one and takes it along as one step',
onePoint.length === 1 &&
onePoint[0].tick === BAR &&
Math.abs(onePoint[0].value + 12) < 1.5 &&
twoPoints.length === 2 &&
twoPoints[1].tick === 4 * BAR &&
Math.abs(twoPoints[1].value) < 1.5 &&
addedLabel === 'Add automation point' &&
laneUndone.length === 1 &&
JSON.stringify(laneRedone) === JSON.stringify(twoPoints),
JSON.stringify({ onePoint, twoPoints, addedLabel, laneUndone, laneRedone }),
);
p = await volumeAt(twoPoints[0].tick, twoPoints[0].value);
to = await volumeAt(6 * BAR, twoPoints[0].value);
await drag(p.x, p.y, to.x, to.y);
await sleep(100);
const heldPoints = await volumePoints();
await press('z', { ctrl: true });
await sleep(100);
check(
'a point dragged past its neighbour stops on its tick: two points there, a jump',
heldPoints.length === 2 &&
heldPoints[0].tick === 4 * BAR &&
heldPoints[1].tick === 4 * BAR &&
Math.abs(heldPoints[0].value - twoPoints[0].value) < 0.2 &&
JSON.stringify(await volumePoints()) === JSON.stringify(twoPoints),
JSON.stringify(heldPoints),
);
p = await volumeAt(twoPoints[0].tick, twoPoints[0].value);
await click(p.x, p.y, 'right');
await sleep(150);
const pointRows = await json('__gm.menuRows()');
await clickText('Ease to next point');
await sleep(150);
await clickText('Sine');
await sleep(150);
await clickText('In');
await sleep(150);
const eased = await volumePoints();
const shownValue = (v) =>
`${Math.round(v * 10) / 10 > 0 ? '+' : ''}${(Math.round(v * 10) / 10).toFixed(1)} dB`;
p = await volumeAt(twoPoints[1].tick, twoPoints[1].value);
await mouse('mouseMoved', p.x - 1, p.y);
await mouse('mouseMoved', p.x, p.y);
await t.waitUntil('__gm.tooltip() !== null', { label: 'the sign over a point of a lane' });
const pointSign = await json('__gm.tooltip()');
check(
"a point's menu sets how it eases to the next, and resting on a point says its value",
pointRows?.includes('Ease to next point') &&
pointRows.includes('Delete point') &&
eased[0].ease === 'sine-in' &&
eased[1].ease === undefined &&
pointSign?.text === shownValue(twoPoints[1].value),
JSON.stringify({ pointRows, eased, pointSign }),
);
p = await json("__gm.song.headAt(1, 'volume', 0.5)");
to = await json("__gm.song.headAt(1, 'volume', 0)");
await drag(p.x, p.y, to.x + 1, to.y);
await sleep(100);
const ruledFader = { volume: (await model()).tracks[1].volume, notice: await json('__gm.state.notice') };
p = await volumeAt(twoPoints[1].tick, twoPoints[1].value);
await doubleClick(p.x, p.y);
await sleep(100);
const afterDouble = await volumePoints();
check(
'the fader a lane rules does not move, and a double-click takes a point away',
ruledFader.volume === 0 &&
ruledFader.notice.startsWith('Volume follows its automation lane') &&
afterDouble.length === 1 &&
afterDouble[0].tick === BAR,
JSON.stringify({ ruledFader, afterDouble }),
);
p = await json("__gm.song.automationHeadAt(1, 'volume')");
await click(p.x, p.y, 'right');
await sleep(150);
await clickText('Clear lane');
await sleep(100);
const laneCleared = { track: (await model()).tracks[1], rows: (await lanesShown()).rows.length };
p = await json("__gm.song.headAt(1, 'lanes')");
await click(p.x, p.y);
await sleep(100);
const closedRows = (await lanesShown()).rows.length;
check(
'Clear lane takes the lane out of the song and leaves it open, the triangle closes the lanes',
!('automation' in laneCleared.track) &&
laneCleared.rows === 7 &&
closedRows === 5 &&
(await lowY()) === lowBefore,
JSON.stringify({ laneCleared, closedRows }),
);
await evaluate("__gm.state.notice = ''");
// 65g3. Markers (#338) on their strip under the ruler. A double-click there adds one on the grid,
// a drag moves it as one step, a click starts playing at it, a double-click names it, resting
// on it says its name, and its menu takes it away.
const songMarkers = async () => (await model()).markers;
const markerAt = (tick) => json(`__gm.song.markerAt(${tick})`);
p = await markerAt(BAR + 100);
await doubleClick(p.x, p.y);
await sleep(100);
const added = await songMarkers();
p = await markerAt(BAR);
to = await markerAt(3 * BAR + 50);
await drag(p.x, p.y, to.x, to.y);
await sleep(100);
const movedMarkers = await songMarkers();
const movedLabel = await json('__gm.state.commands.undoLabel');
await press('z', { ctrl: true });
await sleep(100);
const markerUndone = await songMarkers();
await clickToolbar('redo');
await sleep(100);
const markerDrawn = (await json('__gm.song.markers()'))[0];
const rulerX = (await json(`__gm.song.rulerAt(${3 * BAR})`)).x;
check(
'a double-click on the strip adds a marker on the bar, a drag moves it as one step, drawn at its tick',
added.length === 1 &&
added[0].name === 'Marker 1' &&
added[0].tick === BAR &&
movedMarkers[0].tick === 3 * BAR &&
movedLabel === 'Move marker' &&
markerUndone[0].tick === BAR &&
(await songMarkers())[0].tick === 3 * BAR &&
Math.abs(markerDrawn?.x - rulerX) < 1,
JSON.stringify({ added, movedMarkers, movedLabel, markerUndone, markerDrawn, rulerX }),
);
p = await markerAt(3 * BAR);
await click(p.x, p.y);
await sleep(100);
const startAtMarker = (await json('__gm.song.view()')).start;
await sleep(500);
await doubleClick(p.x, p.y);
await sleep(150);
const nameDialog = await json('__gm.groupDialogOpen()');
await t.typeText('Boss');
await press('Enter');
await sleep(100);
const named = (await songMarkers())[0];
// The presses left the marker's sign quiet: the pointer goes off the strip and back.
to = await json(`__gm.song.rulerAt(${3 * BAR})`);
await mouse('mouseMoved', to.x, to.y);
await mouse('mouseMoved', p.x + 5, p.y);
await mouse('mouseMoved', p.x + 6, p.y);
await t.waitUntil('__gm.tooltip() !== null', { label: 'the sign over a marker' });
const markerSign = await json('__gm.tooltip()');
check(
'a click on a marker starts playing there, a double-click names it, and resting on it says the name',
startAtMarker === 3 * BAR && nameDialog === true && named?.name === 'Boss' && markerSign?.text === 'Boss',
JSON.stringify({ startAtMarker, nameDialog, named, markerSign }),
);
await click(p.x, p.y, 'right');
await sleep(150);
const markerRows = await json('__gm.menuRows()');
await clickText('Delete marker');
await sleep(100);
const markersLeft = await model();
check(
"a marker's menu plays from it, renames it and takes it away",
['Play from here', 'Rename…', 'Delete marker'].every((row) => markerRows?.includes(row)) &&
markersLeft.markers.length === 0,
JSON.stringify({ markerRows, markers: markersLeft.markers }),
);
// 65h. Space plays from the start: the pattern a bar into itself at 2 s a bar, the sound as a
// clip round and round, each on the bus of its track.
p = await json('__gm.song.rulerAt(0)');
@@ -531,6 +740,100 @@ export async function song(t) {
bus(tracks[1])?.gain === 1,
JSON.stringify(buses),
);
// 65h2. A lane of the second track's volume (#337), -40 dB to 0 dB over eight bars, takes its bus
// along while the song plays, from the window after it is set on: the level follows the
// playhead, and the fader turned all the way down meanwhile does not move it. The loop is
// made long, so the playhead does not jump back while it is read.
const loopBefore = (await model()).loop;
await mutate(`
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
song.loop = { start: 0, end: ${64 * BAR} };
song.tracks[1].automation = [{ target: 'volume', points: [{ tick: 0, value: -40 }, { tick: ${8 * BAR}, value: 0 }] }];`);
const laneDb = (ticks) => -40 + (40 * ticks) / (8 * BAR);
const readLane = () =>
json(`({
level: __gm.song.buses().find((b) => b.name === ${JSON.stringify(busOf(tracks[1]))})?.level,
at: __gm.song.position()?.ticks,
})`);
await sleep(300);
const laneFirst = await readLane();
await mutate(`
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
song.tracks[1].volume = -60;`);
await sleep(1000);
const laneLater = await readLane();
const offLane = (r) => Math.abs(20 * Math.log10(r.level) - laneDb(r.at));
check(
'a lane of the volume takes the bus of its track along the playhead, and the fader no longer moves it',
[laneFirst, laneLater].every((r) => r.level > 0 && r.at < 8 * BAR && offLane(r) < 1) &&
laneLater.at - laneFirst.at > 800 &&
20 * Math.log10(laneLater.level / laneFirst.level) > 2,
JSON.stringify({ laneFirst, laneLater, off: [laneFirst, laneLater].map(offLane) }),
);
await mutate(`
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
delete song.tracks[1].automation;
song.tracks[1].volume = 0;`);
// 65h3. A lane of a parameter (#337, stage 3): the sound item of the first track goes through a
// sound FX whose parameter `cut` is its gain in dB, and starts again a bar on, eight bars
// long. Its clip starts at the value the lane has there, and its knob follows the lane while
// it plays: one voice, read twice.
const cutItem = (await model()).items.find((i) => i.kind === 'sound');
const itemBefore = JSON.stringify(cutItem);
const startAt = Math.ceil(((await json('__gm.song.position()')).ticks + BAR) / BAR) * BAR;
await mutate(`
__gm.state.project.assets.push({
id: 'sg65-fx',
type: 'soundfx',
name: 'Cut 65',
props: {},
graph: {
nodes: ${JSON.stringify([
node('sg65-fx-in', 'SoundFxInput', 0, 0, {}, [{ key: 'cut', kind: 'number', default: 0 }]),
node('sg65-fx-gain', 'SoundFxGain', 200, 0),
node('sg65-fx-out', 'SoundFxOutput', 400, 0),
])},
edges: ${JSON.stringify([
edge('sg65-fx-e1', 'sg65-fx-in', 'out', 'sg65-fx-gain', 'in'),
edge('sg65-fx-e2', 'sg65-fx-in', 'cut', 'sg65-fx-gain', 'gain'),
edge('sg65-fx-e3', 'sg65-fx-gain', 'out', 'sg65-fx-out', 'in'),
])},
},
});
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
const item = song.items.find((i) => i.id === ${JSON.stringify(cutItem.id)});
Object.assign(item, { fx: 'sg65-fx', tick: ${startAt}, length: ${8 * BAR} });
song.tracks[0].automation = [
{ target: 'param:cut', min: -40, max: 0, points: [{ tick: 0, value: -40 }, { tick: ${16 * BAR}, value: 0 }] },
];`);
const cutDb = (ticks) => -40 + (40 * ticks) / (16 * BAR);
const readCut = () =>
json(`({
voices: __gm.sound.levels(${JSON.stringify(blip?.file ?? '')}).filter((knobs) => knobs.length > 0),
at: __gm.song.position()?.ticks,
clip: __gm.song.clips().filter((c) => c.fx === 'sg65-fx').at(-1) ?? null,
})`);
const cutFirst = await waitFor(readCut, (r) => r.voices.length > 0 && r.at > startAt + 0.25 * BAR, 40);
await sleep(1500);
const cutLater = await readCut();
const offCut = (r) => Math.abs(20 * Math.log10(r.voices[0]?.at(-1) ?? 0) - cutDb(r.at));
check(
'a lane of a parameter starts a clip at its value there and takes its knob along while it plays',
[cutFirst, cutLater].every((r) => r.voices.length === 1 && offCut(r) < 1) &&
Math.abs(cutFirst.clip?.params?.cut - cutDb(startAt)) < 0.01 &&
cutLater.at - cutFirst.at > 600 &&
20 * Math.log10(cutLater.voices[0].at(-1) / cutFirst.voices[0].at(-1)) > 1,
JSON.stringify({ startAt, cutFirst, cutLater, off: [cutFirst, cutLater].map(offCut) }),
);
await mutate(`
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
const at = song.items.findIndex((i) => i.id === ${JSON.stringify(cutItem.id)});
song.items[at] = ${itemBefore};
delete song.tracks[0].automation;
__gm.state.project.assets = __gm.state.project.assets.filter((a) => a.id !== 'sg65-fx');
${loopBefore ? `song.loop = ${JSON.stringify(loopBefore)};` : 'delete song.loop;'}`);
await click(...Object.values(await json("__gm.song.headAt(0, 'mute')")));
// Mute closes every send of the track (#350); its own level stays as it is set.
const closed = (list) => {
@@ -633,8 +936,17 @@ export async function song(t) {
{ ...base, id: 'sg65-i1', track: song.tracks[0].id, kind: 'sound', asset: ${JSON.stringify(SOUND)}, tick: 0, length: ${BAR}, offset: 0 },
{ ...base, id: 'sg65-i2', track: song.tracks[1].id, kind: 'pattern', asset: ${JSON.stringify(PATTERN)}, tick: ${BAR}, length: ${BAR}, offset: 0 },
];
song.loop = { start: ${BAR}, end: ${2 * BAR} };`);
song.loop = { start: ${BAR}, end: ${2 * BAR} };
song.markers = [
{ id: 'sg65-mark', name: 'Round', tick: ${BAR} },
{ id: 'sg65-half', name: 'Half', tick: ${BAR + BAR / 2} },
];`);
await showTab('scene');
// 65k2 hears it through these (#338): each beat logs bar, beat, how late it is told and where
// Song position stands then, each marker its name.
const joined = (key, a, b, x) => [
node(key, 'Join', x, 400, { ...(a !== null && { a }), ...(b !== null && { b }) }),
];
const rig = await scaffoldBehavior(t, {
assetId: 'sg65-logic',
name: 'Music 65',
@@ -647,10 +959,60 @@ export async function song(t) {
node('sg65-play', 'PlaySong', 260, 0, { song: id, volume: 0.8, fadeIn: 1, loop: true }),
node('sg65-s', 'OnKeyDown', 0, 200, { key: 's' }),
node('sg65-stop', 'StopSong', 260, 200, { song: id, fadeOut: 1 }),
node('sg65-beat', 'OnBeat', 0, 400, { song: id }),
node('sg65-pos', 'SongPosition', 0, 600, { song: id }),
...joined('sg65-j1', 'beat ', null, 200),
...joined('sg65-j2', null, '.', 400),
...joined('sg65-j3', null, null, 600),
...joined('sg65-j4', null, ' ', 800),
...joined('sg65-j5', null, null, 1000),
...joined('sg65-j6', null, ' ', 1200),
...joined('sg65-j7', null, null, 1400),
node('sg65-log', 'Log', 1600, 400),
node('sg65-marker', 'OnSongMarker', 0, 800, { song: id }),
node('sg65-jm', 'Join', 200, 800, { a: 'marker ' }),
node('sg65-logm', 'Log', 400, 800),
// 65k3 (#338): on a beat with W held, Switch song to "Half" on the next bar; with V held, the
// track of the pattern turned down to silence there.
node('sg65-cue', 'OnBeat', 0, 1000, { song: id }),
node('sg65-w', 'KeyIsDown', 0, 1200, { key: 'w' }),
node('sg65-wif', 'Branch', 260, 1000),
node('sg65-switch', 'SwitchSong', 520, 1000, { marker: 'sg65-half', quantize: 'bar', fade: 0 }),
node('sg65-v', 'KeyIsDown', 0, 1400, { key: 'v' }),
node('sg65-vif', 'Branch', 520, 1200),
node('sg65-turn', 'SetSongTrack', 780, 1200, {
track: tracks[1],
volume: -60,
fade: 0,
quantize: 'bar',
}),
],
edges: [
edge('sg65-e3', 'sg65-start', 'exec', 'sg65-play', 'exec'),
edge('sg65-e4', 'sg65-s', 'exec', 'sg65-stop', 'exec'),
edge('sg65-b1', 'sg65-beat', 'bar', 'sg65-j1', 'b'),
edge('sg65-b2', 'sg65-j1', 'text', 'sg65-j2', 'a'),
edge('sg65-b3', 'sg65-j2', 'text', 'sg65-j3', 'a'),
edge('sg65-b4', 'sg65-beat', 'beat', 'sg65-j3', 'b'),
edge('sg65-b5', 'sg65-j3', 'text', 'sg65-j4', 'a'),
edge('sg65-b6', 'sg65-j4', 'text', 'sg65-j5', 'a'),
edge('sg65-b7', 'sg65-beat', 'late', 'sg65-j5', 'b'),
edge('sg65-b8', 'sg65-j5', 'text', 'sg65-j6', 'a'),
edge('sg65-b9', 'sg65-j6', 'text', 'sg65-j7', 'a'),
edge('sg65-b10', 'sg65-pos', 'fraction', 'sg65-j7', 'b'),
edge('sg65-b11', 'sg65-beat', 'exec', 'sg65-log', 'exec'),
edge('sg65-b12', 'sg65-j7', 'text', 'sg65-log', 'message'),
edge('sg65-m1', 'sg65-marker', 'name', 'sg65-jm', 'b'),
edge('sg65-m2', 'sg65-marker', 'exec', 'sg65-logm', 'exec'),
edge('sg65-m3', 'sg65-jm', 'text', 'sg65-logm', 'message'),
edge('sg65-c1', 'sg65-cue', 'exec', 'sg65-wif', 'exec'),
edge('sg65-c2', 'sg65-w', 'down', 'sg65-wif', 'condition'),
edge('sg65-c3', 'sg65-wif', 'then', 'sg65-switch', 'exec'),
edge('sg65-c4', 'sg65-cue', 'playback', 'sg65-switch', 'playback'),
edge('sg65-c5', 'sg65-wif', 'else', 'sg65-vif', 'exec'),
edge('sg65-c6', 'sg65-v', 'down', 'sg65-vif', 'condition'),
edge('sg65-c7', 'sg65-vif', 'then', 'sg65-turn', 'exec'),
edge('sg65-c8', 'sg65-cue', 'playback', 'sg65-turn', 'playback'),
],
},
});
@@ -701,11 +1063,124 @@ export async function song(t) {
resumed === 'running',
JSON.stringify({ heldClock, heldLater, resumed }),
);
// 65k2. The game follows the song (#338): On beat runs on every beat as it is heard, 0.5 s apart
// at 120 BPM, bar 2 again after bar 2's last beat as the loop goes round; On song marker
// runs each time round; the pause above held them with the clock. "late" is how long ago
// the beat was heard, and Song position, read then, stands that far into the beat.
const heardLines = async () =>
(await json('__gm.console.lines()'))
.map((line) => line.text)
.filter((text) => /(beat|marker) /.test(text));
const heardSome = await waitFor(
heardLines,
(lines) => lines.filter((l) => l.includes('marker Round')).length >= 2,
80,
);
const beats = heardSome.flatMap((text) => {
const m = /beat (\d+)\.(\d+) ([-\d.e]+) ([-\d.e]+)/.exec(text);
return m ? [{ bar: +m[1], beat: +m[2], late: +m[3], fraction: +m[4] }] : [];
});
const counted = beats.map((b) => `${b.bar}.${b.beat}`);
const expected = ['1.1', '1.2', '1.3', '1.4', '2.1', '2.2', '2.3', '2.4', '2.1', '2.2', '2.3', '2.4'];
const markersHeard = heardSome.filter((text) => text.includes('marker Round'));
check(
'On beat runs on every beat of the song as heard, round the loop again from bar 2, none lost to the pause; On song marker each round',
counted.length >= 9 &&
counted.every((c, i) => c === expected[i]) &&
markersHeard.length >= 2 &&
beats.every((b) => b.late >= 0 && b.late <= 0.15 && Math.abs(b.fraction * 0.5 - b.late) < 0.02),
JSON.stringify({ beats, markersHeard: markersHeard.length }),
);
// 65k3. Music follows the game (#338): W held over a beat asks Switch song for "Half", bar 2 beat 3,
// on the next bar. The playing keeps its number on buses of its own, heard from that bar line
// to the sample; On beat starts over from the marker. V held turns the pattern's track down to
// silence on the bar after, on its input, its level as the song has it.
const clockNow = () => json('__gm.sound.clock()');
const firstPlaying = (await playbacks())[0];
const linesBefore = (await heardLines()).length;
await t.holdKey('w', 600);
const pair = await waitFor(playbacks, (list) => list.length === 2, 20);
const landed = await waitFor(
playbacks,
(list) => list.length === 1 && list[0].startedAt !== firstPlaying?.startedAt,
40,
);
const switchedTo = landed[0];
const beatsSince = (lines) =>
lines.slice(linesBefore).flatMap((text) => {
const m = /\bbeat (\d+)\.(\d+) /.exec(text);
return m ? [`${m[1]}.${m[2]}`] : [];
});
// Only the jump to the marker hears 2.3 right after 2.4: the loop goes from 2.4 to 2.1.
const jumpIn = (beats) => beats.findIndex((b, i) => b === '2.4' && beats[i + 1] === '2.3') + 1;
const sinceSwitch = beatsSince(
await waitFor(
heardLines,
(lines) => {
const beats = beatsSince(lines);
const at = jumpIn(beats);
return at > 0 && beats.length > at + 1;
},
40,
),
);
const fromHalf = jumpIn(sinceSwitch) || -1;
const bars = switchedTo && firstPlaying ? (switchedTo.startedAt - firstPlaying.startedAt) / 2 : NaN;
check(
'Switch song plays the song from its marker on the next bar line, to the sample, under the same number; On beat goes on from the marker',
pair.length === 2 &&
pair.every((p) => p.id === firstPlaying?.id) &&
switchedTo?.id === firstPlaying?.id &&
Math.abs(bars - Math.round(bars)) < 1e-6 &&
Math.round(bars) >= 1 &&
fromHalf >= 0 &&
sinceSwitch[fromHalf + 1] === '2.4',
JSON.stringify({
firstPlaying,
pair: pair.map((p) => [p.id, p.startedAt]),
switchedTo,
bars,
sinceSwitch,
}),
);
const newMaster = (await json('__gm.song.buses()')).find((b) => /^play:\d+$/.test(b.name))?.name;
const pattern = `${newMaster}:${busOf(tracks[1])}`;
const turnedOf = async () => {
const at = await clockNow();
const bus = (await json('__gm.song.buses()')).find((b) => b.name === pattern);
return { at, turned: bus?.turned ?? 1, gain: bus?.gain };
};
// Read from before V on until it is silent: a cut, so 1 up to a bar line and 0 from it, nothing
// between. The switched song started on beat 3 of bar 2, so its bar lines lie 1 s after it, then
// every 2 s.
const turns = [await turnedOf()];
await t.keyDown('v');
const silent = await waitFor(
async () => (turns.push(await turnedOf()), turns.at(-1)),
(r) => r.turned === 0,
40,
);
await t.keyUp('v');
const lastOn = turns.findLast((r) => r.turned === 1);
const firstBar = (switchedTo?.startedAt ?? 0) + 1;
const barLine = lastOn && firstBar + Math.max(0, Math.ceil((lastOn.at - 0.05 - firstBar) / 2)) * 2;
check(
'Set song track turns the pattern’s track to silence on the next bar line, its own level left as the song has it',
newMaster !== `play:${firstPlaying?.id}` &&
silent.turned === 0 &&
turns.every((r) => r.turned === 1 || r.turned === 0) &&
barLine !== undefined &&
silent.at >= barLine - 0.05 &&
silent.at < barLine + 0.5 &&
Math.abs(silent.gain - 10 ** (-6 / 20)) < 1e-6,
JSON.stringify({ newMaster, barLine, turns }),
);
await press('s');
const fading = await waitFor(playbacks, (list) => list[0]?.fading !== null);
const fadedOut = await waitFor(playbacks, (list) => list.length === 0, 40);
await sleep(100);
const leftBuses = (await json('__gm.song.buses()')).filter((b) => b.name.startsWith(master));
// Every playing's, the one 65k3 switched to among them.
const leftBuses = (await json('__gm.song.buses()')).filter((b) => /^play:/.test(b.name));
await stop();
check(
'Stop song fades it out over its second, playing on meanwhile, then it and its buses are gone',
+63
View File
@@ -637,6 +637,69 @@ export async function mcp(t) {
}),
);
// 71h4. What the agent hears (#436): the sound of 71g9 loops twice from the start of the game, once
// on the master and once on a bus at volume 0. Held at a game time, get_runtime names both voices,
// the master is heard, the quiet bus is silent, and the levels stay as they are while it is paused.
const noisy = await callTool('run_commands', {
label: 'noise',
steps: [
{ command: 'add_file_asset', args: { type: 'sound', file: wav.file }, as: 'tone' },
{ command: 'create_asset', args: { type: 'bus', name: `${TOKEN_NAME} quiet` }, as: 'quiet' },
{ command: 'set_asset_property', args: { asset: '$quiet', key: 'volume', value: 0 } },
{ command: 'create_asset', args: { type: 'behavior', name: `${TOKEN_NAME} noise` }, as: 'noise' },
{ command: 'place_asset', args: { in: sceneId, asset: '$noise', x: 0, y: 0, parent: playerId } },
{ command: 'add_node', args: { asset: '$noise', node: 'OnStart' }, as: 'start' },
{
command: 'add_node',
args: { asset: '$noise', node: 'PlaySound', params: { sound: '$tone', loop: 'true' } },
as: 'loud',
},
{
command: 'add_node',
args: {
asset: '$noise',
node: 'PlaySound',
params: { sound: '$tone', loop: 'true', bus: '$quiet' },
},
as: 'hushed',
},
...['$loud', '$hushed'].map((node) => ({
command: 'connect',
args: { asset: '$noise', from: { node: '$start', port: 'exec' }, to: { node, port: 'exec' } },
})),
],
});
await settle();
const quietName = `${TOKEN_NAME} quiet`;
const heldAt = await callTool('play', { action: 'play', until: 0.6 });
const heardFirst = (await callTool('get_runtime', { console: 0 }))?.structuredContent?.audio;
await sleep(400);
const heardLater = (await callTool('get_runtime', { console: 0 }))?.structuredContent?.audio;
await callTool('play', { action: 'stop' });
await waitUntil('__gm.state.mode === "edit"', { label: 'the noisy game stopped' });
await callTool('undo');
await settle();
const levelOf = (audio, bus) => audio?.levels?.find((row) => row.bus === bus);
const toneVoices = (heardFirst?.voices ?? []).filter((voice) => voice.sound !== undefined);
check(
'get_runtime says what is heard: both voices by sound and bus, the master loud, a bus at 0 silent, held while paused',
noisy?.isError === false &&
heldAt?.structuredContent?.paused === true &&
toneVoices.length === 2 &&
toneVoices.some((voice) => voice.bus === 'master') &&
toneVoices.some((voice) => voice.bus === quietName) &&
heardFirst.window > 0.6 &&
levelOf(heardFirst, 'master')?.peak > -12 &&
levelOf(heardFirst, quietName)?.silent === true &&
JSON.stringify(heardLater?.levels) === JSON.stringify(heardFirst?.levels),
JSON.stringify({
text: noisy?.content?.[0]?.text?.slice(0, 300),
heldAt: heldAt?.structuredContent ?? heldAt?.content,
heardFirst,
laterLevels: heardLater?.levels,
}),
);
// 71i. One Ctrl+Z of the user takes the agent's whole step back.
await press('z', { ctrl: true });
await settle();
+3 -1
View File
@@ -56,7 +56,9 @@ const INSTRUCTIONS = [
'The editor follows your edits: after run_commands the user sees the graph, scene or asset it worked',
'on, with what it touched selected. Use open and select to show anything else.',
'Test what you built yourself: play, then play_input sends keys and mouse clicks into the game,',
'and get_runtime reads where the objects are, their variables and the console. To look at one',
'and get_runtime reads where the objects are, their variables and the console, and under audio',
'what you would hear: the voices sounding and the level of each bus, silent where nothing reaches',
'it. Check there that music and sounds are really heard. To look at one',
'moment, play with until holds the game at that game time, as does a last play_input step',
'{"pause": true}; a call that only reads comes too late for it, the game runs on meanwhile. A game needs a',
'camera in its scene to show anything; get_view says when it has none.',