Compare commits

...
8 Commits
Author SHA1 Message Date
adminandClaude Opus 5.5 56425fca6a Let a song carry named markers on a strip in the song tab (#338)
feuerwerk / gate (push) Successful in 5m22s
feuerwerk / suite (push) Failing after 19m54s
Stage 1 of #338. Game sync needs named places in a song: an event when
the playhead passes one, and a switch that jumps to one. This stage
adds the markers themselves. Games do nothing with them yet.

- A song has markers, each with id, name and tick. A song without
  markers stores nothing for them. Two markers may share tick and
  name, since nodes will name them by id.
- Markers change neither the sound nor the bake fingerprint.
- The song tab draws a strip under the ruler. A marker has a green
  flag with its name and a faint line through the tracks. Tracks and
  lanes sit 18 px lower. The ruler of the pattern tab is unchanged.
- Double click on an empty place adds a marker on the grid, with Alt
  or Shift on the exact tick. Dragging moves it in one undo step.
- A click on a marker sets the start of playback there. A double click
  renames it in the name dialog of the hierarchy groups.
- The context menu offers Play from here, Rename and Delete marker,
  on an empty place Add marker. A tooltip shows the full name. The
  wheel over the strip zooms time like the ruler.
- New agent commands add_song_marker, change_song_marker and
  remove_song_marker. get_asset lists them under markers.

Tests:
- song-markers in core 6 and editor 4, 1 agent test, with the touched
  song, agent and MCP tests 175/175
- 9 counter-probes, all compile, all red
- e2e 65 on atx: ALL PASSED, 70 checks, 3 new in 65g3. Counter-probe
  where a dragged marker is not taken over: 65g3 red
- gate on atx: CHECK EXIT 0, 332 files, 5093 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 18:44:58 +02:00
adminandClaude Opus 5.5 2099511695 Let the agent set automation lanes of a song track (#337)
feuerwerk / gate (push) Successful in 6m3s
feuerwerk / suite (push) Successful in 20m14s
Stage 4 of #337. Lanes could only be drawn in the song tab. Now the
agent can set them in one step and sees which parameters a lane of a
track can move.

- New command set_song_lane. The target is named as it is stored:
  volume, pan or param:<name>. All points come at once, each with
  tick, value and ease, from the same eases as animation keys,
  curve assets included.
- Points on one tick keep the order they come in, which makes a jump.
  points [] removes a volume or pan lane, remove a parameter lane.
- It refuses with a clear message: a value outside the range, an
  unknown ease, a missing curve asset, a range for volume or pan, and
  a new lane for a parameter nothing on the track has. The message
  then names the parameters there are.
- Without min and max a new parameter lane gets the range the dialog
  in the song tab offers.
- get_asset names under params of each song track the parameters a
  lane can move, with their default. The lanes were already in it.

Live in the user's project: get_asset named both parameters of their
two lanes (frequency, env) with defaults 20000 and 0.5. Playing to
3 s and 4.2 s, both tracks were heard, the audio clock running.

Tests:
- agent-music 13/13 (2 new), agent and MCP tests 129/129
- 6 counter-probes, all compile, all red
- gate on atx: CHECK EXIT 0, 330 files, 5082 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 10:54:51 +02:00
adminandClaude Opus 5.5 b50c7dbc35 Let a song track automate parameters of its sounds and FX (#337)
Stage 3 of #337. Lanes moved volume and pan only. Now a lane can
steer a parameter of anything on its track, and a voice already
planned or still waiting for its file follows it too.

- A lane may aim at param:<name>. A parameter has no range of its
  own, so the lane keeps min and max. A parameter lane stays without
  points, so its range is not lost. Remove lane takes it away.
- It reaches the track FX, the FX of an item, the instrument of a
  note with its own FX, and the FX of a clip. When several have the
  name, all of them change, as with Set sound FX parameters.
- The scheduler hands the lane value to a note or clip at its start.
  Values a voice only reads at its start act from the next note.
- While a voice sounds, the knobs of buses and voices of the track
  follow the lane on the audio clock, read every 10 ms, because a
  knob is not always the parameter itself (trackParams.ts).
- A voice waiting for its file takes the marks when it starts.
- When a lane is removed, the parameter goes back to its default.
  An edit while playing does not reset a parameter a lane rules.
- This holds in the song tab, for Play song, in compositions and in
  the bake.
- Song tab: Add parameter lane in the track head menu opens a dialog
  for parameter and range (ParamLaneDialog.ts). The lane head menu
  has Range and Remove lane.
- Inspector: while a lane rules a value, a hint at the track names
  the lanes and says the fields act once the lane is gone.

Known limit: a knob that depends on two automated parameters reckons
each lane with the last value of the other, so it may lag one window.

Tests:
- automation-params.test.ts (9, new), 3 new in sound-library and 3
  in song-lanes. Affected files 407/407.
- 17 counter checks, all compile, all red.
- e2e 65 on melektaus-atx: ALL PASSED, 67 checks. 65h3 is new: a
  clip with FX parameter cut starts at the lane value and its knob
  stays within 0.04 dB of the lane. It goes red when automateBus
  does not pass parameters on.
- Gate on melektaus-atx: CHECK EXIT 0, 330 files, 5080 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 10:30:42 +02:00
adminandClaude Opus 5.5 bd99f9e79d Let automation lanes of a song track be drawn in the song tab (#337)
Stage 2 of #337. The lanes of stage 1 played, but nobody could make
one. Now a track folds out a lane for volume and one for pan, and
points are set, dragged and eased right on the timeline.

- A lane is a row of its own, as high as a track, below its track.
  The view functions stay row based, so zoom, fit, item drags and
  the composition (which shares them) are unchanged.
- itemAt and itemsWithin take a map of track id to row.
- A triangle in the track head folds the lanes out. The head menu
  shows or hides each lane. A lane with points is open by itself.
- Which lanes are open belongs to the view of the tab and is saved
  with it. It is not part of the song and makes no undo step.
- A click sets a point on the grid, Alt or Shift on the exact tick.
  Press and drag sets one and drags it in a single undo step.
- A dragged point stops at its neighbour. Two points on one tick
  make a jump. Double click or Delete removes a point.
- The value shows next to a dragged point and as a tooltip.
- Right click on a point opens "Ease to next point" with the same
  choice as animation keys, curve assets included.
- The lane head shows the value at the playhead. Its menu clears
  or hides the lane.
- While a lane rules a value, the fader shows it in the lane colour
  and cannot be dragged (decision 4). A hint says why.
- Geometry in laneView.ts, edit rules in laneEdit.ts, the easing
  menu in easingMenu.ts. New test hooks automationAt,
  automationHeadAt and automation under __gm.song.

Open: the Volume (dB) field of a track in the inspector can still be
changed while a lane rules it. It has no effect while playing.

Tests:
- song-lanes.test.ts (12, new) and song-edit adapted: 27/27.
  Editor tests of song and composition 62/62.
- 9 counter checks, all compile, all red.
- e2e 65 on melektaus-atx: ALL PASSED, 66 checks. 65g2 is new with
  6 checks. It goes red when a set point is not taken on release.
- Gate on melektaus-atx: CHECK EXIT 0, 329 files, 5065 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 09:31:48 +02:00
adminandClaude Opus 5.5 389a7c0fd1 Let volume and pan of a song track follow automation lanes (#337)
A song track can now change its volume and pan over time, played on
the audio clock wherever the song is heard. This is stage 1 of #337:
the model and the playback. Lanes cannot be drawn in the song tab yet.

- A track holds automation lanes for volume (dB) and pan. Each point
  has a tick, a value and an easing from the shared list, curve assets
  included. Two points on one tick make a jump. Before the first point
  its value holds, after the last point the last value holds.
- The rules are pure functions in core/music/automation.ts: the value
  at a tick and the pieces of a lane inside one scheduling window.
- The scheduler hands those pieces to the host in every window. The
  host sets them as ramps or jumps on the gain and panner of the
  track's bus. Shapes the clock cannot ramp directly (dB as gain,
  curved easings) are sampled every 10 ms. At the loop end the value
  jumps back with the playhead.
- This works in the song tab, for Play song, inside a composition
  (across the song loop, on the item's own bus) and in a bake.
- While a lane has points the fader no longer sets that value. When
  the lane goes, the fader takes over from the current value.
- Automation enters the bake fingerprint only when there is some, so
  existing bakes stay valid.
- The level helpers moved to core/music/level.ts, so song.ts and
  automation.ts do not import each other.

Known gap: a delaying sound FX on a bus after the track (a pitch
shifter) makes the automation late by that delay.

Tests:
- automation.test.ts (13, new), 3 new in sound-library.test.ts.
  Song, bake and composition tests together 286/286.
- 12 counter-checks, all compile, all red.
- e2e 65 on atx: ALL PASSED, 60 checks. New 65h2: in the song tab the
  bus level follows a volume lane, 0.09 dB off, also with the fader at
  -60 dB. Red when the song tab does not pass the automation on.
- Gate on atx: 328 files, 5053 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-19 06:45:55 +02:00
adminandClaude Opus 5.5 b5e071ca68 Let the agent read what is heard in the running game (#436)
The agent has no ears. It could start music and sounds, but not tell
whether anything reached the speakers: a song without an instrument, a
bus at volume 0 or a browser that holds the audio clock all looked the
same from outside.

- get_runtime has a section audio. Under levels it gives the peak and
  RMS in dB of the master, each bus of the mix, each pattern, song or
  composition playing and the tracks of a song. Below -90 dB it says
  silent.
- The levels come from a tap of their own after the level and pan of
  each bus, an AnalyserNode per side as for the meters of the Song tab.
  It looks back about 0.7 s, so the gap between two notes is not
  silence. A pause holds the values, which fits play with until.
- Under voices it names what sounds now: a sound by its asset, a note
  of an instrument with its note name, each with its bus and object.
  Next to them planned, playing, waiting (bytes not here yet) and
  failed.
- Under clock it gives the state of the audio clock, with a hint when
  it stands still while the game runs. The state is told as it was
  before a pause, else the pause would hide that case.
- The buses are tapped only while the agent is switched on.
- The note now travels with a voice from the Start of a pattern or song
  to the SoundLibrary, so a voice can be named by it.
- tools.json and the relay's instructions tell the agent to check under
  audio that music and sounds are really heard.

Found on the user's project: started right after F5, before a click in
the page, the browser kept the audio clock suspended for the whole run.
Nothing was heard and the notes stayed planned. That is what clock now
shows.

Tests:
- Unit: sound-library (taps, levels, voices, clock), agent-play
  (audioView). Pattern and song tests expect the note in their timing.
- 9 counter-checks, 8 red. Not dropping a tap with its bus stays green,
  as the bus is cut off and gone anyway.
- e2e 71 (melektaus-atx): 28/28. New 71h4: a sound on the master and
  one on a bus at volume 0, both voices named, the master loud, the bus
  silent, the values held in the pause.
- Gate (melektaus-atx): CHECK EXIT 0, 327 files, 5037 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 22:14:07 +02:00
adminandClaude Opus 5.5 dd2096e0df Make a bus of the mix at its level, so a quiet bus starts quiet
A bus of the project's mix was made at unity and then glided to its
level. Where the audio context comes with the first sound of a run,
that sound already plays on the new bus. On a bus at volume 0 it was
heard for a moment, loud, while the level glided down.

- setMix now sets the level of a bus it makes at once, without a
  glide. A bus that is there already glides to a new level as before.
- Found while building #436: the e2e check with a sound on a bus at
  volume 0 measured -7.8 dB on that bus. With the fix it is silent.

Tests:
- New unit test in sound-library: buses made at their level with no
  glide, a later change glides. Red without the fix.
- sound-library 130/130, tsc clean on this change alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 22:13:48 +02:00
adminandClaude Opus 5.5 70402aa19a Let the agent hold the game at a given game time (#435)
feuerwerk / gate (push) Successful in 5m31s
feuerwerk / suite (push) Successful in 19m9s
The agent could not look at a short moment of a running game. Its
calls took longer than the moment lasted, so an intro window from
0.5 to 2 s slipped past twice in the 5d test. Now the editor itself
stops the clock at the time the agent names.

- play takes until: play, resume or pause let the game run on to
  that game time and hold it there. The call answers once the game
  is paused, so get_runtime and screenshot show that moment.
- The frame that would step past the time is cut short
  (frameWithin in src/core/time.ts, used in PlaySession). The game
  stands at 0.5 s, not one frame later.
- play_input takes a last step {"pause": true}. After a wait it
  holds the game at the time of the wait, after a key or click at
  once. Its answer names the game time and whether it is paused.
- A pause that comes first (toolbar or a Break node), a Load scene
  or a Restart game call the until off, and the answer says so.
- When the clock stands still, the call gives up after 25 s with
  the time it is at. A tab in the background is refused at once.
- A recording that does not run in real time is not cut. There the
  game stops in the first frame that reaches the time.
- get_runtime and play_input give times to the millisecond, not to
  the hundredth, so 0.505 no longer reads as 0.51.
- The relay's instructions name both ways.

Tests:
- Unit agent-play (6 new) and time (2 new): 19/19, related editor
  and relay tests 1158/1158.
- 6 counter-checks, all compile, all red. One stayed green at
  first: 0.3 s is exactly 18 frames. The test now takes 0.31 s.
- e2e 71 (atx): ALL PASSED, 27 checks. New 71h3 holds at 0.505 s
  and 0.8 s and after a wait of 0.31 s. Without the cut in
  PlaySession it went red at 0.5166 s.
- Gate (atx): CHECK EXIT 0, 327 files, 5033 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-18 21:26:53 +02:00
56 changed files with 5404 additions and 194 deletions

No files matched your search

+31
View File
@@ -8,6 +8,7 @@ import type { EdgeSend, Graph } from '../graph/Graph';
import type { Asset, SongAsset } from '../model/Asset';
import { findGraphAsset, type Project } from '../model/Project';
import { settleEmbedded } from '../music/embedded';
import { heldSongMarkers, type SongMarker } from '../music/markers';
import { routedKind, sendsTapped } from '../music/projectRouting';
import { heldSend, heldSongGraph } from '../music/routing';
import { heldSongItems, heldSongTracks, type SongItem, type SongTrack } from '../music/song';
@@ -75,6 +76,36 @@ export class SetSongCommand implements Command {
}
}
/** Writes the markers of a song whole (#338): one gesture, one step to undo. */
export class SetSongMarkersCommand implements Command {
private previous: SongMarker[] | undefined;
constructor(
private readonly assetId: string,
private readonly markers: readonly SongMarker[],
readonly label = 'Edit markers',
) {}
/** Nothing to do when the song already has exactly these markers – no empty undo step. */
static changes(asset: SongAsset, markers: readonly SongMarker[]): boolean {
return JSON.stringify(asset.markers ?? []) !== JSON.stringify(heldSongMarkers(markers));
}
execute(project: Project): void {
const asset = requireSong(project, this.assetId);
this.previous = asset.markers;
const held = heldSongMarkers(this.markers);
if (held.length > 0) asset.markers = held;
else delete asset.markers;
}
undo(project: Project): void {
const asset = requireSong(project, this.assetId);
if (this.previous) asset.markers = this.previous;
else delete asset.markers;
}
}
/**
* How loud a link of a song's routing sends, and where it taps its track (#350); the same for a link
* of the master routing (#351), where only a bus and a song may tap anywhere but after their level.
+3
View File
@@ -1,5 +1,6 @@
import type { Graph } from '../graph/Graph';
import type { Bake } from '../music/bake';
import type { SongMarker } from '../music/markers';
import type { PatternNote } from '../music/pattern';
import type { SongItem, SongTrack } from '../music/song';
import type { LoopRange } from '../music/sequence';
@@ -490,6 +491,8 @@ export interface SongAsset extends AssetBase {
tracks: SongTrack[];
items: SongItem[];
loop?: LoopRange;
/** Named spots on its time (#338); left out without one. */
markers?: SongMarker[];
graph: Graph;
bake?: Bake;
}
+38
View File
@@ -1,4 +1,6 @@
import type { FadeCurve } from '../model/sound';
import type { FxParams } from '../soundfx/nodes';
import { AUTOMATION_STEP, type AutomationTarget, type LaneMark, laneMarks, paramsAt } from './automation';
import { clipsIn, heldLoop, levelsIn, type LoopRange, notesIn, playheadAt, type Sequence } from './sequence';
import {
COMMON_TIME,
@@ -30,6 +32,8 @@ export interface ScheduledNote {
/** A factor on its loudness, and the track of a song it is heard on (#334). */
volume?: number;
bus?: string;
/** The values the parameter lanes of its track have where it starts (#337). */
params?: FxParams;
}
/** One clip handed to the player (#334): when it starts, how long it plays and from where in the sound. */
@@ -43,6 +47,8 @@ export interface ScheduledClip {
volume: number;
bus: string;
tempo: number;
/** As a note's. */
params?: FxParams;
}
/**
@@ -59,6 +65,18 @@ export interface ScheduledRamp {
done: number;
}
/**
* Where a lane of automation (#337) takes a bus over one window: each mark a second of the audio
* clock with the value the bus is ramped to, or jumps to there. With `fresh` the playing has just
* started, and what was planned for the bus from the first mark on is to go.
*/
export interface ScheduledAutomation {
bus: string;
target: AutomationTarget;
marks: LaneMark[];
fresh: boolean;
}
/** What the scheduler needs of the audio side; a test hands in a clock of its own. */
export interface SchedulerHost {
/** Seconds on the audio clock. */
@@ -77,6 +95,8 @@ export interface SchedulerHost {
playClip?(clip: ScheduledClip): number;
/** Ramps the level of a bus; a host without levels leaves it out. */
ramp?(ramp: ScheduledRamp): void;
/** Takes a bus along its automation; a host without it leaves it out. */
automate?(automation: ScheduledAutomation): void;
playing(voice: number): boolean;
/** Ends these voices; with `letRing` a note falls away in the release of its Envelope. */
stop(voices: readonly number[], letRing?: boolean): void;
@@ -206,7 +226,9 @@ export class Scheduler {
if (to <= this.until) return;
this.voices = this.voices.filter((voice) => this.host.playing(voice));
const sequence = this.sequence();
const lanes = sequence.lanes ?? [];
for (const due of notesIn(sequence, this.from, this.loop, this.until, to)) {
const params = paramsAt(lanes, due.bus, due.tick);
const voice = this.host.play({
at: this.anchor + secondsOfTicks(due.at, this.tempo),
duration: secondsOfTicks(due.length, this.tempo),
@@ -216,11 +238,13 @@ export class Scheduler {
tempo: due.tempo ?? this.tempo,
...(due.volume !== undefined && { volume: due.volume }),
...(due.bus !== undefined && { bus: due.bus }),
...(params && { params }),
});
if (voice !== 0) this.voices.push(voice);
}
if (this.host.playClip)
for (const due of clipsIn(sequence, this.from, this.loop, this.until, to, this.fresh)) {
const params = paramsAt(lanes, due.bus, due.tick + due.skip);
const voice = this.host.playClip({
at: this.anchor + secondsOfTicks(due.at, this.tempo),
duration: secondsOfTicks(due.length, this.tempo),
@@ -230,6 +254,7 @@ export class Scheduler {
volume: due.volume,
bus: due.bus,
tempo: due.tempo ?? this.tempo,
...(params && { params }),
});
if (voice !== 0) this.voices.push(voice);
}
@@ -246,6 +271,19 @@ export class Scheduler {
done: whole > 0 ? due.skip / whole : 0,
});
}
if (this.host.automate) {
const anchor = this.anchor;
const step = ticksOfSeconds(AUTOMATION_STEP, this.tempo);
for (const lane of lanes) {
const marks = laneMarks(lane, this.from, this.loop, this.until, to, this.fresh, step);
this.host.automate({
bus: lane.bus,
target: lane.target,
marks: marks.map((mark) => ({ ...mark, at: anchor + secondsOfTicks(mark.at, this.tempo) })),
fresh: this.fresh,
});
}
}
this.fresh = false;
this.until = to;
if (!this.loop && playheadAt(to, this.from, null) >= this.end) {
+414
View File
@@ -0,0 +1,414 @@
/**
* Automation of a song (#337): the volume and the pan of a track changing over the song, as points
* on a lane. From each point the value runs to the next along that point's easing, as from one
* animation key to the next. While a lane has points it rules its value in place of the fader: before
* its first point it is that point's value, after its last the last one's. Two points on one tick
* are a jump there.
*
* The scheduler reads a lane window by window as marks on the audio clock, each ramped or jumped to.
* Where the value moves along a shape the clock cannot ramp, it is read every `AUTOMATION_STEP`.
*
* A lane can also move a parameter (`param:<name>`): of the track's sound FX, of the sound FX of its
* items, of the instruments of its notes and the sound FX of its clips – wherever one of them has a
* parameter of that name, as `Set sound FX parameters` reaches them. A note or clip starts at the
* value its lane has there, so a parameter a voice only reads when it starts is heard from the next one.
*/
import type { KeyInterpolation, VectorPoint } from '../model/Asset';
import { easedShare, easingOf, isKeyInterpolation } from '../model/animation';
import type { CurveLookup } from '../model/curve';
import type { FxParams } from '../soundfx/nodes';
import { dbToGain, TRACK_MAX_DB, TRACK_MIN_DB } from './level';
import { type LoopRange, playheadAt } from './sequence';
/** What a lane always moves on a track: its fader and its pan. */
export type TrackTarget = 'volume' | 'pan';
/** A parameter by its name (#337, stage 3); no name of a parameter has a colon. */
export type ParamTarget = `param:${string}`;
export type AutomationTarget = TrackTarget | ParamTarget;
export const AUTOMATION_TARGETS: readonly TrackTarget[] = ['volume', 'pan'];
const PARAM_PREFIX = 'param:';
export function paramTarget(name: string): ParamTarget {
return `${PARAM_PREFIX}${name}`;
}
/** The parameter a target moves; null for the volume and the pan. */
export function paramOf(target: AutomationTarget): string | null {
return target.startsWith(PARAM_PREFIX) ? target.slice(PARAM_PREFIX.length) : null;
}
/** Whether a text is a target a lane can have: the volume, the pan or a parameter with a name. */
export function isAutomationTarget(value: unknown): value is AutomationTarget {
if (value === 'volume' || value === 'pan') return true;
if (typeof value !== 'string' || !value.startsWith(PARAM_PREFIX)) return false;
const name = value.slice(PARAM_PREFIX.length);
return name !== '' && !name.includes(':');
}
export interface AutomationPoint {
tick: number;
/** In the units of its target: decibels for the volume, -1 (left) to 1 (right) for the pan. */
value: number;
/** How it runs to the next point; linear where left out. */
ease?: KeyInterpolation;
}
export interface AutomationLane {
target: AutomationTarget;
/** In the order of their ticks. */
points: AutomationPoint[];
/**
* Only on a parameter's lane: the range its values are drawn and held in, `PARAM_RANGE` where left
* out. A parameter says nothing of its own range, so the lane does. Such a lane is kept without a
* point too, for its range; it rules its parameter once it has one.
*/
min?: number;
max?: number;
}
/** What says how a lane's values are held and drawn: its target, and a parameter's range. */
export type LaneKind = Pick<AutomationLane, 'target' | 'min' | 'max'>;
/** A lane's kind, or the volume or the pan by name: neither has a range of its own to name. */
export type LaneSpec = LaneKind | TrackTarget;
export function laneKind(spec: LaneSpec): LaneKind {
return typeof spec === 'string' ? { target: spec } : spec;
}
/** The range of a parameter's lane that sets none. */
export const PARAM_RANGE = { min: 0, max: 1 } as const;
/** Seconds between two readings of a value that moves along a shape the audio clock cannot ramp. */
export const AUTOMATION_STEP = 0.01;
/** The lowest and the highest value a lane of that kind holds. */
export function laneRange(spec: LaneSpec): { min: number; max: number } {
const { target, min, max } = laneKind(spec);
if (target === 'volume') return { min: TRACK_MIN_DB, max: TRACK_MAX_DB };
if (target === 'pan') return { min: -1, max: 1 };
return heldRange(min, max) ?? { ...PARAM_RANGE };
}
/** A range a parameter's lane may have: both ends finite, the top above the bottom; null otherwise. */
export function heldRange(min: unknown, max: unknown): { min: number; max: number } | null {
if (typeof min !== 'number' || typeof max !== 'number') return null;
if (!Number.isFinite(min) || !Number.isFinite(max) || !(max > min)) return null;
return { min, max };
}
/** A lane's value as its bus takes it: the volume as a gain, the pan and a parameter as they are. */
export function audioValueOf(lane: LaneKind, value: number): number {
const { min, max } = laneRange(lane);
const held = Math.max(min, Math.min(max, value));
return lane.target === 'volume' ? dbToGain(held) : held;
}
/**
* Held to what the lanes of a track may be: a target it knows, one lane for each, points in whole
* ticks from 0 with their values in range, in the order of their ticks. A lane of the volume or the
* pan without a point is dropped; a parameter's is kept for its range.
*/
export function heldLanes(raw: unknown): AutomationLane[] {
const lanes: AutomationLane[] = [];
for (const entry of Array.isArray(raw) ? (raw as unknown[]) : []) {
if (typeof entry !== 'object' || entry === null) continue;
const lane = entry as Record<string, unknown>;
const target = lane.target;
if (!isAutomationTarget(target) || lanes.some((l) => l.target === target)) continue;
const range = paramOf(target) !== null ? heldRange(lane.min, lane.max) : null;
const { min, max } = range ?? laneRange({ target });
const points = (Array.isArray(lane.points) ? (lane.points as unknown[]) : []).flatMap(
(point): AutomationPoint[] => {
if (typeof point !== 'object' || point === null) return [];
const { tick, value, ease } = point as Record<string, unknown>;
if (typeof tick !== 'number' || !Number.isFinite(tick) || !(tick >= 0)) return [];
if (typeof value !== 'number' || !Number.isFinite(value)) return [];
return [
{
tick: Math.round(tick),
value: Math.max(min, Math.min(max, value)),
...(isKeyInterpolation(ease) && ease !== 'linear' && { ease }),
},
];
},
);
points.sort((a, b) => a.tick - b.tick);
if (range) lanes.push({ target, points, ...range });
else if (points.length > 0 || paramOf(target) !== null) lanes.push({ target, points });
}
return lanes;
}
/** The parameters a track's lanes move, in their order, those without a point among them. */
export function laneParams(track: { automation?: readonly AutomationLane[] }): string[] {
return (track.automation ?? []).flatMap((lane) => paramOf(lane.target) ?? []);
}
/** The lane of a track for that target, while it has a point. */
export function laneOf(
track: { automation?: readonly AutomationLane[] },
target: AutomationTarget,
): AutomationLane | undefined {
return track.automation?.find((lane) => lane.target === target && lane.points.length > 0);
}
/** A lane on the bus it acts on, the Curve assets its points ease along looked up; one gone is linear. */
export function sequenceLaneOf(lane: AutomationLane, bus: string, curves: CurveLookup): SequenceLane {
return {
bus,
target: lane.target,
...(lane.min !== undefined && lane.max !== undefined && { min: lane.min, max: lane.max }),
points: lane.points.map((point) => {
const { shape, curve } = easingOf(point.ease ?? 'linear');
const found = shape === 'curve' ? curves(curve ?? '') : undefined;
return found ? { ...point, curve: found } : point;
}),
};
}
/** A point as a sequence plays it: a Curve asset it eases along already looked up. */
export interface SequenceLanePoint extends AutomationPoint {
curve?: readonly VectorPoint[];
}
/**
* A lane as a sequence plays it: on the bus it acts on, never without a point. A parameter's acts on
* that bus's sound FX and on everything heard on a bus inside it (`reaches`).
*/
export interface SequenceLane extends LaneKind {
bus: string;
points: readonly SequenceLanePoint[];
}
/** Whether a voice or a bus by that name is on the track of `bus` – it, or a bus inside it (`<bus>:…`). */
export function reaches(bus: string, name: string): boolean {
return name === bus || name.startsWith(`${bus}:`);
}
/**
* The values the parameter lanes of a sequence have at `tick` for what is heard on `bus`, a note's or
* a clip's; undefined where none reaches it.
*/
export function paramsAt(
lanes: readonly SequenceLane[],
bus: string | undefined,
tick: number,
): FxParams | undefined {
if (bus === undefined) return undefined;
let params: Record<string, number> | undefined;
for (const lane of lanes) {
const name = paramOf(lane.target);
if (name === null || !reaches(lane.bus, bus)) continue;
const { min, max } = laneRange(lane);
(params ??= {})[name] = Math.max(min, Math.min(max, laneValueAt(lane.points, tick)));
}
return params;
}
function between(a: SequenceLanePoint, b: SequenceLanePoint, t: number): number {
return a.value + (b.value - a.value) * easedShare(a.ease ?? 'linear', t, () => a.curve);
}
/** The index of the last point on or before `tick`, -1 for none. */
function lastAt(points: readonly SequenceLanePoint[], tick: number): number {
let i = -1;
while (i + 1 < points.length && points[i + 1]!.tick <= tick) i++;
return i;
}
/** The value at `tick`, a jump there already taken. */
export function laneValueAt(points: readonly SequenceLanePoint[], tick: number): number {
const i = lastAt(points, tick);
if (i < 0) return points[0]!.value;
const a = points[i]!;
const b = points[i + 1];
return b ? between(a, b, (tick - a.tick) / (b.tick - a.tick)) : a.value;
}
/** The value just before `tick`: what a jump there jumps from. */
export function laneValueBefore(points: readonly SequenceLanePoint[], tick: number): number {
let i = -1;
while (i + 1 < points.length && points[i + 1]!.tick < tick) i++;
if (i < 0) return points[0]!.value;
const a = points[i]!;
const b = points[i + 1];
if (!b) return a.value;
if (b.tick > tick) return between(a, b, (tick - a.tick) / (b.tick - a.tick));
return a.ease === 'step' ? a.value : between(a, b, 1);
}
/** Whether the way from `a` to `b` holds or is a straight line in lane units. */
function straight(a: SequenceLanePoint, b: SequenceLanePoint): boolean {
const ease = a.ease ?? 'linear';
return a.value === b.value || ease === 'step' || ease === 'linear';
}
/**
* Whether the bus takes it as one ramp: held, or a straight line also in its own units. A parameter
* becomes its knobs along ways of their own, so only one held is a ramp of them.
*/
function rampable(target: AutomationTarget, a: SequenceLanePoint, b: SequenceLanePoint): boolean {
const ease = a.ease ?? 'linear';
return a.value === b.value || ease === 'step' || (ease === 'linear' && target === 'pan');
}
/** One place on the way of a lane: `at` in ticks, the value as its bus takes it, ramped to or jumped to. */
export interface LaneMark {
at: number;
value: number;
jump: boolean;
}
/**
* Where a lane is over the playing time from `start` up to `end`, as `notesIn` lays out notes: a mark
* where the window starts, at every point and every reading in between, and where it ends. Where the
* playhead lands – at the start with `fresh`, after a jump back – the first mark of that stretch is a
* jump. `step` is `AUTOMATION_STEP` in ticks.
*/
export function laneMarks(
lane: SequenceLane,
from: number,
loop: LoopRange | null,
start: number,
end: number,
fresh: boolean,
step: number,
): LaneMark[] {
const { points } = lane;
const audio = (value: number) => audioValueOf(lane, value);
const marks: LaneMark[] = [];
const stretchEnd = loop !== null && from < loop.end ? loop.end : Infinity;
let elapsed = start;
let head = playheadAt(start, from, loop);
let landed = fresh;
while (elapsed < end) {
const span = Math.min(end - elapsed, stretchEnd - head);
if (!(span > 0)) break;
const to = head + span;
const first = laneValueAt(points, head);
marks.push({
at: elapsed,
value: audio(first),
jump: landed || laneValueBefore(points, head) !== first,
});
for (const tick of readings(lane, head, to, step)) {
const before = laneValueBefore(points, tick);
const value = laneValueAt(points, tick);
const at = elapsed + (tick - head);
marks.push({ at, value: audio(before), jump: false });
if (value !== before) marks.push({ at, value: audio(value), jump: true });
}
marks.push({ at: elapsed + span, value: audio(laneValueBefore(points, to)), jump: false });
elapsed += span;
landed = to >= stretchEnd;
head = landed ? loop!.start : to;
}
return thinned(marks);
}
/** The ticks strictly between `head` and `to` to mark: every point, and readings where the bus cannot ramp. */
function readings(lane: SequenceLane, head: number, to: number, step: number): number[] {
const { points, target } = lane;
const ticks: number[] = [];
points.forEach((a, i) => {
if (a.tick > head && a.tick < to) ticks.push(a.tick);
const b = points[i + 1];
if (b && b.tick > head && a.tick < to && !rampable(target, a, b))
ticks.push(...stepsOf(a.tick, head, Math.min(b.tick, to), step));
});
ticks.sort((x, y) => x - y);
return ticks.filter((tick, i) => i === 0 || tick !== ticks[i - 1]);
}
/** The ticks `step` apart from `origin` on, strictly between `after` and `before`; none without a step. */
function stepsOf(origin: number, after: number, before: number, step: number): number[] {
const ticks: number[] = [];
if (!(step > 0)) return ticks;
for (let k = Math.max(1, Math.floor((after - origin) / step) + 1); origin + k * step < before; k++)
if (origin + k * step > after) ticks.push(origin + k * step);
return ticks;
}
/** Without the marks a ramp would pass anyway: a value held from the mark before on. */
function thinned(marks: readonly LaneMark[]): LaneMark[] {
const kept: LaneMark[] = [];
marks.forEach((mark, i) => {
const last = kept.at(-1);
const next = marks[i + 1];
const held =
last !== undefined &&
next !== undefined &&
!mark.jump &&
last.value === mark.value &&
(next.jump || next.value === mark.value);
if (!held) kept.push(mark);
});
return kept;
}
/**
* A lane as it is heard over the playing time from `start` up to `end`, played from 0 round `loop` –
* a song in an item of a composition (#418) –, as a lane of its own in ticks of that playing time:
* each round with its points, cut at the loop end and jumping back there. A way cut in the middle
* that is no straight line is read every `step` ticks.
*/
export function unrolledLane(
points: readonly SequenceLanePoint[],
loop: LoopRange | null,
start: number,
end: number,
step: number,
): SequenceLanePoint[] {
const out: SequenceLanePoint[] = [];
const stretchEnd = loop !== null && loop.end > 0 ? loop.end : Infinity;
let elapsed = start;
let head = playheadAt(start, 0, loop);
while (elapsed < end) {
const span = Math.min(end - elapsed, stretchEnd - head);
if (!(span > 0)) break;
const shift = elapsed - head;
for (const point of stretchOf(points, head, head + span, step))
out.push({ ...point, tick: point.tick + shift });
elapsed += span;
head = head + span >= stretchEnd ? loop!.start : head + span;
}
return out;
}
/** The points of a lane from `head` to `to` as a lane by itself, ending on the value it reaches there. */
function stretchOf(
points: readonly SequenceLanePoint[],
head: number,
to: number,
step: number,
): SequenceLanePoint[] {
const out: SequenceLanePoint[] = [];
const at = lastAt(points, head);
const starts: { tick: number; own: number; whole: boolean }[] = [
{ tick: head, own: Math.max(0, at), whole: at >= 0 && points[at]!.tick === head },
];
points.forEach((point, i) => {
if (point.tick > head && point.tick < to) starts.push({ tick: point.tick, own: i, whole: true });
});
starts.forEach((s, k) => {
const next = starts[k + 1]?.tick ?? to;
const a = points[s.own]!;
const b = points[s.own + 1];
const value = s.whole ? a.value : laneValueAt(points, s.tick);
if (!b || (k === 0 && at < 0)) out.push({ tick: s.tick, value });
else if (s.whole && b.tick <= to) out.push({ ...a, tick: s.tick });
else if (straight(a, b)) out.push({ tick: s.tick, value, ...(a.ease === 'step' && { ease: 'step' }) });
else {
out.push({ tick: s.tick, value });
for (const tick of stepsOf(a.tick, s.tick, next, step))
out.push({ tick, value: laneValueAt(points, tick) });
}
});
out.push({ tick: to, value: laneValueBefore(points, to) });
return out;
}
+2
View File
@@ -233,6 +233,8 @@ export function bakePrint(project: Project, asset: BakedAsset): string {
clips,
buses,
instruments,
// Only where there is any, so a bake made before automation (#337) still holds.
...(sequence.lanes?.length ? [sequence.lanes] : []),
]);
return hashText(text);
}
+16 -2
View File
@@ -13,6 +13,7 @@ import { compositionFps, compositionLength, compositionLoops, itemFades } from '
import { findAssetOf, type Project } from '../model/Project';
import { findSound } from '../model/sound';
import { pickedFx } from '../properties/assetFx';
import { AUTOMATION_STEP, type SequenceLane, unrolledLane } from './automation';
import { itemRoute, MASTER_MIX, type MixRoute } from './mixer';
import { fxVoiceOf } from './needs';
import { patternInstrument, patternLength, patternSequence, patternTempo } from './pattern';
@@ -75,6 +76,7 @@ export function compositionPlayings(
(playing.sequence.notes as SequenceNote[]).push(...laid.notes);
(playing.sequence.clips as SequenceClip[]).push(...laid.clips);
(playing.sequence.levels as SequenceLevel[]).push(...laid.levels);
if (laid.lanes?.length) playing.sequence.lanes = [...(playing.sequence.lanes ?? []), ...laid.lanes];
playing.buses.push(...laid.buses);
}
const playings = [...byRoute.values()];
@@ -100,6 +102,8 @@ interface Laid {
clips: SequenceClip[];
/** Its fades, on its bus. */
levels: SequenceLevel[];
/** A song's automation (#337), round its loop as the song plays inside the item. */
lanes?: SequenceLane[];
buses: BusLevel[];
/** The asset its routing is read from: the sound of a clip, else what it plays. */
routedAs: string;
@@ -168,6 +172,11 @@ function laidOut(project: Project, compId: string, item: CompositionItem, fps: n
return {
notes: heard.notes.map((n) => ({ ...n, tick: begin + n.tick, ...(n.bus && { bus: renamed(n.bus) }) })),
clips: heard.clips.map((c) => ({ ...c, tick: begin + c.tick, bus: renamed(c.bus) })),
lanes: heard.lanes.map((lane) => ({
...lane,
bus: renamed(lane.bus),
points: lane.points.map((p) => ({ ...p, tick: begin + p.tick })),
})),
levels,
buses: [withFx(pickedFx('', song.props), tempo), ...tracks],
routedAs: song.id,
@@ -207,7 +216,7 @@ function played(
tempo: number,
into: number,
seconds: number,
): { notes: SequenceNote[]; clips: SequenceClip[] } {
): { notes: SequenceNote[]; clips: SequenceClip[]; lanes: SequenceLane[] } {
const from = Math.round(ticksOfSeconds(into, tempo));
const to = from + Math.ceil(ticksOfSeconds(seconds, tempo));
const span = compositionTicks(seconds);
@@ -226,5 +235,10 @@ function played(
const { at: _, skip, ...clip } = c;
return [{ ...clip, tick, length, offset: c.offset + secondsOfTicks(skip, tempo), tempo }];
});
return { notes, clips };
const step = ticksOfSeconds(AUTOMATION_STEP, tempo);
const lanes = (sequence.lanes ?? []).map((lane) => ({
...lane,
points: unrolledLane(lane.points, loop, from, to, step).map((p) => ({ ...p, tick: at(p.tick) })),
}));
return { notes, clips, lanes };
}
+7
View File
@@ -0,0 +1,7 @@
/** The volume of a track of a song (#334), in decibels; at the lowest it is silent. */
export const TRACK_MIN_DB = -60;
export const TRACK_MAX_DB = 12;
export function dbToGain(db: number): number {
return db <= TRACK_MIN_DB ? 0 : 10 ** (db / 20);
}
+57
View File
@@ -0,0 +1,57 @@
/**
* Markers of a song (#338): named spots on its time, where an event fires in the game and where a
* switch to the song can start. A marker sits on a whole tick; two may share one, and so may their
* names – a node names a marker by its id.
*/
import { newId } from '../ids';
export interface SongMarker {
id: string;
name: string;
tick: number;
}
const finite = (value: unknown) => (typeof value === 'number' && Number.isFinite(value) ? value : NaN);
/** Held to what a marker may be: a whole tick from 0, a name, an id no other has; in the order of their ticks. */
export function heldSongMarkers(raw: unknown): SongMarker[] {
const ids = new Set<string>();
const markers: SongMarker[] = [];
for (const entry of Array.isArray(raw) ? (raw as unknown[]) : []) {
if (typeof entry !== 'object' || entry === null) continue;
const m = entry as Record<string, unknown>;
const tick = Math.round(finite(m.tick));
if (!(tick >= 0)) continue;
let id = typeof m.id === 'string' && m.id ? m.id : newId();
while (ids.has(id)) id = newId();
ids.add(id);
const name = typeof m.name === 'string' ? m.name.trim() : '';
markers.push({ id, name: name || `Marker ${markers.length + 1}`, tick });
}
return markers.sort((a, b) => a.tick - b.tick);
}
/** "Marker 3": the first number no marker is called by yet. */
export function nextMarkerName(markers: readonly SongMarker[]): string {
const taken = new Set(markers.map((m) => m.name));
let n = 1;
while (taken.has(`Marker ${n}`)) n++;
return `Marker ${n}`;
}
export function newMarker(markers: readonly SongMarker[], tick: number, name?: string): SongMarker {
return { id: newId(), name: name?.trim() || nextMarkerName(markers), tick: Math.max(0, Math.round(tick)) };
}
/** The markers with one of them changed; the others as they are. */
export function changedMarker(
markers: readonly SongMarker[],
id: string,
change: Partial<Pick<SongMarker, 'name' | 'tick'>>,
): SongMarker[] {
return markers.map((m) => (m.id === id ? { ...m, ...change } : m));
}
export function withoutMarker(markers: readonly SongMarker[], id: string): SongMarker[] {
return markers.filter((m) => m.id !== id);
}
+3 -3
View File
@@ -22,9 +22,9 @@ export function fxVoiceOf(
return plan ? { plan, params, tempo } : null;
}
/** The sound FX a clip goes through. */
export function clipFx(project: Project, fxId: string, tempo: number): FxVoice {
return fxVoiceOf(project, fxId, tempo) ?? DRY_VOICE;
/** The sound FX a clip goes through, with the values its parameters start at. */
export function clipFx(project: Project, fxId: string, tempo: number, params: FxParams = {}): FxVoice {
return fxVoiceOf(project, fxId, tempo, params) ?? DRY_VOICE;
}
/** Every layer and FX of every note and every file and FX of every clip a sequence plays. */
+14 -1
View File
@@ -1,9 +1,11 @@
import type { SendTap } from '../graph/Graph';
import type { FxPlan, FxVoice } from '../soundfx/nodes';
import type { AutomationTarget } from './automation';
import type { BakePlayback } from './bake';
import type { MixSend } from './mixer';
import {
LEAD,
type ScheduledAutomation,
type ScheduledClip,
type ScheduledNote,
type ScheduledRamp,
@@ -30,6 +32,11 @@ export interface BusLevel {
sends?: readonly BusSend[];
/** The sound FX all of it goes through as one chain (#346), before its level and pan. */
fx?: FxVoice;
/**
* What its automation sets while it plays (#337): `gain`, `pan` and a parameter of `fx` are then
* only where it starts.
*/
automated?: readonly AutomationTarget[];
}
/** One send of a track (#350): into another track's bus, or the output of the song where `to` is left out. */
@@ -64,6 +71,8 @@ export interface SequencePlay {
strikeClip?(clip: ScheduledClip): number;
/** Ramps the level of one of its buses (#418). */
ramp?(ramp: ScheduledRamp): void;
/** Takes one of its buses along its automation (#337). */
automate?(automation: ScheduledAutomation): void;
/** Its bake (#340), played in place of its notes and clips where the host can. */
bake?: BakePlay;
/**
@@ -91,7 +100,10 @@ export interface Runner {
stop(letRing?: boolean): void;
}
export interface PlaybackHost extends Omit<SchedulerHost, 'play' | 'playClip' | 'ahead' | 'ramp'> {
export interface PlaybackHost extends Omit<
SchedulerHost,
'play' | 'playClip' | 'ahead' | 'ramp' | 'automate'
> {
/** Loads what that playing needs and makes its buses ready, their sound FX among them (#346). */
prepare(needs: SequenceNeeds, id: number): Promise<void>;
/** How far ahead of its time a note of that playing may be started (see `SchedulerHost.ahead`). */
@@ -159,6 +171,7 @@ export class Playbacks {
play: (note) => (keep(planned, note), play.strike(note)),
...(strikeClip && { playClip: (clip: ScheduledClip) => (keep(clips, clip), strikeClip(clip)) }),
...(play.ramp && { ramp: play.ramp }),
...(play.automate && { automate: play.automate }),
},
() => play.sequence,
{ tempo: play.tempo, loop: play.loop, end: play.length },
+3
View File
@@ -5,6 +5,7 @@
* where the playhead stands, which jumps back at the end of a loop.
*/
import type { FadeCurve } from '../model/sound';
import type { SequenceLane } from './automation';
export interface SequenceNote {
/** Where it starts in the sequence, and how long it is held, in ticks. */
@@ -59,6 +60,8 @@ export interface Sequence {
notes: readonly SequenceNote[];
clips?: readonly SequenceClip[];
levels?: readonly SequenceLevel[];
/** The automation of a song's tracks (#337), each lane on the bus it acts on. */
lanes?: readonly SequenceLane[];
}
/** From `start` up to `end`, in ticks of the sequence; `end` is where the playhead jumps back. */
+15 -7
View File
@@ -8,10 +8,13 @@
import { newId } from '../ids';
import { HIGHEST_NOTE, LOWEST_NOTE } from '../instrument/notes';
import type { PatternAsset, SongAsset, SoundAsset } from '../model/Asset';
import { curveLookup } from '../model/curve';
import { type PropertyBag, textOf } from '../model/Properties';
import { findAssetOf, type Project } from '../model/Project';
import { findSound } from '../model/sound';
import { pickedFx } from '../properties/assetFx';
import { type AutomationLane, heldLanes, sequenceLaneOf } from './automation';
import { dbToGain, TRACK_MAX_DB, TRACK_MIN_DB } from './level';
import { gridTicks, GRID_OPTIONS, patternInstrument, patternLength } from './pattern';
import { heldLoop, type LoopRange, type Sequence, type SequenceClip, type SequenceNote } from './sequence';
import { DEFAULT_SIGN, heldTempo, type Meter, secondsOfTicks, signMeter, ticksPerBar } from './time';
@@ -27,6 +30,8 @@ export interface SongTrack {
solo: boolean;
/** The sound FX everything on the track goes through, before its volume and pan (#346); `''` for none. */
fx: string;
/** Its volume, pan and parameters over the song (#337); left out without a lane. */
automation?: AutomationLane[];
}
export type SongItemKind = 'pattern' | 'sound';
@@ -66,8 +71,7 @@ const DEFAULT_TRACKS = 1;
/** A song is never shorter than this many bars, on the ruler or when it plays. */
const MIN_BARS = 1;
export const TRACK_MIN_DB = -60;
export const TRACK_MAX_DB = 12;
export { dbToGain, TRACK_MAX_DB, TRACK_MIN_DB };
export const ITEM_MIN_DB = -60;
/** A sound item can be turned up; a pattern item's notes are held to unity, as a note always is. */
export const SOUND_ITEM_MAX_DB = 12;
@@ -138,10 +142,6 @@ export function itemBus(songId: string, item: Pick<SongItem, 'id' | 'track'>): s
return `${trackBus(songId, item.track)}:${item.id}`;
}
export function dbToGain(db: number): number {
return db <= TRACK_MIN_DB ? 0 : 10 ** (db / 20);
}
/** The sound FX an item goes through (#352): its own pick, else the one of what it plays; '' for none. */
export function itemFx(project: Project, item: SongItem): string {
return pickedFx(item.fx, itemAsset(project, item)?.props ?? {});
@@ -228,7 +228,13 @@ export function songSequence(project: Project, song: SongAsset): Sequence {
}
notes.sort((a, b) => a.tick - b.tick || a.note - b.note);
clips.sort((a, b) => a.tick - b.tick);
return { notes, clips };
const curves = curveLookup(project);
const lanes = song.tracks.flatMap((track) =>
(track.automation ?? [])
.filter((lane) => lane.points.length > 0)
.map((lane) => sequenceLaneOf(lane, trackBus(song.id, track.id), curves)),
);
return { notes, clips, ...(lanes.length > 0 && { lanes }) };
}
const finite = (value: unknown, fallback: number) =>
@@ -255,6 +261,8 @@ export function heldSongTracks(raw: unknown): SongTrack[] {
solo: t.solo === true,
fx: text(t.fx),
});
const automation = heldLanes(t.automation);
if (automation.length > 0) tracks.at(-1)!.automation = automation;
}
return tracks.length > 0 ? tracks : [newTrack(1)];
}
+38 -8
View File
@@ -1,6 +1,7 @@
import type { Project } from '../model/Project';
import type { PropertyBag } from '../model/Properties';
import type { Graph } from '../graph/Graph';
import { audioValueOf, laneOf, laneValueAt, paramOf, type TrackTarget } from './automation';
import { fxVoiceOf } from './needs';
import type { BusLevel } from './playbacks';
import { songRoutes, trackGates } from './routing';
@@ -18,8 +19,10 @@ import {
/**
* The buses of a song: each track's at its volume and pan, through its sound FX (#346), on to where
* its routing sends it (#350); a bus for its own items into it; and one for each pattern item with a
* sound FX into that, at unity – its volume is its notes'. Mute and solo close sends and items
* its routing sends it (#350), a level, pan or parameter its lane rules (#337) from where the lane
* starts – a parameter on the sound FX of its items too; a bus
* for its own items into it; and one for each pattern item with a sound FX into that, at unity – its
* volume is its notes'. Mute and solo close sends and items
* (`trackGates`). A sound FX that does not compile is none, and the bus then only passes it on.
*/
export function songBuses(
@@ -33,18 +36,36 @@ export function songBuses(
},
): BusLevel[] {
const tempo = songTempo(song.props);
const fxOf = (id: string) => {
const fx = fxVoiceOf(project, id, tempo);
const byId = new Map(song.tracks.map((track) => [track.id, track]));
// A parameter a lane rules starts where the lane does (#337).
const fxOf = (id: string, track: SongTrack | undefined) => {
const lanes = (track?.automation ?? []).filter((lane) => paramOf(lane.target) !== null);
const params = Object.fromEntries(
lanes.flatMap((lane) =>
lane.points.length > 0
? [[paramOf(lane.target)!, audioValueOf(lane, laneValueAt(lane.points, 0))]]
: [],
),
);
const fx = fxVoiceOf(project, id, tempo, params);
return fx ? { fx } : {};
};
const routes = songRoutes(song);
const gates = trackGates(song.tracks, routes);
const tracks = song.tracks.map((track) => {
const gate = gates.get(track.id);
const automated = (track.automation ?? []).flatMap((lane) =>
lane.points.length > 0 ? [lane.target] : [],
);
const atStart = (target: TrackTarget, fader: number) => {
const lane = laneOf(track, target);
return lane ? audioValueOf(lane, laneValueAt(lane.points, 0)) : fader;
};
return {
name: trackBus(song.id, track.id),
gain: dbToGain(track.volume),
pan: track.pan,
gain: atStart('volume', dbToGain(track.volume)),
pan: atStart('pan', track.pan),
...(automated.length > 0 && { automated }),
sends: routes
.filter((route) => route.from === track.id)
.map((route) => ({
@@ -52,7 +73,7 @@ export function songBuses(
gain: gate?.passes(route.to) ? dbToGain(route.volume) : 0,
tap: route.tap,
})),
...fxOf(track.fx),
...fxOf(track.fx, track),
};
});
const own = song.tracks.map((track) => ({
@@ -69,7 +90,16 @@ export function songBuses(
gain: 1,
pan: 0,
into: trackItemsBus(song.id, item.track),
...fxOf(itemFx(project, item)),
...fxOf(itemFx(project, item), byId.get(item.track)),
...automatedParams(byId.get(item.track)),
}));
return [...tracks, ...own, ...items];
}
/** The parameters a track's lanes rule, for the bus of one of its items: its sound FX leaves them alone. */
function automatedParams(track: SongTrack | undefined): Pick<BusLevel, 'automated'> {
const automated = (track?.automation ?? []).flatMap((lane) =>
paramOf(lane.target) !== null && lane.points.length > 0 ? [lane.target] : [],
);
return automated.length > 0 ? { automated } : {};
}
+63
View File
@@ -0,0 +1,63 @@
import { VELOCITY_PARAM } from '../instrument/nodes';
import { instrumentParams } from '../instrument/compile';
import { findAssetOf, type Project } from '../model/Project';
import { assetFxOf } from '../properties/assetFx';
import { soundFxParams } from '../soundfx/compile';
import { patternInstrument } from './pattern';
import { itemFx, type SongItem, type SongTrack } from './song';
/** A parameter a lane of a track can move, with the default it has where it is first declared. */
export interface TrackParam {
name: string;
default: number;
}
/**
* The parameters a lane of a track can move (#337): those of its sound FX, of the sound FX of its
* items, of the instruments its pattern items play and of their own sound FX, each name once, in
* that order. A name several of them have moves all of them.
*/
export function trackParams(
project: Project,
song: { items: readonly SongItem[] },
track: SongTrack,
): TrackParam[] {
const found = new Map<string, number>();
const add = (ports: readonly { key: string; default?: number }[]) => {
for (const port of ports)
if (!found.has(port.key) && port.key !== VELOCITY_PARAM) found.set(port.key, port.default ?? 0);
};
const addFx = (fxId: string) => {
const fx = fxId ? findAssetOf(project, fxId, 'soundfx') : undefined;
if (fx) add(soundFxParams(fx.graph));
};
addFx(track.fx);
for (const item of song.items) {
if (item.track !== track.id) continue;
addFx(itemFx(project, item));
if (item.kind !== 'pattern') continue;
const pattern = findAssetOf(project, item.asset, 'pattern');
const id = item.instrument || (pattern ? patternInstrument(pattern.props) : '');
const instrument = id ? findAssetOf(project, id, 'instrument') : undefined;
if (!instrument) continue;
add(instrumentParams(instrument.graph));
addFx(assetFxOf(instrument.props));
}
return [...found].map(([name, value]) => ({ name, default: value }));
}
/**
* The range a new lane of a parameter gets: 0 to 1 for a default in there, else from 0 – or as far
* below – to twice the default, on a round number.
*/
export function defaultParamRange(value: number): { min: number; max: number } {
if (!Number.isFinite(value) || (value >= 0 && value <= 1)) return { min: 0, max: 1 };
const top = roundUp(Math.abs(value) * 2);
return value > 0 ? { min: 0, max: top } : { min: -top, max: top };
}
/** The next of 1, 2 and 5 times a power of ten from `value` up. */
function roundUp(value: number): number {
const power = 10 ** Math.floor(Math.log10(value));
return ([1, 2, 5, 10].find((step) => step * power >= value) ?? 10) * power;
}
+3
View File
@@ -25,6 +25,7 @@ import { heldBake } from '../music/bake';
import { heldStream } from '../music/streams';
import { heldPatternNotes } from '../music/pattern';
import { isEmbeddable } from '../music/embedded';
import { heldSongMarkers } from '../music/markers';
import { heldSongItems, heldSongTracks } from '../music/song';
import { heldSend, heldSongGraph } from '../music/routing';
import { isMasterRoutingDef, isSongRoutingDef } from '../music/routingNodes';
@@ -348,6 +349,7 @@ export function deserializeProject(raw: unknown, fallbackId = ''): Project {
const loop = readLoop(a.loop);
const bake = heldBake(a.bake);
const tracks = heldSongTracks(a.tracks);
const markers = heldSongMarkers(a.markers);
assets.push({
id,
type,
@@ -356,6 +358,7 @@ export function deserializeProject(raw: unknown, fallbackId = ''): Project {
tracks,
items: heldSongItems(a.items, tracks),
...(loop && { loop }),
...(markers.length > 0 && { markers }),
graph: heldSongGraph(normalizeGraph(a.graph, isSongRoutingDef), tracks),
...(bake && { bake }),
});
+6
View File
@@ -41,6 +41,12 @@ export function timeStep(dt: number, scale: number, paused: boolean): number {
return Math.min(Math.max(dt, 0), MAX_FRAME) * effectiveScale(scale, paused);
}
/** The frame that moves the clock by `left` seconds at most, at this scale (#435). */
export function frameWithin(dt: number, scale: number, left: number): number {
const rate = effectiveScale(scale, false);
return rate > 0 ? Math.min(dt, Math.max(0, left) / rate) : dt;
}
export class GameClock {
private seconds = 0;
/** The global time scale: 1 is real time, 2 twice as fast, 0 frozen. */
+8 -2
View File
@@ -984,6 +984,7 @@ export class Editor implements PlayHost {
play: this.play,
follows: followsAgent,
stores,
heard: () => soundLibrary.heard(),
},
wait,
),
@@ -993,7 +994,11 @@ export class Editor implements PlayHost {
this.state.setNotice('Agent switched off: another tab of your account has it now');
},
});
link.onChange = () => this.syncAgentBadge();
link.onChange = () => {
this.syncAgentBadge();
// The buses are tapped for what the agent hears only while it is switched on (#436).
soundLibrary.watch(link.on);
};
return link;
}
@@ -1668,9 +1673,10 @@ export class Editor implements PlayHost {
// and so does a run that waits for its videos at the start (#404).
// A single step out of the pause moves by one fixed step, however long the frame took.
// A recording not in real time (#423) moves it by one frame of the recording, or holds it.
// A pause the agent asked for at a time (#435) cuts the frame that would go past it.
const stepping = this.play.beginFrame();
const recorded = stepping ? null : this.play.recordingStep();
const frame = stepping ? FIXED_STEP : (recorded ?? dt);
const frame = stepping ? FIXED_STEP : (recorded ?? this.play.frameBeforePause(dt, gameClock.scale));
const held = this.state.paused && !stepping;
const step = gameClock.advance(frame, held || this.play.isGamePaused || this.play.isWaiting);
this.play.tick(step, timeStep(frame, gameClock.scale, held));
+33 -1
View File
@@ -12,7 +12,7 @@ import { type BehaviorSystem, behaviorError, type Trace } from '../runtime/syste
import type { AnimationPlayer } from '../core/model/animation';
import type { PropertyBag } from '../core/model/Properties';
import { VIDEO_WAIT_MS } from '../core/model/video';
import { gameClock } from '../core/time';
import { frameWithin, gameClock } from '../core/time';
import { loadPhysics, readyPhysics } from '../runtime/physics/engine';
import { Input } from '../runtime/systems/Input';
import type { World } from '../runtime/World';
@@ -95,6 +95,8 @@ export class PlaySession {
private stepping = false;
/** The object whose `Break` node paused the run, until it is resumed or stepped on. */
private breakAt: string | null = null;
/** The time the agent wants the run held at (#435), on the world it named; a `Load scene` calls it off. */
private pausing: { world: World; time: number } | null = null;
constructor(
private readonly state: EditorState,
@@ -162,10 +164,24 @@ export class PlaySession {
}
this.state.setPaused(!this.state.paused);
this.breakAt = null;
this.pausing = null;
this.applyPause(true);
ui().focus.blur();
}
/** Holds the run once its time reaches `time` (#435), as if the pause button had been pressed; null calls it off. */
pauseAt(time: number | null): void {
this.pausing = time === null || !this.world ? null : { world: this.world, time };
}
/** Called before the clock moves: the frame cut short so the run does not go past the time `pauseAt` named. */
frameBeforePause(dt: number, scale: number): number {
const pausing = this.pausing;
if (!pausing || pausing.world !== this.world || this.state.paused || this.isGamePaused || this.isWaiting)
return dt;
return frameWithin(dt, scale, pausing.time - pausing.world.time);
}
/** Lets a paused run on by exactly one frame, at the next tick; does nothing unless paused. */
step(): void {
if (this.isPaused) this.stepAsked = true;
@@ -356,6 +372,7 @@ export class PlaySession {
this.videoWait = null;
this.stepAsked = false;
this.breakAt = null;
this.pausing = null;
const world = this.world;
this.run = null;
this.input.detach();
@@ -420,6 +437,21 @@ export class PlaySession {
// A game that pauses itself is looked at like one paused from the toolbar, and so is a step.
this.follow((world.paused && !wasPaused) || this.stepping ? this.panels.flush() : update);
this.pauseAtBreak();
this.pauseWhenDue();
}
/** The run reached the time `pauseAt` named: it holds there. A pause that came first calls it off. */
private pauseWhenDue(): void {
const pausing = this.pausing;
if (!pausing) return;
if (pausing.world !== this.world || this.state.paused) {
this.pausing = null;
return;
}
if (pausing.world.time < pausing.time - 1e-9) return;
this.pausing = null;
this.state.setPaused(true);
this.applyPause(true);
}
/**
+40
View File
@@ -11,12 +11,16 @@ import type { StepTarget } from './commandCatalog';
import { cameraOfWorld } from '../../viewport/activeCamera';
import { pixelCels } from '../../viewport/PixelCels';
import {
exact,
IN_BACKGROUND,
type InputStep,
inputProblem,
inputRefusal,
LONG_DEADLINE_MS,
LONG_INPUT_SECONDS,
MAX_INPUT_SECONDS,
type PlayDriver,
runUntil,
SHORT_DEADLINE_MS,
sendInput,
totalSeconds,
@@ -164,6 +168,7 @@ async function run(name: string, args: Record<string, unknown>, host: ToolHost):
case 'select':
return outcome(host.select(args as unknown as AgentSelection), 'Selected.');
case 'play': {
if (args.until !== undefined) return playUntil(args.action as PlayAction, args.until as number, host);
const refusal = host.play(args.action as PlayAction);
if (refusal !== null) return failure(refusal);
const facts = args.action === 'play' ? host.runtime?.() : null;
@@ -215,6 +220,41 @@ async function playInput(steps: InputStep[], host: ToolHost): Promise<ToolResult
return answer(await sendInput(steps, host.game, long ? LONG_DEADLINE_MS : SHORT_DEADLINE_MS));
}
/** `play`, `resume` or `pause` with `until` (#435): the game runs on to that time and holds there. */
async function playUntil(action: PlayAction, until: number, host: ToolHost): Promise<ToolResult> {
const game = host.game;
if (!game) return failure('This editor plays no game');
if (action === 'stop') return failure('until goes with play, resume or pause');
if (game.hidden()) return failure(IN_BACKGROUND);
const run = action === 'play' ? null : game.run();
if (action !== 'play') {
if (!run) return failure('The game is not running: start it with play first');
if (action === 'pause' && game.paused())
return failure('The game is paused already: resume it with until');
if (action === 'resume' && !game.paused())
return failure('The game is running already: pause it with until');
}
const now = run?.world.time ?? 0;
if (until <= now + 1e-6) return failure(`The game is at ${exact(now)} s already: until has to lie ahead`);
const ahead = until - now;
if (ahead > MAX_INPUT_SECONDS)
return failure(`until is ${exact(ahead)} s ahead; at most ${MAX_INPUT_SECONDS} s go in one call`);
if (action !== 'pause') {
const refusal = host.play(action);
if (refusal !== null) return failure(refusal);
}
const long = ahead > LONG_INPUT_SECONDS;
if (long) host.takesLong?.();
const time = await runUntil(game, until, long ? LONG_DEADLINE_MS : SHORT_DEADLINE_MS);
const facts = action === 'play' ? host.runtime?.() : null;
const blind = facts && !cameraOfWorld(facts.world) ? ` ${NO_CAMERA_HINT}` : '';
const text =
time < until - 1e-6
? `Paused at ${time} s of game time, before ${until} s: the user or a Break node paused it first.`
: `Paused at ${time} s of game time.${blind}`;
return done(text, { time, paused: true });
}
async function runCommands(args: Record<string, unknown>, host: ToolHost): Promise<ToolResult> {
const steps = args.steps as EditStep[];
const deletes = deletedAssets(steps);
+171
View File
@@ -0,0 +1,171 @@
import { noteName } from '../../core/instrument/notes';
import { findAsset, type Project } from '../../core/model/Project';
import { clipsOf } from '../../core/model/sound';
import { MASTER_MIX } from '../../core/music/mixer';
import { playedSoundOf } from '../../core/properties/playedSound';
import type { World } from '../../runtime/World';
import type { HeardSound, HeardVoice } from '../../viewport/SoundLibrary';
/** Below this a bus counts as silent: far under anything a speaker gives out. */
export const SILENT_DB = -90;
/** The most voices an answer lists; the rest are counted. */
export const MAX_VOICES = 40;
export const STALLED_HINT =
'The browser holds the audio clock while the game runs, so nothing is heard and notes stay planned. ' +
'The browser starts it only once the user has clicked or pressed a key in the editor page, and not ' +
'without a sound output: ask the user.';
const INSTRUMENT = 'instrument:';
const PLAYING_BUS = /^play:(\d+)(?::(.*))?$/;
/** The bus of a track of a song played on its own: `play:<n>:song:<song>:<track>`. */
const SONG_TRACK = /^song:([^:]+):([^:#]+)$/;
/**
* What `get_runtime` says of the sound (#436): the level of the master and of every bus that is
* heard by itself, the voices sounding now with what they play and who asked, the patterns, songs
* and compositions playing, and what is not heard for lack of its bytes. Names stand in for ids.
*/
export function audioView(heard: HeardSound, project: Project, world: World): object {
const names = new SoundNames(project, heard, world);
const sounding = heard.voices.filter((voice) => voice.startsIn === 0);
const planned = heard.voices.length - sounding.length;
// A pause holds the clock; its state is the one it had while the game ran.
const { state, time } = heard.clock;
return {
clock: { state, time: Math.round(time * 1000) / 1000 },
...(state === 'suspended' && { hint: STALLED_HINT }),
window: Math.round(heard.window * 100) / 100,
levels: heard.buses
.filter((bus) => shownBus(bus.name))
.sort((a, b) => Number(b.name === MASTER_MIX) - Number(a.name === MASTER_MIX))
.map(({ name, peak, rms }) => levelRow(names.bus(name), peak, rms)),
voices: sounding.slice(0, MAX_VOICES).map((voice) => ({
...names.voice(voice),
...(voice.released && { released: true }),
...(voice.ringing && { ringing: true }),
})),
...(sounding.length > MAX_VOICES && { moreVoices: sounding.length - MAX_VOICES }),
...(planned > 0 && { planned }),
playing: heard.playings.map((playing) => ({
...names.playing(playing.name),
...(names.by(playing.owner) && { by: names.by(playing.owner) }),
...(!playing.playing && { ended: true }),
...(playing.fading !== null && { fading: true }),
...(playing.baked && { baked: true }),
})),
...(heard.waiting.length > 0 && { waiting: heard.waiting.map((voice) => names.voice(voice)) }),
...(heard.failed.length > 0 && { failed: heard.failed.map((file) => names.file(file)) }),
};
}
/**
* The buses worth a level of their own: the master, those of the mix, the master of each playing
* and the tracks of a song. What lies inside a track – its items, its own bus for them – is heard on
* the track.
*/
function shownBus(name: string): boolean {
if (name === MASTER_MIX || name.startsWith('mix:')) return true;
const playing = PLAYING_BUS.exec(name);
return playing !== null && (playing[2] === undefined || SONG_TRACK.test(playing[2]));
}
function levelRow(bus: string, peak: readonly number[], rms: readonly number[]): object {
const loudest = decibels(Math.max(...peak));
if (loudest < SILENT_DB) return { bus, silent: true };
const power = rms.reduce((sum, side) => sum + side * side, 0) / rms.length;
return { bus, peak: round(loudest), rms: round(decibels(Math.sqrt(power))) };
}
function decibels(gain: number): number {
return gain > 0 ? 20 * Math.log10(gain) : -Infinity;
}
function round(db: number): number {
return Number.isFinite(db) ? Math.round(db * 10) / 10 : SILENT_DB;
}
/** Names for what the sink knows by file, by bus and by the id of an entity. */
class SoundNames {
private readonly files = new Map<string, Record<string, string>>();
private readonly playings: ReadonlyMap<number, string>;
constructor(
private readonly project: Project,
heard: HeardSound,
private readonly world: World,
) {
for (const asset of project.assets) {
if (asset.type === 'sound') {
this.files.set(asset.file, { sound: asset.name });
for (const clip of clipsOf(asset))
this.files.set(playedSoundOf({ asset, clip }), { sound: `${asset.name} / ${clip.name}` });
}
if ((asset.type === 'pattern' || asset.type === 'song') && asset.bake)
this.files.set(asset.bake.file, { bake: `${asset.type} ${asset.name}` });
}
this.playings = new Map(heard.playings.map(({ id, name }) => [id, this.label(name)]));
}
voice(voice: HeardVoice): object {
const by = this.by(voice.owner);
return {
...this.file(voice.file, voice.note),
bus: this.bus(voice.bus),
...(by && { by }),
};
}
/** What a file plays: a sound or a clip of one, a note of an instrument, the bake of a pattern or song. */
file(file: string, note?: number): Record<string, string> {
if (file.startsWith(INSTRUMENT)) {
const id = file.slice(INSTRUMENT.length);
return {
instrument: findAsset(this.project, id)?.name ?? id,
...(note !== undefined && { note: noteName(note) }),
};
}
return this.files.get(file) ?? { file };
}
bus(name: string): string {
if (name === '' || name === MASTER_MIX) return 'master';
if (name.startsWith('mix:')) {
const id = name.slice('mix:'.length);
return findAsset(this.project, id)?.name ?? id;
}
const playing = PLAYING_BUS.exec(name);
if (!playing) return name;
const of = this.playings.get(Number(playing[1])) ?? `playing ${playing[1]}`;
const track = SONG_TRACK.exec(playing[2] ?? '');
if (!track) return of;
const song = findAsset(this.project, track[1]!);
const named = song?.type === 'song' ? song.tracks.find((t) => t.id === track[2]) : undefined;
return `${of}, track ${named?.name ?? track[2]}`;
}
playing(name: string): Record<string, string> {
const [kind, id] = splitName(name);
return kind ? { [kind]: findAsset(this.project, id)?.name ?? id } : { name };
}
/** The object a voice's behavior sits on, or the behavior itself; '' for no owner. */
by(owner: string): string {
if (owner === '') return '';
const entity = this.world.entities.find((e) => e.id === owner);
if (!entity) return owner;
return entity.parent?.name ?? entity.name;
}
private label(name: string): string {
const [kind, id] = splitName(name);
return kind ? `${kind} ${findAsset(this.project, id)?.name ?? id}` : name;
}
}
/** `pattern:<id>`, `song:<id>` or `composition:<id>` as kind and id; no kind for anything else. */
function splitName(name: string): [string, string] {
const at = name.indexOf(':');
const kind = at > 0 ? name.slice(0, at) : '';
return ['pattern', 'song', 'composition'].includes(kind) ? [kind, name.slice(at + 1)] : ['', name];
}
+1 -1
View File
@@ -65,7 +65,7 @@ const keyOf = (args: Args, name: string): AnimationKey => {
};
/** Every easing a key can name, but the curve ones: those take the id of a Curve asset. */
const EASINGS = EASING_SHAPES.flatMap(({ value }) =>
export const EASINGS = EASING_SHAPES.flatMap(({ value }) =>
value === 'linear' || value === 'step'
? [value]
: value === 'curve'
+207 -1
View File
@@ -9,7 +9,7 @@ import { oneStep } from '../../../core/commands/CompoundCommand';
import { ReplaceGraphCommand } from '../../../core/commands/graphCommands';
import { SetLoopCommand, SetPatternNotesCommand } from '../../../core/commands/patternCommands';
import { requireAsset, requireEdge, requirePattern, requireSong } from '../../../core/commands/require';
import { SetSendCommand, SetSongCommand } from '../../../core/commands/songCommands';
import { SetSendCommand, SetSongCommand, SetSongMarkersCommand } from '../../../core/commands/songCommands';
import type { SendTap } from '../../../core/graph/Graph';
import { newId } from '../../../core/ids';
import {
@@ -20,6 +20,7 @@ import {
TONE_PRESETS,
} from '../../../core/instrument/chipPresets';
import { HIGHEST_NOTE, LOWEST_NOTE, noteName } from '../../../core/instrument/notes';
import { easingOf, isKeyInterpolation } from '../../../core/model/animation';
import type { PatternAsset, SongAsset } from '../../../core/model/Asset';
import { audioSeconds } from '../../../core/model/composition';
import type { PropertyBag } from '../../../core/model/Properties';
@@ -31,6 +32,16 @@ import {
ROUTING_ID,
} from '../../../core/model/Project';
import { clipsOf, findSound, hasClips } from '../../../core/model/sound';
import {
type AutomationLane,
type AutomationPoint,
heldRange,
isAutomationTarget,
type LaneKind,
laneRange,
paramOf,
} from '../../../core/music/automation';
import { changedMarker, newMarker, type SongMarker, withoutMarker } from '../../../core/music/markers';
import { DEFAULT_VELOCITY, patternLength, type PatternNote } from '../../../core/music/pattern';
import { SEND_TAPS } from '../../../core/music/routing';
import {
@@ -47,8 +58,11 @@ import {
TRACK_MIN_DB,
} from '../../../core/music/song';
import { TICKS_PER_QUARTER, ticksOfSeconds, ticksPerBar } from '../../../core/music/time';
import { defaultParamRange, trackParams } from '../../../core/music/trackParams';
import { NO_FX } from '../../../core/properties/assetFx';
import { withLanePoints, withoutLane, withParamLane } from '../../song/laneEdit';
import type { ArgSchema } from '../argCheck';
import { EASINGS } from './animation';
import {
type Args,
type CatalogCommand,
@@ -68,6 +82,7 @@ import {
const PATTERN = id('Id of the pattern asset.');
const SONG = id('Id of the song asset.');
const TRACK = id('Id of the track, as get_asset lists it under tracks.');
const MARKER = id('Id of the marker, as get_asset lists it under markers.');
const ITEM = id('Id of the item, as get_asset lists it under items.');
const INDEX = whole('Place in the list of tracks, from 0 (the top).', 0);
const TICK = whole(`Ticks from the start; ${TICKS_PER_QUARTER} to a quarter note.`, 0);
@@ -280,6 +295,81 @@ export const MUSIC_COMMANDS: readonly CatalogCommand[] = [
},
shows: theTab,
},
{
name: 'set_song_lane',
description:
'Sets an automation lane of a song track: its volume (in dB), its pan (-1 left to 1 right) or a parameter ' +
'changing over the song along points; the points given replace those the lane had. From each point the ' +
'value runs to the next along its ease; before the first point it holds the first value, after the last ' +
'the last one; two points on one tick jump there. While the lane of the volume or the pan has a point, it ' +
"rules that value in place of the track's own; with no point that lane goes and the track's value rules " +
'again. A parameter lane moves every parameter of that name on the track: of its sound FX, of the sound FX ' +
'of its items, of the instruments its patterns play and of their sound FX (get_asset lists them under ' +
"params of the track). It keeps a range its points lie in, round the parameter's default where none is " +
'given, and stays without a point until remove. A note or clip starts at the value there, so what a ' +
'voice reads only when it starts changes from the next one.',
args: object(
{
asset: SONG,
track: TRACK,
target: id(
'"volume", "pan", or "param:<name>" for a parameter, as get_asset lists lanes under automation.',
),
points: {
type: 'array',
items: object(
{
tick: TICK,
value: number(
'In the units of the lane: dB, -1 to 1 for the pan, within the range of a parameter.',
),
ease: id(
`How it runs to the next point: ${EASINGS.join(', ')}, or "curve:<id of a Curve asset>"; ` +
'linear where left out.',
),
},
['tick', 'value'],
),
description: 'All points of the lane, in any order; those on one tick in the order given.',
},
min: number('A parameter lane: the lowest value it holds.'),
max: number('A parameter lane: the highest value it holds.'),
remove: flag("Take the lane away, a parameter's range with it."),
},
['asset', 'track', 'target'],
),
build: (args, { project }) => {
const song = requireSong(project, text(args, 'asset'));
const track = requireTrack(song, text(args, 'track'));
const target = args.target;
if (!isAutomationTarget(target))
throw new CommandError(
`${JSON.stringify(target)} is no lane – give "volume", "pan" or "param:<name>"`,
);
const had = track.automation?.find((lane) => lane.target === target);
const given = [args.points, args.min, args.max].some((value) => value !== undefined);
let changed: SongTrack;
let label: string;
if (args.remove === true) {
if (given) throw new CommandError('remove takes no points and no range');
if (!had) throw new CommandError(`${track.name} has no ${target} lane`);
changed = withoutLane(track, target);
label = 'Remove automation lane';
} else {
const name = paramOf(target);
const range = name === null ? noRange(args) : paramRange(project, song, track, name, had, args);
const kind: LaneKind = { target, ...range };
const points = args.points === undefined ? null : lanePointsOf(project, kind, args.points as Args[]);
if (name === null && !points) throw new CommandError('Give the points, [] to take the lane away');
changed = range ? withParamLane(track, name!, range) : track;
if (points) changed = withLanePoints(changed, target, points);
label = had ? 'Set automation lane' : 'Add automation lane';
}
const tracks = song.tracks.map((t) => (t.id === track.id ? changed : t));
return { command: new SetSongCommand(song.id, { tracks }, label) };
},
shows: theTab,
},
{
name: 'move_song_track',
description: 'Moves a track of a song to another place in its list.',
@@ -309,6 +399,57 @@ export const MUSIC_COMMANDS: readonly CatalogCommand[] = [
},
shows: theTab,
},
{
name: 'add_song_marker',
description:
'Adds a marker to a song: a named spot on its time, drawn on the strip under the ruler of the song tab. ' +
'Markers may share a tick and a name; a marker is known by its id.',
args: object({ asset: SONG, tick: TICK, name: id('Its name; "Marker <n>" where left out.') }, [
'asset',
'tick',
]),
build: (args, { project }) => {
const song = requireSong(project, text(args, 'asset'));
const markers = song.markers ?? [];
const name = typeof args.name === 'string' ? args.name : undefined;
const marker = newMarker(markers, numberOf(args, 'tick'), name);
return {
command: new SetSongMarkersCommand(song.id, [...markers, marker], 'Add marker'),
made: { id: marker.id },
};
},
shows: theTab,
},
{
name: 'change_song_marker',
description: 'Renames a marker of a song or moves it to another tick. Give only what changes.',
args: object({ asset: SONG, marker: MARKER, name: id('Its name.'), tick: TICK }, ['asset', 'marker']),
build: (args, { project }) => {
const song = requireSong(project, text(args, 'asset'));
const marker = requireMarker(song, text(args, 'marker'));
const change: { name?: string; tick?: number } = {};
if (typeof args.name === 'string') {
if (!args.name.trim()) throw new CommandError('A marker needs a name');
change.name = args.name.trim();
}
if (typeof args.tick === 'number') change.tick = numberOf(args, 'tick');
const markers = changedMarker(song.markers ?? [], marker.id, change);
return { command: new SetSongMarkersCommand(song.id, markers, 'Change marker') };
},
shows: theTab,
},
{
name: 'remove_song_marker',
description: 'Removes a marker of a song.',
args: object({ asset: SONG, marker: MARKER }, ['asset', 'marker']),
build: (args, { project }) => {
const song = requireSong(project, text(args, 'asset'));
const marker = requireMarker(song, text(args, 'marker'));
const markers = withoutMarker(song.markers ?? [], marker.id);
return { command: new SetSongMarkersCommand(song.id, markers, 'Delete marker') };
},
shows: theTab,
},
{
name: 'add_song_item',
description:
@@ -537,12 +678,77 @@ function itemLength(project: Project, song: SongAsset, kind: SongItem['kind'], p
return Math.max(1, Math.round(ticksOfSeconds(seconds, songTempo(song.props))));
}
/** Refuses a range given to the lane of the volume or the pan: theirs is fixed. */
function noRange(args: Args): null {
if (args.min !== undefined || args.max !== undefined)
throw new CommandError('Only a parameter lane has a range – the volume and the pan have theirs');
return null;
}
/**
* The range of a parameter's lane: the one given, else the one it has, else one round the default of
* the parameter. A new lane is refused for a parameter nothing on the track has.
*/
function paramRange(
project: Project,
song: SongAsset,
track: SongTrack,
name: string,
had: AutomationLane | undefined,
args: Args,
): { min: number; max: number } {
const offered = trackParams(project, song, track);
const param = offered.find((p) => p.name === name);
if (!had && !param)
throw new CommandError(
offered.length === 0
? `Nothing on ${track.name} has a parameter to move`
: `Nothing on ${track.name} has a parameter ${name} – it has ${offered.map((p) => p.name).join(', ')}`,
);
const given = [args.min, args.max].filter((value) => value !== undefined).length;
if (given === 1) throw new CommandError('Give min and max, or neither');
if (given === 2) {
const range = heldRange(args.min, args.max);
if (!range) throw new CommandError('max has to be above min');
return range;
}
return had ? laneRange(had) : defaultParamRange(param!.default);
}
/**
* The points the agent wrote, refused where a value or an easing is wrong. The song puts them in the
* order of their ticks, those on one tick as they came.
*/
function lanePointsOf(project: Project, kind: LaneKind, raw: readonly Args[]): AutomationPoint[] {
const { min, max } = laneRange(kind);
return raw.map((point): AutomationPoint => {
const tick = point.tick as number;
const value = within(point.value as number, min, max, `value at tick ${tick}`);
const ease = point.ease;
if (ease === undefined || ease === 'linear') return { tick, value };
if (!isKeyInterpolation(ease))
throw new CommandError(
`${JSON.stringify(ease)} is no easing – give ${EASINGS.join(', ')} or "curve:<id>"`,
);
const { curve } = easingOf(ease);
if (curve !== undefined && !findAssetOf(project, curve, 'curve'))
throw new CommandError(`${curve} is no Curve asset`);
return { tick, value, ease };
});
}
function requireTrack(song: SongAsset, trackId: string): SongTrack {
const track = song.tracks.find((t) => t.id === trackId);
if (!track) throw new CommandError(`${song.name} has no track ${trackId}`);
return track;
}
function requireMarker(song: SongAsset, markerId: string): SongMarker {
const marker = song.markers?.find((m) => m.id === markerId);
if (!marker) throw new CommandError(`${song.name} has no marker ${markerId}`);
return marker;
}
/** A pitch the agent wrote as a MIDI number or a name like "C#4". */
export function pitchOf(value: unknown): number {
const number =
+16 -1
View File
@@ -13,6 +13,7 @@ import type { StepTarget } from './commandCatalog';
import { screenshot } from './screenshot';
import type { FileAssetType } from './storedFiles';
import type { FileStore } from '../../core/store/FileStore';
import type { HeardSound } from '../../viewport/SoundLibrary';
/** What the agent's tools reach of the editor beyond its state. */
export interface EditorParts {
@@ -25,12 +26,15 @@ export interface EditorParts {
play(paused: boolean): void;
stop(): void;
togglePause(): void;
pauseAt(time: number | null): void;
agentRun(): { world: World; input: Input; shared: SharedState } | null;
};
/** The user's "Follow agent" switch. */
follows(): boolean;
/** Where the files of each kind are kept (#431). */
stores: Record<FileAssetType, FileStore>;
/** What is heard now (#436). */
heard(): HeardSound;
}
/** How long the user has to answer an agent's question before it counts as a no. */
@@ -82,12 +86,23 @@ export function editorToolHost(parts: EditorParts, wait: () => void): ToolHost {
hidden: () => document.visibilityState === 'hidden',
frame: () => nextFrame(parts.app),
now: () => performance.now(),
pause: () => {
if (parts.play.isPlaying && !state.paused) parts.play.togglePause();
},
pauseAt: (time) => parts.play.pauseAt(time),
},
runtime: () => {
const run = parts.play.agentRun();
if (!run) return null;
const { world, input, shared } = run;
return { world, shared, project: state.project, heldKeys: input.heldKeys, paused: state.paused };
return {
world,
shared,
project: state.project,
heldKeys: input.heldKeys,
paused: state.paused,
audio: parts.heard(),
};
},
takesLong: wait,
files: (type) => parts.stores[type].list(),
+75 -13
View File
@@ -14,6 +14,8 @@ export interface InputStep {
at?: { x?: number; y?: number; object?: string };
wheel?: number;
wait?: number;
/** Holds the game after the step before (#435); it comes last. */
pause?: boolean;
}
/** What `sendInput` needs of the editor. */
@@ -28,6 +30,10 @@ export interface PlayDriver {
frame(): Promise<void>;
/** Wall-clock milliseconds. */
now(): number;
/** Pauses the run, as the toolbar does. */
pause(): void;
/** Pauses the run once its time reaches `time`, with the frame before cut short to land on it; null calls it off. */
pauseAt(time: number | null): void;
}
export interface InputOutcome {
@@ -38,6 +44,9 @@ export interface InputOutcome {
held: string[];
/** Where the mouse is, in world units, and whether it is over the game's picture. */
mouse: { x: number; y: number; inside: boolean; down: boolean };
/** The game's time when the steps were done, and whether a `pause` step holds it there. */
time: number;
paused: boolean;
/** What the console got meanwhile. */
log: string[];
/** Per click on an object: whether it landed on it. */
@@ -63,8 +72,12 @@ export function keyOf(name: string): string {
export function inputProblem(steps: readonly InputStep[]): string | null {
for (const [index, step] of steps.entries()) {
const at = `Step ${index + 1}`;
const kinds = (['key', 'mouse', 'wheel', 'wait'] as const).filter((kind) => step[kind] !== undefined);
if (kinds.length !== 1) return `${at} has to do one thing: key, mouse, wheel or wait`;
const kinds = (['key', 'mouse', 'wheel', 'wait', 'pause'] as const).filter(
(kind) => step[kind] !== undefined,
);
if (kinds.length !== 1) return `${at} has to do one thing: key, mouse, wheel, wait or pause`;
if (step.pause !== undefined && (step.pause !== true || index !== steps.length - 1))
return `${at}: {"pause": true} ends the steps, it comes last`;
if (step.key === undefined && (step.hold !== undefined || step.keep !== undefined))
return `${at}: hold and keep go with a key`;
if (step.hold !== undefined && step.keep !== undefined) return `${at}: a key is held or kept, not both`;
@@ -89,11 +102,13 @@ export function totalSeconds(steps: readonly InputStep[]): number {
export function inputRefusal(driver: PlayDriver): string | null {
if (!driver.run()) return 'The game is not running: start it with play first';
if (driver.paused()) return 'The game is paused: resume it first';
if (driver.hidden())
return 'The editor tab is in the background, where the browser runs no frames: ask the user to bring it to the front';
if (driver.hidden()) return IN_BACKGROUND;
return null;
}
export const IN_BACKGROUND =
'The editor tab is in the background, where the browser runs no frames: ask the user to bring it to the front';
/**
* Plays the steps into the running game, each one taking at least one frame, so an edge a step
* makes is seen before the next one. A tapped or held key and a clicked button are let go again
@@ -109,28 +124,70 @@ export async function sendInput(
const logStart = first.world.log.length;
const startWorld = first.world;
const hits: InputOutcome['hits'] = [];
for (const [index, step] of steps.entries()) {
try {
await playStep(step, clock, (object, hit) => hits.push({ step: index + 1, object, hit }));
} catch (error) {
const why = error instanceof Error ? error.message : String(error);
throw new Error(`Step ${index + 1}: ${why} (after ${round(clock.seconds)} s of game time)`, {
cause: error,
});
const pauses = steps.at(-1)?.pause === true;
try {
for (const [index, step] of steps.entries()) {
try {
if (step.pause) {
if (!driver.paused()) driver.pause();
continue;
}
// A wait the pause comes after ends on its time exactly, not a frame beyond.
if (pauses && index === steps.length - 2 && (step.wait ?? 0) > 0)
driver.pauseAt(clock.run().world.time + step.wait!);
await playStep(step, clock, (object, hit) => hits.push({ step: index + 1, object, hit }));
} catch (error) {
const why = error instanceof Error ? error.message : String(error);
throw new Error(`Step ${index + 1}: ${why} (after ${round(clock.seconds)} s of game time)`, {
cause: error,
});
}
}
} finally {
if (pauses) driver.pauseAt(null);
}
const { world, input } = clock.run();
const { x, y, inside, down } = world.pointer;
return {
seconds: round(clock.seconds),
seconds: exact(clock.seconds),
frames: clock.frames,
held: [...input.heldKeys],
mouse: { x: round(x), y: round(y), inside, down },
time: exact(world.time),
paused: driver.paused(),
log: world === startWorld ? world.log.slice(logStart) : [...world.log],
hits,
};
}
/**
* Lets the run go on until its time reaches `until` and holds it there (#435): the time it holds at,
* earlier when the user or a `Break` node paused it first.
*/
export async function runUntil(driver: PlayDriver, until: number, deadlineMs: number): Promise<number> {
const deadline = driver.now() + deadlineMs;
const world = driver.run()?.world;
driver.pauseAt(until);
try {
for (;;) {
const run = driver.run();
if (!run) throw new Error('The game stopped before it got there');
if (run.world !== world)
throw new Error(
'A Load scene or Restart game started the time over before it got there; the game runs on',
);
if (driver.paused()) return exact(run.world.time);
if (driver.now() > deadline)
throw new Error(
`The game time stood still at ${exact(run.world.time)} s: paused by a behavior, or the editor tab went to the background`,
);
await driver.frame();
}
} finally {
driver.pauseAt(null);
}
}
async function playStep(
step: InputStep,
clock: GameClock,
@@ -255,3 +312,8 @@ class GameClock {
function round(value: number): number {
return Math.round(value * 100) / 100;
}
/** A game time to the millisecond: a pause lands on it, and a hundredth would hide a frame. */
export function exact(value: number): number {
return Math.round(value * 1000) / 1000;
}
+10 -1
View File
@@ -17,6 +17,7 @@ import { instrumentSourcesOf, soundingNotes } from '../../core/instrument/compil
import { noteName } from '../../core/instrument/notes';
import { patternInstrument, patternLength, patternSequence } from '../../core/music/pattern';
import { itemNotes, songLength } from '../../core/music/song';
import { trackParams } from '../../core/music/trackParams';
import { TICKS_PER_QUARTER } from '../../core/music/time';
import { CLIP_ROWS } from '../../core/properties/sounds';
import { VIDEO_CLIP_ROWS } from '../../core/properties/videos';
@@ -100,7 +101,15 @@ export function getAsset(project: Project, assetId: string, readCel?: CelLookup)
plain.notes = asset.notes.map((note) => ({ ...note, name: noteName(note.note) }));
plain.ticks = { perQuarter: TICKS_PER_QUARTER, length: patternLength(asset) };
}
if (asset.type === 'song') plain.ticks = { perQuarter: TICKS_PER_QUARTER, length: songLength(asset) };
if (asset.type === 'song') {
plain.ticks = { perQuarter: TICKS_PER_QUARTER, length: songLength(asset) };
// What a lane of each track can move (#337): set_song_lane takes them as "param:<name>".
plain.tracks = asset.tracks.map((track, i) => {
const params = trackParams(project, asset, track);
const stored = (plain.tracks as object[])[i];
return params.length > 0 ? { ...stored, params } : stored;
});
}
const silent = silentParts(project, asset);
if (silent.length > 0) plain.silent = silent;
if (asset.type === 'sound' || asset.type === 'video') {
+7 -1
View File
@@ -8,6 +8,9 @@ import type { Entity } from '../../runtime/Entity';
import type { World } from '../../runtime/World';
import { cameraOfItems, cameraOfWorld } from '../../viewport/activeCamera';
import { collectHolderItems } from '../../viewport/items';
import type { HeardSound } from '../../viewport/SoundLibrary';
import { audioView } from './audioView';
import { exact } from './playInput';
/** The running game as `get_runtime` reads it (#429). */
export interface RuntimeFacts {
@@ -18,6 +21,8 @@ export interface RuntimeFacts {
heldKeys: ReadonlySet<string>;
/** Paused from the toolbar; the game's own pause is the world's. */
paused: boolean;
/** What is heard (#436); left out where the run plays to nobody. */
audio?: HeardSound;
}
/** How many console lines an answer carries unless the agent asks for more. */
@@ -55,7 +60,7 @@ export function runtimeView(
const lines = options.console ?? CONSOLE_LINES;
return {
scene: { id: world.sceneId, name: findAsset(project, world.sceneId)?.name ?? '' },
time: round(world.time),
time: exact(world.time),
paused: facts.paused ? 'toolbar' : world.paused ? 'by the game' : false,
camera: camera
? { id: camera.id, x: round(camera.x), y: round(camera.y), ...cameraFrame(camera.props) }
@@ -65,6 +70,7 @@ export function runtimeView(
mouse: { ...world.pointer, x: round(world.pointer.x), y: round(world.pointer.y) },
objects: picked.map((entity) => entityRow(entity, project, named.length > 0)),
variables: sharedRows(facts.shared, project),
...(facts.audio && { audio: audioView(facts.audio, project, world) }),
console: lines > 0 ? world.log.slice(-lines) : [],
consoleLines: world.log.length,
};
+14 -6
View File
@@ -59,7 +59,7 @@
{
"name": "get_asset",
"title": "Read an asset",
"description": "One asset as stored: its property values, the rows of its properties (key, kind, default), the objects of a scene or container with their names and positions (the content of a placed container too, marked fromContainer), what the rows of an animation may aim at (targets), the cells of a small tilemap, the pixels of small pixel art as rows of palette indices (colors as \"#rrggbb\"), the notes of a pattern with their names and how many ticks it is long, the rows a clip of a sound or video has (clipProperties), how many seconds a loaded sound lasts, and under silent what of a pattern, song or composition is not heard: all of it without an instrument, or the notes its instrument does not sound. Under usedBy: every asset naming it or one of its frames or clips, with the objects, nodes, items or keys in there that do (none for a property of the asset itself), and the project's start scene, master FX or Master routing; empty when nothing uses it. A graph is only counted here; read it with get_graph.",
"description": "One asset as stored: its property values, the rows of its properties (key, kind, default), the objects of a scene or container with their names and positions (the content of a placed container too, marked fromContainer), what the rows of an animation may aim at (targets), the cells of a small tilemap, the pixels of small pixel art as rows of palette indices (colors as \"#rrggbb\"), the notes of a pattern with their names and how many ticks it is long, on each track of a song its automation lanes and the parameters a lane can move (params), the markers of a song with their ids, names and ticks, the rows a clip of a sound or video has (clipProperties), how many seconds a loaded sound lasts, and under silent what of a pattern, song or composition is not heard: all of it without an instrument, or the notes its instrument does not sound. Under usedBy: every asset naming it or one of its frames or clips, with the objects, nodes, items or keys in there that do (none for a property of the asset itself), and the project's start scene, master FX or Master routing; empty when nothing uses it. A graph is only counted here; read it with get_graph.",
"inputSchema": {
"type": "object",
"properties": { "asset": { "type": "string", "description": "Id of the asset." } },
@@ -187,10 +187,17 @@
{
"name": "play",
"title": "Play the game",
"description": "Plays the game in the editor, pauses, resumes or stops it. Edits made while it runs change the running copy only and are gone when it stops.",
"description": "Plays the game in the editor, pauses, resumes or stops it. With until, play, resume or pause let the game run on to that game time, as get_runtime gives it, and hold it there, exactly: the call answers once it is paused, and get_runtime and screenshot then show that moment. Edits made while it runs change the running copy only and are gone when it stops.",
"inputSchema": {
"type": "object",
"properties": { "action": { "type": "string", "enum": ["play", "pause", "resume", "stop"] } },
"properties": {
"action": { "type": "string", "enum": ["play", "pause", "resume", "stop"] },
"until": {
"type": "number",
"minimum": 0,
"description": "Game seconds to pause at, at most 30 ahead. A Load scene or Restart game starts the time over and calls it off."
}
},
"required": ["action"],
"additionalProperties": false
},
@@ -199,7 +206,7 @@
{
"name": "play_input",
"title": "Keys and mouse in the running game",
"description": "Plays keys and the mouse into the running game like a player, step by step; each step takes at least one frame, and hold and wait count game seconds. A step does one thing: {\"key\": \"ArrowLeft\", \"hold\": 0.5} holds a key; {\"key\": \"Space\"} taps it for one frame; {\"key\": \"Shift\", \"keep\": \"down\"} presses it and leaves it held after the call, \"keep\": \"up\" lets it go; {\"mouse\": \"click\", \"at\": {\"object\": \"<object id>\"}} or {\"mouse\": \"click\", \"at\": {\"x\": 10, \"y\": -20}} clicks the left button there (world units, as get_runtime gives them); \"down\", \"up\" and \"move\" do just that; {\"wheel\": -1} turns the wheel; {\"wait\": 1} lets the game run on. Keys go by their KeyboardEvent.key name: \"a\", \"ArrowUp\", \"Enter\", \"Space\". Answers the game time it took, the keys still held, where the mouse is, whether a click on an object landed on it, and the new console lines; read positions and variables with get_runtime. Needs the game playing, not paused, and the editor tab in front.",
"description": "Plays keys and the mouse into the running game like a player, step by step; each step takes at least one frame, and hold and wait count game seconds. A step does one thing: {\"key\": \"ArrowLeft\", \"hold\": 0.5} holds a key; {\"key\": \"Space\"} taps it for one frame; {\"key\": \"Shift\", \"keep\": \"down\"} presses it and leaves it held after the call, \"keep\": \"up\" lets it go; {\"mouse\": \"click\", \"at\": {\"object\": \"<object id>\"}} or {\"mouse\": \"click\", \"at\": {\"x\": 10, \"y\": -20}} clicks the left button there (world units, as get_runtime gives them); \"down\", \"up\" and \"move\" do just that; {\"wheel\": -1} turns the wheel; {\"wait\": 1} lets the game run on; {\"pause\": true}, the last step, holds the game right after the step before, a wait to its time exactly, so get_runtime and screenshot show that moment. Keys go by their KeyboardEvent.key name: \"a\", \"ArrowUp\", \"Enter\", \"Space\". Answers the game time it took and the game time it is at, whether it is paused, the keys still held, where the mouse is, whether a click on an object landed on it, and the new console lines; read positions and variables with get_runtime. Needs the game playing, not paused, and the editor tab in front.",
"inputSchema": {
"type": "object",
"properties": {
@@ -228,7 +235,8 @@
"additionalProperties": false
},
"wheel": { "type": "integer", "minimum": -10, "maximum": 10 },
"wait": { "type": "number", "minimum": 0, "maximum": 10, "description": "Game seconds." }
"wait": { "type": "number", "minimum": 0, "maximum": 10, "description": "Game seconds." },
"pause": { "type": "boolean", "description": "true: holds the game here; the last step." }
},
"additionalProperties": false
}
@@ -242,7 +250,7 @@
{
"name": "get_runtime",
"title": "Read the running game",
"description": "The game as it runs now: game time, whether it is paused, the camera, the keys held and the mouse, every object with its id, type and position in world units (behaviors with their variables), the values of the Variables assets, and the last console lines. Name objects (ids or names) to get all their properties.",
"description": "The game as it runs now: game time, whether it is paused, the camera, the keys held and the mouse, every object with its id, type and position in world units (behaviors with their variables), the values of the Variables assets, the sound (audio), and the last console lines. Under audio: the audio clock (with a hint when it stands still while the game runs: then nothing is heard); the level of the master, of each bus of the mix, of each pattern, song or composition playing and of the tracks of a song, as peak and RMS in dB over the last window seconds (silent below -90 dB; held while the game is paused); the voices sounding now (a sound, or an instrument with its note), each with its bus and the object it belongs to (by); planned, the notes already set to start; the patterns, songs and compositions playing; waiting, the voices whose bytes have not arrived; and failed, the files that would not load. Name objects (ids or names) to get all their properties.",
"inputSchema": {
"type": "object",
"properties": {
+17 -2
View File
@@ -1,3 +1,4 @@
import type { AutomationTarget } from '../../core/music/automation';
import { findAssetOf } from '../../core/model/Project';
import type { HeadPart } from '../song/songView';
import { soundLibrary } from '../../viewport/SoundLibrary';
@@ -12,12 +13,17 @@ export function songHooks({ state, editor }: HookContext) {
current: () => songEditor.currentAssetId,
info: () => songEditor.shownInfo,
rect: () => songEditor.canvasRect(),
/** The tracks, items and loop as the model holds them. */
/** The tracks, items, loop and markers as the model holds them. */
model: (assetId: string) => {
const asset = findAssetOf(state.project, assetId, 'song');
return asset
? (JSON.parse(
JSON.stringify({ tracks: asset.tracks, items: asset.items, loop: asset.loop ?? null }),
JSON.stringify({
tracks: asset.tracks,
items: asset.items,
loop: asset.loop ?? null,
markers: asset.markers ?? [],
}),
) as unknown)
: null;
},
@@ -29,6 +35,15 @@ export function songHooks({ state, editor }: HookContext) {
headAt: (track: number, part: HeadPart, share?: number) => timeline.headPoint(track, part, share),
rulerAt: (tick: number) => timeline.rulerPoint(tick),
loopAt: (tick: number) => timeline.loopPoint(tick),
/** The markers (#338): where a tick of their strip is on screen, and each as drawn. */
markerAt: (tick: number) => timeline.markerPoint(tick),
markers: () => timeline.markersShown(),
/** The automation lanes (#337): where a value of one is drawn, its head, and the rows as shown. */
automationAt: (track: number, target: AutomationTarget, tick: number, value: number) =>
timeline.automationPoint(track, target, tick, value),
automationHeadAt: (track: number, target: AutomationTarget) =>
timeline.automationHeadPoint(track, target),
automation: () => timeline.automationShown(),
view: () => timeline.viewInfo(),
items: () => timeline.itemsDrawn(),
playAt: () => ({ ...centreOf(songEditor.playButton), text: songEditor.playButton.text }),
+23 -6
View File
@@ -16,6 +16,7 @@ import {
TRACK_MIN_DB,
} from '../../core/music/song';
import type { MenuEntry } from '../../ui';
import { laneLabel } from '../song/laneView';
import { ownFxEntries } from '../song/ownFx';
import type { EditorState } from '../state';
import { createFieldRow } from './fieldFactory';
@@ -33,6 +34,9 @@ const FX_ROW = {
/** What an item's pick says while it is empty (#352): what its asset names. */
const FROM_ASSET = 'From the asset';
/** What a row of a track sets: one of its own values, never its lanes (#337). */
type TrackField = Exclude<keyof SongTrack, 'automation'>;
const TRACK_ROWS: readonly PropertyDef[] = [
{
key: 'volume',
@@ -104,7 +108,13 @@ export function songSignature(
const asset = findAssetOf(project, assetId, 'song');
const item = itemOf(asset, itemIds);
if (item) return `item:${item.id}:${item.kind}`;
return asset?.tracks.some((t) => t.id === trackId) ? `track:${trackId}` : 'missing';
const track = asset?.tracks.find((t) => t.id === trackId);
return track ? `track:${trackId}:${ruledLanes(track).join(',')}` : 'missing';
}
/** The lanes with a point on a track (#337): what its fields no longer set while it plays. */
function ruledLanes(track: SongTrack): string[] {
return (track.automation ?? []).flatMap((lane) => (lane.points.length > 0 ? [laneLabel(lane.target)] : []));
}
/** The item picked last (#334), else the track: its rows. */
@@ -137,7 +147,7 @@ export function refreshSongSection(
const track = asset?.tracks.find((t) => t.id === trackId);
if (!track) return false;
host.nameField?.setValue(track.name);
for (const def of TRACK_ROWS) host.fields.get(def.key)?.setValue(track[def.key as keyof SongTrack]);
for (const def of TRACK_ROWS) host.fields.get(def.key)?.setValue(track[def.key as TrackField]);
return true;
}
@@ -160,22 +170,29 @@ function buildTrack(host: InspectorHost, asset: SongAsset, track: SongTrack): vo
const count = asset.items.filter((i) => i.track === track.id).length;
addNote(host, `Track of the song "${asset.name}", ${count} ${count === 1 ? 'item' : 'items'}.`);
addNote(host, 'Its Sound FX takes everything on it together, before its volume and pan.', 'small');
const ruled = ruledLanes(track);
if (ruled.length > 0)
addNote(
host,
`Automation lanes rule ${ruled.join(', ')} while it plays; a field here is only heard once its lane is cleared.`,
'small',
);
for (const def of TRACK_ROWS) {
const commit = (value: SongTrack[keyof SongTrack]) =>
const commit = (value: SongTrack[TrackField]) =>
new SetSongCommand(
asset.id,
{ tracks: withTrack(state.project, asset.id, track.id, { [def.key]: value }) },
`Set track ${def.key}`,
);
const handlers = previewingHandlers<SongTrack, SongTrack[keyof SongTrack]>(host, {
const handlers = previewingHandlers<SongTrack, SongTrack[TrackField]>(host, {
target: () => findAssetOf(state.project, asset.id, 'song')?.tracks.find((t) => t.id === track.id),
read: (target) => target[def.key as keyof SongTrack],
read: (target) => target[def.key as TrackField],
write: (target, value) => Object.assign(target, { [def.key]: value }),
commit,
});
if (def.key === FX_ROW.key)
handlers.moreChoices = (current) => ownFxEntries(state, asset.id, current, commit);
const row = createFieldRow(host, def, track[def.key as keyof SongTrack], handlers);
const row = createFieldRow(host, def, track[def.key as TrackField], handlers);
host.fields.set(def.key, row);
host.body.add(row.row);
}
+120
View File
@@ -0,0 +1,120 @@
import { heldRange } from '../../core/music/automation';
import { defaultParamRange, type TrackParam } from '../../core/music/trackParams';
import { type Button, Dropdown, Label, Menu, NumberField } from '../../ui';
import { dialogSlot, FormDialog } from '../FormDialog';
const PICKER_WIDTH = 200;
const FIELD_WIDTH = 90;
const slot = dialogSlot<ParamLaneDialog>();
/** The open dialog, for the check scripts. */
export const currentParamLaneDialog = slot.current;
/** Opens the dialog centred, closing whichever one was open before it. */
export const openParamLaneDialog = slot.open;
/**
* A lane for a parameter of a track (#337): which parameter, and the range its lane spans – a
* parameter says nothing of its own. A new one starts at a range round its default; `editing` sets
* the range of a lane there is, its parameter fixed.
*/
export class ParamLaneDialog extends FormDialog {
private param: string;
private min: number;
private max: number;
private readonly picker: Dropdown;
private readonly minField: NumberField;
private readonly maxField: NumberField;
private readonly summary: Label;
/** Public for the check scripts. */
readonly confirm: Button;
constructor(
private readonly offered: readonly TrackParam[],
editing: { name: string; min: number; max: number } | null,
onConfirm: (name: string, range: { min: number; max: number }) => void,
) {
super(340, editing ? `Range of the ${editing.name} lane` : 'Add a parameter lane');
this.param = editing?.name ?? offered[0]?.name ?? '';
this.picker = new Dropdown('', { width: PICKER_WIDTH, onClick: () => this.openPicker() });
this.picker.setEnabled(!editing && offered.length > 1);
this.content.add(this.formRow(this.fieldLabel('Parameter'), this.picker));
({ min: this.min, max: this.max } = editing ?? this.rangeOf(this.param));
const field = (value: number, set: (value: number) => void) => {
const made = new NumberField(value, {
onInput: (now) => (set(now), this.refresh()),
onChange: (now) => (set(now), this.refresh()),
});
made.minWidth = FIELD_WIDTH;
return made;
};
this.minField = field(this.min, (value) => (this.min = value));
this.maxField = field(this.max, (value) => (this.max = value));
this.content.add(this.formRow(this.fieldLabel('From'), this.minField));
this.content.add(this.formRow(this.fieldLabel('To'), this.maxField));
this.summary = new Label('', { style: 'muted' });
this.content.add(this.summary);
this.confirm = this.addButtons(editing ? 'Set range' : 'Add lane', 'primary', () => {
const range = this.range();
if (range && this.param) onConfirm(this.param, range);
});
this.refresh();
}
/** What the dialog holds, for the check scripts. */
debugInfo(): { name: string; offered: string[]; min: number; max: number; ok: boolean } {
return {
name: this.param,
offered: this.offered.map((p) => p.name),
min: this.min,
max: this.max,
ok: this.confirm.enabled,
};
}
/** Picks a parameter and a range by value rather than by hand, for the check scripts. */
pick(name: string, range?: { min: number; max: number }): void {
this.param = name;
({ min: this.min, max: this.max } = range ?? this.rangeOf(name));
this.minField.setNumber(this.min);
this.maxField.setNumber(this.max);
this.refresh();
}
private rangeOf(name: string): { min: number; max: number } {
return defaultParamRange(this.offered.find((p) => p.name === name)?.default ?? 0);
}
private range(): { min: number; max: number } | null {
return heldRange(this.min, this.max);
}
private openPicker(): void {
new Menu(
this.offered.map((param) => ({
label: param.name,
current: param.name === this.param,
onSelect: () => this.pick(param.name),
})),
{ searchable: this.offered.length > 9 },
).openBelow(this.picker);
}
private refresh(): void {
this.picker.setText(this.param || '—');
const range = this.range();
const param = this.offered.find((p) => p.name === this.param);
this.summary.setText(
!this.param
? 'Nothing on this track has a parameter to move.'
: !range
? '"To" has to be above "From".'
: param
? `Its default is ${param.default}.`
: 'Nothing on the track has this parameter now.',
);
this.confirm.setEnabled(!!this.param && range !== null);
this.refreshLayout();
}
}
File diff suppressed because it is too large. Load diff
+43
View File
@@ -0,0 +1,43 @@
import { listedAssets } from '../../core/music/embedded';
import type { KeyInterpolation } from '../../core/model/Asset';
import { EASING_DIRECTIONS, EASING_SHAPES, easingOf, easingValue } from '../../core/model/animation';
import type { Project } from '../../core/model/Project';
import type { MenuEntry } from '../../ui';
/**
* The easings as entries of a menu (#337), the same choice the inspector gives an animation key: Linear
* and Step straight away, every curve with its directions in a submenu, and the Curve assets of the
* project. The one in use is marked.
*/
export function easingEntries(
project: Project,
current: KeyInterpolation,
set: (ease: KeyInterpolation) => void,
): MenuEntry[] {
const now = easingOf(current);
const curves = listedAssets(project)
.filter((a) => a.type === 'curve')
.sort((a, b) => a.name.localeCompare(b.name));
return EASING_SHAPES.map(({ value, label }): MenuEntry => {
if (value === 'linear' || value === 'step')
return { label, current: now.shape === value, onSelect: () => set(value) };
if (value === 'curve')
return {
label,
disabled: curves.length === 0,
submenu: curves.map((curve) => ({
label: curve.name,
current: now.shape === 'curve' && now.curve === curve.id,
onSelect: () => set(easingValue('curve', undefined, curve.id)),
})),
};
return {
label,
submenu: EASING_DIRECTIONS.map((direction) => ({
label: direction.label,
current: now.shape === value && now.direction === direction.value,
onSelect: () => set(easingValue(value, direction.value)),
})),
};
});
}
+132
View File
@@ -0,0 +1,132 @@
import {
AUTOMATION_TARGETS,
type AutomationLane,
type AutomationPoint,
type AutomationTarget,
type LaneSpec,
laneRange,
paramOf,
paramTarget,
} from '../../core/music/automation';
import type { SongTrack } from '../../core/music/song';
import type { KeyInterpolation } from '../../core/model/Asset';
/**
* Editing the points of a lane in the Song tab (#337), apart from Pixi so a test can drive it. Points
* stay in the order of their ticks: one that is dragged stops at its neighbours, so two on one tick –
* a jump – keep which comes first.
*/
/** The points of a track's lane for that target; none while it has no lane. */
export function lanePoints(track: SongTrack, target: AutomationTarget): AutomationPoint[] {
return track.automation?.find((lane) => lane.target === target)?.points ?? [];
}
/**
* The track with these points on that lane. With none the lane of the volume or the pan goes; a
* parameter's stays for its range. With the last lane `automation` goes.
*/
export function withLanePoints(
track: SongTrack,
target: AutomationTarget,
points: readonly AutomationPoint[],
): SongTrack {
const old = track.automation?.find((lane) => lane.target === target);
const range = paramOf(target) !== null ? laneRange(old ?? { target }) : null;
const lane: AutomationLane | null =
points.length > 0 || range ? { target, points: [...points], ...range } : null;
return withLane(track, target, lane);
}
/** The track with a lane for that parameter in this range, its points held to it; a new one is empty. */
export function withParamLane(
track: SongTrack,
name: string,
range: { min: number; max: number },
): SongTrack {
const target = paramTarget(name);
const points = lanePoints(track, target).map((point) => ({
...point,
value: Math.max(range.min, Math.min(range.max, point.value)),
}));
return withLane(track, target, { target, points, min: range.min, max: range.max });
}
/** The track without that lane at all, a parameter's range with it. */
export function withoutLane(track: SongTrack, target: AutomationTarget): SongTrack {
return withLane(track, target, null);
}
/** In a fixed order: the volume, the pan, then the parameters as they came; a new one last. */
function withLane(track: SongTrack, target: AutomationTarget, lane: AutomationLane | null): SongTrack {
const others = (track.automation ?? []).filter((l) => l.target !== target);
const fixed = AUTOMATION_TARGETS.flatMap((t) =>
t === target && lane ? [lane] : others.filter((l) => l.target === t),
);
const params = (track.automation ?? []).flatMap((l) =>
paramOf(l.target) === null ? [] : l.target === target ? (lane ? [lane] : []) : [l],
);
const known = (track.automation ?? []).some((l) => l.target === target);
if (lane && paramOf(target) !== null && !known) params.push(lane);
const lanes = [...fixed, ...params];
const { automation: _, ...rest } = track;
return lanes.length > 0 ? { ...rest, automation: lanes } : rest;
}
function held(kind: LaneSpec, value: number): number {
const { min, max } = laneRange(kind);
return Math.max(min, Math.min(max, value));
}
/** A new point, after those already on its tick, and where it went. */
export function addedPoint(
points: readonly AutomationPoint[],
kind: LaneSpec,
tick: number,
value: number,
): { points: AutomationPoint[]; index: number } {
const at = Math.max(0, Math.round(tick));
let index = 0;
while (index < points.length && points[index]!.tick <= at) index++;
const out = [...points];
out.splice(index, 0, { tick: at, value: held(kind, value) });
return { points: out, index };
}
/** The point at `index` dragged to a tick and a value, the tick held between its neighbours'. */
export function draggedPoint(
points: readonly AutomationPoint[],
kind: LaneSpec,
index: number,
tick: number,
value: number,
): AutomationPoint[] {
const point = points[index];
if (!point) return [...points];
const low = points[index - 1]?.tick ?? 0;
const high = points[index + 1]?.tick ?? Infinity;
const out = [...points];
out[index] = {
...point,
tick: Math.min(high, Math.max(low, Math.round(tick))),
value: held(kind, value),
};
return out;
}
export function withoutPoint(points: readonly AutomationPoint[], index: number): AutomationPoint[] {
return points.filter((_, i) => i !== index);
}
/** The point at `index` easing to the next one along `ease`; linear is stored as no easing. */
export function easedPoint(
points: readonly AutomationPoint[],
index: number,
ease: KeyInterpolation,
): AutomationPoint[] {
return points.map((point, i) => {
if (i !== index) return point;
const { ease: _, ...rest } = point;
return ease === 'linear' ? rest : { ...rest, ease };
});
}
+186
View File
@@ -0,0 +1,186 @@
import {
AUTOMATION_TARGETS,
type AutomationPoint,
type AutomationTarget,
type LaneKind,
laneKind,
laneOf,
laneParams,
laneRange,
laneValueAt,
laneValueBefore,
paramOf,
paramTarget,
type LaneSpec,
type SequenceLanePoint,
} from '../../core/music/automation';
import type { SongTrack } from '../../core/music/song';
import { clamp } from '../../ui';
import { tickToX, xToTick } from '../music/timeView';
import type { SongView } from './songView';
/**
* The automation lanes of the Song tab (#337), apart from Pixi so a test can drive them: each lane a
* row of its own right under its track, as high as a track, so the rows stay evenly spaced and the
* geometry of `songView` holds for them as for the tracks.
*/
/** A row of the tracks: a track itself, or one of its lanes. `track` is the track's index. */
export interface SongRow {
track: number;
lane: AutomationTarget | null;
}
/** The lanes a track was opened to or closed on by hand, by its id. */
export type OpenLanes = ReadonlyMap<string, readonly AutomationTarget[]>;
/** What the head of a lane calls it: the volume, the pan, or the name of its parameter. */
export function laneLabel(target: AutomationTarget): string {
return target === 'volume' ? 'Volume' : target === 'pan' ? 'Pan' : (paramOf(target) ?? target);
}
/** Every lane a track can show: the volume, the pan, and each parameter it has a lane of (#337). */
export function trackLanes(track: SongTrack): AutomationTarget[] {
return [...AUTOMATION_TARGETS, ...laneParams(track).map(paramTarget)];
}
/**
* The lanes a track shows, in a fixed order: as opened by hand, else the ones with a point and every
* parameter's – one is only there because it was added.
*/
export function shownLanes(track: SongTrack, open: OpenLanes): AutomationTarget[] {
const named = open.get(track.id);
return trackLanes(track).filter((target) =>
named ? named.includes(target) : paramOf(target) !== null || !!laneOf(track, target),
);
}
/** A track's lane of that target as far as its kind goes: a parameter's range is on its lane. */
export function laneKindOf(track: SongTrack, target: AutomationTarget): LaneKind {
const lane = track.automation?.find((l) => l.target === target);
return lane ? { target, min: lane.min, max: lane.max } : { target };
}
export function songRows(tracks: readonly SongTrack[], open: OpenLanes): SongRow[] {
return tracks.flatMap((track, i) => [
{ track: i, lane: null },
...shownLanes(track, open).map((lane) => ({ track: i, lane })),
]);
}
/** The row of each track, by its id. */
export function trackRows(tracks: readonly SongTrack[], rows: readonly SongRow[]): Map<string, number> {
const out = new Map<string, number>();
rows.forEach((row, i) => {
if (row.lane === null) out.set(tracks[row.track]!.id, i);
});
return out;
}
/** What is left free above and below the way of a lane, so a point at either end can still be taken. */
export const LANE_INSET = 6;
/** How near a point a press takes it. */
export const POINT_REACH = 6;
/** Where a value is drawn in a lane `h` high, from its top; past the range it is held to the edge. */
export function laneY(kind: LaneSpec, value: number, h: number): number {
const { min, max } = laneRange(kind);
const share = clamp((value - min) / (max - min), 0, 1);
return LANE_INSET + (1 - share) * Math.max(1, h - 2 * LANE_INSET);
}
/**
* The step a lane's values are set in: a tenth of a dB, a hundredth of the pan, and for a parameter
* a power of ten that cuts its range into at least a hundred.
*/
export function laneStep(kind: LaneSpec): number {
const { target } = laneKind(kind);
if (target === 'volume') return 0.1;
if (target === 'pan') return 0.01;
const { min, max } = laneRange(kind);
return 10 ** Math.floor(Math.log10((max - min) / 100));
}
/** The value a height of a lane stands for, in its step. */
export function laneValue(kind: LaneSpec, y: number, h: number): number {
const { min, max } = laneRange(kind);
const share = clamp(1 - (y - LANE_INSET) / Math.max(1, h - 2 * LANE_INSET), 0, 1);
const step = laneStep(kind);
return clamp(roundTo(min + share * (max - min), step), min, max);
}
/** Rounded to a step, without the tail a float adds to a tenth. */
function roundTo(value: number, step: number): number {
const digits = Math.max(0, -Math.floor(Math.log10(step)));
return Number((Math.round(value / step) * step).toFixed(Math.min(digits, 12)));
}
/** The point of a lane under a point of its row, the last drawn first; null for none. */
export function pointAt(
view: SongView,
target: LaneSpec,
points: readonly AutomationPoint[],
x: number,
y: number,
): number | null {
for (let i = points.length - 1; i >= 0; i--) {
const point = points[i]!;
const dx = tickToX(view, point.tick) - x;
const dy = laneY(target, point.value, view.trackHeight) - y;
if (Math.abs(dx) <= POINT_REACH && Math.abs(dy) <= POINT_REACH) return i;
}
return null;
}
/**
* The way of a lane across `width` pixels, a corner every `every` pixels and at each point – a jump
* shows as an upright –, in the row's own pixels.
*/
export function laneWay(
view: SongView,
target: LaneSpec,
points: readonly SequenceLanePoint[],
width: number,
every = 2,
): { x: number; y: number }[] {
if (points.length === 0) return [];
const h = view.trackHeight;
const way: { x: number; y: number }[] = [];
const corners = new Set<number>();
for (let x = 0; x <= width; x += every) corners.add(x);
for (const point of points) {
const x = tickToX(view, point.tick);
if (x > 0 && x < width) corners.add(x);
}
for (const x of [...corners].sort((a, b) => a - b)) {
const at = points.findIndex((p) => tickToX(view, p.tick) === x);
if (at < 0) {
way.push({ x, y: laneY(target, laneValueAt(points, xToTick(view, x)), h) });
continue;
}
// On a point the way comes in with the value before it and leaves from the last point on its tick.
const tick = points[at]!.tick;
const before = laneValueBefore(points, tick);
if (before !== points[at]!.value) way.push({ x, y: laneY(target, before, h) });
for (let i = at; i < points.length && points[i]!.tick === tick; i++)
way.push({ x, y: laneY(target, points[i]!.value, h) });
}
return way;
}
/** A value as the head of a lane and the sign over a point say it. */
export function laneValueText(kind: LaneSpec, value: number): string {
const { target } = laneKind(kind);
if (target === 'volume') {
const { min } = laneRange('volume');
if (value <= min) return '-inf dB';
const shown = Math.round(value * 10) / 10;
return `${shown > 0 ? '+' : ''}${shown.toFixed(1)} dB`;
}
if (target === 'pan') {
const share = Math.round(Math.abs(value) * 100);
return share === 0 ? 'Centre' : `${share}% ${value < 0 ? 'left' : 'right'}`;
}
const digits = Math.max(0, -Math.floor(Math.log10(laneStep(kind))));
return roundTo(value, laneStep(kind)).toFixed(Math.min(digits, 6));
}
+51
View File
@@ -0,0 +1,51 @@
import type { SongMarker } from '../../core/music/markers';
import { RULER_HEIGHT, type TimeView, tickToX } from '../music/timeView';
/** The strip under the song's ruler that holds its markers (#338). */
export const MARKER_STRIP = 18;
/** Where the tracks of the Song tab start: below the ruler and its markers. */
export const SONG_TOP = RULER_HEIGHT + MARKER_STRIP;
/** How far either side of its line a press still takes a marker. */
export const MARKER_REACH = 5;
/** The flag at the head of a marker's line, and the room between it and the name. */
export const MARKER_FLAG = 8;
const NAME_GAP = 3;
/** Where a marker is drawn on the strip: its line at `x`, flag and name over `w` pixels right of it. */
export interface MarkerSpan {
id: string;
x: number;
w: number;
}
/**
* Each marker as drawn on the strip: its flag and as much of its name as there is room for before
* the next one. `measure` gives the width of a name in pixels.
*/
export function markerSpans(
view: TimeView,
markers: readonly SongMarker[],
measure: (name: string) => number,
): MarkerSpan[] {
return markers.map((marker, i) => {
const x = tickToX(view, marker.tick);
const next = markers[i + 1];
const room = next ? tickToX(view, next.tick) - x - 2 : Infinity;
const w = Math.max(MARKER_FLAG, Math.min(MARKER_FLAG + NAME_GAP + measure(marker.name), room));
return { id: marker.id, x, w };
});
}
/** Where a marker's name starts, right of its flag. */
export function markerNameX(span: MarkerSpan): number {
return span.x + MARKER_FLAG + NAME_GAP;
}
/** The marker under `x` on the strip, the last drawn first: near its line, or on its flag and name. */
export function markerAt(spans: readonly MarkerSpan[], x: number): string | null {
for (let i = spans.length - 1; i >= 0; i--) {
const span = spans[i]!;
if (x >= span.x - MARKER_REACH && x <= span.x + Math.max(span.w, MARKER_REACH)) return span.id;
}
return null;
}
+16 -11
View File
@@ -99,17 +99,19 @@ export function itemBox(view: SongView, index: number, item: Pick<SongItem, 'tic
export type ItemPart = 'start' | 'body' | 'end';
/** The item under a point, the last drawn first, and which part of it: an end within reach, else the body. */
/**
* The item under a point, the last drawn first, and which part of it: an end within reach, else the
* body. `rows` is the row of each track by its id (`indexOf` while no lane is open, #337).
*/
export function itemAt(
view: SongView,
items: readonly SongItem[],
tracks: readonly SongTrack[],
rows: ReadonlyMap<string, number>,
p: Vec2,
): { id: string; part: ItemPart } | null {
const index = indexOf(tracks);
for (let i = items.length - 1; i >= 0; i--) {
const item = items[i]!;
const row = index.get(item.track);
const row = rows.get(item.track);
if (row === undefined) continue;
const box = itemBox(view, row, item);
if (p.y < box.y || p.y > box.y + box.h || p.x < box.x || p.x > box.x + box.w) continue;
@@ -125,18 +127,17 @@ export function itemAt(
export function itemsWithin(
view: SongView,
items: readonly SongItem[],
tracks: readonly SongTrack[],
rows: ReadonlyMap<string, number>,
a: Vec2,
b: Vec2,
): string[] {
const index = indexOf(tracks);
const x0 = Math.min(a.x, b.x);
const x1 = Math.max(a.x, b.x);
const y0 = Math.min(a.y, b.y);
const y1 = Math.max(a.y, b.y);
return items
.filter((item) => {
const row = index.get(item.track);
const row = rows.get(item.track);
if (row === undefined) return false;
const box = itemBox(view, row, item);
return box.x <= x1 && box.x + box.w >= x0 && box.y <= y1 && box.y + box.h >= y0;
@@ -180,11 +181,15 @@ export function indexOf(tracks: readonly SongTrack[]): Map<string, number> {
return new Map(tracks.map((t, i) => [t.id, i]));
}
/** The parts of a track's head, in its own pixels: name, sound FX (#346), mute, solo, volume and pan. */
export type HeadPart = 'name' | 'fx' | 'mute' | 'solo' | 'volume' | 'pan';
/**
* The parts of a track's head, in its own pixels: the switch that opens its lanes (#337), name, sound
* FX (#346), mute, solo, volume and pan.
*/
export type HeadPart = 'lanes' | 'name' | 'fx' | 'mute' | 'solo' | 'volume' | 'pan';
export const HEAD_BOXES: Readonly<Record<HeadPart, Box>> = {
name: { x: 8, y: 4, w: 88, h: 18 },
lanes: { x: 4, y: 5, w: 14, h: 16 },
name: { x: 22, y: 4, w: 76, h: 18 },
fx: { x: 102, y: 5, w: 22, h: 16 },
mute: { x: 128, y: 5, w: 18, h: 16 },
solo: { x: 150, y: 5, w: 18, h: 16 },
@@ -202,7 +207,7 @@ export function trackMeterBox(trackHeight: number): Box {
export function headPartAt(x: number, yInTrack: number, trackHeight = TRACK_HEIGHT): HeadPart {
const sliders = trackHeight >= SLIDERS_MIN_HEIGHT;
for (const part of ['fx', 'mute', 'solo', 'volume', 'pan'] as const) {
for (const part of ['lanes', 'fx', 'mute', 'solo', 'volume', 'pan'] as const) {
if (!sliders && (part === 'volume' || part === 'pan')) continue;
const box = HEAD_BOXES[part];
if (x >= box.x - 2 && x <= box.x + box.w + 2 && yInTrack >= box.y - 3 && yInTrack <= box.y + box.h + 3)
+2
View File
@@ -119,6 +119,8 @@ export interface NoteTiming {
tempo?: number;
/** The bus it is heard on: a track of a song (#334) or one of the mix (#325); the master where left out. */
bus?: string;
/** The note it is, for whoever asks what is heard (#436). */
note?: number;
}
/** When and how a clip of a song plays (#334); see `playClip`. */
+16 -5
View File
@@ -349,14 +349,18 @@ export function playNote(
): number {
const asset = findAssetOf(project, instrumentId, 'instrument');
if (!asset) return 0;
const built = instrumentBuildFor(asset, heldNote(struck.note), instrumentSourcesOf(project)).layers;
const note = heldNote(struck.note);
const built = instrumentBuildFor(asset, note, instrumentSourcesOf(project)).layers;
if (built.length === 0) return 0;
const layers = fx ? built.map((layer) => throughScopedFx(fx.plan, layer, FX_SCOPE)) : built;
const velocity = Number.isFinite(struck.velocity) ? Math.max(0, Math.min(1, struck.velocity)) : 1;
const level = Math.max(0, Math.min(1, struck.volume * velocity));
const duration = Number.isFinite(struck.duration) ? Math.max(0, struck.duration) : 0;
const values = { ...params, [VELOCITY_PARAM]: velocity, ...scoped(fx?.params ?? {}) };
return audio.playNote(instrumentVoiceName(instrumentId), layers, values, level, owner, duration, timing);
return audio.playNote(instrumentVoiceName(instrumentId), layers, values, level, owner, duration, {
...timing,
note,
});
}
/** Where the parameters of a note's own sound FX go by (#346): no name of a parameter has a colon. */
@@ -487,8 +491,15 @@ export function strikeNote(
duration: note.duration,
};
const timing = { at: note.at, tempo: note.tempo, ...(note.bus && { bus: note.bus }) };
const fx = fxVoiceOf(sources.project, instrumentFx(sources.project, note.instrument, ''), note.tempo);
return playNote(sources, note.instrument, struck, owner, {}, timing, fx);
// What the lanes of its track start it with (#337) reaches its instrument and its sound FX alike.
const params = note.params ?? {};
const fx = fxVoiceOf(
sources.project,
instrumentFx(sources.project, note.instrument, ''),
note.tempo,
params,
);
return playNote(sources, note.instrument, struck, owner, params, timing, fx);
}
/** A clip of a song (#334): its sound at its own volume times the item's, through its sound FX. */
@@ -505,7 +516,7 @@ export function playClip(
return audio.playClip(
playedSoundOf(ref),
{ at: clip.at, offset: clip.offset, duration: clip.duration, volume, bus: clip.bus },
clipFx(project, clip.fx, clip.tempo),
clipFx(project, clip.fx, clip.tempo, clip.params),
);
}
+298 -16
View File
@@ -30,7 +30,13 @@ import { arrivalOf, sendDelays, type TimedBus } from '../core/music/routing';
import type { SendTap } from '../core/graph/Graph';
import { MASTER_MIX, type MixBus, type MixSend, sameMix } from '../core/music/mixer';
import { limiterLatency } from '../core/soundfx/dynamics';
import type { ScheduledClip, ScheduledNote, ScheduledRamp } from '../core/music/Scheduler';
import { type AutomationTarget, paramOf, reaches } from '../core/music/automation';
import type {
ScheduledAutomation,
ScheduledClip,
ScheduledNote,
ScheduledRamp,
} from '../core/music/Scheduler';
import { dbToGain, SOUND_ITEM_MAX_DB } from '../core/music/song';
import { DEFAULT_TEMPO } from '../core/music/time';
import { playedPlan, type StreamedFile, type StreamPlan } from '../core/music/streams';
@@ -48,11 +54,13 @@ import {
FADE,
type Follow,
followCurve,
followMarks,
fadeOut,
gainNode,
GLIDE,
glideKnobs,
heldBack,
holdParam,
type Knob,
levelsOf,
move,
@@ -79,6 +87,11 @@ const MAX_SEND_DELAY = 1;
/** Samples a level meter looks back over (#359): a frame of 40 ms at 48 kHz misses no peak. */
const METER_SPAN = 2048;
/**
* Samples what the agent hears looks back over (#436): the most an analyser holds, about 0.7 s, so
* the gap between two notes does not read as silence.
*/
const WATCH_SPAN = 32768;
/** The loudest the limiter at the output lets a sample be (#359), and how fast it lets go again. */
const GUARD_DB = -1;
const GUARD_RELEASE = 0.1;
@@ -157,6 +170,8 @@ interface Voice {
owner: string;
/** The bus it is heard on, `''` straight out. */
bus: string;
/** The note it is, for a note of an instrument (#436). */
note?: number;
/** When it started on the audio clock, what its Envelopes count from. */
at: number;
sources: Source[];
@@ -252,6 +267,8 @@ interface Start {
bus?: string;
/** The copy `file` streams from (#349); its bytes are not loaded then, its head is. */
stream?: StreamFile;
/** The note it is, for a note of an instrument (#436). */
note?: number;
}
/**
@@ -260,9 +277,15 @@ interface Start {
*/
interface Pending {
id: number;
/** What it plays and who asked, as its voice will have them (#436). */
file: string;
owner: string;
note?: number;
bus: string;
fx: Layered;
timed: { params: FxParams; time: number; ease: EaseShare }[];
/** The marks of parameter lanes (#337) planned for it meanwhile, taken up once it starts. */
lanes: { param: string; automation: ScheduledAutomation }[];
/** Stopped by its number before it started. */
stopped: boolean;
/** A note released before it started: it starts and is released at once, so a one-shot is heard. */
@@ -298,6 +321,13 @@ export class SoundLibrary implements AudioSink {
*/
private guard: { input: GainNode; limiter: AudioWorkletNode | null; latency: number } | null = null;
private guardMeter: BusMeter | null = null;
/** Whether every bus is tapped for what the agent hears (#436). */
private watching = false;
/**
* The state of the audio clock as a pause took hold (#436): one the browser had suspended already
* stays so after the pause, and nothing was heard before it either.
*/
private heldFrom = 'none';
/**
* `guarded` false leaves the limiter out: for checks that follow every node of a voice. With
@@ -334,6 +364,8 @@ export class SoundLibrary implements AudioSink {
private readonly failed = new Set<string>();
private voices: Voice[] = [];
private pending: Pending[] = [];
/** The knobs a parameter's lane (#337) has planned for: the next window ramps on from there. */
private readonly followed = new WeakSet<Knob>();
/**
* The last note of each Acid (#421), by who plays it and the Acid: when it starts, until when it
* is held, its frequency and voice, and what cuts it short – each of the notes that started then.
@@ -509,7 +541,20 @@ export class SoundLibrary implements AudioSink {
const fx = { layers, params, tempo: timing.tempo ?? DEFAULT_TEMPO };
const at = timing.at ?? 0;
const bus = timing.bus ?? '';
return this.begin({ id: 0, file: name, files, volume, loop: false, owner, fx, duration, at, bus });
const note = timing.note;
return this.begin({
id: 0,
file: name,
files,
volume,
loop: false,
owner,
fx,
duration,
at,
bus,
...(note !== undefined && { note }),
});
}
/**
@@ -540,14 +585,43 @@ export class SoundLibrary implements AudioSink {
/**
* Sets the level and the pan of a track of a song (#334) in a quick glide, so a change reaches
* what is sounding on it already. A bus is made on its first use and kept.
* what is sounding on it already. A bus is made on its first use and kept. What is `automated`
* (#337) glides there once, and is its lane's from then on.
*/
setBus(name: string, gain: number, pan: number): void {
setBus(name: string, gain: number, pan: number, automated: readonly AutomationTarget[] = []): void {
const bus = this.bus(name);
const now = this.audio().currentTime;
bus.set = { gain: Math.max(0, gain), pan: Math.max(-1, Math.min(1, pan)) };
bus.gain.gain.setTargetAtTime(Math.max(0, gain), now, GLIDE / 3);
bus.pan.pan.setTargetAtTime(Math.max(-1, Math.min(1, pan)), now, GLIDE / 3);
const glide = (target: AutomationTarget, param: AudioParam, value: number) => {
const ruled = automated.includes(target);
// Its lane takes it from here (#337); one let go hands it back from where the lane left it.
if (ruled && bus.automated.has(target)) return;
if (!ruled && bus.automated.has(target)) holdParam(param, now);
param.setTargetAtTime(value, now, GLIDE / 3);
};
glide('volume', bus.gain.gain, bus.set.gain);
glide('pan', bus.pan.pan, bus.set.pan);
bus.automated = new Set(automated);
}
/**
* A bus taken along its automation over one window (#337): ramped to each mark, or jumped there.
* A fresh start drops what was planned for it from its first mark on.
*/
automateBus(name: string, automation: ScheduledAutomation): void {
const named = paramOf(automation.target);
if (named !== null) return this.automateParam(name, named, automation);
const bus = this.buses.get(name);
const first = automation.marks[0];
if (!bus || !first) return;
const param = automation.target === 'volume' ? bus.gain.gain : bus.pan.pan;
const now = this.audio().currentTime;
if (automation.fresh) holdParam(param, Math.max(now, first.at));
for (const mark of automation.marks) {
const at = Math.max(now, mark.at);
if (mark.jump) param.setValueAtTime(mark.value, at);
else param.linearRampToValueAtTime(mark.value, at);
}
}
/**
@@ -557,7 +631,7 @@ export class SoundLibrary implements AudioSink {
*/
async setBuses(levels: readonly BusLevel[]): Promise<void> {
const waits: Promise<void>[] = [];
for (const { name, gain, pan, into, sends, fx } of levels) {
for (const { name, gain, pan, into, sends, fx, automated } of levels) {
if (into !== undefined) this.bus(into);
const bus = this.bus(name, into ?? null);
if (sends)
@@ -566,12 +640,76 @@ export class SoundLibrary implements AudioSink {
sends.map((send) => ({ ...send, to: send.to ?? MASTER_MIX })),
);
else this.route(name, bus, into ?? null);
this.setBus(name, gain, pan);
this.setBus(name, gain, pan, automated);
waits.push(this.setFx(bus, fx ?? null));
this.keepParams(bus, fx ?? null, automated ?? []);
}
await Promise.all(waits);
}
/**
* A parameter taken along its lane over one window (#337): on the sound FX of the track's bus and of
* every bus inside it, and on every voice heard inside it – playing, or still waiting on its bytes.
* Each mark is read through the chain, so a knob that is not the parameter itself follows too; one
* without a parameter of that name stays where it is.
*/
private automateParam(name: string, param: string, automation: ScheduledAutomation): void {
if (automation.marks.length === 0) return;
const now = this.audio().currentTime;
for (const [busName, bus] of this.buses) {
const fx = bus.fx;
if (!fx || !reaches(name, busName)) continue;
const values = automation.marks.map((mark) => ({ ...fx.params, [param]: mark.value }));
if (bus.chain) {
const levels = values.map((params) => levelsOf(fxChainOf(fx.plan, params, fx.tempo)));
followMarks(bus.chain.knobs, levels, automation.marks, now, automation.fresh, this.followed);
}
bus.fx = { ...fx, params: values.at(-1)! };
}
for (const voice of this.voices)
if (voice.bus !== '' && reaches(name, voice.bus)) this.followVoice(voice, param, automation);
for (const waiting of this.pending)
if (waiting.bus !== '' && reaches(name, waiting.bus)) waiting.lanes.push({ param, automation });
}
/**
* A voice along a parameter's lane: a note's instrument and its own sound FX (`fx:`, see
* `strikeNote`) alike, as `Set sound FX parameters` reaches both.
*/
private followVoice(voice: Voice, param: string, automation: ScheduledAutomation): void {
const { layers, tempo } = voice.fx;
const values = automation.marks.map((mark) => ({
...voice.fx.params,
[param]: mark.value,
[`fx:${param}`]: mark.value,
}));
const levels = values.map((params) => {
const out: number[] = [];
for (const plan of layers) levelsOf(fxChainOf(plan, params, tempo), out);
return out;
});
const now = this.audio().currentTime;
followMarks(voice.knobs, levels, automation.marks, now, automation.fresh, this.followed);
voice.fx = { ...voice.fx, params: values.at(-1)! };
}
/**
* A sound FX set again as it was (#337): a parameter goes to what it is set to now – one a lane let
* go back to where it stands without one –, but one a lane rules is the lane's.
*/
private keepParams(bus: Bus, fx: FxVoice | null, automated: readonly AutomationTarget[]): void {
const held = bus.fx;
if (!fx || !held || held === fx || held.plan !== fx.plan) return;
const ruled = new Set(automated.flatMap((target) => paramOf(target) ?? []));
const params: Record<string, number> = { ...fx.params };
for (const key of ruled) if (held.params[key] !== undefined) params[key] = held.params[key];
const keys = new Set([...Object.keys(params), ...Object.keys(held.params)]);
if ([...keys].every((key) => params[key] === held.params[key])) return;
bus.fx = { ...held, params };
const levels = levelsOf(fxChainOf(held.plan, params, held.tempo));
if (bus.chain) glideKnobs(bus.chain.knobs, levels, this.audio().currentTime, 0, 'linear');
}
/**
* Lets these buses go once what is on them now has faded (#346): their chains would otherwise run
* on for nothing. One used again before then stays.
@@ -716,23 +854,23 @@ export class SoundLibrary implements AudioSink {
return [...this.buses.values()].filter((bus) => bus.meter).length;
}
private tapMeter(from: AudioNode): BusMeter {
private tapMeter(from: AudioNode, span = METER_SPAN): BusMeter {
const context = this.audio();
const split = context.createChannelSplitter(2);
const sides = [0, 1].map((channel) => {
const side = context.createAnalyser();
side.fftSize = METER_SPAN;
side.fftSize = span;
split.connect(side, channel);
return side;
});
from.connect(split);
return { split, sides, samples: new Float32Array(METER_SPAN) };
return { split, sides, samples: new Float32Array(span) };
}
private dropMeter(bus: Bus): void {
const meter = bus.meter;
private dropMeter(bus: Bus, which: 'meter' | 'watch' = 'meter'): void {
const meter = bus[which];
if (!meter) return;
bus.meter = null;
bus[which] = null;
try {
bus.pan.disconnect(meter.split);
} catch {
@@ -741,10 +879,63 @@ export class SoundLibrary implements AudioSink {
meter.split.disconnect();
}
/** Cuts the pan of a bus from where it went; its meter stays on. */
/** Cuts the pan of a bus from where it went; its meters stay on. */
private cutPan(bus: Bus): void {
bus.pan.disconnect();
if (bus.meter) bus.pan.connect(bus.meter.split);
if (bus.watch) bus.pan.connect(bus.watch.split);
}
/**
* Taps every bus for what the agent hears (#436), those made from now on as well, or lets the taps
* go: on only while the agent is switched on, so the editor pays nothing for it otherwise.
*/
watch(on: boolean): void {
if (this.watching === on) return;
this.watching = on;
for (const bus of this.buses.values()) {
if (!on) this.dropMeter(bus, 'watch');
else bus.watch ??= this.tapMeter(bus.pan, WATCH_SPAN);
}
}
/**
* What is heard now (#436): every voice and playing, the voices still waiting on their bytes, the
* files that would not load, and the level of each bus tapped by `watch` over the last
* `WATCH_SPAN` samples, after its level and pan – held where a pause holds the clock.
*/
heard(): HeardSound {
const now = this.context?.currentTime ?? 0;
const voiceOf = (voice: Voice | Pending): HeardVoice => ({
id: voice.id,
file: voice.file,
owner: voice.owner,
bus: voice.bus,
...(voice.note !== undefined && { note: voice.note }),
});
return {
window: this.context ? WATCH_SPAN / this.context.sampleRate : 0,
clock: { state: this.held ? this.heldFrom : this.clockState(), time: now, held: this.held },
voices: this.voices.map((voice) => ({
...voiceOf(voice),
startsIn: Math.max(0, voice.at - now),
released: voice.released,
ringing: voice.ringing,
})),
waiting: this.pending.filter((waiting) => !waiting.stopped).map(voiceOf),
failed: [...this.failed],
playings: this.playbacks().map(({ id, name, owner, playing, fading, baked }) => ({
id,
name,
owner,
playing,
fading,
baked,
})),
buses: [...this.buses].flatMap(([name, bus]) =>
bus.watch ? [{ name, ...readLevels(bus.watch) }] : [],
),
};
}
/** Where the voices of a bus go in; the master for `''` (#325). */
@@ -770,14 +961,17 @@ export class SoundLibrary implements AudioSink {
gain: context.createGain(),
pan: context.createStereoPanner(),
set: { gain: 1, pan: 0 },
automated: new Set(),
into: target,
sends: null,
fx: null,
chain: null,
leaving: 0,
meter: null,
watch: null,
};
made.input.connect(made.gain).connect(made.pan).connect(this.outOf(target));
if (this.watching) made.watch = this.tapMeter(made.pan, WATCH_SPAN);
this.buses.set(name, made);
return made;
}
@@ -909,10 +1103,15 @@ export class SoundLibrary implements AudioSink {
const names = new Set(this.mix.map((bus) => bus.name));
const waits: Promise<void>[] = [];
for (const { name, volume, into, sends, fx } of this.mix) {
const made = !this.buses.has(name);
const bus = this.bus(name, into);
if (sends) this.setSends(bus, sends);
else this.route(name, bus, into);
this.glideLevel(bus, this.mixVolumes.get(name) ?? volume);
const level = this.mixVolumes.get(name) ?? volume;
// A bus made now is at its level at once: made with the context, it carries the first sound of
// a run, which would otherwise be heard through a glide down from unity (found by #436).
if (made) this.setLevel(bus, level);
else this.glideLevel(bus, level);
waits.push(this.setFx(bus, fx));
}
const gone = [...this.buses.keys()].filter(
@@ -923,6 +1122,13 @@ export class SoundLibrary implements AudioSink {
await Promise.all(waits);
}
/** The level of a bus at `gain` from now on, without a glide. */
private setLevel(bus: Bus, gain: number): void {
const to = Math.max(0, gain);
bus.gain.gain.value = to;
bus.set = { ...bus.set, gain: to };
}
/** A quick glide of the level of a bus to `gain`. */
private glideLevel(bus: Bus, gain: number, time = 0, ease: EaseShare = 'linear'): void {
const to = Math.max(0, gain);
@@ -1122,6 +1328,7 @@ export class SoundLibrary implements AudioSink {
private dropBus(name: string, bus: Bus): void {
this.dropMeter(bus);
this.dropMeter(bus, 'watch');
this.dropSends(bus);
this.unchain(bus);
bus.input.disconnect();
@@ -1160,6 +1367,9 @@ export class SoundLibrary implements AudioSink {
ramp: (ramp) => {
if (tracks.has(ramp.bus)) this.rampBus(groupBus(id, ramp.bus), ramp);
},
automate: (automation) => {
if (tracks.has(automation.bus)) this.automateBus(groupBus(id, automation.bus), automation);
},
};
this.sequences.start(id, name, routed, owner);
return id;
@@ -1553,9 +1763,13 @@ export class SoundLibrary implements AudioSink {
const askedAt = this.cancelsFor(start.file, start.owner);
const waiting: Pending = {
id,
file: start.file,
owner: start.owner,
...(start.note !== undefined && { note: start.note }),
bus: start.bus ?? '',
fx: { ...start.fx, params: { ...start.fx.params } },
timed: [],
lanes: [],
stopped: false,
released: false,
};
@@ -1576,6 +1790,9 @@ export class SoundLibrary implements AudioSink {
if (stream?.failed) delete started.stream;
this.start(started);
for (const { params, time, ease } of waiting.timed) this.tune(id, params, time, ease);
const voice = this.voices.find((other) => other.id === id);
if (voice)
for (const { param, automation } of waiting.lanes) this.followVoice(voice, param, automation);
if (waiting.released) this.release(id);
});
return id;
@@ -1726,6 +1943,7 @@ export class SoundLibrary implements AudioSink {
* wakes the clock – a pattern or song reading it every few milliseconds would undo the pause.
*/
setSuspended(suspended: boolean): void {
if (suspended && !this.held) this.heldFrom = this.clockState();
this.held = suspended;
const context = this.live();
if (!context) return;
@@ -2200,6 +2418,7 @@ export class SoundLibrary implements AudioSink {
clip: asked,
bus = '',
stream,
note,
}: Start): void {
const context = this.audio();
// A context starts suspended where the page has not been interacted with; without this the
@@ -2267,6 +2486,7 @@ export class SoundLibrary implements AudioSink {
file,
owner,
bus,
...(note !== undefined && { note }),
at: when,
sources,
ending: new Set(),
@@ -2924,6 +3144,8 @@ interface Bus {
gain: GainNode;
pan: StereoPannerNode;
set: { gain: number; pan: number };
/** What a lane of automation sets instead (#337), where `setBus` leaves it alone. */
automated: Set<AutomationTarget>;
/** Where it goes as a whole while it has no sends; null straight out. */
into: string | null;
sends: BusSendNodes[] | null;
@@ -2934,6 +3156,8 @@ interface Bus {
leaving: number;
/** Its level meter (#359), while one is asked after. */
meter: BusMeter | null;
/** What the agent hears of it (#436), while the agent is switched on. */
watch: BusMeter | null;
}
/** A tap after the pan of a bus: an analyser for each side, so a peak panned hard is not halved. */
@@ -3061,6 +3285,64 @@ function readMeter(meter: BusMeter): [number, number] {
return [left!, right!];
}
/**
* The loudest sample and the RMS of each side over the stretch a meter looks back over (#436), 1 at
* full scale.
*/
function readLevels(meter: BusMeter): { peak: [number, number]; rms: [number, number] } {
const [left, right] = meter.sides.map((side) => {
side.getFloatTimeDomainData(meter.samples);
let peak = 0;
let squares = 0;
for (const sample of meter.samples) {
peak = Math.max(peak, Math.abs(sample));
squares += sample * sample;
}
return { peak, rms: Math.sqrt(squares / meter.samples.length) };
});
return { peak: [left!.peak, right!.peak], rms: [left!.rms, right!.rms] };
}
/** One voice as `heard` gives it (#436): a sound by its file, a note by `instrument:<id>` and its note. */
export interface HeardVoice {
id: number;
file: string;
/** The behavior it belongs to, `''` for none. */
owner: string;
/** Its bus, `''` the master. */
bus: string;
note?: number;
}
/** What `SoundLibrary.heard` gives (#436). */
export interface HeardSound {
/** Seconds the levels look back over; 0 before there is any sound. */
window: number;
/**
* The audio clock: `running`, `suspended` or `none` before there is any sound – while a pause holds
* it, as it was when the pause took hold –, the second it is at, and whether a pause holds it.
* Suspended, nothing is heard.
*/
clock: { state: string; time: number; held: boolean };
/** Voices started, a planned one `startsIn` seconds from now. */
voices: (HeardVoice & { startsIn: number; released: boolean; ringing: boolean })[];
/** Voices waiting on their bytes. */
waiting: HeardVoice[];
/** Files that would not load. */
failed: string[];
/** Patterns, songs and compositions playing, by `pattern:<id>`, `song:<id>`, `composition:<id>`. */
playings: {
id: number;
name: string;
owner: string;
playing: boolean;
fading: number | null;
baked: boolean;
}[];
/** The tapped buses by their name in the sink: `mix`, `mix:<id>`, `play:<n>`, `play:<n>:<track>`. */
buses: { name: string; peak: [number, number]; rms: [number, number] }[];
}
export const soundLibrary = new SoundLibrary();
/** A composition fits an audio item to the length of its sound once the bytes are here (#418). */
+46 -8
View File
@@ -620,6 +620,18 @@ export function fadeOut(param: AudioParam, now: number, seconds = FADE): void {
param.linearRampToValueAtTime(0, now + seconds);
}
/** Keeps a level where it is at `at` and drops what was planned for it after that. */
export function holdParam(param: AudioParam, at: number): void {
// Firefox has no cancelAndHoldAtTime: there the value it reads out now is held by hand.
const held = param as Omit<AudioParam, 'cancelAndHoldAtTime'> &
Partial<Pick<AudioParam, 'cancelAndHoldAtTime'>>;
if (held.cancelAndHoldAtTime) held.cancelAndHoldAtTime(at);
else {
param.cancelScheduledValues(at);
param.setValueAtTime(param.value, at);
}
}
/**
* Sends one knob from where it is right now to `to`. Whatever was still planned for it is dropped
* first, so a change never queues behind an older one. A timed change runs along `ease` – a `ratio`
@@ -635,14 +647,7 @@ export function move(
ease: EaseShare = 'linear',
): void {
const from = param.value;
// Firefox has no cancelAndHoldAtTime: there the value it reads out now is held by hand.
const held = param as Omit<AudioParam, 'cancelAndHoldAtTime'> &
Partial<Pick<AudioParam, 'cancelAndHoldAtTime'>>;
if (held.cancelAndHoldAtTime) held.cancelAndHoldAtTime(now);
else {
param.cancelScheduledValues(now);
param.setValueAtTime(from, now);
}
holdParam(param, now);
const logarithmic = ratio && from > 0 && to > 0;
if (time <= 0) param.setTargetAtTime(to, now, GLIDE);
else if (ease === 'step') param.setValueAtTime(to, now + time);
@@ -670,6 +675,39 @@ export function glideKnobs(
return moved;
}
/**
* Takes knobs along the marks of a parameter's lane (#337): `levels` are the knobs' levels at each
* mark, in the order of `marks`. A knob that stays where it is all the way is left alone. Each other
* one is ramped to its level at each mark, or jumped there; its first mark is a jump where nothing
* has been planned for it by a lane yet – `followed` says which have –, or where the playing has
* just started (`fresh`), which drops what was planned for it from there on.
*/
export function followMarks(
knobs: readonly Knob[],
levels: readonly (readonly number[])[],
marks: readonly { at: number; jump: boolean }[],
now: number,
fresh: boolean,
followed: WeakSet<Knob>,
): void {
knobs.forEach((knob, i) => {
const param = knob.param;
const way = levels.map((level) => level[i]);
if (!param || way.every((level) => level === undefined || level === knob.target)) return;
const restart = fresh || !followed.has(knob);
followed.add(knob);
marks.forEach((mark, m) => {
const level = way[m];
if (level === undefined) return;
const at = Math.max(now, mark.at);
if (m === 0 && fresh) holdParam(param, at);
if (mark.jump || (m === 0 && restart)) param.setValueAtTime(level, at);
else param.linearRampToValueAtTime(level, at);
});
knob.target = way.at(-1) ?? knob.target;
});
}
/** Samples per second of an eased curve: a bounce keeps its corners, a long sweep stays small. */
const CURVE_RATE = 200;
+1
View File
@@ -83,6 +83,7 @@ export class Music {
),
play: (note) => this.playOne(note),
playClip: (clip) => this.playClip(clip),
automate: (automation) => soundLibrary.automateBus(automation.bus, automation),
playing: (voice) => soundLibrary.playing(voice),
stop: (voices) => {
for (const voice of voices) soundLibrary.stopVoice(voice);
+310
View File
@@ -0,0 +1,310 @@
/**
* Automation of the parameters of a song's track (#337, stage 3): a lane `param:<name>` with a range
* of its own, the value each note and clip starts with, the sound FX of the track and its items set
* where the lane starts, and which parameters a track has to offer.
*/
import { describe, expect, it } from 'vitest';
import { CreateAssetCommand, SetAssetPropertyCommand } from '../../src/core/commands/assetCommands';
import { SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
import { SetSongCommand } from '../../src/core/commands/songCommands';
import type { BehaviorAsset, InstrumentAsset, SongAsset, SoundFxAsset } from '../../src/core/model/Asset';
import { findAssetOf } from '../../src/core/model/Project';
import { chipPresetGraph } from '../../src/core/instrument/chipPresets';
import {
type AutomationLane,
heldLanes,
laneMarks,
laneRange,
paramsAt,
reaches,
type SequenceLane,
} from '../../src/core/music/automation';
import {
type ScheduledClip,
type ScheduledNote,
Scheduler,
type SchedulerHost,
} from '../../src/core/music/Scheduler';
import { newItem, songSequence, trackBus } from '../../src/core/music/song';
import { songBuses } from '../../src/core/music/songBuses';
import { defaultParamRange, trackParams } from '../../src/core/music/trackParams';
import { strikeNote } from '../../src/runtime/systems/behaviorContext';
import { type AudioSink, SILENT_AUDIO } from '../../src/runtime/systems/Audio';
import { graphEditor, makeProject, type ProjectFixture } from '../helpers';
const Q = 480;
const BAR = 4 * Q;
describe('a lane of a parameter (#337)', () => {
it('is held to a name without a colon, its points to its own range, and kept without a point', () => {
expect(
heldLanes([
{
target: 'param:cut',
min: 100,
max: 5000,
points: [
{ tick: 0, value: 20000 },
{ tick: 10, value: 50 },
],
},
{ target: 'param:', points: [{ tick: 0, value: 1 }] },
{ target: 'param:a:b', points: [{ tick: 0, value: 1 }] },
{ target: 'param:mix', min: 1, max: 1, points: [{ tick: 0, value: 3 }] },
{ target: 'param:empty', min: -1, max: 1, points: [] },
{ target: 'param:bare', points: [] },
]),
).toEqual([
{
target: 'param:cut',
min: 100,
max: 5000,
points: [
{ tick: 0, value: 5000 },
{ tick: 10, value: 100 },
],
},
// A range that is none is the one a lane without a range has: 0 to 1.
{ target: 'param:mix', points: [{ tick: 0, value: 1 }] },
{ target: 'param:empty', min: -1, max: 1, points: [] },
{ target: 'param:bare', points: [] },
]);
expect(laneRange({ target: 'param:mix' })).toEqual({ min: 0, max: 1 });
expect(laneRange('volume')).toEqual({ min: -60, max: 12 });
});
it('reaches its track and every bus inside it, and starts what is heard there at its value', () => {
expect(reaches('song:s:t', 'song:s:t')).toBe(true);
expect(reaches('song:s:t', 'song:s:t:#items')).toBe(true);
expect(reaches('song:s:t', 'song:s:t2:#items')).toBe(false);
const lanes: SequenceLane[] = [
{
bus: 'song:s:t',
target: 'param:cut',
min: 0,
max: 10,
points: [
{ tick: 0, value: 0 },
{ tick: 100, value: 20 },
],
},
{ bus: 'song:s:t', target: 'volume', points: [{ tick: 0, value: -6 }] },
{ bus: 'song:s:u', target: 'param:other', points: [{ tick: 0, value: 1 }] },
];
expect(paramsAt(lanes, 'song:s:t:#items', 25)).toEqual({ cut: 5 });
// Past its range the way is held to it.
expect(paramsAt(lanes, 'song:s:t:item', 90)).toEqual({ cut: 10 });
expect(paramsAt(lanes, 'song:s:x:#items', 25)).toBeUndefined();
expect(paramsAt(lanes, undefined, 25)).toBeUndefined();
});
it('is read every step even where it runs straight, as a knob may not be the parameter itself', () => {
const lane: SequenceLane = {
bus: 'b',
target: 'param:cut',
min: 0,
max: 1000,
points: [
{ tick: 0, value: 0 },
{ tick: 960, value: 1000 },
],
};
const marks = laneMarks(lane, 0, null, 0, 480, true, 9.6);
expect(marks).toHaveLength(51);
// In the units of the parameter, not as a gain.
for (const mark of marks) expect(mark.value).toBeCloseTo((1000 * mark.at) / 960, 9);
const held = laneMarks({ ...lane, points: [{ tick: 0, value: 300 }] }, 0, null, 0, 480, true, 9.6);
expect(held).toEqual([
{ at: 0, value: 300, jump: true },
{ at: 480, value: 300, jump: false },
]);
});
});
describe('the scheduler and a parameter lane (#337)', () => {
it('starts each note and clip of the track at the value the lane has where it starts', () => {
const notes: ScheduledNote[] = [];
const clips: ScheduledClip[] = [];
let time = 0;
const host: SchedulerHost = {
now: () => time,
latency: () => 0,
play: (note) => (notes.push(note), 1),
playClip: (clip) => (clips.push(clip), 2),
playing: () => true,
stop: () => {},
wake: () => () => {},
};
const lane: SequenceLane = {
bus: 'song:s:t',
target: 'param:cut',
min: 0,
max: 1000,
points: [
{ tick: 0, value: 0 },
{ tick: 1000, value: 1000 },
],
};
const scheduler = new Scheduler(
host,
() => ({
notes: [
{ tick: 200, length: 10, note: 60, velocity: 1, instrument: 'i', bus: 'song:s:t:#items' },
{ tick: 210, length: 10, note: 60, velocity: 1, instrument: 'i', bus: 'song:s:u:#items' },
],
clips: [{ tick: 100, length: 500, sound: 's', fx: '', offset: 0, volume: 1, bus: 'song:s:t:#items' }],
lanes: [lane],
}),
{ tempo: 120 },
);
scheduler.start(300);
time = 0.5;
scheduler.pump();
expect(notes).toEqual([]);
// Started inside the clip: it is taken up 200 ticks in, at the value there.
expect(clips.map((clip) => clip.params)).toEqual([{ cut: 300 }]);
scheduler.start(150);
time = 1;
scheduler.pump();
expect(notes.map((note) => [note.bus, note.params])).toEqual([
['song:s:t:#items', { cut: 200 }],
['song:s:u:#items', undefined],
]);
});
});
/** A sound FX "Filter" with a parameter `cut` (default 800) into its gain. */
function makeFilter(f: ProjectFixture): SoundFxAsset {
const command = new CreateAssetCommand('soundfx', 'Filter');
f.stack.execute(command);
const fx = command.asset as SoundFxAsset;
const edit = graphEditor(f, fx as unknown as BehaviorAsset);
const input = edit.addNode('SoundFxInput');
const gain = edit.addNode('SoundFxGain');
const output = edit.addNode('SoundFxOutput');
input.ports = [{ key: 'cut', kind: 'number', default: 800 }];
edit.connect(input, 'out', gain, 'in');
edit.connect(input, 'cut', gain, 'gain');
edit.connect(gain, 'out', output, 'in');
return fx;
}
/** A song "Tune" with a pattern of one note on its one track, played by a chip instrument with a parameter `tone`. */
function makeTune() {
const f = makeProject();
const song = new CreateAssetCommand('song', 'Tune');
const pattern = new CreateAssetCommand('pattern', 'Riff');
const chip = new CreateAssetCommand('instrument', 'Chip');
for (const command of [song, pattern, chip]) f.stack.execute(command);
const instrument = chip.asset as InstrumentAsset;
instrument.graph = chipPresetGraph('pickup', () => 0);
instrument.graph.nodes.push({
id: 'params',
type: 'InstrumentParameters',
x: 0,
y: 0,
params: {},
ports: [
{ key: 'tone', kind: 'number', default: 0.25 },
{ key: 'cut', kind: 'number', default: 5 },
],
});
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'bars', 1));
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'instrument', instrument.id));
f.stack.execute(
new SetPatternNotesCommand(pattern.asset.id, [{ id: 'n1', tick: 0, length: Q, note: 60, velocity: 1 }]),
);
const tune = () => findAssetOf(f.project, song.asset.id, 'song') as SongAsset;
const trackId = tune().tracks[0]!.id;
f.stack.execute(
new SetSongCommand(song.asset.id, {
items: [newItem({ track: trackId, kind: 'pattern', asset: pattern.asset.id, tick: 0, length: BAR })],
}),
);
const setTrack = (fields: Partial<SongAsset['tracks'][number]>) =>
f.stack.execute(new SetSongCommand(song.asset.id, { tracks: [{ ...tune().tracks[0]!, ...fields }] }));
return { ...f, tune, trackId, setTrack, instrument, patternId: pattern.asset.id };
}
describe('a song with a parameter lane (#337)', () => {
const cutLane: AutomationLane = {
target: 'param:cut',
min: 0,
max: 2000,
points: [
{ tick: 0, value: 400 },
{ tick: BAR, value: 1600 },
],
};
it('plays it on its track, and sets the sound FX of the track where it starts', () => {
const f = makeTune();
const fx = makeFilter(f);
f.setTrack({ fx: fx.id, automation: [cutLane] });
expect(songSequence(f.project, f.tune()).lanes).toEqual([
{ bus: trackBus(f.tune().id, f.trackId), ...cutLane },
]);
const [track] = songBuses(f.project, f.tune());
expect(track).toMatchObject({ automated: ['param:cut'], fx: { params: { cut: 400 } } });
// Without a point it rules nothing, and is not played.
f.setTrack({ automation: [{ ...cutLane, points: [] }] });
expect(songSequence(f.project, f.tune()).lanes).toBeUndefined();
expect(songBuses(f.project, f.tune())[0]).not.toHaveProperty('automated');
expect(songBuses(f.project, f.tune())[0]!.fx!.params).toEqual({});
});
it('sets the sound FX of a pattern item on the track the same way, which leaves it to the lane', () => {
const f = makeTune();
const fx = makeFilter(f);
const item = f.tune().items[0]!;
f.stack.execute(new SetSongCommand(f.tune().id, { items: [{ ...item, fx: fx.id }] }));
f.setTrack({ automation: [cutLane] });
const own = songBuses(f.project, f.tune()).find((bus) => bus.name.endsWith(item.id));
expect(own).toMatchObject({ automated: ['param:cut'], fx: { params: { cut: 400 } } });
});
it('offers the parameters of the track, its items and their instruments, each name once', () => {
const f = makeTune();
const fx = makeFilter(f);
expect(trackParams(f.project, f.tune(), f.tune().tracks[0]!)).toEqual([
{ name: 'tone', default: 0.25 },
{ name: 'cut', default: 5 },
]);
f.setTrack({ fx: fx.id });
// The track's own first: its `cut` is the one whose default is named.
expect(trackParams(f.project, f.tune(), f.tune().tracks[0]!)).toEqual([
{ name: 'cut', default: 800 },
{ name: 'tone', default: 0.25 },
]);
});
it('starts a new lane at a range round the default, on a round number', () => {
expect(defaultParamRange(0.25)).toEqual({ min: 0, max: 1 });
expect(defaultParamRange(800)).toEqual({ min: 0, max: 2000 });
expect(defaultParamRange(1.5)).toEqual({ min: 0, max: 5 });
expect(defaultParamRange(-3)).toEqual({ min: -10, max: 10 });
expect(defaultParamRange(Number.NaN)).toEqual({ min: 0, max: 1 });
});
it('hands a note its values for its instrument and its own sound FX alike', () => {
const f = makeTune();
const fx = makeFilter(f);
f.stack.execute(new SetAssetPropertyCommand(f.instrument.id, 'fx', fx.id));
const struck: { params: Record<string, number> }[] = [];
const audio: AudioSink = {
...SILENT_AUDIO,
playNote: (_name, _layers, params) => (struck.push({ params: { ...params } }), 1),
};
const note: ScheduledNote = {
at: 0,
duration: 0.1,
note: 60,
velocity: 1,
instrument: f.instrument.id,
tempo: 120,
params: { cut: 1200 },
};
expect(strikeNote({ project: f.project, audio }, note, '')).toBe(1);
expect(struck[0]!.params).toMatchObject({ cut: 1200, 'fx:cut': 1200 });
});
});
+403
View File
@@ -0,0 +1,403 @@
/**
* Automation of a song (#337): lanes of a track's volume and pan, their value along the song, and
* how the scheduler takes a bus along them on the audio clock – in the Song tab, in a game, in a
* composition and in a bake alike.
*/
import { describe, expect, it } from 'vitest';
import { CreateAssetCommand, SetAssetPropertyCommand } from '../../src/core/commands/assetCommands';
import { SetCompositionCommand } from '../../src/core/commands/compositionCommands';
import { SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
import { SetSongCommand } from '../../src/core/commands/songCommands';
import type { CompositionAsset, SongAsset } from '../../src/core/model/Asset';
import { curveLookup } from '../../src/core/model/curve';
import { findAssetOf } from '../../src/core/model/Project';
import {
type AutomationLane,
heldLanes,
laneMarks,
laneValueAt,
laneValueBefore,
type SequenceLane,
type SequenceLanePoint,
sequenceLaneOf,
unrolledLane,
} from '../../src/core/music/automation';
import { bakePrint } from '../../src/core/music/bake';
import { audioItemBus, compositionPlayings } from '../../src/core/music/compositionAudio';
import { dbToGain } from '../../src/core/music/level';
import {
LEAD,
type ScheduledAutomation,
Scheduler,
type SchedulerHost,
} from '../../src/core/music/Scheduler';
import { heldSongTracks, newItem, songSequence, trackBus } from '../../src/core/music/song';
import { songBuses } from '../../src/core/music/songBuses';
import { makeProject, roundTrip } from '../helpers';
const Q = 480;
const BAR = 4 * Q;
const pan = (points: SequenceLanePoint[]): SequenceLane => ({ bus: 'b', target: 'pan', points });
/** The marks with their ticks and values to a millionth, so a sum of floats reads as what it is. */
const rounded = (marks: { at: number; value: number; jump: boolean }[]) =>
marks.map(({ at, value, jump }) => [Math.round(at * 1e6) / 1e6, Math.round(value * 1e6) / 1e6, jump]);
describe('the lanes of a track (#337)', () => {
it('are held to a target each, points in whole ticks from 0 with values in range, in tick order', () => {
expect(
heldLanes([
{
target: 'pan',
points: [
{ tick: 960.4, value: 3 },
{ tick: 0, value: -0.5, ease: 'sine-in' },
{ tick: -1, value: 0 },
{ tick: 10, value: Number.NaN },
],
},
{ target: 'pan', points: [{ tick: 0, value: 1 }] },
{ target: 'volume', points: [] },
{ target: 'speed', points: [{ tick: 0, value: 1 }] },
'volume',
{
target: 'volume',
points: [
{ tick: 5, value: 0, ease: 'wobbly' },
{ tick: 0, value: -100, ease: 'linear' },
],
},
]),
).toEqual([
{
target: 'pan',
points: [
{ tick: 0, value: -0.5, ease: 'sine-in' },
{ tick: 960, value: 1 },
],
},
{
target: 'volume',
points: [
{ tick: 0, value: -60 },
{ tick: 5, value: 0 },
],
},
]);
expect(heldLanes('x')).toEqual([]);
});
it('are kept with their track when the project is saved, and a track without one has none', () => {
const f = makeProject();
const create = new CreateAssetCommand('song', 'Tune');
f.stack.execute(create);
const song = () => findAssetOf(f.project, create.asset.id, 'song') as SongAsset;
const lane: AutomationLane = { target: 'volume', points: [{ tick: 0, value: -6, ease: 'expo-out' }] };
const [first] = song().tracks;
f.stack.execute(
new SetSongCommand(song().id, {
tracks: [
{ ...first!, automation: [lane] },
{ ...first!, id: 'plain', automation: [] },
],
}),
);
const saved = roundTrip(f.project).assets.find((a) => a.id === song().id) as SongAsset;
expect(saved.tracks.map((t) => t.automation)).toEqual([[lane], undefined]);
expect(Object.keys(heldSongTracks([{ id: 'x' }])[0]!)).not.toContain('automation');
});
it('runs from each point to the next along its easing, holds outside them, and jumps where two meet', () => {
const points: SequenceLanePoint[] = [
{ tick: 0, value: 0 },
{ tick: 100, value: 1, ease: 'step' },
{ tick: 200, value: 0, ease: 'sine-in' },
{ tick: 300, value: 1 },
{ tick: 300, value: -1 },
];
const at = (tick: number) => laneValueAt(points, tick);
const before = (tick: number) => laneValueBefore(points, tick);
expect([at(-5), at(50), at(100), before(100)]).toEqual([0, 0.5, 1, 1]);
// A step holds up to the next point, which it jumps to.
expect([at(150), before(200), at(200)]).toEqual([1, 1, 0]);
expect(at(250)).toBeCloseTo(1 - Math.cos(Math.PI / 4), 12);
expect(before(300)).toBeCloseTo(1, 12);
expect([at(300), at(400)]).toEqual([-1, -1]);
});
it('eases along a Curve asset it names, and linear once that is gone', () => {
const f = makeProject();
f.project.assets.push({
id: 'cv',
type: 'curve',
name: 'Knee',
props: {},
points: [
{ x: 0, y: 0 },
{ x: 0.5, y: 0.9 },
{ x: 1, y: 1 },
],
});
const lane: AutomationLane = {
target: 'pan',
points: [
{ tick: 0, value: 0, ease: 'curve:cv' },
{ tick: 100, value: 1 },
],
};
const curves = curveLookup(f.project);
expect(laneValueAt(sequenceLaneOf(lane, 'b', curves).points, 25)).toBeCloseTo(0.45, 9);
f.project.assets.pop();
expect(laneValueAt(sequenceLaneOf(lane, 'b', curveLookup(f.project)).points, 25)).toBe(0.25);
});
});
describe('a lane over a window of the scheduler (#337)', () => {
const sweep = pan([
{ tick: 0, value: -1 },
{ tick: 960, value: 1 },
]);
it('is one ramp where the bus can ramp along it: a straight pan', () => {
expect(rounded(laneMarks(sweep, 0, null, 0, 480, true, 9.6))).toEqual([
[0, -1, true],
[480, 0, false],
]);
// Going on, it starts where the last window ended, as a ramp.
expect(rounded(laneMarks(sweep, 0, null, 480, 600, false, 9.6))).toEqual([
[480, 0, false],
[600, 0.25, false],
]);
});
it('is read every step where the bus cannot: decibels in a gain, or an easing', () => {
const fade: SequenceLane = {
bus: 'b',
target: 'volume',
points: [
{ tick: 0, value: -60 },
{ tick: 960, value: 0 },
],
};
const marks = laneMarks(fade, 0, null, 0, 480, true, 9.6);
// 0 and 480, and the 49 readings between.
expect(marks).toHaveLength(51);
expect(marks[0]).toEqual({ at: 0, value: 0, jump: true });
for (const mark of marks) expect(mark.value).toBeCloseTo(dbToGain(-60 + (60 * mark.at) / 960), 12);
expect(marks.slice(1).every((mark, i) => !mark.jump && mark.at > marks[i]!.at)).toBe(true);
const eased = pan([
{ tick: 0, value: 0, ease: 'sine-in' },
{ tick: 100, value: 1 },
]);
expect(laneMarks(eased, 0, null, 0, 100, true, 25).map((mark) => mark.at)).toEqual([0, 25, 50, 75, 100]);
});
it('jumps back with the playhead at the end of a loop, and where a step lands', () => {
expect(rounded(laneMarks(sweep, 0, { start: 0, end: 960 }, 480, 1440, false, 9.6))).toEqual([
[480, 0, false],
[960, 1, false],
[960, -1, true],
[1440, 0, false],
]);
const step = pan([
{ tick: 0, value: -1, ease: 'step' },
{ tick: 100, value: 1 },
]);
expect(rounded(laneMarks(step, 0, null, 0, 200, true, 9.6))).toEqual([
[0, -1, true],
[100, 1, true],
[200, 1, false],
]);
});
});
describe('a lane round the loop of a song inside a composition (#337)', () => {
it('is cut at the loop end and starts again there, each round on from where it was', () => {
const points: SequenceLanePoint[] = [
{ tick: 0, value: -1 },
{ tick: 960, value: 1 },
];
const unrolled = unrolledLane(points, { start: 0, end: 960 }, 480, 1440, 9.6);
expect(unrolled).toEqual([
{ tick: 480, value: 0 },
{ tick: 960, value: 1 },
{ tick: 960, value: -1 },
{ tick: 1440, value: 0 },
]);
expect(laneValueAt(unrolled, 700)).toBeCloseTo(laneValueAt(points, 700), 12);
expect(laneValueAt(unrolled, 1200)).toBeCloseTo(laneValueAt(points, 240), 12);
});
it('keeps a whole easing as it is, and reads one cut in the middle every step', () => {
const eased: SequenceLanePoint[] = [
{ tick: 0, value: 0, ease: 'sine-in' },
{ tick: 960, value: 1 },
];
expect(unrolledLane(eased, null, 0, 1920, 96)).toEqual([
{ tick: 0, value: 0, ease: 'sine-in' },
{ tick: 960, value: 1 },
{ tick: 1920, value: 1 },
]);
const cut = unrolledLane(eased, null, 480, 960, 96);
expect(cut.map((p) => p.tick)).toEqual([480, 576, 672, 768, 864, 960]);
for (const point of cut) {
expect(point.ease).toBeUndefined();
expect(point.value).toBeCloseTo(laneValueAt(eased, point.tick), 12);
}
});
});
/** A song "Tune" with a one-bar pattern of one note on its track, and a composition "Cut" at 10 fps. */
function makeTune() {
const f = makeProject();
const song = new CreateAssetCommand('song', 'Tune');
const pattern = new CreateAssetCommand('pattern', 'Riff');
const piano = new CreateAssetCommand('instrument', 'Piano');
const comp = new CreateAssetCommand('composition', 'Cut', { fps: 10 });
for (const command of [song, pattern, piano, comp]) f.stack.execute(command);
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'bars', 1));
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'instrument', piano.asset.id));
f.stack.execute(
new SetPatternNotesCommand(pattern.asset.id, [{ id: 'n1', tick: 0, length: Q, note: 60, velocity: 1 }]),
);
const tune = () => findAssetOf(f.project, song.asset.id, 'song') as SongAsset;
const trackId = tune().tracks[0]!.id;
f.stack.execute(
new SetSongCommand(song.asset.id, {
items: [newItem({ track: trackId, kind: 'pattern', asset: pattern.asset.id, tick: 0, length: BAR })],
}),
);
const automate = (...automation: AutomationLane[]) =>
f.stack.execute(new SetSongCommand(song.asset.id, { tracks: [{ ...tune().tracks[0]!, automation }] }));
const cut = () => findAssetOf(f.project, comp.asset.id, 'composition') as CompositionAsset;
return { ...f, tune, trackId, automate, cut };
}
describe('a song with automation (#337)', () => {
it('plays its lanes on the buses of its tracks, which start where the lanes do', () => {
const f = makeTune();
expect(songSequence(f.project, f.tune()).lanes).toBeUndefined();
expect(songBuses(f.project, f.tune())[0]).not.toHaveProperty('automated');
f.automate({
target: 'volume',
points: [
{ tick: 0, value: -12 },
{ tick: BAR, value: 0 },
],
});
expect(songSequence(f.project, f.tune()).lanes).toEqual([
{
bus: trackBus(f.tune().id, f.trackId),
target: 'volume',
points: [
{ tick: 0, value: -12 },
{ tick: BAR, value: 0 },
],
},
]);
// The lane rules the volume, the fader the pan.
expect(songBuses(f.project, f.tune())[0]).toMatchObject({
gain: dbToGain(-12),
pan: 0,
automated: ['volume'],
});
});
it('is a bake of its own, and one without automation keeps the print it had', () => {
const f = makeTune();
const plain = bakePrint(f.project, f.tune());
const sweep = (to: number): AutomationLane => ({
target: 'pan',
points: [
{ tick: 0, value: 0.5 },
{ tick: BAR / 2, value: to },
],
});
f.automate(sweep(1));
const panned = bakePrint(f.project, f.tune());
expect(panned).not.toBe(plain);
// Only later in the song: where its tracks start is the same.
f.automate(sweep(-1));
expect(bakePrint(f.project, f.tune())).not.toBe(panned);
f.automate();
expect(bakePrint(f.project, f.tune())).toBe(plain);
});
it('plays its lanes in a composition on the buses of its item, round its loop, in milliseconds', () => {
const f = makeTune();
f.automate({
target: 'pan',
points: [
{ tick: 0, value: -1 },
{ tick: BAR, value: 1 },
],
});
// From 1 s for 3 s, half a second into the song: it goes round its two-second bar once.
const item = {
id: 'it',
track: f.cut().tracks[0]!.id,
start: 10,
length: 30,
offset: 0.5,
audio: { kind: 'song' as const, asset: f.tune().id, volume: 0, bus: '' },
};
f.stack.execute(new SetCompositionCommand(f.cut().id, { items: [item] }));
const [playing] = compositionPlayings(f.project, f.cut());
expect(playing!.sequence.lanes).toEqual([
{
bus: `${audioItemBus(f.cut().id, 'it')}:${f.trackId}`,
target: 'pan',
points: [
{ tick: 1000, value: -0.5 },
{ tick: 2500, value: 1 },
{ tick: 2500, value: -1 },
{ tick: 4000, value: 0.5 },
],
},
]);
expect(playing!.buses.map((bus) => bus.name)).toContain(`${audioItemBus(f.cut().id, 'it')}:${f.trackId}`);
});
});
describe('the scheduler and a lane (#337)', () => {
it('hands each window to the host on the audio clock, the first fresh, each going on from the last', () => {
const automated: ScheduledAutomation[] = [];
let time = 10;
let pump: (() => void) | null = null;
const host: SchedulerHost = {
now: () => time,
latency: () => 0,
play: () => 0,
automate: (automation) => void automated.push(automation),
playing: () => false,
stop: () => {},
wake: (wake) => ((pump = wake), () => (pump = null)),
};
const lane = {
...pan([
{ tick: 0, value: -1 },
{ tick: BAR, value: 1 },
]),
bus: 'song:s:t',
};
const scheduler = new Scheduler(host, () => ({ notes: [], lanes: [lane] }), { tempo: 120 });
scheduler.start();
for (let i = 0; i < 8; i++) {
time += 0.025;
pump!();
}
expect(automated.map((a) => [a.bus, a.target, a.fresh])).toEqual([
['song:s:t', 'pan', true],
...Array.from({ length: automated.length - 1 }, () => ['song:s:t', 'pan', false]),
]);
expect(automated[0]!.marks[0]).toEqual({ at: 10 + LEAD, value: -1, jump: true });
automated.slice(1).forEach((window, i) => {
expect(window.marks[0]!.at).toBeCloseTo(automated[i]!.marks.at(-1)!.at, 12);
expect(window.marks[0]!.jump).toBe(false);
});
// Two seconds a bar: from -1 by 1 a second.
for (const { marks } of automated)
for (const mark of marks) expect(mark.value).toBeCloseTo(-1 + (mark.at - 10 - LEAD), 9);
});
});
+9 -1
View File
@@ -236,7 +236,15 @@ describe('Play pattern', () => {
expect([asked?.owner, asked?.play.loop]).toEqual(['', null]);
asked!.play.strike({ at: 3, duration: 0.25, note: 60, velocity: 0.5, instrument: f.lead.id, tempo: 90 });
expect(audio.notes).toEqual([
{ name: `instrument:${f.lead.id}`, volume: 0.25, owner: '', duration: 0.25, at: 3, tempo: 90 },
{
name: `instrument:${f.lead.id}`,
volume: 0.25,
owner: '',
duration: 0.25,
at: 3,
tempo: 90,
note: 60,
},
]);
});
+103
View File
@@ -0,0 +1,103 @@
import { describe, expect, it } from 'vitest';
import { CreateAssetCommand } from '../../src/core/commands/assetCommands';
import { SetSongMarkersCommand } from '../../src/core/commands/songCommands';
import type { SongAsset } from '../../src/core/model/Asset';
import {
changedMarker,
heldSongMarkers,
newMarker,
nextMarkerName,
withoutMarker,
} from '../../src/core/music/markers';
import { makeProject, roundTrip } from '../helpers';
function makeSong() {
const f = makeProject();
const create = new CreateAssetCommand('song', 'Tune');
f.stack.execute(create);
const song = () => f.project.assets.find((a) => a.id === create.asset.id) as SongAsset;
return { ...f, song };
}
describe('song markers (#338)', () => {
it('holds markers to whole ticks from 0, a name and ids of their own, in the order of their ticks', () => {
const held = heldSongMarkers([
{ id: 'b', name: ' Boss ', tick: 3840.4 },
{ id: 'a', name: 'Intro', tick: 0 },
{ id: 'a', name: '', tick: 1920 },
{ id: 'c', name: 'Before', tick: -1 },
{ id: 'd', name: 'Nowhere' },
'no marker',
null,
]);
expect(held.map((m) => [m.name, m.tick])).toEqual([
['Intro', 0],
['Marker 3', 1920],
['Boss', 3840],
]);
expect(new Set(held.map((m) => m.id)).size).toBe(3);
expect(held.find((m) => m.tick === 1920)!.id).not.toBe('a');
expect(heldSongMarkers(undefined)).toEqual([]);
});
it('keeps markers on one tick in the order they came', () => {
const held = heldSongMarkers([
{ id: 'x', name: 'Second', tick: 960 },
{ id: 'y', name: 'First', tick: 0 },
{ id: 'z', name: 'Third', tick: 960 },
]);
expect(held.map((m) => m.name)).toEqual(['First', 'Second', 'Third']);
});
it('names a new marker by the first number not taken', () => {
expect(nextMarkerName([])).toBe('Marker 1');
const markers = heldSongMarkers([
{ name: 'Marker 1', tick: 0 },
{ name: 'Marker 3', tick: 10 },
]);
expect(nextMarkerName(markers)).toBe('Marker 2');
const made = newMarker(markers, 479.6);
expect(made).toMatchObject({ name: 'Marker 2', tick: 480 });
expect(newMarker(markers, -5, ' Drop ')).toMatchObject({ name: 'Drop', tick: 0 });
});
it('changes or takes away one marker and leaves the others', () => {
const markers = heldSongMarkers([
{ id: 'a', name: 'A', tick: 0 },
{ id: 'b', name: 'B', tick: 960 },
]);
expect(changedMarker(markers, 'b', { tick: 1920 })).toEqual([markers[0], { ...markers[1], tick: 1920 }]);
expect(changedMarker(markers, 'a', { name: 'Start' })[0]).toEqual({ id: 'a', name: 'Start', tick: 0 });
expect(withoutMarker(markers, 'a')).toEqual([markers[1]]);
});
it('writes the markers as one step, leaves the key out without one and takes it back on undo', () => {
const f = makeSong();
expect(f.song().markers).toBeUndefined();
const one = [{ id: 'm', name: 'Drop', tick: 1920.2 }];
expect(SetSongMarkersCommand.changes(f.song(), one)).toBe(true);
f.stack.execute(new SetSongMarkersCommand(f.song().id, one, 'Add marker'));
expect(f.song().markers).toEqual([{ id: 'm', name: 'Drop', tick: 1920 }]);
expect(SetSongMarkersCommand.changes(f.song(), one)).toBe(false);
f.stack.execute(new SetSongMarkersCommand(f.song().id, [], 'Delete marker'));
expect('markers' in f.song()).toBe(false);
f.stack.undo();
expect(f.song().markers).toEqual([{ id: 'm', name: 'Drop', tick: 1920 }]);
f.stack.undo();
expect('markers' in f.song()).toBe(false);
});
it('keeps the markers through saving and loading', () => {
const f = makeSong();
const markers = [
{ id: 'a', name: 'Intro', tick: 0 },
{ id: 'b', name: 'Boss', tick: 7680 },
];
f.stack.execute(new SetSongMarkersCommand(f.song().id, markers));
const back = roundTrip(f.project).assets.find((a) => a.id === f.song().id) as SongAsset;
expect(back.markers).toEqual(markers);
const none = makeSong();
const plain = roundTrip(none.project).assets.find((a) => a.id === none.song().id) as SongAsset;
expect('markers' in plain).toBe(false);
});
});
+1 -1
View File
@@ -341,7 +341,7 @@ describe('Play song', () => {
tempo: 90,
});
expect(audio.notes).toEqual([
{ name: `instrument:${f.lead.id}`, volume: 1, owner: '', at: 3, tempo: 90, bus },
{ name: `instrument:${f.lead.id}`, volume: 1, owner: '', at: 3, tempo: 90, bus, note: 60 },
]);
expect(audio.clips).toEqual([
{ file: FILE, clip: { at: 4, offset: 0.5, duration: 2, volume: 0.5, bus } },
+14 -1
View File
@@ -1,6 +1,6 @@
/** The global clock: one scaled step a frame, which is what the world, the particles and the shaders share. */
import { beforeEach, describe, expect, it } from 'vitest';
import { effectiveScale, GameClock, MAX_FRAME, timeStep } from '../../src/core/time';
import { effectiveScale, frameWithin, GameClock, MAX_FRAME, timeStep } from '../../src/core/time';
describe('effectiveScale', () => {
it('is the scale itself while running', () => {
@@ -38,6 +38,19 @@ describe('timeStep', () => {
});
});
describe('frameWithin (#435)', () => {
it('cuts the frame that would go past the time left, at the clock scale', () => {
expect(frameWithin(1 / 60, 1, 0.005)).toBeCloseTo(0.005, 12);
expect(timeStep(frameWithin(1 / 60, 2, 0.005), 2, false)).toBeCloseTo(0.005, 12);
});
it('leaves a frame that stays short of it, and one the clock does not move by', () => {
expect(frameWithin(1 / 60, 1, 1)).toBe(1 / 60);
expect(frameWithin(1 / 60, 0, 0.005)).toBe(1 / 60);
expect(frameWithin(1 / 60, 1, -1)).toBe(0);
});
});
describe('GameClock', () => {
let clock: GameClock;
+172 -2
View File
@@ -1,12 +1,19 @@
import { afterEach, describe, expect, it } from 'vitest';
import { CreateSoundCommand } from '../../src/core/commands/assetCommands';
import type { CompositionAsset, PatternAsset, SongAsset, SoundAsset } from '../../src/core/model/Asset';
import type {
BehaviorAsset,
CompositionAsset,
PatternAsset,
SongAsset,
SoundAsset,
SoundFxAsset,
} from '../../src/core/model/Asset';
import { setSoundMeasure } from '../../src/core/model/composition';
import { ROUTING_ID } from '../../src/core/model/Project';
import { settleRouting } from '../../src/core/music/projectRouting';
import { TO_COMPOSITION_BUS } from '../../src/core/properties/buses';
import { runTool } from '../../src/editor/agent/agentTools';
import { makeSpriteScene, type SceneFixture } from '../helpers';
import { graphEditor, makeSpriteScene, type SceneFixture } from '../helpers';
import { hostFor, text } from './agentTestHost';
/** The commands of #429 stage 5d: patterns, songs, sends, clips of sounds and compositions. */
@@ -217,6 +224,169 @@ describe('songs', () => {
expect(noFx.text).toContain('is no Sound FX asset');
});
it('adds, renames, moves and removes markers of a song (#338), and get_asset lists them', async () => {
const fixture = makeSpriteScene();
const { Theme } = await made(fixture, { Theme: 'song' });
const song = () => byId<SongAsset>(fixture, Theme);
const done = await run(fixture, [
{ command: 'add_song_marker', args: { asset: Theme, tick: 7680, name: 'Boss' }, as: 'boss' },
{ command: 'add_song_marker', args: { asset: Theme, tick: 0 } },
{ command: 'change_song_marker', args: { asset: Theme, marker: '$boss', tick: 3840 } },
]);
expect(done.result.isError).toBeUndefined();
expect(song().markers!.map(({ name, tick }) => [name, tick])).toEqual([
['Marker 1', 0],
['Boss', 3840],
]);
const listed = (await read(fixture, Theme)).markers as { id: string; name: string; tick: number }[];
expect(listed.map((m) => m.name)).toEqual(['Marker 1', 'Boss']);
const boss = song().markers![1]!.id;
const blank = await run(fixture, [
{ command: 'change_song_marker', args: { asset: Theme, marker: boss, name: ' ' } },
]);
expect(blank.text).toContain('A marker needs a name');
const renamed = await run(fixture, [
{ command: 'change_song_marker', args: { asset: Theme, marker: boss, name: 'Fight' } },
]);
expect(renamed.result.isError).toBeUndefined();
expect(song().markers![1]).toEqual({ id: boss, name: 'Fight', tick: 3840 });
const gone = await run(fixture, [
{ command: 'remove_song_marker', args: { asset: Theme, marker: boss } },
]);
expect(gone.result.isError).toBeUndefined();
expect(song().markers!.map((m) => m.name)).toEqual(['Marker 1']);
const missing = await run(fixture, [
{ command: 'remove_song_marker', args: { asset: Theme, marker: boss } },
]);
expect(missing.text).toContain(`Theme has no marker ${boss}`);
});
it('sets the volume lane of a track in tick order with its easings, refuses what is out of range, and clears it with []', async () => {
const fixture = makeSpriteScene();
const { Theme, Bend } = await made(fixture, { Theme: 'song', Bend: 'curve' });
const song = () => byId<SongAsset>(fixture, Theme);
const track = song().tracks[0]!.id;
const set = await run(
fixture,
[
{
command: 'set_song_lane',
args: {
asset: Theme,
track,
target: 'volume',
points: [
{ tick: 1920, value: -24, ease: 'step' },
{ tick: 0, value: -6, ease: 'sine-in' },
{ tick: 1920, value: 0, ease: `curve:${Bend}` },
{ tick: 3840, value: -12, ease: 'linear' },
],
},
},
],
true,
);
expect(set.result.isError).toBeUndefined();
expect(set.shown).toEqual([{ kind: 'asset', asset: Theme, open: true }]);
// In tick order, two on one tick as given (a jump); linear is stored as no easing.
expect(song().tracks[0]!.automation).toEqual([
{
target: 'volume',
points: [
{ tick: 0, value: -6, ease: 'sine-in' },
{ tick: 1920, value: -24, ease: 'step' },
{ tick: 1920, value: 0, ease: `curve:${Bend}` },
{ tick: 3840, value: -12 },
],
},
]);
const lanes = (await read(fixture, Theme)).tracks as { automation?: unknown[] }[];
expect(lanes[0]!.automation).toHaveLength(1);
const point = (value: number, ease?: string) => [{ tick: 0, value, ...(ease && { ease }) }];
const refused = async (target: string, points: object[], extra: object = {}) =>
(
await run(fixture, [
{ command: 'set_song_lane', args: { asset: Theme, track, target, points, ...extra } },
])
).text;
expect(await refused('volume', point(13))).toContain('value at tick 0 has to be from -60 to 12');
expect(await refused('pan', point(-2))).toContain('from -1 to 1');
expect(await refused('volume', point(0, 'wobbly'))).toContain('"wobbly" is no easing');
expect(await refused('volume', point(0, 'curve:nope'))).toContain('nope is no Curve asset');
expect(await refused('loudness', point(0))).toContain('is no lane');
expect(await refused('volume', point(0), { min: 0, max: 1 })).toContain('Only a parameter lane');
expect(song().tracks[0]!.automation![0]!.points).toHaveLength(4);
const nothing = await run(fixture, [
{ command: 'set_song_lane', args: { asset: Theme, track, target: 'pan' } },
]);
expect(nothing.text).toContain('Give the points');
const cleared = await run(fixture, [
{ command: 'set_song_lane', args: { asset: Theme, track, target: 'volume', points: [] } },
]);
expect(cleared.result.isError).toBeUndefined();
expect(song().tracks[0]!.automation).toBeUndefined();
});
it('adds a parameter lane only for a parameter on the track, round its default, and get_asset lists what a lane can move', async () => {
const fixture = makeSpriteScene();
const { Theme, Filter } = await made(fixture, { Theme: 'song', Filter: 'soundfx' });
const fx = byId<SoundFxAsset>(fixture, Filter);
const edit = graphEditor(fixture, fx as unknown as BehaviorAsset);
const input = edit.addNode('SoundFxInput');
const gain = edit.addNode('SoundFxGain');
const output = edit.addNode('SoundFxOutput');
input.ports = [{ key: 'cut', kind: 'number', default: 800 }];
edit.connect(input, 'out', gain, 'in');
edit.connect(input, 'cut', gain, 'gain');
edit.connect(gain, 'out', output, 'in');
const song = () => byId<SongAsset>(fixture, Theme);
const track = song().tracks[0]!.id;
const lane = (args: object) => ({
command: 'set_song_lane',
args: { asset: Theme, track, target: 'param:cut', ...args },
});
const none = await run(fixture, [lane({})]);
expect(none.text).toContain('Nothing on Track 1 has a parameter to move');
expect(((await read(fixture, Theme)).tracks as object[])[0]).not.toHaveProperty('params');
await run(fixture, [{ command: 'change_song_track', args: { asset: Theme, track, fx: Filter } }]);
expect(((await read(fixture, Theme)).tracks as { params?: object }[])[0]!.params).toEqual([
{ name: 'cut', default: 800 },
]);
const other = await run(fixture, [{ ...lane({}), args: { asset: Theme, track, target: 'param:tone' } }]);
expect(other.text).toContain('Nothing on Track 1 has a parameter tone – it has cut');
// Without a range it gets one round the default, and keeps it without a point.
const added = await run(fixture, [lane({})]);
expect(added.result.isError).toBeUndefined();
expect(song().tracks[0]!.automation).toEqual([{ target: 'param:cut', points: [], min: 0, max: 2000 }]);
const pointed = await run(fixture, [lane({ points: [{ tick: 0, value: 1500 }] })]);
expect(pointed.result.isError).toBeUndefined();
const over = await run(fixture, [lane({ points: [{ tick: 0, value: 2500 }] })]);
expect(over.text).toContain('from 0 to 2000');
// A narrower range holds the points it had.
await run(fixture, [lane({ min: 100, max: 1000 })]);
expect(song().tracks[0]!.automation).toEqual([
{ target: 'param:cut', points: [{ tick: 0, value: 1000 }], min: 100, max: 1000 },
]);
expect((await run(fixture, [lane({ min: 5 })])).text).toContain('Give min and max');
expect((await run(fixture, [lane({ min: 5, max: 5 })])).text).toContain('max has to be above min');
// A lane there is stays settable when its parameter is gone from the track.
await run(fixture, [{ command: 'change_song_track', args: { asset: Theme, track, fx: '' } }]);
expect((await run(fixture, [lane({ min: 0, max: 10 })])).result.isError).toBeUndefined();
expect((await run(fixture, [lane({ remove: true, min: 0 })])).text).toContain('remove takes no points');
const removed = await run(fixture, [lane({ remove: true })]);
expect(removed.result.isError).toBeUndefined();
expect(song().tracks[0]!.automation).toBeUndefined();
expect((await run(fixture, [lane({ remove: true })])).text).toContain('Track 1 has no param:cut lane');
});
it('sets the send of a song link and of a master routing link, where a sound only sends post', async () => {
const fixture = makeSpriteScene();
const { Theme } = await made(fixture, { Theme: 'song' });
+214 -4
View File
@@ -4,8 +4,11 @@ import { emptySharedState } from '../../src/core/graph';
import { findScene } from '../../src/core/model/Project';
import { runTool, type ToolHost, type ToolResult } from '../../src/editor/agent/agentTools';
import type { AgentView } from '../../src/editor/agent/agentView';
import { frameWithin } from '../../src/core/time';
import { type PlayDriver, pointOf, sendInput } from '../../src/editor/agent/playInput';
import { cameraHint, NO_CAMERA_HINT } from '../../src/editor/agent/runtimeView';
import type { HeardSound } from '../../src/viewport/SoundLibrary';
import { STALLED_HINT } from '../../src/editor/agent/audioView';
import {
graphEditor,
makeBehavior,
@@ -44,19 +47,41 @@ function game(withCamera = false) {
return { f, box, started, ...driverFor(started) };
}
/** Each frame is a 60th of a second of game time and 16 ms of wall time; `frozen` holds the game. */
/**
* Each frame is a 60th of a second of game time and 16 ms of wall time; `frozen` holds the game.
* A pause asked for at a time cuts the frame that reaches it, as `PlaySession` does.
*/
function driverFor(started: StartedWorld) {
const state = { running: true, paused: false, hidden: false, frozen: false, wall: 0 };
const state = {
running: true,
paused: false,
hidden: false,
frozen: false,
wall: 0,
pausing: null as number | null,
};
const driver: PlayDriver = {
run: () => (state.running ? { world: started.world, input: started.input } : null),
paused: () => state.paused,
hidden: () => state.hidden,
frame: () => {
state.wall += 16;
if (!state.frozen) started.frame(1 / 60);
if (state.frozen || state.paused) return Promise.resolve();
const left = state.pausing === null ? Infinity : state.pausing - started.world.time;
started.frame(frameWithin(1 / 60, 1, left));
if (state.pausing !== null && started.world.time >= state.pausing - 1e-9) {
state.paused = true;
state.pausing = null;
}
return Promise.resolve();
},
now: () => state.wall,
pause: () => {
state.paused = true;
},
pauseAt: (time) => {
state.pausing = time;
},
};
return { driver, state };
}
@@ -171,7 +196,10 @@ describe('play_input', () => {
const result = await run([{ key: 'a', mouse: 'click' }]);
expect(result.isError).toBe(true);
expect(text(result)).toBe('Step 1 has to do one thing: key, mouse, wheel or wait');
expect(text(result)).toBe('Step 1 has to do one thing: key, mouse, wheel, wait or pause');
expect(text(await run([{ pause: true }, { wait: 1 }]))).toBe(
'Step 1: {"pause": true} ends the steps, it comes last',
);
expect(text(await run([{ wait: 10 }, { wait: 10 }, { wait: 10 }, { wait: 1 }]))).toMatch(/at most 30 s/);
state.hidden = true;
@@ -196,6 +224,98 @@ describe('play_input', () => {
});
});
describe('pausing at a game time (#435)', () => {
it('play with until holds the game at that time exactly, not at the frame after', async () => {
const { f, started, driver, state } = game();
const host = hostFor(f, started, driver);
const result = await runTool('play', { action: 'play', until: 0.505 }, host);
expect(result.isError).toBeUndefined();
expect(result.structuredContent).toEqual({ time: 0.505, paused: true });
expect(text(result)).toBe(`Paused at 0.505 s of game time. ${NO_CAMERA_HINT}`);
expect(started.world.time).toBeCloseTo(0.505, 9);
expect(state).toMatchObject({ paused: true, pausing: null });
// Paused, it holds: a frame later the time is the same.
await driver.frame();
expect(started.world.time).toBeCloseTo(0.505, 9);
});
it('resume and pause go on from where the game is', async () => {
const { f, started, driver, state } = game();
const host = hostFor(f, started, driver);
await runTool('play', { action: 'pause', until: 0.25 }, host);
expect(started.world.time).toBeCloseTo(0.25, 9);
// The host's play is a stand-in here: resuming is the test's.
const resuming = { ...host, play: () => ((state.paused = false), null) };
const resumed = await runTool('play', { action: 'resume', until: 1 }, resuming);
expect(resumed.structuredContent).toEqual({ time: 1, paused: true });
expect(started.world.time).toBeCloseTo(1, 9);
});
it('turns down what cannot be: stop, a time gone by or too far ahead, paused or running already', async () => {
const { f, started, driver, state } = game();
const host = hostFor(f, started, driver);
const play = async (args: Record<string, unknown>) => text(await runTool('play', args, host));
expect(await play({ action: 'stop', until: 1 })).toBe('until goes with play, resume or pause');
expect(await play({ action: 'pause', until: 31 })).toMatch(/31 s ahead; at most 30 s/);
expect(await play({ action: 'resume', until: 1 })).toMatch(/running already: pause it with until/);
await runTool('play', { action: 'pause', until: 0.5 }, host);
expect(await play({ action: 'pause', until: 1 })).toMatch(/paused already: resume it with until/);
expect(await play({ action: 'resume', until: 0.5 })).toBe(
'The game is at 0.5 s already: until has to lie ahead',
);
state.paused = false;
state.hidden = true;
expect(await play({ action: 'pause', until: 1 })).toMatch(/in the background/);
expect(started.world.time).toBeCloseTo(0.5, 9);
});
it('gives up when the time stands still, and calls the pause off', async () => {
const { f, started, driver, state } = game();
state.frozen = true;
const result = await runTool('play', { action: 'pause', until: 1 }, hostFor(f, started, driver));
expect(result.isError).toBe(true);
expect(text(result)).toMatch(/^The game time stood still at 0 s/);
expect(state.pausing).toBeNull();
});
it('says so when something paused the game before the time', async () => {
const { f, started, driver, state } = game();
const host = hostFor(f, started, driver);
const frame = driver.frame;
driver.frame = async () => {
await frame();
if (started.world.time > 0.2) state.paused = true;
};
const result = await runTool('play', { action: 'pause', until: 1 }, host);
expect(text(result)).toMatch(/^Paused at 0\.21\d* s of game time, before 1 s: the user or a Break node/);
expect(state.pausing).toBeNull();
});
it('a last pause step holds the game after a wait to its time exactly, and after a tap at once', async () => {
const { started, driver, state } = game();
const waited = await sendInput([{ wait: 0.31 }, { pause: true }], driver, 25_000);
expect(waited).toMatchObject({ time: 0.31, paused: true });
expect(started.world.time).toBeCloseTo(0.31, 9);
expect(state.pausing).toBeNull();
state.paused = false;
const tapped = await sendInput([{ key: 'Space' }, { pause: true }], driver, 25_000);
expect(tapped).toMatchObject({ paused: true, frames: 2 });
expect(started.world.time).toBeCloseTo(0.31 + 2 / 60, 9);
state.paused = false;
const plain = await sendInput([{ wait: 0.1 }], driver, 25_000);
expect(plain.paused).toBe(false);
});
});
describe('get_runtime', () => {
it('lists every object with its position, a behavior with its variables, and the console', async () => {
const { f, box, started, driver } = game();
@@ -234,6 +354,96 @@ describe('get_runtime', () => {
);
expect(text(await runTool('get_runtime', {}, hostFor(f, started, driver, false)))).toMatch(/not running/);
});
it('says what is heard: levels in dB by bus name, voices and playings by asset, the rest left out (#436)', async () => {
const { f, started, driver } = game();
const made = (type: 'sound' | 'instrument' | 'song' | 'bus', name: string) => {
const command = new CreateAssetCommand(type, name);
f.stack.execute(command);
return command.asset;
};
const sound = made('sound', 'Jump');
if (sound.type === 'sound') sound.file = 'jump.wav';
const instrument = made('instrument', 'Lead');
const song = made('song', 'Theme');
const track = song.type === 'song' ? song.tracks[0]! : null;
const bus = made('bus', 'Music');
const owner = started.world.entities.find((entity) => entity.type === 'behavior')!.id;
const trackBus = `play:7:song:${song.id}:${track!.id}`;
const quiet = { released: false, ringing: false, startsIn: 0 };
const level = (peak: number, rms: number): Pick<HeardSound['buses'][number], 'peak' | 'rms'> => ({
peak: [peak, peak / 2],
rms: [rms, rms],
});
const heard: HeardSound = {
window: 32768 / 48000,
clock: { state: 'running', time: 12.34567, held: false },
voices: [
{ id: 1, file: 'jump.wav', owner, bus: '', ...quiet },
{
id: 2,
file: `instrument:${instrument.id}`,
owner: '',
bus: trackBus,
note: 64,
...quiet,
released: true,
},
{
id: 3,
file: `instrument:${instrument.id}`,
owner: '',
bus: trackBus,
note: 65,
...quiet,
startsIn: 0.2,
},
],
waiting: [{ id: 4, file: 'boom.wav', owner: '', bus: `mix:${bus.id}` }],
failed: ['boom.wav'],
playings: [{ id: 7, name: `song:${song.id}`, owner, playing: true, fading: null, baked: false }],
buses: [
{ name: trackBus, ...level(0.5, 0.1) },
{ name: `${trackBus}:#items`, ...level(0.5, 0.1) },
{ name: 'mix', ...level(1, 0.01) },
{ name: `mix:${bus.id}`, ...level(0, 0) },
{ name: 'play:7', ...level(0.5, 0.1) },
],
};
const host = hostFor(f, started, driver);
const facts = host.runtime!()!;
host.runtime = () => ({ ...facts, audio: heard });
const view = (await runTool('get_runtime', {}, host)).structuredContent as { audio: object };
expect(view.audio).toEqual({
clock: { state: 'running', time: 12.346 },
window: 0.68,
levels: [
{ bus: 'master', peak: 0, rms: -40 },
{ bus: `song Theme, track ${track!.name}`, peak: -6, rms: -20 },
{ bus: 'Music', silent: true },
{ bus: 'song Theme', peak: -6, rms: -20 },
],
voices: [
{ sound: 'Jump', bus: 'master', by: 'Box' },
{ instrument: 'Lead', note: 'E4', bus: `song Theme, track ${track!.name}`, released: true },
],
planned: 1,
playing: [{ song: 'Theme', by: 'Box' }],
waiting: [{ file: 'boom.wav', bus: 'Music' }],
failed: [{ file: 'boom.wav' }],
});
// A clock the browser holds while the game runs means nothing is heard, paused since or not.
const audioWith = async (clock: HeardSound['clock']) => {
host.runtime = () => ({ ...facts, audio: { ...heard, clock } });
return ((await runTool('get_runtime', {}, host)).structuredContent as { audio: { hint?: string } })
.audio;
};
expect(await audioWith({ state: 'running', time: 3, held: true })).not.toHaveProperty('hint');
expect((await audioWith({ state: 'suspended', time: 0, held: false })).hint).toBe(STALLED_HINT);
expect((await audioWith({ state: 'suspended', time: 0, held: true })).hint).toBe(STALLED_HINT);
});
});
describe('the camera', () => {
+7 -7
View File
@@ -41,7 +41,6 @@ import { MAX_PX_PER_TICK, tickToX, zoomTime } from '../../src/editor/music/timeV
const BAR = 1920;
const TRACKS = ['t1', 't2', 't3'];
const rows = new Map(TRACKS.map((id, i) => [id, i]));
const tracks = TRACKS.map((id) => ({ id, name: id, volume: 0, pan: 0, mute: false, solo: false, fx: '' }));
function item(
id: string,
@@ -175,14 +174,14 @@ describe('where things are in the Song tab (#334)', () => {
const items = [item('a', 't1', 0, BAR), item('b', 't2', BAR, BAR)];
it('finds an item and which end of it is under the pointer', () => {
expect(itemAt(view, items, tracks, { x: 96, y: TRACK_HEIGHT / 2 })).toEqual({ id: 'a', part: 'body' });
expect(itemAt(view, items, tracks, { x: 190, y: TRACK_HEIGHT / 2 })).toEqual({ id: 'a', part: 'end' });
expect(itemAt(view, items, tracks, { x: 194, y: TRACK_HEIGHT * 1.5 })).toEqual({
expect(itemAt(view, items, rows, { x: 96, y: TRACK_HEIGHT / 2 })).toEqual({ id: 'a', part: 'body' });
expect(itemAt(view, items, rows, { x: 190, y: TRACK_HEIGHT / 2 })).toEqual({ id: 'a', part: 'end' });
expect(itemAt(view, items, rows, { x: 194, y: TRACK_HEIGHT * 1.5 })).toEqual({
id: 'b',
part: 'start',
});
expect(itemAt(view, items, tracks, { x: 96, y: TRACK_HEIGHT * 1.5 })).toBeNull();
expect(itemsWithin(view, items, tracks, { x: 150, y: 10 }, { x: 250, y: TRACK_HEIGHT + 10 })).toEqual([
expect(itemAt(view, items, rows, { x: 96, y: TRACK_HEIGHT * 1.5 })).toBeNull();
expect(itemsWithin(view, items, rows, { x: 150, y: 10 }, { x: 250, y: TRACK_HEIGHT + 10 })).toEqual([
'a',
'b',
]);
@@ -202,7 +201,8 @@ describe('where things are in the Song tab (#334)', () => {
expect(headPartAt(solo.x + 4, solo.y + 4)).toBe('solo');
expect(headPartAt(volume.x + 10, volume.y + 4)).toBe('volume');
expect(headPartAt(pan.x + 10, pan.y + 4)).toBe('pan');
expect(headPartAt(20, 8)).toBe('name');
expect(headPartAt(30, 8)).toBe('name');
expect(headPartAt(10, 12)).toBe('lanes');
expect(sliderValue(volume.x, volume, -60, 6, 0.5)).toBe(-60);
expect(sliderValue(volume.x + volume.w, volume, -60, 6, 0.5)).toBe(6);
expect(sliderValue(pan.x + pan.w / 2, pan, -1, 1, 0.05)).toBe(0);
+276
View File
@@ -0,0 +1,276 @@
import { describe, expect, it, vi } from 'vitest';
import type { AutomationPoint } from '../../src/core/music/automation';
import type { SongTrack } from '../../src/core/music/song';
import type { Project } from '../../src/core/model/Project';
import type { MenuEntry, MenuItem, MenuSubmenu } from '../../src/ui';
import { easingEntries } from '../../src/editor/song/easingMenu';
import {
addedPoint,
draggedPoint,
easedPoint,
lanePoints,
withLanePoints,
withoutLane,
withoutPoint,
withParamLane,
} from '../../src/editor/song/laneEdit';
import {
LANE_INSET,
laneKindOf,
laneLabel,
laneStep,
laneValue,
laneValueText,
laneWay,
laneY,
pointAt,
shownLanes,
songRows,
trackLanes,
trackRows,
} from '../../src/editor/song/laneView';
import { trackAtY, type SongView } from '../../src/editor/song/songView';
function track(id: string, automation?: SongTrack['automation']): SongTrack {
return {
id,
name: id,
volume: -6,
pan: 0,
mute: false,
solo: false,
fx: '',
...(automation && { automation }),
};
}
const VOLUME: AutomationPoint[] = [
{ tick: 0, value: -12 },
{ tick: 960, value: 0, ease: 'step' },
{ tick: 1920, value: -24 },
];
describe('the rows of the Song tab with automation lanes (#337)', () => {
const tracks = [track('a'), track('b', [{ target: 'pan', points: [{ tick: 0, value: 0.5 }] }]), track('c')];
it('shows the lanes with points until a track is opened or closed by hand', () => {
expect(shownLanes(tracks[1]!, new Map())).toEqual(['pan']);
expect(shownLanes(tracks[0]!, new Map())).toEqual([]);
// Opened by hand the lanes come in their own order, closed by hand even one with points goes.
expect(shownLanes(tracks[0]!, new Map([['a', ['pan', 'volume'] as const]]))).toEqual(['volume', 'pan']);
expect(shownLanes(tracks[1]!, new Map([['b', []]]))).toEqual([]);
});
it('puts each lane right under its track, and finds a track by the row of its own', () => {
const open = new Map([['a', ['volume', 'pan'] as const]]);
const rows = songRows(tracks, open);
expect(rows).toEqual([
{ track: 0, lane: null },
{ track: 0, lane: 'volume' },
{ track: 0, lane: 'pan' },
{ track: 1, lane: null },
{ track: 1, lane: 'pan' },
{ track: 2, lane: null },
]);
expect([...trackRows(tracks, rows)]).toEqual([
['a', 0],
['b', 3],
['c', 5],
]);
// A lane is as high as a track, so the rows stay evenly spaced.
const view: SongView = { panX: 0, pxPerTick: 0.1, panY: 0, trackHeight: 40 };
expect(trackAtY(view, rows.length, 3 * 40 + 1)).toBe(3);
});
});
describe('the way of a lane (#337)', () => {
const view: SongView = { panX: 10, pxPerTick: 0.1, panY: 0, trackHeight: 52 };
it('draws a value from the bottom of its range to the top, and reads it back to a step', () => {
expect(laneY('volume', -60, 52)).toBe(52 - LANE_INSET);
expect(laneY('volume', 12, 52)).toBe(LANE_INSET);
expect(laneY('pan', 0, 52)).toBe(26);
// Past the range a value is drawn at the edge.
expect(laneY('pan', 3, 52)).toBe(LANE_INSET);
expect(laneValue('pan', 26, 52)).toBe(0);
expect(laneValue('volume', laneY('volume', -6.3, 52), 52)).toBeCloseTo(-6.3, 6);
expect(laneValue('volume', -100, 52)).toBe(12);
expect(laneValue('pan', laneY('pan', 0.333, 52), 52)).toBeCloseTo(0.33, 6);
});
it('finds the point under the pointer, the last drawn first', () => {
const points = [
{ tick: 0, value: 0 },
{ tick: 0, value: 0 },
{ tick: 960, value: 6 },
];
const y = laneY('volume', 0, 52);
expect(pointAt(view, 'volume', points, 10, y)).toBe(1);
expect(pointAt(view, 'volume', points, 10 + 96 + 5, laneY('volume', 6, 52) - 5)).toBe(2);
expect(pointAt(view, 'volume', points, 60, y)).toBeNull();
});
it('holds before the first point and after the last, and jumps upright on a step', () => {
const way = laneWay(view, 'volume', VOLUME, 300, 2);
const at = (x: number) => way.filter((p) => p.x === x).map((p) => p.y);
// Before the first point (x 10) the lane holds its value.
expect(at(0)).toEqual([laneY('volume', -12, 52)]);
// Halfway to the point at tick 960 (x 106) the linear way is at -6 dB, and reaches 0 dB there.
expect(at(58)).toEqual([laneY('volume', -6, 52)]);
expect(at(106)).toEqual([laneY('volume', 0, 52)]);
// The step holds until tick 1920 (x 202), where it jumps to the last point.
expect(at(200)).toEqual([laneY('volume', 0, 52)]);
expect(at(202)).toEqual([laneY('volume', 0, 52), laneY('volume', -24, 52)]);
expect(at(300)).toEqual([laneY('volume', -24, 52)]);
expect(way.map((p) => p.x)).toEqual([...way.map((p) => p.x)].sort((a, b) => a - b));
});
it('says a value as a head and a sign do', () => {
expect(laneValueText('volume', -6)).toBe('-6.0 dB');
expect(laneValueText('volume', 3.25)).toBe('+3.3 dB');
expect(laneValueText('volume', -60)).toBe('-inf dB');
expect(laneValueText('pan', 0)).toBe('Centre');
expect(laneValueText('pan', -0.25)).toBe('25% left');
expect(laneValueText('pan', 1)).toBe('100% right');
});
});
describe('editing the points of a lane (#337)', () => {
it('puts the points on their lane, the lanes in their order, and drops a lane without points', () => {
const a = withLanePoints(
track('a', [{ target: 'pan', points: [{ tick: 0, value: 1 }] }]),
'volume',
VOLUME,
);
expect(a.automation?.map((l) => l.target)).toEqual(['volume', 'pan']);
expect(lanePoints(a, 'volume')).toEqual(VOLUME);
const b = withLanePoints(a, 'pan', []);
expect(b.automation?.map((l) => l.target)).toEqual(['volume']);
expect('automation' in withLanePoints(b, 'volume', [])).toBe(false);
});
it('adds a point after the ones already on its tick, held to whole ticks and the range', () => {
expect(addedPoint(VOLUME, 'volume', 960.4, 3)).toEqual({
points: [VOLUME[0], VOLUME[1], { tick: 960, value: 3 }, VOLUME[2]],
index: 2,
});
expect(addedPoint([], 'pan', -5, 4)).toEqual({ points: [{ tick: 0, value: 1 }], index: 0 });
});
it('drags a point no further than its neighbours, keeping how it eases', () => {
expect(draggedPoint(VOLUME, 'volume', 1, 5000, -3)[1]).toEqual({ tick: 1920, value: -3, ease: 'step' });
expect(draggedPoint(VOLUME, 'volume', 1, -10, 99)[1]).toEqual({ tick: 0, value: 12, ease: 'step' });
expect(draggedPoint(VOLUME, 'volume', 2, 9999.6, -60)[2]).toEqual({ tick: 10000, value: -60 });
expect(draggedPoint(VOLUME, 'volume', 7, 0, 0)).toEqual(VOLUME);
});
it('takes a point away and sets how one eases, linear stored as none', () => {
expect(withoutPoint(VOLUME, 1)).toEqual([VOLUME[0], VOLUME[2]]);
expect(easedPoint(VOLUME, 0, 'sine-in')[0]).toEqual({ tick: 0, value: -12, ease: 'sine-in' });
expect(easedPoint(VOLUME, 1, 'linear')[1]).toEqual({ tick: 960, value: 0 });
});
});
describe('the lane of a parameter (#337)', () => {
const cut = { target: 'param:cut' as const, min: 0, max: 2000 };
const withCut = track('a', [
{ target: 'volume', points: [{ tick: 0, value: -6 }] },
{ ...cut, points: [] },
]);
it('is shown by its name after the volume and the pan, with no point too: it was added', () => {
expect(laneLabel('param:cut')).toBe('cut');
expect(trackLanes(withCut)).toEqual(['volume', 'pan', 'param:cut']);
expect(shownLanes(withCut, new Map())).toEqual(['volume', 'param:cut']);
expect(laneKindOf(withCut, 'param:cut')).toEqual(cut);
expect(laneKindOf(withCut, 'pan')).toEqual({ target: 'pan' });
});
it('draws and reads its values in its own range, to a step that cuts it into a hundred or more', () => {
expect(laneY(cut, 0, 52)).toBe(52 - LANE_INSET);
expect(laneY(cut, 2000, 52)).toBe(LANE_INSET);
expect(laneStep(cut)).toBe(10);
expect(laneStep({ target: 'param:mix' })).toBe(0.01);
expect(laneValue(cut, laneY(cut, 1234, 52), 52)).toBe(1230);
expect(laneValue({ target: 'param:mix' }, 26, 52)).toBe(0.5);
expect(laneValueText(cut, 1234)).toBe('1230');
expect(laneValueText({ target: 'param:mix' }, 0.123)).toBe('0.12');
expect(addedPoint([], cut, 0, 5000).points).toEqual([{ tick: 0, value: 2000 }]);
});
it('keeps its lane and range without a point, takes its points into a new range, and goes only when removed', () => {
const a = withLanePoints(withCut, 'param:cut', [{ tick: 0, value: 1500 }]);
expect(a.automation?.map((l) => l.target)).toEqual(['volume', 'param:cut']);
const emptied = withLanePoints(a, 'param:cut', []);
expect(emptied.automation?.[1]).toEqual({ ...cut, points: [] });
const narrow = withParamLane(a, 'cut', { min: 0, max: 1000 });
expect(narrow.automation?.[1]).toEqual({ ...cut, max: 1000, points: [{ tick: 0, value: 1000 }] });
// A new one comes last; a later volume lane still goes first.
const more = withLanePoints(withParamLane(narrow, 'tone', { min: 0, max: 1 }), 'pan', [
{ tick: 0, value: 0 },
]);
expect(more.automation?.map((l) => l.target)).toEqual(['volume', 'pan', 'param:cut', 'param:tone']);
expect(withoutLane(more, 'param:cut').automation?.map((l) => l.target)).toEqual([
'volume',
'pan',
'param:tone',
]);
});
});
describe('the easings as a menu (#337)', () => {
const project = {
assets: [
{ id: 'c2', type: 'curve', name: 'Swell' },
{ id: 'c1', type: 'curve', name: 'Dip' },
{ id: 'c3', type: 'curve', name: 'Inside', owner: 'song' },
],
} as unknown as Project;
const byLabel = (entries: MenuEntry[], label: string) =>
entries.find((e) => 'label' in e && e.label === label) as MenuItem & MenuSubmenu;
it('offers every easing, marks the one in use and hands back what is picked', () => {
const set = vi.fn();
const entries = easingEntries(project, 'cubic-out', set);
expect(entries.map((e) => ('label' in e ? e.label : '-'))).toEqual([
'Linear',
'Step',
'Sine',
'Quad',
'Cubic',
'Quart',
'Quint',
'Expo',
'Circ',
'Back',
'Elastic',
'Bounce',
'Curve',
]);
expect(byLabel(entries, 'Linear').current).toBe(false);
const cubic = byLabel(entries, 'Cubic').submenu as MenuItem[];
expect(cubic.map((e) => [e.label, e.current])).toEqual([
['In', false],
['Out', true],
['In-out', false],
]);
(byLabel(entries, 'Quad').submenu as MenuItem[])[0]!.onSelect();
byLabel(entries, 'Step').onSelect();
expect(set.mock.calls).toEqual([['ease-in'], ['step']]);
});
it('lists the Curve assets of the project by name, not those a song keeps to itself', () => {
const set = vi.fn();
const curve = byLabel(easingEntries(project, 'curve:c2', set), 'Curve');
const items = curve.submenu as MenuItem[];
expect(items.map((e) => [e.label, e.current])).toEqual([
['Dip', false],
['Swell', true],
]);
items[0]!.onSelect();
expect(set).toHaveBeenCalledWith('curve:c1');
expect(
byLabel(easingEntries({ assets: [] } as unknown as Project, 'linear', set), 'Curve').disabled,
).toBe(true);
});
});
+59
View File
@@ -0,0 +1,59 @@
import { describe, expect, it } from 'vitest';
import { heldSongMarkers } from '../../src/core/music/markers';
import {
MARKER_FLAG,
MARKER_REACH,
MARKER_STRIP,
markerAt,
markerNameX,
markerSpans,
SONG_TOP,
} from '../../src/editor/song/markerView';
import { RULER_HEIGHT } from '../../src/editor/music/timeView';
/** A tick a pixel, from x 100; every name 6 pixels a letter. */
const VIEW = { panX: 100, pxPerTick: 1 };
const measure = (name: string) => name.length * 6;
describe('the marker strip of the song tab (#338)', () => {
it('lies between the ruler and the tracks', () => {
expect(SONG_TOP).toBe(RULER_HEIGHT + MARKER_STRIP);
});
it('draws a marker at its tick, as wide as its flag and name, but not into the next one', () => {
const markers = heldSongMarkers([
{ id: 'a', name: 'Intro', tick: 0 },
{ id: 'b', name: 'A long name', tick: 40 },
{ id: 'c', name: '', tick: 300 },
]);
const spans = markerSpans(VIEW, markers, measure);
expect(spans.map((s) => s.x)).toEqual([100, 140, 400]);
// Room to the next one is 40 pixels, less 2: the name of "Intro" (30) and the flag fit just so.
expect(spans[0]!.w).toBe(38);
expect(spans[1]!.w).toBe(MARKER_FLAG + 3 + measure('A long name'));
expect(spans[2]!.w).toBe(MARKER_FLAG + 3 + measure('Marker 3'));
expect(markerNameX(spans[0]!)).toBe(100 + MARKER_FLAG + 3);
});
it('keeps the flag of a marker with another one close behind it', () => {
const markers = heldSongMarkers([
{ id: 'a', name: 'Close', tick: 0 },
{ id: 'b', name: 'By', tick: 2 },
]);
expect(markerSpans(VIEW, markers, measure)[0]!.w).toBe(MARKER_FLAG);
});
it('takes the marker near its line or on its flag and name, the later one where they overlap', () => {
const markers = heldSongMarkers([
{ id: 'a', name: 'Intro', tick: 0 },
{ id: 'b', name: 'B', tick: 6 },
]);
const spans = markerSpans(VIEW, markers, measure);
expect(markerAt(spans, 100 - MARKER_REACH)).toBe('a');
expect(markerAt(spans, 100 - MARKER_REACH - 1)).toBeNull();
expect(markerAt(spans, 103)).toBe('b');
expect(markerAt(spans, 106 + MARKER_FLAG + 3 + 6)).toBe('b');
expect(markerAt(spans, 106 + MARKER_FLAG + 3 + 7)).toBeNull();
expect(markerAt([], 100)).toBeNull();
});
});
+264 -1
View File
@@ -15,9 +15,11 @@ import {
type FxStagePlan,
type FxVoice,
} from '../../src/core/soundfx/nodes';
import type { AutomationTarget } from '../../src/core/music/automation';
import { dbToGain } from '../../src/core/music/level';
import type { MixBus } from '../../src/core/music/mixer';
import type { BakePlay, SequencePlay } from '../../src/core/music/playbacks';
import type { ScheduledNote } from '../../src/core/music/Scheduler';
import type { ScheduledAutomation, ScheduledNote } from '../../src/core/music/Scheduler';
import type { LfoOptions, PitchOptions, StreamOptions } from '../../src/viewport/fxWorklet';
import { saturationCurve } from '../../src/core/soundfx/saturation';
import { stageNodes } from '../../src/viewport/fxNodes';
@@ -1982,6 +1984,177 @@ describe('the patterns and songs a game plays (#336)', () => {
});
});
describe('the automation of the tracks of a song (#337)', () => {
const automation = (fields: Partial<ScheduledAutomation>): ScheduledAutomation => ({
bus: 'song:s:t',
target: 'volume',
fresh: false,
marks: [],
...fields,
});
it('leaves what a lane sets alone, and hands it back to the fader from where the lane left it', async () => {
const library = new SoundLibrary(false);
// The first gain is where the bus is fed, the second its level.
await library.setBuses([{ name: 'song:s:t', gain: 0.5, pan: 0.25, automated: ['pan'] }]);
await library.setBuses([{ name: 'song:s:t', gain: 0.4, pan: -1, automated: ['pan'] }]);
expect([gains[1]!.glides, panners[0]!.glides, panners[0]!.holds]).toEqual([[0.5, 0.4], [0.25], 0]);
await library.setBuses([{ name: 'song:s:t', gain: 0.4, pan: -1 }]);
expect([panners[0]!.glides, panners[0]!.holds, gains[1]!.holds]).toEqual([[0.25, -1], 1, 0]);
});
it('ramps a bus to each mark or jumps there, a fresh start first dropping what was planned', async () => {
const library = new SoundLibrary(false);
await library.setBuses([{ name: 'song:s:t', gain: 1, pan: 0, automated: ['volume'] }]);
library.automateBus(
'song:s:t',
automation({
fresh: true,
marks: [
{ at: 1, value: 0, jump: true },
{ at: 2, value: 0.5, jump: false },
],
}),
);
library.automateBus('song:s:t', automation({ marks: [{ at: 3, value: 0.25, jump: false }] }));
// The hold by hand of a context without cancelAndHoldAtTime, at the first mark.
expect(gains[1]!.ramps).toEqual([
['set', 1, 1],
['set', 0, 1],
['linear', 0.5, 2],
['linear', 0.25, 3],
]);
library.automateBus(
'song:s:gone',
automation({ bus: 'song:s:gone', marks: [{ at: 1, value: 0, jump: true }] }),
);
expect(library.busLevels().map((bus) => bus.name)).toEqual(['mix', 'song:s:t']);
});
it('takes the tracks of a playing in a game along their lanes, and no bus the song does not have', async () => {
const library = new SoundLibrary(false);
const lane = (bus: string) => ({
bus,
target: 'pan' as const,
points: [
{ tick: 0, value: -1 },
{ tick: 1920, value: 1 },
],
});
const id = library.playSequence(
'song:s',
{
sequence: {
notes: [{ tick: 0, length: 480, note: 60, velocity: 1, instrument: 'i', bus: 'song:s:t' }],
lanes: [lane('song:s:t'), lane('song:s:gone')],
},
tempo: 120,
length: 1920,
loop: null,
needs: { layers: [], files: [] },
buses: [{ name: 'song:s:t', gain: 1, pan: -1, automated: ['pan'] }],
strike: () => 0,
},
'',
);
await settle();
expect(library.busLevels().map((bus) => bus.name)).toContain(`play:${id}:song:s:t`);
// One bus only is taken along: the track's, over its first window – on from the first note, a
// lookahead ahead, 48 ticks at 120 BPM.
const taken = panners.filter((panner) => panner.ramps.length > 0);
expect(taken).toHaveLength(1);
const ramps = taken[0]!.ramps.map(([kind, value, at]) => [
kind,
value,
Math.round(Number(at) * 1e6) / 1e6,
]);
expect(ramps).toEqual([
['set', -1, 0.05],
['set', -1, 0.05],
['linear', -0.95, 0.1],
]);
expect(library.busLevels().some((bus) => bus.name.includes('gone'))).toBe(false);
});
/** A gain whose decibels are the parameter `loud`; with `source` it plays FILE, as a clip's FX does. */
const loud = (fields: Partial<FxPlan> = {}): FxVoice => ({
plan: {
pitch: [],
stages: [{ kind: 'gain', db: { value: 0, param: 'loud', min: -120, max: 24 } }],
...fields,
},
params: {},
tempo: 120,
});
const clip = (library: SoundLibrary, bus: string, file = FILE) =>
library.playClip(file, { at: 0, offset: 0, duration: 10, volume: 1, bus }, loud());
const tracks = (fx: FxVoice, automated: readonly AutomationTarget[] = ['param:loud']) => [
{ name: 'song:s:t', gain: 1, pan: 0, fx, automated },
{ name: 'song:s:t:#items', gain: 1, pan: 0, into: 'song:s:t' },
{ name: 'song:s:u:#items', gain: 1, pan: 0 },
];
const marks = (...values: [number, number][]) =>
automation({
target: 'param:loud',
marks: values.map(([at, value]) => ({ at, value, jump: false })),
});
/** What was planned for each gain before the clips fade out at their end, 10 s on. */
const moved = () =>
gains.map((gain) => gain.ramps.filter(([, , at]) => Number(at) < 5)).filter((ramps) => ramps.length > 0);
it('takes the sound FX of a track and every voice inside it along a parameter, in what their knobs hold', async () => {
const library = new SoundLibrary(false);
await library.setBuses(tracks(loud()));
await settle();
clip(library, 'song:s:t:#items');
clip(library, 'song:s:u:#items');
await settle();
await arrive();
library.automateBus('song:s:t', marks([1, -6], [2, -12]));
// The chain of the track and the voice on it, each first set where nothing was planned yet.
const way = [
['set', expect.closeTo(dbToGain(-6), 9), 1],
['linear', expect.closeTo(dbToGain(-12), 9), 2],
];
expect(moved()).toEqual([way, way]);
// The next window ramps on from there; a mark where a knob stays is left out.
library.automateBus('song:s:t', marks([2, -12], [3, 0]));
expect(moved()[0]).toEqual([...way, ['linear', expect.closeTo(dbToGain(-12), 9), 2], ['linear', 1, 3]]);
library.automateBus('song:s:t', marks([3, 0], [4, 0]));
expect(moved()[0]).toHaveLength(4);
});
it('takes up the marks planned for a voice still waiting on its bytes once it starts', async () => {
const library = new SoundLibrary(false);
await library.setBuses(tracks({ ...loud(), plan: { pitch: [], stages: [] } }, []));
clip(library, 'song:s:t:#items', OTHER);
library.automateBus('song:s:t', marks([1, -6], [2, -12]));
expect(moved()).toEqual([]);
await settle();
await arrive();
expect(moved()).toEqual([
[
['set', expect.closeTo(dbToGain(-6), 9), 1],
['linear', expect.closeTo(dbToGain(-12), 9), 2],
],
]);
});
it('leaves a parameter a lane rules alone when the tracks are set again, and hands one let go back', async () => {
const library = new SoundLibrary(false);
// One plan throughout, as a sound FX compiles to: another would be another chain.
const fx = loud();
await library.setBuses(tracks(fx));
library.automateBus('song:s:t', marks([1, -12]));
const knob = gains.find((gain) => gain.ramps.length > 0)!;
await library.setBuses(tracks({ ...fx, params: { loud: -3 } }));
expect(knob.glides).toEqual([]);
// The lane is gone: back to what the track's FX is set to, its default.
await library.setBuses(tracks(fx, []));
expect(knob.glides).toEqual([1]);
});
});
describe('a bake (#340)', () => {
const BAKED = 'baked.ogg';
/** A pattern of one note, baked: `round` null plays it once to `end`. */
@@ -2342,6 +2515,80 @@ describe('a sound FX on a bus (#346)', () => {
expect(disconnects).toEqual(['panner', 'splitter']);
});
it('taps every bus for the agent while it watches, a later one too, with its peak and RMS (#436)', async () => {
const library = new SoundLibrary(false);
await library.setBuses([{ name: 'song:s:t', gain: 1, pan: 0 }]);
expect(library.heard().buses).toEqual([]);
library.watch(true);
expect(analysers.map((side) => side.fftSize)).toEqual([32768, 32768, 32768, 32768]);
await library.setBuses([
{ name: 'song:s:t', gain: 1, pan: 0 },
{ name: 'song:s:u', gain: 1, pan: 0 },
]);
expect(library.heard().buses.map((bus) => bus.name)).toEqual(['mix', 'song:s:t', 'song:s:u']);
// The fake holds one sample in the middle of the stretch and silence round it.
analysers[2]!.sample = -0.5;
analysers[3]!.sample = 0.25;
const track = library.heard().buses[1]!;
expect(track).toMatchObject({ name: 'song:s:t', peak: [0.5, 0.25] });
expect(track.rms[0]).toBeCloseTo(0.5 / Math.sqrt(32768), 9);
expect(track.rms[1]).toBeCloseTo(0.25 / Math.sqrt(32768), 9);
expect(library.heard().window).toBeCloseTo(32768 / 48000);
// The Song tab's meter is another tap: letting it go leaves the agent's.
library.busPeaks(['song:s:t']);
library.unmeter();
expect(library.heard().buses).toHaveLength(3);
// Moved elsewhere, a bus keeps its tap.
wires.length = 0;
await library.setBuses([{ name: 'song:s:t', gain: 1, pan: 0, into: 'song:s:u' }]);
expect(wires.filter((wire) => wire === 'panner > splitter')).toHaveLength(1);
disconnects.length = 0;
library.watch(false);
expect(library.heard().buses).toEqual([]);
expect(disconnects.filter((node) => node === 'splitter')).toHaveLength(3);
});
it('says what plays: each voice with its owner, bus and note, those planned, waiting or failed (#436)', async () => {
const library = new SoundLibrary(false);
await play(library, FILE, { owner: 'o1' });
const gate = {
pitch: [],
stages: [],
source: { file: FILE, volume: 1, loop: true, loopStart: 0, loopEnd: 0, gate: true },
} as FxPlan;
library.playNote('instrument:i', [gate], {}, 1, '', 0, { at: 5, bus: 'song:s:t', note: 64 });
library.play(OTHER, 1, false, 'o2');
vi.stubGlobal('fetch', () => Promise.resolve({ ok: false, status: 404 }));
vi.spyOn(console, 'error').mockImplementation(() => {});
await library.load('gone.wav');
const heard = library.heard();
expect(heard.voices).toEqual([
{ id: 1, file: FILE, owner: 'o1', bus: '', startsIn: 0, released: false, ringing: false },
{
id: 2,
file: 'instrument:i',
owner: '',
bus: 'song:s:t',
note: 64,
startsIn: 5,
released: false,
ringing: false,
},
]);
expect(heard.waiting).toEqual([{ id: 3, file: OTHER, owner: 'o2', bus: '' }]);
expect(heard.failed).toEqual(['gone.wav']);
library.release(2);
expect(library.heard().voices[1]).toMatchObject({ released: true });
// Paused, the clock is told as it ran before: held by the pause, or by the browser already.
expect(heard.clock).toEqual({ state: 'running', time: 0, held: false });
library.setSuspended(true);
contextState = 'suspended';
expect(library.heard().clock).toEqual({ state: 'running', time: 0, held: true });
library.setSuspended(false);
library.setSuspended(true);
expect(library.heard().clock).toEqual({ state: 'suspended', time: 0, held: true });
});
it('makes the chain of a track in the Song tab once, and again only for another sound FX', async () => {
const library = new SoundLibrary(false);
const track = { name: 'song:s:t', gain: 1, pan: 0, fx: voice(crush) };
@@ -2539,6 +2786,22 @@ describe('the master and the buses of the project (#325)', () => {
expect(wires.filter((wire) => wire.endsWith('> destination'))).toEqual(['panner > destination']);
});
it('makes a bus at its level, so the first sound of a run on a quiet bus is not heard gliding down (#436)', async () => {
const library = new SoundLibrary(false);
await library.setMix(mix);
await play(library);
expect(library.busLevels().map(({ name, level }) => [name, level])).toEqual([
['mix', 0.5],
['mix:a', 1],
['mix:b', 0.25],
]);
expect(gains.filter((gain) => gain.glides.length > 0)).toEqual([]);
// One there already glides to a new level, as before.
await library.setMix([bus('mix', 0.25, null), mix[1]!, mix[2]!]);
expect(library.busLevels()[0]).toMatchObject({ gain: 0.25, level: 0.5 });
expect(gains.some((gain) => gain.glides.includes(0.25))).toBe(true);
});
it('keeps the mix where the first thing heard is a tab of a song, which names its bus before there is a context', async () => {
const library = new SoundLibrary(false);
await library.setMix(mix);
+303
View File
@@ -490,6 +490,215 @@ export async function song(t) {
song.tracks[2].solo = false;
song.tracks[0].mute = false;`);
// 65g2. Automation lanes (#337). The triangle in a head opens a lane for the volume and one for the
// pan right under its track, and what lies below moves down. A click on a lane sets a point on
// the grid, a press dragged on sets one and takes it along, as one step. A dragged point stops
// at its neighbour. A point's menu sets how it eases to the next, it says its value when the
// pointer rests on it, and a double-click takes it away. The fader a lane rules does not move.
// Clear lane leaves the lane open and empty, the triangle closes it.
const lanesShown = () => json('__gm.song.automation()');
const volumeAt = (tick, value) => json(`__gm.song.automationAt(1, 'volume', ${tick}, ${value})`);
const volumePoints = async () =>
(await model()).tracks[1].automation?.find((lane) => lane.target === 'volume')?.points ?? [];
const lowItem = (await model()).items.find((item) => item.track !== tracks[0] && item.track !== tracks[1]);
const lowY = async () => (await json('__gm.song.items()')).find((item) => item.id === lowItem?.id)?.box.y;
const lowBefore = await lowY();
const rowHeight = (await json('__gm.song.view()')).trackHeight;
p = await json("__gm.song.headAt(1, 'lanes')");
await click(p.x, p.y);
await sleep(100);
const lanesOpened = await lanesShown();
const lowAfter = await lowY();
check(
'the triangle of a head opens a volume and a pan lane under its track, and what lies below moves down',
JSON.stringify(lanesOpened.rows.map((row) => row.lane)) ===
JSON.stringify([null, null, 'volume', 'pan', null, null, null]) &&
lanesOpened.rows[2].track === tracks[1] &&
lanesOpened.rows[3].track === tracks[1] &&
lowItem !== undefined &&
Math.abs(lowAfter - lowBefore - 2 * rowHeight) < 0.5,
JSON.stringify({ lanesOpened, lowBefore, lowAfter, rowHeight }),
);
p = await volumeAt(BAR + 100, -12);
await click(p.x, p.y);
await sleep(100);
const onePoint = await volumePoints();
p = await volumeAt(3 * BAR, -30);
to = await volumeAt(4 * BAR + 50, 0);
await drag(p.x, p.y, to.x, to.y);
await sleep(100);
const twoPoints = await volumePoints();
const addedLabel = await json('__gm.state.commands.undoLabel');
await press('z', { ctrl: true });
await sleep(100);
const laneUndone = await volumePoints();
await clickToolbar('redo');
await sleep(100);
const laneRedone = await volumePoints();
check(
'a click on a lane sets a point on the bar, a drag sets one and takes it along as one step',
onePoint.length === 1 &&
onePoint[0].tick === BAR &&
Math.abs(onePoint[0].value + 12) < 1.5 &&
twoPoints.length === 2 &&
twoPoints[1].tick === 4 * BAR &&
Math.abs(twoPoints[1].value) < 1.5 &&
addedLabel === 'Add automation point' &&
laneUndone.length === 1 &&
JSON.stringify(laneRedone) === JSON.stringify(twoPoints),
JSON.stringify({ onePoint, twoPoints, addedLabel, laneUndone, laneRedone }),
);
p = await volumeAt(twoPoints[0].tick, twoPoints[0].value);
to = await volumeAt(6 * BAR, twoPoints[0].value);
await drag(p.x, p.y, to.x, to.y);
await sleep(100);
const heldPoints = await volumePoints();
await press('z', { ctrl: true });
await sleep(100);
check(
'a point dragged past its neighbour stops on its tick: two points there, a jump',
heldPoints.length === 2 &&
heldPoints[0].tick === 4 * BAR &&
heldPoints[1].tick === 4 * BAR &&
Math.abs(heldPoints[0].value - twoPoints[0].value) < 0.2 &&
JSON.stringify(await volumePoints()) === JSON.stringify(twoPoints),
JSON.stringify(heldPoints),
);
p = await volumeAt(twoPoints[0].tick, twoPoints[0].value);
await click(p.x, p.y, 'right');
await sleep(150);
const pointRows = await json('__gm.menuRows()');
await clickText('Ease to next point');
await sleep(150);
await clickText('Sine');
await sleep(150);
await clickText('In');
await sleep(150);
const eased = await volumePoints();
const shownValue = (v) =>
`${Math.round(v * 10) / 10 > 0 ? '+' : ''}${(Math.round(v * 10) / 10).toFixed(1)} dB`;
p = await volumeAt(twoPoints[1].tick, twoPoints[1].value);
await mouse('mouseMoved', p.x - 1, p.y);
await mouse('mouseMoved', p.x, p.y);
await t.waitUntil('__gm.tooltip() !== null', { label: 'the sign over a point of a lane' });
const pointSign = await json('__gm.tooltip()');
check(
"a point's menu sets how it eases to the next, and resting on a point says its value",
pointRows?.includes('Ease to next point') &&
pointRows.includes('Delete point') &&
eased[0].ease === 'sine-in' &&
eased[1].ease === undefined &&
pointSign?.text === shownValue(twoPoints[1].value),
JSON.stringify({ pointRows, eased, pointSign }),
);
p = await json("__gm.song.headAt(1, 'volume', 0.5)");
to = await json("__gm.song.headAt(1, 'volume', 0)");
await drag(p.x, p.y, to.x + 1, to.y);
await sleep(100);
const ruledFader = { volume: (await model()).tracks[1].volume, notice: await json('__gm.state.notice') };
p = await volumeAt(twoPoints[1].tick, twoPoints[1].value);
await doubleClick(p.x, p.y);
await sleep(100);
const afterDouble = await volumePoints();
check(
'the fader a lane rules does not move, and a double-click takes a point away',
ruledFader.volume === 0 &&
ruledFader.notice.startsWith('Volume follows its automation lane') &&
afterDouble.length === 1 &&
afterDouble[0].tick === BAR,
JSON.stringify({ ruledFader, afterDouble }),
);
p = await json("__gm.song.automationHeadAt(1, 'volume')");
await click(p.x, p.y, 'right');
await sleep(150);
await clickText('Clear lane');
await sleep(100);
const laneCleared = { track: (await model()).tracks[1], rows: (await lanesShown()).rows.length };
p = await json("__gm.song.headAt(1, 'lanes')");
await click(p.x, p.y);
await sleep(100);
const closedRows = (await lanesShown()).rows.length;
check(
'Clear lane takes the lane out of the song and leaves it open, the triangle closes the lanes',
!('automation' in laneCleared.track) &&
laneCleared.rows === 7 &&
closedRows === 5 &&
(await lowY()) === lowBefore,
JSON.stringify({ laneCleared, closedRows }),
);
await evaluate("__gm.state.notice = ''");
// 65g3. Markers (#338) on their strip under the ruler. A double-click there adds one on the grid,
// a drag moves it as one step, a click starts playing at it, a double-click names it, resting
// on it says its name, and its menu takes it away.
const songMarkers = async () => (await model()).markers;
const markerAt = (tick) => json(`__gm.song.markerAt(${tick})`);
p = await markerAt(BAR + 100);
await doubleClick(p.x, p.y);
await sleep(100);
const added = await songMarkers();
p = await markerAt(BAR);
to = await markerAt(3 * BAR + 50);
await drag(p.x, p.y, to.x, to.y);
await sleep(100);
const movedMarkers = await songMarkers();
const movedLabel = await json('__gm.state.commands.undoLabel');
await press('z', { ctrl: true });
await sleep(100);
const markerUndone = await songMarkers();
await clickToolbar('redo');
await sleep(100);
const markerDrawn = (await json('__gm.song.markers()'))[0];
const rulerX = (await json(`__gm.song.rulerAt(${3 * BAR})`)).x;
check(
'a double-click on the strip adds a marker on the bar, a drag moves it as one step, drawn at its tick',
added.length === 1 &&
added[0].name === 'Marker 1' &&
added[0].tick === BAR &&
movedMarkers[0].tick === 3 * BAR &&
movedLabel === 'Move marker' &&
markerUndone[0].tick === BAR &&
(await songMarkers())[0].tick === 3 * BAR &&
Math.abs(markerDrawn?.x - rulerX) < 1,
JSON.stringify({ added, movedMarkers, movedLabel, markerUndone, markerDrawn, rulerX }),
);
p = await markerAt(3 * BAR);
await click(p.x, p.y);
await sleep(100);
const startAtMarker = (await json('__gm.song.view()')).start;
await sleep(500);
await doubleClick(p.x, p.y);
await sleep(150);
const nameDialog = await json('__gm.groupDialogOpen()');
await t.typeText('Boss');
await press('Enter');
await sleep(100);
const named = (await songMarkers())[0];
// The presses left the marker's sign quiet: the pointer goes off the strip and back.
to = await json(`__gm.song.rulerAt(${3 * BAR})`);
await mouse('mouseMoved', to.x, to.y);
await mouse('mouseMoved', p.x + 5, p.y);
await mouse('mouseMoved', p.x + 6, p.y);
await t.waitUntil('__gm.tooltip() !== null', { label: 'the sign over a marker' });
const markerSign = await json('__gm.tooltip()');
check(
'a click on a marker starts playing there, a double-click names it, and resting on it says the name',
startAtMarker === 3 * BAR && nameDialog === true && named?.name === 'Boss' && markerSign?.text === 'Boss',
JSON.stringify({ startAtMarker, nameDialog, named, markerSign }),
);
await click(p.x, p.y, 'right');
await sleep(150);
const markerRows = await json('__gm.menuRows()');
await clickText('Delete marker');
await sleep(100);
const markersLeft = await model();
check(
"a marker's menu plays from it, renames it and takes it away",
['Play from here', 'Rename…', 'Delete marker'].every((row) => markerRows?.includes(row)) &&
markersLeft.markers.length === 0,
JSON.stringify({ markerRows, markers: markersLeft.markers }),
);
// 65h. Space plays from the start: the pattern a bar into itself at 2 s a bar, the sound as a
// clip round and round, each on the bus of its track.
p = await json('__gm.song.rulerAt(0)');
@@ -531,6 +740,100 @@ export async function song(t) {
bus(tracks[1])?.gain === 1,
JSON.stringify(buses),
);
// 65h2. A lane of the second track's volume (#337), -40 dB to 0 dB over eight bars, takes its bus
// along while the song plays, from the window after it is set on: the level follows the
// playhead, and the fader turned all the way down meanwhile does not move it. The loop is
// made long, so the playhead does not jump back while it is read.
const loopBefore = (await model()).loop;
await mutate(`
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
song.loop = { start: 0, end: ${64 * BAR} };
song.tracks[1].automation = [{ target: 'volume', points: [{ tick: 0, value: -40 }, { tick: ${8 * BAR}, value: 0 }] }];`);
const laneDb = (ticks) => -40 + (40 * ticks) / (8 * BAR);
const readLane = () =>
json(`({
level: __gm.song.buses().find((b) => b.name === ${JSON.stringify(busOf(tracks[1]))})?.level,
at: __gm.song.position()?.ticks,
})`);
await sleep(300);
const laneFirst = await readLane();
await mutate(`
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
song.tracks[1].volume = -60;`);
await sleep(1000);
const laneLater = await readLane();
const offLane = (r) => Math.abs(20 * Math.log10(r.level) - laneDb(r.at));
check(
'a lane of the volume takes the bus of its track along the playhead, and the fader no longer moves it',
[laneFirst, laneLater].every((r) => r.level > 0 && r.at < 8 * BAR && offLane(r) < 1) &&
laneLater.at - laneFirst.at > 800 &&
20 * Math.log10(laneLater.level / laneFirst.level) > 2,
JSON.stringify({ laneFirst, laneLater, off: [laneFirst, laneLater].map(offLane) }),
);
await mutate(`
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
delete song.tracks[1].automation;
song.tracks[1].volume = 0;`);
// 65h3. A lane of a parameter (#337, stage 3): the sound item of the first track goes through a
// sound FX whose parameter `cut` is its gain in dB, and starts again a bar on, eight bars
// long. Its clip starts at the value the lane has there, and its knob follows the lane while
// it plays: one voice, read twice.
const cutItem = (await model()).items.find((i) => i.kind === 'sound');
const itemBefore = JSON.stringify(cutItem);
const startAt = Math.ceil(((await json('__gm.song.position()')).ticks + BAR) / BAR) * BAR;
await mutate(`
__gm.state.project.assets.push({
id: 'sg65-fx',
type: 'soundfx',
name: 'Cut 65',
props: {},
graph: {
nodes: ${JSON.stringify([
node('sg65-fx-in', 'SoundFxInput', 0, 0, {}, [{ key: 'cut', kind: 'number', default: 0 }]),
node('sg65-fx-gain', 'SoundFxGain', 200, 0),
node('sg65-fx-out', 'SoundFxOutput', 400, 0),
])},
edges: ${JSON.stringify([
edge('sg65-fx-e1', 'sg65-fx-in', 'out', 'sg65-fx-gain', 'in'),
edge('sg65-fx-e2', 'sg65-fx-in', 'cut', 'sg65-fx-gain', 'gain'),
edge('sg65-fx-e3', 'sg65-fx-gain', 'out', 'sg65-fx-out', 'in'),
])},
},
});
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
const item = song.items.find((i) => i.id === ${JSON.stringify(cutItem.id)});
Object.assign(item, { fx: 'sg65-fx', tick: ${startAt}, length: ${8 * BAR} });
song.tracks[0].automation = [
{ target: 'param:cut', min: -40, max: 0, points: [{ tick: 0, value: -40 }, { tick: ${16 * BAR}, value: 0 }] },
];`);
const cutDb = (ticks) => -40 + (40 * ticks) / (16 * BAR);
const readCut = () =>
json(`({
voices: __gm.sound.levels(${JSON.stringify(blip?.file ?? '')}).filter((knobs) => knobs.length > 0),
at: __gm.song.position()?.ticks,
clip: __gm.song.clips().filter((c) => c.fx === 'sg65-fx').at(-1) ?? null,
})`);
const cutFirst = await waitFor(readCut, (r) => r.voices.length > 0 && r.at > startAt + 0.25 * BAR, 40);
await sleep(1500);
const cutLater = await readCut();
const offCut = (r) => Math.abs(20 * Math.log10(r.voices[0]?.at(-1) ?? 0) - cutDb(r.at));
check(
'a lane of a parameter starts a clip at its value there and takes its knob along while it plays',
[cutFirst, cutLater].every((r) => r.voices.length === 1 && offCut(r) < 1) &&
Math.abs(cutFirst.clip?.params?.cut - cutDb(startAt)) < 0.01 &&
cutLater.at - cutFirst.at > 600 &&
20 * Math.log10(cutLater.voices[0].at(-1) / cutFirst.voices[0].at(-1)) > 1,
JSON.stringify({ startAt, cutFirst, cutLater, off: [cutFirst, cutLater].map(offCut) }),
);
await mutate(`
const song = __gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)});
const at = song.items.findIndex((i) => i.id === ${JSON.stringify(cutItem.id)});
song.items[at] = ${itemBefore};
delete song.tracks[0].automation;
__gm.state.project.assets = __gm.state.project.assets.filter((a) => a.id !== 'sg65-fx');
${loopBefore ? `song.loop = ${JSON.stringify(loopBefore)};` : 'delete song.loop;'}`);
await click(...Object.values(await json("__gm.song.headAt(0, 'mute')")));
// Mute closes every send of the track (#350); its own level stays as it is set.
const closed = (list) => {
+100
View File
@@ -600,6 +600,106 @@ export async function mcp(t) {
JSON.stringify({ started: started?.content, x0, x1 }),
);
// 71h3. A moment of the game (#435): play with until holds it at that game time exactly, resume
// with until goes on to the next, and a last pause step holds it at the end of a wait, not a
// frame beyond (the game runs between two calls, so the wait is what is exact there).
const atHalf = await callTool('play', { action: 'play', until: 0.505 });
const halfInfo = await json('__gm.playInfo()');
const halfRead = (await callTool('get_runtime', { console: 0 }))?.structuredContent;
const onward = await callTool('play', { action: 'resume', until: 0.8 });
const onwardInfo = await json('__gm.playInfo()');
await callTool('play', { action: 'resume' });
const waited = await callTool('play_input', { steps: [{ wait: 0.31 }, { pause: true }] });
const waitedInfo = await json('__gm.playInfo()');
await callTool('play', { action: 'stop' });
await waitUntil('__gm.state.mode === "edit"', { label: 'the game stopped again' });
check(
'play with until and a last pause step hold the game at the game time asked for, not a frame beyond',
atHalf?.structuredContent?.time === 0.505 &&
halfInfo.paused &&
Math.abs(halfInfo.time - 0.505) < 1e-6 &&
halfRead?.time === 0.505 &&
onward?.structuredContent?.time === 0.8 &&
onwardInfo.paused &&
Math.abs(onwardInfo.time - 0.8) < 1e-6 &&
waited?.structuredContent?.paused === true &&
waited.structuredContent.seconds === 0.31 &&
waitedInfo.paused &&
Math.abs(waitedInfo.time - waited.structuredContent.time) < 1e-3,
JSON.stringify({
atHalf: atHalf?.structuredContent ?? atHalf?.content,
halfInfo,
halfTime: halfRead?.time,
onward: onward?.structuredContent ?? onward?.content,
onwardInfo,
waited: waited?.structuredContent ?? waited?.content,
waitedInfo,
}),
);
// 71h4. What the agent hears (#436): the sound of 71g9 loops twice from the start of the game, once
// on the master and once on a bus at volume 0. Held at a game time, get_runtime names both voices,
// the master is heard, the quiet bus is silent, and the levels stay as they are while it is paused.
const noisy = await callTool('run_commands', {
label: 'noise',
steps: [
{ command: 'add_file_asset', args: { type: 'sound', file: wav.file }, as: 'tone' },
{ command: 'create_asset', args: { type: 'bus', name: `${TOKEN_NAME} quiet` }, as: 'quiet' },
{ command: 'set_asset_property', args: { asset: '$quiet', key: 'volume', value: 0 } },
{ command: 'create_asset', args: { type: 'behavior', name: `${TOKEN_NAME} noise` }, as: 'noise' },
{ command: 'place_asset', args: { in: sceneId, asset: '$noise', x: 0, y: 0, parent: playerId } },
{ command: 'add_node', args: { asset: '$noise', node: 'OnStart' }, as: 'start' },
{
command: 'add_node',
args: { asset: '$noise', node: 'PlaySound', params: { sound: '$tone', loop: 'true' } },
as: 'loud',
},
{
command: 'add_node',
args: {
asset: '$noise',
node: 'PlaySound',
params: { sound: '$tone', loop: 'true', bus: '$quiet' },
},
as: 'hushed',
},
...['$loud', '$hushed'].map((node) => ({
command: 'connect',
args: { asset: '$noise', from: { node: '$start', port: 'exec' }, to: { node, port: 'exec' } },
})),
],
});
await settle();
const quietName = `${TOKEN_NAME} quiet`;
const heldAt = await callTool('play', { action: 'play', until: 0.6 });
const heardFirst = (await callTool('get_runtime', { console: 0 }))?.structuredContent?.audio;
await sleep(400);
const heardLater = (await callTool('get_runtime', { console: 0 }))?.structuredContent?.audio;
await callTool('play', { action: 'stop' });
await waitUntil('__gm.state.mode === "edit"', { label: 'the noisy game stopped' });
await callTool('undo');
await settle();
const levelOf = (audio, bus) => audio?.levels?.find((row) => row.bus === bus);
const toneVoices = (heardFirst?.voices ?? []).filter((voice) => voice.sound !== undefined);
check(
'get_runtime says what is heard: both voices by sound and bus, the master loud, a bus at 0 silent, held while paused',
noisy?.isError === false &&
heldAt?.structuredContent?.paused === true &&
toneVoices.length === 2 &&
toneVoices.some((voice) => voice.bus === 'master') &&
toneVoices.some((voice) => voice.bus === quietName) &&
heardFirst.window > 0.6 &&
levelOf(heardFirst, 'master')?.peak > -12 &&
levelOf(heardFirst, quietName)?.silent === true &&
JSON.stringify(heardLater?.levels) === JSON.stringify(heardFirst?.levels),
JSON.stringify({
text: noisy?.content?.[0]?.text?.slice(0, 300),
heldAt: heldAt?.structuredContent ?? heldAt?.content,
heardFirst,
laterLevels: heardLater?.levels,
}),
);
// 71i. One Ctrl+Z of the user takes the agent's whole step back.
await press('z', { ctrl: true });
await settle();
+5 -1
View File
@@ -56,7 +56,11 @@ const INSTRUCTIONS = [
'The editor follows your edits: after run_commands the user sees the graph, scene or asset it worked',
'on, with what it touched selected. Use open and select to show anything else.',
'Test what you built yourself: play, then play_input sends keys and mouse clicks into the game,',
'and get_runtime reads where the objects are, their variables and the console. A game needs a',
'and get_runtime reads where the objects are, their variables and the console, and under audio',
'what you would hear: the voices sounding and the level of each bus, silent where nothing reaches',
'it. Check there that music and sounds are really heard. To look at one',
'moment, play with until holds the game at that game time, as does a last play_input step',
'{"pause": true}; a call that only reads comes too late for it, the game runs on meanwhile. A game needs a',
'camera in its scene to show anything; get_view says when it has none.',
'When a tool answers that no editor is connected, ask the user to open the editor and switch on',
'"Agent" in its account menu.',