meleki/feuerwerk
1
0 Fork 0
Code Issues 35 Actions Projects 1 Wiki Activity

Compare commits

base: meleki/feuerwerk:23922ee4a28d38aea5bb522f5b1ad46fe02ff16f
meleki/feuerwerk:main
...
compare: meleki/feuerwerk:main
meleki/feuerwerk:main
145 Commits
23922ee4a2 ... main
Author SHA1 Message Date
adminandClaude Opus 5.5 b5ea7524e6 Let a pattern swing its eighths or sixteenths (#372)
feuerwerk / gate (push) Failing after 12m20s
Details
feuerwerk / suite (push) Failing after 25m17s
Details 
A pattern played its notes strictly on the grid, so a shuffle or a
groove needed every second note moved by hand in the piano roll. Now
a pattern carries its own swing, as agreed in comment 3994.

- Two new rows in the inspector: Swing (%) from 0 to 100, default 0,
  and Swing grid with 1/8 (default) or 1/16.
- Swing is only heard. Model, piano roll, ghost notes and song items
  keep the notes on the grid. notesIn adds a separate late offset at
  planning time, so loops and automation still go by the grid.
- It applies in the pattern tab, as a pattern item of a song, to
  Play pattern and Play song, in compositions and in bakes. A
  composition applies it exactly once. Sound items stay straight.
- A changed swing makes a bake stale. Bakes without swing stay valid.
- Pairs count from each bar start. A pair cut by the bar end stays
  straight, like the last eighth of a 7/8 bar.
- A note that swing would push past a loop end starts one tick
  before it.

Tests:
- Unit: 10 new (swing 9, composition-audio 1). At 1/8 and 120 BPM
  the second eighth comes 1/12 s late at 100 %, 1/24 s at 50 %.
- 13 counter-checks, all red.
- e2e 64 on atx: ALL PASSED, 38 checks. New 64k' measures gaps of
  0.333333 and 0.166667 s in turn. Without late it sees 0.25 s.
- Gate on atx: exit 0, 338 files, 5232 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-30 21:34:21 +01:00
adminandClaude Opus 5.5 b268891f48 Run the CI suite on stein2 next to the gate 
Suite 759 went red on e2e 67a-ii (firstAt 0.083 s) while a job of
another project loaded the GPU on melektaus-atx. The suite measures
frame timings, so it should not share that GPU.

- The suite job now runs on `feuerwerk-ci`, the runner on stein2,
  like the gate. Its config.yaml already passes /dev/dri/renderD128
  and a 1 GB /dev/shm into the job container, and has capacity 2.
- The header of ci.yml says how to move the suite back to
  `feuerwerk-wsl`, and that GM_TEST_WORKERS now matters again,
  because gate and suite share the cores of one machine.

Tests: YAML parses, both jobs name `feuerwerk-ci`. Not run in CI yet,
the next push starts it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-30 15:40:40 +01:00
adminandClaude Opus 5.5 e53121a0bb Let a behavior hear a song, pattern or sound come to its end (#373)
feuerwerk / gate (push) Successful in 5m56s
Details
feuerwerk / suite (push) Failing after 20m55s
Details 
A game should be able to start the next piece of music when one is
over, or wait for a sound before it goes on. Until now a behavior
could start and stop music and sounds, but never learn that they had
ended.

- Two new events: On music finished with the fields Song and Pattern
  and the input playback, and On sound finished with the field Sound
  and the input voice. Both give out what ended and stopped.
- Two new data nodes, Music playing and Sound playing, ask whether
  something is still heard.
- Over means nothing is heard any more: no tail of the effects, and
  a fade-out has run down.
- stopped is on for Stop song, pattern or sound, when the behavior
  that owns it ends and on a scene change. For a sound also when a
  newer voice needs its place or the file does not load.
- Only what the Play nodes of this run started is heard, across
  scene changes too. Previews in the tabs, compositions and notes of
  instruments do not count.
- Switch song ends nothing. The number goes on, and the end is told
  under the song heard last.
- Playbacks and the SoundLibrary tell each end once to a new
  PlayedEnds per run. The AudioSwitch passes them on from both sinks.

Found and fixed: a pattern whose last bar ends in silence counted as
done up to one planning window before its end. In e2e 65 the next
pattern started after 1.962 s instead of 2 s. Scheduler.done now also
wants the playhead to have reached the end. Cleaning up done
playbacks therefore comes a few tenths of a second later. One test
that expected the early end is adjusted.

Tests:
- Unit: 23 new (played-ends 12, sound-library 5, node catalog 5,
  audio-switch 1), all green.
- Counter-probes: 19 bends, all red. One was green at first for want
  of a case, which is added now.
- e2e 65 on atx: ALL PASSED, 78 checks. New 65k6: the event came
  once with stopped off, and the second pattern started 2.051 s after
  the stinger. Stop song gave exactly one end with stopped on, and
  the switch from 65k3 gave none.
- Gate on atx: CHECK EXIT 0, 337 files, 5222 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-30 14:48:16 +01:00
adminandClaude Opus 5.5 2333f66ba2 Keep e2e 65 and 67 green in the full suite after #410
feuerwerk / gate (push) Successful in 5m55s
Details
feuerwerk / suite (push) Successful in 20m21s
Details 
Suite 755 for b17a924 was red in three places. Both e2e 65 and e2e 67
were green on their own. Two of the new checks did not fit into the
full run. The product is not at fault.

- e2e 66 (66s): the per-frame pixel read never stopped. splice(0)
  emptied the array, so the length bound was never reached. Each
  pixelAt leaves the stage shifted to its pixel until the next
  drawing. The editor, idle after Stop, kept it at (-710, -376), so
  the Play click of e2e 67 hit no object. The loop now ends on a flag
  while the run still draws each frame.
- e2e 65 (65k2, 65k3): the console panel keeps 500 lines and is never
  cleared in the suite. Lines of earlier steps pushed the first beats
  out, so the beats began at 1.3. The checks now read the harness's
  console log since Play, which has no limit.
- 65k2 also writes panelBusiest, the three most frequent other lines
  in the panel. The next CI run names the module that logs that much.

Tests (melektaus-atx):
- e2e 66+67 before the fix: 67 breaks off as in CI.
- e2e 66+67 with the fix: ALL PASSED, 52 checks.
- e2e 65 under a flood of 100 lines/s: ALL PASSED, 76 checks.
  The old reading under the same flood shows the CI picture.
- e2e 65+66+67: ALL PASSED, 126 checks (a first run had the known
  e2e 67 firstAt wobble, 0.1).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-30 13:05:46 +01:00
adminandClaude Opus 5.5 b17a924157 Let a behavior play, pause and seek a video and hear it end (#410)
feuerwerk / gate (push) Successful in 5m58s
Details
feuerwerk / suite (push) Failing after 17m35s
Details 
A game should be able to stop a video on a sprite, jump to a spot in
it and start it again, and react when it has run out. Until now a
video on a sprite either ran free on its own or followed a
composition, and no behavior could touch it.

- Five new nodes: Play video, Pause video and Seek video (time in s
  from the clip start), On video finished with a Video field, and the
  data node Video time with time, length and playing.
- Play, Pause and Seek act on the owner when object is not linked.
- On video finished with an empty field hears every video. A chosen
  video also hears its clips.
- A new VideoSystem keeps a clock for every sprite with a video. It
  counts from the moment the sprite appears, as long as it is on. For
  a Driven sprite the clock is its Video time. A paused game holds all
  clocks.
- A free sprite that one of the nodes touches gets the clock
  "steered" at run time. It then has its own player, and other
  sprites with the same clip play on. Pause of the game and sound go
  the existing Driven way.
- On a Driven sprite and in the editor run the nodes do nothing and
  write a line to the log.
- Until its own player has a frame, a steered sprite shows the frame
  of the shared player. Without that it would blink out on the first
  Pause video.
- Play at the end of a clip without loop starts it from the start.
  The issue does not say, this is my choice.

Found, not fixed: with a Driven sprite in the scene the run waits up
to 3 s for its frame at the start. The world does not tick then, but
the free videos already play. Their clocks lag behind the picture by
that wait, so On video finished comes that much late.

Tests:
- Unit: 23 new (video-steer 16, node catalog 7), affected groups
  4820/4820.
- Counter-probes: 16 bends, all red.
- e2e 66 on atx: ALL PASSED, 39 checks. New 66s: Pause at 0.15 s
  held its own player at 0.2 s, the picture stayed red for 1 s. Seek
  landed on 0.5 s exactly, green, held. Play showed green, blue,
  white, and white stayed at the end. On video finished came once.
  Without the clock "steered" all three 66s checks are red.
- Gate on atx: CHECK EXIT 0, 336 files, 5199 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-30 10:13:52 +01:00
adminandClaude Opus 5.5 f9428c5bc2 Let a game take a song's automation lane over and give it back (#338) 
A game should be able to steer a lane of a song, the filter of the
pads in a fight, the pan of a track when the player turns, and later
hand it back to the curve the composer drew. Set song lane holds a
lane of a track at a value, Release song lane glides it back.

- Set song lane takes playback, track ("Song / Track"), lane, value,
  fade and quantize (Next bar by default). lane is volume, pan or a
  parameter name, with or without "param:".
- From where the lane stands it goes over fade to value and stays.
  The value is clamped to the lane's range. It replaces the lane and
  is not mixed with it (decision 10).
- Release song lane glides over fade back to the curve, to where the
  curve stands at the end of the fade. Then the lane follows the
  curve again.
- The override sits in the Scheduler, where the lane marks are made.
  Volume, pan and parameters go one way. The maths is pure functions
  in the new module laneHolds.ts.
- Notes that start during a parameter hold get the held value. A
  Switch song to the same song keeps the holds.
- The holds are a list per lane. The running one holds until the next
  one lands. A key held down fires the node on every beat, and with a
  single hold per lane the new call replaced the running one before it
  landed, so the lane fell back to the curve. e2e 65 found this.
- A track without that lane and a baked song write a line to the log
  (with a hint at "live tracks").
- The test harness knows the keys l and r.

Tests:
- Unit: 24 new (lane-holds 14, song-play 5, sound-library 2, node
  catalog 3), affected groups 748/748.
- Counter-probes: 12 bends, all red.
- e2e 65 on atx: ALL PASSED, 76 checks. New 65k5: the hold landed on
  the bar line, curve until 0.02 s before, -40 dB 0.14 s after, no
  values between. It stood 1.8 s. The release began at -39.8 dB and
  followed the curve again from 0.55 s on. With one hold per lane
  65k5 is red.
- Gate on atx: CHECK EXIT 0, 335 files, 5176 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-30 07:22:12 +01:00
adminandClaude Opus 5.5 e390d39314 Let a sound, note or pattern land on the beat of the music (#338) 
A stinger has to start on the next beat or bar of the song that is
playing, not whenever the frame happens to run. Play sound, Play note
and Play pattern now take the number of a song or pattern playback and
plan their start on the audio clock, as the music is heard.

- New inputs sync (a playback number) and quantize (Next bar by
  default, decision 6) on Play sound, Play note and Play pattern.
- The landing is worked out in Playbacks (landOn), on the next beat,
  bar or marker of the playback as it is heard. SoundLibrary and the
  audio switch only pass it on, and only on the clock the stinger
  plays on.
- A pattern with sync plays in the tempo of the playback. It takes its
  bake only when both tempos match.
- Without a link, or when the playback is not heard, the node starts
  at once as before.
- The sync ports live in their own module musicSync.ts. Putting them
  in songSync.ts would have made an import cycle through Play song.

Tests:
- Unit: 8 new (6 in song-play, 1 each in sound-library and
  audio-switch), all green.
- Counter-probes: 12 bends, all red.
- e2e 65 on atx: ALL PASSED, 74 checks. New 65k4: the pattern (own
  tempo 100) started exactly 8 beats after the song and played at the
  song's 120 BPM. Without sync it is red (6.2 beats, 100 BPM).
- Gate on atx: CHECK EXIT 0, 334 files, 5152 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-30 06:40:25 +01:00
adminandClaude Opus 5.5 53ec1094f4 Let a game switch songs and turn their tracks on the beat (#338) 
A game could start and stop a song, but not steer one while it plays.
The battle theme could not take over on the next bar, a song could not
jump to a marker, and a track could not fall away when the player
hides. This is stage 3 of 5 in #338.

- Quantize: now, next beat, next bar (the default) or next marker. The
  spot is the next one as the song is heard, and the change is planned
  on the audio clock to the sample, round the loop as the ruler counts.
- Set song track turns one track in dB on top of the song's level, over
  a fade, on the bus input before its FX and sends.
- Switch song plays another song, or the same one from a marker, in
  place of the playing one. It keeps the playback number, so Stop song,
  Set song track and On beat go on reaching it.
- A playing now has a key for its buses apart from its public number.
  During a switch the old and the new song each have buses of their own.
- Play song has a "live tracks" switch. With it off a baked song plays
  its bake, and Set song track on it writes a line to the log.
- A Switch song to a marker plays the notes, as a bake only plays from
  its start.

Found on the way:

- A cut on a bar line faded over seconds in Chrome. cancelAndHoldAtTime
  holds nothing where nothing is planned later, so the ramp ran from the
  bus's last change. The turn now sets its start value on the spot. It
  had been measured at 0.52 of full level 0.2 s past the bar line.

Tests:
- Unit: 23 new cases in beats, song-play, sound-library and the node
  catalog. Counter-checks: 17 bends, all compile, all red.
- e2e 65 (atx): ALL PASSED, 73 checks. 65k3 is new. The switch landed
  three bars after the start under the same number, and the track cut
  on the next bar line with nothing in between. With the old ramp 65k3
  is red (0.53 and 0.51 between).
- Gate (atx): CHECK EXIT 0, 334 files, 5144 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-29 21:17:52 +01:00
adminandClaude Opus 5.5 a6a41b6b69 Let a behavior follow the beat, bar and markers of the music (#338)
feuerwerk / gate (push) Failing after 15m41s
Details
feuerwerk / suite (push) Canceled after 18m24s
Details 
Stage 2 of #338. A game that syncs to its music has to know where the
music is heard. This stage reads that on the audio clock and hands it
to behaviors as events and as a position.

- New event nodes On beat (playback, bar, beat, late), On bar
  (playback, bar, late) and On song marker (playback, marker, name,
  late).
- New data node Song position with bar, beat, fraction, seconds and
  playing.
- The nodes have the fields Song and Pattern. With neither picked,
  every playback counts. A linked playback input picks that one.
  Compositions do not count.
- The Marker field offers the markers of all songs as Song / Marker,
  as a sound field offers its clips.
- Everything follows what is heard: the audio clock minus the output
  latency. That holds for baked songs and for Play pattern too.
- Only the last beat since the frame before fires, and only when it is
  at most half a beat late, 0.15 s at most. Bars and markers alike.
- A copied song gets markers with ids of its own, so a Marker field
  never names two songs.
- The audio clock is only asked while something plays. Asked in every
  frame, it made the AudioContext without a user gesture (#436).

Tests:
- beats 14 and music-events 6 new, 4 in song-play, 2 in song-markers,
  2 catalog cases, the touched groups 756/756
- 16 counter-probes, all compile, all red
- e2e 65 on atx: ALL PASSED, 71 checks, 65k2 new: 11 beats 1.1 to 2.4
  and 2.1 again, none missing or twice, across the pause too. late
  0.015 to 0.027 s. Counter-probe without the last position: red
- gate on atx: CHECK EXIT 0, 334 files, 5121 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-29 19:48:51 +01:00
adminandClaude Opus 5.5 5cc1cc9ef6 Tell a stream that fell behind apart from a pause that lost samples
feuerwerk / gate (push) Failing after 15m58s
Details
feuerwerk / suite (push) Canceled after 19m19s
Details 
CI task 750 failed the pause check in e2e 34 although the pause lost
nothing. The stream on the runner never got a second ahead. It had
skipped 1.1 s before the pause was clicked, with 0.04 s left ahead.
The wait for the second ran out without a word, so the check blamed
the pause for samples the stream had lost from the start.

- A check of its own before the pause asks for a second ahead and not
  a sample skipped. A runner too slow to decode fails there now.
- When the stream was behind, the pause check judges only the clock,
  the resume and the stop, and says in its name that the samples were
  not judged.
- Nothing in the product changes.

Tests:
- e2e 34 on melektaus-atx: ALL PASSED, 149 checks. The new check saw
  1.3 MB ahead and 0 samples skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-29 19:48:01 +01:00
adminandClaude Opus 5.5 56425fca6a Let a song carry named markers on a strip in the song tab (#338)
feuerwerk / gate (push) Successful in 5m22s
Details
feuerwerk / suite (push) Failing after 19m54s
Details 
Stage 1 of #338. Game sync needs named places in a song: an event when
the playhead passes one, and a switch that jumps to one. This stage
adds the markers themselves. Games do nothing with them yet.

- A song has markers, each with id, name and tick. A song without
  markers stores nothing for them. Two markers may share tick and
  name, since nodes will name them by id.
- Markers change neither the sound nor the bake fingerprint.
- The song tab draws a strip under the ruler. A marker has a green
  flag with its name and a faint line through the tracks. Tracks and
  lanes sit 18 px lower. The ruler of the pattern tab is unchanged.
- Double click on an empty place adds a marker on the grid, with Alt
  or Shift on the exact tick. Dragging moves it in one undo step.
- A click on a marker sets the start of playback there. A double click
  renames it in the name dialog of the hierarchy groups.
- The context menu offers Play from here, Rename and Delete marker,
  on an empty place Add marker. A tooltip shows the full name. The
  wheel over the strip zooms time like the ruler.
- New agent commands add_song_marker, change_song_marker and
  remove_song_marker. get_asset lists them under markers.

Tests:
- song-markers in core 6 and editor 4, 1 agent test, with the touched
  song, agent and MCP tests 175/175
- 9 counter-probes, all compile, all red
- e2e 65 on atx: ALL PASSED, 70 checks, 3 new in 65g3. Counter-probe
  where a dragged marker is not taken over: 65g3 red
- gate on atx: CHECK EXIT 0, 332 files, 5093 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-19 18:44:58 +02:00
adminandClaude Opus 5.5 2099511695 Let the agent set automation lanes of a song track (#337)
feuerwerk / gate (push) Successful in 6m3s
Details
feuerwerk / suite (push) Successful in 20m14s
Details 
Stage 4 of #337. Lanes could only be drawn in the song tab. Now the
agent can set them in one step and sees which parameters a lane of a
track can move.

- New command set_song_lane. The target is named as it is stored:
  volume, pan or param:<name>. All points come at once, each with
  tick, value and ease, from the same eases as animation keys,
  curve assets included.
- Points on one tick keep the order they come in, which makes a jump.
  points [] removes a volume or pan lane, remove a parameter lane.
- It refuses with a clear message: a value outside the range, an
  unknown ease, a missing curve asset, a range for volume or pan, and
  a new lane for a parameter nothing on the track has. The message
  then names the parameters there are.
- Without min and max a new parameter lane gets the range the dialog
  in the song tab offers.
- get_asset names under params of each song track the parameters a
  lane can move, with their default. The lanes were already in it.

Live in the user's project: get_asset named both parameters of their
two lanes (frequency, env) with defaults 20000 and 0.5. Playing to
3 s and 4.2 s, both tracks were heard, the audio clock running.

Tests:
- agent-music 13/13 (2 new), agent and MCP tests 129/129
- 6 counter-probes, all compile, all red
- gate on atx: CHECK EXIT 0, 330 files, 5082 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-19 10:54:51 +02:00
adminandClaude Opus 5.5 b50c7dbc35 Let a song track automate parameters of its sounds and FX (#337) 
Stage 3 of #337. Lanes moved volume and pan only. Now a lane can
steer a parameter of anything on its track, and a voice already
planned or still waiting for its file follows it too.

- A lane may aim at param:<name>. A parameter has no range of its
  own, so the lane keeps min and max. A parameter lane stays without
  points, so its range is not lost. Remove lane takes it away.
- It reaches the track FX, the FX of an item, the instrument of a
  note with its own FX, and the FX of a clip. When several have the
  name, all of them change, as with Set sound FX parameters.
- The scheduler hands the lane value to a note or clip at its start.
  Values a voice only reads at its start act from the next note.
- While a voice sounds, the knobs of buses and voices of the track
  follow the lane on the audio clock, read every 10 ms, because a
  knob is not always the parameter itself (trackParams.ts).
- A voice waiting for its file takes the marks when it starts.
- When a lane is removed, the parameter goes back to its default.
  An edit while playing does not reset a parameter a lane rules.
- This holds in the song tab, for Play song, in compositions and in
  the bake.
- Song tab: Add parameter lane in the track head menu opens a dialog
  for parameter and range (ParamLaneDialog.ts). The lane head menu
  has Range and Remove lane.
- Inspector: while a lane rules a value, a hint at the track names
  the lanes and says the fields act once the lane is gone.

Known limit: a knob that depends on two automated parameters reckons
each lane with the last value of the other, so it may lag one window.

Tests:
- automation-params.test.ts (9, new), 3 new in sound-library and 3
  in song-lanes. Affected files 407/407.
- 17 counter checks, all compile, all red.
- e2e 65 on melektaus-atx: ALL PASSED, 67 checks. 65h3 is new: a
  clip with FX parameter cut starts at the lane value and its knob
  stays within 0.04 dB of the lane. It goes red when automateBus
  does not pass parameters on.
- Gate on melektaus-atx: CHECK EXIT 0, 330 files, 5080 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-19 10:30:42 +02:00
adminandClaude Opus 5.5 bd99f9e79d Let automation lanes of a song track be drawn in the song tab (#337) 
Stage 2 of #337. The lanes of stage 1 played, but nobody could make
one. Now a track folds out a lane for volume and one for pan, and
points are set, dragged and eased right on the timeline.

- A lane is a row of its own, as high as a track, below its track.
  The view functions stay row based, so zoom, fit, item drags and
  the composition (which shares them) are unchanged.
- itemAt and itemsWithin take a map of track id to row.
- A triangle in the track head folds the lanes out. The head menu
  shows or hides each lane. A lane with points is open by itself.
- Which lanes are open belongs to the view of the tab and is saved
  with it. It is not part of the song and makes no undo step.
- A click sets a point on the grid, Alt or Shift on the exact tick.
  Press and drag sets one and drags it in a single undo step.
- A dragged point stops at its neighbour. Two points on one tick
  make a jump. Double click or Delete removes a point.
- The value shows next to a dragged point and as a tooltip.
- Right click on a point opens "Ease to next point" with the same
  choice as animation keys, curve assets included.
- The lane head shows the value at the playhead. Its menu clears
  or hides the lane.
- While a lane rules a value, the fader shows it in the lane colour
  and cannot be dragged (decision 4). A hint says why.
- Geometry in laneView.ts, edit rules in laneEdit.ts, the easing
  menu in easingMenu.ts. New test hooks automationAt,
  automationHeadAt and automation under __gm.song.

Open: the Volume (dB) field of a track in the inspector can still be
changed while a lane rules it. It has no effect while playing.

Tests:
- song-lanes.test.ts (12, new) and song-edit adapted: 27/27.
  Editor tests of song and composition 62/62.
- 9 counter checks, all compile, all red.
- e2e 65 on melektaus-atx: ALL PASSED, 66 checks. 65g2 is new with
  6 checks. It goes red when a set point is not taken on release.
- Gate on melektaus-atx: CHECK EXIT 0, 329 files, 5065 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-19 09:31:48 +02:00
adminandClaude Opus 5.5 389a7c0fd1 Let volume and pan of a song track follow automation lanes (#337) 
A song track can now change its volume and pan over time, played on
the audio clock wherever the song is heard. This is stage 1 of #337:
the model and the playback. Lanes cannot be drawn in the song tab yet.

- A track holds automation lanes for volume (dB) and pan. Each point
  has a tick, a value and an easing from the shared list, curve assets
  included. Two points on one tick make a jump. Before the first point
  its value holds, after the last point the last value holds.
- The rules are pure functions in core/music/automation.ts: the value
  at a tick and the pieces of a lane inside one scheduling window.
- The scheduler hands those pieces to the host in every window. The
  host sets them as ramps or jumps on the gain and panner of the
  track's bus. Shapes the clock cannot ramp directly (dB as gain,
  curved easings) are sampled every 10 ms. At the loop end the value
  jumps back with the playhead.
- This works in the song tab, for Play song, inside a composition
  (across the song loop, on the item's own bus) and in a bake.
- While a lane has points the fader no longer sets that value. When
  the lane goes, the fader takes over from the current value.
- Automation enters the bake fingerprint only when there is some, so
  existing bakes stay valid.
- The level helpers moved to core/music/level.ts, so song.ts and
  automation.ts do not import each other.

Known gap: a delaying sound FX on a bus after the track (a pitch
shifter) makes the automation late by that delay.

Tests:
- automation.test.ts (13, new), 3 new in sound-library.test.ts.
  Song, bake and composition tests together 286/286.
- 12 counter-checks, all compile, all red.
- e2e 65 on atx: ALL PASSED, 60 checks. New 65h2: in the song tab the
  bus level follows a volume lane, 0.09 dB off, also with the fader at
  -60 dB. Red when the song tab does not pass the automation on.
- Gate on atx: 328 files, 5053 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-19 06:45:55 +02:00
adminandClaude Opus 5.5 b5e071ca68 Let the agent read what is heard in the running game (#436) 
The agent has no ears. It could start music and sounds, but not tell
whether anything reached the speakers: a song without an instrument, a
bus at volume 0 or a browser that holds the audio clock all looked the
same from outside.

- get_runtime has a section audio. Under levels it gives the peak and
  RMS in dB of the master, each bus of the mix, each pattern, song or
  composition playing and the tracks of a song. Below -90 dB it says
  silent.
- The levels come from a tap of their own after the level and pan of
  each bus, an AnalyserNode per side as for the meters of the Song tab.
  It looks back about 0.7 s, so the gap between two notes is not
  silence. A pause holds the values, which fits play with until.
- Under voices it names what sounds now: a sound by its asset, a note
  of an instrument with its note name, each with its bus and object.
  Next to them planned, playing, waiting (bytes not here yet) and
  failed.
- Under clock it gives the state of the audio clock, with a hint when
  it stands still while the game runs. The state is told as it was
  before a pause, else the pause would hide that case.
- The buses are tapped only while the agent is switched on.
- The note now travels with a voice from the Start of a pattern or song
  to the SoundLibrary, so a voice can be named by it.
- tools.json and the relay's instructions tell the agent to check under
  audio that music and sounds are really heard.

Found on the user's project: started right after F5, before a click in
the page, the browser kept the audio clock suspended for the whole run.
Nothing was heard and the notes stayed planned. That is what clock now
shows.

Tests:
- Unit: sound-library (taps, levels, voices, clock), agent-play
  (audioView). Pattern and song tests expect the note in their timing.
- 9 counter-checks, 8 red. Not dropping a tap with its bus stays green,
  as the bus is cut off and gone anyway.
- e2e 71 (melektaus-atx): 28/28. New 71h4: a sound on the master and
  one on a bus at volume 0, both voices named, the master loud, the bus
  silent, the values held in the pause.
- Gate (melektaus-atx): CHECK EXIT 0, 327 files, 5037 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 22:14:07 +02:00
adminandClaude Opus 5.5 dd2096e0df Make a bus of the mix at its level, so a quiet bus starts quiet 
A bus of the project's mix was made at unity and then glided to its
level. Where the audio context comes with the first sound of a run,
that sound already plays on the new bus. On a bus at volume 0 it was
heard for a moment, loud, while the level glided down.

- setMix now sets the level of a bus it makes at once, without a
  glide. A bus that is there already glides to a new level as before.
- Found while building #436: the e2e check with a sound on a bus at
  volume 0 measured -7.8 dB on that bus. With the fix it is silent.

Tests:
- New unit test in sound-library: buses made at their level with no
  glide, a later change glides. Red without the fix.
- sound-library 130/130, tsc clean on this change alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 22:13:48 +02:00
adminandClaude Opus 5.5 70402aa19a Let the agent hold the game at a given game time (#435)
feuerwerk / gate (push) Successful in 5m31s
Details
feuerwerk / suite (push) Successful in 19m9s
Details 
The agent could not look at a short moment of a running game. Its
calls took longer than the moment lasted, so an intro window from
0.5 to 2 s slipped past twice in the 5d test. Now the editor itself
stops the clock at the time the agent names.

- play takes until: play, resume or pause let the game run on to
  that game time and hold it there. The call answers once the game
  is paused, so get_runtime and screenshot show that moment.
- The frame that would step past the time is cut short
  (frameWithin in src/core/time.ts, used in PlaySession). The game
  stands at 0.5 s, not one frame later.
- play_input takes a last step {"pause": true}. After a wait it
  holds the game at the time of the wait, after a key or click at
  once. Its answer names the game time and whether it is paused.
- A pause that comes first (toolbar or a Break node), a Load scene
  or a Restart game call the until off, and the answer says so.
- When the clock stands still, the call gives up after 25 s with
  the time it is at. A tab in the background is refused at once.
- A recording that does not run in real time is not cut. There the
  game stops in the first frame that reaches the time.
- get_runtime and play_input give times to the millisecond, not to
  the hundredth, so 0.505 no longer reads as 0.51.
- The relay's instructions name both ways.

Tests:
- Unit agent-play (6 new) and time (2 new): 19/19, related editor
  and relay tests 1158/1158.
- 6 counter-checks, all compile, all red. One stayed green at
  first: 0.3 s is exactly 18 frames. The test now takes 0.31 s.
- e2e 71 (atx): ALL PASSED, 27 checks. New 71h3 holds at 0.505 s
  and 0.8 s and after a wait of 0.31 s. Without the cut in
  PlaySession it went red at 0.5166 s.
- Gate (atx): CHECK EXIT 0, 327 files, 5033 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 21:26:53 +02:00
adminandClaude Opus 5.5 e4ccf5f72f Keep other nodes out of a group the agent makes round new nodes
feuerwerk / gate (push) Successful in 5m18s
Details
feuerwerk / suite (push) Successful in 19m7s
Details 
A group made in an agent call is fitted round its nodes only after
the call has arranged them, and the arrangement knew nothing of it.
With On key held, Move and Owner placed fresh, a group round key and
move and one round the owner, the Owner landed beside the key. Move
is taller than the key, so the Owner stood in the first group and the
two groups lay on each other. lint_graph counted none of it. Found
while testing copy_nodes (#434).

- arrange takes the new option blocks: sets of arranged nodes a group
  is fitted round afterwards. Once the columns are laid out, whatever
  would stand in such a group goes below it, a node or the group of
  another set, with everything arranged under it.
- The flow keeps its columns. A first try laid each set out as one
  block instead, but then the link from the Owner ran over the key
  inside the group (throughNodes 1).
- AgentEdit passes every group made in the call whose nodes are all
  new. Groups round nodes already there and the Arrange button stay
  as they were.

Tests:
- agent-edit: new test with the setup that showed the fault. No node
  stands in a group not its own, the groups lie apart, lint_graph has
  0 overlaps, throughNodes, backwards and coveredNotes.
- Counter-check: without the new step exactly that test is red.
- Agent, graph-arrange and graph-follow tests 120/120.
- e2e 71 on atx: ALL PASSED, 26 checks.
- Gate on atx: CHECK EXIT 0, 327 files, 5025 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 20:53:45 +02:00
adminandClaude Opus 5.5 fb981b5b0f Let the agent copy nodes, also into another graph (#434) 
The agent built every part again that already stood in a graph,
node by node, while the editor has Ctrl+C and Ctrl+V for it. The new
command `copy_nodes` in the area graphs copies nodes into the same
graph or another one, in one undo step with the rest of the call.

- The links between the copied nodes come along, and so do groups
  and comments that are only about copied nodes. A node belongs to a
  group by its middle, as when it is dragged. `boxesAbout` in
  annotationFollow.ts decides this.
- `made` names each copy by the id of its original, so later steps
  link to it with `$copy.<id>`. `$copy` alone is the first one.
- Without x and y the copies are arranged at the end of the call like
  new nodes. With them they keep their layout, top left corner there.
- Nodes the target graph does not take are refused by name, not left
  out as Ctrl+V does. The rule is the graph tab's: `fitsGraph` and
  `fittingPart` moved from clipboard.ts to paletteEntries.ts.
- A copied Variables node takes the variables of its behavior along
  into the target behavior (`CarryVariablesCommand`), with default,
  value, Keep and Private. A variable of that name of another kind
  there is refused.
- The relay's instructions name `copy_nodes`.

Found on the way:
- Groups made by the agent can take in a node of another group once
  the nodes are arranged. lint_graph does not count that.
- Undo after a paste leaves an empty `groups: []`, as
  AddAnnotationCommand does. It is the same as no list.

Tests:
- agent-copy (new, 3 tests) with the agent, relay, clipboard and
  palette tests: 118/118.
- Counter-probes: 6 bendings, all compile, all red.
- e2e 71 (atx): ALL PASSED, 26 checks.
- Gate (atx): CHECK EXIT 0, 327 files, 5024 tests.
- Live in the MCP Test Project: 5 nodes of Arrow keys walk into a new
  behavior, 6 links, group, comment and speed (300) along, lint_graph
  0 everywhere, taken back with undo.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 20:39:37 +02:00
adminandClaude Opus 5.5 e4b0419a53 Measure e2e 68h against what the sound is held to (#418) 
The check failed in CI twice in a row (tasks 740 and 741) with the
sound 61 and 52 ms ahead of the picture, against a fixed limit of
50 ms. Neither commit touched the sound timing. The limit was tighter
than what the composition promises.

- The sound follows the unrounded `player.time` and is started again
  once it lies more than `SLIP` (40 ms) off.
- The check reads the picture from `videoTime`, which counts in whole
  frames of the cut. It can lie up to one frame (1/60 s) behind.
- The first frame at 0.5 s may also come past it (0.5167 s in 740).
  That overshoot is now taken off.
- The limit is SLIP plus one frame plus 5 ms for the audio clock
  moving on between the game's frame and the test's.

Recomputed, the two red runs give 44.5 and 52.1 ms, under the new
limit of 61.7 ms.

Tests:
- e2e 68 (atx): ALL PASSED, 21 checks, 68h at 16 ms.
- Counter-probe: sound planned 100 ms early, 68h red at 116 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 20:39:14 +02:00
adminandClaude Opus 5.5 24cd9fdeaa Let the agent use stored files, presets and usages (#431 #432 #433)
feuerwerk / gate (push) Successful in 5m16s
Details
feuerwerk / suite (push) Failing after 19m8s
Details 
The agent got stuck where the editor has it easy: it could not make a
Sample play without a sound asset, it built instruments node by node
although the Preset menu has them ready, and it could not tell what
names an asset before reworking or deleting it.

- list_files is a new tool. It lists the pictures, fonts, sounds or
  videos the server holds, with size, video state and the asset that
  already wears a file. Bakes and stream copies of songs are left out.
- add_file_asset makes an asset of such a file, as the picker does.
  It reuses the create rules of the import pipeline, so the four
  import kinds are exported now. It refuses a file already in the
  project (naming that asset), a picture without a size, a video not
  yet converted and a file the server does not hold.
- run_commands fetches the file lists its steps need before it runs,
  so the commands themselves stay synchronous.
- apply_preset sets one of the 16 instrument presets or 7 sfxr
  presets, like the Preset menu. Presets take every note, so the
  C4 to B4 range of a new Notes node is no trap there.
- get_asset gains usedBy: every asset naming the asset or one of its
  frames or clips, with the object, node, item or key that does, plus
  start scene, master FX and master routing of the project.
- usesOf in src/core/model/usages.ts walks the project data instead of
  a list of fields, so no new field can be missed. The node every
  asset has in the master routing does not count as a use.
- The relay tells the agent about all three ways.

Found on the way: suite 740 for 2e4c0a0 (#430) was red in e2e 68h
only, a sound clip 61 ms early against 50 allowed. #430 did not touch
sound timing, and e2e 68 on the same tree was green on atx (-4 ms).
Taken as a timing wobble under load until the next suite says more.

Tests:
- agent-files (new, 4 tests) with the other agent and relay tests:
  130/130.
- 8 counter checks, all compiling, all red.
- e2e 71 on atx: 26/26, new 71g9 lists an uploaded sound, makes an
  asset of it and fills an instrument from a preset.
- Gate on atx: CHECK EXIT 0, 326 files, 5021 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 20:06:15 +02:00
adminandClaude Opus 5.5 2e4c0a09ea Open clips and frames from the asset panel by double click (#430)
feuerwerk / gate (push) Successful in 4m58s
Details
feuerwerk / suite (push) Failing after 19m37s
Details 
A double click on a clip row did not open its tab. The first click of
the double click had already selected the clip for the inspector, so
the second click selected the same thing again and the editor saw no
change. Frames of textures and pixel art had no rows at all.

- A double click now asks for the tab with openSelection, even when
  the selection is already right.
- Textures with frames and pixel art with more than one frame get a
  chevron and one row per frame, like sounds and videos. Which rows an
  asset has is decided in one place, src/core/model/assetParts.ts,
  used by the panel and the inspector link.
- A texture frame shows in the inspector on click and opens the
  texture with the frame selected on double click.
- A pixel art frame selects the pixel art on click, so an open tab
  paints on that frame. A double click opens the tab on the frame.
- A frame can be dragged onto a composition track like a video clip.
  The item carries the sprite of the whole image, the frame becomes
  the texture of its object, and it is 3 s long. The agent can do the
  same with add_composition_item.
- A texture label that names a frame now points at the frame row.

Found on the way: the e2e check from #425 only double clicked the
texture label, where the first click is a ping, so the bug never
showed there.

Tests:
- Unit: asset-parts (new, 6), composition-sprites and references
  extended, related tests 303/303, agent tests 21/21
- 5 mutations, all red in the matching test
- e2e 69 (atx): ALL PASSED, 26 checks, 69t new
- Gate (atx): CHECK EXIT 0, 325 files, 5017 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 19:30:29 +02:00
adminandClaude Opus 5.5 37779a2070 Fold object groups in the timeline, with summary keys (#414)
feuerwerk / gate (push) Successful in 5m21s
Details
feuerwerk / suite (push) Successful in 20m7s
Details 
Long animations with many objects fill the timeline with rows. A
group per object can now fold away, and its header row still shows
where the group has keys, so they stay reachable while folded.

- A click on an object header or its triangle folds the group. The
  groups of the objects below it fold along.
- Which groups fold along follows the real parent in the hierarchy,
  not the indent. The root shows at the level of its children, so
  folding it hides only its own rows.
- Every header shows a summary diamond on each frame where a row of
  the group or of a group below has a key. Click selects all those
  keys, drag moves them, Ctrl+drag duplicates, a band catches them
  and Delete removes them, each through the existing commands.
- The fold state is a view of the session per animation. It is no
  undo step and is not saved.
- Selecting a row without keys inside a folded group unfolds it, so
  a fresh row from + Row does not stay hidden.
- The fold rules live in the new timelineFold.ts. keyAt became
  keysAt, which returns every key behind a summary diamond.

Found on the way: the triangle moved the object name to x 22 while
its rows stayed at x 20, left of the name (#408 check in e2e 25).
Rows now sit one step behind the name, as in the hierarchy.

Tests:
- Unit animation + timeline-view 114/114, 6 new
- 6 counter-checks, all red in the matching test
- e2e 25 (atx) ALL PASSED, 41 checks, 4 new (25k)
- Gate (atx) CHECK EXIT 0, 324 files, 5008 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 19:01:30 +02:00
adminandClaude Opus 5.5 19902a9f02 Keep groups, comments and silent music in view for the agent (#429)
feuerwerk / gate (push) Successful in 4m47s
Details
feuerwerk / suite (push) Successful in 19m5s
Details 
A live test of the music areas built a pattern, a song and a
composition that stayed silent: no instrument at first, and later
notes outside the C4 to B4 range of a new Notes node. The agent got
no hint of either. The same test showed that new nodes left groups
behind, that a comment could end up under other nodes, and that an
end node like Output stayed left of the devices put in front of it.

- get_asset lists under silent what of a pattern, song or
  composition is not heard. That is a missing instrument, an
  instrument that sounds no note, or notes outside what its Notes
  nodes take in, in a song after the transpose.
- soundingNotes moves from the pattern tab to
  src/core/instrument/compile.ts. The tab and the agent use the
  same rule.
- When new nodes go between two that have no room, everything from
  the fed node on moves right together. This holds for Arrange with
  a selection too, so no link runs backwards.
- A new node linked to a node in a group joins that group, and the
  group grows in the same undo step. A group that would take in a
  foreign node stays as it is.
- Arrange takes groups and comments along: a group whose nodes all
  moved the same keeps its size, otherwise it is fitted anew. The
  rules live in the new src/editor/graph/annotationFollow.ts.
- add_comment and fit_group put a comment below its nodes when a
  node or a group title is in the way above. lint_graph counts
  covered comments as coveredNotes.
- add_notes, get_asset, lint_graph and the relay instructions name
  the default range and how to mend a covered comment.

Tests:
- graph-follow (new, 12), agent-edit, agent-music (11) and the
  other agent, graph, pattern and relay tests green
- 10 deliberate breakages, each one red in its own test
- e2e 71: 25/25
- Gate: CHECK EXIT 0, 324 files, 5002 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 18:27:57 +02:00
adminandClaude Opus 5.5 10f5e3ba8f Let the agent write music and build compositions (#429)
feuerwerk / gate (push) Successful in 5m17s
Details
feuerwerk / suite (push) Successful in 18m54s
Details 
The agent could already draw, animate and wire graphs, but songs,
patterns, sound clips and compositions were out of its reach. This
last step of #429 adds the two missing catalog areas, so the agent
can now use 168 of the editor's commands.

- Area music (15 commands): notes, pattern and song loops, song
  tracks and items, sends, and clips of sounds and videos.
- Area composition (9 commands): tracks, items for objects, images
  and audio, changes with fades, volume and bus, removal, and
  animation clips.
- A pitch is a MIDI number or a name like C4 or F#3. A note past the
  end of its pattern is refused.
- Song and master routing stay graphs. The agent links them with
  connect, and set_send sets level and tap. get_graph now shows the
  send of every link.
- Lengths follow the same rules as dragging in the tab. A sound has
  a length only after the editor loaded it. get_asset reports it as
  seconds, and without it the agent has to give a length.
- Moving a composition item takes its clips along, as dragging does.
  A clip that would then overlap another one is refused. Fades are
  only allowed on items with sound.
- get_asset reads notes back with their names and gives the length
  of a pattern in ticks.

Found while testing live: two song tracks in the same column made
lint_graph report a backwards link. arrange_graph cleared it, as the
agent instructions ask.

Tests:
- agent-music (new, 9 tests) and the other agent and relay tests:
  109/109.
- 9 mutations of the new code, each one red in its own test.
- e2e 71 on melektaus-atx: 25/25. New check: notes are read back by
  name, and a composition with a pattern and a sprite opens in its
  tab.
- Gate on melektaus-atx: CHECK EXIT 0, 323 files, 4987 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 17:32:44 +02:00
adminandClaude Opus 5.5 0684e76dc5 Let the agent draw pixel art and vector art (#429)
feuerwerk / gate (push) Successful in 5m46s
Details
feuerwerk / suite (push) Successful in 19m14s
Details 
Stage 5c of the MCP work. The agent could place sprites and edit
graphs, but it could not draw: pixel art, palettes and vector art had
no commands in its catalog. Two new areas bring 29 commands, so the
catalog now holds 144.

- pixelart (13): paint, layers (add, copy, remove, change, move,
  merge down), frames (add, also as a copy, remove, move), canvas size
  with an anchor, palette colors set and removed.
- paint_pixels edits one layer on one frame. Its steps run in order:
  rows (a small picture as characters, a space keeps the pixel),
  pixels, line, rect, ellipse, fill, clear. An index the palette does
  not have, a pixel off the canvas and an unknown character are
  refused.
- vectorart (16): shapes from rings or as rect and ellipse, outline,
  style (fill, opacity, gradient or image fill, stroke), move, scale,
  rotate and flip, copy, remove, combine, rename, order, move into
  groups, group, layers.
- get_asset gives the palette of pixel art as colors. Up to 16384
  pixels over all layers and frames each layer comes as rows, above
  that only the number of painted pixels. Palette and vector colors
  come as #rrggbb.
- Merge down, canvas size and removing a color are now builders in
  pixelActions.ts. The panel and the agent share the same code.
- Color and point helpers moved into catalog/shared.ts, and graphs.ts
  uses them too.

Found while testing:

- The ellipse tests only checked the four extreme points, which stay
  right even with wrong handle lengths. The test now checks that every
  point of the drawn curve lies on the ellipse.

Tests:

- agent-drawing (new, 9) and the other agent, pixel and relay tests:
  133/133.
- 7 mutations, all red in the matching test.
- e2e 71 on melektaus-atx: 24/24. New check: the agent's pixel art
  opens in its tab, reads back as rows, and its cel is on the server
  after saving (status 200).
- Live in a test project: an 8x8 sprite and a copied second frame
  read back as painted, a rotated square and a radial gradient disc
  with their corners where computed.
- Gate on melektaus-atx: CHECK EXIT 0, 322 files, 4978 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 16:59:08 +02:00
adminandClaude Opus 5.5 d01e2b832e Let the agent edit animations, frames, tiles and tilemaps (#429)
feuerwerk / gate (push) Successful in 5m14s
Details
feuerwerk / suite (push) Successful in 18m43s
Details 
After stage 5a the agent could build behaviors and scenes, but not
animate an object, cut a texture into frames or paint a tilemap.
Stage 5b adds three areas with 30 commands, 115 in all.

- animation (12): set the root, add, retarget and make rows relative,
  set, move, copy and delete keys, easing, insert or remove frames,
  and curve points.
- textures (8): add, change and delete frames, slice a grid, nine
  slice, sorting line and collision outline.
- tiles (10): the images of a tile, painting tilemaps, layers (add,
  copy, move, change) and the border of a placed tilemap.
- get_asset names the targets of an animation with their animatable
  properties. A tilemap up to 400 cells shows its cells, a bigger one
  only their count.
- After such a call the editor opens the tab of the animation, curve,
  texture, tile or tilemap. A frame edit selects the frame there.
- One paint step takes rectangles and single cells. Rectangles go
  first, so a cell can cut a hole into a rectangle. Cells outside the
  grid, ids that are no tile and more than 20 000 cells are refused.
  The grid is the tilemap's largest, a grown placement included.

Left out on purpose: Generate collision, because it reads the pixels
of the texture, and pasting keys, which goes through the clipboard.

Found on the way:
- Nine slice without fill now keeps the fill the image has. The
  model's default is repeat, not stretch as first described.
- Painting ran the other way round, rectangles overwrote cells.

Tests:
- agent-art (new, 11) and the other agent tests green, 64/64.
  5 bends of the code, all red.
- e2e 71 on atx: 23/23, the new check opens the agent's animation in
  its tab and reads its targets.
- Gate on atx: 321 files, 4969 tests.
- Live on a test project: a Spin animation of 360 degrees in 30
  frames matched 12 degrees per frame to 0.01 frames in the game.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 16:24:27 +02:00
adminandClaude Opus 5.5 0d8fd6ff52 Let the agent use the graph, scene and project commands (#429)
feuerwerk / gate (push) Successful in 5m10s
Details
feuerwerk / suite (push) Failing after 18m49s
Details 
The agent knew 23 of the editor's commands. It could not point a
Variables node at a variable, group nodes, edit ports, transitions,
containers, colliders or the project settings. Stage 5a brings the
catalog to 85 commands in five areas and keeps every answer short.

- The catalog lists names and descriptions only. With a query (an
  area or command names) it adds the arguments. All schemas at once
  would pass the 60 000 character limit of one answer.
- add_port points a Variables node at a variable. The port takes the
  variable's kind, and an unknown name is refused with the list of
  the ones that exist.
- add_group and add_comment take nodes instead of coordinates. They
  follow the nodes after the arrange step, in the same undo step.
- The rule for which list a port picks from moved into portChoices.ts.
  Inspector and agent now share it.
- The frame of a group around nodes moved into annotationDrag.ts, so
  the button and the agent get the same frame.
- list_assets returns the project settings. get_asset shows the
  content of placed containers with the ids overrides need. get_graph
  shows declared ports and frame colors.
- A new node is placed at the height of its port, in the free spot
  whose links cross the fewest others. A source goes left of the node
  it feeds. Before, new nodes always went right of the rest.
- Arrange breaks a tie in crossings by putting the flow on top, and
  exec links pull harder when a column is sorted.
- A comment over a group keeps clear of the group's title bar.
- The agent's instructions and lint_graph ask to look at crossings
  too and to call arrange_graph when there are more than needed.

Left out on purpose: file assets (no upload), paste from the
clipboard, internal wrapper commands and switching plugins on.

Found on the way:
- Variable commands on a Struct asset did not run through
  WithStructTypes. New fields did not reach variables of that type.
- A query "variables" also matched set_variables_scope. A word that
  names an area now means only that area.

Tests:
- agent-catalog (new, 10), agent-edit, graph-arrange (3 new) and
  relay green. 11 bends of the code, all red.
- e2e 71 on atx: 22/22, the new check sees port speed and a group
  around all 4 drawn nodes.
- Live on a test project: a speed variable of 300 moved the player
  to x 150.03 in 0.5 s. A re-placed Variables node landed below Axis
  input with 4 crossings instead of 7.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 16:01:15 +02:00
adminandClaude Opus 5.5 1943539a7a Let the agent play the game and read its state through MCP (#429)
feuerwerk / gate (push) Successful in 5m7s
Details
feuerwerk / suite (push) Successful in 18m44s
Details 
An agent could build a game object but not see whether it works. It
had to ask the user to press the keys. Now it can play the game in the
editor itself, send keys and mouse input and read what happened.

- play_input sends steps into the running game: hold or tap a key,
  keep it down or let it go, click, press, release or move the mouse,
  turn the wheel, wait. Holding and waiting count game seconds.
- A mouse target is a point in world units or an object by id or
  name. A click on an object reports whether it landed on it.
- The answer names the game time used, the keys still held, the mouse
  and the new console lines.
- One tool instead of press_keys and pointer from the plan. A step
  list can then mix both, like holding Shift during a click.
- get_runtime reads the running game: objects with id, type and
  position, the own variables of their behaviors, the Variables
  assets, the camera, keys, mouse and the console. Named objects come
  with all their properties.
- A hidden editor tab gets no frames. play_input says so at once. If
  game time stands still for 25 s it stops, names the step and lets
  go of every key it held.
- get_view and play say when the scene has no active camera. The
  first agent test missed that and only saw it on a screenshot.
- The relay instructions tell the agent to test what it built.

Found in the live test on a real project:

- A click on an object by its name reported hit false. The hit test
  looked the target up by id only.
- get_runtime listed the built-in switches of a behavior (enabled,
  runInEditor) as its variables.

Tests:
- agent-play (new), agent-edit, agent-view, relay green.
- 5 counter-checks red: hold counts one frame, key never released,
  click ignores the object, hidden tab not refused, no variable
  filter.
- e2e 71: 21/21. The agent holds ArrowLeft 0.5 s (x -120 to -270)
  and clicks the object (hit true).
- Gate: CHECK EXIT 0, 319 files, 4944 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 15:01:32 +02:00
adminandClaude Opus 5.5 f296f427ef Measure the stream's pause before the forced garbage collection
feuerwerk / gate (push) Successful in 4m38s
Details
feuerwerk / suite (push) Successful in 18m39s
Details 
e2e 34 "the pause holds the stream" was red in CI tasks 712, 719 and
721. The pause was not at fault. Right before it the step forces a
full garbage collection for its memory check. On the CI runner that
held the page for seconds, and the player ran dry before the pause was
even clicked: 0.02 s and 0.05 s were left ahead in 719 and 721, against
about 2 s in the green CI runs and 4 s on melektaus-atx.

- The worklet counts a late sample only once it arrives and reports
  every 4800 samples read. So the underrun from the collection showed
  up after the resume and looked like samples the pause had lost.
- The pause is now measured first, on a stream held at least a second
  ahead, with half a second for the worklet to report. The collection
  and the renderer's size follow while the game still plays.
- The check judges what the pause adds: as many skipped samples after
  the resume as before the pause.

Tests:
- e2e 34 on melektaus-atx with a 5 s busy loop on the page where the
  collection was: 12375 samples skipped before the pause, the same
  after it. The old condition, none skipped, fails on these values.
  The new one passes.
  ALL PASSED, 148 checks
- Lint and Prettier on the module are clean

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 13:13:00 +02:00
adminandClaude Opus 5.5 97e0d93168 Let the editor follow the agent's edits on screen (#429)
feuerwerk / gate (push) Successful in 4m45s
Details
feuerwerk / suite (push) Failing after 18m0s
Details 
The user asked to see what the agent is working on through MCP. Until
now the agent had to call open for that, and mostly did not.

- After run_commands the editor shows what the call worked on last. A
  graph opens its tab with the touched nodes selected, deleted ones
  left out. A scene or container shows with its objects selected. Any
  other asset shows in the inspector.
- Objects in another scene switch the scene inside the same undo step.
  A switch of its own would be its own step, and Ctrl+Z would take back
  the switch before the agent's edit.
- A graph tab that is open already moves its camera after the nodes.
  It stays put while they are in view, moves only as far as needed with
  the zoom kept, and frames them like Fit only when they do not fit.
  This is revealBoxes in framing.ts and the state's reveal event.
- A tab that has just opened frames itself first and moves after that.
- The account menu has "Follow agent" below "Agent", on by default and
  kept in the browser. While the game plays the editor never follows.
- The run_commands description and the relay's instructions tell the
  agent about it.

Tests:
- agent-edit.test.ts: 3 new tests, each red with its bent-back fix
  (scene switch outside the undo step, deleted nodes selected, switch
  ignored)
- graph-framing.test.ts 29/29, 6 of them new
- e2e 71 on melektaus-atx: ALL PASSED, 20 checks. 71g4 adds a node
  3000 px to the right of an open tab: it ends 24 px inside the edge,
  zoom still 1
- Tried live on a test project through a real MCP session: the tab
  opened, the nodes and the object were selected. With the old build
  a far node stayed out of view, after the reload the camera went there
- Gate on melektaus-atx: CHECK EXIT 0, 318 files, 4931 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 12:48:35 +02:00
adminandClaude Opus 5.5 732989a452 Keep Fit inside the song tab's header again (#429) 
Suite 719 for 0dd671e was red in e2e 65. The Arrange button that came
with b9d1602 sits in every graph tab's header, the song's too. There it
pushed Bake and Fit past the right edge of the tab. The Fit click hit
nothing, the last track stayed below the edge, and its head menu with
"Add track below" never opened.

- The song tab shows Arrange with its routing only. On the timeline it
  was greyed out and of no use anyway.
- SnapToggle.setArrangeShown hides the button and lays its header out
  again, so the timeline's header is as wide as before #429.
- The magnet stays where it was, greyed out on the timeline, as e2e 65
  expects.

The other red check in suite 719, e2e 34 "the pause holds the stream",
is the wobble left open in 5b9d084 (task 712). Its message now shows
the stream behind before the pause already (skipped 12950), so the
pause did not lose its place. On melektaus-atx it passed.

Tests:
- snap-toggle.test.ts 6/6, the new test is red without the relayout
- e2e 65 on melektaus-atx: red before (same error as in CI), now
  ALL PASSED, 59 checks
- Gate on melektaus-atx: CHECK EXIT 0, 318 files, 4931 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 12:48:15 +02:00
adminandClaude Opus 5.5 0dd671ea26 Count each connected agent once in the editor badge (#429)
feuerwerk / gate (push) Successful in 5m17s
Details
feuerwerk / suite (push) Failing after 14m54s
Details 
The Agent badge showed 2 or 3 while one Claude Code session was
connected. The relay log against the open TCP connections in the
container showed two causes.

- A client may drop its event stream while the relay still asks the
  API about its token. The close was over before the relay listened
  for it, so the stream stayed in the list and counted for good. The
  relay now listens before the token check and keeps no stream that
  is already gone. It listens on the response, not the request.
- Claude Code opens three sessions whenever it connects and keeps a
  stream on one of them. It never ends the others with a DELETE. Any
  session counted for two minutes after its last request, so the
  badge stood at 3 after each start. Now a session counts while its
  stream is open or it called a tool in the last two minutes.
- The relay logs session start, stream open and close and each change
  of the count, with session ids cut to 8 characters.

One case is left: after a relay restart Claude Code sometimes repeats
its first call on a spare session. The badge then shows 2 for two
minutes and drops to 1 on its own.

Tests:
- relay.test.ts: 27/27, three new. A dropped stream counts out, a
  stream dropped during the token check never counts, and spare
  sessions without a tool call do not count.
- Counter-checks: without the early close listener the drop test
  turns red, with the old count two tests turn red.
- Live: the badge showed 2 with one open SSE connection before. After
  the fix it shows "Agent · 1" and stays there after a normal call.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 12:10:02 +02:00
adminandClaude Opus 5.5 b9d1602b94 Arrange graph nodes for the agent and with a button (#429) 
An agent that built a graph through MCP left every new node in one
row. In the first real test that put nine nodes on a line 2080 px
long, with the Owner's links running straight through all the others.
Placing nodes by hand cost nine move_node calls and guessed heights.
The agent now gives no coordinates, and the editor lays the nodes out.

- arrange.ts is a pure function. It puts the flow into columns from
  left to right and sets an Owner or Get right before the first node
  it feeds. It orders each column for the fewest crossings and lines
  nodes up by their linked ports without overlaps.
- Node sizes come from the drawn NodeView in the browser and from an
  estimate in unit tests. Both share the constants in nodeBox.ts.
- run_commands arranges the nodes that came without x and y when the
  call ends, inside the same undo step. Nodes that were there stay.
- New command arrange_graph and new tool lint_graph. The lint counts
  overlaps, crossings, links through foreign nodes, backward links
  and the longest links, with example ids.
- The relay instructions tell the agent not to give coordinates and
  to fetch far values with a second Get or Owner node.
- An Arrange button next to the snap magnet in all eight graph tabs.
  It arranges the selected nodes, or all of them, as one undo step.

Found while testing on a real project:

- lint_graph missed a crossing where two links swap rows exactly
  mirrored. Both curves then share a sample point in the middle, and
  only a crossing between two sample points was counted. Links that
  meet on a point count now (segmentsMeet).
- The Arrow-key graph with one Owner for four Moves cannot get below
  8 crossings. The Owner feeds object in row 2 of Move, between exec
  and dt of the key, so it always crosses one of them.

Tests:
- graph-arrange, agent-edit, snap-toggle: 33/33. The swapped-rows
  test was red before the fix (0 instead of 1).
- Four counter-checks that bend arrange or the edit plan, all red.
- e2e 71 on melektaus-atx: 18/18. Lint after the build shows 0
  overlaps, 0 links through nodes, 0 backward. Arrange takes a
  tangled graph from 3 backward links and 2 crossings to 0, and one
  Ctrl+Z restores it.
- Gate on melektaus-atx: CHECK EXIT 0, 318 files, 4917 tests,
  measured before the lint fix.
- Live through MCP on the user's project: lint_graph reports 8
  crossings after Arrange, matching the screenshot.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 12:09:52 +02:00
adminandClaude Opus 5.5 25af064249 Let an AI agent read and edit the open project through MCP (#429)
feuerwerk / gate (push) Successful in 4m40s
Details
feuerwerk / suite (push) Successful in 18m43s
Details 
Stage 2 of #429. Until now an agent could only look at the editor.
Now it reads the project and changes it through the same commands
the user's own edits run, so every change shows up live and one
Ctrl+Z takes back a whole agent call.

- Read tools: list_assets, get_asset, get_graph and catalog. The
  catalog lists the commands with their arguments, the nodes of a
  graph from the same palette the user sees, asset types and
  variable kinds.
- run_commands runs its steps as one undo step named "Agent: ...".
  A failing step rolls back the steps before it. The answer names
  the step and the reason the editor gave.
- A step can name what it made ("as") and later steps use "$name".
  Each step is built after the one before has run, so one call can
  make a behavior and fill it.
- 23 commands cover assets, variables, scene objects and graphs.
  Textures, fonts, sounds and videos need a file and are left out.
- open, select, play and undo/redo drive the editor.
- Deleting asks the user with the same text as their own delete.
  The tab sends "wait", and the relay then gives the call 2 minutes
  instead of 30 seconds.
- tools.json is the only source of the argument schemas. argCheck
  checks every call against them before anything runs.
- An open menu now names its first entries in get_view, instead of
  an empty title.
- The new host functions live in editorToolHost.ts, so Editor.ts
  does not grow further.

Found while testing: e2e 71 hung for 30 s in Page.navigate after a
failure. The beforeunload guard of an unsaved project held the page
and hid the real error. The module now reports the error first and
waits for the save before it navigates.

Tests:
- Gate on melektaus-atx: CHECK EXIT 0, 317 files, 4903 tests.
- agent-edit 14/14, agent-view 10/10, relay 24/24.
- e2e 71 16/16 on atx and against the live site. One call builds a
  sprite with an arrow-key behavior, play moves it from x 0 to -125,
  and one undo restores the assets exactly.
- Counter-checks turned red: no rollback, a partial undo, no delete
  question, a relay that ignores "wait".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 11:08:39 +02:00
adminandClaude Opus 5.5 48ec722660 Let an AI agent see the open editor through MCP (#429)
feuerwerk / gate (push) Successful in 5m41s
Details
feuerwerk / suite (push) Successful in 18m44s
Details 
Claude Code should be able to work inside the user's open editor tab.
This first stage connects the two and lets the agent look: what the
user sees and a picture of it. Changing the project comes in stage 2.

- An agent token per account opens only /mcp, no API route. The
  dialog "Agent tokens..." in the account menu creates it once, shows
  the ready "claude mcp add" line and revokes it. The server keeps
  only its sha256 (api/lib/AgentTokens.php).
- Wrong tokens count into the sign-in brake from #427. Changing the
  password, renaming or deleting an account ends its tokens.
- The relay (tools/mcp/) is a small Node service without packages,
  WebSocket server included. It runs in the container feuerwerk-mcp
  because PHP-FPM cannot hold connections. Whether a token or a tab
  ticket is valid is still decided by the PHP API, through two routes
  that answer only with the relay key.
- The tab joins over WebSocket on the same origin with the normal
  session, an Origin check and a one-time ticket. It connects only
  after "Agent" is switched on in the account menu. One tab per
  account is active, a new one takes over. Guests get nothing.
- Tools: get_view (project, tab, document, selection with names,
  camera, play state, open dialog) and screenshot. The resource
  feuerwerk://view can be subscribed. Claude Code does not act on
  that reliably, so a changed view is also noted on the next answer.
- A badge next to the account menu shows the connected agents. It
  counts only agents with an open stream or a request in the last
  two minutes.
- CI starts the relay in front of php -S (tools/ci/serve.sh), and
  e2e 71 runs the whole path in the browser.

Found on the live test: the badge showed four agents because old
sessions of revoked tokens were still counted.

Tests:
- Gate on atx: 316 files, 4888 tests
- relay protocol test with fake API and fake tabs: 23/23
- get_view, tools, socketUrl: 16/16
- PHP agents 29, rights 103 checks
- e2e 71 on atx 10/10, against the live site 10/10, no CSP violation
- real Claude Code against the live site: get_view answered
- ZAP against /mcp: nothing at High or Medium
- every new check fails when the guard it covers is taken out (6)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-18 10:39:34 +02:00
adminandClaude Opus 5.5 edef2b6323 Serve the editor from feuerwerk.nockal.com as well
feuerwerk / gate (push) Successful in 4m45s
Details
feuerwerk / suite (push) Successful in 19m52s
Details 
The subdomain proxies to the same folder, but the build had the path
/feuerwerk/ baked in. On feuerwerk.nockal.com the page loaded its
assets, the API and the data URLs from /feuerwerk/... and got 404. The
session cookie had Path=/feuerwerk/, so the browser would never have
sent it to /api/ there either.

- site.ts takes the base path from the folder of the open page:
  /feuerwerk/ on nockal.com, / on the subdomain. Release links point at
  the address you are working on.
- vite.config.ts builds with a relative base. The copy plugin rewrites
  the pages to ./dist/assets/ and stops the build if it finds nothing
  to rewrite. The dev server now runs at http://localhost:5175/.
- Auth.php sets the cookie without a Path, so it belongs to the API
  folder the browser sees. Sign-in and sign-out also clear the old
  /feuerwerk/ cookie, otherwise it would keep signing in after a
  logout. Existing sessions stay valid.
- README: the new dev server address.

Tests:
- curl on both addresses: pages, JS, CSS and API 200
- sign-in, session, sign-out on both addresses
- e2e 70 against both addresses: 13/13 each, no CSP violation
- tsc, lint, prettier clean, site and releases tests 14/14
- PHP tests 7 of 8 files green, videos.test.php fails only because
  the web-php-fpm container has no ffmpeg

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 08:40:18 +02:00
adminandClaude Opus 5.5 5b9d084dfe Keep a clip that starts later playing at its first frame 
The e2e 68 check "a clip that starts later" was red in 3 of the last
8 suites (CI tasks 697, 703, 712). Two causes sat on top of each other.

- The check measured too late. It started sampling after play(), and
  under CI load the clock was already at 0.05 to 0.22 s. It now starts
  before the click, like 68d does.
- Measured early, a real bug showed: the video held for one frame at
  its first picture. The first picture triggers a redraw, and the game
  takes its first step in the same frame. readSpeed got both calls with
  the same frame time and divided by 1 ms. The speed came out at 16.7x,
  which counted as a jump, so the player paused.
- readSpeed now measures a step without elapsed wall time against one
  frame at 60 fps. A real jump still counts as a jump.
- e2e 34 "the pause holds the stream" also failed in task 712. Its
  cause is still open. The failure message now names the state before
  the pause, so the next red run shows where the samples got lost.

Tests:
- video-follow.test.ts 18/18, the new test is red without the fix
- e2e 68 on melektaus-atx: ALL PASSED, 21 checks
- Gate on melektaus-atx: CHECK EXIT 0, 314 files, 4853 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 08:40:11 +02:00
adminandClaude Opus 5.5 0d7672a5af Play the guest's changed demo in e2e 70 (#428)
feuerwerk / gate (push) Successful in 4m34s
Details
feuerwerk / suite (push) Failing after 18m31s
Details 
The test plan of #428 asks a guest to change the demo and play it,
with no request that writes reaching the server. e2e 70 only changed
it.

- 70f now plays the changed demo and stops it before it counts the
  requests.

Tests: e2e 70 (atx) ALL PASSED, 13 checks, mode "play", no writes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 07:19:31 +02:00
adminandClaude Opus 5.5 925a39a276 Let guests try a demo project without an account (#428)
feuerwerk / gate (push) Successful in 4m51s
Details
feuerwerk / suite (push) Successful in 19m23s
Details 
The editor was only open to somebody with an account. A guest now picks
a demo project on the sign-in screen and gets the whole editor on a copy
held in the tab. The server writes nothing for a guest, so there is no
guest session, no guest folder and no quota to keep.

- Demo copies are frozen copies like a release, in data/demos/. An
  administrator makes, updates and removes one with "Demo copy..." in
  the project's main menu. GET demos and GET demos/{token} need no
  session. An empty project is just a demo that is empty.
- Demo material: an administrator offers an uploaded file with "Offer
  as demo" in the pickers (PATCH {demo}, data/demo-files.json). Every
  list carries it with demo: true, for accounts and guests alike.
  Signed out, a list holds the demo material alone, without the owner.
- An offered file is only an administrator's to delete. A file a demo
  copy names counts as used, like one a release names.
- The guest's project lives in MemoryProjectStore. Every store that
  would write refuses in the tab before a request is made, and the
  autosave is not attached at all.
- The main menu drops projects, snapshots and release for a guest. The
  pickers drop Upload and Delete, a dropped or pasted file is refused,
  and a recording is saved as a download.
- The toolbar says the work is not saved. The Account menu holds
  "Sign in..." instead of Logout and the password.
- A guest's demo never goes into the address: a reload brings the
  sign-in screen back, not the copy. Found by e2e 70.
- GET plugins needs no session any more, since a demo may switch one
  on. It only reads the server's own modules.

Tests:
- e2e 70 (atx): ALL PASSED, 13 checks. Counter-test with the guest
  autosave sent to the server: 70f and 70g red on the PUT.
- tests/core/guest.test.ts: 9 tests. PHP: demos 20, files 46,
  rights 80 checks (the matrix asks every new route).
- Gate (atx): CHECK EXIT 0, 314 files, 4852 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-16 06:58:30 +02:00
adminandClaude Opus 5.5 ce1f25f2eb Prune old sign-in failures by their current time (#427)
feuerwerk / gate (push) Successful in 5m12s
Details
feuerwerk / suite (push) Canceled after 4m52s
Details 
The CI gate of cfd6d7a was red in limits.test.php: "pruning drops an
address whose failures are all old". PHP 8.2 in the CI image still
answered the modification time it had cached before the test moved it
back, PHP 8.5 on stein2 did not.

- LoginThrottle::prune() clears the stat cache before it reads times.

Tests: limits.test.php in feuerwerk-ci (PHP 8.2.34) 21/21, red before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 23:12:10 +02:00
adminandClaude Opus 5.5 cfd6d7a8df Lock the server down before the editor goes public (#427)
feuerwerk / gate (push) Failing after 5m1s
Details
feuerwerk / suite (push) Canceled after 4m52s
Details 
The editor is about to be opened to strangers: a post on Reddit and a
guest access (#428). nginx served the whole workspace and ran every PHP
file in it, the API let anybody guess passwords as fast as they liked,
and one account could fill the disk.

- nginx (shared web stack, outside this repo, backup in
  /root/config-backup-20261015/) hands out only the two pages, dist/,
  public/brand/, the uploads by kind and ending, and api/index.php.
  Everything else under /feuerwerk/ is a 404.
- PHP behind it has display_errors off and no X-Powered-By. Plain HTTP
  forwards to HTTPS, the API refuses it.
- Headers: a Content-Security-Policy without inline code or eval,
  nosniff, frame-ancestors, HSTS, Referrer-, Permissions-, COOP, CORP.
- vite.config.ts writes the same policy into the built pages, so CI
  runs the whole suite under it. Pixi gets by without eval through
  pixi.js/unsafe-eval. The page CSS left its inline style for a file.
- The harness fails a module for anything the policy refuses, from
  Chrome's log and from the page's securitypolicyviolation event.
- LoginThrottle: five wrong passwords from one address within 15
  minutes, and it gets 429 until 15 minutes after the last of them.
  The address is the last X-Forwarded-For entry, the Brandmauer's own.
- Quota: 1 GB per account for uploads, projects and snapshots. Only a
  write that grows the account is refused (413). Admins have no limit.
- A write whose Sec-Fetch-Site says another site is refused (403).
- The session cookie carries Secure, nginx passes HTTPS on.
- showFailure puts the message in as text, not as HTML.
- Every PHP script outside api/ stops unless it runs on the command
  line.
- The video worker sees the checkout read-only and can only write
  data/videos. No capabilities, 2 GB and 256 processes at most.
- tests/php/rights.test.php runs the API under php -S on a scratch
  directory and asks every route as signed out, owner, stranger and
  admin. A route named in api/index.php but missing in the matrix
  fails it. limits.test.php covers the brake and the quota.

Findings:

- tmp/ was readable from outside, with 79 copies of project files.
  Closed on 15.10. at 20:02 with chmod 750. The log back to 11.10.
  shows only our own requests.
- Port 8080 is bound on 0.0.0.0, so the LAN could reach nginx past the
  Brandmauer. Left as it is: the web stack is not this project's.
- npm audit finds nothing in what is shipped. Dev tools (vitest,
  brace-expansion, source-map-js) have advisories, not touched here.

Tests: check 313 files / 4843 tests. PHP limits 21, rights 60,
accounts 56, snapshots 15, releases 20, files 46 (videos 50/51, no
ffmpeg in the web container). e2e on stein2, atx was off: 34+38+41
197/198, the one red check (a click in 41) passes with 41 alone,
27/27. 67 misses frames at 1080p on stein2 with and without this.
Path list from outside: 10 served, 26 refused. ZAP baseline against
the live site: nothing High or Medium under /feuerwerk/.
Counter-checks: rights.test.php without the cross-site guard, the
quota, the brake, the admin check of users, or with an undocumented
route fails each time. A policy without wasm-unsafe-eval fails e2e 41
with the harness's own CSP line.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 23:05:32 +02:00
adminandClaude Opus 5.5 a9e0b6d9fa Show the README without Gitea's Unicode warnings (#426)
feuerwerk / gate (push) Successful in 4m30s
Details
feuerwerk / suite (push) Failing after 19m58s
Details 
The repo's start page marked eight characters in the README and
offered an Escape button. All eight were en dashes (U+2013), which
Gitea flags as ambiguous because they look like a hyphen. The wiki
dropped them in #416, the README now follows the same rule.

- Prose: the dash became a colon, a comma or a full stop.
- Comments in the code blocks: a semicolon.

Tests:
- grep over README.md for en dash, em dash, times, minus and
  single guillemet finds nothing.
- prettier --check README.md is clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 08:35:13 +02:00
adminandClaude Opus 5.5 2f13f9a0b9 Keep a recording behind another tab from dropping its frames
feuerwerk / gate (push) Successful in 4m52s
Details
feuerwerk / suite (push) Successful in 19m5s
Details 
In CI run 699 check 68k (real time behind a tab) got 21 of 120 frames.
The beat ticked 100 times, but 83 frames were dropped as busy. Late in
a long browser session the GPU behind a hidden tab took up to 1.5 s to
finish a read back, so all three read-back buffers were still on their
way when the next frame was due. Locally on melektaus-atx this showed
only in the full run of modules 01 to 68, and not in every one.

- The render loop no longer draws the screen while the page is hidden.
  Nobody sees it there, and the drawing was competing with the read
  backs for the slow GPU.
- A recording of the editor copies the canvas, so it turns the drawing
  back on with `drawWhileHidden` while it runs.
- `PictureReader.collect` reads the oldest buffer at once when the page
  is hidden and no buffer is free, instead of dropping the frame. The
  main thread waits for the GPU there, which nobody sees either.
- More buffers are no fix: with 30 of them the frames piled up at the
  encoder instead (45 frames, 56 dropped).

Tests:
- render.test.ts: 18/18, the new test is red without the hidden check
- Full run of modules 01 to 68 on melektaus-atx with a counting probe:
  read at once only 18 frames, 63 dropped; both parts 106 frames,
  2 dropped
- e2e 67 and 68 on melektaus-atx without probe: 68k green, 97 frames,
  0 dropped. Red there only "a clip that starts later", the known
  timing wobble from task 697
- Gate on melektaus-atx: CHECK EXIT 0, 313 files, 4841 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 08:02:08 +02:00
adminandClaude Opus 5.5 406839baea Drag clips of videos and sounds straight onto a track (#425) 
A clip of a video could not be laid on a composition track at all, and
neither could a clip of a sound, although #418 and the wiki said so.
The only place a drag starts is the asset panel, and it listed assets,
not clips. The texture label of an item playing a video clip was dead
too, because the reference lookup did not know video clips.

- The asset panel gives a sound or video with clips a chevron of its
  own. It unfolds one row per clip and starts closed.
- A clip row drags like an asset. One click shows the clip in the
  inspector without leaving the tab (`EditorState.glance`). A double
  click opens its sound or video with the clip picked.
- A video clip dropped on a composition track becomes an item as long
  as the clip. It wears the sprite of its whole video, so a composition
  keeps one sprite per video, and the clip is a texture override on
  the object. Sprite, item and override are one undo step
  (`droppedPicture`).
- A sound clip becomes an audio item as long as the clip. The song tab
  takes sound clips on the same way, through `findSound`.
- `referencedTarget` lets a reference label link to the clip itself.
  One click pings the clip row, a double click opens the asset with the
  clip picked. The Plays row of an audio item reads "Sound / Clip".
- `clipOwner` in `sound.ts` finds the sound or video a clip id belongs
  to. `activateAssetOrClip` uses it for the double clicks.

Found on the way: the Blip of the e2e seed is only 0.25 s long, so a
clip of 0.2 to 0.7 s on it is cut to one frame. 69s takes the Drone
(0.75 s) instead.

Tests:
- tsc, eslint, Prettier clean
- Unit on stein2 (composition, references, sound, video, editor):
  94 files, 1238 tests green
- e2e 69 on melektaus-atx: ALL PASSED, 25 checks, 2 of them new (69s)
- Gate on melektaus-atx with the fix for 68k on top: CHECK EXIT 0,
  313 files, 4841 tests
- Counter checks, all red: no clip rows in e2e 69 (no drag starts),
  no clip override in the unit test, the link aimed at the asset
  instead of the clip
- Not covered by its own e2e check: sound clips on the song tab

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-15 08:01:04 +02:00
adminandClaude Opus 5.5 0701fe3856 Share the parts the sound devices and timelines repeated (#85)
feuerwerk / gate (push) Successful in 5m5s
Details
feuerwerk / suite (push) Failing after 18m39s
Details 
Round 12 of the clean-up. Duplicate code had grown from 33 clones
(249 lines) after round 11 to 140 clones (1263 lines), almost all of
it in the panels and faces of the sound FX devices (#379 to #401).
It is down to 57 clones (416 lines) now, and src is 816 lines
shorter.

- deviceParts.ts holds what the device panels built by hand: the
  head, rows of knobs, the screen style and the row layout for
  groups of knobs. Twelve panels use it. panel.knobs stays because
  the debug hooks read it.
- DeviceFace is the base of the twelve faces. It finds the node and
  does wired, preview, commit, set, hide, reload and audition.
  compressorWired lives there as paramWired.
- TextPool replaces ten copies of "make texts as needed, hide the
  rest" in the wave and video tabs, the rulers, the piano roll and
  the song and composition timelines. clipStrip.ts is the clip label
  strip the wave and video tabs share.
- core/util/bytes.ts has the concat that ogg, matroska and avcColour
  each wrote out.
- Dead code is gone: clear on the video players (never called since
  #402), posterFileOf and videoTimeAt (only tests called them) and
  easeOut (the same as glide(t, 3)). Values only used in their own
  file lose their export.
- Comments over 125 columns are wrapped, and comments that told the
  history ("as before", "used to") say what is, in src and in the
  e2e modules.
- Left alone on purpose: AnimationEditor and CompositionEditor share
  only the play button, and the curve panels are built differently.
- Chip now places its boxes through put, which rounds. They were
  whole numbers already. The three row layouts are byte-identical
  before and after.
- README lists the video and composition assets and the Videos and
  Compositions pages.

Tests:
- Gate (melektaus-atx): CHECK EXIT 0, 313 files, 4835 tests.
- e2e 25, 34, 64, 65, 66, 69 (melektaus-atx): ALL PASSED, 330 checks.
- TextPool without keeping its texts and framesToRoom without making
  room both go red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 20:40:05 +02:00
adminandClaude Opus 5.5 df6dbe97c5 Build only the meshes of the tiles a painted cell changes (#424)
feuerwerk / gate (push) Successful in 4m34s
Details
feuerwerk / suite (push) Failing after 18m22s
Details 
Since #348 a painted cell worked out only the cells around it, but the
layer's tiles still came as a new list. groupByTile grouped that list
afresh, so meshFor saw no group again and every mesh on the layer was
built anew. On 120x120 that was the last cost that grew with the map.

- A patched list of tiles remembers the list it came from and which
  entries went and came (tilesPatch). A cell that is drawn the same
  keeps its entry, and a patch that changes nothing drawn hands back
  the very same list.
- groupByTile and layerRows patch the groups of the list before
  (patchedGroups). An untouched group stays the same array, so its
  mesh is not built again. On a layer sorted by Y this holds for each
  row and for the groups in it.
- A tile new to the layer still works the whole layer out again, as
  since #348.
- Stage 2 of the issue (patching quads inside a mesh) is not needed.
  The tilemap geometry no longer shows in the CPU profile.

Measured with the #348 probe, a brush stroke on a copy of Garden of
Sacrifices on stein2:

- Meshes built per frame: 21.9 to 0.94 on 60x60, 40.7 to 1.22 on
  120x120.
- fps while painting: 49.4 to 51.1 on 60x60, 42.0 to 45.8 on 120x120.
- Painting now costs both sizes about 10 fps. The gap left between the
  sizes is drawing the larger map, which shows without painting too.

Tests:
- Gate on melektaus-atx green, 312 files, 4832 tests.
- The random walk from #348 also compares groups and rows with a fresh
  grouping after each of its 400 steps, and counts reused groups.
- A fixed test paints a cell and keeps every other group and row.
- Three counter-checks turn red: entries not removed from a group,
  rows sorted by their first cell, groupByTile ignoring the patch.
- e2e 39 + 40 on melektaus-atx: ALL PASSED, 107 checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 18:41:41 +02:00
adminandClaude Opus 5.5 88e1cf1c7d Work out only the cells around a painted one on a tilemap (#348) 
A painted cell worked out the tiles and blocking cells of the whole
map again, so painting got slower the bigger the map. One cell cost
16.8 ms on 60x60 and 54.6 ms on 120x120. Now it costs what is around
it: 3.5 ms and 4.0 ms, against 3.4 ms without painting.

- The brush, the paint command and its undo name the keys they
  changed. Each new version of the cells keeps them for the last 16
  versions (cellsChangedSince).
- The cell index, the tiles of a layer and the blocking cells patch
  only the changed cells and their eight neighbours. Everything else
  stays the same object as before.
- Both lists are in cell order now, so a patch is one walk that keeps
  every other entry.
- The rectangles are merged row by row. A patch joins again only the
  rows around it, one row further than the changed cells, and stacks
  the rest as they were.
- A writer that names no keys costs one walk over the cells. A
  replaced cell object (load, resize, edits at run time) is worked out
  in full as before.
- The tiles a map names come sorted by id. Two trigger tiles on one
  cell now keep that order, whichever way they were painted.
- resolveTilemap walked all tiles every frame, even without painting.
  It now does so only when the list changed.

Found on the way: deleting an outlined autotile can turn the one below
into a full box, which cuts a scaled box two rows further down. Merging
only the patched rows missed that. It is a test of its own now.

A brush stroke on the map doubled to 120x120 runs at 42 fps instead of
28, p95 of a frame 52 ms instead of 92 ms. What still grows with the
map is the mesh of a layer, which is built again on every paint.

Tests:
- tests/core/tilemap-patch.test.ts (new, 7): 400 random steps on two
  layers with every kind of tile, each compared with a fresh full run
- Unit, tilemap and viewport: 1284 green in 57 files
- e2e 39 + 40 (atx): ALL PASSED, 107 checks
- Gate (atx): 312 files, 4831 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 16:14:39 +02:00
adminandClaude Opus 5.5 8a49392d7f Keep the shifter check in e2e 65 from losing the clip's start (#415) 
The check that a track plays through one shifter on its bus asked for
the start of the clip only after the note was planned. The clip ends
where the note starts, and the note is planned just 0.126 s ahead. In
a slow browser the clip's voice was already gone by then, and
`__gm.sound.startedAt` gave `null`.

- Each reading takes the plan, both starts and the audio clock in one
  `evaluate`. The clock goes into the detail of the check.
- The start of the clip is kept from the first reading that has it.
  Readings come every 150 ms plus the round trip, which is longer than
  the 0.126 s window, so one reading alone would not be enough.
- The note's start is still read where the note is planned.

Tests:
- Prettier, eslint on 65-song.mjs: clean
- e2e 65 on melektaus-atx: ALL PASSED, 59 checks
- Bent check (wait until the clip is over, read the start as before):
  red as the only one of 59, `clipStart: null` at clock 21.943 after
  the clip ended at 21.828

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 15:33:17 +02:00
adminandClaude Opus 5.5 39c509a8e5 Keep a video item in a recording when the tab goes back and forth 
A recording made while switching between tabs had stretches without
the video item and without its sound, and in between the video stood
for several frames. Two things went wrong behind another tab.

- A player opened there got no first picture. It waited for
  requestVideoFrameCallback, which Chrome never calls behind a tab.
  A recording not in real time then waited 3 s per frame and took it
  without the video. Now the player takes its first picture as soon
  as it has data while the tab is hidden.
- The span of a picture was learnt from steps that skipped pictures.
  After a time behind the tab it learnt 0.133 s instead of 1/30 s,
  left out seeks within that span and never corrected it. Now it
  learns only from steps where the time sought moved at most half a
  step, so no picture can lie between them.
- Behind the tab a seek forgets which picture shows but keeps the
  span. A landing without a frame counts as the same picture only
  once the tab has been in front for 500 ms.

Found in the user's recording 7ea9146b: 5363 frames, all 1/60 s
apart, with the video missing at 10.63 s (8 frames), 43.17 s
(5 frames) and 46.93 s (28 frames), and the sound gone with it.

Tests:
- Unit tests/viewport: 377 green, 4 new or rebuilt. With the old
  rule and the first picture only by requestVideoFrameCallback both
  new tests go red.
- e2e 68 on melektaus-atx: ALL PASSED, 21 checks. New 68l: a video
  item that begins behind another tab has its colours exactly, and
  a tab switched every 350 ms gives colour changes at 0/30/60/90.
- One run in five with switches had a change one frame late, so the
  check allows one frame either way.
- Gate on melektaus-atx: green, 311 files, 4824 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 14:36:00 +02:00
adminandClaude Opus 5.5 b1173ba6bb Show how fast a recording not in real time goes as a speed 
The tempo of a recording not in real time read as how many times
slower than real time it was made. "5.1x real time" for a recording
that took five times as long as its video looked like it ran five
times faster.

- The log line at the end and the notice now give seconds of video per
  second it took, with two decimals: 0.20x for the example above.
- It may go above 1x. Nothing holds a recording not in real time back
  to the wall clock, and a cheap scene on a fast screen makes more
  frames a second than the video has.
- A recording of a single frame took 0 s and has no tempo line now,
  instead of a division by zero.

Tests:
- tsc, eslint, Prettier clean
- Gate on melektaus-atx: green, 311 files, 4821 tests
- No check reads the figure itself. 68j looks for "x real time, and
  has every frame", which still matches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 14:05:20 +02:00
adminandClaude Opus 5.5 abb8fa51d8 Keep a recording going behind another browser tab 
Chrome stops calling requestAnimationFrame for a hidden tab, and Pixi's
ticker hangs on it. The run, the drawing and the recording all stood
still as soon as the user switched to another tab. A plain run keeps
standing still there, so a forgotten game does not play on unheard.

- backgroundBeat.ts ticks the editor from a worker at the recording's
  fps while a recording goes and the tab is hidden, or the page has
  not drawn a frame of its own for 250 ms. Once the page draws again,
  the beat keeps out of the way.
- The worker looks every 4 ms whether a tick is due. A 16 ms interval
  came in bursts behind a busy page, and the recording threw away the
  second tick of each burst.
- backgroundTimers.ts and timerWorker.ts give the short waits of a
  recording timers in a worker. Chrome slows the page's own timers to
  once a second in a hidden tab. This covers the waits while saving
  and the wait for a video's picture after a seek.
- Behind a tab Chrome does not call requestVideoFrameCallback, but the
  picture is there at seeked. The wait there is 10 ms instead of
  150 ms while the tab is hidden.
- The beat runs for Start recording, the exported game and Start
  recording... in the main menu.

Tests:
- tsc, eslint, Prettier clean
- tests/viewport: 374 green, 3 new in background-beat.test.ts
- e2e 68k (3 checks) on melektaus-atx with the tab really hidden:
  not in real time all 120 frames, colour changes every 30 frames,
  the blip 8 ms from green; in real time 97 of 120 frames (73 with
  the tab in front)
- Counter-check without the beat: all three 68k checks red, 3 frames
  in 60 s not in real time, 7 frames in real time
- Gate on melektaus-atx: green, 311 files, 4821 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 13:56:56 +02:00
adminandClaude Opus 5.5 87b2ee9402 Record every frame with Real time off on Start recording (#423) 
At 4K and 60 fps the user's computer reaches about 39 fps in real time.
Reading each frame back from the GPU takes longer than a frame, and
neither reading back later nor the canvas route helped. With Real time
off the run moves on by one frame of the recording at a time, however
long the computer takes, so the file has every frame.

- Start recording has a new check Real time, on by default. Only the
  node records so, the editor recording stays in real time.
- SteadyPace in recording.ts is the pace as a pure class. A step is
  1/fps once the frame before is taken and the sound up to it is made,
  otherwise 0 and the run holds. A frame's time is its number.
- The sound is made apart, as decided in the issue. AudioSwitch routes
  what the run plays to the live library, or to the offline library of
  a SteadySound while such a recording lasts. That library has the same
  mix, buses, FX and limiter and is rendered up to each next frame.
- A driven video holds at every picture asked for, and the frame waits
  for it. Its sound is decoded from the MP4 and played in step through
  its bus.
- Two Chrome quirks measured and worked around. A held video hands
  over the picture of a seek before seeked, so the frame callback is
  asked for when the seek begins. A seek right onto where a picture
  begins shows that one or the one before, so such a seek lands half a
  millisecond further. Together 30 s became 10.7 s for 2 s of video.
- The notice tells how far the recording has got, and the line at the
  end how long it took.

Deviations and limits:

- Input is not ignored, it still reaches the game, which runs slower.
- At most ten minutes per recording. Chrome holds the whole buffer of
  the OfflineAudioContext while rendering, 230 MB for ten minutes.
- A free video plays on in real time and does not fit. A sound playing
  before the start stays at the speakers.

Found along the way:

- A check in e2e 60 was red since 77365f1. It expected one log line,
  the start line of the recording came on top. CI was off, so nobody
  saw it. The expectation is updated.

Tests:

- First gate run red: 1 test in node-catalog.test.ts still expected
  the settings of Start recording without realTime. Updated.
- Second gate run on melektaus-atx green: 310 files,
  4818 tests.
- e2e 66 + 68 all passed, 50 checks, 68j new with 3 checks. ffmpeg
  reads all 120 frames 1/60 s apart, the colour changes at frames
  30/60/90, the video's sound from the first frame, the blip 8 ms off.
- e2e 60 (20 checks) and 67 green.
- Counter-check without the pace: all three 68j checks red, 32 of 119
  frames, colour changes at 33/62/96, no sound. AudioSwitch's unit test
  goes red when the ids are mixed up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 12:57:15 +02:00
adminandClaude Opus 5.5 77365f17d3 Say when a recording starts and when it cannot keep up 
A 4K60 game view recording on the user's machine had 517 of 827
frames. The rest were skipped without a word: when the editor's tick
came late, FrameGate let the due slots pass, and only frames lost to a
busy encoder or read back counted as dropped. The file kept its time
but stuttered, and nothing said so.

- FrameGate counts the frames it took and the slots the recording's
  time has room for. The difference is skipped, next to dropped in
  __gm.recording().
- FramePace judges the missed frames over spans of 2 s. The first
  time more than 10 % are missed, the log warns once with the frames
  a second the recording really got.
- At the end, when more than 5 % are missed, a warning gives the
  frames of the whole file and what helps (30 fps or a smaller
  resolution). It comes after the upload's notice, which would cover
  it at once in the editor.
- The first second counts for neither warning. Without that, a short
  e2e recording warned about 3 frames lost while starting.
- A recording that starts logs an info line with its size, fps, codec
  and sound. TakeHost.log takes a level for it. The editor recording
  keeps the line to the console, its notice would be in the video.
- consoleLine is the one place with an eslint exception for
  console.info, so the console panel shows the line plain.

Found along the way: headless Chrome in WSL records the editor with
real stutter in e2e 67 (1280x800@60 27 %, 2304x1440@30 12-14 %). That
is the software encoder there, and only now shown.

Tests:
- Gate on melektaus-atx: green, 309 files, 4810 tests.
- Unit recording.test.ts: 36 green, 6 new for FrameGate.skipped and
  FramePace. With skipped stuck at 0 the test turns red.
- e2e 67 on melektaus-atx: ALL PASSED, 15 checks, run twice. The new
  67b-ii slows the editor to about 20 fps: a 640x360@60 recording
  says "about 19 of 60 fps" and saves with 95 of 192 frames.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 11:21:46 +02:00
adminandClaude Opus 5.5 4d0a696327 Keep composition sound in step and give compositions a bus of their own 
A composition's sound ran on the audio clock while its picture ran on
the frames. A frame counts at most 0.1 s, so after a hitch the sound
stayed ahead of the picture until the next stop. Songs in a trailer
drifted off the cut. The items of a composition also had no bus of
their own, and a video sprite could not pick where its sound goes.

- CompositionSound.follow compares the audio clock with the picture
  each frame. More than 40 ms apart, it starts the sound again where
  the picture is. This holds in the tab and in a run.
- A time scale other than 1 now restarts the sound again and again.
  It stays in time but comes in pieces.
- The sprite of a video item shows "Driven (by composition)" as a
  fixed Video clock row. Video time is hidden there, because the
  composition sets it every frame.
- Every composition has a bus in the mix and a node in the master
  routing. A new one goes to the master unless other compositions are
  linked elsewhere, apart from where the sounds go.
- Audio items pick From asset, To composition, Master or a bus. New
  items start with To composition. Items saved before keep From asset,
  so music already sent through a bus does not bypass it.
- Video sprites have a new row Video bus: From asset by default, the
  master or a bus. In a composition the default is the composition's
  bus, also for video sprites inside a container item.
- The chosen bus is part of the key of the video player, so a sprite
  with another bus gets a player of its own.

Found along the way:

- Six tests with full lists of sprite defaults expected the old rows
  and now list Video bus too.

Tests:
- tsc, eslint, Prettier clean
- Unit: 971 green in the 28 files touched
- npm run check on melektaus-atx: green, 309 files, 4804 tests
- e2e 68 (melektaus-atx): 13 checks green, 68i new. The sound item
  and the video item go into mix:cp68-cut, which goes to the master.
- e2e 69 (melektaus-atx): ALL PASSED, 23 checks, 69q and 69r new.
  After a hitch of 500 ms the sound restarted twice and then lay
  0.5 to 5 ms off the playhead.
- Counter-checks in unit tests: without the comparison the hitch test
  fails, without the bus in the preload the run test fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 10:31:16 +02:00
adminandClaude Opus 5.5 875c76026a Silence a free video once its sprite is destroyed (#420) 
A sprite whose Video clock is free played its video on after it was
destroyed, heard in a game, until the player was let go 2 to 4 s
later. A driven one was already held and quiet after 100 ms.

- The library holds and quiets a free player nothing asked for in
  HUSH_MS, as it did a driven one. That covers Destroy, a scene load,
  the end of a run and a sprite switched off.
- Asked for again before it is let go, it plays on from where it was
  held, heard only in a run. A hidden page leaves it alone, as the
  sweep does.
- A sprite only out of view keeps its free video playing and heard.
  Whether a video is heard should not hang on the camera. Before, it
  ran 2 to 4 s and was then let go. Now it plays on, which opens its
  player even where it was never in view.

Tests:
- Unit: video-library 1 new. A free player losing its wearer is held
  and quiet 200 ms later, stays so through a new run, plays on heard
  when asked again with the same element, and runs on on a hidden
  page.
- e2e 66 on melektaus-atx: ALL PASSED, 36 checks, 66g-ii new. Heard
  31 frames before Destroy, held and muted in every frame read after.
- Counter-probe without hushing free players: the unit test and only
  66g-ii fail, the player plays on heard (paused false, muted false).
- Gate on melektaus-atx: 309 files, 4797 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 09:49:12 +02:00
adminandClaude Opus 5.5 acd12326dc Fade sounds and videos in and out on the items of a composition 
A trailer needs its music and its clips to come in softly and leave
softly. Until now an item of a composition could only play at one
level, and a video item could not even be turned down on its own.

- Every item with sound gets a grip at its upper left and upper right
  corner. Dragging one inwards fades the item in or out. Dragging it
  back to the edge removes the fade. Lower down the same edge still
  trims as before.
- The fade is drawn as its curve, with the part above it darkened.
  The inspector shows Fade in and Fade out in seconds, each with the
  curves sound clips already know.
- The model keeps fadeIn and fadeOut in frames plus curveIn and
  curveOut on the item. Old projects load unchanged. An item shorter
  than its fades plays them cut to its length but keeps the setting.
- Sound, pattern and song items ramp the gain of their own bus. The
  scheduler plans the ramps on the audio clock next to the notes.
  Starting in the middle of a fade picks it up from there.
- Video items set the new sprite property Video volume each frame.
  It multiplies with the fade and is only shown for a driven video.
  The player glides to each new level over 15 ms so it does not click
  between frames.

Not covered: a container item that carries a video further down gets
no grips. Only a video on the item's own sprite counts.

Tests:
- tsc, eslint, Prettier clean.
- Unit tests for composition, music, song and sound (12 files): 269
  passed, 7 of them new.
- e2e 69 on melektaus-atx: ALL PASSED, 21 checks. 69p drags a grip to
  a fade in of 10 frames and hears the item bus rise from 0.127 over
  0.229 and 0.339 to 1 within one second.
- Counter-checks as unit tests: without the audio ramps, and without
  the level on the video sprite in the game, both go red.
- Gate on melektaus-atx: 309 files, 4796 tests passed, 8 more than
  at 77ac9d1. The first run failed 14 tests in 6 files that list the
  default sprite properties or the bus levels in full. They now
  expect videoVolume and the new level field of busLevels.
- Not tried in a browser: video fades, pattern and song items with a
  fade.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-14 09:05:36 +02:00
adminandClaude Opus 5.5 77ac9d1aaf Play sounds, patterns and songs on the tracks of a composition (#418)
feuerwerk / gate (push) Successful in 4m31s
Details
feuerwerk / suite (push) Successful in 18m21s
Details 
A trailer needs its music and effects on the same timeline as its
pictures. Until now a composition could only place objects, so sound
had to be started by hand from a behavior and drifted against the
picture.

- Audio items: a sound, a clip of a sound, a pattern or a song dragged
  from the asset panel onto a track becomes an item as long as one
  pass of it. It may share a track with object and video items and
  overlap them.
- Editing works as for other items: move, resize, trim on the left
  (moves the offset), fit to media, delete, copy and paste, box
  selection. Sounds draw their waveform, the colour tells the kind.
- The inspector shows what it plays, start, length, offset, volume
  (-60 to +12 dB) and bus. The bus is "From asset", the master or one
  of the buses, as asked for in the issue.
- An item longer than its material plays in rounds. Sounds and
  patterns start over, a song runs on in its loop. Loop marks on the
  item follow the same rule, a song's first mark sits at its loop end.
- compositionAudio.ts turns the items into sequences, each with its
  own tempo. CompositionSound plays them, in the tab and in a game.
- In the tab the items are heard from the playhead on, straight into
  the master like every tab. A scrub is silent. Playing a song or
  pattern in its own tab stops the composition, and the other way
  round.
- In a game they are planned on the audio clock for the moment the
  composition clock reaches them, with the scheduler's LEAD made up.
  While the clock stands (waiting for a video, paused, placement off)
  they are silent and resume at the right spot.

Found along the way:

- The asset panel never let a song out of its list, so a song could
  not be dragged anywhere. Song and scene tab reject it as they reject
  a pattern.
- The asset panel scrolled to the selected asset on every rebuild of
  its list, so folding a group far below the selection jumped back up.
  Only a new selection scrolls now.
- At the end of the list, folding a group pulled every row down by the
  rows that went away. ScrollContainer has a new opt-in holdOffset,
  set only in the asset panel, that keeps the offset until the user
  scrolls up.

Limits: a time scale other than 1 or a frame longer than 0.1 s moves
the picture and not the sound. A baked song plays its notes here, not
the bake.

Tests:
- tsc, eslint, Prettier clean. Gate on melektaus-atx green,
  309 files, 4788 tests (4772 plus 16 new).
- Unit: composition-audio.test.ts 13 new, scroll-container 3 new.
  Counter-checks: without catching up on load time or without the
  silence on pause the unit tests go red.
- e2e 68h: the sound is planned 1.2 ms after the composition reaches
  0.5 s. Without the LEAD make-up it is 54 ms late and red.
- e2e 04 and 69 ALL PASSED, 86 checks. 69o-ii drags a pattern and a
  song onto a track, 20 frames each.
- e2e 04: folding Sound at the end of the list keeps its header at
  y 234. With the old panel it jumps to 984 and the check is red.
- Not run in a browser: the loop marks of a song item, copy and paste
  of audio items, the bus choice.
- README and wiki count 2072 checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 21:28:36 +02:00
adminandClaude Opus 5.5 30b618d9a9 Show game view recordings in their true colours in other players 
A game view recording looked darker and its colours off in VLC, and
each round of recording a recording made it worse. Chrome's encoder,
fed RGB frames, wrote full-range H.264 and left the colour out of the
stream. Only our Matroska head said "full range", and VLC reads only
the stream. Chrome itself plays the converted MP4, which was right.

- The GPU now turns the game view into I420, limited range BT.709,
  before it is read back (PictureReader). The encoder gets the planes
  with that colour, as it does for a recording of the whole editor.
- Where the planes lie in the read-back picture is a plain function,
  i420Layout in recording.ts.
- Only 1.5 instead of 4 bytes a pixel come back from the GPU.
- avcColour.ts writes the colour into the SPS of the encoder's avcC as
  well, so a player that reads only the stream knows it. An SPS with a
  VUI only gets its colour block replaced. One without gets a VUI that
  holds the colour. An SPS it cannot read stays as the encoder made it.
- A first try sent the frames through a 2D canvas. That gave limited
  range too, but only 18 to 20 of 33 frames at 1080p30.

Found on the way:

- With the colour in the stream alone, VLC 3.0.8 still showed the
  full-range picture too dark. It ignores the range.
- The one old full-range original on the server got the same SPS fix
  without loss: the clusters are byte-equal, the decoded frames are
  equal (framemd5), backup in /root/config-backup-20261013/videos/.
- Older game view recordings stay full range. Fixing that in VLC would
  mean encoding them again.

Tests:
- tsc, eslint, Prettier clean.
- Gate on melektaus-atx: CHECK EXIT 0, 308 files, 4772 tests.
- avc-colour.test.ts new (9), expectations read back with ffmpeg
  trace_headers. Counter-check: the old bug in the no-VUI path turns
  it red.
- e2e 67: ALL PASSED, 12 checks. 67d-ii reads tv/bt709 from the file
  and from the bare H.264 stream, red with the old code.
- 1080p30 34 frames, 640x360@60 68 frames, none dropped (was 33/47).
- Checked by the user: a new recording looks right in VLC 3.0.8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 19:48:44 +02:00
adminandClaude Opus 5.5 31ce6480b2 Give video posters the colours the video shows (#411)
feuerwerk / gate (push) Successful in 4m30s
Details
feuerwerk / suite (push) Successful in 18m12s
Details 
The worker made the poster with a plain scale=320:-2. ffmpeg then
read every picture as BT.601 in the limited range, so a red of 255 in
a BT.709 video came out as 229 in the JPEG. The picker showed paler
posters than the video in the editor.

- VideoFormat::colour reads matrix and range of the converted MP4.
  When the file says nothing, it follows Chrome: BT.709 from 720 lines
  up, BT.601 below. That guess was measured in Chromium with a red
  H.264 video without colour tags.
- posterArgs turns the picture from that matrix into BT.601 in the
  full range, the way a JPEG is read. accurate_rnd and
  full_chroma_int keep dark greys from landing two steps too low.
- The search for the first moving video stream is now one helper,
  used by plan and colour.
- Posters made before this change stay as they are.

Tests:
- videos.test.php in the worker image: ALL PASSED, 90 checks. Three
  red videos go through the real worker: BT.709 tagged, 720p without
  tags and 360p in BT.601. Red comes out as 254.
- Counter-check with the old filter: 3 checks red, red is 229 again.
- php -l clean on all three files. The gate was not run, the change
  has no TypeScript.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 16:45:20 +02:00
adminandClaude Opus 5.5 ba68167704 Greet with the Feuerwerk logo on the sign-in screen
feuerwerk / gate (push) Successful in 5m8s
Details
feuerwerk / suite (push) Successful in 18m38s
Details 
The sign-in screen was a plain box with the name as a title. It now
shows the brand, so the first thing anyone sees is Feuerwerk itself.

- The stacked logo (name below the icon) sits above the card. It is
  drawn as vectors straight from public/brand/, so it stays sharp at
  any zoom. Its <title> is stripped first, Pixi warns about it.
- The card is a floating panel, 300 px wide, with more padding and a
  thin strip in the three colours of the logo along its top.
- The small "Feuerwerk" title in the card is gone, the logo says it.
- Logo and card sit a little above the middle of the screen.
- Labels, field names and error texts are unchanged, the tests look
  for them.

Tests:
- Gate on melektaus-atx: tsc, eslint, Prettier clean, 307 files,
  4762 tests green
- e2e 45 (login): ALL PASSED, 19 checks, run while the screen still
  had a glow behind the logo. The glow was taken out by hand after
  that, e2e 45 did not run again.
- Looked at in Chromium at 1280x800, empty and with a wrong password

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 15:59:35 +02:00
adminandClaude Opus 5.5 d4ff652293 Keep descenders of a text whole and frame it at its drawn size
feuerwerk / gate (push) Successful in 5m21s
Details
feuerwerk / suite (push) Successful in 18m31s
Details 
A text was drawn into a texture as tall as Pixi measures the string
"|ÉqÅM". Whatever reached further, like the "g" of Manrope (0.015 em
below the "q"), was cut off at the bottom. The blue frame around a
text was only estimated from the font size and the number of letters,
so it was far too wide for most texts.

- textPicture.ts takes the height of a line from the font itself
  (fontBoundingBoxAscent and fontBoundingBoxDescent, as a browser
  does). Pixi draws with a margin, and the picture is cut to that box
  afterwards. Where Pixi's own box is larger, Pixi's box wins.
- Both raster paths (linear and nearest) use the cut.
- The height no longer depends on the letters typed, so a text does
  not jump while it is edited.
- bounds.ts asks a measurer that textPicture.ts registers
  (setTextMeasure). Selection, picking in a running game, framing the
  camera, the drop preview and culling all get the measured size.
  The footprint functions take the font file for this.
- Unit tests in node have no canvas and keep the estimate
  (estimatedTextSize).
- Texts get taller: Source Sans 3 at 48 px is 68 instead of 55 px,
  Manrope is 1.366 em instead of 1.25 em. The glyphs stay in place,
  the centre moves by less than half a pixel. A shader on a text
  gets a taller UV rectangle.

Tests:
- Gate on melektaus-atx green: 307 files, 4762 tests.
- 5 new unit tests for textBox, textCut and the measurer hook.
- e2e 30: 26 checks, 30f-iii new. The texture is 68 high, the font
  box is 68, the frame equals the drawn picture (237x68).
- Counter test: with Pixi's old box the texture is 55 high and
  30f-iii is red.
- Not checked in the browser: the Manrope caption of the user and
  texts with several lines.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 15:18:59 +02:00
adminandClaude Opus 5.5 9445f39c8d Space the letters of a text object closer or wider apart 
The Feuerwerk wordmark is set with -0.03em tracking. Text in the
engine had no letter spacing, so the wordmark came out about 3 % wider
than the logo.

- Text has a new property Letter spacing, default 0, in the same unit
  as the font size. It goes between each two characters, so ten
  characters get it nine times. Negative values pull them together.
- TextPicture hands the value to Pixi's text style. Both rasters, the
  smooth and the sharp one, use it, and it is part of the raster key.
- The estimated text width in bounds.ts adds the spacing too, so
  selection, picking and "Add collider" fit the drawn text.
- Like every number property it can be animated and set from a graph.
  That is not tried in the browser.

Tests:
- New unit test for the footprint with spacing. 102 tests in the
  three text files green.
- e2e 30f-ii measures the width of "Handgloves" on screen: 237 without
  spacing, 282 at +5 (expected +45) and 210 at -3 (expected -27).
  e2e 30 on melektaus-atx: ALL PASSED, 25 checks.
- Gate on melektaus-atx: CHECK EXIT 0, 307 files, 4757 tests.
- No counter-check run for 30f-ii.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 14:46:38 +02:00
adminandClaude Opus 5.5 2d3b957551 Add the Feuerwerk logo files and a favicon for the editor 
The project has a name and a logo now, but neither the editor tab nor
the repo carried it. The logo existed only as a design mockup.

- public/brand/ holds the logo in three layouts: the icon alone, the
  wordmark to the right and the wordmark below. Each comes in a light
  and a dark variant, with a transparent background.
- The geometry is taken from the mockup. Strokes are rebuilt as filled
  shapes. The wordmark is Manrope Bold converted to paths, with the
  font's kerning and -0.03em tracking, so no installed font is needed.
- The favicon is the dark icon on a rounded #15171B tile, as SVG plus
  favicon.ico (16, 32, 48 px) and a 180 px apple-touch-icon. Only the
  editor page links it. play.html shows the games and stays without.
- nginx serves the folder as /feuerwerk/public/brand/. Vite has no
  publicDir, so the files are not copied into dist/.

Findings:
- At 16 px the strokes get thinner than one pixel and the icon blurs.
  From 32 px on it is clean.

Tests:
- Rendered in Chromium at 3x against the mockup: the icon matches to
  7 of 255 per channel, wordmark edges sit within 1 device pixel.
- Prettier on web/index.html clean. No full gate, the change has no
  code in it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 14:23:54 +02:00
adminandClaude Opus 5.5 6097307b6d Move repo, CI and the video worker over to the name Feuerwerk (#422)
feuerwerk / gate (push) Successful in 4m30s
Details
feuerwerk / suite (push) Successful in 18m38s
Details 
Stage 3 of the rename. The repo now lives at meleki/feuerwerk, the
runners carry feuerwerk labels and the WSL distro on melektaus-atx is
called feuerwerk-ci. The code has to follow, or CI finds no runner and
the suite no longer picks the GPU path.

- ci.yml: workflow name feuerwerk, jobs on feuerwerk-ci (gate, stein2)
  and feuerwerk-wsl (suite, melektaus-atx).
- GAME_MAKER_* variables are now FEUERWERK_*. This includes
  FEUERWERK_BROWSER=wsl, which the runner config on melektaus-atx sets.
- The video worker runs as container feuerwerk-video with the image
  feuerwerk-video and works on /srv/feuerwerk.
- e2e.sh starts the unit feuerwerk-e2e, the CI image keeps its paths
  under the new name.
- README and repo links point to git.nockal.com/meleki/feuerwerk. The
  old URLs still redirect with 301.

Both runners keep their old labels until the first CI run with this
ci.yml is through. Then they go, together with the image tag
game-maker-ci:latest and GAME_MAKER_BROWSER in the atx config.

Tests:
- Gate on melektaus-atx through the renamed distro: CHECK EXIT 0,
  307 files, 4756 tests.
- e2e 60 on melektaus-atx: ALL PASSED, 20 checks. Renderer is the
  RTX 4070 through D3D12, so FEUERWERK_BROWSER reaches the GPU path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 13:42:52 +02:00
adminandClaude Opus 5.5 043c224cc3 Rename Game Maker to Feuerwerk and serve it at /feuerwerk/ (#422) 
The editor gets a name of its own that fits beside Brandmauer,
Glutspeicher and Feuerwehr. This is stage 1, 2 and 4 of #422: the
name, the URL and the stored state in the browser. Repository, CI
runners, containers and the GAME_MAKER_* variables follow in stage 3.

- All paths come from SITE_PATH in src/core/site.ts: the API endpoint,
  the data folders, the play page and the base in vite.config.ts. A
  later move is one line instead of fifteen.
- The window title, the sign-in screen, the playground and the README
  say Feuerwerk. The package, __feuerwerkPlay, the take worklet and the
  writing app in Ogg and Matroska follow.
- The session cookie is now feuerwerk_session, so everyone signs in
  once more.
- localStorage keys are now feuerwerk.*. moveOldStoredKeys copies the
  old game-maker.* keys over on the first start, so settings, panel
  sizes and save games survive.
- tools/ci/serve.sh serves the build under /feuerwerk/. The variable
  GAME_MAKER_BASE_URL keeps its name until stage 3.

Outside the repository, nockal.com/feuerwerk/ is a second symlink to
the same tree. nginx sends the bare /game-maker/ folder there with a
301. Everything else under /game-maker/ is still served directly, so
shared play links and the API keep working.

Finding: when nginx redirects a path without a trailing slash or with
a query, the Brandmauer answers with its own 307 to the same URL plus
a slash, placed after the query. A play link would lose its token that
way, which is why only the bare folder is redirected.

Tests:
- Gate on melektaus-atx: CHECK EXIT 0, 307 files, 4756 tests
  (4 new in site.test.ts).
- Counter-check: when the migration misses the old prefix, 3 of the
  4 new tests fail.
- e2e 60 under /feuerwerk/: ALL PASSED, 20 checks.
- From outside: /feuerwerk/ 200, /game-maker/ 301, an old play
  link 200.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 13:27:43 +02:00
adminandClaude Opus 5.5 f92d69d902 Rebuild the Acid node after the acid voice of nodeseq (#421)
game-maker / gate (push) Successful in 4m23s
Details
game-maker / suite (push) Failing after 18m20s
Details 
The first Acid hardly squelched. Its tanh ate the resonance, and its
sweep fell evenly in octaves. The user pointed at the acid tone of
nodeseq.com/studio, so the node now follows how that voice is built
from Web Audio parts.

- The filter is a two-pole 12 dB low-pass with Q in dB up to 17.5,
  like a BiquadFilterNode. The ladder and its tanh are gone.
- One ADSR drives both the volume and the filter. The sweep falls
  back in Hz, fast from the top and long at the bottom.
- Every note glides from the one before. A new note cuts the last
  one off, and the node plays its release itself.
- Knobs: cutoff, reso, env, attack, decay, sustain, release and
  glide. The defaults are the acid preset of nodeseq.
- Accent, slide, drive and tuning are dropped. Old projects were not
  kept, the user had only placed the node for a try.
- A hook __gm.acid.compare renders a line offline through our
  worklet and through the nodeseq graph, for e2e 34.

Found on the way: in Chrome an oscillator that starts inside a
render block runs its first samples at 440 Hz. The comparison evens
this out.

Tests:
- tsc, eslint, Prettier clean
- Unit: acid.test.ts rewritten with 12 tests. chip-presets and
  sound-library follow the new node.
- e2e 34 on melektaus-atx: ALL PASSED, 148 checks. Against nodeseq
  the loudness per 10 ms window is at most 0.14 dB apart, and 95 to
  97 % of the windows are equally bright.
- With a linear Q instead of Q in dB the comparison is red.
- Gate on melektaus-atx: green, 306 files, 4752 tests. That is
  4754 less the 14 old and plus the 12 new Acid tests.
- Not checked: the sound by ear, the master chain of nodeseq, the
  keyboard and a bake with the new node.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 12:39:01 +02:00
adminandClaude Opus 5.5 d844f2734a Add an Acid instrument node for TB-303 style bass lines (#421) 
The instruments could only come near a 303 with a Chip, a Filter and
two envelopes. The biquad filter does not saturate, every note strikes
anew, and there was no slide and no accent.

- Acid is a new instrument node that plays a saw or a square through a
  four-pole ladder low-pass with tanh saturation, oversampled twice.
  Knobs: cutoff, resonance, env mod, decay, accent, tuning and slide
  time.
- An accent comes from the velocity, from 0.9 on. It is louder, opens
  the filter further with a shorter decay, and accents close behind
  each other build up.
- Two notes that touch or overlap slide. Each note stays a voice of
  its own, so the scheduler is unchanged. The notes of a line share
  one synth in the worklet, and the new voice takes it over from its
  first sample on.
- The node sets its own volume, so it needs no envelope behind it.
- The graph shows it as a device with the filter curve at rest and at
  the top of the sweep.
- A preset Acid Bass puts an Acid in front of a short echo.

Tests:
- tsc, eslint, Prettier clean
- Unit: 14 new tests in acid.test.ts for pitch, low-pass, resonance,
  sweep, accent, slide and the handover between two voices.
- e2e 34 on melektaus-atx: ALL PASSED, 148 checks (3 new). The
  handover sits on sample 4800 and jumps by 0.0012, of three notes
  only the second slides.
- Without the handover, or with a slide that strikes anew, the unit
  tests are red.
- Gate on melektaus-atx: tsc, eslint, Prettier clean, 4753 of 4754
  tests green. The red one was sound-library.test.ts, which did not
  know the new slid count in the stats. Fixed, and the file alone
  is green again with 128 tests. No second full gate.
- Not checked: the sound by ear, slide at the keyboard, a bake with
  Acid, the CPU load of many notes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 11:08:57 +02:00
adminandClaude Opus 5.5 3443368855 Record a game from its very first frame
game-maker / gate (push) Successful in 4m18s
Details
game-maker / suite (push) Successful in 18m17s
Details 
Start recording has to set up its encoders first. It asks the browser
whether it can encode H.264 at the size and hooks in the sound tap,
both asynchronously, and it only took its size at the first frame.
Meanwhile the game ran on unrecorded. A caption that fades in from
alpha 0 at the start was already visible in the first recorded frame.

- A run now waits while a recording sets up its encoders, at most 3 s,
  as it already waits for its videos at Play. That holds in the editor
  and in play.html.
- The recording takes its size at Play, right after the game view has
  the world, rather than at the first frame. A Start recording in
  On start is so ready before the first tick.
- A recording started in the middle of a game holds the picture for
  that moment. A sound that already plays goes on.

Tests:
- tsc, eslint, Prettier clean
- e2e 67 on melektaus-atx: ALL PASSED, 11 checks (67a-ii new). The
  first recorded frame is at game time 0.0168 s, the first frame.
- Without the wait 67a-ii is red: the first recorded frame is at
  0.1667 s, so 10 frames were lost.
- Gate on melektaus-atx: green, 305 files, 4740 tests, run on this
  commit and the one before together.
- Not checked in the browser: a start in the middle of a game, and
  play.html.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 10:11:27 +02:00
adminandClaude Opus 5.5 2ba41c1bd0 Record 1440p and 4K, and choose sound and conversion in the node 
The Start recording node only offered the game view, 720p and 1080p,
and always recorded with sound, converted and imported. The recording
from the main menu could already leave those out.

- Resolution has 1440p and 4K. As with 720p and 1080p the height is
  fixed and the width follows the camera, so 16:9 gives 2560x1440 and
  3840x2160.
- Every resolution is capped at 3840x2160, as the game view already
  was. A 21:9 camera in 4K would be 5040 px wide, which the encoder
  levels are not made for. It is scaled down to 3840x1646.
- The node has Sound, Convert and Import after conversion, all on by
  default as in "Start recording...". Import only counts with Convert.
  An old graph without the inputs records as before.
- A released game ignores Convert and Import and downloads the MKV.
- Both recordings now hand their upload options the same way, from
  GameRecorder over PlaySession to RecordingUploads.
  RecordingUploadOptions moved to core/recording/recording.ts.

Tests:
- tsc, eslint, Prettier clean
- Unit recording and node-catalog: 451 green, recording 29 with
  3 new tests and 1 extended
- e2e 67 on melektaus-atx: ALL PASSED, 10 checks
- Not checked in the browser: a real 4K recording, and the node with
  its switches off.
- The gate ran once on this commit and the next one together.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 10:08:54 +02:00
adminandClaude Opus 5.5 65d7bf6a1f Silence a composition video as soon as its item ends
game-maker / gate (push) Successful in 4m19s
Details
game-maker / suite (push) Failing after 19m52s
Details 
A video in a composition plays on a driven player that the item tells
where to stand every frame. When the item ended, its object was gone
and stopped asking, but the player ran on with its sound until it was
let go for being unused. That took 2 to 4 seconds, and in a pause it
never happened at all. The same held for the composition preview.

- A driven player nobody asked for in 100 ms is now paused and muted.
  When its wearer comes back it carries on where it should be, as
  before. It is still let go after 2 to 4 seconds.
- A player still without its first picture keeps playing, without
  sound, because it only gets that picture by playing.
- Leaving the animation or composition tab now silences their video
  preview at once instead of after 2 to 4 seconds.
- Free videos (Video clock free) keep the old behaviour. A sprite
  destroyed by a behavior is still heard for 2 to 4 seconds.

Tests:
- Unit video-library: 10 green (1 new). Without the new timer the
  new test is red.
- e2e 68: ALL PASSED, 11 checks (68g new). An item of 0.5 s with a
  2 s video plays audibly while it is there, and the player is
  paused and muted in every frame from about 200 ms after it ends.
- Gate on melektaus-atx: green, 305 files, 4737 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 08:24:04 +02:00
adminandClaude Opus 5.5 126a5a3a62 Let a composition item's clips move along with it 
A clip only hung at its track and frame and worked on whatever item it
overlapped. Moving an item left its clips behind, so they started to
animate the next item that came along.

- A clip belongs to the item that lies on its track at its start frame.
  With two there, it belongs to the one in front. A clip that only
  reaches into an item stays with the item before. The model does not
  change, the rule is worked out each time.
- Dragging an item, also onto another track, and typing its Start in
  the inspector take its clips along in the same undo step.
- Where a carried clip would land on a foreign one, the whole move
  holds at the last place where everything fits, as dragging clips
  already does.
- Pressing Alt during the drag moves the item alone. Alt on the press
  already starts the clip band since 0648853.
- Copying an item takes its clips along with new ids. On a collision
  items and clips slide on together.
- Deleting an item deletes its clips. Trimming and dragging its edges
  leave them where they are.

Tests:
- tsc, eslint, Prettier clean
- Unit composition-edit 36 green (6 new), composition-sprites 11
- Without the rule 5 of the 6 new tests are red
- e2e 69 ALL PASSED, 18 checks (69n new)
- Gate on melektaus-atx green: 305 files, 4736 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 08:03:35 +02:00
adminandClaude Opus 5.5 480db9c4e4 Scroll the rows of the animation timeline up and down (#413)
game-maker / gate (push) Successful in 4m28s
Details
game-maker / suite (push) Successful in 18m13s
Details 
An animation with more rows than the panel is tall cut the bottom rows
off. There was no way to reach them, their fields or their keys.

- The view has a new `scrollY`. Rows scroll up under the ruler, which
  stays where it is together with the tab head.
- The wheel over the header column scrolls the rows. Over the cells it
  still zooms, and `Shift`+wheel still scrolls across.
- A scrollbar at the right edge shows up only when the rows do not
  fit. Its thumb can be dragged, a click beside it jumps there.
- The middle button pans both ways. `Shift`+drag stays across only.
- A row picked or added out of view is scrolled just far enough to be
  seen whole. Value fields out of view are not drawn.
- The tab keeps its scroll like its zoom when switching.

Found on the way: `zoomCells` built a new view from `panX` and
`cellWidth` only, so every zoom would have dropped the scroll. The
type check caught it, and a unit test now holds it.

Tests:
- tsc, eslint, Prettier clean
- Unit timeline-view: 20 green, 6 new. Bending `rowAt` to ignore the
  scroll or `revealRow` to do nothing turns exactly those tests red.
- e2e 25 on melektaus-atx: ALL PASSED, 37 checks (25j, 25j-ii new).
  32 rows, the wheel brings the last field into view and typing keys
  77 there. Dragging the thumb up goes back to 0.
- Gate on melektaus-atx: green, 305 files, 4730 tests.
- Not checked in the browser: the middle button, revealing a picked
  row, the wheel right over a value field.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 07:22:11 +02:00
adminandClaude Opus 5.5 0648853356 Select composition items with a band and copy clips too (#417)
game-maker / gate (push) Successful in 4m57s
Details
game-maker / suite (push) Successful in 18m29s
Details 
The composition tab only took single clicks and Ctrl-clicks, and
animation clips could not be copied at all. A band selection and
Copy/Paste for clips make longer compositions quicker to arrange.

- Dragging from an empty spot draws a band. It selects every item it
  touches, across tracks. Ctrl adds to the selection, because Shift
  plus drag already pans the view.
- With Alt held the band selects clips instead and may start on an
  item, since items fill most of a track.
- Ctrl+C copies the selected clips as their own clipboard kind.
  Ctrl+V places them like items: under the pointer over the tracks,
  otherwise at the playhead on the selected track.
- Clips of a track never overlap. A paste onto taken frames moves the
  whole copy on frame by frame until every clip fits (`pastedClips`).
- The paste is one undo step and selects the new clips.

Tests:
- Gate on melektaus-atx: tsc, eslint, Prettier clean,
  305 files, 4724 tests (2 new).
- e2e 69: ALL PASSED, 17 checks (69m new). Band over three items on
  two tracks, Alt band from an item picks both clips, first paste at
  frames 50 and 60, second paste moves on to 56 and 66.
- Counter-check: without moving on, the paste unit test is red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 06:56:25 +02:00
adminandClaude Opus 5.5 5df2173559 Show Delete in red in the track and item menus of both timelines 
A composition clip had a red button in the inspector that did the
same as its menu entry. Deleting a track looked like any other entry
in the menu, although it takes the track with everything on it.

- The inspector of a clip drops "Remove clip(s)". Right-click and
  the Delete key still remove it.
- "Delete track" is red in the Composition and Song tabs, behind a
  separator like "Remove item" in the other menus.
- "Delete" on a Song item is red, too.

Tests:
- Gate on melektaus-atx: tsc, eslint and Prettier clean,
  305 files, 4722 tests passed.
- No check clicks the inspector button or counts the track menu, so
  no e2e module ran. The red entries were not looked at in a picture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 06:38:54 +02:00
adminandClaude Opus 5.5 7a952cb20d Start paused, step one frame and break from a behavior 
Finding out what a game does in its first frames, or right at one
spot of a flow, needed a quick hand on the pause button. Now the
editor can start a play paused, step it frame by frame and let a
behavior pause it.

- The pause button can be pressed in edit mode. Then Play starts
  paused, and every further Play does too until it is released. A
  reload of the page clears it.
- A third button, Next frame, right of Pause, or Ctrl+Shift+Enter.
  It works only while paused and runs exactly one frame of 1/60 s,
  whatever the frame took on screen. Hierarchy and inspector follow.
- The Break node pauses the play like the button, at the end of the
  frame it ran in. The status line names the behavior's object. It
  is the editor's pause, not Pause game.
- Break reaches the editor through a service only the editor hands
  to a play. In play.html and Run in editor the node does nothing.
- Restart game keeps the running play's state. Start paused only
  counts when Play is pressed.
- Wiki Concepts, Behaviors, Keyboard: the buttons, the node and the
  shortcut.

Not tried: sound, free-running videos and a running recording while
stepping. Sound and videos stay held during the pause.

Tests:
- tsc, eslint and Prettier are clean.
- Unit pause, node-catalog, tooltip: 465 green. svg-icons now
  expects the step icon too.
- e2e 18 and 46 on stein2 (melektaus-atx was off): ALL PASSED,
  63 checks, 7 new in 18j. A step moves the box by 1.667 px at
  100 px/s, Break stops the play with "Break in Breaker".
- Counter-checks: a session that ignores the step turns 18j red. A
  Break that does not report its object turns the unit test red.
- Gate on stein2: tsc, eslint and Prettier clean, 305 files,
  4722 tests, 6 more than before. One was red at first, the icon
  list of the top bar did not know step yet. Fixed in
  svg-icons.test.ts and run again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-13 06:02:35 +02:00
adminandClaude Opus 5.5 64d0d472e5 Show a spawned object's first animation frame right away 
An object that appears during a frame and plays an animation in its
On start was drawn once in its original state. A composition item
showed frame 0 one frame late. An object from the Spawn node even ran
its On start only on the next frame, after it had been drawn.

- BehaviorSystem: at the end of the behavior pass, objects that came
  to be in this frame hear On start and On enable. This covers spawned
  objects and behaviors switched on for the first time. Their
  On update starts with the next frame.
- AnimationSystem.showStarted: right after that, every animation
  started in this frame shows its frame 0. The clock runs from the next
  frame, and an end is told no earlier than then. A paused frame skips
  this, its animations stand still anyway.
- What such a late On start spawns in turn starts in the next frame.
  Otherwise a spawn chain could stretch one frame without end.
- Scene start is unchanged. There On start already ran before the
  first frame.
- Wiki Behaviors, Lifecycle: the new timing.

Tests: gate green on stein2 (melektaus-atx went offline mid-run),
305 files, 4716 tests (2 new). e2e 68 on melektaus-atx green,
10 checks. Without the late pass the Spawn test
reads 7 instead of 0, without showStarted both new tests are red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 21:55:35 +02:00
adminandClaude Opus 5.5 33891acfb3 Move, turn and scale an item in the composition picture live 
Dragging an object in the Composition tab's picture writes the copy
of the composition at the playhead and asks for a preview once a
frame. The tab answered every such preview by building that copy
again from the composition, which still held the old value. So the
object stood still while dragged, and a move often did nothing at
all, because its end was read from a copy that had just been rebuilt.
Values typed in the inspector were not affected.

- SceneViewport gets editing(), which says whether a gesture of this
  view is under way: a drag, a size or turn handle, a polygon corner,
  a joint anchor or a tilemap stroke.
- CompositionCanvas leaves its copy alone while such a gesture sends
  its frame previews. The command at the end writes the composition
  as before, and the copy is built again from it.

Tests:
- tsc, eslint and Prettier are clean.
- e2e 69 on melektaus-atx: ALL PASSED, 16 checks, 69l is new. An item
  dragged 60 px to the right stands at +60 px while the button is
  still down (590 to 650) and has x = 60 in the model afterwards.
- Counter-check: rebuilding the copy on every preview again turns 69l
  red. The item stays put while dragged and x is 0 after letting go.
- Gate on melektaus-atx: green, 305 files, 4714 tests.
- Turning and scaling by the handles were not tried in the browser.
  They take the same path and editing() covers them.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 19:52:11 +02:00
adminandClaude Opus 5.5 b16fe8c718 Drop a picture or video onto a composition track (#419) 
A track item has always carried an object, and an object needs an
asset. Instead of objects without one, a dropped picture or video gets
a sprite the composition owns, the way a song owns its sounds (#355).
To the rest of the code it is a sprite like any other.

- The asset list starts a drag for textures, vector art, pixel art and
  videos. Dropped on a track, they make the sprite and the item in one
  undo step. A video item is as long as the video and follows it.
- One sprite per picture and composition. A second drop of the same
  video shares it. Once no item wears it, it goes in the same step.
- Deleting a composition takes its sprites along. A duplicate gets
  sprites of its own.
- Pasting into a scene or container turns the sprite into a project
  asset. Pasting into another composition gives that one a copy. The
  clipboard carries the sprite, so cut and paste loses nothing.
- Owned sprites stay out of the asset list and the pickers. The
  inspector shows them with a dot in front.
- "Make project asset" in the item's menu, on the timeline and in the
  hierarchy, turns the sprite into an ordinary one.
- The first track of the Composition tab gets its top line, as the
  Song tab's has.

Found on the way: loading kept an owner only for the asset types a
song can hold. A composition's sprite would have lost its owner and
turned up in the list after a reload.

Tests:
- Unit composition-sprites: 11 new. Without the rule 5 of them are
  red, without the loading fix the loading test is.
- e2e 69 on melektaus-atx: ALL PASSED, 15 checks, 69k new
- Gate on melektaus-atx: 305 files, 4714 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 18:48:20 +02:00
adminandClaude Opus 5.5 f171c77f91 Turn the own mouse pointer off again after signing out and in 
CI was red on d4fc425 with two checks, both from this week's work.

- Signing out builds the editor anew on the same renderer. The second
  useCursorFiles then took the SVG pointers of the first for Pixi's
  own styles, so switching the option off kept them. The editor now
  undoes the pointers on dispose and Pixi's styles are back in place.
- e2e 62d still looked for "Start recording". The menu reads "Start
  recording…" since ba01d14, and 62 had not run then.

Tests:
- e2e 45, 46 and 62 on melektaus-atx: ALL PASSED, 51 checks
- Without the dispose, 46i-ii is red after 45, as it was in CI
- Gate, measured together with #419 on top: 305 files, 4714 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 18:48:18 +02:00
adminandClaude Opus 5.5 c425eea004 Key what an empty timeline row shows on a double-click (#412) 
A double-click into a timeline row without keys set the property's
default, while the field in the row header showed the value the target
actually holds. A sprite standing on x 40 got a key with 0, so the
first key moved it. The double-click now keys the value the row shows.

- trackHeld in src/core/model/animation.ts says what a row without keys
  holds. The field in the row header and SetAnimationKeyCommand both
  ask it, so they cannot drift apart again.
- A relative row still gets 0, a row with keys still takes the value of
  its curve at that frame, and a row whose target is gone still only
  gives a notice and no key.
- README and the wiki page Development name 2047 checks.

Tests:
- Unit animation: 87 green, 1 new. A sprite on x 40 with the override
  scale 2 gets keys with 40 and 2. Red with the old def.default.
- e2e 25 on melektaus-atx: ALL PASSED, 35 checks, 25i-ii new. The arm
  stands on x 40 and a double-click on frame 3 keys { frame: 3,
  value: 40 }.
- Gate on melektaus-atx: green, 304 files, 4703 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 18:09:02 +02:00
adminandClaude Opus 5.5 d4fc425233 Draw the editor's mouse pointer from SVG files, on screen too
game-maker / gate (push) Successful in 4m19s
Details
game-maker / suite (push) Failing after 17m35s
Details 
The pointer drawn into an editor recording lived as paths in code, so
it could not be looked at or edited by hand. The shapes are now SVG
files, and the editor can show the same pointer on screen, so what is
recorded is what was seen.

- src/core/recording/cursors/ holds one file per shape: arrow, hand,
  open-hand, fist, move, crosshair, text and resize. Resize is turned
  for the vertical and diagonal directions.
- data-hot="x y" on the <svg> is the hotspot in viewBox units. Width
  and height in the file are ignored, and anything before <svg>, such
  as <?xml ...> from a drawing program, is cut off.
- The open hand had six fingers. It has a thumb and four now, the
  middle one longest.
- CURSOR_SIZE = 0.8 scales every pointer the same on screen and in a
  recording, so the files keep their units.
- General settings has "Custom mouse pointer", on by default. It hangs
  the files into Pixi's cursorStyles as CSS cursors with the keyword as
  fallback, so the system still draws the pointer and the canvas does
  not. Off gives Pixi's own values back at once.
- A recording draws the pointer from the same files at its own size,
  so it stays sharp in 4K. It still reads the shape from the canvas
  when that holds a url(...) cursor.

Tests:
- Unit recording and view-tween: 3 new, green. They read viewBox and
  data-hot in an edited form, the boxes of turned shapes, the default
  size and old settings without the switch. Red without data-hot and
  without cutting before <svg>.
- e2e 46i-ii: the canvas cursor is a url(...) ending in default and
  the image loads. Off gives inherit and stores false, on brings the
  file back. Red without useCursorFiles.
- e2e 67e: the recording is still black at the mouse position.
- The README names 2046 e2e checks.
- Gate on melektaus-atx: 304 files, 4702 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 17:33:57 +02:00
adminandClaude Opus 5.5 ba01d146e9 Choose size, rate and what follows when recording the editor 
The editor recording had fixed settings: the canvas size at 60 fps,
always converted and imported. Recording in 4K needs a choice, and the
server shrank every video to 1080p anyway. Start recording... now opens
a dialog, and a recording always goes to the server as it is.

- The dialog holds name, resolution (Screen, 1080p, 1440p, 4K), fps
  (60 or 30), mouse pointer, sound, Convert and Import after
  conversion. Import is greyed out while Convert is off. The browser
  keeps the last choice, not the name.
- A larger resolution makes the editor draw finer while it records,
  so 4K is sharp, not stretched. It is never below the screen's own
  resolution and goes back after Stop.
- Stop always uploads. Without Convert the file only stays on the
  server. Without Import it is converted but no asset is added. Both
  are found in the video picker.
- The server keeps the original next to each MP4 for every upload, so
  Download in the video tab and the picker gives the original. Videos
  from before have none, they still download their MP4.
- A file left unconverted shows in the picker as not converted, with
  a Convert button. POST videos/{file}/convert queues it, and upload
  takes convert=0.
- The worker now caps a video at 3840x2160 instead of 1920x1080. Videos
  converted before stay as they are.

Tests:
- PHP videos.test.php: 81 checks green, with the original, convert=0,
  a later Convert and the 4K cap.
- Unit recording: 5 new, green. Red without the rule that a recording
  is never below the screen's resolution.
- e2e 67e-ii: the video tab downloads the .source.mkv, a Matroska
  file. 67f: 1440p from a 1280x800 window gives 2304x1440 at 30 fps
  without sound, kept and not imported, then converted in the picker.
  Both red with the MP4 download and without the finer drawing.
- The README names 2045 e2e checks.
- Gate on melektaus-atx: 304 files, 4699 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 17:01:55 +02:00
adminandClaude Opus 5.5 93190e185c Route a video's sound through its own Master routing node 
An editor recording had no sound while the video tab played a video.
The tab played its <video> straight to the speakers, past the Web
Audio graph, so the recording tap behind the limiter never heard it.
Fixing only that would have left videos as the one sound source the
Master routing cannot reach, so they get their own node, like sounds.

- A video asset with sound has a routing node labelled "Video". One
  without sound has none, it would have nothing to send.
- Its links take the sound after volume and FX. Several links or
  another level give the video its own bus, as with a sound.
- A new video follows the bus most other videos use. With none linked
  yet it goes to the Master, not to where the sounds go. Songs and
  sounds no longer follow videos either.
- Players only know their file, so videoBusesOf maps each file to the
  bus of its first asset. setMix takes that map and moves the gains of
  running videos at once when the routing changes.
- In the game, played in the editor or released, a video's sound
  follows its links. The tabs (video, composition, animation preview)
  play straight into the Master, as the sound tabs have since #368.
- The video tab's sound now goes through the Master, so its volume, FX
  and limiter apply there and the editor recording hears it.
- The recording badge beside the menu is an HTML element over the
  canvas now. The second render that drew it on the desynchronized
  canvas let the screen show a frame without it, so it flickered. The
  element is never in the copy, and clicks go through it.

Tests:
- Unit master-routing and sound-library: 3 new, green. Red without
  video in the routing kinds, without moving the gains, and with the
  old rule for the links of a new video.
- e2e 66e-iii: the tab plays a video linked to a bus, the level is
  0.09 on the Master and 0 on the bus. 66g: in the game the bus has
  0.09. Both red with the old ways.
- e2e 67e reads the badge from a real screenshot, still green.
- The README names 2043 e2e checks.
- Gate on melektaus-atx: 304 files, 4694 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 16:15:59 +02:00
adminandClaude Opus 5.5 846a8b70b3 Let ?w=&h= fix the editor canvas to a set size 
A recording of the editor is as large as its canvas, and so far the
canvas always followed the window. With ?w=1920&h=1080 the editor is
laid out at exactly that size in CSS pixels, whatever the window.

- src/ui/pageSize.ts reads w and h (both needed, 200 to 8192 each) and
  works out how the canvas fits the window.
- A smaller window shows the canvas shrunk to fit, centred, with the
  same aspect. A larger one shows it 1:1, centred on a dark page.
- Clicks still land right, since Pixi maps them over the canvas's
  on-screen box. Switching projects keeps w and h in the URL.
- The pointer drawn into an editor recording now scales against the
  layout size, so a shrunk canvas still records a full-size pointer.

Tests:
- Unit page-size: 5 new, green.
- e2e 62d: ?w=1600&h=900 in a 960x720 window gives a 1600x900 canvas
  shown at 960x540, y 90, and a click on the menu button opens it.
  Red without the shrinking.
- Gate on melektaus-atx: 304 files, 4691 tests green.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 14:44:28 +02:00
adminandClaude Opus 5.5 d6a325978c Name the e2e check count CI actually measures 
Since a62d1d0 the README named two checks more than the suite counts.
CI on 080df2d reported 2039, the README said 2041.

- The count now matches the last green CI run.
- Which two checks were counted twice back then is not traced.

Tests: documentation only, CI suite on 080df2d counted 2039 checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 14:19:56 +02:00
adminandClaude Opus 5.5 080df2d211 Let the video tab's Play start at the playhead and reset its button
game-maker / gate (push) Successful in 4m18s
Details
game-maker / suite (push) Successful in 17m57s
Details 
Two faults in the video tab. A click into the ruler while a clip
played stopped the playback, but the button kept showing "Stop". And
Play always started at the start of the clip, wherever the playhead
stood.

- The tab now reports when the ruler stops the playback, and the
  button switches back to "Play".
- Play starts at the playhead when it lies inside the clip. Without
  a selected clip the whole video counts. Outside or at the end,
  Play starts at the beginning.
- So Stop and Play continue where the playback stopped. A clip that
  ran to its end without loop starts again at the beginning.

Tests: gate on melektaus-atx green. e2e 66 ALL PASSED, 32 checks (2
new in 66e-i). With both fixes taken back, both checks are red with
exactly the reported faults: the button shows "Stop", and Play runs
from 0.71 s, i.e. from the clip start at 0.6 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 13:55:13 +02:00
adminandClaude Opus 5.5 f177ee90a4 Record the whole editor from the main menu 
The game view could already record itself through nodes. Making a
video of the editor itself, to show how something is done, needed an
outside screen recorder. The main menu now starts and stops one, with
no options to set.

- "Start recording" sits in its own group before General settings, and
  reads "Stop recording" while one runs. The name is
  "Recording <time>", the size is the whole canvas in device pixels
  (an odd row or column is cut), 60 fps always. Sound is the master,
  saving goes the same way as for the game view.
- The encoder, the sound and the Matroska writing of a take moved from
  GameRecorder.ts to RecordingTake.ts. A take now reads its pictures
  from a FrameSource, so the game view and the editor share it.
- The editor's picture is copied from the canvas after each render, in
  the same frame, so nothing is drawn twice. A resting editor does not
  render, and the recording repeats its last picture.
- The mouse pointer is drawn into the recording in the shape the editor
  shows (arrow, hand, resize arrows, I-beam, crosshair, move).
- A red dot with the running time stands beside the menu button on
  screen only. It is drawn after the copy, and the toolbar makes room
  for it.
- Drawing that dot made Pixi take it as the root for every hit test,
  and the menu would not open any more. The stage is set back as the
  last rendered object after it.

Tests:
- Gate on melektaus-atx green: 303 files, 4686 tests.
- Unit recording: 20 green, 5 new.
- e2e 67: 8 checks, 67e new. It records through the menu and decodes
  the MP4: 1280x800 at 60 fps, the pointer black at the mouse, the dot
  missing, 238 plain spots alike with the canvas. Without the pointer
  and with the dot drawn before the copy, 67e is red.
- e2e 46: the main menu lists "Start recording".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 13:40:47 +02:00
adminandClaude Opus 5.5 c0ba2673ba Keep the video tab's picture following while scrubbing (#417) 
Scrubbing in the video tab set currentTime on every pointer move.
Each new seek aborts the running one. With a large video a seek takes
longer than the gap between two moves, so none finished and the
picture froze. The composition tab never had this, because its
players run through the VideoLibrary, which only keeps the newest
wish while a seek runs.

- The pending-seek logic moves into a shared helper,
  src/viewport/videoSeek.ts. The VideoLibrary uses it unchanged.
- The video tab seeks through the same helper and uploads the
  picture again after each finished seek.
- Play drops a pending scrub wish. Otherwise the video would jump
  back right after it starts.
- Scrubbing before the first picture no longer jumps back to the
  time the tab opened with. The video stays at the last chosen time.

Tests: gate on melektaus-atx green, 303 files, 4681 tests (2 new in
video-seek). e2e 66 ALL PASSED, 30 checks. Without the helper the
new unit test is red ([0.1, 0.2, 0.3, 0.4, 0.5] instead of [0.1]).
There is no browser check for the freeze itself, the test video
seeks too fast to show it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 12:56:37 +02:00
adminandClaude Opus 5.5 40b6290d24 Sample 68d's video wait from before the click on Play (#417)
game-maker / gate (push) Successful in 4m13s
Details
game-maker / suite (push) Successful in 17m57s
Details 
CI task 669 failed in e2e 68d with no loading frame seen. The check
began to sample after play(), and play() settles after the start. A
video that loads quickly was ready by then, so the wait it checks had
already passed. The composition itself had waited: its clock read 0
when the player was ready.

- 68d starts its sampler before the click on Play and counts only
  frames in play mode.
- The sampler stops after 20 s, not after a number of frames. Before
  the run the page draws far faster than 60 frames a second.

Tests:
- e2e 66, 67, 68 on melektaus-atx: ALL PASSED, 43 checks, 68d saw 6
  loading frames at 0 and 0.32 s after

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 12:31:32 +02:00
adminandClaude Opus 5.5 10c2c3c308 Expect compositions in the scene view's visibility list (#417)
game-maker / gate (push) Successful in 4m14s
Details
game-maker / suite (push) Failing after 16m59s
Details 
CI task 668 failed in e2e 46. A composition placed in a scene has its
own symbol and name since 0334d25, so the eye in the scene header
lists it as a placeable type. The check still expected the list from
before.

- 46-topbar: composition joins the expected types, between collider
  and container.

Tests:
- e2e 46 on melektaus-atx: ALL PASSED, 24 checks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 11:58:34 +02:00
adminandClaude Opus 5.5 b2b4cbad28 Keep a video's loading out of recordings and first frames (#417)
game-maker / gate (push) Successful in 4m21s
Details
game-maker / suite (push) Failing after 17m16s
Details 
A Start recording right after Play composition still began with
empty frames. The composition's clock waited for its video, but the
recording went on and took every frame while the video loaded. A
clip that starts later in the video also slipped through, because
its player already had one picture while it still sought to the
clip's start.

- The view now counts every video sprite it leaves out as missing,
  not only one without any first picture.
- GameRecorder takes no frame while the game view misses a video,
  for at most VIDEO_WAIT_MS (3 s) at a time. The recorded time
  stands and the sound of that time is dropped, so it stays in sync.
- RecordingClock.start forgets a hold from before the first frame.
  Without that, the first real frames all got the time 0.
- A followed player starts at 1x until the first speed reading.
  Before, it paused and sought every frame for about 100 ms, so the
  first pictures after a wait stuck and then jumped. A step faster
  than 4x and a time running backwards do not count as a start.
- Test hooks: __gm.gameVideo(id) reads the game view's own player,
  TakeInfo.lastFrameAt shows when the last frame was taken.

Tests:
- Unit recording (clock after an early hold), video-follow (2 new,
  1 adapted): 449 green in tests/core/recording, tests/viewport and
  tests/core/composition*
- e2e 68e (a clip from 0.8 s shows that picture first and then
  plays), 68f (a recording that already runs takes no frame while
  the composition's video loads)
- 68e is red without the start at 1x, 68f without the hold
- README: 2038 editor checks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 11:38:37 +02:00
adminandClaude Opus 5.5 6e6b6a4d9a Wait for a composition's videos and copy and paste its items (#417) 
A composition lost the first frames of a video item. Its clock ran
while the player was still loading, in the tab and in the game. The
game only waited for videos once, when it started. A composition that
a behavior starts later spawned its items after that wait.

- The view now records which video sprites it left out for lack of a
  first picture, one entry per object.
- The composition tab holds its playhead while the preview misses a
  video, for at most VIDEO_WAIT_MS (3 s) like the game.
- CompositionSystem holds a composition's clock while one of its live
  items misses its video. The rest of the game keeps running.
- The editor play and the published game pass their view to the run
  as RunServices.pictures.
- VIDEO_WAIT_MS moves to core/model/video.ts, because the runtime
  does not import from viewport/.
- Videos loaded ahead for coming items do not hold the clock.

Items can now be copied and pasted inside the composition tab.

- Ctrl+C keeps the timing of the copied items next to the objects.
- Ctrl+V pastes objects and items in one undo step. The earliest item
  lands at the frame under the pointer, on the track there.
- Without the pointer over the tracks it lands at the playhead, on the
  selected track or on the top track of the selected items.
- Several items keep their distances and tracks to each other.
- The same copy pastes into a scene as plain objects. Objects copied
  from a scene do not paste into a composition, they have no timing.

Tests:
- Unit composition-edit, clipboard, composition-play: 3 new for
  pasting, 1 new for the clock waiting in the game
- e2e 69i (play from the end waits for the video), 69j (paste under
  the pointer and at the playhead), 68d (a composition started from
  On start waits for its video)
- each new check is red without its fix
- README: 2036 editor checks

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 10:38:19 +02:00
adminandClaude Opus 5.5 c4650e43f0 Let composition items overlap and show where their video lies (#417) 
Trying the tab turned up six things to fix. Items of a track could
not overlap, so a drag stalled as soon as the pointer reached another
item. An item's ends gave no hint they could be dragged, a trimmed
video showed nowhere, clip names were hard to read and vanished when
picked, and the track menu did not match the song tab.

- Items of a track may overlap and nothing is cut or pushed aside.
  The later one lies in front, in the picture and on the timeline.
  The overlap is shaded and a line marks the covered end.
- Edges win over bodies when hit, so the covered end of the lower
  item can still be dragged. Hovering an edge lights it up and shows
  the ew-resize cursor, as in the song tab.
- A clip in the overlap plays on both items. Clips themselves still
  do not overlap; a moved clip now runs up to its neighbour instead
  of stopping short.
- Fit to media and a new video no longer cap the length at the next
  item, which only made sense while overlaps were forbidden.
- The left edge can be trimmed past the start of the video. The
  offset goes negative and the sprite shows the first picture until
  the video begins.
- Video marks: a corner triangle where the item cuts into its video,
  a line with a triangle like the song loop where the video starts or
  ends inside the item. Edges snap to the video's ends within 6 px.
- Clip names sit in their own layer above the bars, in text colour.
- The track menu matches the song tab: add above or below, move up or
  down, delete. It opens on the head and on an empty spot of the
  track. The Remove track button left the inspector.

Tests:
- Unit composition and composition-edit: 67 green, several new or
  rewritten. Bending three spots in the rules turned their tests red.
- e2e 69: ALL PASSED, 12 checks. 69b and 69c now drop the second item
  over the first and drag the covered end past it. Without edges
  winning the hit, 69c fails with hover: null.
- Gate on melektaus-atx: CHECK EXIT 0, 302 files, 4672 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 08:57:26 +02:00
adminandClaude Opus 5.5 4f0272638e Load the next item's video before a composition cuts to it (#417) 
Two sprites with videos back to back left a blank frame at the cut.
The video player only came into being once the item began, and a
sprite is not drawn until its video has a first frame. So the second
item was missing until its video had loaded.

- compositionAhead lists the items that begin within the next second.
  Near the end of a looping composition it also lists the items at
  its start.
- An item that is alive already is not loaded ahead. Two times would
  otherwise pull at the same player.
- The frame carries these videos hidden and paused on their first
  picture at the item's offset. Their players get the id the item
  will have, so the cut reuses them instead of opening a second one.
- This works the same in the tab's preview and in the game.
- One second of lead (AHEAD_SECONDS) is an assumption. It can go up
  if large videos still flash.

Tests:
- Unit composition: 27 green, 2 new. Unit composition-play: 14
  green, 1 new for the path in the game.
- e2e 69: ALL PASSED, 12 checks. The new 69h puts two video items
  back to back and checks the second player is open and paused at
  its offset before the cut, and drawn in the same redraw as the
  cut.
- Without loading ahead in the preview 69h fails with cut: null.
  Bending two spots in the model turned their unit tests red.
- Gate on melektaus-atx: CHECK EXIT 0, 302 files, 4665 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 08:07:46 +02:00
adminandClaude Opus 5.5 e435c54afb Edit a composition's tracks, items and clips in its own tab (#417) 
A composition could be placed and played, but not filled. Its tab
puts the picture at the playhead above a timeline of tracks, so a cut
or a cutscene is laid out by dragging assets onto tracks.

- The timeline takes the song tab's geometry with frames for ticks:
  ruler in seconds, scrubbing, play and pause, + Track. Zoom and pan
  glide through viewTween like the other tabs.
- A placeable asset dropped on a track becomes an item, 3 s long or
  as long as its sprite's video. Dropped on an item it lands behind.
- Items move without overlapping. The right edge stops at the next
  item. The left edge trims into a video and not before its start.
- An animation dropped on a track becomes a narrower clip bar above
  the item it plays on. The asset panel now starts a drag for
  animations too.
- The picture is a scene view over compositionHolder. A click picks
  an item or its child, a drag moves the real object. It takes no
  assets, since an object there would have no item.
- The hierarchy lists the items alive at the playhead. Reparenting is
  off there, items stay at the top level.
- The inspector shows a TIMING block on an item (start, length,
  offset for a video, Fit to media) and edits tracks and clips.
- Delete, the context menu and Remove item take item and object in
  one undo step.
- fitMedia: an override that gives an item another video sets its
  length in the same undo step, capped at the next item. Dragging an
  edge or typing a length turns it off.
- A composition inside a composition is refused.

Tests:
- Unit composition-edit (new): 19 tests. Four counter-checks each
  turned their test red. The "back to 3 s" test got a step in
  between after one stayed green.
- e2e 69 (new): 11 checks, 9 own. The hierarchy check reads the rows
  shown, after a counter-check on the model's rows stayed green.
- Gate on melektaus-atx: CHECK EXIT 0, 302 files, 4662 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 07:47:22 +02:00
adminandClaude Opus 5.5 0334d2508d Place compositions in scenes and play them from behaviors (#417) 
A composition can now sit in a scene like a container and play there.
Its items become real objects while they run, so behaviors, colliders
and events work on them as on any other object.

- A composition is placeable with its own symbol, transform and render
  layer. The scene view shows its picture at frame 0. A click on that
  picture selects the placement, not a single item.
- CompositionSystem spawns each item as an object under the placement
  when it begins, with the id <placement>~<item>. On start and the
  other events run. The object is destroyed when the item ends.
- The top track lies in front no matter which item started first.
  World.arrange() orders the entity list and the scene objects.
- Clips and driven videos follow the same rules as compositionAt().
  Loop, restart, stop and the game pause work.
- New nodes: Play composition (with restart), Stop composition and the
  event On composition finished. They were planned for step 4 but come
  now, because without autoplay nothing could play otherwise.
- The editor run ignores Play composition and says so in the log. It
  writes into the saved project, and the items would stay behind.
- contentId() moved to instance.ts. The animation system uses it for
  nested ids too.
- Compositions get no "Sort with parent" for now.

Tests: the gate passed with 301 files and 4643 tests, 18 more than
step 1. 13 of them are in tests/core/composition-play.test.ts. The
first gate run failed in event-order.test.ts, which counts the event
nodes and now finds 19. Five counter-checks each turned exactly their
test red. Two order tests stayed green without the fix at first and
were rebuilt. e2e 68 is new and passed with 5 own checks. Its
counter-check without arrange() failed as expected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 07:06:33 +02:00
adminandClaude Opus 5.5 73e675835e Add the composition asset with its tracks, items and clips (#417) 
A composition is a timeline of scene objects for cuts and cutscenes.
This first step brings the model and the pure evaluation behind it.
The tab and placing it in a scene follow in the next steps.

- CompositionAsset holds objects, tracks, items and clips. Each item
  owns exactly one object with overrides and container edits.
- The composition is a third object holder next to scene and
  container. Delete cascade, undo, usages and references work with it.
- Duplicating gives the copy new object ids and moves the item ids
  along with them.
- compositionAt() tells which items live at a frame. The top track is
  drawn in front.
- A clip acts on the item of its own track that it overlaps. After its
  end it holds the last value until the item ends. The animation's
  play mode is ignored, a clip is one pass.
- Clip targets are found by template path, else by name path, like
  the runtime's animation system.
- The root sprite of an item plays its video driven at offset plus
  time. Children follow unless they are already driven by hand.
- fittedLength() gives the video or clip length minus the offset, or
  3 s without media.
- fps (60) and loop live in their own properties file. Keeping them in
  PropertySchema would close an import cycle.
- The type is listed under Visuals with its own icon. It is not
  placeable yet.

Tests: the gate passed with 300 files and 4625 tests. That is 25 new
tests in tests/core/composition.test.ts and 2 icon checks that run
for every icon. Four counter-checks (track order, overlap rule, child
videos, ids on duplicate) each turned exactly their test red. e2e 04
passed with 67 checks and now counts 33 asset types.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-12 06:36:30 +02:00
adminandClaude Opus 5.5 597367c804 Keep a transition's pace through new steps and pans (#309) 
A wheel step in the middle of a transition restarted the ease-out
curve at full speed, so a run of quick steps kicked at each one.
A pan called finish() first and made the zoom jump to its target.
Both made the zoom and the gliding lists feel jerky.

- A new step starts from the speed the view already has and then
  eases out. Quick steps in one direction move in one stroke. A step
  the other way still starts from rest.
- A pan takes the transition along. The zoom goes on while dragging
  and lands where it was heading, moved by the pan. This covers all
  eight pan places: scene and container, graph, plugin board,
  texture and pixel art, timeline, pattern, song and sound.
- finish() is gone, it has no callers left. Dragging the scrollbar,
  a jump to a row, another document or a new view size still end
  the transition as before.

Found: on a second step the old curve jumped to 2.47 times the
speed the view had just before.

Tests:
- npm run check on stein2: 299 files, 4598 tests (5 new)
- e2e 46 on stein2: ALL PASSED, 24 checks (1 new). A pan by 40/30
  px during a 1000 ms zoom step leaves it moving (2.33 on the way
  from 2 to 3), it lands on 3 with the world point moved by the pan.
- Counter-checks: with the pace reset to 3 the new unit test fails
  with 2.47 instead of at most 1.1. Without carry the pan test fails.
- melektaus-atx went offline mid-run, so both ran on stein2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 23:08:43 +02:00
adminandClaude Opus 5.5 be41b1f0d7 Glide the asset and object lists and scroll past their end (#309)
game-maker / gate (push) Successful in 4m21s
Details
game-maker / suite (push) Canceled after 8m42s
Details 
The asset panel stopped with its last group flush on the bottom edge,
and both lists on the left jumped on every wheel step while the tabs
next to them already zoom with a transition.

- ScrollContainer gets tailSpace, empty room always kept below the
  content. Content that fits only without it scrolls by the rest. The
  content is made that much taller, so clicks and the context menu on
  that room work as on any empty part of the list.
- The asset panel keeps six rows of it. The other lists keep none.
- ScrollContainer gets wheelStep, which by default scrolls at once.
- scrollTween() in viewTween.ts hands the wheel step to an axisTween.
  The duration is the zoom transition's from the General settings,
  quick steps add up and stop at the end of the list, and dragging the
  bar or a jump to a row ends the glide where it stands.
- Only the asset panel and Scene objects use it. Inspector, menus and
  the console still jump.
- Wiki: Concepts describes the glide and the room below the assets.

Not checked in the browser by me. The user picked the six rows and
asked for the glide after trying the room below the list.

Tests:
- Unit scroll-container: 10 tests (3 new for tailSpace).
- Unit view-tween: 4 new with a fake clock (glide lands on target,
  quick steps add up and stop at the end, a jump ends it, 0 ms jumps).
- Gate on melektaus-atx: CHECK EXIT 0, 299 files,
  4593 tests (4586 before).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 22:14:54 +02:00
adminandClaude Opus 5.5 322fc39d16 Zoom the video tab's time smoothly like the other tabs (#377)
game-maker / gate (push) Successful in 4m41s
Details
game-maker / suite (push) Successful in 17m13s
Details 
The video tab jumped on every wheel step and on Fit. The Sound,
Pattern and Song tabs already ease the zoom since #377. The video
tab now takes the same transition, so all tabs feel alike.

- VideoCanvas runs wheel and Fit through `axisTween`. The duration
  comes from the General settings. Quick wheel steps add up, panning
  ends the transition, and the view that is stored is the one the
  transition lands on.
- Opening a video still fits at once without a transition. Only the
  Fit button eases, as in the Sound tab.
- `__gm.videoTab.zoom()` reports the zoom and whether a transition is
  on its way.

Tests:
- e2e 66 on melektaus-atx: ALL PASSED, 30 checks (1 new). The new
  check uses `sampleZoom` like the Sound tab. At 1000 ms the zoom is
  on its way two frames after the wheel (319 -> 321) and lands where
  the jump does (370.63). No counter-check against the old code.
- Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 21:52:29 +02:00
adminandClaude Opus 5.5 e191dc4b2e Record the game view from a behavior into a video asset (#405)
game-maker / gate (push) Successful in 4m14s
Details
game-maker / suite (push) Successful in 17m7s
Details 
A game can now film itself. A behavior starts and stops a take, and
the editor turns it into a video asset with the run's picture and the
sound the player hears. The exported game hands the file over as a
download, since there is no project to keep it in.

- New action nodes `Start recording` (Name, Resolution, FPS) and
  `Stop recording`. An empty name becomes `Recording <timestamp>`.
- Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes
  the height is set and the width follows the camera.
- The picture is the camera only, without hints, overlays or the
  letterbox. It is drawn into a render texture of the target size,
  so a smaller window still gives full 1080p.
- `PictureReader` reads the texture back through pixel buffers with
  a fence. The editor never waits on the GPU, and a frame with all
  three buffers busy is dropped and counted.
- Nothing of the take reaches the visible canvas. A first version
  drew tiles into it and copied them out, and with the low-latency
  canvas that midway state flickered on screen.
- The sound is tapped behind the master limiter in the FX worklet
  and encoded as Opus.
- H.264 is chosen when the browser has it, VP9 otherwise. A small
  own Matroska writer packs both tracks, no new dependency. It names
  the frame rate and BT.709 so ffmpeg keeps both.
- A second `Start recording` is ignored with a warning in the log.
  A take goes on through `Load scene` and a restart, pauses with the
  editor and is saved when the run ends. It stops at 2 GB.
- The editor uploads the take on the existing video route and makes
  the asset once the run is over. The video tab gets a Download
  button for the converted MP4.
- `downloadBlob` in `src/ui` replaces the song editor's own copy.

Findings:
- `On start` runs before the game screen knows the world. A take
  therefore fixes its size on the first frame that has a camera.
- Without a frame rate in the file, ffmpeg turned 30 fps into 59.94.
- The worker's poster JPEG is off in colour for every upload (red
  255 comes out as 229, BT.601). Not touched here.

Tests:
- Unit recording + matroska: 21 green, plus the node catalog cases.
  The Matroska test reads the file back with its own EBML reader.
- e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a
  1280x800 window gives a 1920x1080 MP4 with sound and no dropped
  frame. Every 4th pixel is checked, a green quarter catches an
  upside-down picture. A take ended by Esc is saved with the
  timestamp name.
- e2e 60j: the exported game downloads an `.mkv` with EBML magic.
- The flicker is gone on the user's machine without `?vsync`.
- Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 21:11:05 +02:00
adminandClaude Opus 5.5 36ffd9a61a Land a driven video's seek without a frame so it is never empty (#404)
game-maker / gate (push) Successful in 4m9s
Details
game-maker / suite (push) Successful in 17m18s
Details 
Since b9fcf48 a seek of a driven video only counted as done once
Chrome handed over the next frame. For some seeks Chrome hands over
none. The player then waited for good, the view left the sprite out
after 0.3 s, and the picture stayed empty until the next jump.

- A seek now lands after `seeked` with the next frame, or after
  150 ms at the latest. The worst case is a picture 150 ms late.
- The landing is set up in the `seeked` handler instead of `seek`.
  A frame from before the last seek of a chase no longer counts.
- `__gm.videos()` reports `landing`, so an empty picture can be told
  apart from a player that is still loading.

Findings:
- The empty picture could not be reproduced in Chrome headless.
  The test video is 320x180 and decoded in software. The user's
  videos are 1080p60 H.264, likely decoded in hardware.
- The user confirmed the fix in the editor.

Tests:
- Unit video-library + video-follow: 20 green (1 new). The new test
  plays a video that hands over no frame after a seek. With the old
  code it stays not ready, so it fails.
- e2e 66 on melektaus-atx: ALL PASSED, 29 checks. The new 66r cuts
  LATE ten times quickly and was also green with the old code.
- Gate on melektaus-atx: CHECK EXIT 0, 297 files, 4563 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 19:04:40 +02:00
adminandClaude Opus 5.5 8118b8042b Let Texture size feed the Vertex output to scale about the middle (#409)
game-maker / gate (push) Successful in 4m16s
Details
game-maker / suite (push) Successful in 17m10s
Details 
A scale about the picture's middle moves each corner by its distance
to the middle times (scale - 1). In pixels that distance needs the
picture's size, and Texture size was locked at the Vertex output as a
per-pixel node. So the size had to be typed in by hand and was only
right for pictures of exactly that size.

- At a corner Texture size reads the uniform uPictureSize. It holds
  the size the Vertex output counts its pixels in, the same number the
  node gives in the fragment. The offset is divided by that size again,
  so a scale comes out right for every picture and sprite size.
- A node can say what it is at a corner through emitPerCorner. The
  walker takes it there and emit everywhere else.
- Fed a Pass's picture, Texture size stays locked at the Vertex output.
  A Pass has no picture at a corner.
- 9-slice, field and camera shaders do not move corners. Their size is
  0 there and nothing changes for them.

Tests:
- Unit shader-vertex: the node compiles at a corner to uPictureSize
  and the Output still reads textureSize. shader-passes: a Pass's
  picture into Texture size at the Vertex output is refused.
- Against the old perPixel flag the new unit test fails.
- e2e 28l-iv: width * 0.5 at x moves the 32 px picture as far as the
  constant 16 did, same three colours. With the size forced to 0 on
  the GPU the check fails and the picture stands still.
- The case "Texture size is refused at a corner" left the it.each
  that lists refused nodes, so the test count stays the same.
- e2e 28 on melektaus-atx: ALL PASSED, 101 checks.
- Gate on melektaus-atx: CHECK EXIT 0, 297 files, 4562 tests.
- tsc, eslint, Prettier clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 18:35:20 +02:00
adminandClaude Opus 5.5 b9fcf48097 Show a cut into the middle of a clip without its old picture (#404) 
A switched-off driven video waits at the video time it has where it
is, before its first key mostly the start value. Switched on at
another spot, its player has to seek first. Until the seek is done the
texture still holds the old picture, so the start of the clip flashed
for a frame or two at the cut.

- A view holds its last picture while a driven video that just came
  into sight is still seeking, for at most 0.3 s. At a cut the old clip
  stays one frame longer and the right picture of the new one follows.
- A seek counts as done only when the browser hands in the next video
  frame (requestVideoFrameCallback), and the texture is uploaded right
  then. The seeked event alone was too early, one red frame was left.
- A seek that takes longer than 0.3 s leaves the sprite out until its
  picture is there. The wrong picture does not show either way.

Tests:
- e2e 66q switches a sprite off at 0.2 s (red) and on at 1.2 s (blue)
  and reads every frame in the browser. No red may show, blue must
  stand at the end.
- Against the old code 66q fails with 2 red frames before the blue.
  With only the seeked event it still fails with 1 red frame.
- e2e 66 on melektaus-atx: ALL PASSED, 28 checks, run twice.
- Unit video-library, video-follow: 19 passed.
- Gate on melektaus-atx: CHECK EXIT 0, 297 files, 4562 tests.
- tsc, eslint, Prettier clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 18:18:56 +02:00
adminandClaude Opus 5.5 3a1e420806 Read a driven video's speed on the frame clock so 1x stays 1x (#404)
game-maker / gate (push) Successful in 4m12s
Details
game-maker / suite (push) Successful in 16m58s
Details 
A driven video measures how fast its wearer's time runs and sets its
playbackRate from that. The time moves on with the ticker, but the
reading took its clock from performance.now() while drawing. A frame
that had more to do before the draw shifted that clock against the
time, and a 1x animation read as up to 1.09. Outside 0.9 to 1.1 the
video goes mute, so the sound could drop out for a moment. In CI this
made e2e 66k red now and then.

- VideoLibrary reads the speed with the timestamp of the frame
  (document.timeline.currentTime), the same clock the time runs on.
  It falls back to performance.now() where there is no document.
- debugInfo() reports the measured speed of a driven player.
- e2e 66k checks the measured speed and the correction of the rate
  apart. The rate alone may differ from 1 by up to 10 percent on
  purpose, so checking it at +-0.1 was too strict.

Tests:
- Unit video-library: 8 tests green, 1 new. With performance.now()
  back in place the new test is red and reads 1.089.
- e2e 66 on melektaus-atx: all passed, 27 checks, speed 1.000 in
  all 5 readings.
- Gate on melektaus-atx: CHECK EXIT 0, 297 files, 4562 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 17:49:45 +02:00
adminandClaude Opus 5.5 8679dfbe65 Group timeline rows by object and edit values in the row (#408) 
The rows of the animation timeline stood in the order they were keyed.
With several objects and properties the list was hard to read, and a
value could only be changed by picking a key and using the inspector.
The Play button also kept saying Pause after scrubbing had paused.

- Each object gets a header row with its name, indented by depth. The
  groups follow the scene hierarchy, the root first. Rows whose object
  is missing go to the end.
- Under an object the rows follow the order in which it lists its
  properties: X, Y, Z, then the order of the inspector. Only the view
  is sorted, the animation itself is unchanged and there is no undo
  step. After a move in the hierarchy the order is right by itself.
- The row head shows the same field as the inspector with the value at
  the playhead. Without keys it shows the value of the object, for a
  relative row the offset.
- A change in that field sets or changes the key at the playhead,
  selects it and is one undo step. A running playback pauses first.
- The Play/Pause button follows every change of the playing state, so
  scrubbing that pauses shows Play again.

Tests:
- Unit animation + timeline-view: 100 passed, 3 new.
- e2e 25 on melektaus-atx: ALL PASSED, 34 checks, 3 new (groups,
  order and values at frame 4, a typed 55 becomes a selected key on
  frame 4, Play then a click on the ruler shows Play and pauses).
- Gate on melektaus-atx: exit 0, 297 files, 4561 tests.
- tsc, eslint, Prettier clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 17:42:10 +02:00
adminandClaude Opus 5.5 dd60a795e1 Wait for driven videos before a run starts so no sprite flashes (#404)
game-maker / gate (push) Successful in 4m28s
Details
game-maker / suite (push) Failing after 16m24s
Details 
The game view opens a player for each driven video only when it draws,
and it draws only while its tab is visible. Without a detour over the
Game tab the players came up with the click on Play. Until a video had
loaded, its sprite showed its base colour, white in the user's cut.
Sharing the players of the scene view would not have helped. From the
Animations tab those are discarded after two to four seconds.

- Play holds the run at second 0 until every driven video in the game
  view has its first picture, at most 3 s. The status line says
  "Loading the videos… · Esc stops".
- The game clock stands while it waits. Behaviors and sounds start
  only after it, so picture and sound stay together.
- Load scene in a running game waits the same way.
- A driven video without a picture is not drawn at all instead of
  white. That holds everywhere, also on a switch to the Game tab
  without Play and at a cut whose video is not ready.
- The exported player waits the same way.
- __gm.playInfo() reports waiting for the e2e check.

Tests:
- e2e 66p puts the sprite in front of the camera and waits 4.5 s so
  the game view has no player left. Then it reads every frame of the
  run: 14 frames of waiting on #22252c at time 0, then blue, no white.
- Against the old code 66p fails: 14 frames #ffffff, then blue.
- Unit video-library: 7 passed, 1 new for loading().
- e2e 66 on melektaus-atx: ALL PASSED, 27 checks.
- Gate on melektaus-atx: exit 0, 297 files, 4558 tests.
- tsc, eslint, Prettier clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 17:07:53 +02:00
adminandClaude Opus 5.5 843b110bcb Hold a driven video at the end of its file instead of restarting (#404) 
A driven video plays on for about 100 ms after the last key, until the
speed reading says it stands. With the key close to the end of the
file, that runs the video past it. The browser then marks it ended,
and follow() called play() on it in every frame. Per the HTML spec an
ended video starts over from the beginning of its file on play(). The
sprite flashed the first picture one to three times before it held.

- An ended video now counts as standing at its end. follow() holds it
  and seeks to the wanted picture instead of playing it again.
- The end of the file counts as an end too, not only the end of the
  clip.

Tests:
- e2e 66o plays an animation in once mode to 1.95 s of the 2 s test
  video and records the player time in every frame. It must never jump
  back, and the last picture must stand.
- Against the old follow() 66o fails, the player jumps back to 0 once.
  A first version with the key at 1.9 s stayed green, the video never
  reached the end of the file.
- e2e 66 on melektaus-atx: ALL PASSED, 26 checks.
- Unit video-library, video-follow, video-clock: 31 passed.
- tsc, eslint, Prettier clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 15:46:03 +02:00
adminandClaude Opus 5.5 43fd22a763 Keep a driven video ready while its sprite is off (#404) 
A cut between two videos flashed white: the sprite switched on had
no picture yet, because its player only opened then or had been
dropped meanwhile. A driven video now waits muted at its time while
its sprite is off, in the editor and in a run.

- A disabled sprite with Video clock Driven asks for its player
  without drawing it. A Free video still opens when shown.
- A run's draw list keeps disabled driven sprites, marked as off.
  Before, collectWorldItems dropped every disabled object, so the
  game view never held a player for them.
- VideoLibrary.wear() counts as asking. The sweeper dropped a held
  player after 2 to 4 s and it was loading again at the cut.
- The speed of the asked time is read over a window that grows to
  500 ms. Over 100 ms one 1/60 s step more or less read as 17 % off,
  the playback rate jumped and the sound dropped out.
- A stream feeds every packet it has loaded to the decoder at once
  and waits once. One round trip per 20 ms packet let the decoder
  fall behind on a busy main thread, which made e2e 34 flaky in CI.

Tests:
- Unit streaming, video-library, video-follow: green, 4 new
- e2e 34 + 66 (atx): ALL PASSED, stream skipped 0, 66m new
- e2e 66 (atx), two runs: ALL PASSED, 25 checks, 66n new
- Gate (atx): CHECK EXIT 0, 297 files, 4557 tests
- Counter-checks: without the draw list fix 66n finds no player,
  without the wear() fix and with the old speed reading the new
  unit tests turn red

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 15:37:16 +02:00
adminandClaude Opus 5.5 774207208c Key a video to its end at 1x and zoom the timeline further out (#407) 
To play a clip at its real speed the user had to know its length,
work out the last whole frame by hand and then scroll far to the
right to set the end key. The timeline now does the sum, and it
zooms out far enough to see a long clip in one piece.

- The context menu of a Video time row gets "Video at 1x to its end
  from frame N". It keys the playhead with the row's value there and
  the last whole frame before the clip ends with the matching time.
- The start key turns linear and keys in between are removed. The
  whole run is one undo step (KeyStraightRunCommand).
- The clip is the one the sprite wears at the playhead, also when
  the texture is keyed in the same animation (rowVideoAt).
- The entry is greyed out for relative rows, without a video, while
  the length is unknown, or when the playhead is already at the end.
- The status line names the result, e.g. "frame 572 shows 57.2 s of
  57.27 s".
- The timeline zooms out to 0.02 px per frame instead of 4 px. The
  ruler labels follow a 1-2-5 step without an upper limit.

Tests:
- Unit video-clock + timeline-view: 27 tests green, 8 new
- e2e 66 (atx, alone): ALL PASSED, 23 checks, 66l new
- Gate (atx): CHECK EXIT 0, 297 files, 4553 tests
- Counter-check: the command keeping the keys in between turns
  exactly the command test red

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 14:51:20 +02:00
adminandClaude Opus 5.5 c9e8a3f6ac Let an animation drive a sprite's video picture by picture (#404)
game-maker / gate (push) Successful in 4m16s
Details
game-maker / suite (push) Failing after 16m12s
Details 
A video used to run on its own clock as soon as a sprite wore it. An
animation could switch the sprite on and off, but never say which
picture shows at which frame. Cutting a video in an animation needs
the opposite: the animation sets the time and the video follows.

- Sprites wearing a video or a clip get two rows, Video clock and
  Video time. They show only while the texture holds a video.
- With Video clock on Driven the video does not play by itself. It
  shows the picture at Video time, counted from the clip's start.
  Past the end it goes round with Loop and holds the last picture
  without.
- Video time is a number like any other, so keys, step, easing and
  curves give stills, slow motion and ramps.
- A driven sprite gets a player of its own, keyed by the wearer's
  name. The same clip can run twice at different times.
- Following has two ways (`videoFollow.ts`). Running forward at 0.25x
  to 4x the video plays at that rate and is nudged. A still, a scrub
  or a step backwards seeks, and only the newest seek counts.
- Sound is heard only near 1x. The animation tab now plays it too, so
  a cut can follow the sound. Scrubbing is silent.
- Scene, game view, the inspector's PREVIEW and the shader preview
  show the driven picture. In a running game Play animation sets it.

Found on the way:

- After a scrub the video in the animation tab kept running. The
  preview redrew only while the playhead moved, so nobody asked for
  the video once it stood. `AnimationEditor.tick` now redraws it every
  frame while it shows a video.
- Every sprite has two more defaults (`videoClock: 'free'`,
  `videoTime: 0`). Eight tests in five files pinned the old list and
  are updated.

Tests:
- Unit: video-clock and video-follow new, 4 files with 42 tests green.
- e2e 66 (melektaus-atx): ALL PASSED, 22 checks, 3 new. 66i: two
  sprites on one clip stand at 0.2 s red and 0.7 s green. 66j: a
  scrub to frames 2, 7, 12 and 3 shows red, green, blue, red, and
  each colour still stands 0.7 s later. 66k: playing runs 10-30 ms
  behind at rate 1.00-1.03 and is not muted.
- Counterchecks: a seek that keeps the rate turns the seek test red.
  Not seeing Driven turns 66i, 66j and 66k red.
- Gate (melektaus-atx): exit 0, 297 files, 4546 tests.
- Not tested in the browser: Play animation in a running game.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 14:23:07 +02:00
adminandClaude Opus 5.5 a62d1d0faf Give e2e 66 its own lane so the video sprite is not covered (#406)
game-maker / gate (push) Successful in 4m36s
Details
game-maker / suite (push) Successful in 16m31s
Details 
In the full suite e2e 66f saw the same purple #cc00f4 fourteen times
instead of the clip's green and blue. Alone the module was green. The
video sprite stood at (0, 0), and there earlier modules keep objects
on purpose that later modules still need. Every other module has its
own lane for that reason, only 66 had none.

- scaffold.mjs: new lane `video` at y -7000.
- 66-video.mjs: the sprite and lookAt use that lane, and the cleanup
  looks back at (0, 0) like the other lane modules do.

Found with a diagnosis in a full run on melektaus-atx: under the
sprite's centre lay o3, otex (z 900) and filler40 to filler1 (z 5)
from e2e 10 to 12, only then vd66-object. Beside them the sprite
showed green as it should. The purple is otex's texture in the state
later modules leave it in. It has nothing to do with video seeking:
task 645 saw the same colour while seeking was still broken.

Tests:
- e2e 10-12 + 66, old state (melektaus-atx): 66f FAIL, otex in front
- e2e 10-12 + 66, with the fix: ALL PASSED, 81 checks, 66f sees
  #00ff01 and #0000ff
- Prettier, eslint: clean

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 13:40:43 +02:00
adminandClaude Opus 5.5 ce933a4f32 Let a drag change only the picked clip in the Sound and Video tab (#403)
game-maker / gate (push) Successful in 4m15s
Details
game-maker / suite (push) Failing after 15m40s
Details 
A drag with a clip picked could span a new clip in the strip or span
the picked one anew over the wave. Both happened by accident while
working on a clip. Now a drag only makes a new clip while none is
picked. With one picked it moves its edges or the whole of it.

- With no clip picked, a drag in the strip or over the wave (in the
  Video tab the track) spans a new clip, as before.
- A click over the wave or in the track picks the clip at that point,
  as a click on its band does. Beside the clips it picks the sound or
  the video. The picked clip wins where clips lie over each other.
- With a clip picked, its handles move its edges as before. A drag on
  its band in the strip moves the whole clip, its length kept and
  inside the file. Any other drag does nothing.
- The rules live in waveView.ts (clipGrip, clipToPick, movedSpan) and
  both tabs use them. The cursor shows grab over the picked band.
- The Video tab's track draws the clips not picked faint, as the wave
  does, so a click there has something to aim at.

Tests:
- Unit wave-view.test.ts: 16 tests (4 new)
- Counter-check: a strip drag beside the picked clip spanning a new
  one again fails the picked-clip rule test
- e2e 34 + 66 (atx): ALL PASSED, 162 checks, 1 new check in each
- Gate (atx): 295 files, 4527 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 12:51:46 +02:00
adminandClaude Opus 5.5 c815d2c35f Keep a playing video when the window comes back to the front (#402) 
Chrome draws no frames for a hidden page, so the scene stopped asking
for its videos. The sweeper of the VideoLibrary kept running and
dropped every player after 2 s. Back in the window, the video started
over, and Pixi warned that a texture source was destroyed while still
bound to a shader.

- The sweeper skips a hidden page. The time hidden does not count as
  idle, the 2 s start once the page is shown again.
- A dropped player only unloads its video source instead of destroying
  it. A shader that still holds the source no longer warns.
- CI serves videos through a new router for php -S. The built-in
  server answers no Range requests, so Chrome could not seek. e2e 66
  had 4 red checks in the suite job of d48f174 because of it. nginx on
  stein2 hid this.

Tests:
- Unit video-library.test.ts (new): 3 tests
- e2e 66 (atx): ALL PASSED, 18 checks
- Router checked with curl: 206 with the right bytes
- Counter-check: the old sweep fails both hidden-page tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 12:32:59 +02:00
adminandClaude Opus 5.5 d48f174caf Play videos on sprites, converted on the server with clips (#402)
game-maker / gate (push) Successful in 4m15s
Details
game-maker / suite (push) Failing after 15m49s
Details 
A video becomes an asset like a sound: uploaded once, converted to one
format by a worker of its own, cut into clips in its tab, and worn by a
sprite through its Texture row. Only Chrome is a target, without alpha.

- Server: POST ?r=videos queues the upload under the hash of its bytes,
  so the MP4's name is known at once. The worker (VideoWorker over
  VideoFormat) writes H.264/AAC, yuv420p, constant frame rate, a
  keyframe every 15 frames, at most 1920x1080, with a poster and the
  details beside it. A failed job says why and is tried again on the
  next upload.
- An upload may be up to 2 GB. It is streamed from the request to disk
  and hashed there, so php-fpm holds none of it in memory. The other
  routes keep their 256 MB.
- The worker runs in the container game-maker-video on stein2, a
  compose project of its own (tools/video/), and in CI beside the PHP
  server. The CI image has ffmpeg for it.
- Model: asset type video with size, length, frame rate and sound, and
  clips like a sound's. The clip helpers and commands serve both.
- A sprite's Texture row offers videos and clips after the pictures.
  resolvePicture names a playing video by a key, and the texture library
  hands it to a VideoLibrary with one player per key.
- The editor shows a placed video as a quiet preview. Play starts it
  over at its clip's start and lets it be heard through the master bus,
  a pause holds it, Esc makes it quiet again.
- Video tab: the picture, a ruler, the clip strip and a track on the
  Sound tab's time axis. A drag spans a clip, a handle moves an edge,
  Play plays the picked clip.
- The picker shows a video on its way with its progress and adopts an
  upload from it once it is ready. Dropping a video file does the same.

Decided on my own:

- Two sprites wearing the same clip share one player and show the same
  picture. Control per instance belongs to the follow-up with nodes.
- A video plays in real time, the game's time scale does not slow it.
- The container may use 3 of the 4 cores of stein2.

Findings:

- The first real upload (279 MB) got 413: the API's own ceiling of
  256 MB for every request, not nginx or the Brandmauer.
- Chrome has no picture of a video that never played, and a Pixi
  VideoSource drawn before the first frame stays empty for good. The
  texture is made in the first frame callback (openVideo).
- A player nobody asks for in 2 s is let go. The inspector's PREVIEW
  asked only on an edit, so with the sprite not on screen its texture
  was destroyed under it and the batcher threw on every frame. The
  preview now redraws every frame while it shows a video, like a Time
  shader. A let-go texture keeps an empty source instead of none.
- VideoSource.load() reloads the element and cuts off an earlier
  play(). Playing starts after loadeddata.

Tests:

- PHP videos.test.php: 68 checks, with the real ffmpeg
- 300 MB upload through Brandmauer and nginx: 201 in 21 s, php-fpm at
  13.7 MB RSS at most
- Unit videos.test.ts: 20 tests
- e2e 66 (stein2, atx was off): ALL PASSED, 18 checks; e2e 04 with the
  new asset count green
- Gate (atx): 294 files, 4520 tests. serialize.test.ts used video as
  its example of an unknown type and now uses a made-up one.
- Counter-checks: a keyframe gap of 30 fails the PHP check, a key that
  keeps the video's end fails 2 unit tests, a clip without its end
  fails "plays only its stretch" in e2e 66

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-11 12:12:39 +02:00
adminandClaude Opus 5.5 2f376262d3 Let the Reverb's screen ring out and its reflections drift (#401)
game-maker / gate (push) Successful in 4m13s
Details
game-maker / suite (push) Successful in 16m5s
Details 
The round screen of the Reverb stood still. It now moves, only for
the look of it, and says nothing it did not say before.

- The sound rings out again and again in real time. The dot in the
  middle flashes, and a thin front runs out over the rings as fast as
  the tail does. It fades as the tail falls, then 0.8 s of quiet.
- A reflection lights up for a moment once the front has passed it.
- The reflections turn slowly round the dot, each at its own pace and
  way (at most 0.16 rad/s). They sway 1.5 px in and out and twinkle.
  Their drift comes from a noise of its own, so they sit where they
  always did.
- Rings, grid and dashed end stay one drawing. Only the dots, the
  front and the flash are drawn again each frame.
- Nothing moves without a tail to see: no sound picked, or wet at 0.

Tests:
- reverb-view: 13 tests green (3 new)
- e2e 34 on melektaus-atx: ALL PASSED, 144 checks
- gate on melektaus-atx: 293 files, 4498 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 23:32:49 +02:00
adminandClaude Opus 5.5 742f240a62 Make the Bitcrusher and Delay plates shorter (#399, #393) 
Both devices were 280 tall, but their knobs end well above that. The
gap between the lamps or levers and the knobs was wide, and below the
knobs about 70 units stayed empty.

- The Bitcrusher is 680x220. Its knobs sit 14 higher, closer to the
  row of bit lamps.
- The Delay is 680x224. Its knob groups start 16 higher, closer to the
  two levers.
- The screen's frame is now worked out from the plate's bottom edge.
  It stays as far from the bottom as from the side, so the screen
  shrinks with the plate (116 and 120 tall).

Tests:
- crusher-view and delay-view: 30 tests green
- e2e 34 on melektaus-atx: ALL PASSED, 144 checks
- gate on melektaus-atx: 293 files, 4495 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 23:16:47 +02:00
adminandClaude Opus 5.5 6383cc0d8e Show a Sample node as a device that draws its waveform (#400)
game-maker / gate (push) Successful in 4m14s
Details
game-maker / suite (push) Successful in 16m49s
Details 
The Sample was the last node of an instrument that still showed rows.
It now draws the sound or clip it plays, so the loop and what plays
after the note can be seen instead of read off two numbers.

- SamplePanel, SampleFace and sampleView: a compact gold plate of
  480x280 with the sound picker beside the name, a waveform screen,
  the four ways of playing it as buttons two by two, the root as a
  knob that reads as a note, a lever for following the note and the
  loop's start above its end.
- The waveform comes from the library's peaks of the played sound,
  a clip's own cut included, scaled by the sound's volume. It is
  loaded on demand as the Reverb's sound is.
- Each part has its colour: played once, looped, played after the
  note, never heard. The loop is held as playback holds it: an end
  of 0 or past the sound is the end, a start not before it loops
  all of it.
- Idle settings come from the node's own idleParams, so the device
  greys out what the rows did.
- A new instrument places its Output at x 840, past the device.
  At 460 it lay under the Sample.
- Hook __gm.sample.panel and a check in e2e 34 behind the first
  instrument check.

Tests:
- Unit sample-view: 11 tests, new. pattern and chip-presets green.
- e2e 34 (atx): ALL PASSED, 144 checks (1 new).
- Gate (atx): 293 files, 4495 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 22:54:17 +02:00
adminandClaude Opus 5.5 a6c00f7b8b Show a Bitcrusher node as a device that draws its pixels (#399)
game-maker / gate (push) Successful in 4m11s
Details
game-maker / suite (push) Failing after 15m21s
Details 
A Bitcrusher only said what it does in rows of numbers. How coarse
its steps and holds are could not be seen. Now the node is a device
of its own size. Its screen sends a sine through the node's own crush
and draws what comes out in cells, as tall as a step and as wide as a
hold, so the coarser the crush, the coarser the picture.

- CrusherPanel draws a 680x280 plate in bubblegum pink. Bits and rate
  are large knobs, wet is small. Rate gives its low end more of the
  turn, 8 kHz sits just past the middle.
- crushTrace runs a 440 Hz sine at 0 dBFS through Crusher at 48 kHz
  and blends it as the worklet does. Where each hold starts comes from
  Crusher too: a ramp through it at 16 bits takes a new value per hold.
- cellOf shapes a hold. From 5 px a step or hold gets a grid line and
  a gap. Finer steps give a 2 px bar. A cell is never more than twice
  as tall as wide, so 1 bit gives bars at -1, 0 and 1.
- The screen shows two periods, or 6 holds where the rate is low. The
  sine going in is a grey ghost. Below wet 1 the heard line lies over
  the cells.
- Above the screen a word for how crushed it is, CLEAN to BROKEN.
  Under it the noise the bits leave and where the rate folds back.
- 16 lamps show the bits kept, the highest on the left. A CRUSH lamp
  beside the name glows the more it crushes.
- COLUMNS_OF gives the Bitcrusher 4 columns in the presets.
- Hook __gm.bitcrusher.panel(id) for the sound FX and instrument
  graphs.

Decided myself:

- The screen is still. Nothing on it moves, so tick() says false.
- How crushed counts whichever of bits or rate takes more, the rate in
  octaves, times wet.

Found: at 8 kHz the first hold is 7 samples long, the rest 6. Six
sixths add up to just under 1 in Crusher. The worklet does the same,
so the screen shows what is heard.

Every sfxr effect preset ends in a Bitcrusher and gets 3 columns
wider. The Laser preset still fits whole on Fit.

Tests:
- Unit: crusher-view 13 tests (new). The screen matches Crusher with
  its wet on three settings.
- Counter-test: without the wet in the blend the worklet test fails.
- e2e 34: ALL PASSED, 143 checks (1 new).
- Gate: CHECK EXIT 0, 292 files, 4484 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 22:05:47 +02:00
adminandClaude Opus 5.5 b145e1f288 Draw the Flanger's comb as a thin line with a fading trail (#398)
game-maker / gate (push) Successful in 4m8s
Details
game-maker / suite (push) Failing after 16m2s
Details 
The comb was still drawn too heavy. Each frame showed the whole band
the comb had swept over the last 1/20 s, edged by two lines 1.5 wide,
and the faster the delay moved, the thicker that band got.

- flangerLine draws the comb where the light is as one line, 1 wide.
  It runs through the ends of each column and the peaks and notches
  in it, where they are. A comb finer than a column zigzags over it.
- Behind the line a faint trail shows what the comb swept over the
  last 1/20 s, and over the last 1/40 s once more, so it fades with
  age. A Flanger that stands still has none.
- The line fades out where its teeth move more than a quarter of a
  tooth within the trail (clearAt), and is gone from half a tooth on.
  There only the calm band of the trail is left, so nothing flickers.
- responseFrame gives the screen the line, how clear each point is,
  the sharp columns for the notches and the trail. It replaces
  responseColumnsOf, and RESPONSE_BLUR is now RESPONSE_TRAIL.
- The notches are counted on the sharp comb again, not on the band.
- The Phaser's response is drawn as the same thin line, without a
  trail.

Tests:
  Unit modulation-view (stein2): 26 tests, 2 new. The line holds the
    comb's dB at each point and every notch. On a fast swing it fades
    out over 8 kHz, stays clear under 1 kHz, and the trail is the same
    band over 16 kHz in three frames. A still Flanger has no trail.
  Counter-check: with clearAt always 1, the fade test fails.
  e2e 34 (atx): ALL PASSED, 142 checks.
  Gate (atx): CHECK EXIT 0, 291 files, 4471 tests.
  Pictures of the default Flanger, a fast one (2 Hz, 5 ms) and one
    sweeping to 20 ms, and of the Phaser: a thin line, the trail
    behind it, a calm band high up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 21:30:03 +02:00
adminandClaude Opus 5.5 f9a953f3d3 Keep the Flanger's comb calm and smooth where its teeth crowd (#398)
game-maker / gate (push) Successful in 4m24s
Details
game-maker / suite (push) Failing after 16m35s
Details 
High up the Flanger's comb flickered and looked pixelated. Each column
of its screen took six readings, and its teeth pass there faster than
frames are drawn, up to 0.4 of a tooth a frame at 15 kHz by default.
Each frame caught other teeth, and each column was drawn as a sharp
rectangle.

- flangerColumns reckons each column exactly. The response only hangs
  on the phase, and between two multiples of pi it runs one way. So a
  column holds its lowest and highest at its ends or on the peaks and
  notches inside it.
- Each frame is blurred over the delays of the last 1/20 s
  (RESPONSE_BLUR), about what an eye blends. Where teeth pass faster
  than that, they become a calm band from the deepest notch to the
  highest peak. Further down, where they can be followed, they stay
  sharp and move.
- The response is drawn as a band between two smooth lines through
  the columns, no longer as one rectangle per column.
- responseColumnsOf picks the Phaser's readings or the Flanger's exact
  columns. responseDb had no caller left and is gone.

Measured before choosing the blur, as the mean change of a column from
one frame to the next on the default Flanger (2-5 / 5-10 / 10-20 kHz):

  readings as before   5.6 / 10.6 / 14.3 dB
  exact, no blur       5.5 / 10.2 / 12.9 dB
  exact, 1/60 s        5.0 /  7.6 /  4.0 dB
  exact, 1/20 s        3.9 /  2.6 /  0.04 dB

Under 5 kHz most of what is left is the teeth really moving.

Tests:
  Unit modulation-view (stein2): 24 tests, 2 new. The exact columns
    hold what 4000 readings a column find there, and a fast swing
    leaves the same band over 16 kHz in three frames in a row.
  Counter-check: without the peaks and notches inside a column, both
    new tests fail.
  e2e 34 (atx): ALL PASSED, 142 checks.
  Gate (atx): CHECK EXIT 0, 291 files, 4469 tests.
  Pictures of the default Flanger and of one sweeping to 20 ms, near
    and far, two of each 40 ms apart: smooth edges, the band the same.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 21:15:53 +02:00
adminandClaude Opus 5.5 b6749cc43d Show Tremolo, Vibrato, Phaser and Flanger as devices that swing (#398) 
The four effects that swing were still drawn as rows of numbers. Like
the Delay, the Saturation and the Limiter, each node now shows a device
of a fixed size. Its screen draws what it does over the first seconds
of a sound, and a light plays that in real time, over and over.

- One device for all four (ModulationPanel, ModulationFace and
  modulationView), as KnobPanel is for Gain, Pitch, Mix and Pan. A short
  description gives each its title, plate, buttons and knobs.
- Tremolo, 640x280 in cream: a wave its volume shapes, the volume as
  its outline. It says DOWN TO 50%, EVERY 200 ms and DIPS -6 dB.
- Vibrato, 640x280 in plum: the pitch in semitones, cool below and warm
  above. Under a wet below 1 the dry pitch stands beside it. It says
  how late the voice is (LATE 0.93 ms).
- Phaser, 760x280 in mustard, and Flanger, 680x280 in steel blue: the
  response from 20 Hz to 20 kHz, and under it a lane of the turn or the
  delay over time. The response is drawn where the light is, so the
  notches swirl and the comb jets. A comb finer than a column fills it.
- The light loops over 1, 2, 4 or 8 s, two LFO swings at least, for a
  Flanger as long as its sweep takes to reach 20 ms. The face only
  keeps the editor drawing while something swings and is heard.
- A Rate lamp beside each name goes from dark to lit with the swing.
- Nothing is listened to. The pictures use the worklets' own numbers:
  lfoAt, vibratoSpeedUp (now exported from lfo.ts), the allpass chain
  with its feedback, and the comb, reckoned at 48 kHz.
- Phaser and Flanger take 4 preset columns. Phased Pad was looked at
  in a picture, nothing overlaps.
- Exported from nodes.ts: LFO_SHAPE, lfoShapeOf, TREMOLO_ROWS,
  VIBRATO_ROWS, PHASER_STAGE_ROW and phaserStagesOf. FLANGER_MAX_MS is
  exported from phaser.ts.

Decided here, as the request left the look open:

- A notch counts where the response falls below -3 dB, not some dB
  under the loudest point. Feedback rings peaks far above 0 dB, and the
  first rule counted 7 notches on a Phaser of 12 stages.
- Rate and Depth of Tremolo and Vibrato are big knobs, Wet is small.

Tests:
  Unit modulation-view (stein2): 22 tests, new. Phaser and Flanger
    give the dB of their worklets on a sine within 0.005 dB, the
    Tremolo its samples, the Vibrato the pitch it reads at.
  Counter-check: with the sign of the Phaser's feedback flipped in one
    place, exactly the worklet test fails.
  e2e 34 (atx): ALL PASSED, 142 checks, 4 new. Each device has its
    size, its ports outside, a light that runs by itself, and a button
    and a knob as one undo step each. Picking 8 stages on the Phaser
    draws 4 notches.
  Gate (atx): CHECK EXIT 0, 291 files, 4467 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 21:02:50 +02:00
adminandClaude Opus 5.5 28909f0adf Show a Limiter node as a device that holds what it took away (#397)
game-maker / gate (push) Successful in 4m23s
Details
game-maker / suite (push) Successful in 16m6s
Details 
A Limiter only said what it does in rows of numbers. How much it took
away, and when, could not be seen at all, and a short sound is over
before any needle could show it. Now the node is a device of its own
size, and its screen catches the reduction from the moment a sound
starts and holds it once the sound ends.

- LimiterPanel draws a 720x280 plate of brushed brick. The threshold
  knob is large, ceiling and release are small beside it, auto-gain is
  a lever. A LIMIT lamp lights while it takes something away.
- The screen hangs the reduction down from the top, 0 to 24 dB by the
  square root like the Compressor's meter. The level coming in runs
  along the bottom, orange above the limit. Time runs from the start
  of the sound over 1, 2, 4 or 8 s, and a longer sound rolls.
- ReductionCapture in limiterView holds the capture after 0.4 s of
  silence, without the silence, until the next sound. It writes the
  most taken away and when, and NOW or LASTED on the right.
- The Limiter is tapped now, like the Compressor. The face reads the
  loudest sample since the last frame on both sides.
- SpectrumTaps.levels takes a lateBy. The side in front is read 5 ms
  earlier, so a peak going in is met by what it became.
- limiterReduction leaves the auto-gain out and solves Wet's blend for
  the limited part. Less than 0.2 dB counts as none.
- A capture is also held when the node leaves the picture. Changing a
  knob or the lever leaves it as it is.
- Hook __gm.limiter.panel(id) for the sound FX and instrument graphs.

Decided myself:

- The capture stays held until the next sound, not for a fixed time.
- The screen has no wet knob, as the Compressor's has none.
- The Limiter's two taps cost two gains of 1 per voice. The analysers
  only exist while the node is in the picture.

Measured on stein2: the seeded blip at -4.4 dBFS against -12 dB showed
7.56 dB. Three hits at 0, -8 and -2 dBFS showed 11.9 dB at the first.
In e2e 34 on melektaus-atx, -10.44 dBFS in against -20 dB showed
9.56 dB, the difference to the hundredth.

Tests:
- Unit: limiter-view 13 tests (new), spectrum 6 (1 new), soundfx 67
  with the Limiter's taps.
- Counter-test: without Wet's share in the formula the blend test
  fails.
- e2e 34: ALL PASSED, 138 checks (1 new).
- Gate: CHECK EXIT 0, 290 files, 4445 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 20:30:59 +02:00
adminandClaude Opus 5.5 80b25488ce Show a Saturation node as a device that glows with its drive (#396)
game-maker / gate (push) Successful in 4m5s
Details
game-maker / suite (push) Successful in 16m2s
Details 
A Saturation in the graph was a stack of rows. The user wanted a device
of its own style and size, like the Delay and the Reverb, that shows
warmth and heat the harder the sound is driven.

- The node is a copper plate of 720x280 with a button per type, the
  auto-gain lever, a big drive knob and small knobs for tone, output
  and wet. Each change is one undo step, wired numbers lock their knob.
- The screen draws a sine at -12 dBFS, the level auto-gain is reckoned
  at, as a grey ghost. Over it is the wave that comes out: bent by the
  type, its DC taken out, auto-gain and output after it, blended by
  wet. Under it stand the auto-gain and the peak.
- The heat is the drive over 48 dB, as far as wet lets it be heard.
  The wave goes from cold steel through ember red to white, a glow
  rises from below and flickers, and embers fly up from about 8 dB on.
  A lamp beside the name glows with it.
- The node keeps the editor drawing only while there is heat. Without
  drive it draws nothing, unlike the Delay, which always animates.
- The Reverb's private type button is now PickButton with its own
  colours, and the Saturation's type buttons use it as well.
- No instrument preset uses a Saturation, so no preset needs more
  columns.

Tests:
- Unit saturation-view: 14 new tests.
- Counter-check: without taking out the DC, exactly the DC test fails.
- e2e 34 (atx): ALL PASSED, 137 checks, one new for type, lever and
  drive with undo.
- Gate (atx): CHECK EXIT 0, 289 files, 4431 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 20:05:11 +02:00
adminandClaude Opus 5.5 8c53ce0cbe Replace the 5-Band-EQ with the Parametric EQ and remove it (#395)
game-maker / gate (push) Successful in 4m13s
Details
game-maker / suite (push) Successful in 16m21s
Details 
The Parametric EQ does all the 5-Band-EQ did and more, so the editor
keeps only one EQ. The old node was used in one project only, with
nothing linked to its band inputs.

- "Garden of Sacrifices" was converted once on the project file, not
  by a load migration (#137). Its 5 nodes are Parametric EQs now with
  the same filters: low shelf 80 Hz, peaks at 250 Hz, 1 kHz and 4 kHz
  with Q 1, high shelf 12 kHz. A band at 0 became no band.
- Two values below -24 dB were held to -24, as the Parametric EQ goes
  no further. Both are in the asset "Delay" (high -60, low -36). The
  user will rework them by ear.
- The write went through the API's own Store (lock, revision 20181 to
  20182), so a tab with the old revision gets a 409 instead of
  overwriting it.
- Removed with the node: its stage kind "filter" and the type
  FxFilter, which nothing else made, and the player's branch for it.
  A Pan is now the last branch of fxNodes, a Flanger that of stagesOf.
- Tests that took the 5-Band-EQ as "some effect" use the Pan or the
  Parametric EQ now. The test for a parameter feeding a band now
  feeds the cutoff of a Filter, which had no such test yet.
- A project that still names the type, such as an old snapshot of
  "Garden of Sacrifices", loads without those nodes. Unknown types
  were always left out on load.

Tests:
- Unit (stein2): soundfx 67, instrument 36, envelope 13, sound-library
  127, all green
- Counter-check: with the cutoff not taken from the parameter, only
  the new Filter test goes red
- The converted project loads and compiles: Delay, Vocals and Drums
  play their bands (3+1, 4 and 5)
- e2e 34 (atx): ALL PASSED, 136 checks, the palette check now looks
  for the Parametric EQ
- Gate (atx): CHECK EXIT 0, 288 files, 4417 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 19:46:03 +02:00
adminandClaude Opus 5.5 cb3183227b Keep drags at full speed in large projects (#394) 
Dragging a band of a Filter or a Parametric EQ ran at about 30 fps
in a large project. Each frame of a drag announces the changed
project, and every listener of that event ran, also those whose work
nobody sees while the drag goes on.

- FramePreview marks its own frame, read through isDragFrame().
- The Scene view behind another tab skips such a frame and draws on
  coming back, as it already did while a game runs (#265).
- preloadProjectFiles skips it as well. It walks every asset and
  checks every bake, which costs milliseconds in a large project.
- The command that ends a drag emits for itself, and a drag that
  changes nothing ends with a plain preview, so both catch up then.
- This holds for every drag through FramePreview, the knobs of the
  devices in the graph included.

Measured on stein2 with a copy of a project of 162 assets, a band of
a Parametric EQ dragged at 250 pointer moves a second for 3 s:

- Listeners per frame: 17 ms before, 7.4 ms after.
- Hidden Scene view: 814 ms before, 53 ms after.
- preloadProjectFiles: 514 ms before, 0.4 ms after.
- What is left is mostly the Hierarchy and Assets panels on screen
  (about 3 and 1.3 ms a frame). They stay as they are.

Tests:

- Unit frame-preview, play-throttle: 29 passed (1 new)
- Counter-check: without the mark the new test fails
- e2e 34 (melektaus-atx): ALL PASSED, 136 checks
- Gate (melektaus-atx): CHECK EXIT 0, 288 files, 4417 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 19:45:59 +02:00
adminandClaude Opus 5.5 3f7eb03143 Check what a streamed sound holds by the library's own count (#378)
game-maker / gate (push) Successful in 4m8s
Details
game-maker / suite (push) Successful in 16m22s
Details 
The e2e check that a streamed sound costs less than a decoded one
compared how much the renderer process grew. Alone, module 34 saw it
grow by 1 to 8 MB while streaming. After the 33 modules before it in
CI it grew by 25 to 66 MB, and suite went red (CI tasks 611, 614 and
617, the last with 66.7 MB streamed against 95.9 MB decoded).

- SoundLibrary.stats tells decodedBytes, the samples of every file
  decoded whole, and streaming.held, every head of a copy and what is
  sent ahead to the worklet.
- StreamFile.headBytes and StreamFeed.ahead give their share.
- The e2e check now asks the library: streamed it holds less than
  4 MB more than before and nothing decoded whole, decoded it holds
  more than 40 MB, the five minutes being 50 MB at 44.1 kHz.
- The growth of the renderer stays in the log of that check. The
  counter-proof that the renderer grows by more than 40 MB when the
  sound is decoded stays as it was.

Tests:
- Unit sound-library: the streaming test also checks the bytes held.
- e2e 34 (melektaus-atx): 136 checks. Streamed 0.76 MB more held and
  none decoded whole, decoded 50.5 MB. The renderer grew by 4 MB and
  49 MB.
- Gate (melektaus-atx): 288 files, 4416 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 19:01:00 +02:00
adminandClaude Opus 5.5 d4a45c127b Show a Delay node as a device with its echoes over time (#393) 
After the Chip, the Envelope, the Reverb and the one-knob devices, the
Delay is drawn as a device of its own size too. Its screen shows what
time, feedback, damping, wet and ping-pong make of a sound, and a light
runs over the echoes at the beat they come at.

- DelayPanel: a 680x280 plate in the Compressor's brushed aluminium,
  now exported as ALUMINIUM. A screen, levers for Sync and Ping-pong,
  knobs for Time, Feedback, Damping and Wet, and an Echo lamp.
- The screen draws the sound as a white bar and each echo as a bar one
  step later. A bar is as tall as the echo is loud, evenly in dB to
  -60. It turns from warm white to rust as the damping takes the highs.
  With ping-pong the bars alternate above and below, the first left.
- At most 24 bars are drawn, and the screen says how many more are
  heard. Under it stand the step and the end of the tail, above it
  where the highs of the first echo end.
- The running light: DelayFace.tick returns true while the node is in
  the picture, so the editor keeps drawing, as for the Particles tab.
  The light goes round the span in real time, a bar glows as it is
  passed, and the lamp blinks with every echo but not with the sound.
- On beats the screen counts at 120 BPM, as the tempo of the voice is
  not known in the graph. The time knob then names the note, dotted
  and triplet ones included. In seconds, 2 s and up reads "2 s max".
- delayView holds the rules apart from the drawing: layout, tapers,
  echoTrain, echoCutoff for k rounds of the Butterworth low-pass,
  sweepAt and the texts. At wet 0 the rest is drawn faint.
- delayEchoes in the core counts the echoes heard. The tail of a voice
  and the screen use it both. DELAY_ROWS, DELAY_SYNC and PING_PONG are
  exported.
- Knob.reformat writes the number anew when its format changes, here
  when Sync switches between seconds and beats.
- The test hook __gm.delay.panel returns knobs, levers, idle settings,
  the echoes drawn, the texts and where the light is.
- Instrument presets give the Delay 4 columns. Echo Lead and Bell use
  one. In the picture of Echo Lead nothing overlaps.

Found on the way: a node with a face draws no setting rows. The e2e
check of a bypassed Delay (#354) read "Bypass: true" from them. It now
checks the faded node and the stages heard, as before.

Tests:
- Unit delay-view: 17 tests, new. Counter-check: the first ping-pong
  echo on the right turns the side test red.
- e2e 34 (melektaus-atx): 136 checks, 1 new.
- Gate (melektaus-atx): 288 files, 4416 tests.
- Pictures looked at: default, ping-pong, beats with damping, 83
  echoes at 50 ms, wet 0, and the preset Echo Lead.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 18:51:06 +02:00
adminandClaude Opus 5.5 389d9555fa Show a Pan node as a one-knob device in cyan (#392)
game-maker / gate (push) Successful in 4m13s
Details
game-maker / suite (push) Failing after 15m23s
Details 
After the Gain, the Pitch and the Mix, the Pan is the next node that
is just one number. It now gets the same device in the graph instead
of a row for its pan.

- PAN_DEVICE: the Gain's 160x160 plate in cyan with one bipolar knob
  for Pan. The middle stands straight up, and the turn goes evenly to
  L and R, which are engraved at its ends like A and B on the Mix.
- PAN_TAPER in hardwareView is 1, so left and right are heard alike.
  Device and unit test read the same constant.
- PAN_ROW is exported from the core, so the knob has the node's range.
- A linked Pan greys the knob, as for the Gain.
- The test hook __gm.pan.panel returns the same shape as the Gain's.
- No instrument preset uses a Pan, so no preset columns change.

Tests:
- Unit hardware-view: 17 tests, 2 new (the turn, and L and R inside
  the plate and clear of the screws). Counter-check: PAN_TAPER 2
  turns the turn test red.
- e2e 34 (melektaus-atx): 135 checks, 1 new.
- Gate (melektaus-atx): 287 files, 4399 tests.
- Pictures looked at: Gain, Pitch, Mix and Pan in a row, Pan at -0.35.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 18:26:17 +02:00
adminandClaude Opus 5.5 fe4e0d9f1a Snap to the grid from the start in every graph tab (#391)
game-maker / gate (push) Successful in 4m12s
Details
game-maker / suite (push) Successful in 15m55s
Details 
Laying out a graph by hand mostly means lining nodes up, so the
magnet was switched on in nearly every session. Every graph tab now
starts with it lit: Behavior, State machine, Shader, Particles,
Sound FX, Instrument, the song's Routing and the Routing tab.

- SnapToggle takes the state it starts in and lights its button to
  match. GRAPH_SNAPS_AT_FIRST sets it for the toggles and for
  GraphCanvas alike, so button and canvas cannot disagree.
- Curve and Vector Art are no graphs and still start without it.
- New hook __gm.graphSnapping(on?) reads every graph tab's magnet,
  or "mixed" where button and canvas differ, and switches them all.
- The e2e harness reads the magnets after each navigate() and then
  switches them off, as it already sets the zoom transition to 0.
  Most checks drag a node to an exact spot. e2e 45 does the same
  itself, because the editor comes up there through the sign-in.
- e2e 06 checks that all eight graph tabs started with the magnet on.
  The checks in 06, 26 and 65 that read it off now say they read
  what the harness left.

Tests:
- Unit snap-toggle: 4 tests, 1 new (lit at first, nothing asked of
  the target before a click).
- e2e 06 (melektaus-atx): 150 checks, 1 new. e2e 34: 134 checks.
- Gate (melektaus-atx, together with #390): 287 files, 4397 tests.
- Pictures looked at: none, the state comes from the hook.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 17:37:13 +02:00
adminandClaude Opus 5.5 e70954115c Show Pitch and Mix nodes as one-knob devices in green and blue (#390) 
The Gain has been a small device with one big knob since #384. The
Pitch and the Mix are just as much a single number, so they now look
and work the same way. The Mix also gets a lever for its curve, which
until now was only a row in the inspector.

- GainPanel and GainFace become KnobPanel and KnobFace. Each node is
  described by a KnobDevice: its plate, its row, how the knob turns,
  and an optional lever. The Gain looks as before.
- Pitch: a green 160x160 plate with a bipolar knob for Semitones. 0
  stands straight up and taper 2 puts an octave at 43 % of each half,
  so small steps near 0 are easy and the full +-64 is still there.
- Mix: a blue 180x200 plate, the lever on top and the knob below.
  The knob has A and B engraved at the ends of its turn. The lever
  reads "crossfade" down and "both at center" up with its lamp lit.
  A flip is one undo step.
- A wired Semitones or Mix greys the knob, as for the Gain.
- The instrument presets give Pitch and Mix two columns. In the graph
  they are 281 and 261 units wide, more than one column of 240.
- The test hooks __gm.pitch.panel and __gm.mix.panel return the same
  shape as __gm.gain.panel, plus the lever.

Tests:
- Unit hardware-view: 15 tests, 3 new (Pitch turn, lever above the
  knob and ends on the Mix plate). Counter-check: taper 1 turns the
  Pitch test red.
- e2e 34 (melektaus-atx): 134 checks, 1 new.
- Gate (melektaus-atx, together with #391): 287 files, 4397 tests.
- Pictures looked at: Gain, Pitch and Mix in a row, Pitch at -24.5,
  Mix with both curves, the Bell preset without overlaps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 17:37:03 +02:00
adminandClaude Opus 5.5 e8021b1fd3 Show a Reverb node as a device with a round screen of its tail (#389)
game-maker / gate (push) Successful in 4m5s
Details
game-maker / suite (push) Successful in 15m49s
Details 
The Compressor, the Gain, the Chip and the Envelope are devices in the
graph. The Reverb was still a list of rows, and how long and how dark
its tail rings was left to the numbers.

- The node is a plate of 640x280 in pale lavender. Its inputs are on
  the left, its output on the right.
- A round screen draws the tail as rings out from the sound. A ring's
  radius is its time, as the root of its seconds, on one scale up to
  10.5 s for every Reverb. So a Room is a small disc and a Cave fills
  two thirds.
- A ring is as bright as the tail is loud there, evenly in dB. Its
  colour is how much of the highs is left, from ice to violet, with
  the cut-off of the impulse response (reverbCutoff, now shared).
- The pre-delay is a dark gap round the dot, a dashed ring marks
  -60 dB. Seeded dots stand for the reflections. Wet fades the tail
  against the dot.
- A type of Sound draws the peaks of its sound from the library, each
  against its loudest. It says LOADING until they are there.
- A button for each type, a list of the sounds and four chrome knobs:
  Decay, Damping, Pre-delay and Wet.
- A knob the type does not take is greyed but turns. It goes over to
  Custom with the type's numbers, as one undo step. Picking a sound
  makes the type Sound, also one step.
- NodeParamDrag can write several settings as one drag for that.
- Test hook __gm.reverb.panel(nodeId), for the sound FX and the
  instrument graph.

Tests:
- Unit reverb-view: 10 new tests for the plate, the knob turn, the
  numbers per type, the scale, the tail and the texts. A screen that
  ignores damping turns the highs test red.
- e2e 34 (atx): 1 new check (device between its ports, decay knob of
  a Hall goes to Custom as one undo step, Cave button, a sound from
  the list makes it Sound and draws that sound).
- Probe pictures on stein2: Room, Hall, Cave, Plate, Custom with
  damping 1 and pre-delay 0.3 s, Sound.
- Gate (atx): CHECK EXIT 0, 287 files, 4393 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 16:35:59 +02:00
adminandClaude Opus 5.5 c75e72d4ee Show an Envelope node as a device with its ADSR curve (#388)
game-maker / gate (push) Successful in 4m24s
Details
game-maker / suite (push) Successful in 15m50s
Details 
The Compressor, the Gain and the Chip are devices in the graph. The
Envelope was still a list of rows, and its shape was left to the
imagination of whoever read the numbers.

- The node is a plate of 520x280 in pale sea green. Its inputs are
  on the left, out and value on the right.
- A dark screen draws the curve over a note in amber. A letter sits
  above each stage, its time below it, and the sustain in percent
  above its line.
- The time on the screen is squeezed. Each stage is as wide as the
  root of its seconds, at least 3 %, and the hold has a fixed fifth.
  So 5 ms of attack still shows beside a release of 3 s.
- The falls use SETTLE, as the player does. With sustain 0 there is
  no hold and no release, and the screen says NOTE ENDS.
- Attack, Decay, Sustain and Release are chrome knobs. The times use
  taper 4, so 5 ms sits an eighth of the way and 1 s not quite half.
- From and To have no range a knob could turn through. They are
  NumberFields on the plate, typed in or scrubbed.
- The decay at sustain 1 and From/To while nothing takes value are
  faded. While a link takes value, the bezel says what it moves
  between, as VALUE 400 TO 3500. The UI font has no arrow.
- Instrument presets give an Envelope 3 columns of 240.
- Test hook __gm.envelope.panel(nodeId).

Tests:
- Unit envelope-view: 6 new tests for the plate, the knob turn, the
  idle settings and the curve. A linear decay turns two of them red.
- e2e 34 (atx): 1 new check on the Laser preset (device between its
  ports, curve ends with the decay, decay knob and scrubbed To field
  each one undo step).
- Probe pictures on stein2: Synth Brass preset, pad, pluck, organ
  and an Envelope whose value moves a Chip.
- Gate (atx): CHECK EXIT 0, 286 files, 4383 tests. The first run was
  red on the font test (#214) for an arrow in the bezel text.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 16:06:15 +02:00
adminandClaude Opus 5.5 ab37d3423a Let graphs zoom out to an eighth (#387) 
With the Chip and the Filter drawn as devices, a preset such as the
Laser is about 3060 units wide. In a window of 1280 px a fit then hit
the floor of 0.25 and cut the Notes node off on the left.

- ZOOM_LIMITS.min goes from 0.25 to 0.125. The graph zooms without
  steps, so one more step is the floor halved. 0.125 is also the
  smallest step of the scene view.
- It holds for the wheel, for Fit and for the node board of plugins.
- The grid still hides itself below 8 px, so it does not flicker.

Tests:
- Unit plugins and graph-framing: 55 tests green, the floor test now
  expects 0.125.
- e2e 06 (atx): ALL PASSED, 149 checks. Only a comment that named 0.25
  changed there.
- Gate (atx): CHECK EXIT 0, 285 files, 4377 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 15:09:05 +02:00
adminandClaude Opus 5.5 eb78f3cbf3 Show a Chip node as a device with its wave and its pitch (#386)
game-maker / gate (push) Successful in 4m5s
Details
game-maker / suite (push) Failing after 16m5s
Details 
The Chip had eleven numbers in rows and nothing that showed what they
do to a note. Its node is now a device of one size like the
Compressor's, and two screens draw the wave and the pitch from the
settings, worked right in the instrument graph.

- ChipPanel is 840x280 in the warm grey of a handheld console. A
  button per wave, the "follow note" lever and a chrome knob per
  number, grouped Oscillator, Slide, Vibrato, Arpeggio and Repeat.
- The wave screen draws two periods as loud as the Chip plays them,
  a square at its duty. toneValue and periodicSteps come out of
  chip.ts, so the screen and the worklet share one formula.
- The pitch screen draws what slide, delta slide, vibrato, arpeggio
  and repeat make of the pitch over a note. It ends where the note
  falls below "min frequency" and starts from A4 while the Chip
  follows the note.
- Idle settings are faded, wired ones greyed and locked, as the rows
  were. Every change is one undo step.
- Knobs take a bipolar range: 0 straight up, the taper bends each
  half. Slide, delta slide, duty sweep and arpeggio use it.
- The Compressor's auto-gain lever is now a Lever with a name of its
  own, which the Chip uses for "follow note".
- Presets give a Chip five columns, as a Filter has four since #385.

Found on the way:

- With the Chip this wide the Laser preset is about 3060 units. In a
  1280 px window Fit stops at the least zoom of 0.25 and cuts the
  Notes node off. The zoom limit is left as it is.

Tests:
- Unit: chip-view (9 new), knob (1 new), chip, chip-presets and
  hardware-view, 53 green
- e2e 34 on melektaus-atx: ALL PASSED, 131 checks (1 new). The
  first run was red on the check itself: a Laser's "min frequency"
  can lie above A4, and the screen then rightly says the note ends
  at once.
- Gate on melektaus-atx: CHECK EXIT 0, 285 files, 4377 tests
- Counter-check: a square drawn with its duty mirrored turns the wave
  test red

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 14:55:35 +02:00
adminandClaude Opus 5.5 c087993000 Edit a Filter in its node and resize EQ and Filter nodes (#385) 
A Filter sounds like one band of the Parametric EQ, but it keeps what
the EQ cannot: linked numbers, the all-pass and +-40 dB. So it stays a
node of its own and gets the EQ's editor with itself as the one band.
The editors also stop running out of their room: what does not fit is
left out instead of being cut.

- The curve of the EQ is now BandCurve with a host the EQ and the
  Filter both fill. CurvePanel holds what their panels share.
- FilterPanel shows the Filter as one band. A drag of its handle sets
  frequency and gain as one undo step. A number fed by a link is
  greyed out and the drag leaves it alone.
- A double-click on the Filter's title opens the same in a tab of its
  own. Delete there does nothing, the node stays.
- The Filter is tapped like the EQ, so its node and its tab draw the
  spectrum of what passes it. A tap is two gain nodes per voice, the
  analysers are made only while someone listens.
- EqFace and FilterFace are one CurveFace now, given the panel.
- EQ and Filter nodes have a corner to resize them, drawn like the
  floating window's. The size is saved on the node as `size`, each
  drag is one undo step, the least is 240 x 120.
- In the node the help line is gone. The row of numbers comes in only
  once the curve keeps 120 px of height. A group of the row that does
  not fit is not drawn, the one on the right first.
- A frequency or dB label that would run into the one before it is
  left out.
- The all-pass is drawn flat. The curve's maths takes every shape a
  Filter has.

Found on the way:
- The instrument presets set nodes 240 apart. A Filter with its
  editor is about 700 wide and covered the Bitcrusher, so a Filter
  now takes four columns.

Tests:
- Unit: 8 new (filter band, resize command and save, fitting rules),
  stage lists with a Filter now hold its taps
- e2e 34 (atx): ALL PASSED, 130 checks (4 new)
- Gate (atx): CHECK EXIT 0, 284 files, 4367 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 14:30:00 +02:00
adminandClaude Opus 5.5 17f47f6956 Drop the Gain's engraved name and centre its knob (#384) 
The plate was taller than its one knob needs, and the engraved
"GAIN" only repeated the node's title. Without it the plate is
square, and the knob sits in its middle.

- The plate is 160x160 instead of 160x220, without the "GAIN"
  engraving.
- The knob is centred on what it shows: the "DB" caption and the
  dial. Since #382 its number hangs inside the dial's open bottom,
  so it adds no height.
- CAPTION_AIR (4) takes out the empty space above the caption's
  capitals. Without it the knob sat a little too low.

Measured on a picture at zoom 0.8: 19 px of air above "DB" and
19 px below the number. The user's screenshot showed about 14
above and 35 below.

Tests:
- Unit hardware-view: 12 tests green. The layout test now asks
  for the same air above and below.
- e2e 34 (melektaus-atx): ALL PASSED, 126 checks.
- Gate (melektaus-atx): CHECK EXIT 0, 283 files, 4359 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 13:29:28 +02:00
adminandClaude Opus 5.5 01ee61868b Show a Gain node as an orange device with one knob (#384) 
A Gain node now draws a small device in the graph, as the Compressor
does since #380: the brushed plate in orange, 160x220 graph units, with
one chrome knob between its inputs and its output.

- GainPanel and GainFace join the node faces. The knob drags, resets
  on a double-click and keeps the wheel for the graph (#383). Each turn
  is one undo step, a linked Gain (dB) greys it.
- The knob turns with a taper of 0.26, so 0 dB stands straight up and
  the range below -24 dB takes a fifth of the turn.
- drawPlate, engrave and the brushed look move out of CompressorPanel
  so both devices share them. The Knob takes an arc colour, because
  the orange arc vanished on the orange plate.
- No meter and no tap. Gain sits in every voice of an instrument, and
  a tap would add two audio nodes to each note for a display nobody
  asked for.

Found on the way:
- Knob.dialGlobal added half the dial unscaled. In a zoomed graph it
  pointed beside the dial. The Compressor's e2e only hit by luck.

Tests:
- Unit: 2 new for the Gain plate and its taper.
- e2e 34 on atx: ALL PASSED, 126 checks (1 new).
- Gate on atx: CHECK EXIT 0, 283 files, 4359 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 13:07:44 +02:00
adminandClaude Opus 5.5 79a04e6764 Edit a Parametric EQ in its node, let the wheel only zoom graphs (#381) 
The small curve below the EQ node's rows (#375) only showed the bands.
Like the Compressor since #380, the node now shows the editor of its
tab between its inputs and its output, and is worked right there. A
double-click on the node still opens the same editor in a tab.

Two wishes for the nodes with a panel came on top and share the same
files, so they are in this commit as well: the numbers below the
Compressor's knobs read as plain labels (#382), and the wheel over a
node zooms the graph instead of turning a value (#383).

#381, the EQ in its node:
- EqPanel holds the curve, the band's numbers and the help line, taken
  out of EqEditor. The tab puts its one panel under a header, each EQ
  node has a panel of its own at 720 x 360 (EqFace).
- A press on the curve neither picks nor moves the node. A
  double-click on the curve adds a band, one on the title or the edge
  opens the tab.
- In the node a press on the Type or Slope button stays with the
  button, so the node is not picked or moved.
- The spectrum runs in the node while it is in the picture. The tab
  lets go of the tap once, like the Compressor's, and Editor.tick lets
  every unseen tab and graph go before any seen one listens.
- The count of bands moved from the tab's header into the band row,
  so the node shows it as well.
- The node picture of #375 is gone with its only user: NodePicture,
  EQ_PICTURE and NodeView's drawing of it.

#382, the numbers below the Compressor's knobs in its node:
- A Knob with typed: false shows its number as a centred Label without
  a box, in the panel's ink. The ratio reads 4:1 (ratioText).
- The label is as high as its text and sits up in the dial's open
  bottom, right under the ends of the scale.
- Label takes a colour of its own.

#383, the wheel in a graph:
- GraphCanvas takes the wheel on the way down and stops it there. It
  zooms over a node too, and no knob or curve in a node sees it.
- The tabs keep their wheel: a knob's step, a band's Q and the zoom
  of the EQ's dB.

Found on the way:
- A Menu opened below a button in the zoomed graph would have been
  placed by the button's own height. openBelow now takes the bottom
  edge as drawn.
- The band row did not lay out its labels after a text change when
  its own size stayed. In the tab a later resize hid it, in the node
  "Band 1" and the count were missing. The row now lays out itself.
- Handle and point positions for the checks were unscaled. They now
  go through toGlobal and hold at the graph's zoom.

Tests:
- Unit parametric-eq: 16 tests. The #375 tests now check
  eqCurvePoints and eqRangeOf directly. compressor-view: ratioText.
- e2e 34 (atx): ALL PASSED, 125 checks (3 new: the EQ panel and its
  ports, editing in the node, the spectrum in the node with the tab
  open). The Compressor node's check reads its numbers as labels and
  8:1. Both node checks now expect the wheel to zoom the graph and
  leave the ratio, the Q and the dB zoom alone.
- The unseen-tab check now leaves for the Scene tab, since the node
  in the graph listens to the same tap.
- Counter-check: the tab letting go of the tap every unseen frame
  stalls the node's spectrum with the tab open (1 frame against 22,
  probe on stein2).
- Gate (atx): CHECK EXIT 0, 283 files, 4357 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 12:40:11 +02:00
adminandClaude Opus 5.5 1f2b64fb4c Show a Compressor's panel in its node, its tab on a double-click (#380)
game-maker / gate (push) Successful in 4m2s
Details
game-maker / suite (push) Successful in 16m14s
Details 
The panel of #379 opened in a window of its own and replaced the tab.
The user wanted the device in the graph instead, and the tab of #378
back on a double-click. The switch Hardware look goes again.

- A node can show a face instead of its rows (`NodeFace`). Its inputs
  sit on the left of it and its outputs on the right, in the middle of
  the height. Each side is as wide as its longest port name.
- A Compressor node shows the panel at full size, 820x280 graph units.
  Its settings and numbers are not listed as rows, the panel has them.
- The knobs and the lever work in the node. A drag, a notch of the
  wheel and a double-click are one undo step each. A press on them
  neither picks nor moves the node, and the wheel does not zoom the
  graph. The number below a knob is only shown, typing stays in the
  tab and the inspector.
- The meter and the LEDs run while the node is in the picture of a sound
  FX or instrument graph that is seen. Out of the picture, folded or
  hidden, the node lets go of its taps.
- The window of one size, `fixedSize`, `refit`, `keptSize` and the rule
  that kept a Compressor out of the tab row are gone again.
- A knob's drag as one undo step lives in `nodeParamDrag.ts` now, for
  the tab and the node. The numbers of a node and what auto-gain says
  moved to `compressorView.ts`.

Found and fixed on the way:
- The tab and the node listen to the same taps. The tab let go of them
  on every frame it was not seen, so the node's meter stood still while
  the tab was open. Now the tab lets go once, and in a frame everything
  unseen lets go before anything seen listens.

Tests:
- Unit: compressor-view (2 new), documents (the test of `keptSize` is
  gone with it)
- e2e 34 (stein2, melektaus-atx was off): ALL PASSED, 122 checks. The
  four checks of #379 are now four of the node: ports beside the panel,
  knobs and lever in the node, the needle when played (21.7 dB with the
  tab open), and the double-click opening the tab.
- Gate (stein2): CHECK EXIT 0, 283 files, 4356 tests
- Counter-checks: an old auto-gain number read as off turns exactly its
  unit test red. A tab that lets go on every frame leaves the node's
  needle at 0 dB and no LED lit (probe on stein2).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 11:41:28 +02:00
adminandClaude Opus 5.5 9917253751 Open a Compressor as a device in a window of one size (#379) 
The tab from #378 shows a Compressor as an editor. The user wanted it
to look like a device as well, after a brushed aluminium plug-in, with
a meter of the gain reduction instead of the pictures. The tab stays
and is switched to per node.

- Hardware look is a switch under Settings, on by default for new and
  existing Compressors. Only the editor reads it.
- The panel is 820x280 px and opens straight in a window of that size.
  The window has no sizing corner and no Dock. A pane says so through
  `fixedSize`, and `keptSize` keeps the window's middle when it
  changes to that size.
- A Compressor in this look never sits in the tab row. Opening one
  floats it, and switching the look on moves its tab into the window.
- The panel has chrome knobs (threshold and makeup large), a meter
  with a needle that moves like a VU's, an LCD with the name and the
  reading, a lever with a lamp for auto-gain, and LEDs for the level
  going out. Everything is drawn in code.
- The Knob takes a size and a chrome look, and leaves out its caption
  when it has no name, so the panel engraves its own.
- `autoGainOn` is now `switchOn`, because the new switch reads the
  same way.
- Where parts sit, the meter's scale, the needle's speed, the LEDs and
  the plate's grain are in `hardwareView.ts`, apart from the drawing.

Decided myself:
- The meter's scale runs by the square root, so the first dB is about
  as wide as the last nine. It is red from 10 dB on.
- The LEDs light one every 3 dB from -39 to 0 dB and fall by 20 dB a
  second.
- Switching the look changes the graph's fingerprint, like any other
  setting of a node. The sound FX is built again once.

Findings, fixed on the way:
- Pixi's `arc` goes on from the last point of the path. The meter's
  scale had a line from the corner of the face until each arc got a
  `moveTo` to its own start.
- A text put into a Graphics makes Pixi warn that only Containers take
  children. The LCD is a Container now.
- The first browser check read the needle at one moment and missed the
  short sound. The panel keeps the most its needle and LEDs showed.

Tests:
- Unit: hardware-view (10 new), documents (1 new)
- e2e 34 (melektaus-atx): ALL PASSED, 122 checks (4 new). Played,
  the needle reached 20.9 dB and 5 LEDs lit.
- Gate (melektaus-atx): CHECK EXIT 0, 283 files, 4355 tests (11 new)
- Counter-checks: a window that keeps its corner instead of its
  middle, and a linear meter scale, each turn exactly their test red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 02:06:56 +02:00
adminandClaude Opus 5.5 2d4909d248 Fit the Compressor's tab to any size, make auto-gain a switch (#378) 
Two things came up on the new tab. Floated and sized down, it spilled
its knobs out of its window, and sized up it left the window empty.
Auto-gain lifted far too much: it brought 0 dB back to 0 dB, which at
-40 dB, 4:1 and knee 6 is +28.1 dB, so a sound at -24 dB came out at
-6 dB. And as a knob from 0 to 1 it blended between makeup and
auto-gain, so at 1 the makeup knob did nothing.

Layout:

- The knobs stay on the bottom line, centred, with the help line
  above them.
- The levels fill the height left above. The curve is a square at
  their right, centred up and down. It grows with the tab from 180 to
  360 px and takes no more than 40 % of its width (picturesLayout).
- A floated window cannot be sized below what its pane needs. A pane
  says so in its minWidth and minHeight, and the Compressor's is every
  knob and the smallest pictures. Other panes keep 320 x 200.
- A pane put into an open window that is too small grows the window.

Auto-gain, as decided in #378:

- A Compressor's auto-gain gives back what it takes from a sound at
  -18 dB, so that one comes out as loud as it went in. At the numbers
  above that is +14.6 dB.
- Auto-gain is a switch on the Compressor, the Limiter and the
  Saturation alike, a setting and no longer an input. The makeup or
  output is added on top of it instead of being blended with it.
- A project from before reads a number as on above 0. Nodes never
  set keep their defaults: off on the Compressor, on on the other two.
- The tab shows the switch at the right of the knobs, with the dB it
  gives below it. With it on, the curve has a line at -18 dB from the
  curve up to the diagonal.
- Every level reading keeps the gain of its time. Before, the whole
  picture was read by the current makeup, so after a change the
  tinting of the last seconds was wrong or missing.

Findings:

- My first e2e run of the layout clicked Dock where it was before the
  window was sized up, so the window stayed open and the next check
  went red. The check reads the button again.

Tests:

- Gate (melektaus-atx): 282 files, 4344 tests (6 new)
- e2e 34 (melektaus-atx): ALL PASSED, 118 checks (2 new)
- Offline in Chrome: a sine at -18 dB through -24 dB and 4:1 came out
  at -17.92 dB with auto-gain on and -22.42 dB with it off.
- Counter-checks: the curve set at the top instead of the middle, and
  every reading read at a gain of 0, each fail exactly their test.
- Looked at the window in screenshots: at its least every knob and
  the switch show. At 1280 x 797 the levels fill the height, the
  curve is centred at the right and the knobs are at the bottom.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-10 00:57:26 +02:00
adminandClaude Opus 5.5 1df38bd430 Edit a Compressor in a tab of its own, with knobs and levels (#378) 
A Compressor had only its number rows in the node. Its numbers are
easier to set by feel on a dial, and what it does is easier to judge
when the level going in and the level going out are seen side by side.

- Knob is a new widget in the toolkit: a slider bent into a circle,
  its name above and a number field below. A drag up or down turns
  it, Shift a tenth as far, a notch of the wheel a fiftieth of the
  turn, a double-click sets the default, the number takes a typed
  value. A taper gives the low end more of the turn.
- The Compressor's tab opens on a double-click, one per node, like
  the Parametric EQ's. It has a knob for each of the seven numbers,
  greyed while its input is linked.
- Its curve shows level in to level out from -60 to 0 dB. The point
  where it bends sets the threshold, the end of the curve sets the
  ratio that ends there.
- Its levels show the last 4 s on both sides of the node. Where it
  takes more than 0.5 dB away, the stretch below the level coming in
  is tinted. The makeup is left out of that.
- The Compressor is tapped like the EQ. SpectrumTaps listens to more
  than one node at once and reads the loudest sample of the last
  25 ms on each side.
- The node tabs share one map in the editor (nodeTabs), so a third
  kind needs no new plumbing.

Findings:

- CI task 614 for 23922ee was red on the memory check of the stream:
  the tab's renderer grew by 94 MB with nothing decoded. Play mode
  made garbage of its own and was read before a collection. Both
  readings in play now come after one, and the check compares the
  growth streamed with the growth decoded (3.7 MB against 59.8 MB).
- The ratio handle sat on the right edge of the picture, where a
  widget's hit area ends. The curve is inset by the grab radius.

Tests:

- Gate (melektaus-atx): 282 files, 4338 tests (17 new)
- e2e 34 (melektaus-atx): ALL PASSED, 116 checks (6 new)
- Counter-checks: an inverted ratio search, stop() letting go of
  every node and a drag without its least step each fail exactly
  their test.
- Looked at the tab in a screenshot: a blip at -4.4 dB came out at
  -16.7 dB, the gap tinted, the threshold line at -24.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
 2026-10-09 22:40:12 +02:00
589 changed files with 75327 additions and 3103 deletions
Expand all files Collapse all files

No files matched your search

.gitea/workflows/ci.yml
+20 -14
View File
@@ -1,20 +1,20 @@
# Runs the whole proof on the Gitea runner: gate, the editor suite and the three example games.
#
# The suite runs on `game-maker-wsl`, the runner in the WSL2 distro game-maker-ci on
# melektaus-atx (#283). Its label picks the image built from tools/ci/Dockerfile. The GPU
# (/dev/dxg and the Windows driver libraries), /dev/shm and GAME_MAKER_BROWSER=wsl come from that
# runner's own config.yaml, not from here – without them chromium falls back to software WebGL and
# the harness stops. The gate needs no GPU and runs on `game-maker-ci`, the runner on stein2 with
# the same image. Pointing the suite at that label too moves the browser runs back to stein2.
# Both jobs run on `feuerwerk-ci`, the runner on stein2. Its label picks the image built from
# tools/ci/Dockerfile. The GPU (/dev/dri/renderD128) and /dev/shm come from that runner's own
# config.yaml, not from here – without them chromium falls back to software WebGL and the harness
# stops. The suite ran on `feuerwerk-wsl` in the WSL2 distro feuerwerk-ci on melektaus-atx (#283),
# whose config adds /dev/dxg and FEUERWERK_BROWSER=wsl. Pointing the suite at that label again
# moves the browser runs back there. It moved to stein2 because other jobs on melektaus-atx share
# its GPU and made the suite's frame timings wobble.
#
# Two jobs without `needs:`, so they start together. The gate does not stand in front of the
# suite: a lint error used to cost the full e2e run before it was reported, and the run reported
# it after six minutes instead of one. Both still have to pass for the push to be green.
#
# On two runners the gate's vitest no longer shares cores with the suite's chromium, which
# measures text widths and frame counts. GM_TEST_WORKERS still caps vitest at two workers, for the
# day both jobs share one runner again.
name: game-maker
# On one runner the gate's vitest shares cores with the suite's chromium, which measures text
# widths and frame counts. GM_TEST_WORKERS caps vitest at two workers for that.
name: feuerwerk
on:
push:
@@ -22,7 +22,7 @@ on:
jobs:
gate:
runs-on: game-maker-ci
runs-on: feuerwerk-ci
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
@@ -35,17 +35,23 @@ jobs:
env:
GM_TEST_WORKERS: 2
# The API's own rules (#180, #179, #231, #266). Not part of check.sh: the host the editor is built
# on has no PHP, the image does.
# The API's own rules (#180, #179, #231, #266, #427, #429). Not part of check.sh: the host the
# editor is built on has no PHP, the image does. rights.test.php runs the API itself under
# `php -S`.
- name: PHP tests
run: |
php tests/php/accounts.test.php
php tests/php/snapshots.test.php
php tests/php/releases.test.php
php tests/php/files.test.php
php tests/php/videos.test.php
php tests/php/limits.test.php
php tests/php/demos.test.php
php tests/php/agents.test.php
php tests/php/rights.test.php
suite:
runs-on: game-maker-wsl
runs-on: feuerwerk-ci
timeout-minutes: 45
steps:
- uses: actions/checkout@v4
.gitignore
+7
View File
@@ -4,12 +4,19 @@
/data/e2e-password.txt
/data/e2e-admin-password.txt
/data/sessions/
/data/sign-ins/
/data/agent-tokens/
/data/agent-tickets/
/data/mcp-relay.key
/data/snapshots/
/data/releases/
/data/demos/
/data/demo-files.json
/data/users.json
/data/textures/
/data/fonts/
/data/sounds/
/data/videos/
/data/pixels/
/data/plugins/
/dist/
README.md
+47 -45
View File
@@ -1,16 +1,16 @@
# Game Maker
# Feuerwerk
[![game-maker](https://git.nockal.com/meleki/game-maker/actions/workflows/ci.yml/badge.svg?branch=main)](https://git.nockal.com/meleki/game-maker/actions?workflow=ci.yml)
[![feuerwerk](https://git.nockal.com/meleki/feuerwerk/actions/workflows/ci.yml/badge.svg?branch=main)](https://git.nockal.com/meleki/feuerwerk/actions?workflow=ci.yml)
A browser-based tool "one step before a game engine": create **assets** (scenes, cameras,
sprites), place them into scenes, tweak their properties in an inspector and press **Play** –
sprites), place them into scenes, tweak their properties in an inspector and press **Play**,
all in one program, all drawn inside a single WebGL canvas.
![](screenshot.png)
Live: <https://nockal.com/game-maker/> · Widget playground: <https://nockal.com/game-maker/?playground>
Live: <https://nockal.com/feuerwerk/> · Widget playground: <https://nockal.com/feuerwerk/?playground>
**The documentation is the wiki:** <https://git.nockal.com/meleki/game-maker/wiki> – every concept,
**The documentation is the wiki:** <https://git.nockal.com/meleki/feuerwerk/wiki>: every concept,
node, tab and shortcut, plus the architecture and the development workflow. This file is the short
way in.
@@ -19,13 +19,13 @@ way in.
- An **asset** is a named, typed thing (scene, camera, sprite, text, behavior, variables, struct,
collider, rigidbody, joint, container, clip area, texture, vector art, pixel art, palette,
animation, curve, tile, tilemap, state machine, shader, shader function, font, sound, sound FX,
instrument, pattern, song, bus, particles) with properties and no position.
instrument, pattern, song, bus, video, composition, particles) with properties and no position.
- A **scene object** places an asset: position, sparse **overrides** of its properties, an optional
parent. Only what differs from the asset is stored; the inspector paints it gold and offers ↺ / ↑.
- A **container** is a reusable group edited once and placed many times; its content is resolved
from the template on every read. A **screen** is a container switched on and off.
- A **behavior** is logic as a node graph, placed like anything else and parented to the object it
acts on. Play has no built-in behaviour – everything that happens is a graph.
acts on. Play has no built-in behaviour: everything that happens is a graph.
- An **account** owns projects: the editor asks for a user name and a password first, and each
account sees only its own projects. The Account menu at the far right of the toolbar changes
the password and signs out.
@@ -39,61 +39,63 @@ Node 22 is required.
```bash
npm install
npm run dev # http://localhost:5175/game-maker/dist/ – API calls are proxied to production
npm run check # typecheck + lint + format:check + unit tests – the gate CI runs too
npm run dev # http://localhost:5175/; API and data calls are proxied to production
npm run check # typecheck + lint + format:check + unit tests; the gate CI runs too
npm run build # writes dist/ and regenerates index.html; the folder is what nginx serves
```
The browser suite and the three example games need a machine with a GPU:
```bash
node tools/seed-e2e.mjs && node tools/e2e.mjs # the editor, 1941 checks (GM_STEPS=20 for one step)
# every tool signs in first – see the Accounts page
node tools/seed-e2e.mjs && node tools/e2e.mjs # the editor, 2090 checks (GM_STEPS=20 for one step)
# every tool signs in first; see the Accounts page
node tools/seed-jumprun.mjs && node tools/smoke-jumprun.mjs
node tools/seed-tetris.mjs && node tools/smoke-tetris.mjs
node tools/seed-actor.mjs && node tools/smoke-actor.mjs
```
Every push runs all of it on the Gitea runners – the gate as one job, build and suite as another
beside it – so locally take the gate plus the step your change touches and leave the full run to
Every push runs all of it on the Gitea runners: the gate as one job, build and suite as another
beside it. So locally take the gate plus the step your change touches and leave the full run to
CI. Details, the GPU requirement and the detached runner are in
[Development](https://git.nockal.com/meleki/game-maker/wiki/Development).
[Development](https://git.nockal.com/meleki/feuerwerk/wiki/Development).
## Documentation
| Page | What it covers |
|---|---|
| [Concepts](https://git.nockal.com/meleki/game-maker/wiki/Concepts) | Assets, scene objects, overrides, play mode, the camera, the console |
| [Accounts](https://git.nockal.com/meleki/game-maker/wiki/Accounts) | Signing in and out, the accounts, who a project belongs to, what the check tools need |
| [Scenes](https://git.nockal.com/meleki/game-maker/wiki/Scenes) | The edited scene, the start scene, and switching scenes from a graph |
| [Scene editing](https://git.nockal.com/meleki/game-maker/wiki/Scene-editing) | Placing, z-order, hierarchy, the editor offset |
| [Behaviors](https://git.nockal.com/meleki/game-maker/wiki/Behaviors) | Node graphs: variables, the node set, structs and lists, loops, nesting, editing |
| [Structs](https://git.nockal.com/meleki/game-maker/wiki/Structs) | Struct assets: a named shape that struct variables, fields and `Pack`/`Expand` follow |
| [Colliders](https://git.nockal.com/meleki/game-maker/wiki/Colliders) | Shapes (box, circle, capsule, polygon), collision events, blocking movement, one-way colliders and raycasts |
| [Physics](https://git.nockal.com/meleki/game-maker/wiki/Physics) | Rigidbodies, the collider material, gravity, the fixed step, joints |
| [Textures](https://git.nockal.com/meleki/game-maker/wiki/Textures) | Images, frames, nine-slice, the filter, upload limits |
| [Vector art](https://git.nockal.com/meleki/game-maker/wiki/Vector-Art) | Pictures drawn from shapes: the pen, curves, gradients and texture fills, combining shapes |
| [Pixel art](https://git.nockal.com/meleki/game-maker/wiki/Pixel-Art) | Pictures painted pixel by pixel: the palette, the tools, layers and frames |
| [Fonts](https://git.nockal.com/meleki/game-maker/wiki/Fonts) | Font assets, the picker, what a text falls back to |
| [Sounds](https://git.nockal.com/meleki/game-maker/wiki/Sounds) | Sound assets and clips, `Play sound`, buses and the Master routing, sound FX |
| [Instruments](https://git.nockal.com/meleki/game-maker/wiki/Instruments) | Instruments as node graphs, Chips and presets, the keys of the Instrument tab, `Play note` |
| [Patterns](https://git.nockal.com/meleki/game-maker/wiki/Patterns) | Patterns and the piano roll, playing one in a loop or from a game |
| [Songs](https://git.nockal.com/meleki/game-maker/wiki/Songs) | Songs, their tracks and items, routing, recording, playing one from a game |
| [Animations](https://git.nockal.com/meleki/game-maker/wiki/Animations) | Animation assets and the Animation tab |
| [Curves](https://git.nockal.com/meleki/game-maker/wiki/Curves) | Easings drawn by hand, the Curve tab, using one on a key or a `Lerp` |
| [Shaders](https://git.nockal.com/meleki/game-maker/wiki/Shaders) | Shader graphs that sprites, texts, particle fields and cameras wear, their node set, passes, shader functions |
| [Particles](https://git.nockal.com/meleki/game-maker/wiki/Particles) | Particle graphs: a particle as a function of its age |
| [State machines](https://git.nockal.com/meleki/game-maker/wiki/State-machines) | The asset, named transitions, `State branch`, `Set state` and `State transition` |
| [Containers](https://git.nockal.com/meleki/game-maker/wiki/Containers) | Templates, instances, screens |
| [Tiles](https://git.nockal.com/meleki/game-maker/wiki/Tiles) | The tile asset, plain tiles and the two autotile kinds, the pictures as animation steps, the Tile tab |
| [Tilemaps](https://git.nockal.com/meleki/game-maker/wiki/Tilemaps) | The tilemap asset, painting in the scene view, layers, what it looks like in the game |
| [Plugins](https://git.nockal.com/meleki/game-maker/wiki/Plugins) | Extending the editor: the plugin file, how it is loaded, switching one on and off, API v1 |
| [Keyboard](https://git.nockal.com/meleki/game-maker/wiki/Keyboard) | Every shortcut |
| [Architecture](https://git.nockal.com/meleki/game-maker/wiki/Architecture) | Stack, source layout, the rules that keep it organised |
| [Development](https://git.nockal.com/meleki/game-maker/wiki/Development) | Commands, the e2e suite, the example games, CI |
| [API](https://git.nockal.com/meleki/game-maker/wiki/API) | The PHP JSON API and the project document |
| [Concepts](https://git.nockal.com/meleki/feuerwerk/wiki/Concepts) | Assets, scene objects, overrides, play mode, the camera, the console |
| [Accounts](https://git.nockal.com/meleki/feuerwerk/wiki/Accounts) | Signing in and out, the accounts, who a project belongs to, what the check tools need |
| [Scenes](https://git.nockal.com/meleki/feuerwerk/wiki/Scenes) | The edited scene, the start scene, and switching scenes from a graph |
| [Scene editing](https://git.nockal.com/meleki/feuerwerk/wiki/Scene-editing) | Placing, z-order, hierarchy, the editor offset |
| [Behaviors](https://git.nockal.com/meleki/feuerwerk/wiki/Behaviors) | Node graphs: variables, the node set, structs and lists, loops, nesting, editing |
| [Structs](https://git.nockal.com/meleki/feuerwerk/wiki/Structs) | Struct assets: a named shape that struct variables, fields and `Pack`/`Expand` follow |
| [Colliders](https://git.nockal.com/meleki/feuerwerk/wiki/Colliders) | Shapes (box, circle, capsule, polygon), collision events, blocking movement, one-way colliders and raycasts |
| [Physics](https://git.nockal.com/meleki/feuerwerk/wiki/Physics) | Rigidbodies, the collider material, gravity, the fixed step, joints |
| [Textures](https://git.nockal.com/meleki/feuerwerk/wiki/Textures) | Images, frames, nine-slice, the filter, upload limits |
| [Vector art](https://git.nockal.com/meleki/feuerwerk/wiki/Vector-Art) | Pictures drawn from shapes: the pen, curves, gradients and texture fills, combining shapes |
| [Pixel art](https://git.nockal.com/meleki/feuerwerk/wiki/Pixel-Art) | Pictures painted pixel by pixel: the palette, the tools, layers and frames |
| [Fonts](https://git.nockal.com/meleki/feuerwerk/wiki/Fonts) | Font assets, the picker, what a text falls back to |
| [Sounds](https://git.nockal.com/meleki/feuerwerk/wiki/Sounds) | Sound assets and clips, `Play sound`, buses and the Master routing, sound FX |
| [Instruments](https://git.nockal.com/meleki/feuerwerk/wiki/Instruments) | Instruments as node graphs, Chips and presets, the keys of the Instrument tab, `Play note` |
| [Patterns](https://git.nockal.com/meleki/feuerwerk/wiki/Patterns) | Patterns and the piano roll, playing one in a loop or from a game |
| [Songs](https://git.nockal.com/meleki/feuerwerk/wiki/Songs) | Songs, their tracks and items, routing, recording, playing one from a game |
| [Videos](https://git.nockal.com/meleki/feuerwerk/wiki/Videos) | Video assets and clips, wearing one on a sprite, recording the game view or the editor |
| [Animations](https://git.nockal.com/meleki/feuerwerk/wiki/Animations) | Animation assets and the Animation tab |
| [Curves](https://git.nockal.com/meleki/feuerwerk/wiki/Curves) | Easings drawn by hand, the Curve tab, using one on a key or a `Lerp` |
| [Shaders](https://git.nockal.com/meleki/feuerwerk/wiki/Shaders) | Shader graphs that sprites, texts, particle fields and cameras wear, their node set, passes, shader functions |
| [Particles](https://git.nockal.com/meleki/feuerwerk/wiki/Particles) | Particle graphs: a particle as a function of its age |
| [State machines](https://git.nockal.com/meleki/feuerwerk/wiki/State-machines) | The asset, named transitions, `State branch`, `Set state` and `State transition` |
| [Containers](https://git.nockal.com/meleki/feuerwerk/wiki/Containers) | Templates, instances, screens |
| [Compositions](https://git.nockal.com/meleki/feuerwerk/wiki/Compositions) | Objects, videos and sounds laid out in time, the Composition tab, playing one in a game |
| [Tiles](https://git.nockal.com/meleki/feuerwerk/wiki/Tiles) | The tile asset, plain tiles and the two autotile kinds, the pictures as animation steps, the Tile tab |
| [Tilemaps](https://git.nockal.com/meleki/feuerwerk/wiki/Tilemaps) | The tilemap asset, painting in the scene view, layers, what it looks like in the game |
| [Plugins](https://git.nockal.com/meleki/feuerwerk/wiki/Plugins) | Extending the editor: the plugin file, how it is loaded, switching one on and off, API v1 |
| [Keyboard](https://git.nockal.com/meleki/feuerwerk/wiki/Keyboard) | Every shortcut |
| [Architecture](https://git.nockal.com/meleki/feuerwerk/wiki/Architecture) | Stack, source layout, the rules that keep it organised |
| [Development](https://git.nockal.com/meleki/feuerwerk/wiki/Development) | Commands, the e2e suite, the example games, CI |
| [API](https://git.nockal.com/meleki/feuerwerk/wiki/API) | The PHP JSON API and the project document |
## Ideas, gaps and bugs
They live in the issue tracker: <https://git.nockal.com/meleki/game-maker/issues>. What is not
there yet is listed on the wiki's [Home](https://git.nockal.com/meleki/game-maker/wiki) page.
They live in the issue tracker: <https://git.nockal.com/meleki/feuerwerk/issues>. What is not
there yet is listed on the wiki's [Home](https://git.nockal.com/meleki/feuerwerk/wiki) page.
api/index.php
+454 -53
View File
@@ -2,18 +2,36 @@
declare(strict_types=1);
/**
* Game Maker API.
* Feuerwerk API.
*
* Routing uses the `r` query parameter (e.g. `index.php?r=projects/abc123`) because the
* site's nginx configuration does not pass PATH_INFO to PHP.
*
* GET ?r=auth/session → {user, role} (both null while nobody is signed in)
* POST ?r=auth/login {username,password} → {user, role} | 401
* POST ?r=auth/login {username,password} → {user, role} | 401 | 429
* POST ?r=auth/logout → 204
* POST ?r=auth/password {currentPassword,newPassword} → 204 | 401 | 422
* POST ?r=auth/password {currentPassword,newPassword} → 204 | 401 | 422 | 429
* (the only auth route that needs a session; ends the account's
* other logins, keeps the one asking)
*
* Five wrong passwords from one address within 15 minutes, and it gets 429 with `Retry-After` for
* the next 15 (see LoginThrottle, #427). Wrong agent tokens count alike.
*
* The MCP relay (#429, tools/mcp/) asks two routes of its own, with the key both sides read from
* `data/mcp-relay.key` in `X-Relay-Key`; without it they answer 404 like a route that is not there:
*
* POST ?r=agent/verify {token,address} → {user, role} | 401 | 429
* (`address` is the agent's, which the relay saw; the brake counts it)
* POST ?r=agent/redeem {ticket} → {user, role} | 401
*
* A request that changes something (anything but GET) is refused with 403 when the browser says it
* comes from another site, `Sec-Fetch-Site` being `cross-site` or `same-site`: the session cookie is
* SameSite=Lax, and every subdomain of the site counts as the same site to it. Tools send no such
* header and their own cookie.
*
* An account takes up at most 1 GB with its uploads, projects and snapshots (see Quota); a write that
* would make it larger is refused with 413. Administrators have no limit.
*
* Everything below needs a signed-in user (401 otherwise), and projects belong to one: a project
* of somebody else answers like one that is not there – unless the one asking is an administrator
* (#179), who reads and saves it like their own; a save keeps its owner. Deleting a project is
@@ -28,6 +46,15 @@ declare(strict_types=1);
* (renaming or deleting signs the account out everywhere; a new
* password ends its other sessions)
*
* Agent tokens (#429) open the relay for an agent of the account and nothing else; a ticket gets a
* tab onto it (see AgentTokens). Neither is for a guest:
*
* GET ?r=agent-tokens → [{id,name,createdAt,expiresAt,usedAt}], oldest first
* POST ?r=agent-tokens {name,days?} → 201 {id,name,createdAt,expiresAt,usedAt,token} | 409 | 422
* (the one answer the token is in; `days` 1–365, 90 without)
* DELETE ?r=agent-tokens/{id} → 204 | 404
* POST ?r=agent/ticket → {ticket, expiresIn}
*
* GET ?r=projects → [{id,name,revision,updatedAt}]
* POST ?r=projects {name} → {id,name,revision,updatedAt,project}
* GET ?r=projects/{id} → {id,name,revision,updatedAt,project}
@@ -47,10 +74,19 @@ declare(strict_types=1);
* DELETE ?r=projects/{id}/release → 204 | 404
* (one link per project, kept when publishing again, #266)
*
* Released games are the one thing open to everybody, the token being the key:
* Released games are open to everybody, the token being the key:
*
* GET ?r=releases/{token} → {name,publishedAt,project} | 404
*
* So are the demo copies a guest opens in the editor (#428), frozen copies like a release that an
* administrator makes and updates per project:
*
* GET ?r=demos → [{token,name,publishedAt}], by name
* GET ?r=demos/{token} → {name,publishedAt,project} | 404
* GET ?r=projects/{id}/demo → {token,revision,publishedAt} | null (administrators only)
* POST ?r=projects/{id}/demo → {token,revision,publishedAt} (administrators only)
* DELETE ?r=projects/{id}/demo → 204 | 404 (administrators only)
*
* GET ?r=textures → [{file,name,width,height,bytes,updatedAt,owner,deletable}]
* POST ?r=textures&name=x.png raw image bytes → {file,name,width,height,bytes}
* DELETE ?r=textures/{file} → 204 | 403 | 404 | 409 (still used)
@@ -63,34 +99,59 @@ declare(strict_types=1);
* POST ?r=sounds&name=x.wav raw sound bytes → {file,name,bytes}
* DELETE ?r=sounds/{file} → 204 | 403 | 404 | 409
*
* GET ?r=videos → [{file,name,status,…,original,owner,deletable}]
* POST ?r=videos&name=x.mov[&convert=0] raw video bytes → {file,name,status,…}
* POST ?r=videos/{file}/convert → {file,name,status,…} | 403 | 404
* DELETE ?r=videos/{file} → 204 | 403 | 404 | 409
* (an upload is converted by the worker in `feuerwerk-video` (#402):
* `status` is queued, converting with `progress`, failed with `error`,
* or ready with width, height, duration, fps, audio and `poster`;
* with `convert=0` it is only kept, `unconverted`, until it is asked
* to be. `original` is the upload as it came, null for a video
* uploaded before uploads were kept)
*
* Uploads land in the uploader's own folder, and `file` carries it: `{folder}/{sha256}.{ext}`
* (#344). A `file` without a folder belongs to nobody. A list shows the files of the one asking and
* nobody's, an administrator's every file; `owner` is the account a file belongs to (null for
* nobody, '' for a deleted account). A file goes only at its owner's or an administrator's hand,
* and only while no project, snapshot or release uses it.
* and only while no project, snapshot, release or demo copy uses it.
*
* An administrator offers a file to everybody as demo material (#428); every list carries those
* files too, with `demo: true`, and without a session a list holds nothing else. Only an
* administrator deletes one while it is offered.
*
* PATCH ?r=textures/{file} {demo} → 204 | 403 | 404 | 422 (administrators only)
* PATCH ?r=fonts/{file} {demo} → the same
* PATCH ?r=sounds/{file} {demo} → the same
* PATCH ?r=videos/{file} {demo} → the same
*
* POST ?r=pixels raw cel bytes → {file,bytes} (pixel art, #130)
*
* GET ?r=plugins → [{file,bytes,updatedAt}], sorted by file name
* GET ?r=plugins/{file} → the module source as text/javascript
* (both without a session as well: a guest's demo may switch one on, #428)
*
* `name` is the file's original name, kept beside it as `{file}.json` (see OriginalName) and
* empty when none is known. It is a caption, never a path: the stored file is always `{hash}.{ext}`
* in a folder.
*
* Stored images, fonts, sounds and cels are served straight from `data/textures/`, `data/fonts/`,
* `data/sounds/` and `data/pixels/` as static files; the API only writes them.
* Stored images, fonts, sounds, videos and cels are served straight from `data/textures/`, `data/fonts/`,
* `data/sounds/`, `data/videos/` and `data/pixels/` as static files; the API only writes them.
*/
require __DIR__ . '/lib/AccountRules.php';
require __DIR__ . '/lib/AgentTokens.php';
require __DIR__ . '/lib/AtomicFile.php';
require __DIR__ . '/lib/Auth.php';
require __DIR__ . '/lib/DemoFiles.php';
require __DIR__ . '/lib/FileUsage.php';
require __DIR__ . '/lib/HashedFileStore.php'; // before the font, sound and texture stores built on it
require __DIR__ . '/lib/FontStore.php';
require __DIR__ . '/lib/LoginThrottle.php';
require __DIR__ . '/lib/OriginalName.php';
require __DIR__ . '/lib/PixelStore.php';
require __DIR__ . '/lib/PluginStore.php';
require __DIR__ . '/lib/Quota.php';
require __DIR__ . '/lib/ReleaseStore.php';
require __DIR__ . '/lib/Response.php';
require __DIR__ . '/lib/SessionStore.php';
@@ -99,8 +160,13 @@ require __DIR__ . '/lib/SoundStore.php';
require __DIR__ . '/lib/Store.php';
require __DIR__ . '/lib/TextureStore.php';
require __DIR__ . '/lib/UserStore.php';
require __DIR__ . '/lib/VideoQueue.php';
require __DIR__ . '/lib/VideoStore.php';
/** Has to clear the largest single upload below, which is a sound; the web server has its own say. */
/**
* Has to clear the largest upload held in memory, which is a sound; a video is streamed to disk and
* has its own ceiling. The web server and the proxy in front of it have their own say.
*/
const MAX_BODY_BYTES = 256 * 1024 * 1024;
/** Upload ceiling for a single image. The editor downscales larger pictures before sending. */
@@ -112,6 +178,26 @@ const MAX_FONT_BYTES = 4 * 1024 * 1024;
/** The same for a sound, far higher: a recorded take is plain WAV, about 46 minutes in mono. */
const MAX_SOUND_BYTES = 256 * 1024 * 1024;
/**
* A video goes up as it was recorded and is made smaller by the worker, not on the way. It is
* streamed to disk, so memory is no limit; nginx and the Brandmauer let 2 GB through (#402).
*/
const MAX_VIDEO_BYTES = 2048 * 1024 * 1024;
/**
* Where everything the API keeps lies, and how much of it an account may fill. The rights test
* (tests/php/rights.test.php) runs the API under `php -S` on a scratch directory with a small quota;
* behind nginx neither can be moved.
*/
define('DATA_DIR', testSetting('FEUERWERK_DATA') ?? dirname(__DIR__) . '/data');
define('QUOTA_BYTES', (int) (testSetting('FEUERWERK_QUOTA_BYTES') ?? Quota::BYTES));
function testSetting(string $name): ?string
{
$value = PHP_SAPI === 'cli-server' ? getenv($name) : false;
return is_string($value) && $value !== '' ? $value : null;
}
set_error_handler(static function (int $severity, string $message, string $file, int $line): bool {
// PHP calls the handler for a call silenced with `@` as well, only with less reported: that
// warning was expected and is answered where it happens (an unreadable image is a 422).
@@ -125,6 +211,7 @@ set_error_handler(static function (int $severity, string $message, string $file,
function routeAuth(array $parts, string $method, Auth $auth): never
{
$what = $parts[1] ?? '';
$throttle = new LoginThrottle(DATA_DIR . '/sign-ins');
if ($what === 'session') {
if ($method !== 'GET') {
@@ -141,8 +228,10 @@ function routeAuth(array $parts, string $method, Auth $auth): never
Response::error('Method not allowed', 405);
}
$body = Response::body();
refuseWhileThrottled($throttle);
$user = $auth->logIn((string) ($body['username'] ?? ''), (string) ($body['password'] ?? ''));
if ($user === null) {
countFailure($throttle);
// One message for both halves: which of the two was wrong is the attacker's question.
Response::error('Wrong user name or password', 401);
}
@@ -161,9 +250,13 @@ function routeAuth(array $parts, string $method, Auth $auth): never
if (!UserStore::isValidPassword($new)) {
Response::error('The new password needs at least ' . UserStore::MIN_PASSWORD_LENGTH . ' characters', 422);
}
// Guessing the current password from an open tab is guessing all the same.
refuseWhileThrottled($throttle);
if (!$auth->changePassword($user, (string) ($body['currentPassword'] ?? ''), $new)) {
countFailure($throttle);
Response::error('Wrong password', 401);
}
agentTokens()->endAllOf($user);
Response::noContent();
}
@@ -178,10 +271,47 @@ function routeAuth(array $parts, string $method, Auth $auth): never
Response::error('Not found', 404);
}
/** @param list<string> $parts */
function routeProjects(array $parts, string $method, string $owner, bool $seesAll): never
/** The address a request came from: nginx puts the one the Brandmauer saw there (#427). */
function clientAddress(): string
{
$store = new Store(dirname(__DIR__) . '/data/projects', $owner, $seesAll);
return (string) ($_SERVER['REMOTE_ADDR'] ?? '');
}
function countFailure(LoginThrottle $throttle, ?string $address = null): void
{
$throttle->fail($address ?? clientAddress());
// Now and then, so the folder holds the addresses of the last quarter of an hour and no more.
if (random_int(1, 20) === 1) {
$throttle->prune();
}
}
function refuseWhileThrottled(LoginThrottle $throttle, ?string $address = null): void
{
$wait = $throttle->waitFor($address ?? clientAddress());
if ($wait === 0) {
return;
}
header('Retry-After: ' . $wait);
Response::error('Too many failed sign-ins. Try again in ' . (int) ceil($wait / 60) . ' minutes', 429);
}
/**
* Bytes a project takes up as stored, 0 for one that is not there.
*/
function projectBytes(string $id): int
{
$path = DATA_DIR . "/projects/{$id}.json";
return is_file($path) ? (int) filesize($path) : 0;
}
/**
* @param list<string> $parts
* @param Closure(int):void $fits answers 413 when that many more bytes do not fit the quota
*/
function routeProjects(array $parts, string $method, string $owner, bool $seesAll, Closure $fits): never
{
$store = new Store(DATA_DIR . '/projects', $owner, $seesAll);
if (count($parts) === 1) {
if ($method === 'GET') {
@@ -193,6 +323,8 @@ function routeProjects(array $parts, string $method, string $owner, bool $seesAl
if ($name === '' || mb_strlen($name) > 120) {
Response::error('Project name must be 1–120 characters', 422);
}
// What an empty project takes up, give or take its name.
$fits(1024);
Response::json($store->create($name), 201);
}
Response::error('Method not allowed', 405);
@@ -203,14 +335,22 @@ function routeProjects(array $parts, string $method, string $owner, bool $seesAl
Response::error('Invalid project id', 400);
}
$snapshots = new SnapshotStore(dirname(__DIR__) . '/data/snapshots');
$snapshots = new SnapshotStore(DATA_DIR . '/snapshots');
if (($parts[2] ?? null) === 'snapshots') {
routeSnapshots($parts, $method, $store, $snapshots, $id);
routeSnapshots($parts, $method, $store, $snapshots, $id, $fits);
}
$releases = new ReleaseStore(dirname(__DIR__) . '/data/releases');
$releases = new ReleaseStore(DATA_DIR . '/releases');
if (($parts[2] ?? null) === 'release' && count($parts) === 3) {
routeRelease($method, $store, $releases, $id);
}
$demos = new ReleaseStore(DATA_DIR . '/demos');
if (($parts[2] ?? null) === 'demo' && count($parts) === 3) {
// What guests get to see is the administrators' call (#428).
if (!$seesAll) {
Response::error('Administrators only', 403);
}
routeRelease($method, $store, $demos, $id, 'Project has no demo copy');
}
if (count($parts) > 2) {
Response::error('Not found', 404);
}
@@ -228,6 +368,11 @@ function routeProjects(array $parts, string $method, string $owner, bool $seesAl
if (!isset($body['revision']) || !is_int($body['revision']) || !is_array($body['project'] ?? null)) {
Response::error('Body must contain integer "revision" and object "project"', 422);
}
// Asked after whether the project is there for them: a 413 would tell a stranger it is.
if ($store->load($id) === null) {
Response::error('Project not found', 404);
}
$fits((int) ($_SERVER['CONTENT_LENGTH'] ?? 0) - projectBytes($id));
try {
$result = $store->save($id, $body['revision'], $body['project']);
} catch (StaleRevisionException $e) {
@@ -250,6 +395,7 @@ function routeProjects(array $parts, string $method, string $owner, bool $seesAl
}
$snapshots->deleteAll($id);
$releases->takeDown($id);
$demos->takeDown($id);
Response::noContent();
}
@@ -260,9 +406,13 @@ function routeProjects(array $parts, string $method, string $owner, bool $seesAl
* The snapshots of one project (#231). Whoever may load the project may see, make, restore and
* delete its snapshots; for everybody else the project, and with it this route, is not there.
*
* A snapshot copies the project, and so does restoring one (the stand it replaces is kept), so both
* count against the quota with the project's size.
*
* @param list<string> $parts
* @param Closure(int):void $fits
*/
function routeSnapshots(array $parts, string $method, Store $store, SnapshotStore $snapshots, string $id): never
function routeSnapshots(array $parts, string $method, Store $store, SnapshotStore $snapshots, string $id, Closure $fits): never
{
$doc = $store->load($id);
if ($doc === null) {
@@ -275,6 +425,7 @@ function routeSnapshots(array $parts, string $method, Store $store, SnapshotStor
}
if ($method === 'POST') {
$name = snapshotName(Response::body());
$fits(projectBytes($id));
Response::json($snapshots->create($id, $doc, $name === '' ? gmdate('Y-m-d H:i') . ' UTC' : $name), 201);
}
Response::error('Method not allowed', 405);
@@ -307,6 +458,7 @@ function routeSnapshots(array $parts, string $method, Store $store, SnapshotStor
if (!isset($body['revision']) || !is_int($body['revision'])) {
Response::error('Body must contain integer "revision"', 422);
}
$fits(projectBytes($id));
// The same guard a save has: a tab that is behind must not throw away what another one saved.
if ($body['revision'] !== $doc['revision']) {
Response::error('Project was modified elsewhere', 409, ['revision' => $doc['revision']]);
@@ -327,11 +479,17 @@ function routeSnapshots(array $parts, string $method, Store $store, SnapshotStor
}
/**
* The released copy of one project (#266). Whoever may load the project may publish it, update it
* and take it down; publishing copies the project as stored, not what a tab has not saved yet.
* The released copy of one project (#266), or its demo copy (#428). Whoever may load the project may
* publish it, update it and take it down; publishing copies the project as stored, not what a tab
* has not saved yet.
*/
function routeRelease(string $method, Store $store, ReleaseStore $releases, string $id): never
{
function routeRelease(
string $method,
Store $store,
ReleaseStore $releases,
string $id,
string $none = 'Project is not released',
): never {
$doc = $store->load($id);
if ($doc === null) {
Response::error('Project not found', 404);
@@ -344,26 +502,32 @@ function routeRelease(string $method, Store $store, ReleaseStore $releases, stri
}
if ($method === 'DELETE') {
if (!$releases->takeDown($id)) {
Response::error('Project is not released', 404);
Response::error($none, 404);
}
Response::noContent();
}
Response::error('Method not allowed', 405);
}
/** The one route open without a sign-in: a released game, found by its token. */
function routeReleases(array $parts, string $method): never
/**
* What is open without a sign-in: a released game (#266) and a demo copy (#428), each found by its
* token. The demos are listed as well, for a guest to pick one.
*/
function routeCopies(array $parts, string $method, ReleaseStore $copies, string $missing, bool $listed): never
{
if ($method !== 'GET' || count($parts) !== 2) {
if ($method !== 'GET' || count($parts) > 2 || (count($parts) === 1 && !$listed)) {
Response::error('Not found', 404);
}
$release = (new ReleaseStore(dirname(__DIR__) . '/data/releases'))->load($parts[1]);
if ($release === null) {
Response::error('Game not found', 404);
if (count($parts) === 1) {
Response::json($copies->list());
}
$copy = $copies->load($parts[1]);
if ($copy === null) {
Response::error($missing, 404);
}
// The copy goes out under the token, not the project's own id: that one is nobody's business.
$project = ['id' => $release['token']] + $release['project'];
Response::json(['name' => $release['name'], 'publishedAt' => $release['publishedAt'], 'project' => $project]);
$project = ['id' => $copy['token']] + $copy['project'];
Response::json(['name' => $copy['name'], 'publishedAt' => $copy['publishedAt'], 'project' => $project]);
}
function snapshotName(array $body): string
@@ -375,11 +539,112 @@ function snapshotName(array $body): string
return $name;
}
function agentTokens(): AgentTokens
{
return new AgentTokens(DATA_DIR . '/agent-tokens', DATA_DIR . '/agent-tickets');
}
/**
* Whether the request is the MCP relay's (#429): it carries the key that lies in `data/` for both
* sides. Nobody else can read that folder, nginx serves none of it.
*/
function fromRelay(): bool
{
$path = DATA_DIR . '/mcp-relay.key';
$key = is_file($path) ? trim((string) file_get_contents($path)) : '';
$given = (string) ($_SERVER['HTTP_X_RELAY_KEY'] ?? '');
return strlen($key) >= 32 && hash_equals($key, $given);
}
/**
* The relay asks who is behind an agent's token or a tab's ticket. A wrong token is a wrong password
* to the brake, counted against the agent's address, not the relay's.
*
* @param list<string> $parts
*/
function routeRelay(array $parts, string $method, Auth $auth): never
{
if (!fromRelay() || count($parts) !== 2) {
Response::error('Not found', 404);
}
if ($method !== 'POST') {
Response::error('Method not allowed', 405);
}
$body = Response::body();
$tokens = agentTokens();
if ($parts[1] === 'verify') {
$throttle = new LoginThrottle(DATA_DIR . '/sign-ins');
$address = (string) ($body['address'] ?? '');
refuseWhileThrottled($throttle, $address);
$user = $tokens->verify((string) ($body['token'] ?? ''));
if ($user === null) {
countFailure($throttle, $address);
Response::error('Unknown or expired agent token', 401);
}
Response::json(['user' => $user, 'role' => $auth->role($user)]);
}
if ($parts[1] === 'redeem') {
$user = $tokens->redeemTicket((string) ($body['ticket'] ?? ''));
if ($user === null) {
Response::error('Unknown or used ticket', 401);
}
Response::json(['user' => $user, 'role' => $auth->role($user)]);
}
Response::error('Not found', 404);
}
/** @param list<string> $parts */
function routeAgentTokens(array $parts, string $method, string $user): never
{
$tokens = agentTokens();
if (count($parts) === 1) {
if ($method === 'GET') {
Response::json($tokens->listOf($user));
}
if ($method === 'POST') {
$body = Response::body();
$name = (string) ($body['name'] ?? '');
$days = $body['days'] ?? AgentTokens::DEFAULT_DAYS;
if (!AgentTokens::isValidName($name)) {
Response::error('A token name is 1–60 characters', 422);
}
if (!is_int($days) || $days < 1 || $days > AgentTokens::MAX_DAYS) {
Response::error('A token lasts 1–' . AgentTokens::MAX_DAYS . ' days', 422);
}
try {
Response::json($tokens->create($user, $name, $days), 201);
} catch (TooManyAgentTokensException $e) {
Response::error($e->getMessage(), 409);
}
}
Response::error('Method not allowed', 405);
}
if ($method !== 'DELETE') {
Response::error('Method not allowed', 405);
}
if (!$tokens->revoke($user, $parts[1])) {
Response::error('Agent token not found', 404);
}
Response::noContent();
}
/** @param list<string> $parts */
function routeAgent(array $parts, string $method, string $user): never
{
if (($parts[1] ?? '') !== 'ticket' || count($parts) !== 2) {
Response::error('Not found', 404);
}
if ($method !== 'POST') {
Response::error('Method not allowed', 405);
}
Response::json(['ticket' => agentTokens()->issueTicket($user), 'expiresIn' => AgentTokens::TICKET_SECONDS]);
}
/** @param list<string> $parts */
function routeUsers(array $parts, string $method, Auth $auth, UserStore $users): never
{
$actor = $auth->requireAdmin();
$projects = new Store(dirname(__DIR__) . '/data/projects', $actor, true);
$projects = new Store(DATA_DIR . '/projects', $actor, true);
$shown = static function (string $name) use ($users): array {
foreach ($users->accounts() as $account) {
if (strcasecmp($account['username'], $name) === 0) {
@@ -458,6 +723,7 @@ function routeUsers(array $parts, string $method, Auth $auth, UserStore $users):
if ($password !== null) {
$auth->resetPassword($target, $password);
agentTokens()->endAllOf($target);
}
if ($newRole !== null && $newRole !== $role) {
$users->setRole($target, $newRole);
@@ -467,6 +733,7 @@ function routeUsers(array $parts, string $method, Auth $auth, UserStore $users):
$users->rename($name, $newName);
$projects->transfer($name, $newName);
$auth->endSessionsOf($name);
agentTokens()->endAllOf($name);
$name = $newName;
}
Response::json($shown($name));
@@ -483,6 +750,7 @@ function routeUsers(array $parts, string $method, Auth $auth, UserStore $users):
}
$users->remove($target);
$auth->endSessionsOf($target);
agentTokens()->endAllOf($target);
Response::noContent();
}
@@ -493,10 +761,12 @@ function routeUsers(array $parts, string $method, Auth $auth, UserStore $users):
* Images, fonts and sounds, stored under the hash of their bytes in a folder per account (#344).
* Everybody signed in uploads into their own folder and lists their own files and nobody's; an
* administrator lists every file. Deleting one is for its owner or an administrator, and refused
* while a project of any account still uses it.
* while a project of any account still uses it. What an administrator offers as demo material
* (#428) is in every list.
*
* @param list<string> $parts
* @param class-string<RuntimeException> $invalid what the store throws for bytes it cannot take
* @param Closure(int):void $fits
*/
function routeFiles(
array $parts,
@@ -508,18 +778,32 @@ function routeFiles(
int $maxBytes,
string $noun,
string $invalid,
Closure $fits,
): never {
$kind = $parts[0];
$demos = new DemoFiles(DATA_DIR . '/demo-files.json');
if (count($parts) === 1) {
if ($method === 'GET') {
Response::json($store->list($users->folderOf($user), $isAdmin, $users->folderOwners()));
Response::json(filesOffered($store, $kind, $user, $isAdmin, $users));
}
if ($method === 'POST') {
$bytes = Response::rawBody();
if (strlen($bytes) > $maxBytes) {
Response::error("{$noun} must be at most " . ($maxBytes / 1024 / 1024) . ' MB', 413);
}
$tooLarge = "{$noun} must be at most " . ($maxBytes / 1024 / 1024) . ' MB';
$name = (string) ($_GET['name'] ?? '');
$fits((int) ($_SERVER['CONTENT_LENGTH'] ?? 0));
try {
Response::json($store->store($bytes, (string) ($_GET['name'] ?? ''), $users->folderOf($user, true)), 201);
// A video is too big for memory and goes from the request straight to disk.
if ($store instanceof VideoStore) {
$in = fopen('php://input', 'rb');
$convert = ($_GET['convert'] ?? '1') !== '0';
Response::json($store->storeStream($in, $maxBytes, $name, $users->folderOf($user, true), $convert), 201);
}
$bytes = Response::rawBody();
if (strlen($bytes) > $maxBytes) {
Response::error($tooLarge, 413);
}
Response::json($store->store($bytes, $name, $users->folderOf($user, true)), 201);
} catch (VideoTooLargeException) {
Response::error($tooLarge, 413);
} catch (RuntimeException $e) {
if (!$e instanceof $invalid) {
throw $e;
@@ -530,6 +814,36 @@ function routeFiles(
Response::error('Method not allowed', 405);
}
// A video kept without being converted is converted later, at its owner's word.
if ($store instanceof VideoStore && $method === 'POST' && end($parts) === 'convert') {
$file = implode('/', array_slice($parts, 1, -1));
$folder = HashedFileStore::folderOf($file);
if (!$isAdmin && ($folder === '' || $folder !== $users->folderOf($user))) {
Response::error('Only its owner may convert this video', 403);
}
$record = $store->convert($file);
if ($record === null) {
Response::error("{$noun} not found", 404);
}
Response::json($record);
}
if ($method === 'PATCH') {
$file = implode('/', array_slice($parts, 1));
if (!$isAdmin) {
Response::error('Administrators only', 403);
}
if (!$store->exists($file)) {
Response::error("{$noun} not found", 404);
}
$body = Response::body();
if (!is_bool($body['demo'] ?? null)) {
Response::error('Body must contain boolean "demo"', 422);
}
$demos->set($kind, $file, $body['demo']);
Response::noContent();
}
if ($method === 'DELETE') {
$file = implode('/', array_slice($parts, 1));
// Asked before whether there is one: somebody else's folder tells a stranger nothing.
@@ -540,24 +854,63 @@ function routeFiles(
if (!$store->exists($file)) {
Response::error("{$noun} not found", 404);
}
$uses = (new FileUsage(dirname(__DIR__) . '/data'))->of($file);
// Everybody may have it in a project by now; whether that ends is the administrators' call.
if (!$isAdmin && $demos->has($kind, $file)) {
Response::error('Offered to everybody as demo material: only an administrator can take that back', 409);
}
$uses = (new FileUsage(DATA_DIR))->of($file);
if ($uses !== []) {
Response::error(FileUsage::refusal($uses, $user, $isAdmin), 409);
}
$store->delete($file);
$demos->set($kind, $file, false);
Response::noContent();
}
Response::error('Method not allowed', 405);
}
/**
* What a picker of `$kind` lists: the files of the one asking and nobody's, an administrator's every
* file (#344), and everything offered as demo material (#428), with `demo` saying which those are.
* Signed out (`$user` null) it is the demo material alone, without who uploaded it.
*
* @return list<array<string,mixed>>
*/
function filesOffered(HashedFileStore $store, string $kind, ?string $user, bool $isAdmin, UserStore $users): array
{
$demo = (new DemoFiles(DATA_DIR . '/demo-files.json'))->of($kind);
$owners = $users->folderOwners();
$items = $user === null ? [] : $store->list($users->folderOf($user), $isAdmin, $owners);
if ($demo !== [] && !$isAdmin) {
$listed = array_column($items, 'file');
$hidden = $user === null ? ['owner' => null] : [];
foreach ($store->list('', true, $owners) as $item) {
if (in_array($item['file'], $demo, true) && !in_array($item['file'], $listed, true)) {
$items[] = ['deletable' => false, ...$hidden] + $item;
}
}
}
return array_map(static fn(array $item): array => $item + ['demo' => in_array($item['file'], $demo, true)], $items);
}
function fileStore(string $kind): HashedFileStore
{
return match ($kind) {
'textures' => new TextureStore(DATA_DIR . '/textures'),
'fonts' => new FontStore(DATA_DIR . '/fonts'),
'sounds' => new SoundStore(DATA_DIR . '/sounds'),
'videos' => new VideoStore(DATA_DIR . '/videos'),
};
}
/** @param list<string> $parts */
function routePixels(array $parts, string $method): never
{
if (count($parts) !== 1 || $method !== 'POST') {
Response::error('Method not allowed', 405);
}
$store = new PixelStore(dirname(__DIR__) . '/data/pixels');
$store = new PixelStore(DATA_DIR . '/pixels');
$bytes = Response::rawBody();
if (strlen($bytes) > PixelStore::MAX_BYTES) {
Response::error('A cel is at most ' . PixelStore::MAX_BYTES . ' bytes', 413);
@@ -572,7 +925,7 @@ function routePixels(array $parts, string $method): never
/** @param list<string> $parts */
function routePlugins(array $parts, string $method): never
{
$store = new PluginStore(dirname(__DIR__) . '/data/plugins');
$store = new PluginStore(DATA_DIR . '/plugins');
if (count($parts) === 1) {
if ($method !== 'GET') {
@@ -600,18 +953,26 @@ function routePlugins(array $parts, string $method): never
}
try {
if ((int) ($_SERVER['CONTENT_LENGTH'] ?? 0) > MAX_BODY_BYTES) {
Response::error('Request body too large', 413);
}
$method = $_SERVER['REQUEST_METHOD'] ?? 'GET';
$route = trim((string) ($_GET['r'] ?? ''), '/');
$parts = $route === '' ? [] : explode('/', $route);
$site = (string) ($_SERVER['HTTP_SEC_FETCH_SITE'] ?? '');
if ($method !== 'GET' && $method !== 'HEAD' && ($site === 'cross-site' || $site === 'same-site')) {
Response::error('Requests from other sites are refused', 403);
}
$maxBody = ($parts[0] ?? '') === 'videos' ? MAX_VIDEO_BYTES : MAX_BODY_BYTES;
if ((int) ($_SERVER['CONTENT_LENGTH'] ?? 0) > $maxBody) {
Response::error('Request body too large', 413);
}
// A project reaches deepest, `projects/{id}/snapshots/{snapshotId}/restore`; a stored file is
// `textures/{folder}/{file}`; everything else stops at two parts.
// `textures/{folder}/{file}`, a video to convert `videos/{folder}/{file}/convert`; everything
// else stops at two parts.
$depth = match ($parts[0] ?? '') {
'projects' => 5,
'videos' => 4,
'textures', 'fonts', 'sounds' => 3,
default => 2,
};
@@ -619,8 +980,8 @@ try {
Response::error('Not found', 404);
}
$users = new UserStore(dirname(__DIR__) . '/data/users.json');
$auth = new Auth($users, new SessionStore(dirname(__DIR__) . '/data/sessions'));
$users = new UserStore(DATA_DIR . '/users.json');
$auth = new Auth($users, new SessionStore(DATA_DIR . '/sessions'));
if ($parts[0] === 'auth') {
routeAuth($parts, $method, $auth);
}
@@ -628,26 +989,50 @@ try {
routeUsers($parts, $method, $auth, $users);
}
if ($parts[0] === 'releases') {
routeReleases($parts, $method);
routeCopies($parts, $method, new ReleaseStore(DATA_DIR . '/releases'), 'Game not found', false);
}
if ($parts[0] === 'demos') {
routeCopies($parts, $method, new ReleaseStore(DATA_DIR . '/demos'), 'Demo not found', true);
}
if ($parts[0] === 'agent' && in_array($parts[1] ?? '', ['verify', 'redeem'], true)) {
routeRelay($parts, $method, $auth);
}
// A guest's editor runs the plugins a demo switches on (#428); they are the server's own code,
// put there by its administrator, and only to be read.
if ($parts[0] === 'plugins' && $method === 'GET') {
routePlugins($parts, $method);
}
// A guest's pickers list the demo material without a session (#428), and nothing else.
if (in_array($parts[0], DemoFiles::KINDS, true) && count($parts) === 1 && $method === 'GET' && $auth->user() === null) {
Response::json(filesOffered(fileStore($parts[0]), $parts[0], null, false, $users));
}
// Being signed in is the price of admission for everything below, so an uploaded file cannot be
// dropped in by a stranger.
$user = $auth->require();
$isAdmin = $auth->role($user) === 'admin';
$data = dirname(__DIR__) . '/data';
$fits = static function (int $adding) use ($user, $isAdmin, $users): void {
if ($isAdmin) {
return;
}
$refusal = (new Quota(DATA_DIR, QUOTA_BYTES))->refusal($user, $users->folderOf($user), $adding);
if ($refusal !== null) {
Response::error($refusal, 413);
}
};
match ($parts[0]) {
'projects' => routeProjects($parts, $method, $user, $isAdmin),
'projects' => routeProjects($parts, $method, $user, $isAdmin, $fits),
'textures' => routeFiles(
$parts,
$method,
$user,
$isAdmin,
$users,
new TextureStore("{$data}/textures"),
fileStore('textures'),
MAX_TEXTURE_BYTES,
'Image',
InvalidImageException::class,
$fits,
),
'fonts' => routeFiles(
$parts,
@@ -655,10 +1040,11 @@ try {
$user,
$isAdmin,
$users,
new FontStore("{$data}/fonts"),
fileStore('fonts'),
MAX_FONT_BYTES,
'Font',
InvalidFontException::class,
$fits,
),
'sounds' => routeFiles(
$parts,
@@ -666,16 +1052,31 @@ try {
$user,
$isAdmin,
$users,
new SoundStore("{$data}/sounds"),
fileStore('sounds'),
MAX_SOUND_BYTES,
'Sound',
InvalidSoundException::class,
$fits,
),
'videos' => routeFiles(
$parts,
$method,
$user,
$isAdmin,
$users,
fileStore('videos'),
MAX_VIDEO_BYTES,
'Video',
InvalidVideoException::class,
$fits,
),
'pixels' => routePixels($parts, $method),
'plugins' => routePlugins($parts, $method),
'agent-tokens' => routeAgentTokens($parts, $method, $user),
'agent' => routeAgent($parts, $method, $user),
default => Response::error('Not found', 404),
};
} catch (Throwable $e) {
error_log('[game-maker api] ' . $e->getMessage());
error_log('[feuerwerk api] ' . $e->getMessage());
Response::error('Internal server error', 500);
}
api/lib/AgentTokens.php
+238
View File
@@ -0,0 +1,238 @@
<?php
declare(strict_types=1);
/**
* The keys of the MCP relay (#429): agent tokens, which an account makes for an agent such as
* Claude Code, and tickets, with which an editor tab gets onto the relay.
*
* A token is shown once, when it is made, and kept as `{dir}/{sha256(token)}.json` like a session:
* the file on disk cannot be replayed as the token. It opens the relay and nothing else, the API
* takes no token. It ends at its expiry, at its owner's word, and with every login of the account
* when the password changes or the account is renamed or deleted.
*
* A ticket is what a tab gets from the API with its session cookie, and hands to the relay on its
* WebSocket: the cookie belongs to the API's folder and never reaches the relay. It lasts a minute
* and is taken once.
*/
final class AgentTokens
{
public const PREFIX = 'fwa_';
public const DEFAULT_DAYS = 90;
public const MAX_DAYS = 365;
public const MAX_PER_ACCOUNT = 20;
public const TICKET_SECONDS = 60;
/** How often a use is written down: every request of an agent would cost a write otherwise. */
private const USE_RESOLUTION_SECONDS = 60;
/** @param (Closure():int)|null $clock the tests' way to move time on */
public function __construct(
private readonly string $dir,
private readonly string $ticketDir,
private readonly ?Closure $clock = null,
) {}
public static function isValidName(string $name): bool
{
return trim($name) !== '' && mb_strlen($name) <= 60;
}
/**
* Makes a token for `$user`. The answer is the only place the token itself ever appears.
*
* @return array{id:string,name:string,createdAt:int,expiresAt:int,usedAt:null,token:string}
*/
public function create(string $user, string $name, int $days): array
{
if (count($this->listOf($user)) >= self::MAX_PER_ACCOUNT) {
throw new TooManyAgentTokensException('An account has at most ' . self::MAX_PER_ACCOUNT . ' agent tokens');
}
$token = self::PREFIX . bin2hex(random_bytes(32));
$now = $this->now();
$record = [
'id' => bin2hex(random_bytes(8)),
'username' => $user,
'name' => trim($name),
'createdAt' => $now,
'expiresAt' => $now + max(1, min(self::MAX_DAYS, $days)) * 86400,
'usedAt' => null,
];
$this->write($this->dir . '/' . hash('sha256', $token) . '.json', $record);
return self::shown($record) + ['token' => $token];
}
/**
* The tokens of `$user` that have not expired, oldest first, without the tokens themselves.
*
* @return list<array{id:string,name:string,createdAt:int,expiresAt:int,usedAt:?int}>
*/
public function listOf(string $user): array
{
$found = [];
foreach ($this->records() as $record) {
if (strcasecmp($record['username'], $user) === 0) {
$found[] = self::shown($record);
}
}
usort($found, static fn(array $a, array $b): int => $a['createdAt'] <=> $b['createdAt']);
return $found;
}
/** Ends one token of `$user`; false when they have none by that id. */
public function revoke(string $user, string $id): bool
{
foreach ($this->records() as $path => $record) {
if ($record['id'] === $id && strcasecmp($record['username'], $user) === 0) {
return @unlink($path);
}
}
return false;
}
/** Ends every token of `$user`, and answers how many there were. */
public function endAllOf(string $user): int
{
$ended = 0;
foreach ($this->records() as $path => $record) {
if (strcasecmp($record['username'], $user) === 0 && @unlink($path)) {
$ended++;
}
}
return $ended;
}
/** The account a token belongs to, or null when it is unknown, malformed or expired. */
public function verify(string $token): ?string
{
if (!preg_match('/^' . self::PREFIX . '[0-9a-f]{64}$/', $token)) {
return null;
}
$path = $this->dir . '/' . hash('sha256', $token) . '.json';
$record = self::read($path);
if ($record === null) {
return null;
}
$now = $this->now();
if ($record['expiresAt'] <= $now) {
@unlink($path);
return null;
}
if ($record['usedAt'] === null || $now - $record['usedAt'] >= self::USE_RESOLUTION_SECONDS) {
$record['usedAt'] = $now;
$this->write($path, $record);
}
return $record['username'];
}
/** A ticket for `$user`'s tab to get onto the relay with. */
public function issueTicket(string $user): string
{
$this->sweepTickets();
$ticket = bin2hex(random_bytes(32));
$this->write($this->ticketDir . '/' . hash('sha256', $ticket) . '.json', [
'username' => $user,
'createdAt' => $this->now(),
]);
return $ticket;
}
/** The account a ticket was issued to, once: a second redemption finds nothing. */
public function redeemTicket(string $ticket): ?string
{
if (!preg_match('/^[0-9a-f]{64}$/', $ticket)) {
return null;
}
$path = $this->ticketDir . '/' . hash('sha256', $ticket) . '.json';
// Moved aside before it is read: of two redemptions at once, one gets the file.
$taken = $path . '.taken-' . bin2hex(random_bytes(4));
if (!@rename($path, $taken)) {
return null;
}
$raw = file_get_contents($taken);
@unlink($taken);
$data = $raw === false ? null : json_decode($raw, true);
if (!is_array($data) || !is_string($data['username'] ?? null) || !is_int($data['createdAt'] ?? null)) {
return null;
}
return $this->now() - $data['createdAt'] <= self::TICKET_SECONDS ? $data['username'] : null;
}
/** Throws out the tickets nobody redeemed. */
private function sweepTickets(): void
{
foreach (glob($this->ticketDir . '/*.json*') ?: [] as $path) {
if ($this->now() - (int) @filemtime($path) > self::TICKET_SECONDS) {
@unlink($path);
}
}
}
/**
* Every stored token by its path; expired ones are dropped on the way.
*
* @return array<string,array{id:string,username:string,name:string,createdAt:int,expiresAt:int,usedAt:?int}>
*/
private function records(): array
{
$records = [];
foreach (glob($this->dir . '/*.json') ?: [] as $path) {
$record = self::read($path);
if ($record === null) {
continue;
}
if ($record['expiresAt'] <= $this->now()) {
@unlink($path);
continue;
}
$records[$path] = $record;
}
return $records;
}
/** @return array{id:string,username:string,name:string,createdAt:int,expiresAt:int,usedAt:?int}|null */
private static function read(string $path): ?array
{
$raw = is_file($path) ? file_get_contents($path) : false;
$data = $raw === false ? null : json_decode($raw, true);
if (
!is_array($data)
|| !is_string($data['id'] ?? null)
|| !is_string($data['username'] ?? null)
|| !is_string($data['name'] ?? null)
|| !is_int($data['createdAt'] ?? null)
|| !is_int($data['expiresAt'] ?? null)
|| !(is_int($data['usedAt'] ?? null) || ($data['usedAt'] ?? null) === null)
) {
return null;
}
return $data;
}
/** @return array{id:string,name:string,createdAt:int,expiresAt:int,usedAt:?int} */
private static function shown(array $record): array
{
return [
'id' => $record['id'],
'name' => $record['name'],
'createdAt' => $record['createdAt'],
'expiresAt' => $record['expiresAt'],
'usedAt' => $record['usedAt'],
];
}
private function write(string $path, array $data): void
{
$dir = dirname($path);
if (!is_dir($dir) && !mkdir($dir, 0775, true) && !is_dir($dir)) {
throw new RuntimeException("Cannot create {$dir}");
}
AtomicFile::write($path, (string) json_encode($data, JSON_UNESCAPED_SLASHES), AtomicFile::HIDDEN, 'agent token');
}
private function now(): int
{
return $this->clock === null ? time() : ($this->clock)();
}
}
final class TooManyAgentTokensException extends RuntimeException {}
api/lib/Auth.php
+15 -5
View File
@@ -109,24 +109,34 @@ final class Auth
return (string) ($_COOKIE[SessionStore::COOKIE] ?? '');
}
/**
* The cookie has no `Path`: the browser takes the folder of the API as it sees it, which is
* `/feuerwerk/api` on nockal.com and `/api` on feuerwerk.nockal.com. nginx sees the second as
* the first, so no path this side writes down would be right for both. The editor's pages and
* the data never needed the cookie.
*
* Before that, the cookie was set for the whole site (`/feuerwerk/`). Such an old one is ended
* with every new one, or it would still sign a browser in after signing out.
*/
private function setCookie(string $value, int $expires): void
{
setcookie(SessionStore::COOKIE, $value, [
$options = [
'expires' => $expires,
'path' => self::basePath(),
'httponly' => true,
'samesite' => 'Lax',
'secure' => ($_SERVER['HTTPS'] ?? '') !== '',
]);
];
setcookie(SessionStore::COOKIE, $value, $options);
setcookie(SessionStore::COOKIE, '', ['expires' => time() - 3600, 'path' => self::basePath()] + $options);
}
/**
* The path the editor is served under (`/game-maker/`), read off the script rather than
* The path the old cookie was set for (`/feuerwerk/`), read off the script rather than
* written down: the same checkout answers under a different prefix in CI.
*/
private static function basePath(): string
{
$script = (string) ($_SERVER['SCRIPT_NAME'] ?? '/game-maker/api/index.php');
$script = (string) ($_SERVER['SCRIPT_NAME'] ?? '/feuerwerk/api/index.php');
$base = dirname(dirname($script));
return $base === '/' || $base === '\\' || $base === '.' ? '/' : $base . '/';
}
api/lib/DemoFiles.php
+62
View File
@@ -0,0 +1,62 @@
<?php
declare(strict_types=1);
/**
* Uploaded files an administrator offers to everybody as demo material (#428): every account's
* picker lists them beside its own, and a guest's picker lists nothing else. The files stay where
* they were uploaded, in their owner's folder; this only says which of them are offered.
*
* data/demo-files.json { "textures": ["{folder}/{sha256}.png", …], "sounds": […], … }
*
* Written 0640 like everything that is not a file to hand out.
*/
final class DemoFiles
{
public const KINDS = ['textures', 'fonts', 'sounds', 'videos'];
public function __construct(private readonly string $path) {}
/** @return list<string> the offered files of one kind, in the order they were offered */
public function of(string $kind): array
{
$files = $this->read()[$kind] ?? [];
return array_values(array_filter($files, 'is_string'));
}
public function has(string $kind, string $file): bool
{
return in_array($file, $this->of($kind), true);
}
/** Offers a file or takes the offer back; answers whether that changed anything. */
public function set(string $kind, string $file, bool $offered): bool
{
if (!in_array($kind, self::KINDS, true)) {
throw new InvalidArgumentException("Unknown kind: {$kind}");
}
$all = $this->read();
$files = $this->of($kind);
$has = in_array($file, $files, true);
if ($has === $offered) {
return false;
}
$all[$kind] = $offered ? [...$files, $file] : array_values(array_diff($files, [$file]));
$json = json_encode($all, JSON_UNESCAPED_SLASHES | JSON_PRETTY_PRINT);
if ($json === false) {
throw new RuntimeException('Cannot encode the demo files');
}
AtomicFile::write($this->path, $json . "\n", AtomicFile::HIDDEN, 'demo file list');
return true;
}
/** @return array<string,list<string>> */
private function read(): array
{
if (!is_file($this->path)) {
return [];
}
$raw = file_get_contents($this->path);
$data = $raw === false ? null : json_decode($raw, true);
return is_array($data) ? $data : [];
}
}
api/lib/FileUsage.php
+6 -4
View File
@@ -4,8 +4,9 @@ declare(strict_types=1);
/**
* Which projects still use a stored image, font or sound (#344) – asked before one is deleted.
*
* A project counts when the file is named in the project itself, in one of its snapshots or in its
* release: restoring a snapshot or playing a released game must not find a hole. Any string value
* A project counts when the file is named in the project itself, in one of its snapshots, in its
* release or in its demo copy: restoring a snapshot, playing a released game or trying a demo must
* not find a hole. Any string value
* equal to the stored name counts, wherever in the document it is; that may refuse a file nothing
* really draws, but never lets one go that something does.
*/
@@ -31,7 +32,8 @@ final class FileUsage
$used[basename(dirname($path))] = true;
}
}
foreach (glob("{$this->data}/releases/*.json") ?: [] as $path) {
// A demo copy (#428) is a release a guest opens in the editor, and stored like one.
foreach ([...glob("{$this->data}/releases/*.json") ?: [], ...glob("{$this->data}/demos/*.json") ?: []] as $path) {
$release = $this->decoded($path, $file);
if ($release !== null && is_string($release['projectId'] ?? null) && self::uses($release['project'] ?? null, $file)) {
$used[$release['projectId']] = true;
@@ -74,7 +76,7 @@ final class FileUsage
$named[] = $others === 1 ? 'a project of another account' : "{$others} projects of other accounts";
}
$list = count($named) > 1 ? implode(', ', array_slice($named, 0, -1)) . ' and ' . end($named) : $named[0];
return "Still used by {$list}, counting snapshots and releases";
return "Still used by {$list}, counting snapshots, releases and demo copies";
}
/** Whether a value holds `$file` as a string anywhere inside it. */
api/lib/LoginThrottle.php
+122
View File
@@ -0,0 +1,122 @@
<?php
declare(strict_types=1);
/**
* The brake on guessing passwords (#427): five failed sign-ins from one address within 15 minutes,
* and that address is refused for 15 minutes after the last of them. The right password is refused
* as well while it lasts, or guessing would only have to wait for its hit. A success does not wipe
* the count either: with an account of one's own, that would be the way to reset it.
*
* `{dir}/{sha256(address)}.json` holds the times of the recent failures, written 0640 like
* everything else that is private. The address is the one nginx found behind the Brandmauer.
*/
final class LoginThrottle
{
public const MAX_FAILURES = 5;
public const WINDOW_SECONDS = 15 * 60;
/** @param (Closure():int)|null $clock the tests' way to move time on */
public function __construct(
private readonly string $dir,
private readonly ?Closure $clock = null,
) {}
/**
* Seconds until another attempt counts, 0 when one may be made now.
*
* @param list<int> $failures when the recent failures were
*/
public static function wait(array $failures, int $now): int
{
$times = self::recent($failures, $now);
sort($times);
$until = 0;
for ($i = self::MAX_FAILURES - 1; $i < count($times); $i++) {
if ($times[$i] - $times[$i - self::MAX_FAILURES + 1] < self::WINDOW_SECONDS) {
$until = $times[$i] + self::WINDOW_SECONDS;
}
}
return max(0, $until - $now);
}
/** Seconds until `$address` may try again, 0 when it may now. */
public function waitFor(string $address): int
{
$path = $this->path($address);
$raw = is_file($path) ? file_get_contents($path) : false;
return self::wait($raw === false ? [] : self::decode($raw), $this->now());
}
/** Counts a failed attempt of `$address` and answers the wait that follows from it. */
public function fail(string $address): int
{
if (!is_dir($this->dir) && !mkdir($this->dir, 0775, true) && !is_dir($this->dir)) {
throw new RuntimeException('Cannot create the directory of failed sign-ins');
}
$path = $this->path($address);
$handle = fopen($path, 'c+');
if ($handle === false) {
throw new RuntimeException('Cannot count a failed sign-in');
}
try {
chmod($path, AtomicFile::HIDDEN);
flock($handle, LOCK_EX);
$now = $this->now();
$failures = self::recent(self::decode((string) stream_get_contents($handle)), $now);
$failures[] = $now;
ftruncate($handle, 0);
rewind($handle);
fwrite($handle, (string) json_encode(['failures' => $failures]));
fflush($handle);
return self::wait($failures, $now);
} finally {
flock($handle, LOCK_UN);
fclose($handle);
}
}
/** Removes the files of addresses whose failures are all too old to matter. */
public function prune(): int
{
// PHP keeps what it learned of a file for the rest of the request; a time read earlier
// would keep a file that has gone stale since, or drop one written since.
clearstatcache();
$removed = 0;
foreach (glob($this->dir . '/*.json') ?: [] as $path) {
if ((int) filemtime($path) < $this->now() - 2 * self::WINDOW_SECONDS && unlink($path)) {
$removed++;
}
}
return $removed;
}
/**
* The failures that can still matter: five of them lie within one window, and the lock lasts one
* window after the last, so nothing older than two windows can.
*
* @param list<int> $failures
* @return list<int>
*/
private static function recent(array $failures, int $now): array
{
return array_values(array_filter($failures, static fn(int $at): bool => $at > $now - 2 * self::WINDOW_SECONDS));
}
/** @return list<int> */
private static function decode(string $raw): array
{
$data = json_decode($raw, true);
$failures = is_array($data) && is_array($data['failures'] ?? null) ? $data['failures'] : [];
return array_values(array_map('intval', array_filter($failures, 'is_int')));
}
private function path(string $address): string
{
return $this->dir . '/' . hash('sha256', $address) . '.json';
}
private function now(): int
{
return $this->clock === null ? time() : ($this->clock)();
}
}
api/lib/Quota.php
+77
View File
@@ -0,0 +1,77 @@
<?php
declare(strict_types=1);
/**
* How much of the disk an account may take (#427): 1 GB for its uploads, its projects and their
* snapshots together. Administrators have no limit; the router does not ask for them.
*
* Only a write that makes an account larger is refused, so one that is full can still save a smaller
* project or delete files. Pixel cels are kept apart: they belong to nobody and are small by rule
* (`PixelStore::MAX_BYTES`).
*/
final class Quota
{
public const BYTES = 1024 * 1024 * 1024;
/** The upload folders that are counted, each with a folder per account. */
private const KINDS = ['textures', 'fonts', 'sounds', 'videos'];
public function __construct(
private readonly string $data,
private readonly int $limit = self::BYTES,
) {}
/** Bytes `$owner` takes up: everything in their upload folder, their projects and their snapshots. */
public function usage(string $owner, string $folder): int
{
$bytes = 0;
if (preg_match('/^' . UserStore::FOLDER_PATTERN . '$/', $folder)) {
foreach (self::KINDS as $kind) {
$bytes += self::sizeOf("{$this->data}/{$kind}/{$folder}");
}
}
$projects = new Store("{$this->data}/projects", $owner);
foreach ($projects->listOf($owner) as $project) {
$bytes += (int) @filesize("{$this->data}/projects/{$project['id']}.json");
$bytes += self::sizeOf("{$this->data}/snapshots/{$project['id']}");
}
return $bytes;
}
/** Why `$adding` more bytes do not fit, or null when they do (or add nothing). */
public function refusal(string $owner, string $folder, int $adding): ?string
{
if ($adding <= 0) {
return null;
}
$used = $this->usage($owner, $folder);
if ($used + $adding <= $this->limit) {
return null;
}
return sprintf(
'Not enough space: this account uses %s of its %s',
self::megabytes($used),
self::megabytes($this->limit),
);
}
private static function megabytes(int $bytes): string
{
return number_format($bytes / 1024 / 1024, $bytes < 10 * 1024 * 1024 ? 1 : 0, '.', '') . ' MB';
}
private static function sizeOf(string $dir): int
{
if (!is_dir($dir)) {
return 0;
}
$bytes = 0;
$files = new RecursiveIteratorIterator(new RecursiveDirectoryIterator($dir, FilesystemIterator::SKIP_DOTS));
foreach ($files as $file) {
if ($file->isFile()) {
$bytes += $file->getSize();
}
}
return $bytes;
}
}
api/lib/ReleaseStore.php
+22
View File
@@ -13,6 +13,9 @@ declare(strict_types=1);
*
* Like the snapshots, the store checks nothing about who asks: the router reaches the project half
* only after `Store::load` found the project for that user.
*
* Demo copies (#428) are the same thing on `data/demos/`: a frozen copy a guest opens in the editor
* instead of the player, made and updated by an administrator.
*/
final class ReleaseStore
{
@@ -82,6 +85,25 @@ final class ReleaseStore
];
}
/**
* Every copy there is, by name: what a guest picks a demo from (#428), which is this store on a
* directory of its own.
*
* @return list<array{token:string,name:string,publishedAt:string}>
*/
public function list(): array
{
$items = [];
foreach (glob($this->dir . '/*.json') ?: [] as $path) {
$release = $this->load(basename($path, '.json'));
if ($release !== null) {
$items[] = ['token' => $release['token'], 'name' => $release['name'], 'publishedAt' => $release['publishedAt']];
}
}
usort($items, static fn(array $a, array $b): int => strnatcasecmp($a['name'], $b['name']));
return $items;
}
/** Makes the link dead. False when the project had none. */
public function takeDown(string $projectId): bool
{
api/lib/SessionStore.php
+1 -1
View File
@@ -14,7 +14,7 @@ final class SessionStore
/** How long a login lasts without being used; every request that finds it pushes it out again. */
public const LIFETIME_SECONDS = 30 * 24 * 60 * 60;
public const COOKIE = 'game_maker_session';
public const COOKIE = 'feuerwerk_session';
public function __construct(private readonly string $dir) {}
api/lib/VideoFormat.php
+248
View File
@@ -0,0 +1,248 @@
<?php
declare(strict_types=1);
/**
* The one format every uploaded video is turned into (#402), as plain rules: what ffprobe said in,
* the ffmpeg arguments out. Nothing here runs a process, so all of it is testable without ffmpeg.
*
* The target is MP4 with H.264 and AAC, `yuv420p`, at a constant frame rate and with a keyframe
* every 15 frames: the editor seeks to clip starts and loops back to them, and a seek can only land
* on a keyframe quickly. The picture fits into 3840 × 2160 (2160 × 3840 standing up), so a 4K
* recording of the editor stays 4K, and is never made larger than it came.
*/
final class VideoFormat
{
public const MAX_LONG_SIDE = 3840;
public const MAX_SHORT_SIDE = 2160;
public const MAX_FPS = 60;
public const KEYFRAME_EVERY = 15;
/** Used when a file does not say its frame rate at all. */
private const FALLBACK_FPS = '30';
/**
* What the conversion makes of a probed file: the size it is shown at after rotation, scaled to
* fit, the frame rate as ffmpeg takes it, and whether there is sound to keep.
*
* @param array<string,mixed> $probe ffprobe's `-show_streams -show_format` as JSON
* @return array{width:int,height:int,fps:string,audio:bool,duration:float}
* @throws InvalidArgumentException when there is no picture in it
*/
public static function plan(array $probe): array
{
$streams = is_array($probe['streams'] ?? null) ? $probe['streams'] : [];
$video = self::picture($probe);
$audio = in_array('audio', array_column($streams, 'codec_type'), true);
if ($video === null || (int) ($video['width'] ?? 0) <= 0 || (int) ($video['height'] ?? 0) <= 0) {
throw new InvalidArgumentException('There is no picture in this file');
}
[$width, $height] = [(int) $video['width'], (int) $video['height']];
if (abs(self::rotation($video)) % 180 === 90) {
[$width, $height] = [$height, $width];
}
[$width, $height] = self::fit($width, $height);
return [
'width' => $width,
'height' => $height,
'fps' => self::fps($video),
'audio' => $audio,
'duration' => self::duration($probe, $video),
];
}
/**
* The size inside the box, keeping the shape, both sides even (H.264 in `yuv420p` needs that).
*
* @return array{0:int,1:int}
*/
public static function fit(int $width, int $height): array
{
[$boxW, $boxH] = $width >= $height
? [self::MAX_LONG_SIDE, self::MAX_SHORT_SIDE]
: [self::MAX_SHORT_SIDE, self::MAX_LONG_SIDE];
$scale = min(1.0, $boxW / $width, $boxH / $height);
$even = static fn(float $side): int => max(2, 2 * (int) floor(round($side, 3) / 2));
return [$even($width * $scale), $even($height * $scale)];
}
/**
* The frame rate the file is written at: its own average, unless that is above 60 or unknown.
* Kept as ffmpeg's fraction, so 29.97 stays 30000/1001 and frames do not drift against the time.
*
* @param array<string,mixed> $video
*/
public static function fps(array $video): string
{
foreach (['avg_frame_rate', 'r_frame_rate'] as $key) {
$rate = self::rate((string) ($video[$key] ?? ''));
if ($rate !== null) {
return $rate[1] > self::MAX_FPS ? (string) self::MAX_FPS : $rate[0];
}
}
return self::FALLBACK_FPS;
}
/** A fraction such as `30000/1001` as a number of frames per second. */
public static function fpsValue(string $fps): float
{
$rate = self::rate($fps);
return $rate === null ? 0.0 : $rate[1];
}
/**
* @param array{width:int,height:int,fps:string,audio:bool} $plan
* @return list<string>
*/
public static function convertArgs(string $source, string $target, array $plan): array
{
$args = [
'ffmpeg', '-nostdin', '-hide_banner', '-loglevel', 'error', '-y',
'-i', $source,
'-map', '0:v:0',
];
if ($plan['audio']) {
$args = [...$args, '-map', '0:a:0'];
}
$args = [
...$args,
'-vf', "scale={$plan['width']}:{$plan['height']}:flags=lanczos,setsar=1",
'-r', $plan['fps'], '-fps_mode', 'cfr',
'-c:v', 'libx264', '-preset', 'veryfast', '-crf', '21',
'-pix_fmt', 'yuv420p',
'-g', (string) self::KEYFRAME_EVERY, '-keyint_min', (string) self::KEYFRAME_EVERY, '-sc_threshold', '0',
];
$args = $plan['audio']
? [...$args, '-c:a', 'aac', '-b:a', '160k', '-ac', '2', '-ar', '48000']
: [...$args, '-an'];
return [...$args, '-sn', '-dn', '-map_metadata', '-1', '-movflags', '+faststart', '-progress', 'pipe:1', '-nostats', '-f', 'mp4', $target];
}
/**
* A still from a little way in (not the very first frame, which is often black), as wide as the
* picker draws it. A JPEG is always read as BT.601 in the full range, so the picture is turned
* into that from what the video is; left alone, a red of 255 in BT.709 came out as 229 (#411).
*
* @param array{matrix:string,range:string} $colour
* @return list<string>
*/
public static function posterArgs(string $video, string $target, float $duration, array $colour): array
{
$at = sprintf('%.3f', max(0.0, min(1.0, $duration / 2)));
$scale = "scale=320:-2:in_color_matrix={$colour['matrix']}:in_range={$colour['range']}"
. ':out_color_matrix=bt601:out_range=pc:flags=accurate_rnd+full_chroma_int';
return [
'ffmpeg', '-nostdin', '-hide_banner', '-loglevel', 'error', '-y',
'-ss', $at, '-i', $video, '-frames:v', '1',
'-vf', $scale, '-q:v', '4', '-f', 'image2', $target,
];
}
/**
* How the picture of a probed file is to be read, as ffmpeg's scale filter names it. Without a
* word in the file this is Chrome's guess, measured: BT.709 from 720 lines up, BT.601 below.
*
* @param array<string,mixed> $probe
* @return array{matrix:string,range:string}
*/
public static function colour(array $probe): array
{
$video = self::picture($probe) ?? [];
$matrix = match ($video['color_space'] ?? '') {
'bt709' => 'bt709',
'smpte170m', 'bt470bg' => 'bt601',
'bt2020nc', 'bt2020c' => 'bt2020',
'smpte240m' => 'smpte240m',
'fcc' => 'fcc',
default => (int) ($video['height'] ?? 0) >= 720 ? 'bt709' : 'bt601',
};
return ['matrix' => $matrix, 'range' => ($video['color_range'] ?? '') === 'pc' ? 'pc' : 'tv'];
}
/** @return list<string> */
public static function probeArgs(string $path): array
{
return ['ffprobe', '-v', 'error', '-print_format', 'json', '-show_streams', '-show_format', $path];
}
/**
* What the editor is told about a converted file, read back from the file itself rather than
* the plan: a crop or a rounding inside ffmpeg is what the editor has to draw.
*
* @param array<string,mixed> $probe
* @return array{width:int,height:int,duration:float,fps:float,audio:bool}
*/
public static function details(array $probe): array
{
$plan = self::plan($probe);
return [
'width' => $plan['width'],
'height' => $plan['height'],
'duration' => round($plan['duration'], 3),
'fps' => round(self::fpsValue($plan['fps']), 3),
'audio' => $plan['audio'],
];
}
/** The share of the work done, from one `out_time_us=` line of ffmpeg's progress. */
public static function progress(string $line, float $duration): ?float
{
if ($duration <= 0 || !preg_match('/^out_time_us=(\d+)$/', trim($line), $m)) {
return null;
}
return max(0.0, min(1.0, ((int) $m[1]) / 1e6 / $duration));
}
/**
* @param array<string,mixed> $probe
* @return array<string,mixed>|null the first video stream that moves
*/
private static function picture(array $probe): ?array
{
foreach (is_array($probe['streams'] ?? null) ? $probe['streams'] : [] as $stream) {
// A cover picture in a music file is a video stream as well, one that is a single frame.
$still = (int) (($stream['disposition'] ?? [])['attached_pic'] ?? 0) === 1;
if (($stream['codec_type'] ?? '') === 'video' && !$still) {
return $stream;
}
}
return null;
}
/** @param array<string,mixed> $video */
private static function rotation(array $video): int
{
foreach (is_array($video['side_data_list'] ?? null) ? $video['side_data_list'] : [] as $side) {
if (isset($side['rotation']) && is_numeric($side['rotation'])) {
return (int) $side['rotation'];
}
}
$tag = ($video['tags'] ?? [])['rotate'] ?? null;
return is_numeric($tag) ? (int) $tag : 0;
}
/** @param array<string,mixed> $probe @param array<string,mixed> $video */
private static function duration(array $probe, array $video): float
{
foreach ([$video['duration'] ?? null, ($probe['format'] ?? [])['duration'] ?? null] as $value) {
if (is_numeric($value) && (float) $value > 0) {
return (float) $value;
}
}
return 0.0;
}
/** @return array{0:string,1:float}|null the fraction as written and its value, null for none */
private static function rate(string $text): ?array
{
if (!preg_match('#^(\d+)(?:/(\d+))?$#', $text, $m)) {
return null;
}
$num = (int) $m[1];
$den = isset($m[2]) ? (int) $m[2] : 1;
if ($num <= 0 || $den <= 0) {
return null;
}
return [$den === 1 ? (string) $num : "{$num}/{$den}", $num / $den];
}
}
api/lib/VideoQueue.php
+188
View File
@@ -0,0 +1,188 @@
<?php
declare(strict_types=1);
/**
* The videos waiting to be converted (#402): `data/videos/queue/{folder}-{hash}.json` says what
* became of an upload, and its `source` where the upload lies – kept as it came beside the MP4,
* `{folder}/{hash}.source.{ext}`, for downloading it. The API adds to it, the worker in the
* container `feuerwerk-video` takes from it, one at a time, oldest first.
*
* A job is `queued`, `converting` (with `progress` from 0 to 1) or `failed` (with `error`). A done
* one is no job any more: its MP4 lies in the account's folder and the job is gone. A failed one
* stays, so the picker can say why, until the file is deleted or uploaded again.
*
* A job from before the uploads were kept has no `source`: its bytes wait hidden in the queue as
* `{folder}-{hash}.src` and go once they are converted.
*/
final class VideoQueue
{
public const QUEUED = 'queued';
public const CONVERTING = 'converting';
public const FAILED = 'failed';
public function __construct(private readonly string $dir)
{
if (!is_dir($this->dir) && !mkdir($this->dir, 0775, true) && !is_dir($this->dir)) {
throw new RuntimeException("Cannot create directory {$this->dir}");
}
}
/**
* Copies an upload into a temp file beside the jobs, never more than `$maxBytes` of it: a video
* is too big to be held in memory. The caller keeps the file or deletes it.
*
* @param resource $in
* @throws VideoTooLargeException when the stream has more than `$maxBytes`
*/
public function receive($in, int $maxBytes): string
{
$tmp = "{$this->dir}/upload-" . bin2hex(random_bytes(8)) . '.tmp';
$out = fopen($tmp, 'xb');
if ($out === false) {
throw new RuntimeException('Cannot write uploaded video');
}
$copied = stream_copy_to_stream($in, $out, $maxBytes + 1);
fclose($out);
if ($copied === false || $copied > $maxBytes) {
unlink($tmp);
throw $copied === false ? new RuntimeException('Cannot write uploaded video') : new VideoTooLargeException();
}
return $tmp;
}
/**
* Queues the upload kept at `$source`, a path inside the videos' directory such as
* `{folder}/{hash}.source.mkv`. The upload stays where it is.
*/
public function add(string $folder, string $hash, string $source, string $name): void
{
$this->write($folder, $hash, [
'folder' => $folder,
'hash' => $hash,
'source' => $source,
'name' => $name,
'state' => self::QUEUED,
'progress' => 0,
'error' => '',
'createdAt' => gmdate('c'),
]);
}
/** @return array<string,mixed>|null */
public function job(string $folder, string $hash): ?array
{
$path = $this->jobPath($folder, $hash);
if (!is_file($path)) {
return null;
}
$raw = @file_get_contents($path);
$job = $raw === false ? null : json_decode($raw, true);
return is_array($job) ? $job : null;
}
/** @return list<array<string,mixed>> every job, oldest first */
public function jobs(): array
{
$jobs = [];
foreach (glob($this->dir . '/*.json') ?: [] as $path) {
if (!preg_match('/^(' . UserStore::FOLDER_PATTERN . ')-([a-f0-9]{64})\.json$/', basename($path), $m)) {
continue;
}
$job = $this->job($m[1], $m[2]);
if ($job !== null) {
$jobs[] = $job;
}
}
usort($jobs, static fn(array $a, array $b): int => strcmp((string) $a['createdAt'], (string) $b['createdAt']));
return $jobs;
}
/** The oldest job nobody has started on, or null when there is none. */
public function next(): ?array
{
foreach ($this->jobs() as $job) {
if ($job['state'] === self::QUEUED && is_file($this->source($job['folder'], $job['hash']))) {
return $job;
}
}
return null;
}
/**
* Changes a job, unless it was deleted in the meantime: then false, and the one converting it
* knows to stop.
*
* @param array<string,mixed> $changes
*/
public function update(string $folder, string $hash, array $changes): bool
{
$job = $this->job($folder, $hash);
if ($job === null) {
return false;
}
$this->write($folder, $hash, $changes + $job);
return true;
}
/** A job that was being converted when the worker stopped is started again. */
public function requeueInterrupted(): int
{
$count = 0;
foreach ($this->jobs() as $job) {
if ($job['state'] === self::CONVERTING) {
$this->update($job['folder'], $job['hash'], ['state' => self::QUEUED, 'progress' => 0]);
$count++;
}
}
return $count;
}
/** Drops the job, and an old one's hidden upload; a kept upload is the store's to delete. */
public function remove(string $folder, string $hash): bool
{
$removed = false;
foreach ([$this->hiddenUpload($folder, $hash), $this->jobPath($folder, $hash)] as $path) {
if (is_file($path)) {
$removed = unlink($path) || $removed;
}
}
return $removed;
}
/**
* Drops an old job's hidden upload once it cannot be converted; the job itself says why. A kept
* upload stays: it can still be downloaded as it came.
*/
public function dropSource(string $folder, string $hash): void
{
$path = $this->hiddenUpload($folder, $hash);
if (is_file($path)) {
unlink($path);
}
}
/** Where the bytes of a job lie, for ffmpeg to read. */
public function source(string $folder, string $hash): string
{
$source = (string) ($this->job($folder, $hash)['source'] ?? '');
$kept = '#^' . UserStore::FOLDER_PATTERN . '/[a-f0-9]{64}\.source\.[a-z0-9]+$#';
return preg_match($kept, $source) ? dirname($this->dir) . "/{$source}" : $this->hiddenUpload($folder, $hash);
}
private function hiddenUpload(string $folder, string $hash): string
{
return "{$this->dir}/{$folder}-{$hash}.src";
}
private function jobPath(string $folder, string $hash): string
{
return "{$this->dir}/{$folder}-{$hash}.json";
}
/** @param array<string,mixed> $job */
private function write(string $folder, string $hash, array $job): void
{
$json = json_encode($job, JSON_UNESCAPED_UNICODE | JSON_UNESCAPED_SLASHES);
AtomicFile::write($this->jobPath($folder, $hash), (string) $json, AtomicFile::HIDDEN, 'video job');
}
}
api/lib/VideoStore.php
+359
View File
@@ -0,0 +1,359 @@
<?php
declare(strict_types=1);
/**
* Videos (#402): `data/videos/{folder}/{hash}.mp4`, beside it `{hash}.jpg` (the poster),
* `{hash}.meta.json` (size, length, frame rate, sound) and `{hash}.source.{ext}`, the upload as it
* came, all served as they are.
*
* What a project plays is the MP4 the worker makes of the upload (see VideoFormat), which waits in
* the VideoQueue until then. The upload itself is kept for downloading it, and one sent up without
* being converted stays only that until it is asked to be. The hash is the one of the uploaded
* bytes, so the name a project stores is known the moment the upload is in, long before the file
* exists, and uploading the same file twice converts it once. Videos uploaded before the uploads
* were kept have none.
*/
final class VideoStore extends HashedFileStore
{
/** An upload kept, waiting for nobody to convert it. */
public const UNCONVERTED = 'unconverted';
/**
* What an upload has to start with to be sent to ffmpeg at all: the ISO family (MP4, MOV, M4V,
* 3GP), Matroska (MKV, WebM), AVI and MPEG transport streams. ffmpeg has the last word.
*/
private const SIGNATURE_HINT = 'MP4, MOV, WebM, MKV or AVI';
/** The endings a kept upload may have, taken from its name; anything else is told by its bytes. */
private const SOURCE_EXTENSIONS = ['mp4', 'mov', 'm4v', '3gp', 'mkv', 'webm', 'avi', 'ts', 'mts', 'm2ts'];
private readonly VideoQueue $queue;
public function __construct(private readonly string $root)
{
parent::__construct($root);
$this->queue = new VideoQueue("{$root}/queue");
}
protected static function filePattern(): string
{
return '[a-f0-9]{64}\.mp4';
}
public function queue(): VideoQueue
{
return $this->queue;
}
/**
* Takes an upload: a file already converted answers as it is, anything else waits in the
* queue, a failed one is tried again. Without `$convert` it is only kept.
*
* @return array<string,mixed>
* @throws InvalidVideoException when the bytes cannot be a video
*/
public function store(string $bytes, string $originalName, string $folder, bool $convert = true): array
{
$in = fopen('php://memory', 'r+b');
fwrite($in, $bytes);
rewind($in);
try {
return $this->storeStream($in, strlen($bytes), $originalName, $folder, $convert);
} finally {
fclose($in);
}
}
/**
* Like store(), straight from the request: the bytes go to disk as they come, and only their
* first ones and their hash are looked at, so a video of 2 GB needs no 2 GB of memory.
*
* @param resource $in
* @return array<string,mixed>
* @throws VideoTooLargeException when the stream has more than `$maxBytes`
* @throws InvalidVideoException when the bytes cannot be a video
*/
public function storeStream($in, int $maxBytes, string $originalName, string $folder, bool $convert = true): array
{
$upload = $this->queue->receive($in, $maxBytes);
try {
$head = (string) file_get_contents($upload, false, null, 0, 256);
if (!self::looksLikeVideo($head)) {
throw new InvalidVideoException('Not a video file – use ' . self::SIGNATURE_HINT);
}
if (!preg_match('/^' . UserStore::FOLDER_PATTERN . '$/', $folder)) {
throw new InvalidArgumentException("Invalid folder: {$folder}");
}
$hash = hash_file('sha256', $upload);
$dir = "{$this->root}/{$folder}";
if (!is_dir($dir) && !mkdir($dir, 0775, true) && !is_dir($dir)) {
throw new RuntimeException("Cannot create directory {$dir}");
}
OriginalName::write($dir, "{$hash}.mp4", $originalName);
if ($this->sourceOf($folder, $hash) === null) {
$source = "{$dir}/{$hash}.source." . self::sourceExtension($originalName, $head);
if (!chmod($upload, AtomicFile::SERVED) || !rename($upload, $source)) {
throw new RuntimeException('Cannot write uploaded video');
}
}
if ($convert) {
$this->enqueue($folder, $hash);
}
return $this->record("{$folder}/{$hash}.mp4");
} finally {
if (is_file($upload)) {
unlink($upload);
}
}
}
/**
* Converts an upload that was kept without: it waits in the queue like any other, a failed one
* is tried again. Null when there is no such video.
*
* @return array<string,mixed>|null
*/
public function convert(string $file): ?array
{
$folder = self::isValidName($file) ? self::folderOf($file) : '';
if ($folder === '') {
return null;
}
$this->enqueue($folder, basename($file, '.mp4'));
return $this->record($file);
}
/** Queues a kept upload, unless it is converted or on its way already. */
private function enqueue(string $folder, string $hash): void
{
$source = $this->sourceOf($folder, $hash);
if ($source === null || is_file("{$this->root}/{$folder}/{$hash}.mp4")) {
return;
}
$job = $this->queue->job($folder, $hash);
if ($job === null || $job['state'] === VideoQueue::FAILED) {
$this->queue->add($folder, $hash, $source, OriginalName::read("{$this->root}/{$folder}", "{$hash}.mp4"));
}
}
/**
* One file as the picker shows it, done or not; null when there is neither file nor job.
*
* @return array<string,mixed>|null
*/
public function record(string $file): ?array
{
if (!self::isValidName($file)) {
return null;
}
if (is_file("{$this->root}/{$file}")) {
$folder = self::folderOf($file);
$dir = $folder === '' ? $this->root : "{$this->root}/{$folder}";
return [
'file' => $file,
'name' => OriginalName::read($dir, basename($file)),
...$this->details("{$this->root}/{$file}"),
'bytes' => (int) filesize("{$this->root}/{$file}"),
];
}
$job = $this->jobOf($file);
if ($job !== null) {
return $this->pending($job);
}
$folder = self::folderOf($file);
return $folder === '' ? null : $this->unconverted($folder, basename($file, '.mp4'));
}
/** A video is there for deleting while it is converted, failed or only kept, too. */
public function exists(string $file): bool
{
return $this->record($file) !== null;
}
/**
* Lists the converted files like every store does, then the ones on their way, then the ones
* kept without being converted.
*/
public function list(string $folder, bool $seesAll, array $owners): array
{
$items = parent::list($folder, $seesAll, $owners);
foreach ($this->queue->jobs() as $job) {
if ($seesAll || $job['folder'] === $folder) {
$items[] = $this->pending($job) + [
'updatedAt' => (string) $job['createdAt'],
'owner' => $owners[$job['folder']] ?? '',
'deletable' => true,
];
}
}
foreach (glob("{$this->root}/*/*.source.*") ?: [] as $path) {
$owner = basename(dirname($path));
if (!preg_match('/^([a-f0-9]{64})\.source\.[a-z0-9]+$/', basename($path), $m) || (!$seesAll && $owner !== $folder)) {
continue;
}
$record = $this->unconverted($owner, $m[1]);
if ($record !== null) {
$items[] = $record + [
'updatedAt' => gmdate('c', (int) filemtime($path)),
'owner' => $owners[$owner] ?? '',
'deletable' => true,
];
}
}
return $items;
}
public function delete(string $file): bool
{
if (!self::isValidName($file)) {
return false;
}
$folder = self::folderOf($file);
$hash = basename($file, '.mp4');
$removed = $folder !== '' && $this->queue->remove($folder, $hash);
$source = $folder === '' ? null : $this->sourceOf($folder, $hash);
if ($source !== null) {
$removed = unlink("{$this->root}/{$source}") || $removed;
}
if (is_file("{$this->root}/{$file}")) {
$removed = parent::delete($file) || $removed;
$dir = $folder === '' ? $this->root : "{$this->root}/{$folder}";
foreach (["{$dir}/{$hash}.jpg", "{$dir}/{$hash}.meta.json"] as $path) {
if (is_file($path)) {
unlink($path);
}
}
} elseif ($removed) {
OriginalName::delete("{$this->root}/{$folder}", "{$hash}.mp4");
}
return $removed;
}
/** Called by the worker once the MP4 is in place: what the editor reads instead of probing. */
public static function writeDetails(string $dir, string $hash, array $details): void
{
$json = json_encode($details, JSON_UNESCAPED_SLASHES);
AtomicFile::write("{$dir}/{$hash}.meta.json", (string) $json, AtomicFile::SERVED, 'video details');
}
/** @return array<string,mixed> */
protected function details(string $path): array
{
$meta = substr($path, 0, -strlen('.mp4')) . '.meta.json';
$raw = is_file($meta) ? file_get_contents($meta) : false;
$details = $raw === false ? null : json_decode($raw, true);
$details = is_array($details) ? $details : [];
$file = substr($path, strlen($this->root) + 1);
$folder = self::folderOf($file);
return [
'status' => 'ready',
'width' => (int) ($details['width'] ?? 0),
'height' => (int) ($details['height'] ?? 0),
'duration' => (float) ($details['duration'] ?? 0),
'fps' => (float) ($details['fps'] ?? 0),
'audio' => (bool) ($details['audio'] ?? false),
'poster' => substr($file, 0, -strlen('.mp4')) . '.jpg',
'original' => $folder === '' ? null : $this->sourceOf($folder, basename($file, '.mp4')),
];
}
/**
* Where the upload of a video is kept, `{folder}/{hash}.source.{ext}` as it is served; null for
* one uploaded before uploads were kept.
*/
private function sourceOf(string $folder, string $hash): ?string
{
foreach (glob("{$this->root}/{$folder}/{$hash}.source.*") ?: [] as $path) {
if (preg_match('/^[a-f0-9]{64}\.source\.[a-z0-9]+$/', basename($path))) {
return "{$folder}/" . basename($path);
}
}
return null;
}
/**
* A kept upload that is neither converted nor on its way, as the picker shows it; null when
* the video is either, or there is no upload.
*
* @return array<string,mixed>|null
*/
private function unconverted(string $folder, string $hash): ?array
{
$source = $this->sourceOf($folder, $hash);
if ($source === null || is_file("{$this->root}/{$folder}/{$hash}.mp4") || $this->queue->job($folder, $hash) !== null) {
return null;
}
return [
'file' => "{$folder}/{$hash}.mp4",
'name' => OriginalName::read("{$this->root}/{$folder}", "{$hash}.mp4"),
'status' => self::UNCONVERTED,
'progress' => 0,
'error' => '',
'bytes' => (int) filesize("{$this->root}/{$source}"),
'original' => $source,
];
}
/**
* The ending a kept upload is served with: the one of its name when that is a video's, else the
* one its first bytes tell.
*/
private static function sourceExtension(string $name, string $head): string
{
$extension = strtolower(pathinfo($name, PATHINFO_EXTENSION));
if (in_array($extension, self::SOURCE_EXTENSIONS, true)) {
return $extension;
}
return match (true) {
substr($head, 4, 4) === 'ftyp' => substr($head, 8, 4) === 'qt ' ? 'mov' : 'mp4',
str_starts_with($head, "\x1A\x45\xDF\xA3") => str_contains($head, 'webm') ? 'webm' : 'mkv',
str_starts_with($head, 'RIFF') => 'avi',
default => 'ts',
};
}
/** @return array<string,mixed>|null */
private function jobOf(string $file): ?array
{
$folder = self::folderOf($file);
return $folder === '' ? null : $this->queue->job($folder, basename($file, '.mp4'));
}
/**
* @param array<string,mixed> $job
* @return array<string,mixed>
*/
private function pending(array $job): array
{
return [
'file' => "{$job['folder']}/{$job['hash']}.mp4",
'name' => (string) $job['name'],
'status' => (string) $job['state'],
'progress' => (float) $job['progress'],
'error' => (string) $job['error'],
'bytes' => 0,
'original' => $this->sourceOf((string) $job['folder'], (string) $job['hash']),
];
}
private static function looksLikeVideo(string $bytes): bool
{
if (strlen($bytes) < 16) {
return false;
}
return substr($bytes, 4, 4) === 'ftyp'
|| str_starts_with($bytes, "\x1A\x45\xDF\xA3")
|| (str_starts_with($bytes, 'RIFF') && substr($bytes, 8, 4) === 'AVI ')
|| ($bytes[0] === "\x47" && strlen($bytes) > 188 && $bytes[188] === "\x47");
}
}
/** Thrown by VideoStore::store when the uploaded bytes cannot be a video. */
final class InvalidVideoException extends RuntimeException
{
}
/** Thrown by VideoQueue::receive when an upload is bigger than the API takes. */
final class VideoTooLargeException extends RuntimeException
{
}
api/lib/VideoWorker.php
+151
View File
@@ -0,0 +1,151 @@
<?php
declare(strict_types=1);
/**
* Converts what waits in the VideoQueue, one video at a time (#402). Runs in the container
* `feuerwerk-video` through `tools/video-worker.php`, and in CI beside the PHP server.
*
* Each job goes: probe the upload, convert it into a temp file beside its target, read the result
* back for its details, cut a poster, then move all three into place and drop the job. Whatever
* fails on the way fails the job with ffmpeg's own words, and the temp files go.
*/
final class VideoWorker
{
/** How often a running conversion writes its progress, in seconds. */
private const PROGRESS_EVERY = 0.5;
public function __construct(private readonly VideoStore $store, private readonly string $videos)
{
}
/** Converts the oldest waiting video; false when there was none. */
public function runOnce(): bool
{
$job = $this->store->queue()->next();
if ($job === null) {
return false;
}
$this->convert((string) $job['folder'], (string) $job['hash']);
return true;
}
private function convert(string $folder, string $hash): void
{
$queue = $this->store->queue();
if (!$queue->update($folder, $hash, ['state' => VideoQueue::CONVERTING, 'progress' => 0, 'error' => ''])) {
return;
}
$dir = "{$this->videos}/{$folder}";
$tag = bin2hex(random_bytes(4));
$tmpVideo = "{$dir}/{$hash}.{$tag}.tmp.mp4";
$tmpPoster = "{$dir}/{$hash}.{$tag}.tmp.jpg";
try {
$plan = VideoFormat::plan(self::probe($queue->source($folder, $hash)));
$finished = $this->run(
VideoFormat::convertArgs($queue->source($folder, $hash), $tmpVideo, $plan),
$plan['duration'],
static fn(float $p): bool => $queue->update($folder, $hash, ['progress' => round($p, 3)]),
);
if (!$finished) {
return;
}
$converted = self::probe($tmpVideo);
$details = VideoFormat::details($converted);
$this->run(VideoFormat::posterArgs($tmpVideo, $tmpPoster, $details['duration'], VideoFormat::colour($converted)), 0, null);
if ($queue->job($folder, $hash) === null) {
return;
}
chmod($tmpVideo, AtomicFile::SERVED);
chmod($tmpPoster, AtomicFile::SERVED);
VideoStore::writeDetails($dir, $hash, $details);
rename($tmpPoster, "{$dir}/{$hash}.jpg");
rename($tmpVideo, "{$dir}/{$hash}.mp4");
$queue->remove($folder, $hash);
} catch (Throwable $e) {
$queue->dropSource($folder, $hash);
$queue->update($folder, $hash, ['state' => VideoQueue::FAILED, 'error' => self::shorten($e->getMessage())]);
} finally {
foreach ([$tmpVideo, $tmpPoster] as $path) {
if (is_file($path)) {
unlink($path);
}
}
}
}
/** @return array<string,mixed> */
private static function probe(string $path): array
{
[$code, $out, $err] = self::capture(VideoFormat::probeArgs($path));
$probe = json_decode($out, true);
if ($code !== 0 || !is_array($probe)) {
throw new RuntimeException('ffmpeg cannot read this file' . ($err === '' ? '' : ": {$err}"));
}
return $probe;
}
/**
* Runs ffmpeg, reporting progress from its `-progress pipe:1` lines to `$report`. Returns false
* when `$report` says the job is gone, after stopping ffmpeg.
*
* @param list<string> $args
* @param (callable(float):bool)|null $report
*/
private function run(array $args, float $duration, ?callable $report): bool
{
$err = tempnam(sys_get_temp_dir(), 'video-err-');
$process = proc_open($args, [0 => ['file', '/dev/null', 'r'], 1 => ['pipe', 'w'], 2 => ['file', $err, 'w']], $pipes);
if (!is_resource($process)) {
throw new RuntimeException('Cannot start ffmpeg');
}
$last = 0.0;
$alive = true;
while (($line = fgets($pipes[1])) !== false) {
$share = $report === null ? null : VideoFormat::progress($line, $duration);
if ($share !== null && microtime(true) - $last >= self::PROGRESS_EVERY) {
$last = microtime(true);
if (!$report($share)) {
$alive = false;
proc_terminate($process);
break;
}
}
}
fclose($pipes[1]);
$code = proc_close($process);
$message = trim((string) file_get_contents($err));
unlink($err);
if (!$alive) {
return false;
}
if ($code !== 0) {
throw new RuntimeException($message === '' ? "ffmpeg stopped with code {$code}" : $message);
}
return true;
}
/**
* @param list<string> $args
* @return array{0:int,1:string,2:string}
*/
private static function capture(array $args): array
{
$process = proc_open($args, [0 => ['file', '/dev/null', 'r'], 1 => ['pipe', 'w'], 2 => ['pipe', 'w']], $pipes);
if (!is_resource($process)) {
throw new RuntimeException('Cannot start ffprobe');
}
$out = (string) stream_get_contents($pipes[1]);
$err = trim((string) stream_get_contents($pipes[2]));
fclose($pipes[1]);
fclose($pipes[2]);
return [proc_close($process), $out, $err];
}
/** The last lines of what ffmpeg said: enough to tell why, not a log. */
private static function shorten(string $message): string
{
$lines = array_values(array_filter(array_map('trim', explode("\n", $message)), static fn(string $l): bool => $l !== ''));
$text = implode(' · ', array_slice($lines, -2));
return mb_strlen($text) > 300 ? mb_substr($text, 0, 300) . '…' : $text;
}
}
e2e.sh
+1 -1
View File
@@ -10,7 +10,7 @@
set -u
UNIT=game-maker-e2e
UNIT=feuerwerk-e2e
DIR=$(dirname "$(readlink -f "$0")")
LOGDIR=$DIR/logs
package-lock.json
Generated
+2 -2
View File
@@ -1,11 +1,11 @@
{
"name": "game-maker",
"name": "feuerwerk",
"version": "0.1.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "game-maker",
"name": "feuerwerk",
"version": "0.1.0",
"dependencies": {
"@dimforge/rapier2d-compat": "^0.20.0",
package.json
+1 -1
View File
@@ -1,5 +1,5 @@
{
"name": "game-maker",
"name": "feuerwerk",
"private": true,
"version": "0.1.0",
"type": "module",
public/brand/apple-touch-icon.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.3 KiB

public/brand/favicon.ico
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 2.9 KiB

public/brand/favicon.svg
+8
View File
@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 344 344" width="344" height="344" role="img" aria-label="Feuerwerk">
<title>Feuerwerk</title>
<rect width="344" height="344" rx="64" fill="#15171B"/>
<path fill="#3FB6A8" d="M39.2 92L39.2 39.2L92 39.2L92 48.8L48.8 48.8L48.8 92ZM252 39.2L304.8 39.2L304.8 92L295.2 92L295.2 48.8L252 48.8ZM304.8 252L304.8 304.8L252 304.8L252 295.2L295.2 295.2L295.2 252ZM92 304.8L39.2 304.8L39.2 252L48.8 252L48.8 295.2L92 295.2Z"/>
<path fill="#F5B83D" d="M162.4 42.4L181.6 42.4L181.6 140L162.4 140ZM162.4 204L181.6 204L181.6 301.6L162.4 301.6ZM42.4 162.4L140 162.4L140 181.6L42.4 181.6ZM204 162.4L301.6 162.4L301.6 181.6L204 181.6Z"/>
<path fill="#F0782F" d="M208 147.314L246.514 108.8L235.2 97.486L196.686 136ZM147.314 136L108.8 97.486L97.486 108.8L136 147.314ZM196.686 208L235.2 246.514L246.514 235.2L208 196.686ZM136 196.686L97.486 235.2L108.8 246.514L147.314 208Z"/>
<path fill="#FFF3D1" d="M172 147.11L196.89 172L172 196.89L147.11 172Z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 987 B

public/brand/feuerwerk-icon-dark.svg
+7
View File
@@ -0,0 +1,7 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 344 344" width="344" height="344" role="img" aria-label="Feuerwerk">
<title>Feuerwerk</title>
<path fill="#3FB6A8" d="M6 72L6 6L72 6L72 18L18 18L18 72ZM272 6L338 6L338 72L326 72L326 18L272 18ZM338 272L338 338L272 338L272 326L326 326L326 272ZM72 338L6 338L6 272L18 272L18 326L72 326Z"/>
<path fill="#F5B83D" d="M160 10L184 10L184 132L160 132ZM160 212L184 212L184 334L160 334ZM10 160L132 160L132 184L10 184ZM212 160L334 160L334 184L212 184Z"/>
<path fill="#F0782F" d="M217 141.142L265.142 93L251 78.858L202.858 127ZM141.142 127L93 78.858L78.858 93L127 141.142ZM202.858 217L251 265.142L265.142 251L217 202.858ZM127 202.858L78.858 251L93 265.142L141.142 217Z"/>
<path fill="#FFF3D1" d="M172 140.887L203.113 172L172 203.113L140.887 172Z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 795 B

public/brand/feuerwerk-icon-light.svg
+7
View File
@@ -0,0 +1,7 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 344 344" width="344" height="344" role="img" aria-label="Feuerwerk">
<title>Feuerwerk</title>
<path fill="#1F8F82" d="M6 72L6 6L72 6L72 18L18 18L18 72ZM272 6L338 6L338 72L326 72L326 18L272 18ZM338 272L338 338L272 338L272 326L326 326L326 272ZM72 338L6 338L6 272L18 272L18 326L72 326Z"/>
<path fill="#E08A00" d="M160 10L184 10L184 132L160 132ZM160 212L184 212L184 334L160 334ZM10 160L132 160L132 184L10 184ZM212 160L334 160L334 184L212 184Z"/>
<path fill="#D9531E" d="M217 141.142L265.142 93L251 78.858L202.858 127ZM141.142 127L93 78.858L78.858 93L127 141.142ZM202.858 217L251 265.142L265.142 251L217 202.858ZM127 202.858L78.858 251L93 265.142L141.142 217Z"/>
<path fill="#1B1E24" d="M172 140.887L203.113 172L172 203.113L140.887 172Z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 795 B

public/brand/feuerwerk-logo-horizontal-dark.svg
+8
View File
@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 460.372 88" width="460.372" height="88" role="img" aria-label="Feuerwerk">
<title>Feuerwerk</title>
<path fill="#3FB6A8" d="M1.535 18.419L1.535 1.535L18.419 1.535L18.419 4.605L4.605 4.605L4.605 18.419ZM69.581 1.535L86.465 1.535L86.465 18.419L83.395 18.419L83.395 4.605L69.581 4.605ZM86.465 69.581L86.465 86.465L69.581 86.465L69.581 83.395L83.395 83.395L83.395 69.581ZM18.419 86.465L1.535 86.465L1.535 69.581L4.605 69.581L4.605 83.395L18.419 83.395Z"/>
<path fill="#F5B83D" d="M40.93 2.558L47.07 2.558L47.07 33.767L40.93 33.767ZM40.93 54.233L47.07 54.233L47.07 85.442L40.93 85.442ZM2.558 40.93L33.767 40.93L33.767 47.07L2.558 47.07ZM54.233 40.93L85.442 40.93L85.442 47.07L54.233 47.07Z"/>
<path fill="#F0782F" d="M55.512 36.106L67.827 23.791L64.209 20.173L51.894 32.488ZM36.106 32.488L23.791 20.173L20.173 23.791L32.488 36.106ZM51.894 55.512L64.209 67.827L67.827 64.209L55.512 51.894ZM32.488 51.894L20.173 64.209L23.791 67.827L36.106 55.512Z"/>
<path fill="#FFF3D1" d="M44 36.041L51.959 44L44 51.959L36.041 44Z"/>
<path fill="#F2F0EE" d="M117.04 71V19.16H148.252V27.836H125.716V40.76H143.932V49.4H125.716V71ZM169.168 72.08Q163.264 72.08 158.8 69.524Q154.336 66.968 151.834 62.432Q149.332 57.896 149.332 51.992Q149.332 45.62 151.78 40.922Q154.228 36.224 158.584 33.632Q162.94 31.04 168.664 31.04Q174.712 31.04 178.942 33.866Q183.172 36.692 185.224 41.84Q187.276 46.988 186.772 53.972H178.168V50.804Q178.132 44.468 175.936 41.552Q173.74 38.636 169.024 38.636Q163.696 38.636 161.104 41.93Q158.512 45.224 158.512 51.56Q158.512 57.464 161.104 60.704Q163.696 63.944 168.664 63.944Q171.868 63.944 174.19 62.522Q176.512 61.1 177.772 58.4L186.34 60.992Q184.108 66.248 179.446 69.164Q174.784 72.08 169.168 72.08ZM155.776 53.972V47.42H182.524V53.972ZM207.256 72.116Q202.936 72.116 200.128 70.676Q197.32 69.236 195.664 67.004Q194.008 64.772 193.216 62.252Q192.424 59.732 192.19 57.482Q191.956 55.232 191.956 53.828V32.12H200.74V50.84Q200.74 52.64 200.992 54.89Q201.244 57.14 202.162 59.21Q203.08 61.28 204.898 62.612Q206.716 63.944 209.812 63.944Q211.468 63.944 213.088 63.404Q214.708 62.864 216.022 61.55Q217.336 60.236 218.128 57.932Q218.92 55.628 218.92 52.1L224.068 54.296Q224.068 59.264 222.142 63.296Q220.216 67.328 216.472 69.722Q212.728 72.116 207.256 72.116ZM219.964 71V58.94H218.92V32.12H227.632V71ZM253.228 72.08Q247.324 72.08 242.86 69.524Q238.396 66.968 235.894 62.432Q233.392 57.896 233.392 51.992Q233.392 45.62 235.84 40.922Q238.288 36.224 242.644 33.632Q247 31.04 252.724 31.04Q258.772 31.04 263.002 33.866Q267.232 36.692 269.284 41.84Q271.336 46.988 270.832 53.972H262.228V50.804Q262.192 44.468 259.996 41.552Q257.8 38.636 253.084 38.636Q247.756 38.636 245.164 41.93Q242.572 45.224 242.572 51.56Q242.572 57.464 245.164 60.704Q247.756 63.944 252.724 63.944Q255.928 63.944 258.25 62.522Q260.572 61.1 261.832 58.4L270.4 60.992Q268.168 66.248 263.506 69.164Q258.844 72.08 253.228 72.08ZM239.836 53.972V47.42H266.584V53.972ZM276.736 71V32.12H284.404V41.588L283.468 40.364Q284.188 38.42 285.394 36.818Q286.6 35.216 288.292 34.172Q289.732 33.2 291.478 32.642Q293.224 32.084 295.06 31.958Q296.896 31.832 298.624 32.12V40.22Q296.896 39.716 294.646 39.896Q292.396 40.076 290.596 41.156Q288.796 42.128 287.662 43.64Q286.528 45.152 285.988 47.078Q285.448 49.004 285.448 51.236V71ZM311.584 71 299.704 32.048 308.2 32.12 315.796 57.068 323.5 32.12H330.808L338.476 57.068L346.108 32.12H354.604L342.724 71H335.956L327.136 44.252L318.352 71ZM375.16 72.08Q369.256 72.08 364.792 69.524Q360.328 66.968 357.826 62.432Q355.324 57.896 355.324 51.992Q355.324 45.62 357.772 40.922Q360.22 36.224 364.576 33.632Q368.932 31.04 374.656 31.04Q380.704 31.04 384.934 33.866Q389.164 36.692 391.216 41.84Q393.268 46.988 392.764 53.972H384.16V50.804Q384.124 44.468 381.928 41.552Q379.732 38.636 375.016 38.636Q369.688 38.636 367.096 41.93Q364.504 45.224 364.504 51.56Q364.504 57.464 367.096 60.704Q369.688 63.944 374.656 63.944Q377.86 63.944 380.182 62.522Q382.504 61.1 383.764 58.4L392.332 60.992Q390.1 66.248 385.438 69.164Q380.776 72.08 375.16 72.08ZM361.768 53.972V47.42H388.516V53.972ZM398.668 71V32.12H406.336V41.588L405.4 40.364Q406.12 38.42 407.326 36.818Q408.532 35.216 410.224 34.172Q411.664 33.2 413.41 32.642Q415.156 32.084 416.992 31.958Q418.828 31.832 420.556 32.12V40.22Q418.828 39.716 416.578 39.896Q414.328 40.076 412.528 41.156Q410.728 42.128 409.594 43.64Q408.46 45.152 407.92 47.078Q407.38 49.004 407.38 51.236V71ZM425.164 71 425.236 19.16H434.02V50.84L448.24 32.12H459.076L443.992 51.56L460.372 71H448.888L434.02 52.28V71Z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.5 KiB

public/brand/feuerwerk-logo-horizontal-light.svg
+8
View File
@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 460.372 88" width="460.372" height="88" role="img" aria-label="Feuerwerk">
<title>Feuerwerk</title>
<path fill="#1F8F82" d="M1.535 18.419L1.535 1.535L18.419 1.535L18.419 4.605L4.605 4.605L4.605 18.419ZM69.581 1.535L86.465 1.535L86.465 18.419L83.395 18.419L83.395 4.605L69.581 4.605ZM86.465 69.581L86.465 86.465L69.581 86.465L69.581 83.395L83.395 83.395L83.395 69.581ZM18.419 86.465L1.535 86.465L1.535 69.581L4.605 69.581L4.605 83.395L18.419 83.395Z"/>
<path fill="#E08A00" d="M40.93 2.558L47.07 2.558L47.07 33.767L40.93 33.767ZM40.93 54.233L47.07 54.233L47.07 85.442L40.93 85.442ZM2.558 40.93L33.767 40.93L33.767 47.07L2.558 47.07ZM54.233 40.93L85.442 40.93L85.442 47.07L54.233 47.07Z"/>
<path fill="#D9531E" d="M55.512 36.106L67.827 23.791L64.209 20.173L51.894 32.488ZM36.106 32.488L23.791 20.173L20.173 23.791L32.488 36.106ZM51.894 55.512L64.209 67.827L67.827 64.209L55.512 51.894ZM32.488 51.894L20.173 64.209L23.791 67.827L36.106 55.512Z"/>
<path fill="#1B1E24" d="M44 36.041L51.959 44L44 51.959L36.041 44Z"/>
<path fill="#1B1E24" d="M117.04 71V19.16H148.252V27.836H125.716V40.76H143.932V49.4H125.716V71ZM169.168 72.08Q163.264 72.08 158.8 69.524Q154.336 66.968 151.834 62.432Q149.332 57.896 149.332 51.992Q149.332 45.62 151.78 40.922Q154.228 36.224 158.584 33.632Q162.94 31.04 168.664 31.04Q174.712 31.04 178.942 33.866Q183.172 36.692 185.224 41.84Q187.276 46.988 186.772 53.972H178.168V50.804Q178.132 44.468 175.936 41.552Q173.74 38.636 169.024 38.636Q163.696 38.636 161.104 41.93Q158.512 45.224 158.512 51.56Q158.512 57.464 161.104 60.704Q163.696 63.944 168.664 63.944Q171.868 63.944 174.19 62.522Q176.512 61.1 177.772 58.4L186.34 60.992Q184.108 66.248 179.446 69.164Q174.784 72.08 169.168 72.08ZM155.776 53.972V47.42H182.524V53.972ZM207.256 72.116Q202.936 72.116 200.128 70.676Q197.32 69.236 195.664 67.004Q194.008 64.772 193.216 62.252Q192.424 59.732 192.19 57.482Q191.956 55.232 191.956 53.828V32.12H200.74V50.84Q200.74 52.64 200.992 54.89Q201.244 57.14 202.162 59.21Q203.08 61.28 204.898 62.612Q206.716 63.944 209.812 63.944Q211.468 63.944 213.088 63.404Q214.708 62.864 216.022 61.55Q217.336 60.236 218.128 57.932Q218.92 55.628 218.92 52.1L224.068 54.296Q224.068 59.264 222.142 63.296Q220.216 67.328 216.472 69.722Q212.728 72.116 207.256 72.116ZM219.964 71V58.94H218.92V32.12H227.632V71ZM253.228 72.08Q247.324 72.08 242.86 69.524Q238.396 66.968 235.894 62.432Q233.392 57.896 233.392 51.992Q233.392 45.62 235.84 40.922Q238.288 36.224 242.644 33.632Q247 31.04 252.724 31.04Q258.772 31.04 263.002 33.866Q267.232 36.692 269.284 41.84Q271.336 46.988 270.832 53.972H262.228V50.804Q262.192 44.468 259.996 41.552Q257.8 38.636 253.084 38.636Q247.756 38.636 245.164 41.93Q242.572 45.224 242.572 51.56Q242.572 57.464 245.164 60.704Q247.756 63.944 252.724 63.944Q255.928 63.944 258.25 62.522Q260.572 61.1 261.832 58.4L270.4 60.992Q268.168 66.248 263.506 69.164Q258.844 72.08 253.228 72.08ZM239.836 53.972V47.42H266.584V53.972ZM276.736 71V32.12H284.404V41.588L283.468 40.364Q284.188 38.42 285.394 36.818Q286.6 35.216 288.292 34.172Q289.732 33.2 291.478 32.642Q293.224 32.084 295.06 31.958Q296.896 31.832 298.624 32.12V40.22Q296.896 39.716 294.646 39.896Q292.396 40.076 290.596 41.156Q288.796 42.128 287.662 43.64Q286.528 45.152 285.988 47.078Q285.448 49.004 285.448 51.236V71ZM311.584 71 299.704 32.048 308.2 32.12 315.796 57.068 323.5 32.12H330.808L338.476 57.068L346.108 32.12H354.604L342.724 71H335.956L327.136 44.252L318.352 71ZM375.16 72.08Q369.256 72.08 364.792 69.524Q360.328 66.968 357.826 62.432Q355.324 57.896 355.324 51.992Q355.324 45.62 357.772 40.922Q360.22 36.224 364.576 33.632Q368.932 31.04 374.656 31.04Q380.704 31.04 384.934 33.866Q389.164 36.692 391.216 41.84Q393.268 46.988 392.764 53.972H384.16V50.804Q384.124 44.468 381.928 41.552Q379.732 38.636 375.016 38.636Q369.688 38.636 367.096 41.93Q364.504 45.224 364.504 51.56Q364.504 57.464 367.096 60.704Q369.688 63.944 374.656 63.944Q377.86 63.944 380.182 62.522Q382.504 61.1 383.764 58.4L392.332 60.992Q390.1 66.248 385.438 69.164Q380.776 72.08 375.16 72.08ZM361.768 53.972V47.42H388.516V53.972ZM398.668 71V32.12H406.336V41.588L405.4 40.364Q406.12 38.42 407.326 36.818Q408.532 35.216 410.224 34.172Q411.664 33.2 413.41 32.642Q415.156 32.084 416.992 31.958Q418.828 31.832 420.556 32.12V40.22Q418.828 39.716 416.578 39.896Q414.328 40.076 412.528 41.156Q410.728 42.128 409.594 43.64Q408.46 45.152 407.92 47.078Q407.38 49.004 407.38 51.236V71ZM425.164 71 425.236 19.16H434.02V50.84L448.24 32.12H459.076L443.992 51.56L460.372 71H448.888L434.02 52.28V71Z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.5 KiB

public/brand/feuerwerk-logo-stacked-dark.svg
+8
View File
@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="-62.876 0 247.962 175.806" width="247.962" height="175.806" role="img" aria-label="Feuerwerk">
<title>Feuerwerk</title>
<path fill="#3FB6A8" d="M2.023 24.279L2.023 2.023L24.279 2.023L24.279 6.07L6.07 6.07L6.07 24.279ZM91.721 2.023L113.977 2.023L113.977 24.279L109.93 24.279L109.93 6.07L91.721 6.07ZM113.977 91.721L113.977 113.977L91.721 113.977L91.721 109.93L109.93 109.93L109.93 91.721ZM24.279 113.977L2.023 113.977L2.023 91.721L6.07 91.721L6.07 109.93L24.279 109.93Z"/>
<path fill="#F5B83D" d="M53.953 3.372L62.047 3.372L62.047 44.512L53.953 44.512ZM53.953 71.488L62.047 71.488L62.047 112.628L53.953 112.628ZM3.372 53.953L44.512 53.953L44.512 62.047L3.372 62.047ZM71.488 53.953L112.628 53.953L112.628 62.047L71.488 62.047Z"/>
<path fill="#F0782F" d="M73.174 47.594L89.408 31.36L84.64 26.592L68.406 42.826ZM47.594 42.826L31.36 26.592L26.592 31.36L42.826 47.594ZM68.406 73.174L84.64 89.408L89.408 84.64L73.174 68.406ZM42.826 68.406L26.592 84.64L31.36 89.408L47.594 73.174Z"/>
<path fill="#FFF3D1" d="M58 47.509L68.491 58L58 68.491L47.509 58Z"/>
<path fill="#F2F0EE" d="M-62.876 175V137.56H-40.334V143.826H-56.61V153.16H-43.454V159.4H-56.61V175ZM-25.228 175.78Q-29.492 175.78 -32.716 173.934Q-35.94 172.088 -37.747 168.812Q-39.554 165.536 -39.554 161.272Q-39.554 156.67 -37.786 153.277Q-36.018 149.884 -32.872 148.012Q-29.726 146.14 -25.592 146.14Q-21.224 146.14 -18.169 148.181Q-15.114 150.222 -13.632 153.94Q-12.15 157.658 -12.514 162.702H-18.728V160.414Q-18.754 155.838 -20.34 153.732Q-21.926 151.626 -25.332 151.626Q-29.18 151.626 -31.052 154.005Q-32.924 156.384 -32.924 160.96Q-32.924 165.224 -31.052 167.564Q-29.18 169.904 -25.592 169.904Q-23.278 169.904 -21.601 168.877Q-19.924 167.85 -19.014 165.9L-12.826 167.772Q-14.438 171.568 -17.805 173.674Q-21.172 175.78 -25.228 175.78ZM-34.9 162.702V157.97H-15.582V162.702ZM2.28 175.806Q-0.84 175.806 -2.868 174.766Q-4.896 173.726 -6.092 172.114Q-7.288 170.502 -7.86 168.682Q-8.432 166.862 -8.601 165.237Q-8.77 163.612 -8.77 162.598V146.92H-2.426V160.44Q-2.426 161.74 -2.244 163.365Q-2.062 164.99 -1.399 166.485Q-0.736 167.98 0.577 168.942Q1.89 169.904 4.126 169.904Q5.322 169.904 6.492 169.514Q7.662 169.124 8.611 168.175Q9.56 167.226 10.132 165.562Q10.704 163.898 10.704 161.35L14.422 162.936Q14.422 166.524 13.031 169.436Q11.64 172.348 8.936 174.077Q6.232 175.806 2.28 175.806ZM11.458 175V166.29H10.704V146.92H16.996V175ZM35.482 175.78Q31.218 175.78 27.994 173.934Q24.77 172.088 22.963 168.812Q21.156 165.536 21.156 161.272Q21.156 156.67 22.924 153.277Q24.692 149.884 27.838 148.012Q30.984 146.14 35.118 146.14Q39.486 146.14 42.541 148.181Q45.596 150.222 47.078 153.94Q48.56 157.658 48.196 162.702H41.982V160.414Q41.956 155.838 40.37 153.732Q38.784 151.626 35.378 151.626Q31.53 151.626 29.658 154.005Q27.786 156.384 27.786 160.96Q27.786 165.224 29.658 167.564Q31.53 169.904 35.118 169.904Q37.432 169.904 39.109 168.877Q40.786 167.85 41.696 165.9L47.884 167.772Q46.272 171.568 42.905 173.674Q39.538 175.78 35.482 175.78ZM25.81 162.702V157.97H45.128V162.702ZM52.46 175V146.92H57.998V153.758L57.322 152.874Q57.842 151.47 58.713 150.313Q59.584 149.156 60.806 148.402Q61.846 147.7 63.107 147.297Q64.368 146.894 65.694 146.803Q67.02 146.712 68.268 146.92V152.77Q67.02 152.406 65.395 152.536Q63.77 152.666 62.47 153.446Q61.17 154.148 60.351 155.24Q59.532 156.332 59.142 157.723Q58.752 159.114 58.752 160.726V175ZM77.628 175 69.048 146.868 75.184 146.92 80.67 164.938 86.234 146.92H91.512L97.05 164.938L102.562 146.92H108.698L100.118 175H95.23L88.86 155.682L82.516 175ZM123.544 175.78Q119.28 175.78 116.056 173.934Q112.832 172.088 111.025 168.812Q109.218 165.536 109.218 161.272Q109.218 156.67 110.986 153.277Q112.754 149.884 115.9 148.012Q119.046 146.14 123.18 146.14Q127.548 146.14 130.603 148.181Q133.658 150.222 135.14 153.94Q136.622 157.658 136.258 162.702H130.044V160.414Q130.018 155.838 128.432 153.732Q126.846 151.626 123.44 151.626Q119.592 151.626 117.72 154.005Q115.848 156.384 115.848 160.96Q115.848 165.224 117.72 167.564Q119.592 169.904 123.18 169.904Q125.494 169.904 127.171 168.877Q128.848 167.85 129.758 165.9L135.946 167.772Q134.334 171.568 130.967 173.674Q127.6 175.78 123.544 175.78ZM113.872 162.702V157.97H133.19V162.702ZM140.522 175V146.92H146.06V153.758L145.384 152.874Q145.904 151.47 146.775 150.313Q147.646 149.156 148.868 148.402Q149.908 147.7 151.169 147.297Q152.43 146.894 153.756 146.803Q155.082 146.712 156.33 146.92V152.77Q155.082 152.406 153.457 152.536Q151.832 152.666 150.532 153.446Q149.232 154.148 148.413 155.24Q147.594 156.332 147.204 157.723Q146.814 159.114 146.814 160.726V175ZM159.658 175 159.71 137.56H166.054V160.44L176.324 146.92H184.15L173.256 160.96L185.086 175H176.792L166.054 161.48V175Z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.6 KiB

public/brand/feuerwerk-logo-stacked-light.svg
+8
View File
@@ -0,0 +1,8 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="-62.876 0 247.962 175.806" width="247.962" height="175.806" role="img" aria-label="Feuerwerk">
<title>Feuerwerk</title>
<path fill="#1F8F82" d="M2.023 24.279L2.023 2.023L24.279 2.023L24.279 6.07L6.07 6.07L6.07 24.279ZM91.721 2.023L113.977 2.023L113.977 24.279L109.93 24.279L109.93 6.07L91.721 6.07ZM113.977 91.721L113.977 113.977L91.721 113.977L91.721 109.93L109.93 109.93L109.93 91.721ZM24.279 113.977L2.023 113.977L2.023 91.721L6.07 91.721L6.07 109.93L24.279 109.93Z"/>
<path fill="#E08A00" d="M53.953 3.372L62.047 3.372L62.047 44.512L53.953 44.512ZM53.953 71.488L62.047 71.488L62.047 112.628L53.953 112.628ZM3.372 53.953L44.512 53.953L44.512 62.047L3.372 62.047ZM71.488 53.953L112.628 53.953L112.628 62.047L71.488 62.047Z"/>
<path fill="#D9531E" d="M73.174 47.594L89.408 31.36L84.64 26.592L68.406 42.826ZM47.594 42.826L31.36 26.592L26.592 31.36L42.826 47.594ZM68.406 73.174L84.64 89.408L89.408 84.64L73.174 68.406ZM42.826 68.406L26.592 84.64L31.36 89.408L47.594 73.174Z"/>
<path fill="#1B1E24" d="M58 47.509L68.491 58L58 68.491L47.509 58Z"/>
<path fill="#1B1E24" d="M-62.876 175V137.56H-40.334V143.826H-56.61V153.16H-43.454V159.4H-56.61V175ZM-25.228 175.78Q-29.492 175.78 -32.716 173.934Q-35.94 172.088 -37.747 168.812Q-39.554 165.536 -39.554 161.272Q-39.554 156.67 -37.786 153.277Q-36.018 149.884 -32.872 148.012Q-29.726 146.14 -25.592 146.14Q-21.224 146.14 -18.169 148.181Q-15.114 150.222 -13.632 153.94Q-12.15 157.658 -12.514 162.702H-18.728V160.414Q-18.754 155.838 -20.34 153.732Q-21.926 151.626 -25.332 151.626Q-29.18 151.626 -31.052 154.005Q-32.924 156.384 -32.924 160.96Q-32.924 165.224 -31.052 167.564Q-29.18 169.904 -25.592 169.904Q-23.278 169.904 -21.601 168.877Q-19.924 167.85 -19.014 165.9L-12.826 167.772Q-14.438 171.568 -17.805 173.674Q-21.172 175.78 -25.228 175.78ZM-34.9 162.702V157.97H-15.582V162.702ZM2.28 175.806Q-0.84 175.806 -2.868 174.766Q-4.896 173.726 -6.092 172.114Q-7.288 170.502 -7.86 168.682Q-8.432 166.862 -8.601 165.237Q-8.77 163.612 -8.77 162.598V146.92H-2.426V160.44Q-2.426 161.74 -2.244 163.365Q-2.062 164.99 -1.399 166.485Q-0.736 167.98 0.577 168.942Q1.89 169.904 4.126 169.904Q5.322 169.904 6.492 169.514Q7.662 169.124 8.611 168.175Q9.56 167.226 10.132 165.562Q10.704 163.898 10.704 161.35L14.422 162.936Q14.422 166.524 13.031 169.436Q11.64 172.348 8.936 174.077Q6.232 175.806 2.28 175.806ZM11.458 175V166.29H10.704V146.92H16.996V175ZM35.482 175.78Q31.218 175.78 27.994 173.934Q24.77 172.088 22.963 168.812Q21.156 165.536 21.156 161.272Q21.156 156.67 22.924 153.277Q24.692 149.884 27.838 148.012Q30.984 146.14 35.118 146.14Q39.486 146.14 42.541 148.181Q45.596 150.222 47.078 153.94Q48.56 157.658 48.196 162.702H41.982V160.414Q41.956 155.838 40.37 153.732Q38.784 151.626 35.378 151.626Q31.53 151.626 29.658 154.005Q27.786 156.384 27.786 160.96Q27.786 165.224 29.658 167.564Q31.53 169.904 35.118 169.904Q37.432 169.904 39.109 168.877Q40.786 167.85 41.696 165.9L47.884 167.772Q46.272 171.568 42.905 173.674Q39.538 175.78 35.482 175.78ZM25.81 162.702V157.97H45.128V162.702ZM52.46 175V146.92H57.998V153.758L57.322 152.874Q57.842 151.47 58.713 150.313Q59.584 149.156 60.806 148.402Q61.846 147.7 63.107 147.297Q64.368 146.894 65.694 146.803Q67.02 146.712 68.268 146.92V152.77Q67.02 152.406 65.395 152.536Q63.77 152.666 62.47 153.446Q61.17 154.148 60.351 155.24Q59.532 156.332 59.142 157.723Q58.752 159.114 58.752 160.726V175ZM77.628 175 69.048 146.868 75.184 146.92 80.67 164.938 86.234 146.92H91.512L97.05 164.938L102.562 146.92H108.698L100.118 175H95.23L88.86 155.682L82.516 175ZM123.544 175.78Q119.28 175.78 116.056 173.934Q112.832 172.088 111.025 168.812Q109.218 165.536 109.218 161.272Q109.218 156.67 110.986 153.277Q112.754 149.884 115.9 148.012Q119.046 146.14 123.18 146.14Q127.548 146.14 130.603 148.181Q133.658 150.222 135.14 153.94Q136.622 157.658 136.258 162.702H130.044V160.414Q130.018 155.838 128.432 153.732Q126.846 151.626 123.44 151.626Q119.592 151.626 117.72 154.005Q115.848 156.384 115.848 160.96Q115.848 165.224 117.72 167.564Q119.592 169.904 123.18 169.904Q125.494 169.904 127.171 168.877Q128.848 167.85 129.758 165.9L135.946 167.772Q134.334 171.568 130.967 173.674Q127.6 175.78 123.544 175.78ZM113.872 162.702V157.97H133.19V162.702ZM140.522 175V146.92H146.06V153.758L145.384 152.874Q145.904 151.47 146.775 150.313Q147.646 149.156 148.868 148.402Q149.908 147.7 151.169 147.297Q152.43 146.894 153.756 146.803Q155.082 146.712 156.33 146.92V152.77Q155.082 152.406 153.457 152.536Q151.832 152.666 150.532 153.446Q149.232 154.148 148.413 155.24Q147.594 156.332 147.204 157.723Q146.814 159.114 146.814 160.726V175ZM159.658 175 159.71 137.56H166.054V160.44L176.324 146.92H184.15L173.256 160.96L185.086 175H176.792L166.054 161.48V175Z"/>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 4.6 KiB

src/core/commands/animationCommands.ts
+41 -2
View File
@@ -10,6 +10,7 @@ import {
targetAssetOf,
targetDefs,
trackDef,
trackHeld,
trackValueAt,
} from '../model/animation';
import type { AnimationAsset, AnimationKey, AnimationTrack, KeyInterpolation } from '../model/Asset';
@@ -215,7 +216,7 @@ export class SetTrackRelativeCommand extends TimelineCommand {
/**
* Sets the key at `frame`, replacing the value of one already there but keeping its interpolation.
* Without a value it takes the value the row has at that frame – what a double click in the timeline
* means – and on an empty row the property's default, which for a relative row is no offset at all.
* means – and on an empty row what the row's header shows: the object's value, or no offset when relative.
*/
export class SetAnimationKeyCommand extends TimelineCommand {
readonly label = 'Set key';
@@ -235,7 +236,7 @@ export class SetAnimationKeyCommand extends TimelineCommand {
const def = trackDef(project, asset, track);
if (!def) throw new CommandError('The row aims at something that is gone');
const current = trackValueAt(track.keys, def.kind, this.frame, false, curveLookup(project));
const raw = this.value ?? current ?? (track.relative ? 0 : def.default);
const raw = this.value ?? current ?? trackHeld(project, asset, track);
const key: AnimationKey = { frame: this.frame, value: coerceValue(def, raw) };
const interpolation = track.keys.find((k) => k.frame === this.frame)?.interpolation;
if (interpolation) key.interpolation = interpolation;
@@ -243,6 +244,44 @@ export class SetAnimationKeyCommand extends TimelineCommand {
}
}
/**
* Keys a number row straight from one value to another (#407): both keys set, the first one linear,
* the keys between them gone – what a driven video needs to run at 1× without a bend.
*/
export class KeyStraightRunCommand extends TimelineCommand {
readonly label = 'Key video at 1×';
/** How many keys between the two were taken out. */
removed = 0;
constructor(
assetId: string,
private readonly trackId: string,
private readonly from: AnimationKey,
private readonly to: AnimationKey,
) {
super(assetId);
}
protected apply(project: Project, asset: AnimationAsset): void {
const track = requireTrack(asset, this.trackId);
requireFrame(this.from.frame, 'A key sits on a whole frame from 0');
requireFrame(this.to.frame, 'A key sits on a whole frame from 0');
if (this.to.frame <= this.from.frame) throw new CommandError('The run has to end after it starts');
const def = trackDef(project, asset, track);
if (def?.kind !== 'number') throw new CommandError('Only a number row can run straight');
const kept = track.keys.filter((k) => k.frame < this.from.frame || k.frame > this.to.frame);
const end = track.keys.find((k) => k.frame === this.to.frame);
this.removed = track.keys.filter((k) => k.frame > this.from.frame && k.frame < this.to.frame).length;
const last: AnimationKey = { frame: this.to.frame, value: coerceValue(def, this.to.value) };
if (end?.interpolation) last.interpolation = end.interpolation;
track.keys = normalizeKeys([
...kept,
{ frame: this.from.frame, value: coerceValue(def, this.from.value) },
last,
]);
}
}
/**
* Sets how each key blends into the next one. `linear` is stored as no field at all. A key on a row
* that cannot blend is left as it is – it jumps whatever it says – and only such keys is refused.
src/core/commands/assetCommands.ts
+38 -1
View File
@@ -10,11 +10,13 @@ import type {
TextureAsset,
TilemapLayer,
VariablesAsset,
VideoAsset,
} from '../model/Asset';
import type { AssetType } from '../model/AssetTypes';
import { embeddedIn } from '../music/embedded';
import { defaultTracks } from '../music/song';
import { heldSongGraph } from '../music/routing';
import { newCompositionTrack } from '../model/composition';
import { linearCurve } from '../model/curve';
import { isFontFile } from '../model/font';
import { DEFAULT_PALETTE, PIXEL_CANVAS_DEFAULT } from '../model/pixelart';
@@ -33,6 +35,7 @@ import {
} from '../model/Scope';
import type { SceneObject } from '../model/SceneObject';
import { isTextureFile } from '../model/texture';
import { isVideoFile } from '../model/video';
import { cellsWithout } from '../model/tilemap';
import { ASSET_FX_KEY } from '../properties/assetFx';
import { coerceValue, defaultsFor, initialOf, initialPropsFor } from '../properties/PropertySchema';
@@ -108,6 +111,8 @@ export function buildAsset(type: AssetType, name: string, id = newId()): Asset {
return { ...base, type: 'font', file: '' };
case 'sound':
return { ...base, type: 'sound', file: '' };
case 'video':
return { ...base, type: 'video', file: '', width: 0, height: 0, duration: 0, fps: 0, audio: false };
case 'soundfx':
return { ...base, type: 'soundfx', graph: createEmptyGraph() };
case 'instrument':
@@ -122,6 +127,15 @@ export function buildAsset(type: AssetType, name: string, id = newId()): Asset {
return { ...base, type: 'bus' };
case 'particles':
return { ...base, type: 'particles', graph: createEmptyGraph() };
case 'composition':
return {
...base,
type: 'composition',
objects: [],
tracks: [newCompositionTrack(1)],
items: [],
clips: [],
};
}
}
@@ -217,13 +231,17 @@ function copyOf(original: Asset, id: string, name: string): Asset {
if (copy.type === 'texture' || copy.type === 'pixelart')
for (const frame of copy.frames) frame.id = newId();
renewClipIds(copy, new Map());
if (copy.type === 'scene' || copy.type === 'container') {
// A node names a marker by its id alone (#338), so a copy's markers are told apart.
if (copy.type === 'song') for (const marker of copy.markers ?? []) marker.id = newId();
if (copy.type === 'scene' || copy.type === 'container' || copy.type === 'composition') {
const ids = new Map(copy.objects.map((o) => [o.id, newId()]));
for (const object of copy.objects) {
object.id = ids.get(object.id)!;
object.parentId = ids.get(object.parentId) ?? object.parentId;
}
swapReferences(copy.objects, (ref) => renamedReference(ref, ids), new Recorder());
// An item spawns the object of its own id.
if (copy.type === 'composition') for (const item of copy.items) item.id = ids.get(item.id) ?? item.id;
}
return copy;
}
@@ -311,6 +329,25 @@ export class CreateSoundCommand extends ImportFileCommand<SoundAsset> {
}
}
/** What a video says about itself once the worker has made it (#402), copied into the asset. */
export interface VideoDetails {
width: number;
height: number;
duration: number;
fps: number;
audio: boolean;
}
export class CreateVideoCommand extends ImportFileCommand<VideoAsset> {
constructor(name: string, file: string, details: VideoDetails) {
super(
{ id: newId(), name, props: defaultsFor('video'), type: 'video', file, ...details },
'video',
isVideoFile,
);
}
}
/**
* Makes an asset a song kept to itself a project asset (#355): it shows in the lists, and a name a
* listed asset has already gets a number.
src/core/commands/clipCommands.ts
+33 -18
View File
@@ -1,29 +1,41 @@
/** The clips of a sound (#357): stretches of its file with a name, as frames are of a texture. */
/**
* The clips of a sound (#357) or a video (#402): stretches of its file with a name, as frames are
* of a texture.
*/
import { newId } from '../ids';
import type { SoundAsset, SoundClip } from '../model/Asset';
import type { SoundClip } from '../model/Asset';
import type { PropertyBag, PropertyValue } from '../model/Properties';
import type { Project } from '../model/Project';
import { clipsOf, nextClipName, uniqueClipName } from '../model/sound';
import { type ClipOwner, clipsOf, hasClips, nextClipName, uniqueClipName } from '../model/sound';
import type { PropertyDef } from '../properties/PropertySchema';
import { CLIP_ROWS } from '../properties/sounds';
import { VIDEO_CLIP_ROWS } from '../properties/videos';
import { type Removed, removeFirst, restoreAt } from '../util/list';
import { type Command, CommandError } from './Command';
import { requireAsset } from './require';
function requireSound(project: Project, assetId: string): SoundAsset {
/** A clip is the same shape on a sound and on a video; only its rows differ. */
type Clip = SoundClip;
function requireSound(project: Project, assetId: string): ClipOwner {
const asset = requireAsset(project, assetId);
if (asset.type !== 'sound') throw new CommandError(`${asset.name} is not a sound`);
if (!hasClips(asset)) throw new CommandError(`${asset.name} has no clips`);
return asset;
}
function requireClip(asset: SoundAsset, clipId: string): SoundClip {
const clip = clipsOf(asset).find((c) => c.id === clipId);
function requireClip(asset: ClipOwner, clipId: string): Clip {
const clip = clipsOf<Clip>(asset).find((c) => c.id === clipId);
if (!clip) throw new CommandError(`Clip ${clipId} not found in ${asset.name}`);
return clip;
}
/** What a clip may hold: a row of `CLIP_ROWS`, a finite number at least 0, a curve it offers. */
function heldClipValue(key: string, value: PropertyValue): PropertyValue {
const def = CLIP_ROWS.find((row) => row.key === key);
function clipRowsOf(asset: ClipOwner): readonly PropertyDef[] {
return asset.type === 'video' ? VIDEO_CLIP_ROWS : CLIP_ROWS;
}
/** What a clip may hold: a row of its asset's clip rows, a finite number at least 0, a curve it offers. */
function heldClipValue(asset: ClipOwner, key: string, value: PropertyValue): PropertyValue {
const def = clipRowsOf(asset).find((row) => row.key === key);
if (!def) throw new CommandError(`A clip has no "${key}"`);
if (def.kind === 'number' && (typeof value !== 'number' || !Number.isFinite(value) || value < 0))
throw new CommandError(`${def.label} needs a number of at least 0`);
@@ -47,14 +59,14 @@ export class AddClipCommand implements Command {
execute(project: Project): void {
const asset = requireSound(project, this.assetId);
const props: PropertyBag = {};
for (const [key, value] of Object.entries(this.props)) props[key] = heldClipValue(key, value);
for (const [key, value] of Object.entries(this.props)) props[key] = heldClipValue(asset, key, value);
// Named per execution: a redo has to fit the names that exist when it runs.
asset.clips = [...clipsOf(asset), { id: this.clipId, name: nextClipName(asset), props }];
asset.clips = [...clipsOf<Clip>(asset), { id: this.clipId, name: nextClipName(asset), props }];
}
undo(project: Project): void {
const asset = requireSound(project, this.assetId);
asset.clips = clipsOf(asset).filter((c) => c.id !== this.clipId);
asset.clips = clipsOf<Clip>(asset).filter((c) => c.id !== this.clipId);
}
}
@@ -73,8 +85,11 @@ export class SetClipPropsCommand implements Command {
) {}
execute(project: Project): void {
const clip = requireClip(requireSound(project, this.assetId), this.clipId);
const held = Object.entries(this.values).map(([key, value]) => [key, heldClipValue(key, value)] as const);
const asset = requireSound(project, this.assetId);
const clip = requireClip(asset, this.clipId);
const held = Object.entries(this.values).map(
([key, value]) => [key, heldClipValue(asset, key, value)] as const,
);
this.previous = held.map(([key]) => [key, clip.props[key]]);
for (const [key, value] of held) clip.props[key] = value;
}
@@ -117,7 +132,7 @@ export class RenameClipCommand implements Command {
*/
export class RemoveClipCommand implements Command {
readonly label = 'Delete clip';
private removed?: Removed<SoundClip>;
private removed?: Removed<Clip>;
constructor(
private readonly assetId: string,
@@ -126,7 +141,7 @@ export class RemoveClipCommand implements Command {
execute(project: Project): void {
const asset = requireSound(project, this.assetId);
const clips = [...clipsOf(asset)];
const clips = [...clipsOf<Clip>(asset)];
const removed = removeFirst(clips, (c) => c.id === this.clipId);
if (!removed) throw new CommandError(`Clip ${this.clipId} not found in ${asset.name}`);
this.removed = removed;
@@ -136,7 +151,7 @@ export class RemoveClipCommand implements Command {
undo(project: Project): void {
if (!this.removed) return;
const asset = requireSound(project, this.assetId);
const clips = [...clipsOf(asset)];
const clips = [...clipsOf<Clip>(asset)];
restoreAt(clips, [this.removed]);
asset.clips = clips;
}
src/core/commands/compositionCommands.ts
+224
View File
@@ -0,0 +1,224 @@
/**
* Editing a composition (#417). As with a song, one gesture is one command: tracks, items and clips
* are written whole once the pointer lets go. An item comes and goes together with its object.
*/
import type { CompositionAsset, CompositionClip, CompositionItem, CompositionTrack } from '../model/Asset';
import { newId } from '../ids';
import {
clipLength,
fittedLength,
heldCompositionClips,
heldCompositionItems,
heldCompositionTracks,
} from '../model/composition';
import type { ObjectClip } from '../model/clipboard';
import { carriedClips, mediaKey, pastedItemsWithClips } from '../model/compositionEdit';
import type { Project } from '../model/Project';
import type { Command } from './Command';
import { oneStep } from './CompoundCommand';
import { PasteObjectsCommand } from './pasteCommands';
import { requireComposition } from './require';
import { PlaceAssetCommand, RemoveSceneObjectsCommand } from './sceneObjectCommands';
export interface CompositionParts {
tracks?: readonly CompositionTrack[];
items?: readonly CompositionItem[];
clips?: readonly CompositionClip[];
}
/** Writes the tracks, items and clips whole, held to what each may be. */
export class SetCompositionCommand implements Command {
private previous: {
tracks: CompositionTrack[];
items: CompositionItem[];
clips: CompositionClip[];
} | null = null;
constructor(
private readonly assetId: string,
private readonly parts: CompositionParts,
readonly label = 'Edit composition',
) {}
/** Nothing to do when the composition already reads exactly like this – no empty undo step. */
static changes(project: Project, assetId: string, parts: CompositionParts): boolean {
const asset = requireComposition(project, assetId);
return JSON.stringify(held(asset, parts)) !== JSON.stringify([asset.tracks, asset.items, asset.clips]);
}
execute(project: Project): void {
const asset = requireComposition(project, this.assetId);
this.previous = { tracks: asset.tracks, items: asset.items, clips: asset.clips };
[asset.tracks, asset.items, asset.clips] = held(asset, this.parts);
}
undo(project: Project): void {
if (!this.previous) return;
const asset = requireComposition(project, this.assetId);
({ tracks: asset.tracks, items: asset.items, clips: asset.clips } = this.previous);
}
}
function held(
asset: CompositionAsset,
parts: CompositionParts,
): [CompositionTrack[], CompositionItem[], CompositionClip[]] {
const tracks = heldCompositionTracks(parts.tracks ?? asset.tracks);
const items = heldCompositionItems(parts.items ?? asset.items, tracks, asset.objects);
return [tracks, items, heldCompositionClips(parts.clips ?? asset.clips, tracks)];
}
/**
* An asset laid on a track as a new item: its object at the composition's origin and the item
* spanning `start` and `length`. `objectId` is the item's id, to pick it afterwards.
*/
export function addCompositionItem(
project: Project,
assetId: string,
placed: string,
item: Omit<CompositionItem, 'id'>,
): { command: Command; objectId: string } {
const place = new PlaceAssetCommand(assetId, placed, { x: 0, y: 0 });
const comp = requireComposition(project, assetId);
const set = new SetCompositionCommand(assetId, { items: [...comp.items, { ...item, id: place.objectId }] });
return { command: oneStep('Add item', [place, set]), objectId: place.objectId };
}
/**
* Copied items put back into a composition: their objects where the copied ones stood, the items
* and the clips that came with them as `pastedItemsWithClips` places them. Null when the copy holds
* anything but items – a scene's objects have no time to land at. `itemIds` are the new items', to
* pick them afterwards.
*/
export function pasteCompositionItems(
project: Project,
assetId: string,
clip: ObjectClip,
at: number,
row?: number,
): { command: Command; itemIds: string[] } | null {
const copied = new Map((clip.items ?? []).map((i) => [i.id, i]));
if (clip.roots.length === 0 || clip.roots.some((r) => !copied.has(r.id))) return null;
const comp = requireComposition(project, assetId);
const paste = new PasteObjectsCommand(assetId, clip, {}, { x: 0, y: 0 });
const ids = new Map(clip.roots.map((r, n) => [r.id, paste.rootIds[n]!]));
for (const c of [...(clip.clips ?? []), ...(clip.audio ?? [])]) ids.set(c.id, newId());
const { items, clips } = pastedItemsWithClips(
[...clip.roots.map((r) => copied.get(r.id)!), ...(clip.audio ?? [])],
clip.clips ?? [],
comp.clips,
ids,
at,
comp.tracks.map((t) => t.id),
(c) => Math.max(1, clipLength(project, comp, c)),
row,
);
const set = new SetCompositionCommand(assetId, {
items: [...comp.items, ...items],
clips: [...comp.clips, ...clips],
});
return {
command: oneStep(items.length === 1 ? 'Paste item' : 'Paste items', [paste, set]),
itemIds: items.map((i) => i.id),
};
}
/** Items and clips taken off together, the items' objects and the clips that belong to them with them. */
export function removeFromComposition(
project: Project,
assetId: string,
itemIds: readonly string[],
clipIds: readonly string[],
): Command {
const comp = requireComposition(project, assetId);
const carried = carriedClips(comp.items, comp.clips, new Set(itemIds));
const gone = new Set([...itemIds, ...clipIds, ...carried]);
const objects = comp.objects.filter((o) => gone.has(o.id)).map((o) => o.id);
const set = new SetCompositionCommand(assetId, {
items: comp.items.filter((i) => !gone.has(i.id)),
clips: comp.clips.filter((c) => !gone.has(c.id)),
});
const count = itemIds.length + clipIds.length;
const label = count === 1 ? (itemIds.length === 1 ? 'Remove item' : 'Remove clip') : 'Remove items';
return oneStep(label, objects.length > 0 ? [new RemoveSceneObjectsCommand(assetId, objects), set] : [set]);
}
/** A track taken off with everything on it. */
export function removeCompositionTrack(project: Project, assetId: string, trackId: string): Command {
const comp = requireComposition(project, assetId);
const items = comp.items.filter((i) => i.track === trackId).map((i) => i.id);
const clips = comp.clips.filter((c) => c.track === trackId).map((c) => c.id);
const objects = comp.objects.filter((o) => items.includes(o.id)).map((o) => o.id);
const set = new SetCompositionCommand(assetId, {
tracks: comp.tracks.filter((t) => t.id !== trackId),
items: comp.items.filter((i) => !items.includes(i.id)),
clips: comp.clips.filter((c) => !clips.includes(c.id)),
});
return oneStep(
'Delete track',
objects.length > 0 ? [new RemoveSceneObjectsCommand(assetId, objects), set] : [set],
);
}
interface Refit {
assetId: string;
itemId: string;
from: number;
to: number;
}
/**
* Any edit, with every item that follows its media fitted to it again where the edit changed what it
* wears (#417): a texture turned to a video takes the video's length, one turned back the default.
* In the same undo step as the edit, and done again with it.
*/
export class FitMediaCommand implements Command {
private refits: Refit[] | null = null;
constructor(readonly inner: Command) {}
get label(): string {
return this.inner.label;
}
execute(project: Project): void {
const before = this.refits ? null : mediaKeys(project);
this.inner.execute(project);
this.refits ??= refitsAfter(project, before!);
for (const r of this.refits) setLength(project, r, r.to);
}
undo(project: Project): void {
for (const r of [...(this.refits ?? [])].reverse()) setLength(project, r, r.from);
this.inner.undo(project);
}
}
function mediaKeys(project: Project): Map<string, string> {
const keys = new Map<string, string>();
for (const comp of project.assets) {
if (comp.type !== 'composition') continue;
for (const item of comp.items)
if (item.fitMedia) keys.set(`${comp.id}/${item.id}`, mediaKey(project, comp, item));
}
return keys;
}
function refitsAfter(project: Project, before: ReadonlyMap<string, string>): Refit[] {
const refits: Refit[] = [];
for (const comp of project.assets) {
if (comp.type !== 'composition') continue;
for (const item of comp.items) {
const was = before.get(`${comp.id}/${item.id}`);
if (!item.fitMedia || was === undefined || was === mediaKey(project, comp, item)) continue;
const to = fittedLength(project, comp, item);
if (to !== item.length) refits.push({ assetId: comp.id, itemId: item.id, from: item.length, to });
}
}
return refits;
}
function setLength(project: Project, refit: Refit, length: number): void {
const comp = requireComposition(project, refit.assetId);
comp.items = comp.items.map((i) => (i.id === refit.itemId ? { ...i, length } : i));
}
src/core/commands/compositionSprites.ts
+240
View File
@@ -0,0 +1,240 @@
/**
* What a composition keeps to itself (#419): a texture or video laid on a track wears a sprite the
* composition owns. It is an asset like any other, so everything that finds an asset by its id finds
* it, but the lists leave it out, as with the takes of a song (#355). After every edit the project is
* held to that: a composition using the sprite of another gets a copy, one placed anywhere else makes
* it a project asset, and one nothing uses any more goes.
*/
import { newId } from '../ids';
import type { Asset, CompositionItem, SpriteAsset } from '../model/Asset';
import { isCompositionSprite } from '../model/composition';
import { generatesPictures } from '../model/generated';
import { findPixelFrame, pixelArtWornId } from '../model/pixelart';
import type { PropertyBag } from '../model/Properties';
import { findAsset, objectHolders, type Project } from '../model/Project';
import { findFrame } from '../model/texture';
import { findVideo } from '../model/video';
import { spriteFromTexture } from '../properties/aspect';
import { type Removed, removeWhere, restoreAt } from '../util/list';
import type { Command } from './Command';
import { addCompositionItem } from './compositionCommands';
import { oneStep } from './CompoundCommand';
import { nextAssetName } from './naming';
import { SetOverrideCommand } from './sceneObjectCommands';
/**
* What a sprite made of `asset` wears: a texture, a drawn picture (#185) or a video; a sequence of
* pixel art frames its first frame. Null for an asset that is no picture.
*/
export function wornPicture(asset: Asset): string | null {
if (asset.type === 'pixelart') return pixelArtWornId(asset);
return asset.type === 'texture' || asset.type === 'video' || generatesPictures(asset.type)
? asset.id
: null;
}
/** What an item dropped on a track wears: the picture its sprite is made of, and a part (#425, #430). */
export interface DroppedPicture {
picture: string;
/**
* The clip of a video the item plays or the frame it shows, as an override on its object; null for
* what its sprite wears.
*/
part: string | null;
/** The name a new sprite for the picture gets. */
name: string;
}
/**
* What an asset, a clip of a video (#425) or a frame of a picture (#430) dragged onto a track wears. A
* part shares the sprite of its asset, so one composition keeps one sprite per video or picture. Null
* for anything that is no picture.
*/
export function droppedPicture(project: Project, id: string): DroppedPicture | null {
const asset = findAsset(project, id);
if (asset) {
const picture = wornPicture(asset);
return picture === null ? null : { picture, part: null, name: asset.name };
}
const video = findVideo(project, id);
if (video?.clip) return { picture: video.asset.id, part: video.clip.id, name: video.asset.name };
const frame = findFrame(project, id);
if (frame) return { picture: frame.asset.id, part: id, name: frame.asset.name };
const pixel = findPixelFrame(project, id);
if (!pixel) return null;
// Its first frame is what the sprite of a sequence wears already.
const picture = pixelArtWornId(pixel.asset);
return { picture, part: id === picture ? null : id, name: pixel.asset.name };
}
/** The sprite the composition already keeps for this picture: one serves every item wearing it. */
function keptSprite(project: Project, compId: string, textureId: string): SpriteAsset | undefined {
return project.assets.find(
(a): a is SpriteAsset => isCompositionSprite(a) && a.owner === compId && a.props['texture'] === textureId,
);
}
/** A new sprite of the composition's own, wearing the picture at its own size. */
function newCompositionSprite(
project: Project,
compId: string,
textureId: string,
name: string,
): SpriteAsset {
const props: PropertyBag = spriteFromTexture(project, textureId);
return { id: newId(), type: 'sprite', name, props, owner: compId };
}
/**
* A texture, a drawn picture, a video or a part of one laid on a track: an item wearing the sprite the
* composition keeps for that picture, made first when it has none yet, in one undo step. A part is
* the texture of the item's object.
*/
export function addPictureItem(
project: Project,
compId: string,
dropped: DroppedPicture,
item: Omit<CompositionItem, 'id'>,
): { command: Command; objectId: string } {
const kept = keptSprite(project, compId, dropped.picture);
const sprite = kept ?? newCompositionSprite(project, compId, dropped.picture, dropped.name);
const { command, objectId } = addCompositionItem(project, compId, sprite.id, item);
const steps = [
...(kept ? [] : [new AddSpriteCommand(sprite)]),
command,
...(dropped.part ? [new SetOverrideCommand(compId, objectId, 'texture', dropped.part)] : []),
];
return { command: steps.length === 1 ? command : oneStep('Add item', steps), objectId };
}
class AddSpriteCommand implements Command {
readonly label = 'Add sprite';
constructor(private readonly sprite: SpriteAsset) {}
execute(project: Project): void {
project.assets.push(structuredClone(this.sprite));
}
undo(project: Project): void {
project.assets = project.assets.filter((a) => a.id !== this.sprite.id);
}
}
interface Retarget {
holderId: string;
objectId: string;
from: string;
to: string;
}
interface Owned {
assetId: string;
from: { owner: string; name: string };
to: { owner: string; name: string };
}
export interface SpriteChange {
/** Copies for compositions using a sprite another one keeps. */
added: Asset[];
retargets: Retarget[];
/** Sprites made project assets or handed to the composition now using them. */
owners: Owned[];
dropped: Set<string>;
}
/**
* Holds every composition's sprites to what uses them, in place. A sprite placed in a scene or a
* container becomes a project asset. Otherwise it stays with its composition while that uses it, else
* goes to the first composition that does; every other composition using it gets a copy of its own.
*/
function settleCompositionSprites(project: Project): SpriteChange {
const change: SpriteChange = { added: [], retargets: [], owners: [], dropped: new Set() };
const sprites = project.assets.filter(isCompositionSprite);
if (sprites.length === 0) return change;
const holders = objectHolders(project);
for (const sprite of sprites) {
const using = holders.filter((h) => h.objects.some((o) => o.assetId === sprite.id));
const from = { owner: sprite.owner!, name: sprite.name };
if (using.some((h) => h.type !== 'composition')) {
const to = { owner: '', name: nextAssetName(project, sprite.name, sprite.id) };
setOwner(sprite, to);
change.owners.push({ assetId: sprite.id, from, to });
continue;
}
if (using.length === 0) {
change.dropped.add(sprite.id);
continue;
}
const keeper = using.find((h) => h.id === sprite.owner) ?? using[0]!;
if (keeper.id !== sprite.owner) {
const to = { owner: keeper.id, name: sprite.name };
setOwner(sprite, to);
change.owners.push({ assetId: sprite.id, from, to });
}
for (const holder of using) {
if (holder === keeper) continue;
const copy: SpriteAsset = { ...structuredClone(sprite), id: newId(), owner: holder.id };
change.added.push(copy);
for (const object of holder.objects) {
if (object.assetId !== sprite.id) continue;
change.retargets.push({ holderId: holder.id, objectId: object.id, from: sprite.id, to: copy.id });
object.assetId = copy.id;
}
}
}
project.assets.push(...change.added);
return change;
}
function setOwner(asset: Asset, to: { owner: string; name: string }): void {
if (to.owner) asset.owner = to.owner;
else delete asset.owner;
asset.name = to.name;
}
function retarget(project: Project, retargets: readonly Retarget[], to: (r: Retarget) => string): void {
for (const r of retargets) {
const holder = objectHolders(project).find((h) => h.id === r.holderId);
const object = holder?.objects.find((o) => o.id === r.objectId);
if (object) object.assetId = to(r);
}
}
/**
* Any edit, with the compositions' sprites settled after it (#419). Worked out once, so a redo brings
* back the very same copies; taken back along with the edit.
*/
export class KeepCompositionSpritesCommand implements Command {
private change: SpriteChange | null = null;
private removed: Removed<Asset>[] = [];
constructor(readonly inner: Command) {}
get label(): string {
return this.inner.label;
}
execute(project: Project): void {
this.inner.execute(project);
if (!this.change) this.change = settleCompositionSprites(project);
else {
for (const o of this.change.owners) setOwner(findAsset(project, o.assetId)!, o.to);
project.assets.push(...this.change.added);
retarget(project, this.change.retargets, (r) => r.to);
}
const dropped = this.change.dropped;
this.removed = removeWhere(project.assets, (a) => dropped.has(a.id));
}
undo(project: Project): void {
if (this.change) {
restoreAt(project.assets, this.removed);
retarget(project, this.change.retargets, (r) => r.from);
const added = new Set(this.change.added.map((a) => a.id));
project.assets = project.assets.filter((a) => !added.has(a.id));
for (const o of [...this.change.owners].reverse()) setOwner(findAsset(project, o.assetId)!, o.from);
}
this.inner.undo(project);
}
}
src/core/commands/graphCommands.ts
+27
View File
@@ -332,6 +332,33 @@ function setCollapsed(node: GraphNode, collapsed: boolean): void {
else delete node.collapsed;
}
/**
* The size a node's face was dragged to (#385). The previous size is read at execution time, so a
* caller that previewed the drag must restore the node before executing.
*/
export class ResizeNodeCommand implements Command {
readonly label = 'Resize node';
private previous: GraphNode['size'];
constructor(
private readonly assetId: string,
private readonly nodeId: string,
private readonly size: { w: number; h: number },
) {}
execute(project: Project): void {
const node = requireNode(project, this.assetId, this.nodeId);
this.previous = node.size && { ...node.size };
node.size = { w: Math.round(this.size.w), h: Math.round(this.size.h) };
}
undo(project: Project): void {
const node = requireNode(project, this.assetId, this.nodeId);
if (this.previous) node.size = { ...this.previous };
else delete node.size;
}
}
/** Other nodes and edges in place of a graph's, as one step: an instrument's Preset (#329). Groups and notes stay. */
export class ReplaceGraphCommand implements Command {
private previous?: Pick<Graph, 'nodes' | 'edges'>;
src/core/commands/pasteCommands.ts
+9 -1
View File
@@ -2,6 +2,7 @@ import { annotationList, type Graph } from '../graph/Graph';
import { isTerminalType, terminalNode } from '../graph/nested';
import { nodeDef } from '../graph/NodeRegistry';
import { newId } from '../ids';
import type { Asset } from '../model/Asset';
import type { GraphClip, ObjectClip } from '../model/clipboard';
import { findGroup } from '../model/hierarchy';
import { isLiveInstance } from '../model/instance';
@@ -28,6 +29,7 @@ function freshIds(ids: readonly string[]): Map<string, string> {
export class PasteObjectsCommand implements Command {
readonly label: string;
private readonly ids: Map<string, string>;
private restored: Asset[] = [];
constructor(
private readonly holderId: string,
@@ -46,8 +48,10 @@ export class PasteObjectsCommand implements Command {
execute(project: Project): void {
const holder = requireHolder(project, this.holderId);
// A cut took the sprites a composition kept for it along (#419); who keeps them is settled after.
const restored = (this.clip.sprites ?? []).filter((s) => !findAsset(project, s.id));
for (const object of this.clip.objects) {
const asset = findAsset(project, object.assetId);
const asset = findAsset(project, object.assetId) ?? restored.find((s) => s.id === object.assetId);
if (!asset) throw new CommandError('A copied object belongs to an asset that no longer exists');
const rejection = placementRejection(project, holder, asset);
if (rejection) throw new CommandError(rejection);
@@ -55,6 +59,8 @@ export class PasteObjectsCommand implements Command {
for (const parentId of new Set(Object.values(this.parents)))
if (parentId && isLiveInstance(project, holder, requireStructuralObject(project, holder, parentId)))
throw new CommandError('A placed container shows its template; unpack it to put objects inside');
this.restored = structuredClone(restored);
project.assets.push(...this.restored);
const roots = new Map(this.clip.roots.map((root) => [root.id, root]));
const pasted = this.clip.objects.map((object): SceneObject => {
const copy = structuredClone(object);
@@ -87,6 +93,8 @@ export class PasteObjectsCommand implements Command {
const holder = requireHolder(project, this.holderId);
const ids = new Set(this.ids.values());
holder.objects = holder.objects.filter((o) => !ids.has(o.id));
const restored = new Set(this.restored.map((s) => s.id));
project.assets = project.assets.filter((a) => !restored.has(a.id));
}
}
src/core/commands/require.ts
+7
View File
@@ -3,6 +3,7 @@ import { findNode, type Graph, type GraphEdge, type GraphNode } from '../graph/G
import { type NodeDef, nodeDef } from '../graph/NodeRegistry';
import type {
Asset,
CompositionAsset,
CurveAsset,
PatternAsset,
SongAsset,
@@ -155,6 +156,12 @@ export function requireSong(project: Project, assetId: string): SongAsset {
return asset;
}
export function requireComposition(project: Project, assetId: string): CompositionAsset {
const asset = findAssetOf(project, assetId, 'composition');
if (!asset) throw new CommandError(`Composition ${assetId} not found`);
return asset;
}
export function requireVariablesAsset(project: Project, assetId: string): VariablesAsset {
const asset = findAssetOf(project, assetId, 'variables');
if (!asset) throw new CommandError(`Variables ${assetId} not found`);
src/core/commands/songCommands.ts
+31
View File
@@ -8,6 +8,7 @@ import type { EdgeSend, Graph } from '../graph/Graph';
import type { Asset, SongAsset } from '../model/Asset';
import { findGraphAsset, type Project } from '../model/Project';
import { settleEmbedded } from '../music/embedded';
import { heldSongMarkers, type SongMarker } from '../music/markers';
import { routedKind, sendsTapped } from '../music/projectRouting';
import { heldSend, heldSongGraph } from '../music/routing';
import { heldSongItems, heldSongTracks, type SongItem, type SongTrack } from '../music/song';
@@ -75,6 +76,36 @@ export class SetSongCommand implements Command {
}
}
/** Writes the markers of a song whole (#338): one gesture, one step to undo. */
export class SetSongMarkersCommand implements Command {
private previous: SongMarker[] | undefined;
constructor(
private readonly assetId: string,
private readonly markers: readonly SongMarker[],
readonly label = 'Edit markers',
) {}
/** Nothing to do when the song already has exactly these markers – no empty undo step. */
static changes(asset: SongAsset, markers: readonly SongMarker[]): boolean {
return JSON.stringify(asset.markers ?? []) !== JSON.stringify(heldSongMarkers(markers));
}
execute(project: Project): void {
const asset = requireSong(project, this.assetId);
this.previous = asset.markers;
const held = heldSongMarkers(this.markers);
if (held.length > 0) asset.markers = held;
else delete asset.markers;
}
undo(project: Project): void {
const asset = requireSong(project, this.assetId);
if (this.previous) asset.markers = this.previous;
else delete asset.markers;
}
}
/**
* How loud a link of a song's routing sends, and where it taps its track (#350); the same for a link
* of the master routing (#351), where only a bus and a song may tap anywhere but after their level.
src/core/commands/structTypeCommands.ts
+10
View File
@@ -227,6 +227,11 @@ export class MakeStructTypeCommand implements Command {
private readonly path: readonly string[],
) {}
/** The Struct asset it made, once it has run. */
get madeId(): string | undefined {
return this.asset?.id;
}
execute(project: Project): void {
const owner = requireVariableOwner(project, this.ownerId);
const def = requireStructVariable(project, this.ownerId, this.key);
@@ -372,6 +377,11 @@ export class MakeListItemTypeCommand implements Command {
constructor(private readonly target: ListTarget) {}
/** The Struct asset it made, once it has run. */
get madeId(): string | undefined {
return this.asset?.id;
}
execute(project: Project): void {
const { def, decl } = requireListDecl(project, this.target);
if (decl.itemType && structAssetOf(project, decl.itemType))
src/core/commands/tilemapCommands.ts
+2 -2
View File
@@ -72,7 +72,7 @@ export class PaintTilesCommand implements Command {
if (edit.tileId) layer.cells[key] = edit.tileId;
else delete layer.cells[key];
}
cellsChanged(layer.cells);
cellsChanged(layer.cells, this.previous.keys());
}
undo(project: Project): void {
@@ -81,7 +81,7 @@ export class PaintTilesCommand implements Command {
if (value) layer.cells[key] = value;
else delete layer.cells[key];
}
cellsChanged(layer.cells);
cellsChanged(layer.cells, this.previous.keys());
}
}
src/core/commands/variableCommands.ts
+42
View File
@@ -65,6 +65,48 @@ export class AddVariableCommand implements Command {
}
}
/**
* Declares variables of one behavior in another as they are declared there (#434): default, Keep
* and Private included, for copied nodes that name them. A key the target has already is left alone.
*/
export class CarryVariablesCommand implements Command {
readonly label = 'Carry variables';
private added: string[] = [];
constructor(
private readonly assetId: string,
private readonly fromId: string,
private readonly keys: readonly string[],
) {}
execute(project: Project): void {
const from = requireBehavior(project, this.fromId);
const to = requireBehavior(project, this.assetId);
this.added = [];
for (const key of this.keys) {
const def = from.variables.find((v) => v.key === key);
if (!def) throw new CommandError(`${from.name} has no variable "${key}"`);
if (to.variables.some((v) => v.key === key)) continue;
to.variables.push(structuredClone(def));
to.props[key] = structuredClone(from.props[key] ?? def.default);
if (isKeptVariable(from, key)) setKeptVariable(to, key, true);
if (isPrivateVariable(from, key)) setPrivateVariable(to, key, true);
this.added.push(key);
}
}
undo(project: Project): void {
const to = requireBehavior(project, this.assetId);
const added = new Set(this.added);
to.variables = to.variables.filter((v) => !added.has(v.key));
for (const key of added) {
delete to.props[key];
setKeptVariable(to, key, false);
setPrivateVariable(to, key, false);
}
}
}
/** One placed object's own value for a variable, remembered so that undo can put it back. */
interface TakenOverride {
holderId: string;
src/core/graph/Graph.ts
+2
View File
@@ -52,6 +52,8 @@ export interface GraphNode {
splits?: PortSplit[];
/** Drawn as its title bar alone, every link meeting at one point per side. Purely visual. */
collapsed?: boolean;
/** The size its face was dragged to in the graph (#385), for the nodes whose face may be. Purely visual. */
size?: { w: number; h: number };
}
/**
src/core/graph/NodeContext.ts
+76 -7
View File
@@ -1,7 +1,11 @@
import type { AnimationPlayer, EaseShare } from '../model/animation';
import type { CompositionPlayer } from '../model/composition';
import type { Mat } from '../model/matrix';
import type { Position, PropertyBag } from '../model/Properties';
import type { FxParams } from '../soundfx/nodes';
import type { HeardPosition, Quantize } from '../music/beats';
import type { RecordingSettings } from '../recording/recording';
import type { VideoReading, VideoSteer } from '../model/videoClock';
import { type ChainTransform, composeAlongChain } from '../model/walk';
import { enabledAlongChain } from '../properties/enabled';
import { localMatrixOf, positionOf, transformOf } from '../properties/transform';
@@ -17,6 +21,15 @@ export const ENTITY_HANDLE: unique symbol = Symbol('entity handle');
* What a graph sees of a running entity; the runtime's Entity satisfies this shape. Mutating
* `x`/`y`/`props` is how actions take effect.
*/
/**
* A sound, note or pattern started in time with a playing song or pattern (#338), a stinger: on the
* spot `quantize` names of the playing numbered `playback`; 0 for none, which starts it at once.
*/
export interface MusicSync {
playback: number;
quantize: Quantize;
}
export interface EntityHandle {
readonly [ENTITY_HANDLE]: true;
id: string;
@@ -35,6 +48,8 @@ export interface EntityHandle {
emits?: number;
/** What `Play animation` started on this object (see `model/animation.ts`). */
animations?: AnimationPlayer[];
/** What `Play composition` started on a placed composition (see `model/composition.ts`). */
composition?: CompositionPlayer;
}
/** Position, rotation, scale and mirror of every ancestor applied, nearest last. */
@@ -228,6 +243,13 @@ export interface NodeContext {
/** Absent where nothing is simulated: the editor run, and a test double that does not ask for it. */
physics?: PhysicsControl;
log(message: string): void;
/**
* Plays, pauses or seeks the video the sprite `target` wears (#410). A free one gets a player of its
* own from then on; a driven one, and the editor run, are left alone and say so in the log.
*/
steerVideo(target: EntityHandle, steer: VideoSteer): void;
/** Where the video the sprite `target` wears stands (#410); null for none. */
videoReading(target: EntityHandle): VideoReading | null;
/**
* Starts a sound asset, named by its id. `volume` multiplies the asset's own (1 leaves it alone);
* `loop` is the caller's answer to whether it repeats, and `undefined` leaves that to the asset,
@@ -242,7 +264,8 @@ export interface NodeContext {
* names (#352), `NO_FX` none.
* `fxParams` are values for its parameters; one not given is the default the FX declares.
* `tempo` is what a Delay synced to beats counts by (#332). `bus` is the bus it is heard on
* (#325): a bus asset or `MASTER_BUS`, `''` for the one the sound asset names.
* (#325): a bus asset or `MASTER_BUS`, `''` for the one the sound asset names. `sync` starts it in
* time with a playing (#338).
* Returns the number of the voice for `tuneSound`, or 0 where nothing is played.
*/
playSound(
@@ -254,6 +277,7 @@ export interface NodeContext {
fxParams: FxParams,
tempo: number,
bus: string,
sync: MusicSync,
): number;
/**
* Plays one note of an instrument asset (#331): `note` as MIDI counts it (60 is C4), struck with
@@ -262,7 +286,7 @@ export interface NodeContext {
* values for the instrument's parameters, `tempo` as for `playSound`. `fx` sends the note through a
* sound FX of its own (#346), `fxParams` with values for its parameters, apart from the instrument's;
* `''` the one the instrument names, `NO_FX` none (#352).
* `bus` as for `playSound`, `''` for the one the instrument names.
* `bus` as for `playSound`, `''` for the one the instrument names, and `sync` too.
* Returns the number of the voice, 0 where nothing plays.
*/
playNote(
@@ -277,13 +301,14 @@ export interface NodeContext {
fx: string,
fxParams: FxParams,
bus: string,
sync: MusicSync,
): number;
/**
* Plays a pattern asset through its instrument, at its own tempo, once or round and round, every
* note at `volume`. Owned like a `playSound` voice. `fx` is a sound FX all of it goes through as
* one chain (#346), `''` the one the pattern names, `NO_FX` none (#352); `fxParams` values for its
* parameters, `bus` as for `playSound`. Returns its
* number, for `stopVoice` and `tuneSound`; 0 where there is nothing to play.
* parameters, `bus` as for `playSound`. Synced to a playing (#338), it starts on its spot and plays
* at its tempo. Returns its number, for `stopVoice` and `tuneSound`; 0 where there is nothing to play.
*/
playPattern(
patternId: string,
@@ -293,6 +318,7 @@ export interface NodeContext {
fx: string,
fxParams: FxParams,
bus: string,
sync: MusicSync,
): number;
/** Stops every playing of that pattern, `''` every pattern; `letRing` lets its notes fall away in their release. */
stopPattern(patternId: string, letRing: boolean): void;
@@ -300,8 +326,9 @@ export interface NodeContext {
* Plays a song asset (#336) at its tempo, its tracks as they are set: from its start, then round
* the loop of its ruler with `loop`, else once to its end. `volume` (0 to 1) is the level of the
* whole of it, rising there from silence over `fadeIn` seconds. Owned like a `playSound` voice.
* `fx`, `fxParams` and `bus` as for `playPattern`, after its tracks. Returns its number; 0 where
* there is nothing to play.
* `fx`, `fxParams` and `bus` as for `playPattern`, after its tracks. With `live` it plays its notes
* even where it has a bake, so its tracks can be turned (#338). Returns its number; 0 where there is
* nothing to play.
*/
playSong(
songId: string,
@@ -312,7 +339,33 @@ export interface NodeContext {
fx: string,
fxParams: FxParams,
bus: string,
live: boolean,
): number;
/**
* Turns one track of a playing song (#338) to `volumeDb` on top of its own level, over `fade`
* seconds from where `quantize` lands; a playing of its bake has no tracks, which is logged.
*/
setSongTrack(playback: number, track: string, volumeDb: number, fade: number, quantize: Quantize): void;
/**
* Holds the lane `lane` of one track of a playing song (#338) on `value`, in the lane's units,
* reached over `fade` seconds from where `quantize` lands; its points are not heard until it is
* let go. A track without such a lane, or a playing of its bake, is logged.
*/
setSongLane(
playback: number,
track: string,
lane: string,
value: number,
fade: number,
quantize: Quantize,
): void;
/** Lets such a held lane go (#338): back to its curve over `fade` seconds, as soon as can be heard. */
releaseSongLane(playback: number, track: string, lane: string, fade: number): void;
/**
* Plays `songId` in place of a playing song (#338), `''` the same one, from the marker `marker`
* (`''` its start), where `quantize` lands: over `fade` seconds, else in a cut. It keeps its number.
*/
switchSong(playback: number, songId: string, marker: string, quantize: Quantize, fade: number): void;
/** Stops every playing of that song, `''` every song, falling silent over `fadeOut` seconds. */
stopSong(songId: string, fadeOut: number): void;
/** Stops one playing of a song or pattern by its number, falling silent over `fadeOut` seconds. */
@@ -350,6 +403,16 @@ export interface NodeContext {
setBusVolume(bus: string, volume: number, time: number, ease: EaseShare): void;
/** The volume a bus is at, `''` the master: what a game set, else that of its asset; 0 for none. */
busVolume(bus: string): number;
/**
* Where a playing song or pattern stands as it is heard (#338): the one numbered `playback`, else
* the one started last of `song` or `pattern`, of every one where both are `''`; null for none.
*/
songPosition(playback: number | null, song: string, pattern: string): HeardPosition | null;
/**
* Whether the voice or playing numbered `id` is still to be heard (#373): waiting on its files,
* playing, fading or ringing on; false for 0.
*/
playing(id: number): boolean;
/**
* Asks for another scene, by asset id. The swap happens once the frame is over – a graph must be
* able to finish the flow it is in, and half a frame in the old world and half in the new one is
@@ -358,7 +421,7 @@ export interface NodeContext {
loadScene(sceneId: string): void;
/**
* The position is local to `parent` (world when none); null when the asset does not exist or
* cannot be placed. A spawned behavior runs its `On start` on the next tick.
* cannot be placed. A spawned behavior runs its `On start` at the end of this tick, before the frame is drawn.
*/
spawn(
assetId: string,
@@ -391,4 +454,10 @@ export interface NodeContext {
isPaused(): boolean;
/** From the next frame on, only behaviors that run while paused are ticked. */
setPaused(paused: boolean): void;
/** Starts recording the game view (#405); where it cannot, the log says why. */
startRecording(settings: RecordingSettings): void;
/** Ends the recording and saves it; without one, only the log hears of it. */
stopRecording(): void;
/** Pauses the editor's Play after this frame, as its pause button does; elsewhere nothing. */
breakpoint(): void;
}
src/core/graph/NodeRegistry.ts
-17
View File
@@ -111,21 +111,6 @@ export interface DerivedShape {
inOrder?: true;
}
/**
* A small picture a node draws below its rows (#375), as wide as the node and `height` tall. Its
* shapes are flat point lists `[x0, y0, x1, y1, …]` in that box, y down: a `guide` is a faint line, an
* `area` is filled, a `line` is drawn strong.
*/
export interface NodePicture {
height: number;
shapes(params: PropertyBag, w: number): PictureShape[];
}
export interface PictureShape {
kind: 'guide' | 'area' | 'line';
points: number[];
}
/** Palette, port rendering, connection validation and inspector fields are all derived from this. */
export interface NodeDef {
type: string;
@@ -150,8 +135,6 @@ export interface NodeDef {
idleParams?(params: PropertyBag, fed?: (key: string) => boolean): string[];
/** Whether the node does nothing at this value set and is drawn faded, like a bypassed effect (#354). */
dimmed?(params: PropertyBag): boolean;
/** What the node shows of its settings below its rows, drawn rather than written (#375). */
picture?: NodePicture;
/**
* Which side the ports this node declares for itself (`GraphNode.ports`) appear on. Setting it
* is what gives a node the port editor in the inspector. `both`: every port is an output, and an
src/core/graph/nodes/action/PlayNote.ts
+6 -1
View File
@@ -3,6 +3,7 @@ import { heldTempo } from '../../../music/time';
import type { NodeDef } from '../../NodeRegistry';
import { asBoolean, asNumber, asString } from '../../values';
import { detachedPort, exec, fxParamsPort, num, playedBusPort, soundFxPort, withDefault } from '../shared';
import { quantizePort, SYNC, syncOf, syncPort } from './musicSync';
import { fxParamsOf, tempoPort } from './PlaySound';
export const PlayNote: NodeDef = {
@@ -10,7 +11,8 @@ export const PlayNote: NodeDef = {
label: 'Play note',
category: 'action',
description:
'Plays one note of an instrument asset. "note" is its number as MIDI counts it: 60 is C4, 61 C#4, 72 C5. "velocity" is how hard it is struck, 0 to 1 – it makes the note quieter and is handed to what the instrument does with it. "duration" is how many seconds until the note is released; at 0 it is held until a Stop sound ends it. A note belongs to this object like a sound and stops when the object is destroyed, unless "detached" is on. "params" takes values for the parameters of the instrument from an Instrument parameters node. "fx" sends the note through a sound FX on top of what the instrument does; left empty, the one the instrument names, and None through none. "fx params" takes values for its parameters from a Sound FX parameters node – the two sets stay apart. "voice" is the number of this note, for Set sound FX parameters and Stop sound; 0 where the instrument plays nothing for it. "tempo" is what a Delay in the instrument synced to beats counts by, in quarter notes a minute. "bus" is the bus it is heard on; left empty, the one the instrument names.',
'Plays one note of an instrument asset. "note" is its number as MIDI counts it: 60 is C4, 61 C#4, 72 C5. "velocity" is how hard it is struck, 0 to 1 – it makes the note quieter and is handed to what the instrument does with it. "duration" is how many seconds until the note is released; at 0 it is held until a Stop sound ends it. A note belongs to this object like a sound and stops when the object is destroyed, unless "detached" is on. "params" takes values for the parameters of the instrument from an Instrument parameters node. "fx" sends the note through a sound FX on top of what the instrument does; left empty, the one the instrument names, and None through none. "fx params" takes values for its parameters from a Sound FX parameters node – the two sets stay apart. "voice" is the number of this note, for Set sound FX parameters and Stop sound; 0 where the instrument plays nothing for it. "tempo" is what a Delay in the instrument synced to beats counts by, in quarter notes a minute. "bus" is the bus it is heard on; left empty, the one the instrument names. ' +
SYNC,
inputs: [
exec(),
{
@@ -41,6 +43,8 @@ export const PlayNote: NodeDef = {
{ ...soundFxPort(), key: 'soundFx' },
fxParamsPort('soundFxParams'),
playedBusPort(),
syncPort(),
quantizePort(),
],
outputs: [exec(), num('voice')],
params: [],
@@ -59,6 +63,7 @@ export const PlayNote: NodeDef = {
asString(inputs.soundFx),
fxParamsOf(inputs.soundFxParams),
asString(inputs.bus),
syncOf(inputs),
)
: 0;
return { next: 'exec', outputs: { voice } };
src/core/graph/nodes/action/PlayPattern.ts
+6 -1
View File
@@ -10,6 +10,7 @@ import {
soundFxPort,
withDefault,
} from '../shared';
import { quantizePort, SYNC, syncOf, syncPort } from './musicSync';
import { fxParamsOf } from './PlaySound';
export const patternPort = (): PortDef => ({
@@ -34,7 +35,8 @@ export const PlayPattern: NodeDef = {
'none. "fx params" takes values for its parameters from a Sound ' +
'FX parameters node. "playback" is the number of this playing, for Stop pattern and for Set sound ' +
'FX parameters; 0 where the pattern has no instrument or no notes. "bus" is the bus all of it is ' +
'heard on; left empty, the one the pattern names.',
`heard on; left empty, the one the pattern names. ${SYNC} Synced, it plays at the tempo of that ` +
'playing.',
inputs: [
exec(),
patternPort(),
@@ -44,6 +46,8 @@ export const PlayPattern: NodeDef = {
soundFxPort(),
fxParamsPort(),
playedBusPort(),
syncPort(),
quantizePort(),
],
outputs: [exec(), num('playback')],
params: [],
@@ -58,6 +62,7 @@ export const PlayPattern: NodeDef = {
asString(inputs.fx),
fxParamsOf(inputs.fxParams),
asString(inputs.bus),
syncOf(inputs),
)
: 0;
return { next: 'exec', outputs: { playback } };
src/core/graph/nodes/action/PlaySong.ts
+6 -2
View File
@@ -33,8 +33,10 @@ export const PlaySong: NodeDef = {
'is on. "fx" sends the whole song through one sound FX, after its tracks and before "volume"; ' +
'left empty, the one the song names, and None is none. ' +
'"fx params" takes values for its parameters from a Sound FX parameters node. "playback" is the ' +
'number of this playing, for Stop song and for Set sound FX parameters; 0 where the song has ' +
'nothing to play. "bus" is the bus all of it is heard on; left empty, the one the song names.',
'number of this playing, for Stop song, Set song track, Switch song and Set sound FX parameters; ' +
'0 where the song has nothing to play. "bus" is the bus all of it is heard on; left empty, the ' +
'one the song names. A song with a bake plays the bake, one sound for all its tracks; with "live ' +
'tracks" it plays its notes instead, as the Song tab does, so Set song track can turn its tracks.',
inputs: [
exec(),
songPort(),
@@ -45,6 +47,7 @@ export const PlaySong: NodeDef = {
soundFxPort(),
fxParamsPort(),
playedBusPort(),
checkPort('live', 'live tracks', 'Live tracks', false),
],
outputs: [exec(), num('playback')],
params: [],
@@ -60,6 +63,7 @@ export const PlaySong: NodeDef = {
asString(inputs.fx),
fxParamsOf(inputs.fxParams),
asString(inputs.bus),
asBoolean(inputs.live),
)
: 0;
return { next: 'exec', outputs: { playback } };
src/core/graph/nodes/action/PlaySound.ts
+5 -1
View File
@@ -13,6 +13,7 @@ import {
soundPort,
withDefault,
} from '../shared';
import { quantizePort, SYNC, syncOf, syncPort } from './musicSync';
/**
* Three answers, not two: a checkbox cannot say "leave it to the asset", and without that a sound
@@ -64,7 +65,7 @@ export const PlaySound: NodeDef = {
'the sound has not been loaded yet. A pitch never changes it. "tail" is that length and the ' +
'echoes of its FX after it, as they are when it starts – how long until the voice is quiet. ' +
'"tempo" is what a Delay of its FX synced to beats counts by, in quarter notes a minute. "bus" is ' +
'the bus it is heard on; left empty, the one the sound names.',
`the bus it is heard on; left empty, the one the sound names. ${SYNC}`,
inputs: [
exec(),
soundPort(),
@@ -75,6 +76,8 @@ export const PlaySound: NodeDef = {
fxParamsPort(),
tempoPort(),
playedBusPort(),
syncPort(),
quantizePort(),
],
outputs: [exec(), num('voice'), num('length'), num('tail')],
params: [],
@@ -90,6 +93,7 @@ export const PlaySound: NodeDef = {
fxParamsOf(inputs.fxParams),
heldTempo(asNumber(inputs.tempo)),
asString(inputs.bus),
syncOf(inputs),
)
: 0;
const length = voice ? ctx.soundLength(sound) : 0;
src/core/graph/nodes/action/compositionNodes.ts
+40
View File
@@ -0,0 +1,40 @@
import { playComposition, stopComposition } from '../../../model/composition';
import type { NodeDef } from '../../NodeRegistry';
import { asBoolean, asEntity } from '../../values';
import { bool, exec, obj, withDefault } from '../shared';
export const PlayComposition: NodeDef = {
type: 'PlayComposition',
label: 'Play composition',
category: 'action',
description:
'Plays a composition placed in the scene from its first frame: each item becomes an object below ' +
'it when the item begins – its behaviors start then – and is destroyed when it ends. One already ' +
'playing plays on; Restart starts it over. Without Loop it ends after its last item, and On ' +
'composition finished hears that. The editor run plays none and says so in the log.',
inputs: [exec(), obj('object'), withDefault(bool('restart'), false)],
outputs: [exec()],
params: [],
run: (ctx, inputs) => {
const target = asEntity(inputs.object);
if (target) playComposition(target, asBoolean(inputs.restart), ctx.self);
return { next: 'exec' };
},
};
export const StopComposition: NodeDef = {
type: 'StopComposition',
label: 'Stop composition',
category: 'action',
description:
'Stops a playing composition: the objects of its items are destroyed on the next frame. It does ' +
'not count as finished, and Play composition starts it from its first frame again.',
inputs: [exec(), obj('object')],
outputs: [exec()],
params: [],
run: (_ctx, inputs) => {
const target = asEntity(inputs.object);
if (target) stopComposition(target);
return { next: 'exec' };
},
};
src/core/graph/nodes/action/gameTimeNodes.ts
+17
View File
@@ -65,6 +65,23 @@ export const ResumeGame: NodeDef = {
},
};
export const Break: NodeDef = {
type: 'Break',
label: 'Break',
category: 'action',
description:
"Pauses the editor's Play once this frame is done, as the pause button in the top bar does, to " +
'look at the game the moment something happens. Next frame steps on from there, Ctrl+Enter ' +
'resumes. A released game and the editor run go past it.',
inputs: [exec()],
outputs: [exec()],
params: [],
run: (ctx) => {
ctx.breakpoint();
return { next: 'exec' };
},
};
export const IsGamePaused: NodeDef = {
type: 'IsGamePaused',
label: 'Is game paused',
src/core/graph/nodes/action/musicSync.ts
+28
View File
@@ -0,0 +1,28 @@
import { heldQuantize, QUANTIZE_OPTIONS } from '../../../music/beats';
import type { MusicSync } from '../../NodeContext';
import type { PortDef } from '../../NodeRegistry';
import { asNumber, type Value } from '../../values';
import { num } from '../shared';
/** Where a change to the music lands (#338): at once, or on the next beat, bar or marker of the playing. */
export const quantizePort = (): PortDef => ({
key: 'quantize',
label: 'quantize',
type: 'string',
default: 'bar',
field: { key: 'quantize', label: 'Quantize', kind: 'enum', default: 'bar', options: [...QUANTIZE_OPTIONS] },
});
/** The playing a sound, note or pattern starts in time with (#338), a stinger: not typed in either. */
export const syncPort = (): PortDef => ({ ...num('sync'), inline: false });
/** What `syncPort` and `quantizePort` ask for. */
export function syncOf(inputs: Record<string, Value>): MusicSync {
return { playback: asNumber(inputs.sync), quantize: heldQuantize(inputs.quantize) };
}
/** How the nodes that play a sound, note or pattern tell of `sync` (#338). */
export const SYNC =
'"sync" takes the playback number of a playing song or pattern to start in time with, as a ' +
'stinger: on the next beat, bar or marker of it as it is heard, as "quantize" says, planned on ' +
'the audio clock to the sample. Unconnected, or where that playing is not heard, it starts at once.';
src/core/graph/nodes/action/recordingNodes.ts
+84
View File
@@ -0,0 +1,84 @@
import { RECORDING_FPS, RECORDING_RESOLUTIONS, recordingSettingsOf } from '../../../recording/recording';
import type { NodeDef, PortDef } from '../../NodeRegistry';
import { asBoolean, asString } from '../../values';
import { checkPort, exec } from '../shared';
const choicePort = (
key: string,
label: string,
options: readonly { value: string; label: string }[],
): PortDef => ({
key,
label: key,
type: 'string',
default: options[0]!.value,
field: { key, label, kind: 'enum', default: options[0]!.value, options: [...options] },
});
export const StartRecording: NodeDef = {
type: 'StartRecording',
label: 'Start recording',
category: 'action',
description:
'Records the game view as a video, with what the master plays: only the picture of the camera, ' +
'none of what lies over it. "name" is what the recording is saved as; left empty, it is named ' +
'after the moment it starts. "resolution" Game view records at the size the picture has on ' +
'screen when it starts, 720p, 1080p, 1440p and 4K at that height with the width of the camera, ' +
'both at most 3840 × 2160: a wider picture is scaled down to fit. "sound" off records the ' +
'picture alone. "real time" off moves the game on by one frame of the recording at a time, ' +
'however long the computer takes for it, so not a frame is missed even at 4K and 60 fps: the ' +
'game view runs slower meanwhile, and the sound is made apart, in step with the picture. Such a ' +
'recording holds at most ten minutes. The recording goes on through Load scene and Restart ' +
'game, holds while the editor pauses the run, and ends with Stop recording or with the run. ' +
'In the editor it is ' +
'always uploaded: "convert" has the server make an MP4 of it, "import" makes that a video asset ' +
'of the project once it is ready; without them the video picker has it. A released game ' +
'downloads it as an MKV file and takes neither. A second start while one records is ignored and ' +
'says so in the log.',
inputs: [
exec(),
{
key: 'name',
label: 'name',
type: 'string',
field: { key: 'name', label: 'Name', kind: 'text', default: '' },
},
choicePort('resolution', 'Resolution', RECORDING_RESOLUTIONS),
choicePort('fps', 'FPS', RECORDING_FPS),
checkPort('sound', 'sound', 'Sound', true),
checkPort('convert', 'convert', 'Convert', true),
checkPort('adopt', 'import', 'Import after conversion', true),
checkPort('realTime', 'real time', 'Real time', true),
],
outputs: [exec()],
params: [],
run: (ctx, inputs) => {
// Graphs from before these flags have no value for them: on, as they recorded then.
const flag = (key: string) => (inputs[key] === undefined ? true : asBoolean(inputs[key]));
ctx.startRecording(
recordingSettingsOf(asString(inputs.name), asString(inputs.resolution), asString(inputs.fps), {
sound: flag('sound'),
convert: flag('convert'),
adopt: flag('adopt'),
realTime: flag('realTime'),
}),
);
return { next: 'exec' };
},
};
export const StopRecording: NodeDef = {
type: 'StopRecording',
label: 'Stop recording',
category: 'action',
description:
'Ends the recording Start recording began and saves it. Without one running it does nothing but ' +
'say so in the log.',
inputs: [exec()],
outputs: [exec()],
params: [],
run: (ctx) => {
ctx.stopRecording();
return { next: 'exec' };
},
};
src/core/graph/nodes/action/songSync.ts
+173
View File
@@ -0,0 +1,173 @@
import { heldQuantize } from '../../../music/beats';
import { TRACK_MAX_DB, TRACK_MIN_DB } from '../../../music/level';
import type { NodeDef, PortDef } from '../../NodeRegistry';
import { asNumber, asString } from '../../values';
import { exec, num, withDefault } from '../shared';
import { quantizePort } from './musicSync';
import { songPort } from './PlaySong';
/** The playing a node changes: not typed in, as it is a number Play song hands out. */
const playbackPort = (): PortDef => ({ ...num('playback'), inline: false });
const trackPort = (): PortDef => ({
key: 'track',
label: 'track',
type: 'string',
field: { key: 'track', label: 'Track', kind: 'asset', default: '', assetTypes: ['song'], tracks: true },
});
const markerPort = (): PortDef => ({
key: 'marker',
label: 'marker',
type: 'string',
field: { key: 'marker', label: 'Marker', kind: 'asset', default: '', assetTypes: ['song'], markers: true },
});
const QUANTIZE =
'"quantize" says where it lands: Now, or on the next beat, bar or marker of the song as it is ' +
'heard, planned on the audio clock to the sample; Next marker in a song without markers is the ' +
'next bar.';
export const SetSongTrack: NodeDef = {
type: 'SetSongTrack',
label: 'Set song track',
category: 'action',
description:
'Turns one track of a playing song up or down, by the "playback" number Play song handed out: ' +
'"volume" in decibels on top of the level the song gives it, 0 as the song has it, -60 silent – ' +
'what goes into the track, its sound FX and its sends alike. It gets there over "fade" seconds. ' +
`${QUANTIZE} The track is picked as "Song / Track"; a track the playing song does not have is ` +
'left alone. A song played from its bake has no tracks to turn: switch on "live tracks" in Play ' +
'song, else the log says so. A Switch song to the same song keeps what was turned.',
inputs: [
exec(),
playbackPort(),
trackPort(),
withDefault(
{
...num('volume', 'volume (dB)'),
field: {
key: 'volume',
label: 'Volume (dB)',
kind: 'number',
default: 0,
min: TRACK_MIN_DB,
max: TRACK_MAX_DB,
},
},
0,
),
withDefault(num('fade'), 1),
quantizePort(),
],
outputs: [exec()],
params: [],
run: (ctx, inputs) => {
ctx.setSongTrack(
asNumber(inputs.playback),
asString(inputs.track),
asNumber(inputs.volume),
asNumber(inputs.fade),
heldQuantize(inputs.quantize),
);
return { next: 'exec' };
},
};
const lanePort = (): PortDef => ({
key: 'lane',
label: 'lane',
type: 'string',
default: 'volume',
field: { key: 'lane', label: 'Lane', kind: 'text', default: 'volume' },
});
const LANE =
'The track is picked as "Song / Track"; "lane" names one of its automation lanes: "volume", ' +
'"pan" or a parameter by its name. A track the playing song does not have is left alone; one ' +
'without that lane is logged, and so is a song played from its bake ("live tracks" in Play song).';
export const SetSongLane: NodeDef = {
type: 'SetSongLane',
label: 'Set song lane',
category: 'action',
description:
'Takes an automation lane of one track of a playing song over, by the "playback" number Play ' +
'song handed out: from where the lane is, it goes to "value" over "fade" seconds and stays there, ' +
'the points of the lane no longer heard, until Release song lane lets it go. "value" is in the ' +
'units of the lane – decibels for the volume, -1 (left) to 1 (right) for the pan, a parameter in ' +
`the range of its lane – and held to that range. ${QUANTIZE} ${LANE} A Switch song to the same ` +
'song keeps what is held.',
inputs: [
exec(),
playbackPort(),
trackPort(),
lanePort(),
withDefault(num('value'), 0),
withDefault(num('fade'), 1),
quantizePort(),
],
outputs: [exec()],
params: [],
run: (ctx, inputs) => {
ctx.setSongLane(
asNumber(inputs.playback),
asString(inputs.track),
asString(inputs.lane),
asNumber(inputs.value),
asNumber(inputs.fade),
heldQuantize(inputs.quantize),
);
return { next: 'exec' };
},
};
export const ReleaseSongLane: NodeDef = {
type: 'ReleaseSongLane',
label: 'Release song lane',
category: 'action',
description:
'Lets an automation lane go that Set song lane took over, by the "playback" number Play song ' +
'handed out: as soon as it can be heard, it goes back over "fade" seconds to where the curve of ' +
'the lane is by then, and follows it from there. A lane that is not held stays as it is. ' +
LANE,
inputs: [exec(), playbackPort(), trackPort(), lanePort(), withDefault(num('fade'), 1)],
outputs: [exec()],
params: [],
run: (ctx, inputs) => {
ctx.releaseSongLane(
asNumber(inputs.playback),
asString(inputs.track),
asString(inputs.lane),
asNumber(inputs.fade),
);
return { next: 'exec' };
},
};
export const SwitchSong: NodeDef = {
type: 'SwitchSong',
label: 'Switch song',
category: 'action',
description:
'Plays another song, or the same one from a marker, in place of a playing song, by the ' +
'"playback" number Play song handed out; the playing keeps its number, so Stop song and Set ' +
'song track go on reaching it. "song" left empty is the same song, "marker" left empty its ' +
`start. ${QUANTIZE} With "fade" 0 it is a cut on that spot; else the old song falls silent over ` +
'that many seconds as the new one rises from it. The new song plays with the volume, loop, ' +
'bus, sound FX and "live tracks" of its Play song. A marker the song does not have is logged, ' +
'and nothing changes.',
inputs: [exec(), playbackPort(), songPort(), markerPort(), quantizePort(), withDefault(num('fade'), 0)],
outputs: [exec()],
params: [],
run: (ctx, inputs) => {
ctx.switchSong(
asNumber(inputs.playback),
asString(inputs.song),
asString(inputs.marker),
heldQuantize(inputs.quantize),
asNumber(inputs.fade),
);
return { next: 'exec' };
},
};
src/core/graph/nodes/action/videoNodes.ts
+60
View File
@@ -0,0 +1,60 @@
import type { VideoSteer } from '../../../model/videoClock';
import type { EntityHandle, NodeContext } from '../../NodeContext';
import type { NodeDef } from '../../NodeRegistry';
import { asEntity, asNumber, type Value } from '../../values';
import { exec, num, obj } from '../shared';
const STEERS =
'A free video gets a player of its own from then on, so other sprites wearing the same clip play ' +
'on. A driven one and the editor run are left alone and say so in the log. Unconnected, object is the owner.';
/** The sprite a video node works on: the connected object, or the owner. */
export function videoTarget(ctx: NodeContext, value: Value | undefined): EntityHandle | null {
return asEntity(value) ?? ctx.owner;
}
function steering(steer: (inputs: Record<string, Value>) => VideoSteer): NodeDef['run'] {
return (ctx, inputs) => {
const target = videoTarget(ctx, inputs.object);
if (target) ctx.steerVideo(target, steer(inputs));
return { next: 'exec' };
};
}
export const PlayVideo: NodeDef = {
type: 'PlayVideo',
label: 'Play video',
category: 'action',
description:
'Plays on the video a sprite wears from where it stands, heard as before. At the end of a clip ' +
`without Loop it starts over. ${STEERS}`,
inputs: [exec(), obj('object')],
outputs: [exec()],
params: [],
run: steering(() => ({ kind: 'play' })),
};
export const PauseVideo: NodeDef = {
type: 'PauseVideo',
label: 'Pause video',
category: 'action',
description: `Holds the video a sprite wears at the picture it shows, without sound. ${STEERS}`,
inputs: [exec(), obj('object')],
outputs: [exec()],
params: [],
run: steering(() => ({ kind: 'pause' })),
};
export const SeekVideo: NodeDef = {
type: 'SeekVideo',
label: 'Seek video',
category: 'action',
description:
'Sends the video a sprite wears to time, in seconds from the start of its clip; playing or held, ' +
'it stays so. Past the end it goes round with Loop and holds the last picture without, as Video ' +
`time does. ${STEERS}`,
inputs: [exec(), obj('object'), num('time', 'time (s)')],
outputs: [exec()],
params: [],
run: steering((inputs) => ({ kind: 'seek', time: asNumber(inputs.time) })),
};
src/core/graph/nodes/data/MusicPlaying.ts
+33
View File
@@ -0,0 +1,33 @@
import type { NodeDef } from '../../NodeRegistry';
import { asNumber } from '../../values';
import { PLAYBACK_INPUT, playbackPort } from '../events/musicEvents';
import { bool, num } from '../shared';
export const MusicPlaying: NodeDef = {
type: 'MusicPlaying',
label: 'Music playing',
category: 'data',
description:
'Whether the song or pattern numbered "playback" is still to be heard: from Play song or Play ' +
'pattern on, while its files load, while it plays and fades, until its last echoes are gone – ' +
'the moment On music finished runs. Unlike "playing" of Song position it is on before the first ' +
'note is heard. Off for 0 and for a number nothing plays under.',
inputs: [playbackPort()],
outputs: [bool('playing')],
params: [],
read: (ctx, _params, input) => ({ playing: ctx.playing(asNumber(input(PLAYBACK_INPUT))) }),
};
export const SoundPlaying: NodeDef = {
type: 'SoundPlaying',
label: 'Sound playing',
category: 'data',
description:
'Whether the voice numbered "voice", the number Play sound handed out, is still to be heard: ' +
'while its file loads, while it plays, until the echoes of its FX are gone – the moment On sound ' +
'finished runs. Off for 0 and for a voice that has ended.',
inputs: [{ ...num('voice'), inline: false }],
outputs: [bool('playing')],
params: [],
read: (ctx, _params, input) => ({ playing: ctx.playing(asNumber(input('voice'))) }),
};
src/core/graph/nodes/data/SongPosition.ts
+34
View File
@@ -0,0 +1,34 @@
import type { NodeDef } from '../../NodeRegistry';
import { asNumber, asString } from '../../values';
import { patternPort } from '../action/PlayPattern';
import { songPort } from '../action/PlaySong';
import { PLAYBACK_INPUT, playbackPort } from '../events/musicEvents';
import { bool, num } from '../shared';
export const SongPosition: NodeDef = {
type: 'SongPosition',
label: 'Song position',
category: 'data',
description:
'Where a playing song or pattern stands, as it is heard: the output latency of the device taken ' +
'off, a baked song too, so what is drawn by it keeps to the sound. "bar" and "beat" count from 1 ' +
'as the ruler does, "fraction" is how far into the beat (0 to below 1), "seconds" how far from ' +
'the start of the song, back with the loop. Connect "playback" for that playing; else it is the ' +
'one started last of the Song or the Pattern picked (both picked, either), of every song and ' +
'pattern where none is. "playing" is off before its first note is heard and after its end; with ' +
'no such playing everything is 0.',
inputs: [songPort(), patternPort(), playbackPort()],
outputs: [num('bar'), num('beat'), num('fraction'), num('seconds'), bool('playing')],
params: [],
read: (ctx, _params, input) => {
const playback = input(PLAYBACK_INPUT);
const at = ctx.songPosition(
playback === null ? null : asNumber(playback),
asString(input('song')),
asString(input('pattern')),
);
return at
? { bar: at.bar, beat: at.beat, fraction: at.fraction, seconds: at.seconds, playing: at.playing }
: { bar: 0, beat: 0, fraction: 0, seconds: 0, playing: false };
},
};
src/core/graph/nodes/data/VideoTime.ts
+22
View File
@@ -0,0 +1,22 @@
import type { NodeDef } from '../../NodeRegistry';
import { videoTarget } from '../action/videoNodes';
import { bool, num, obj } from '../shared';
export const VideoTime: NodeDef = {
type: 'VideoTime',
label: 'Video time',
category: 'data',
description:
'Where the video a sprite wears stands: "time" in seconds from the start of its clip, "length" ' +
'how long the clip lasts, "playing" whether it moves. A free video counts from when the sprite ' +
'appeared, while it is switched on; a driven one stands where its Video time says. All 0 for a ' +
'sprite without a video. Unconnected, object is the owner.',
inputs: [obj('object')],
outputs: [num('time', 'time (s)'), num('length', 'length (s)'), bool('playing')],
params: [],
read: (ctx, _params, input) => {
const target = videoTarget(ctx, input('object'));
const reading = target ? ctx.videoReading(target) : null;
return reading ? { ...reading } : { time: 0, length: 0, playing: false };
},
};
src/core/graph/nodes/events/OnCompositionFinished.ts
+24
View File
@@ -0,0 +1,24 @@
import { textOf } from '../../../model/Properties';
import type { NodeDef } from '../../NodeRegistry';
import { hiddenOrder, orderParam } from '../../eventOrder';
import { exec, obj } from '../shared';
export const OnCompositionFinished: NodeDef = {
type: 'OnCompositionFinished',
label: 'On composition finished',
category: 'event',
description:
'Runs once when a composition placed at the owner, or below it, plays past its last item – and when ' +
'one this behavior played itself does, wherever it stands. A looping one never finishes, and Stop ' +
'composition does not count. Leave Composition at None to hear every composition.',
inputs: [],
outputs: [exec(), obj('object')],
params: [
{ key: 'composition', label: 'Composition', kind: 'asset', default: '', assetTypes: ['composition'] },
orderParam,
],
hiddenParams: (params) => [
...(textOf(params.composition ?? '') ? [] : ['composition']),
...hiddenOrder(params),
],
};
src/core/graph/nodes/events/OnSoundFinished.ts
+25
View File
@@ -0,0 +1,25 @@
import { textOf } from '../../../model/Properties';
import type { NodeDef } from '../../NodeRegistry';
import { hiddenOrder, orderParam } from '../../eventOrder';
import { bool, exec, num } from '../shared';
/** The input that picks one play (#373): not typed in, so unconnected it is null and the Sound decides. */
export const VOICE_INPUT = 'voice';
export const OnSoundFinished: NodeDef = {
type: 'OnSoundFinished',
label: 'On sound finished',
category: 'event',
description:
'Runs once when a sound a Play sound of this game started is over: once nothing of it is heard ' +
'any more, the echoes of its FX included. "stopped" is on where it did not play to its end but ' +
'was stopped – by Stop sound, by the end of the behavior that owns it, or to make room for ' +
'newer voices – and where it never started because its file would not load. A looping sound ' +
'ends only so. Pick a Sound to hear only that one, or connect "voice" to hear only that play, ' +
'the number Play sound handed out. A note of an instrument is none of them. It comes in the ' +
'first frame after the end, so up to a frame late.',
inputs: [{ ...num(VOICE_INPUT), inline: false }],
outputs: [exec(), num('voice'), bool('stopped')],
params: [{ key: 'sound', label: 'Sound', kind: 'sound', default: '' }, orderParam],
hiddenParams: (params) => [...(textOf(params.sound ?? '') ? [] : ['sound']), ...hiddenOrder(params)],
};
src/core/graph/nodes/events/OnVideoFinished.ts
+19
View File
@@ -0,0 +1,19 @@
import { textOf } from '../../../model/Properties';
import type { NodeDef } from '../../NodeRegistry';
import { hiddenOrder, orderParam } from '../../eventOrder';
import { exec, obj } from '../shared';
export const OnVideoFinished: NodeDef = {
type: 'OnVideoFinished',
label: 'On video finished',
category: 'event',
description:
'Runs once when a video without Loop reaches its last picture on the owner, or on a sprite below ' +
'it – and when one this behavior played, paused or sought does, on whatever sprite. A free video ' +
'counts from when its sprite appeared, while it is switched on; a driven one ends when its Video ' +
'time passes the end. Leave Video at None to hear every video; a video hears its clips as well.',
inputs: [],
outputs: [exec(), obj('object')],
params: [{ key: 'video', label: 'Video', kind: 'asset', default: '', assetTypes: ['video'] }, orderParam],
hiddenParams: (params) => [...(textOf(params.video ?? '') ? [] : ['video']), ...hiddenOrder(params)],
};
src/core/graph/nodes/events/musicEvents.ts
+112
View File
@@ -0,0 +1,112 @@
import { textOf } from '../../../model/Properties';
import type { PropertyDef } from '../../../properties/PropertySchema';
import type { NodeDef, PortDef } from '../../NodeRegistry';
import { hiddenOrder, orderParam } from '../../eventOrder';
import { bool, exec, num, str } from '../shared';
/** The input that picks one playing (#338): not typed in, so unconnected it is null and hears every one. */
export const PLAYBACK_INPUT = 'playback';
export const playbackPort = (): PortDef => ({ ...num(PLAYBACK_INPUT), inline: false });
const songParam: PropertyDef = {
key: 'song',
label: 'Song',
kind: 'asset',
default: '',
assetTypes: ['song'],
};
const patternParam: PropertyDef = {
key: 'pattern',
label: 'Pattern',
kind: 'asset',
default: '',
assetTypes: ['pattern'],
};
const markerParam: PropertyDef = {
key: 'marker',
label: 'Marker',
kind: 'asset',
default: '',
assetTypes: ['song'],
markers: true,
};
/** A pick left at None is not shown on the node: it hears every one. */
const hiddenEmpty =
(...keys: string[]) =>
(params: Parameters<NonNullable<NodeDef['hiddenParams']>>[0]) => [
...keys.filter((key) => textOf(params[key] ?? '') === ''),
...hiddenOrder(params),
];
const HEARS =
'It hears every song a Play song plays and every pattern a Play pattern plays, wherever it was ' +
'played; pick a Song or a Pattern to hear only that one (both picked, either), or connect ' +
'"playback" to hear only that playing. A composition plays none of them. It goes by what is heard, ' +
'the output latency of the device taken off, a baked song too. A frame comes now and then, so ' +
'"late" says how many seconds ago it was heard; one more than half a beat (at most 0.15 s) ago is ' +
'not told, and of several passed in one frame only the last.';
export const OnBeat: NodeDef = {
type: 'OnBeat',
label: 'On beat',
category: 'event',
description:
'Runs on every beat of a playing song or pattern, the first beat of a bar too. A beat is a note ' +
'of the value under the line of the time signature: a quarter in 4/4, an eighth in 6/8. "bar" ' +
'and "beat" count from 1, as the ruler does, and start over with the loop. ' +
HEARS,
inputs: [playbackPort()],
outputs: [exec(), num('playback'), num('bar'), num('beat'), num('late')],
params: [songParam, patternParam, orderParam],
hiddenParams: hiddenEmpty('song', 'pattern'),
};
export const OnBar: NodeDef = {
type: 'OnBar',
label: 'On bar',
category: 'event',
description:
'Runs at the start of every bar of a playing song or pattern, "bar" counting from 1 as the ruler ' +
'does; with the loop it starts over at the bar the loop starts in. ' +
HEARS,
inputs: [playbackPort()],
outputs: [exec(), num('playback'), num('bar'), num('late')],
params: [songParam, patternParam, orderParam],
hiddenParams: hiddenEmpty('song', 'pattern'),
};
export const OnMusicFinished: NodeDef = {
type: 'OnMusicFinished',
label: 'On music finished',
category: 'event',
description:
'Runs once when a song or pattern a Play song or Play pattern of this game started is over: once ' +
'nothing of it is heard any more, its last echoes and a fade out included. "stopped" is on where ' +
'it did not play to its end but was stopped – by Stop song, Stop pattern or Stop sound, or by ' +
'the end of the behavior that owns it. One with a loop ends only so. After a Switch song it ' +
'counts as the song heard last. Pick a Song or a Pattern to hear only that one (both picked, ' +
'either), or connect "playback" to hear only that playing. It comes in the first frame after ' +
'the end, so up to a frame late.',
inputs: [playbackPort()],
outputs: [exec(), num('playback'), bool('stopped')],
params: [songParam, patternParam, orderParam],
hiddenParams: hiddenEmpty('song', 'pattern'),
};
export const OnSongMarker: NodeDef = {
type: 'OnSongMarker',
label: 'On song marker',
category: 'event',
description:
'Runs when a playing song passes a marker of its ruler, every time round the loop. Pick a Marker ' +
'to hear only that one, or a Song to hear only its markers; markers on one tick each run it. ' +
'"marker" is the id a marker is known by, "name" what it is called. Connect "playback" to hear ' +
'only that playing. It goes by what is heard, a baked song too. "late" says how many seconds ago ' +
'it was heard; one more than half a beat (at most 0.15 s) ago is not told, and of several passed ' +
'in one frame only the last.',
inputs: [playbackPort()],
outputs: [exec(), num('playback'), str('marker'), str('name'), num('late')],
params: [songParam, markerParam, orderParam],
hiddenParams: hiddenEmpty('song', 'marker'),
};
src/core/graph/nodes/index.ts
+35 -1
View File
@@ -2,8 +2,9 @@ import { type NodeDef, registerNode } from '../NodeRegistry';
import { AddJoint } from './action/AddJoint';
import { Destroy } from './action/Destroy';
import { PauseAnimation, PlayAnimation, StopAnimation } from './action/animationNodes';
import { PlayComposition, StopComposition } from './action/compositionNodes';
import { Emit } from './action/Emit';
import { IsGamePaused, PauseGame, ResumeGame, SetTimeScale, TimeScale } from './action/gameTimeNodes';
import { Break, IsGamePaused, PauseGame, ResumeGame, SetTimeScale, TimeScale } from './action/gameTimeNodes';
import { LoadScene } from './action/LoadScene';
import { Log } from './action/Log';
import { IgnoreOneWay } from './action/IgnoreOneWay';
@@ -30,9 +31,12 @@ import { PlayPattern } from './action/PlayPattern';
import { StopPattern } from './action/StopPattern';
import { PlaySong } from './action/PlaySong';
import { StopSong } from './action/StopSong';
import { ReleaseSongLane, SetSongLane, SetSongTrack, SwitchSong } from './action/songSync';
import { SetZ } from './action/SetZ';
import { Spawn } from './action/Spawn';
import { StopGame } from './action/StopGame';
import { StartRecording, StopRecording } from './action/recordingNodes';
import { PauseVideo, PlayVideo, SeekVideo } from './action/videoNodes';
import { StopSound } from './action/StopSound';
import { CellToWorld, SetTile, TileAt, TileLiteral, WorldToCell } from './action/tileNodes';
import { Abs } from './data/Abs';
@@ -122,6 +126,13 @@ import { SetItem } from './data/SetItem';
import { SubList } from './data/SubList';
import { TextureSize } from './data/TextureSize';
import { OnAnimationFinished } from './events/OnAnimationFinished';
import { OnCompositionFinished } from './events/OnCompositionFinished';
import { OnVideoFinished } from './events/OnVideoFinished';
import { OnBar, OnBeat, OnMusicFinished, OnSongMarker } from './events/musicEvents';
import { OnSoundFinished } from './events/OnSoundFinished';
import { MusicPlaying, SoundPlaying } from './data/MusicPlaying';
import { SongPosition } from './data/SongPosition';
import { VideoTime } from './data/VideoTime';
import { OnBehavior } from './events/OnBehavior';
import { OnClick } from './events/OnClick';
import { OnMouseWheel } from './events/OnMouseWheel';
@@ -182,6 +193,13 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
OnClick,
OnMouseWheel,
OnAnimationFinished,
OnCompositionFinished,
OnVideoFinished,
OnBeat,
OnBar,
OnSongMarker,
OnMusicFinished,
OnSoundFinished,
OnBehavior,
OnStateEntered,
OnStateExited,
@@ -307,6 +325,11 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
PlayAnimation,
PauseAnimation,
StopAnimation,
PlayComposition,
StopComposition,
PlayVideo,
PauseVideo,
SeekVideo,
PlaySound,
StopSound,
SetSoundFxParameters,
@@ -317,15 +340,26 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
StopPattern,
PlaySong,
StopSong,
SetSongTrack,
SetSongLane,
ReleaseSongLane,
SwitchSong,
SongPosition,
MusicPlaying,
SoundPlaying,
VideoTime,
SetBusVolume,
BusVolume,
LoadScene,
Log,
StopGame,
RestartGame,
StartRecording,
StopRecording,
PauseGame,
ResumeGame,
IsGamePaused,
Break,
SetTimeScale,
TimeScale,
];
src/core/instrument/chipPresets.ts
+35 -2
View File
@@ -21,8 +21,10 @@ import {
SoundFxPitch,
} from '../soundfx/nodes';
import {
ACID_ROWS,
CHIP_ROWS,
ENVELOPE_ROWS,
INSTRUMENT_ACID,
INSTRUMENT_CHIP,
INSTRUMENT_ENVELOPE,
INSTRUMENT_NOTES,
@@ -58,6 +60,7 @@ export type TonePreset =
| 'pluck'
| 'triangleBass'
| 'pluckBass'
| 'acidBass'
| 'drumKit';
export type ChipPreset = EffectPreset | TonePreset;
@@ -76,6 +79,7 @@ export const TONE_PRESETS: readonly { value: TonePreset; label: string }[] = [
{ value: 'pluck', label: 'Pluck' },
{ value: 'triangleBass', label: 'Triangle Bass' },
{ value: 'pluckBass', label: 'Pluck Bass' },
{ value: 'acidBass', label: 'Acid Bass' },
{ value: 'drumKit', label: 'Drum Kit' },
];
@@ -393,6 +397,26 @@ function held<K extends string>(
/** Room between the nodes of a preset, left to right and top to bottom: a Chip is a tall node. */
const COLUMN = 240;
const ROW = 340;
/**
* Columns a node takes that is drawn as its editor in the graph (#385): a Filter's curve is wide,
* a Chip's device (#386) wider, an Envelope's (#388) a little less, a Pitch's and a Mix's (#390) two,
* a Delay's (#393), a Phaser's, a Flanger's (#398) and a Bitcrusher's (#399) four, an Acid's (#421) three.
*/
const COLUMNS_OF: Readonly<Record<string, number>> = {
[SoundFxFilter.type]: 4,
[SoundFxBitcrusher.type]: 4,
[SoundFxDelay.type]: 4,
[SoundFxPhaser.type]: 4,
[SoundFxFlanger.type]: 4,
[SoundFxPitch.type]: 2,
[SoundFxMix.type]: 2,
[INSTRUMENT_CHIP]: 5,
[INSTRUMENT_ENVELOPE]: 3,
[INSTRUMENT_ACID]: 3,
};
/** The column right of a node, past all it takes. */
const columnAfter = (node: GraphNode): number => node.x / COLUMN + (COLUMNS_OF[node.type] ?? 1);
/** A node to make: its type and its params. */
type Step = readonly [type: string, params: PropertyBag];
@@ -416,7 +440,7 @@ class Sketch {
line(from: GraphNode, ...steps: readonly Step[]): GraphNode {
let last = from;
for (const [type, params] of steps) {
const next = this.add(type, last.x / COLUMN + 1, last.y / ROW, params);
const next = this.add(type, columnAfter(last), last.y / ROW, params);
this.wire(last, 'out', next, 'in');
last = next;
}
@@ -510,7 +534,7 @@ const echo = (time: number, feedback: number, wet: number, pingPong = false) =>
/** Two ways into one Mix, a column right of the further one; `b` weighs `share`. */
function mixed(s: Sketch, a: GraphNode, b: GraphNode, share: number, curve = 'crossfade'): GraphNode {
const mix = s.add(SoundFxMix.type, Math.max(a.x, b.x) / COLUMN + 1, Math.min(a.y, b.y) / ROW, {
const mix = s.add(SoundFxMix.type, Math.max(columnAfter(a), columnAfter(b)), Math.min(a.y, b.y) / ROW, {
curve,
mix: share,
});
@@ -692,6 +716,15 @@ const TONES: Record<Exclude<TonePreset, 'drumKit'>, (s: Sketch, notes: GraphNode
sustain: 0.15,
release: 0.08,
}),
// nodeseq's acid (#421): a saw through a filter that rings and snaps shut, with a short echo.
acidBass: (s, notes) => {
const acid = s.add(INSTRUMENT_ACID, 1, 0, {
wave: 'saw',
...held({ cutoff: 250, resonance: 0.78, envMod: 0.62 }, ACID_ROWS),
});
s.wire(notes, 'any', acid, 'trigger');
return s.line(acid, echo(0.75, 0.25, 0.1));
},
};
/** An instrument preset: every note at its own pitch, the same graph each time it is picked. */
src/core/instrument/compile.ts
+37
View File
@@ -21,6 +21,7 @@ import { resolveAssetProps } from '../properties/resolve';
import { volumeOf } from '../properties/sounds';
import { soundFxBuildFor, soundFxKey } from '../soundfx/compile';
import {
type FxAcidPlan,
type FxAmount,
type FxChipPlan,
type FxEnvelopePlan,
@@ -41,9 +42,12 @@ import {
withTaps,
} from '../soundfx/nodes';
import {
ACID_ROWS,
acidWave,
CHIP_ROWS,
chipWave,
ENVELOPE_ROWS,
INSTRUMENT_ACID,
INSTRUMENT_CHIP,
INSTRUMENT_ENVELOPE,
INSTRUMENT_NOTES,
@@ -210,6 +214,11 @@ export function compileInstrument(
if (!trigger || !('on' in trigger) || !trigger.on) return { out: SILENT };
return { out: { plans: [{ pitch: [], stages: [], chip: chipOf(node, note, amounts) }] } };
}
case INSTRUMENT_ACID: {
const trigger = inputs.trigger;
if (!trigger || !('on' in trigger) || !trigger.on) return { out: SILENT };
return { out: { plans: [{ pitch: [], stages: [], acid: acidOf(node, note, amounts) }] } };
}
case INSTRUMENT_SOUND_FX: {
const fx = sources.soundFx(textOf(node.params.fx ?? ''));
const arriving = plansAt('in');
@@ -293,6 +302,25 @@ export function compileInstrument(
};
}
/** An Acid's numbers, held to its rows (#421): it always plays the note, as loud as its velocity. */
function acidOf(node: GraphNode, played: number, amounts: Readonly<Record<string, FxAmount>>): FxAcidPlan {
const rows = ACID_ROWS;
return {
id: node.id,
wave: acidWave(node.params),
frequency: { value: noteFrequency(played) },
velocity: { value: 1, param: VELOCITY_PARAM, min: 0, max: 1 },
cutoff: held(amounts.cutoff, rows.cutoff),
resonance: held(amounts.resonance, rows.resonance),
envMod: held(amounts.envMod, rows.envMod),
attack: held(amounts.attack, rows.attack),
decay: held(amounts.decay, rows.decay),
sustain: held(amounts.sustain, rows.sustain),
release: held(amounts.release, rows.release),
glide: held(amounts.glide, rows.glide),
};
}
const output = graph.nodes.find((n) => n.type === INSTRUMENT_OUTPUT);
if (!output) return { layers: [], problems: walk.problems, reason: 'No Output node' };
// Saved while the Output still took any number of sounds: only the first would be heard.
@@ -449,6 +477,15 @@ export function playedNotes(graph: Graph): Set<number> {
return notes;
}
/** Which notes are heard: inside a Notes node's range, and where they lead to something that sounds. */
export function soundingNotes(asset: InstrumentAsset, sources: InstrumentSources): Set<number> {
return new Set(
[...playedNotes(asset.graph)].filter(
(note) => instrumentBuildFor(asset, note, sources).layers.length > 0,
),
);
}
/** The notes a Notes node with single notes has given a name of their own (a drum kit's). */
export function namedNotes(graph: Graph): Map<number, string> {
const names = new Map<number, string>();
src/core/instrument/nodes.ts
+65 -6
View File
@@ -11,6 +11,7 @@ import type { NodeDef, PortDef } from '../graph/NodeRegistry';
import { asNumber } from '../graph/values';
import { type PropertyBag, textOf } from '../model/Properties';
import type { FlatDef, PropertyDef } from '../properties/PropertySchema';
import type { AcidWave } from '../soundfx/acid';
import type { ChipWave } from '../soundfx/chip';
import { type FxChipPlan, soundFxParamPort } from '../soundfx/nodes';
import { HIGHEST_NOTE, heldNote, LOWEST_NOTE, MIDDLE_C, NOTE_OPTIONS, noteName } from './notes';
@@ -120,7 +121,7 @@ const ROOT = noteRow('root', 'Root', MIDDLE_C);
const FOLLOW = { key: 'followNote', label: 'Follow note', kind: 'boolean' as const, default: true };
/** How a Sample plays while its note is held and once it is let go. */
export type SamplePlay = 'gate' | 'oneshot' | 'loop' | 'release';
const SAMPLE_PLAYS: { value: SamplePlay; label: string }[] = [
export const SAMPLE_PLAYS: { value: SamplePlay; label: string }[] = [
{ value: 'gate', label: 'Gate' },
{ value: 'oneshot', label: 'One-shot' },
{ value: 'loop', label: 'Loop' },
@@ -154,7 +155,7 @@ export function withSamplePlay(params: PropertyBag): PropertyBag {
const { mode: _mode, loop: _loop, ...rest } = params;
return { ...rest, [PLAY.key]: samplePlay(params) };
}
const LOOP_START: PropertyDef = {
const LOOP_START: FlatDef<'number'> = {
key: 'loopStart',
label: 'Loop start (s)',
kind: 'number',
@@ -162,7 +163,7 @@ const LOOP_START: PropertyDef = {
min: 0,
step: 0.01,
};
const LOOP_END: PropertyDef = {
const LOOP_END: FlatDef<'number'> = {
key: 'loopEnd',
label: 'Loop end (s)',
kind: 'number',
@@ -170,6 +171,8 @@ const LOOP_END: PropertyDef = {
min: 0,
step: 0.01,
};
/** Where a Sample loops, in seconds; its device (#400) types them in. */
export const SAMPLE_LOOP_ROWS = { loopStart: LOOP_START, loopEnd: LOOP_END };
export const InstrumentSample: InstrumentNodeDef = {
type: INSTRUMENT_SAMPLE,
@@ -320,7 +323,7 @@ export const InstrumentChip: InstrumentNodeDef = {
category: 'sound',
instrument: true,
description:
'A raw wave as old game consoles made it, for every note that triggers it, the way sfxr makes its sounds. "Wave" is Square, Triangle, Saw, Sine, Noise, or Noise (periodic), the short metallic noise of the NES; a square and that noise are half as loud as the others, as in sfxr. With "follow note" it plays at the pitch of the note, else at "frequency" in Hz; a number connected to "frequency" goes before both. "duty" is how much of a square is up: 0.5 is even, 0.125 thin and nasal; "duty sweep" moves it by that much per second. "slide" glides the pitch in semitones per second, "delta slide" changes the slide by that much per second. Where it falls below "min frequency" the note ends; at 0 it never does. "vibrato depth" in semitones and "vibrato speed" in Hz make it wobble. "arpeggio" jumps it by that many semitones "arpeggio time" seconds into the note. "repeat" starts slide, duty sweep and arpeggio over every so many seconds, 0 never. The Chip plays until its note ends, faded out in a few milliseconds; under an Envelope it plays on through the release. An Envelope can move "frequency" and "duty" while it plays, a Pitch moves its frequency itself. Settings that do nothing as things stand are grey.',
'A raw wave as old game consoles made it, for every note that triggers it, the way sfxr makes its sounds. "Wave" is Square, Triangle, Saw, Sine, Noise, or Noise (periodic), the short metallic noise of the NES; a square and that noise are half as loud as the others, as in sfxr. With "follow note" it plays at the pitch of the note, else at "frequency" in Hz; a number connected to "frequency" goes before both. "duty" is how much of a square is up: 0.5 is even, 0.125 thin and nasal; "duty sweep" moves it by that much per second. "slide" glides the pitch in semitones per second, "delta slide" changes the slide by that much per second. Where it falls below "min frequency" the note ends; at 0 it never does. "vibrato depth" in semitones and "vibrato speed" in Hz make it wobble. "arpeggio" jumps it by that many semitones "arpeggio time" seconds into the note. "repeat" starts slide, duty sweep and arpeggio over every so many seconds, 0 never. The Chip plays until its note ends, faded out in a few milliseconds; under an Envelope it plays on through the release. An Envelope can move "frequency" and "duty" while it plays, a Pitch moves its frequency itself. Settings that do nothing as things stand are grey. In the graph the node is a device: it draws its wave over two periods, and what slide, vibrato, arpeggio and repeat make of the pitch over a note, from A4 when it follows the note.',
inputs: [
triggerIn,
...Object.values(CHIP_ROWS).map((row): PortDef =>
@@ -342,6 +345,61 @@ export const InstrumentChip: InstrumentNodeDef = {
},
};
export const INSTRUMENT_ACID = 'InstrumentAcid';
export const ACID_WAVES: { value: AcidWave; label: string }[] = [
{ value: 'saw', label: 'Saw' },
{ value: 'square', label: 'Square' },
];
const ACID_WAVE: PropertyDef = {
key: 'wave',
label: 'Wave',
kind: 'enum',
default: 'saw',
options: ACID_WAVES,
};
export function acidWave(params: PropertyBag): AcidWave {
return ACID_WAVES.find((option) => option.value === params[ACID_WAVE.key])?.value ?? 'saw';
}
/** The defaults are nodeseq's acid bass: Cutoff 30, Reso 78, Env 62 of its 100, its ADSR and its glide. */
export const ACID_ROWS = {
cutoff: chipRow('cutoff', 'Cutoff (Hz)', 40, 18000, 1, 250),
resonance: chipRow('resonance', 'Resonance', 0, 1, 0.01, 0.78),
envMod: chipRow('envMod', 'Env mod', 0, 1, 0.01, 0.62),
attack: chipRow('attack', 'Attack (s)', 0.001, 2, 0.001, 0.003),
decay: chipRow('decay', 'Decay (s)', 0.02, 3, 0.01, 0.16),
sustain: chipRow('sustain', 'Sustain', 0, 1, 0.01, 0.15),
release: chipRow('release', 'Release (s)', 0.02, 4, 0.01, 0.08),
glide: chipRow('glide', 'Glide (s)', 0, 0.5, 0.001, 0.05),
} satisfies Record<string, FlatDef<'number'>>;
export const InstrumentAcid: InstrumentNodeDef = {
type: INSTRUMENT_ACID,
label: 'Acid',
category: 'sound',
instrument: true,
description:
'An acid bass, one note at a time, as nodeseq.com plays it; its defaults are nodeseq\'s acid. "Wave" is a Saw or a Square at the pitch of the note, through a low-pass with a sharp resonance. "cutoff" is where the filter rests, "resonance" how sharply it rings there. One ADSR shapes the volume and the filter alike: in "attack" seconds the filter opens by up to five octaves with "env mod", in "decay" it falls back to where "sustain" holds it, and when the note ends it closes in "release". Every note strikes anew and glides from the pitch of the note before in "glide" seconds; a note that starts while the one before still sounds cuts it short. Notes that start together all play. The velocity makes it louder or softer. The Acid shapes its own volume and plays out its release, so it needs no Envelope behind it. An Envelope can move "cutoff", a Pitch moves its frequency. In the graph the node is a device: it draws the filter where it rests, at the top of its sweep and where the sustain holds it.',
inputs: [
triggerIn,
...Object.values(ACID_ROWS).map((row): PortDef =>
row.key === ACID_ROWS.cutoff.key ? { ...numberIn(row), followsEnvelope: true } : numberIn(row),
),
],
outputs: [audioOut],
params: [ACID_WAVE],
// At full sustain and without a sweep nothing falls: the decay has nothing to do.
idleParams: (params, fed = () => false) =>
!fed('sustain') &&
!fed('envMod') &&
asNumber(params.sustain ?? ACID_ROWS.sustain.default) >= 1 &&
asNumber(params.envMod ?? ACID_ROWS.envMod.default) <= 0
? [ACID_ROWS.decay.key]
: [],
};
export const InstrumentOutput: InstrumentNodeDef = {
type: INSTRUMENT_OUTPUT,
label: 'Output',
@@ -358,6 +416,7 @@ export const INSTRUMENT_NODES: readonly InstrumentNodeDef[] = [
InstrumentNotes,
InstrumentSample,
InstrumentChip,
InstrumentAcid,
InstrumentParameters,
InstrumentSoundFx,
InstrumentEnvelope,
@@ -366,7 +425,7 @@ export const INSTRUMENT_NODES: readonly InstrumentNodeDef[] = [
/**
* What a new instrument starts as: every note of an octave from C4 plays one Sample, which waits for
* its sound to be picked.
* its sound to be picked. The Output sits past the Sample's device (#400), some 580 wide.
*/
export function starterInstrumentGraph(): Graph {
const notes = newId();
@@ -376,7 +435,7 @@ export function starterInstrumentGraph(): Graph {
nodes: [
{ id: notes, type: INSTRUMENT_NOTES, x: 0, y: 0, params: {} },
{ id: sample, type: INSTRUMENT_SAMPLE, x: 220, y: 0, params: {} },
{ id: output, type: INSTRUMENT_OUTPUT, x: 460, y: 0, params: {} },
{ id: output, type: INSTRUMENT_OUTPUT, x: 840, y: 0, params: {} },
],
edges: [
{ id: newId(), from: { node: notes, port: 'any' }, to: { node: sample, port: 'trigger' } },
src/core/model/Asset.ts
+108 -2
View File
@@ -1,5 +1,6 @@
import type { Graph } from '../graph/Graph';
import type { Bake } from '../music/bake';
import type { SongMarker } from '../music/markers';
import type { PatternNote } from '../music/pattern';
import type { SongItem, SongTrack } from '../music/song';
import type { LoopRange } from '../music/sequence';
@@ -8,13 +9,17 @@ import type { AssetType } from './AssetTypes';
import type { PropertyBag, PropertyValue, Vec2 } from './Properties';
import type { Scope } from './Scope';
import type { ObjectGroup, SceneObject } from './SceneObject';
import type { FadeCurve } from './sound';
export interface AssetBase {
id: string;
type: AssetType;
name: string;
props: PropertyBag;
/** The song a sound, pattern or sound FX made in it belongs to (#355); absent on a project asset. */
/**
* The song a sound, pattern or sound FX made in it belongs to (#355), or the composition a sprite
* worn by its items does (#419); absent on a project asset.
*/
owner?: string;
}
@@ -260,6 +265,78 @@ export interface AnimationAsset extends AssetBase {
tracks: AnimationTrack[];
}
/** A lane of a composition (#417); the first one lies in front. */
export interface CompositionTrack {
id: string;
name: string;
}
/**
* One stretch on a track: the object of the same id lives from `start` for `length` frames.
* `offset` is how many seconds into its video the item begins, which a trim of the left edge moves;
* below 0 the item begins before the video and shows its first picture until then. `fitMedia` says
* its length still follows that video, or the default while there is none. Items of a track may
* overlap. An item with `audio` spawns no object: it plays a sound, a pattern or a song (#418), and
* `offset` is how far into that it begins. `fadeIn` and `fadeOut` are the frames its sound – what it
* plays, or its video's – rises from silence at its start and falls back at its end, shaped by
* `curveIn` and `curveOut`; absent for none.
*/
export interface CompositionItem {
id: string;
track: string;
start: number;
length: number;
offset: number;
fitMedia?: boolean;
audio?: CompositionAudio;
fadeIn?: number;
fadeOut?: number;
curveIn?: FadeCurve;
curveOut?: FadeCurve;
}
export type CompositionAudioKind = 'sound' | 'pattern' | 'song';
/**
* What an audio item plays (#418), round and round for as long as it lasts – a song round its loop.
* `asset` is a sound, a clip of one, a pattern or a song; `volume` is in decibels. `bus` is the bus
* of the mix it is heard on: `''` where the routing sends its asset, else a bus or the master.
*/
export interface CompositionAudio {
kind: CompositionAudioKind;
asset: string;
volume: number;
/** '' along the routing of its asset, `TO_COMPOSITION_BUS` into the composition's bus, else as a Play node picks. */
bus: string;
}
/**
* An animation laid on a track from `start`, one pass long. It plays on every item of its track it
* overlaps; its rows find their targets there as `Play animation` finds them.
*/
export interface CompositionClip {
id: string;
track: string;
animation: string;
start: number;
}
/**
* Objects along a time axis (#417), for a cut or a cutscene: each item spawns its object for as long
* as it lasts, and animation clips play on the items they overlap. The objects are stored like a
* scene's, overrides and container edits included, one per item and all at the top level. Time is
* counted in frames of its own `fps` (see `model/composition.ts`).
*/
export interface CompositionAsset extends AssetBase {
type: 'composition';
objects: SceneObject[];
/** Hierarchy folders, as a scene's (#241); absent when there are none. */
groups?: ObjectGroup[];
tracks: CompositionTrack[];
items: CompositionItem[];
clips: CompositionClip[];
}
/**
* The graph holds one `State` node per state and an edge per transition; a behavior reads the
* machine's current state through `State branch` and moves it through `Set state` or `State transition`.
@@ -348,6 +425,31 @@ export interface SoundClip {
stream?: SoundStream;
}
/**
* `file` names the video in `data/videos/` (#402): `{folder}/{sha256}.mp4`, the MP4 the worker made
* from the upload, with a poster `.jpg` and its details beside it. The size, length, frame rate and
* whether it has sound are copied in when it is picked, so a sprite can be measured before the file
* loads. A sprite wears it through `texture`; its clips are stretches of it, as a sound's are.
*/
export interface VideoAsset extends AssetBase {
type: 'video';
file: string;
width: number;
height: number;
duration: number;
fps: number;
audio: boolean;
/** Absent for a video without clips. */
clips?: VideoClip[];
}
/** A stretch of a video with a name, `props` its start and end (see `properties/videos.ts`). */
export interface VideoClip {
id: string;
name: string;
props: PropertyBag;
}
/**
* What a sound is played through (#311): the graph runs from its `Input` over effect nodes to its
* `Output`, and a `Play sound` naming it sends the voice along that line (see `core/soundfx`).
@@ -389,6 +491,8 @@ export interface SongAsset extends AssetBase {
tracks: SongTrack[];
items: SongItem[];
loop?: LoopRange;
/** Named spots on its time (#338); left out without one. */
markers?: SongMarker[];
graph: Graph;
bake?: Bake;
}
@@ -645,6 +749,7 @@ export type Asset =
| PixelArtAsset
| PaletteAsset
| AnimationAsset
| CompositionAsset
| CurveAsset
| TileAsset
| TilemapAsset
@@ -658,7 +763,8 @@ export type Asset =
| PatternAsset
| SongAsset
| BusAsset
| ParticlesAsset;
| ParticlesAsset
| VideoAsset;
/** The assets whose `variables` are declared by the user – see `requireVariableOwner`. */
export type VariableOwnerAsset = BehaviorAsset | VariablesAsset | StructAsset;
src/core/model/AssetTypes.ts
+9 -1
View File
@@ -30,6 +30,8 @@ export const ASSET_TYPES = [
'song',
'bus',
'particles',
'video',
'composition',
] as const;
export type AssetType = (typeof ASSET_TYPES)[number];
@@ -66,6 +68,8 @@ export const ASSET_TYPE_LABELS: Record<AssetType, string> = {
song: 'Song',
bus: 'Bus',
particles: 'Particles',
video: 'Video',
composition: 'Composition',
};
/**
@@ -79,7 +83,8 @@ export const ASSET_TYPE_LABELS: Record<AssetType, string> = {
* A shader function is part of other shaders (#303), worn by nothing itself. A sound FX is what a
* sound is played through (#311), heard but never seen, an instrument what a note is played
* with (#331), a pattern the notes it plays (#333) and a song what lays them out (#334). A bus is
* where sounds are mixed on their way out (#325).
* where sounds are mixed on their way out (#325). A video is a moving picture a sprite wears through
* `texture` (#402), as it wears an image.
*/
export type PlaceableAssetType = Exclude<
AssetType,
@@ -103,6 +108,7 @@ export type PlaceableAssetType = Exclude<
| 'bus'
| 'variables'
| 'struct'
| 'video'
>;
const NOT_PLACEABLE: ReadonlySet<AssetType> = new Set([
@@ -126,6 +132,7 @@ const NOT_PLACEABLE: ReadonlySet<AssetType> = new Set([
'bus',
'variables',
'struct',
'video',
]);
export function isPlaceableType(type: AssetType): type is PlaceableAssetType {
@@ -145,6 +152,7 @@ export function isSymbolType(type: AssetType): boolean {
type === 'rigidbody' ||
type === 'joint' ||
type === 'container' ||
type === 'composition' ||
type === 'clip'
);
}
src/core/model/Project.ts
+6 -4
View File
@@ -1,5 +1,6 @@
import {
type Asset,
type CompositionAsset,
type ContainerAsset,
type GraphAsset,
isGraphAsset,
@@ -151,13 +152,14 @@ export function findGraphAsset(project: Project, assetId: string): GraphHolder |
}
/**
* Everything that holds placed objects. A container's `objects` are a template, so anything
* walking "all placed objects" must cover both or it silently skips the templates.
* Everything that holds placed objects. A container's `objects` are a template, and a composition's
* live only while their items run, so anything walking "all placed objects" must cover all three or
* it silently skips some.
*/
export type ObjectHolder = SceneAsset | ContainerAsset;
export type ObjectHolder = SceneAsset | ContainerAsset | CompositionAsset;
function isObjectHolder(asset: Asset): asset is ObjectHolder {
return asset.type === 'scene' || asset.type === 'container';
return asset.type === 'scene' || asset.type === 'container' || asset.type === 'composition';
}
export function objectHolders(project: Project): ObjectHolder[] {
src/core/model/animation.ts
+122 -2
View File
@@ -23,6 +23,7 @@ import { instanceChildId, resolveObjects } from './instance';
import { findAsset, type Project } from './Project';
import { colorOf, type PropertyBag, type PropertyValue, textOf } from './Properties';
import { objectName, type SceneObject } from './SceneObject';
import { findVideo, type VideoRef } from './video';
import { mixColor } from '../util/color';
/** `once` holds the last value when it ends, `once-reset` puts back what was there before it started. */
@@ -427,11 +428,12 @@ export function placedRoot(root: Asset): SceneObject {
};
}
/** One object a row may aim at below the root, under the id the row stores. */
/** One object a row may aim at below the root, under the id the row stores; `parent` is the one above it, '' at the top. */
export interface AnimationTarget {
id: string;
object: SceneObject;
depth: number;
parent: string;
}
/**
@@ -446,12 +448,18 @@ export function animationTargets(project: Project, root: Asset | undefined): Ani
id: object.id,
object,
depth,
parent: depth > 0 ? object.parentId : '',
}));
if (root?.type !== 'container') return [];
const prefix = instanceChildId(PLACED_ROOT_ID, '');
return treeOrder({ objects: resolveObjects(project, { objects: [placedRoot(root)] }) })
.filter(({ object }) => object.id.startsWith(prefix))
.map(({ object, depth }) => ({ id: object.id.slice(prefix.length), object, depth: depth - 1 }));
.map(({ object, depth }) => ({
id: object.id.slice(prefix.length),
object,
depth: depth - 1,
parent: object.parentId.startsWith(prefix) ? object.parentId.slice(prefix.length) : '',
}));
}
/**
@@ -522,6 +530,26 @@ export function trackDef(
return targetDefs(project, animation, track.target)?.find((d) => d.key === track.property);
}
/**
* The video the object a row aims at wears at `frame`, for a row shown only with one (#407): what
* the texture row the def names holds there – keyed in this animation, or else on the object.
*/
export function rowVideoAt(
project: Project,
animation: AnimationAsset,
track: AnimationTrack,
frame: number,
): VideoRef | null {
const aimed = aimedAt(project, animation, track.target);
const def =
aimed && animatableDefsOf(project, aimed.asset, aimed.bag).find((d) => d.key === track.property);
const textureKey = def && 'shownForVideo' in def ? def.shownForVideo : undefined;
if (!aimed || !textureKey) return null;
const keyed = animation.tracks.find((t) => t.target === track.target && t.property === textureKey);
const value = (keyed && trackValueAt(keyed.keys, 'texture', frame)) ?? aimed.bag[textureKey];
return findVideo(project, textOf(value ?? ''));
}
/**
* The rows the dialog offers for the object `target` names: what it can animate, less a behavior's
* private variables (#270) and what another row animates already. A row on a private variable keeps
@@ -556,6 +584,98 @@ export function trackLabel(project: Project, animation: AnimationAsset, track: A
return `${who} · ${def ? def.label : aimed ? `${track.property} (missing)` : track.property}`;
}
/**
* A line of the timeline's header column (#408): the head of an object's group, or a row under it,
* named by its property alone. `depth` is the object's in the hierarchy, `above` the objects it sits
* under, the nearest last – the root not among them: its head stands level with its children's (#414).
* A row carries its property (none once that is gone) and `held`, what it shows before it has keys:
* it writes nothing then, so that is what the object holds – or no offset at all for a relative row.
*/
export type TimelineLine =
| { kind: 'group'; target: string; name: string; depth: number; above: readonly string[] }
| {
kind: 'row';
track: AnimationTrack;
label: string;
def: PropertyDef | undefined;
held: PropertyValue;
depth: number;
};
/**
* The rows grouped by the object they aim at, in the order the objects stand in the hierarchy – the
* root first – and under each object in the order it lists its properties. Rows aiming at something
* gone come last, as they were added. Only the view is sorted: the asset keeps its own order.
*/
export function timelineLines(project: Project, animation: AnimationAsset): TimelineLine[] {
const placed = new Map<string, { at: number; depth: number; parent: string }>([
['', { at: 0, depth: 0, parent: '' }],
]);
animationTargets(project, findAsset(project, animation.root)).forEach((t, i) =>
placed.set(t.id, { at: i + 1, depth: t.depth, parent: t.parent }),
);
const aboveOf = (target: string): string[] => {
const above: string[] = [];
let at = placed.get(target)?.parent ?? '';
while (at !== '' && !above.includes(at)) {
above.unshift(at);
at = placed.get(at)?.parent ?? '';
}
return above;
};
const targets = [...new Set(animation.tracks.map((t) => t.target))];
const rank = (target: string) => placed.get(target)?.at ?? placed.size + targets.indexOf(target);
const lines: TimelineLine[] = [];
for (const target of targets.sort((a, b) => rank(a) - rank(b))) {
const aimed = aimedAt(project, animation, target);
const defs = aimed ? animatableDefsOf(project, aimed.asset, aimed.bag) : [];
const depth = placed.get(target)?.depth ?? 0;
const name = !aimed
? '(missing)'
: aimed.object
? objectName(aimed.object, aimed.asset)
: aimed.asset.name;
lines.push({ kind: 'group', target, name, depth, above: aboveOf(target) });
const order = (track: AnimationTrack) => {
const i = defs.findIndex((d) => d.key === track.property);
return i < 0 ? defs.length : i;
};
const rows = animation.tracks.filter((t) => t.target === target).sort((a, b) => order(a) - order(b));
for (const track of rows) {
const def = defs.find((d) => d.key === track.property);
const label = def ? def.label : `${track.property} (missing)`;
lines.push({ kind: 'row', track, label, def, held: heldBy(aimed, def, track), depth });
}
}
return lines;
}
function heldBy(
aimed: Aimed | undefined,
def: PropertyDef | undefined,
track: AnimationTrack,
): PropertyValue {
if (!aimed || !def || track.relative) return 0;
if (isPositionRow(aimed.asset.type, def.key)) return aimed.object?.position[def.key] ?? 0;
return aimed.bag[def.key] ?? def.default;
}
/** What a row holds without keys (#412): the value its object has, no offset for a relative row. */
export function trackHeld(project: Project, animation: AnimationAsset, track: AnimationTrack): PropertyValue {
return heldBy(aimedAt(project, animation, track.target), trackDef(project, animation, track), track);
}
/** What a row's header shows at `frame` (#408): the row's value there, else what it holds without keys. */
export function lineValueAt(
line: TimelineLine & { kind: 'row' },
frame: number,
reverse = false,
curves: CurveLookup = NO_CURVES,
): PropertyValue | undefined {
if (!line.def) return undefined;
return trackValueAt(line.track.keys, line.def.kind, frame, reverse, curves) ?? line.held;
}
/** Whether a value names something in the project – what deleting that something has to count. */
export function referencesAny(
track: AnimationTrack,
src/core/model/animationPreview.ts
+4 -3
View File
@@ -41,7 +41,7 @@ export function previewHolder(
const at: ReadAt = { frame, reverse, curves: curveLookup(project) };
for (const track of reset ? [] : animation.tracks) {
if (root.type === 'scene' && track.target === '') writeScene(holder, track, at);
else writeObject(project, holder, previewObjectId(root, track.target), track, at);
else writeObjectRow(project, holder, previewObjectId(root, track.target), track, at);
}
return holder;
}
@@ -71,7 +71,7 @@ function placedHolder(root: Asset): ObjectHolder {
}
/** Where on the timeline a row is read, and the Curve assets its keys may blend along. */
interface ReadAt {
export interface ReadAt {
frame: number;
reverse: boolean;
curves: CurveLookup;
@@ -86,7 +86,8 @@ function writeScene(holder: ObjectHolder, track: AnimationTrack, at: ReadAt): vo
if (value !== undefined) holder.props[track.property] = value;
}
function writeObject(
/** A row on the object `id` of the holder, stored there or resolved from a container – also a composition's (#417). */
export function writeObjectRow(
project: Project,
holder: ObjectHolder,
id: string,
src/core/model/assetParts.ts
+28
View File
@@ -0,0 +1,28 @@
import type { Asset } from './Asset';
import { frameLabel } from './pixelart';
import { findAsset, type Project } from './Project';
import { clipsOf, hasClips } from './sound';
/** A named piece of an asset with an id of its own: a clip of a sound or video, a frame of a picture. */
export interface AssetPart {
id: string;
name: string;
}
/**
* What the asset panel lists under an asset (#425, #430): the clips of a sound or a video, the frames
* of a texture, and those of pixel art once it has more than one – the frames a texture field offers.
*/
export function partsOf(asset: Asset): AssetPart[] {
if (hasClips(asset)) return clipsOf<AssetPart>(asset).map(({ id, name }) => ({ id, name }));
if (asset.type === 'texture') return asset.frames.map(({ id, name }) => ({ id, name }));
if (asset.type === 'pixelart' && asset.frames.length > 1)
return asset.frames.map((frame, index) => ({ id: frame.id, name: frameLabel(index) }));
return [];
}
/** The asset `id` is a listed part of; undefined for an asset's own id or anything else. */
export function partOwner(project: Project, id: string): Asset | undefined {
if (id === '' || findAsset(project, id)) return undefined;
return project.assets.find((asset) => partsOf(asset).some((part) => part.id === id));
}
src/core/model/clipboard.ts
+28 -1
View File
@@ -14,6 +14,9 @@ import {
import { editorTransform, parentTransform, positionOf } from '../properties/transform';
import { findObject, subtreeIds, subtreeRoots, treeOrder } from './hierarchy';
import { isLiveInstance, isTemplateContent, resolvedHolder, splitInstanceId } from './instance';
import type { CompositionClip, CompositionItem, SpriteAsset } from './Asset';
import { isCompositionSprite } from './composition';
import { carriedClips } from './compositionEdit';
import type { Mat } from './matrix';
import type { Position } from './Properties';
import type { ObjectHolder, Project } from './Project';
@@ -34,6 +37,14 @@ export interface ObjectClip {
/** Parents before children; a root's `parentId` is the one it had where it was copied. */
objects: SceneObject[];
roots: ClipRoot[];
/** Copied out of a composition (#417): the items of the roots that are items, for a paste there. */
items?: CompositionItem[];
/** The animation clips that belong to those items, to go with them. */
clips?: CompositionClip[];
/** The sprites of a composition's own the objects wear (#419), back for a paste after a cut. */
sprites?: SpriteAsset[];
/** Audio items copied with them (#418): they have no object. */
audio?: CompositionItem[];
}
export interface GraphClip {
@@ -44,7 +55,14 @@ export interface GraphClip {
annotations: { kind: AnnotationKind; annotation: GraphAnnotation }[];
}
export type Clip = ObjectClip | GraphClip;
/** Animation clips or audio items of a composition (#417, #418), for a paste into the same one or another. */
export interface CompositionClipsClip {
kind: 'compositionClips';
clips: CompositionClip[];
items?: CompositionItem[];
}
export type Clip = ObjectClip | GraphClip | CompositionClipsClip;
/**
* The selected objects with everything below them. A placed container keeps showing its template,
@@ -76,9 +94,18 @@ export function copyObjects(
.map((entry) => entry.object)
.filter((o) => members.has(o.id) && !shownByInstance(o.id))
.map((o) => ({ ...structuredClone(o), editorOffset: { x: 0, y: 0 } }));
const items = holder.type === 'composition' ? holder.items.filter((i) => roots.includes(i.id)) : [];
const carried =
holder.type === 'composition' ? carriedClips(holder.items, holder.clips, new Set(roots)) : new Set();
const clips = holder.type === 'composition' ? holder.clips.filter((c) => carried.has(c.id)) : [];
const worn = new Set(objects.map((o) => o.assetId));
const sprites = project.assets.filter(isCompositionSprite).filter((a) => worn.has(a.id));
return {
kind: 'objects',
objects,
...(items.length > 0 && { items: structuredClone(items) }),
...(clips.length > 0 && { clips: structuredClone(clips) }),
...(sprites.length > 0 && { sprites: structuredClone(sprites) }),
roots: roots.map((id) => {
const object = findObject(view, id)!;
return {
src/core/model/composition.ts
+539
View File
@@ -0,0 +1,539 @@
/**
* Compositions (#417): objects along a time axis, counted in frames of the composition's own `fps`.
* What runs at a frame is one pure function, `compositionAt`, so the tab's preview and the running
* game read the same cut; `compositionHolder` writes it out as objects for a view to draw.
*/
import { newId } from '../ids';
import { patternLength, patternTempo } from '../music/pattern';
import { ITEM_MIN_DB, songLoop, songTempo, SOUND_ITEM_MAX_DB } from '../music/song';
import { secondsOfTicks } from '../music/time';
import {
COMPOSITION_FPS_KEY,
COMPOSITION_LOOP_KEY,
DEFAULT_COMPOSITION_FPS,
} from '../properties/compositions';
import { playedSoundById } from '../properties/playedSound';
import { resolveSceneObjectProps } from '../properties/resolve';
import { DEFAULT_FADE_CURVE } from '../properties/sounds';
import {
VIDEO_CLOCK_KEY,
VIDEO_LEVEL_KEY,
VIDEO_TIME_KEY,
videoDrivenOf,
videoLevelOf,
} from '../properties/videos';
import { animationDuration, fpsOf, frameCount, targetNamePath } from './animation';
import { type ReadAt, writeObjectRow } from './animationPreview';
import type {
AnimationAsset,
AnimationTrack,
Asset,
CompositionAsset,
CompositionAudio,
CompositionAudioKind,
CompositionClip,
CompositionItem,
CompositionTrack,
SpriteAsset,
} from './Asset';
import { curveLookup } from './curve';
import {
contentId,
instanceChildId,
objectRef,
placedWithContent,
resolveObjects,
splitInstanceId,
} from './instance';
import { findAsset, findAssetOf, type Project } from './Project';
import { booleanOf, type PropertyBag } from './Properties';
import { objectName, type SceneObject } from './SceneObject';
import { type FadeCurve, fadeGain, isFadeCurve } from './sound';
import { findVideo, videoLengthOf, type VideoRef } from './video';
/** How long an item lasts while it has no video to take its length from. */
export const DEFAULT_ITEM_SECONDS = 3;
export function compositionFps(comp: CompositionAsset): number {
const fps = Number(comp.props[COMPOSITION_FPS_KEY]);
return Number.isFinite(fps) && fps > 0 ? fps : DEFAULT_COMPOSITION_FPS;
}
export function compositionLoops(comp: CompositionAsset): boolean {
return booleanOf(comp.props[COMPOSITION_LOOP_KEY]);
}
/** A sprite a composition keeps to itself (#419), worn by the items a texture or video was laid as. */
export function isCompositionSprite(asset: Asset): asset is SpriteAsset {
return asset.type === 'sprite' && (asset.owner ?? '') !== '';
}
export function newCompositionTrack(index: number): CompositionTrack {
return { id: newId(), name: `Track ${index}` };
}
export function itemEnd(item: CompositionItem): number {
return item.start + item.length;
}
/** The object an item spawns; undefined once it is gone, and then the item does nothing – or for an audio item. */
export function itemObject(comp: CompositionAsset, item: CompositionItem): SceneObject | undefined {
return item.audio ? undefined : comp.objects.find((o) => o.id === item.id);
}
/** Whether an item stands on its track: an audio item always (#418), any other while it has its object. */
export function itemStands(comp: CompositionAsset, item: CompositionItem): boolean {
return item.audio !== undefined || itemObject(comp, item) !== undefined;
}
/** Seconds a sound file lasts once its bytes are there, as the editor's sound library knows it (#418). */
let soundSeconds: (file: string) => number | null = () => null;
export function setSoundMeasure(measure: (file: string) => number | null): void {
soundSeconds = measure;
}
/** Whether an item has a sound its fades shape: what an audio item plays, or the video its object wears. */
export function itemSounds(project: Project, comp: CompositionAsset, item: CompositionItem): boolean {
return item.audio !== undefined || itemVideo(project, comp, item) !== null;
}
/**
* The fades of an item in whole frames as they work: the fade in no longer than the item, the fade
* out no longer than the rest.
*/
export interface ItemFades {
fadeIn: number;
fadeOut: number;
curveIn: FadeCurve;
curveOut: FadeCurve;
}
export function itemFades(item: CompositionItem): ItemFades {
const frames = (value: number | undefined, most: number) =>
Math.max(0, Math.min(most, Math.round(Number.isFinite(value) ? value! : 0)));
const fadeIn = frames(item.fadeIn, item.length);
return {
fadeIn,
fadeOut: frames(item.fadeOut, item.length - fadeIn),
curveIn: item.curveIn ?? DEFAULT_FADE_CURVE,
curveOut: item.curveOut ?? DEFAULT_FADE_CURVE,
};
}
/** How loud an item's sound is `at` frames after its start, its fades considered: 0 to 1. */
export function fadeLevel(item: CompositionItem, at: number): number {
const { fadeIn, fadeOut, curveIn, curveOut } = itemFades(item);
const rise = fadeIn > 0 && at < fadeIn ? fadeGain(curveIn, at / fadeIn) : 1;
const left = item.length - at;
const fall = fadeOut > 0 && left < fadeOut ? fadeGain(curveOut, left / fadeOut) : 1;
return rise * fall;
}
/**
* Seconds the first pass of what an audio item plays lasts: its sound once loaded, its pattern, its
* song to the end of its loop – it goes round there, what lies behind is never heard –; 0 while that
* is not known or the asset is gone.
*/
export function audioSeconds(project: Project, audio: CompositionAudio): number {
if (audio.kind === 'sound') {
const file = playedSoundById(project, audio.asset);
return (file && soundSeconds(file)) || 0;
}
if (audio.kind === 'pattern') {
const pattern = findAssetOf(project, audio.asset, 'pattern');
return pattern ? secondsOfTicks(patternLength(pattern), patternTempo(pattern.props)) : 0;
}
const song = findAssetOf(project, audio.asset, 'song');
return song ? secondsOfTicks(songLoop(song).end, songTempo(song.props)) : 0;
}
/** Seconds every pass after the first lasts: a song's loop, else the same as the first. */
export function audioRoundSeconds(project: Project, audio: CompositionAudio): number {
const song = audio.kind === 'song' ? findAssetOf(project, audio.asset, 'song') : undefined;
if (!song) return audioSeconds(project, audio);
const loop = songLoop(song);
return secondsOfTicks(loop.end - loop.start, songTempo(song.props));
}
/**
* The animation a clip plays, unless it is gone or built over a scene: a scene's rows aim at that
* scene's objects, nothing an item holds.
*/
function clipAnimation(project: Project, clip: CompositionClip): AnimationAsset | undefined {
const animation = findAssetOf(project, clip.animation, 'animation');
return animation && findAsset(project, animation.root)?.type !== 'scene' ? animation : undefined;
}
/** Frames one pass of the clip's animation takes in the composition; 0 when it plays nothing. */
export function clipLength(project: Project, comp: CompositionAsset, clip: CompositionClip): number {
const animation = clipAnimation(project, clip);
const seconds = animation ? animationDuration(animation) : 0;
return seconds > 0 ? Math.max(1, Math.ceil(seconds * compositionFps(comp) - 1e-6)) : 0;
}
/** The frame after the last item or clip ends: how long one run of the composition is. */
export function compositionLength(project: Project, comp: CompositionAsset): number {
let end = 0;
for (const item of comp.items) if (itemStands(comp, item)) end = Math.max(end, itemEnd(item));
for (const clip of comp.clips) end = Math.max(end, clip.start + clipLength(project, comp, clip));
return end;
}
/**
* The video the item's own object wears through its texture – the override's, else its sprite's. A
* video worn further down a container item runs with the item but gives it no length.
*/
export function itemVideo(project: Project, comp: CompositionAsset, item: CompositionItem): VideoRef | null {
const object = itemObject(comp, item);
const asset = object && findAsset(project, object.assetId);
if (!object || asset?.type !== 'sprite') return null;
return videoWorn(resolveSceneObjectProps(project, asset, object), project);
}
function videoWorn(props: PropertyBag, project: Project): VideoRef | null {
const texture = props.texture;
return typeof texture === 'string' ? findVideo(project, texture) : null;
}
/**
* The length an item with `fitMedia` takes: from its start to the end of its video, or of what an
* audio item plays; the default while it has none or its length is not known yet.
*/
export function fittedLength(project: Project, comp: CompositionAsset, item: CompositionItem): number {
const fps = compositionFps(comp);
const video = item.audio ? null : itemVideo(project, comp, item);
const seconds = item.audio
? audioSeconds(project, item.audio) - item.offset
: video
? videoLengthOf(video) - item.offset
: 0;
return seconds > 0 ? Math.max(1, Math.round(seconds * fps)) : Math.round(DEFAULT_ITEM_SECONDS * fps);
}
/**
* A composition placed in a running game that `Play composition` started; kept on its entity, out of
* `props`. Its items exist only while it plays.
*/
export interface CompositionPlayer {
/** Tells a start apart from the one before, so a restart builds every item anew. */
run: number;
/** Seconds since its first frame. */
time: number;
/** Whether the first frame was shown: the clock runs from the tick after the start. */
shown: boolean;
/** Set by `Stop composition`: the next tick takes its items away. */
stopped: boolean;
/** The behaviors that played it; they hear `On composition finished` wherever it stands. */
startedBy: object[];
}
let runCount = 0;
/**
* Starts the composition placed as `holder` from its first frame. One already playing plays on, unless
* `restart` – or a `Stop composition` this tick – asks to start over.
*/
export function playComposition(
holder: { composition?: CompositionPlayer },
restart = false,
startedBy?: object,
): void {
const playing = holder.composition;
if (playing && !playing.stopped && !restart) {
if (startedBy && !playing.startedBy.includes(startedBy)) playing.startedBy.push(startedBy);
return;
}
holder.composition = {
run: ++runCount,
time: 0,
shown: false,
stopped: false,
startedBy: startedBy ? [startedBy] : [],
};
}
export function stopComposition(holder: { composition?: CompositionPlayer }): void {
if (holder.composition) holder.composition.stopped = true;
}
/** The frame a player's time stands on; the small guard keeps a time of exactly `n / fps` on frame n. */
export function compositionFrameAt(comp: CompositionAsset, time: number): number {
return Math.floor(time * compositionFps(comp) + 1e-6);
}
/** A clip playing on an item, at `frame` of its animation – held on the last one once it is through. */
export interface ClipAt {
clip: CompositionClip;
animation: AnimationAsset;
frame: number;
}
/** An item alive at a frame of the composition. */
export interface LiveItem {
item: CompositionItem;
object: SceneObject;
/** The index of its track, 0 the first. */
track: number;
/** Seconds since the item began. */
time: number;
/** Seconds into what its videos play: `time` past the item's offset. */
videoTime: number;
/** How loud its videos are heard, 0 to 1: its fades at this frame. */
level: number;
/** In the order they started; a later one writes over an earlier one. */
clips: ClipAt[];
}
/**
* Everything alive at `frame`, in drawing order: the last track first, so the first lies in front –
* the order the items take below the composition, which decides between equal `z`. An item lives
* from its start up to, not including, its end; one on a track that is gone, or whose object is gone,
* does not. Items of a track that overlap both live, the later one in front. A clip plays on every
* item of its track it overlaps once it has started, and holds its last frame for as long as that
* item lasts.
*/
export function compositionAt(project: Project, comp: CompositionAsset, frame: number): LiveItem[] {
const fps = compositionFps(comp);
const tracks = new Map(comp.tracks.map((t, index) => [t.id, index]));
const clips = comp.clips
.map((clip, order) => ({ clip, order, animation: clipAnimation(project, clip) }))
.filter((c) => c.animation !== undefined && c.clip.start <= frame)
.sort((a, b) => a.clip.start - b.clip.start || a.order - b.order);
const live: { item: LiveItem; order: number }[] = [];
comp.items.forEach((item, order) => {
const track = tracks.get(item.track);
const object = itemObject(comp, item);
if (track === undefined || !object || frame < item.start || frame >= itemEnd(item)) return;
const time = (frame - item.start) / fps;
const playing: ClipAt[] = [];
for (const { clip, animation } of clips) {
const end = clip.start + clipLength(project, comp, clip);
if (clip.track !== item.track || clip.start >= itemEnd(item) || end <= item.start) continue;
const at = ((frame - clip.start) / fps) * fpsOf(animation!);
playing.push({ clip, animation: animation!, frame: Math.min(at, frameCount(animation!) - 1) });
}
const level = fadeLevel(item, frame - item.start);
live.push({
item: { item, object, track, time, videoTime: item.offset + time, level, clips: playing },
order,
});
});
live.sort(
(a, b) => b.item.track - a.item.track || a.item.item.start - b.item.item.start || a.order - b.order,
);
return live.map((l) => l.item);
}
/**
* The composition at `frame` as objects a view draws: a copy holding only what is alive, in drawing
* order, each with its clips' rows and its video time written over it. Every sprite of an item that
* wears a video and is not driven by hand is driven at the item's video time. Nothing here touches
* the document.
*/
export function compositionHolder(project: Project, comp: CompositionAsset, frame: number): CompositionAsset {
return holderOf(project, comp, compositionAt(project, comp, frame));
}
/** How far ahead of where a composition stands the items about to begin are written out, in seconds. */
const AHEAD_SECONDS = 1;
/**
* The items that begin within `seconds` after `frame` – once it loops, those at the start too when
* the end is that close – each written out as at its first frame, like `compositionHolder` does. A
* view opens their videos ahead of the cut: opened when its item began, a video had no picture yet,
* and the cut showed nothing until it loaded. One alive at `frame` is left out, it wears its own time.
*/
export function compositionAhead(
project: Project,
comp: CompositionAsset,
frame: number,
seconds = AHEAD_SECONDS,
): CompositionAsset {
const until = frame + Math.ceil(seconds * compositionFps(comp));
const length = compositionLength(project, comp);
const wrapped = compositionLoops(comp) && until >= length ? until - length : -1;
const coming = comp.items.filter(
(item) => (item.start > frame && item.start <= until) || item.start <= wrapped,
);
if (coming.length === 0) return holderOf(project, comp, []);
const alive = new Set(compositionAt(project, comp, frame).map((l) => l.item.id));
const live = coming
.filter((item) => !alive.has(item.id))
.flatMap((item) => compositionAt(project, comp, item.start).filter((l) => l.item.id === item.id));
return holderOf(project, comp, live);
}
function holderOf(project: Project, comp: CompositionAsset, live: LiveItem[]): CompositionAsset {
const holder: CompositionAsset = {
...comp,
props: { ...comp.props },
objects: live.map((l) => copyOf(l.object)),
};
const curves = curveLookup(project);
for (const item of live) {
driveVideos(project, holder, item);
for (const clip of item.clips) {
const at: ReadAt = { frame: clip.frame, reverse: false, curves };
for (const track of clip.animation.tracks) {
const id = rowTarget(project, holder, item.object.id, clip.animation, track);
if (id) writeObjectRow(project, holder, id, track, at);
}
}
}
return holder;
}
/**
* What a composition placed as `placementId` shows in the editor: its first frame, as objects below
* the placement with the ids its items take in a run. Stored nowhere – a placement is sealed, and
* its items are edited in the composition only.
*/
export function placedPicture(project: Project, comp: CompositionAsset, placementId: string): SceneObject[] {
return compositionHolder(project, comp, 0).objects.flatMap((o) =>
placedWithContent(project, o, contentId(placementId, o.id), placementId),
);
}
const record = (value: unknown): Record<string, unknown> =>
typeof value === 'object' && value !== null ? (value as Record<string, unknown>) : {};
const text = (value: unknown) => (typeof value === 'string' ? value : '');
const finite = (value: unknown) => (typeof value === 'number' && Number.isFinite(value) ? value : NaN);
const list = (raw: unknown): Record<string, unknown>[] => (Array.isArray(raw) ? raw.map(record) : []);
/** Held to what a track may be: ids no other has, a name, and at least one track. */
export function heldCompositionTracks(raw: unknown): CompositionTrack[] {
const tracks: CompositionTrack[] = [];
for (const t of list(raw)) {
const id = text(t.id);
if (!id || tracks.some((other) => other.id === id)) continue;
tracks.push({ id, name: text(t.name) || `Track ${tracks.length + 1}` });
}
return tracks.length > 0 ? tracks : [newCompositionTrack(1)];
}
/**
* Held to what an item may be: the id of an object it spawns and no other item does – an audio item
* an id of its own and what it plays –, a track of the composition, a start in whole frames from 0,
* at least a frame long, a finite offset, fades in whole frames above 0 and curves there are.
*/
export function heldCompositionItems(
raw: unknown,
tracks: readonly CompositionTrack[],
objects: readonly SceneObject[],
): CompositionItem[] {
const known = new Set(tracks.map((t) => t.id));
const placed = new Set(objects.map((o) => o.id));
const items: CompositionItem[] = [];
for (const i of list(raw)) {
const id = text(i.id);
const start = Math.round(finite(i.start));
const length = Math.round(finite(i.length));
const audio = heldAudio(i.audio);
if (!id || (audio ? placed.has(id) : !placed.has(id))) continue;
if (items.some((other) => other.id === id) || !known.has(text(i.track))) continue;
if (!(start >= 0) || !(length >= 1)) continue;
const offset = finite(i.offset);
const item: CompositionItem = {
id,
track: text(i.track),
start,
length,
offset: Number.isFinite(offset) ? offset : 0,
};
if (i.fitMedia === true) item.fitMedia = true;
if (audio) item.audio = audio;
const fadeIn = Math.round(finite(i.fadeIn));
const fadeOut = Math.round(finite(i.fadeOut));
if (fadeIn > 0) item.fadeIn = fadeIn;
if (fadeOut > 0) item.fadeOut = fadeOut;
if (isFadeCurve(i.curveIn)) item.curveIn = i.curveIn;
if (isFadeCurve(i.curveOut)) item.curveOut = i.curveOut;
items.push(item);
}
return items;
}
const AUDIO_KINDS: readonly CompositionAudioKind[] = ['sound', 'pattern', 'song'];
/** Held to what an audio item plays: a kind, an asset id kept even once the asset is gone, a volume. */
function heldAudio(raw: unknown): CompositionAudio | null {
if (typeof raw !== 'object' || raw === null) return null;
const a = raw as Record<string, unknown>;
const kind = AUDIO_KINDS.find((k) => k === a.kind);
if (!kind || !text(a.asset)) return null;
const volume = finite(a.volume);
return {
kind,
asset: text(a.asset),
volume: Number.isFinite(volume) ? Math.max(ITEM_MIN_DB, Math.min(SOUND_ITEM_MAX_DB, volume)) : 0,
bus: text(a.bus),
};
}
/** Held like an item; an animation that is gone is kept, as an animation keeps a root that is gone. */
export function heldCompositionClips(raw: unknown, tracks: readonly CompositionTrack[]): CompositionClip[] {
const known = new Set(tracks.map((t) => t.id));
const clips: CompositionClip[] = [];
for (const c of list(raw)) {
const id = text(c.id);
const start = Math.round(finite(c.start));
if (!id || clips.some((other) => other.id === id) || !known.has(text(c.track)) || !(start >= 0)) continue;
clips.push({ id, track: text(c.track), animation: text(c.animation), start });
}
return clips;
}
function copyOf(object: SceneObject): SceneObject {
return {
...object,
position: { ...object.position },
overrides: { ...object.overrides },
instanceEdits: structuredClone(object.instanceEdits),
};
}
function driveVideos(project: Project, holder: CompositionAsset, live: LiveItem): void {
const below = resolveObjects(project, holder).filter(
(o) => splitInstanceId(o.id)?.instanceId === live.object.id,
);
for (const object of [holder.objects.find((o) => o.id === live.object.id)!, ...below]) {
const asset = findAsset(project, object.assetId);
if (asset?.type !== 'sprite') continue;
const props = resolveSceneObjectProps(project, asset, object);
const own = object.id === live.object.id;
if (!videoWorn(props, project) || (!own && videoDrivenOf(props))) continue;
const ref = objectRef(project, holder, object.id);
ref?.setOverride(VIDEO_CLOCK_KEY, 'driven');
ref?.setOverride(VIDEO_TIME_KEY, live.videoTime);
if (live.level < 1) ref?.setOverride(VIDEO_LEVEL_KEY, videoLevelOf(props) * live.level);
}
}
/**
* The object of the holder a row of a clip aims at on the item `rootId`: the item itself, the object
* at the row's template path below it, or else the one its names lead to – the way `Play animation`
* finds a target on an object built from another container.
*/
function rowTarget(
project: Project,
holder: CompositionAsset,
rootId: string,
animation: AnimationAsset,
track: AnimationTrack,
): string | undefined {
if (track.target === '') return rootId;
const objects = resolveObjects(project, holder);
const byPath = instanceChildId(rootId, track.target);
if (objects.some((o) => o.id === byPath)) return byPath;
const names = targetNamePath(project, findAsset(project, animation.root), track.target);
let at: string | undefined = rootId;
for (const name of names ?? []) {
const found = objects.filter(
(o) => o.parentId === at && objectName(o, findAsset(project, o.assetId)) === name,
);
at = found.length === 1 ? found[0]!.id : undefined;
if (!at) return undefined;
}
return names ? at : undefined;
}
src/core/model/compositionEdit.ts
+431
View File
@@ -0,0 +1,431 @@
/**
* What the Composition tab does to items and clips (#417), as plain functions on lists so a test can
* check them. Items of a track may overlap – both live there, as on two tracks –, clips of a track
* do not: a clip's edit that would make them is held short, a dropped clip lands in the gap it falls
* into.
*/
import { videoLoopsOf } from '../properties/videos';
import type { CompositionAsset, CompositionAudioKind, CompositionClip, CompositionItem } from './Asset';
import {
audioRoundSeconds,
audioSeconds,
compositionFps,
DEFAULT_ITEM_SECONDS,
fittedLength,
itemEnd,
itemFades,
itemVideo,
} from './composition';
import { findAsset, type Project } from './Project';
import { findSound } from './sound';
import { findVideo, videoLengthOf } from './video';
/** A stretch of a track, in frames. */
export interface Span {
start: number;
length: number;
}
interface Placed {
id: string;
track: string;
start: number;
}
/**
* Where something `length` frames long lands when dropped at `at`: there, or right after whatever
* of `taken` covers `at`, and cut short where the next begins. Null where not a frame is free.
*/
export function droppedSpan(taken: readonly Span[], at: number, length: number): Span | null {
let start = Math.max(0, Math.round(at));
for (const span of [...taken].sort((a, b) => a.start - b.start))
if (span.start <= start && start < span.start + span.length) start = span.start + span.length;
const next = Math.min(...taken.filter((s) => s.start >= start).map((s) => s.start));
const room = Number.isFinite(next) ? next - start : Infinity;
return room >= 1 ? { start, length: Math.min(Math.max(1, length), room) } : null;
}
/** The stretches the clips of a track cover, all but `except`. */
export function clipSpans(
clips: readonly CompositionClip[],
track: string,
lengthOf: (clip: CompositionClip) => number,
except: ReadonlySet<string> = new Set(),
): Span[] {
return clips
.filter((c) => c.track === track && !except.has(c.id))
.map((c) => ({ start: c.start, length: Math.max(1, lengthOf(c)) }));
}
/** Whether no two of `spans` on the same track share a frame. */
function apart<T extends Placed>(list: readonly T[], lengthOf: (t: T) => number): boolean {
const byTrack = new Map<string, T[]>();
for (const t of list) byTrack.set(t.track, [...(byTrack.get(t.track) ?? []), t]);
for (const on of byTrack.values()) {
const sorted = [...on].sort((a, b) => a.start - b.start);
for (let i = 1; i < sorted.length; i++)
if (sorted[i]!.start < sorted[i - 1]!.start + Math.max(1, lengthOf(sorted[i - 1]!))) return false;
}
return true;
}
/**
* A move of the picked ones held so that none starts before 0 or leaves the tracks there are: what
* the pointer asks for, as far as it goes.
*/
export function clampedMove(
list: readonly Placed[],
picked: ReadonlySet<string>,
trackIds: readonly string[],
frames: number,
tracks: number,
): { frames: number; tracks: number } {
const chosen = list.filter((t) => picked.has(t.id));
if (chosen.length === 0) return { frames: 0, tracks: 0 };
const rows = chosen.map((t) => trackIds.indexOf(t.track));
return {
// `|| 0`: no -0 for one at frame 0.
frames: Math.max(frames, -Math.min(...chosen.map((t) => t.start))) || 0,
tracks: Math.max(-Math.min(...rows), Math.min(trackIds.length - 1 - Math.max(...rows), tracks)) || 0,
};
}
function moved<T extends Placed>(
list: readonly T[],
picked: ReadonlySet<string>,
trackIds: readonly string[],
frames: number,
tracks: number,
): T[] | null {
const out: T[] = [];
for (const t of list) {
if (!picked.has(t.id)) {
out.push(t);
continue;
}
const track = trackIds[trackIds.indexOf(t.track) + tracks];
if (track === undefined || t.start + frames < 0) return null;
out.push({ ...t, start: t.start + frames, track });
}
return out;
}
/** The picked items moved by `frames` and `tracks`; null where one would start before 0 or leave the tracks. */
export function movedItems(
items: readonly CompositionItem[],
picked: ReadonlySet<string>,
trackIds: readonly string[],
frames: number,
tracks: number,
): CompositionItem[] | null {
return moved(items, picked, trackIds, frames, tracks);
}
/**
* The clips that belong to the picked items: a clip belongs to the item lying on its track at its
* first frame, of two there the one in front – the later start, at the same start the later in the
* list. A clip that begins before an item and only reaches into it belongs to the one before. An
* audio item (#418) has nothing a clip could move, and so has no clips.
*/
export function carriedClips(
items: readonly CompositionItem[],
clips: readonly CompositionClip[],
picked: ReadonlySet<string>,
): Set<string> {
const carried = new Set<string>();
if (picked.size === 0) return carried;
for (const clip of clips) {
let owner: CompositionItem | undefined;
for (const item of items)
if (!item.audio && item.track === clip.track && item.start <= clip.start && clip.start < itemEnd(item))
if (!owner || item.start >= owner.start) owner = item;
if (owner && picked.has(owner.id)) carried.add(clip.id);
}
return carried;
}
/**
* The picked items moved, their clips with them unless `alone`; null where an item would start
* before 0 or leave the tracks, or a clip would come to lie over another.
*/
export function movedWithClips(
items: readonly CompositionItem[],
clips: readonly CompositionClip[],
picked: ReadonlySet<string>,
trackIds: readonly string[],
frames: number,
tracks: number,
lengthOf: (clip: CompositionClip) => number,
alone = false,
): { items: CompositionItem[]; clips: CompositionClip[] } | null {
const outItems = movedItems(items, picked, trackIds, frames, tracks);
const carried = alone ? new Set<string>() : carriedClips(items, clips, picked);
if (!outItems) return null;
if (carried.size === 0) return { items: outItems, clips: [...clips] };
const outClips = movedClips(clips, carried, trackIds, frames, tracks, lengthOf);
return outClips && { items: outItems, clips: outClips };
}
/**
* Where a move asked to go `wanted` lands when only places that `fit` will do: on the way back to
* `last`, on the track asked for, else on the one it was on; `last` itself when nothing fits.
*/
export function heldMove(
wanted: { frames: number; tracks: number },
last: { frames: number; tracks: number },
fits: (frames: number, tracks: number) => boolean,
): { frames: number; tracks: number } {
for (const on of [wanted.tracks, last.tracks]) {
const step = Math.sign(last.frames - wanted.frames);
for (let at = wanted.frames; ; at += step) {
if (fits(at, on)) return { frames: at, tracks: on };
if (at === last.frames) break;
}
}
return last;
}
/** The picked clips moved like items; null, too, where one would come to lie over another. */
export function movedClips(
clips: readonly CompositionClip[],
picked: ReadonlySet<string>,
trackIds: readonly string[],
frames: number,
tracks: number,
lengthOf: (clip: CompositionClip) => number,
): CompositionClip[] | null {
const out = moved(clips, picked, trackIds, frames, tracks);
return out && apart(out, lengthOf) ? out : null;
}
/**
* Copied clips placed as `placing` places what is pasted, then moved on frame by frame as long as
* one would lie over a clip of `clips` or over another of the copy – clips of a track never overlap.
*/
export function pastedClips(
copied: readonly CompositionClip[],
clips: readonly CompositionClip[],
ids: ReadonlyMap<string, string>,
at: number,
trackIds: readonly string[],
lengthOf: (clip: CompositionClip) => number,
row?: number,
): CompositionClip[] {
const place = placing(copied, at, trackIds, row);
const placed = copied.map((c) => place(c, ids));
const by = framesToRoom(placed, clips, lengthOf);
return placed.map((c) => ({ ...c, start: c.start + by }));
}
/**
* Copied items placed as `placing` places them, the copied clips that came with them moved by as
* much; then both moved on frame by frame as long as one of the clips would lie over a clip of
* `clips` – clips of a track never overlap.
*/
export function pastedItemsWithClips(
items: readonly CompositionItem[],
copied: readonly CompositionClip[],
clips: readonly CompositionClip[],
ids: ReadonlyMap<string, string>,
at: number,
trackIds: readonly string[],
lengthOf: (clip: CompositionClip) => number,
row?: number,
): { items: CompositionItem[]; clips: CompositionClip[] } {
const place = placing(items, at, trackIds, row);
const placedItems = items.map((i) => place(i, ids));
const placedClips = copied.map((c) => place(c, ids));
const by = framesToRoom(placedClips, clips, lengthOf);
return {
items: placedItems.map((i) => ({ ...i, start: i.start + by })),
clips: placedClips.map((c) => ({ ...c, start: c.start + by })),
};
}
/** The frames `placed` moves on until none of it lies over a clip of `clips` or over another of it. */
function framesToRoom(
placed: readonly CompositionClip[],
clips: readonly CompositionClip[],
lengthOf: (clip: CompositionClip) => number,
): number {
// Past the end of every clip there is room, so the search ends there at the latest.
const last = Math.max(0, ...clips.map((c) => c.start + Math.max(1, lengthOf(c))));
for (let by = 0; ; by++) {
const moved = placed.map((c) => ({ ...c, start: c.start + by }));
if (apart([...clips, ...moved], lengthOf) || moved.every((c) => c.start >= last)) return by;
}
}
/**
* How a paste moves what was copied, under the ids `ids` gives it: the earliest of `items` to `at`,
* on their tracks, or, given `row`, the topmost on that track and the others as far below as they
* were, as far as there are tracks – a song's paste. One whose track is gone counts as on the first.
*/
function placing(
items: readonly Placed[],
at: number,
trackIds: readonly string[],
row?: number,
): <T extends Placed>(t: T, ids: ReadonlyMap<string, string>) => T {
if (items.length === 0) return (t) => t;
const first = Math.min(...items.map((i) => i.start));
const index = new Map(trackIds.map((id, i) => [id, i]));
const rows = items.map((i) => index.get(i.track) ?? 0);
const top = Math.min(...rows);
const bottom = Math.max(...rows);
const shift = row === undefined ? 0 : Math.max(-top, Math.min(trackIds.length - 1 - bottom, row - top));
return (t, ids) => ({
...t,
id: ids.get(t.id) ?? t.id,
start: Math.max(0, Math.round(at)) + (t.start - first),
track: trackIds[(index.get(t.track) ?? 0) + shift] ?? t.track,
});
}
/** The item with its end edge dragged to make it `wanted` frames long, at least one; its length is its own from then on. */
export function resizedItem(item: CompositionItem, wanted: number): CompositionItem {
const { fitMedia: _, ...rest } = item;
return { ...rest, length: Math.max(1, Math.round(wanted)) };
}
/**
* The item with its start edge dragged to `wanted`: never onto its end or before 0. What it shows
* stays where it was in time – the video starts that much later or sooner –, and pulled out before
* the video's beginning the item shows its first picture until the video starts.
*/
export function trimmedItem(
item: CompositionItem,
wanted: number,
fps: number,
video: boolean,
): CompositionItem {
const start = Math.max(0, Math.min(Math.round(wanted), itemEnd(item) - 1));
const by = start - item.start;
const { fitMedia: _, ...rest } = item;
const offset = video ? Math.round((item.offset + by / fps) * 1e6) / 1e6 || 0 : item.offset;
return { ...rest, start, length: item.length - by, offset };
}
/**
* The item with the handle of its fade in or out dragged to `frame`: the fade from its edge to there,
* none where that is at or past the edge, no further than the other fade lets it. A fade of 0 is left
* out of the item.
*/
export function fadedItem(item: CompositionItem, end: 'in' | 'out', frame: number): CompositionItem {
const fades = itemFades(item);
const asked = end === 'in' ? frame - item.start : itemEnd(item) - frame;
const room = item.length - (end === 'in' ? fades.fadeOut : fades.fadeIn);
const frames = Math.max(0, Math.min(room, Math.round(asked)));
const key = end === 'in' ? 'fadeIn' : 'fadeOut';
const { [key]: _, ...rest } = item;
return frames > 0 ? { ...rest, [key]: frames } : rest;
}
/**
* Where an item's video lies in the composition, in frames: where it begins, how long its first pass
* is, whether it goes round, and how long every pass after the first is where that differs – a song
* goes round its loop only.
*/
export interface ItemMedia {
begins: number;
length: number;
loops: boolean;
round?: number;
}
/**
* The video of an item as it lies on its track, or what an audio item plays (#418) – a sound and a
* pattern go round, a song to the end of its loop and then round it –; null without one, or while
* its length is not known.
*/
export function itemMedia(project: Project, comp: CompositionAsset, item: CompositionItem): ItemMedia | null {
const video = item.audio ? null : itemVideo(project, comp, item);
const seconds = item.audio ? audioSeconds(project, item.audio) : video ? videoLengthOf(video) : 0;
if (!(seconds > 0)) return null;
const fps = compositionFps(comp);
const media = {
begins: item.start - item.offset * fps,
length: seconds * fps,
loops: item.audio ? true : videoLoopsOf(video!.asset.props),
};
const round = item.audio?.kind === 'song' ? audioRoundSeconds(project, item.audio) * fps : 0;
return round > 0 ? { ...media, round } : media;
}
/** The frames from `from` to `to` where the video begins or a pass of it ends. */
export function mediaEdges(media: ItemMedia, from: number, to: number): number[] {
const edges = media.begins >= from && media.begins <= to ? [media.begins] : [];
const round = media.round ?? media.length;
const first = media.begins + media.length;
for (let pass = Math.max(0, Math.ceil((from - first) / round)); pass === 0 || media.loops; pass++) {
const end = first + pass * round;
if (end > to) break;
if (end >= from) edges.push(end);
}
return edges;
}
/** Half a frame: closer than this an edge counts as on the video's. */
const ON_EDGE = 0.5;
/**
* What the timeline marks on an item wearing a video: where inside it the video begins or a pass
* ends (`seams`), and which of its ends cut into the video – starting after its beginning, ending
* before its end.
*/
export function mediaMarks(
item: CompositionItem,
media: ItemMedia,
): { seams: number[]; cutStart: boolean; cutEnd: boolean } {
return {
seams: mediaEdges(media, item.start + ON_EDGE, itemEnd(item) - ON_EDGE),
cutStart: item.start > media.begins + ON_EDGE,
cutEnd: itemEnd(item) < media.begins + media.length - ON_EDGE,
};
}
/** A frame pulled onto the nearest edge of the video within `reach` frames; else as it is. */
export function snappedToMedia(frame: number, media: ItemMedia | null, reach: number): number {
const edges = media ? mediaEdges(media, frame - reach, frame + reach) : [];
return edges.reduce(
(best, edge) => (Math.abs(edge - frame) < Math.abs(best - frame) ? edge : best),
edges[0] ?? frame,
);
}
/** What an asset is laid on a track as when it is to be heard (#418): a sound, a clip of one, a pattern or a song. */
export function audioKindOf(project: Project, assetId: string): CompositionAudioKind | null {
const asset = findAsset(project, assetId);
if (asset?.type === 'pattern' || asset?.type === 'song') return asset.type;
return findSound(project, assetId) ? 'sound' : null;
}
/**
* How long an item of the asset `id` names is when it is dropped in: as long as the video or clip of
* one it is (#419, #425) or its sprite wears, or what it plays as an audio item (#418); else 3 s.
*/
export function droppedLength(project: Project, comp: CompositionAsset, id: string): number {
const kind = audioKindOf(project, id);
if (kind)
return fittedLength(project, comp, {
id: '',
track: '',
start: 0,
length: 1,
offset: 0,
audio: { kind, asset: id, volume: 0, bus: '' },
});
const asset = findAsset(project, id);
const texture = asset?.type === 'sprite' ? asset.props.texture : id;
const video = typeof texture === 'string' ? findVideo(project, texture) : null;
const fps = compositionFps(comp);
const seconds = video ? videoLengthOf(video) : 0;
return seconds > 0 ? Math.max(1, Math.round(seconds * fps)) : Math.round(DEFAULT_ITEM_SECONDS * fps);
}
/** What an item wears that its length follows: the video or clip's id, what an audio item plays, '' for none. */
export function mediaKey(project: Project, comp: CompositionAsset, item: CompositionItem): string {
if (item.audio) return item.audio.asset;
const video = itemVideo(project, comp, item);
return video ? (video.clip?.id ?? video.asset.id) : '';
}
src/core/model/instance.ts
+34
View File
@@ -436,6 +436,40 @@ export function materializeInstance(
}));
}
/**
* The id the object at `path` below `ownerId` has – one level deeper when the owner is content
* itself, so the content of `q~a` is `q~a/b`, the id a template path through nested containers gives.
*/
export function contentId(ownerId: string, path: string): string {
const split = splitInstanceId(ownerId);
return split
? instanceChildId(split.instanceId, `${split.path}${PATH_SEP}${path}`)
: instanceChildId(ownerId, path);
}
/**
* `object` as `id` below `parentId`, followed by its container content with the ids below that one –
* how a composition puts its objects into a scene (#417). Copies, free to be stored and written to.
*/
export function placedWithContent(
project: Project,
object: SceneObject,
id: string,
parentId: string,
): SceneObject[] {
const rename = (at: string) => (at === object.id ? id : contentId(id, at.slice(object.id.length + 1)));
const { groupId: _, ...top } = object;
return resolveObjects(project, { objects: [{ ...top, parentId: '' }] }).map((o) => ({
...o,
id: rename(o.id),
parentId: o.id === object.id ? parentId : rename(o.parentId),
position: { ...o.position },
editorOffset: { ...o.editorOffset },
overrides: { ...o.overrides },
instanceEdits: {},
}));
}
/**
* Spells out every live instance of the holder. The runtime needs it: entities are built from
* stored objects, and a behavior may destroy or move one of them – which a resolved copy could
src/core/model/placement.ts
+2
View File
@@ -14,5 +14,7 @@ export function placementRejection(project: Project, holder: ObjectHolder, asset
return `A ${ASSET_TYPE_LABELS[asset.type].toLowerCase()} cannot be placed into a scene`;
if (asset.type === 'container' && containerReaches(project, asset, holder.id))
return `${asset.name} cannot contain ${holder.name}: that nests forever`;
if (asset.type === 'composition' && holder.type === 'composition')
return 'A composition cannot be placed into a composition';
return '';
}
src/core/model/projectFiles.ts
+1 -1
View File
@@ -64,7 +64,7 @@ const UNNAMING: ReadonlySet<AssetType> = new Set(['sound', 'texture', 'pixelart'
* Every text the project holds outside the assets that cannot name a sound. Where a sound is played
* whole – a sound field, an item of a song, a Sample, a Reverb's impulse – its id is one of them; a
* clip has an id of its own. Looked at as data, not field by field, so no place is missed: one that
* only looks like a reference loads the file as before.
* only looks like a reference just loads a file nothing plays.
*/
function namedStrings(project: Project): Set<string> {
const seen = new Set<string>();
src/core/model/sceneVisibility.ts
+1
View File
@@ -17,6 +17,7 @@ export const VISIBILITY_TYPES: readonly PlaceableAssetType[] = [
'camera',
'clip',
'collider',
'composition',
'container',
'joint',
'particles',
src/core/model/sound.ts
+28 -13
View File
@@ -1,6 +1,6 @@
import { nextFreeName, nextNumberedName } from '../commands/naming';
import { newId } from '../ids';
import type { Asset, SoundAsset, SoundClip } from './Asset';
import type { Asset, SoundAsset, SoundClip, VideoAsset } from './Asset';
import { findAsset, type Project } from './Project';
import { storedFilePattern } from './storedFile';
@@ -13,10 +13,18 @@ export function isSoundFile(name: string): boolean {
const CLIP_SEPARATOR = ' / ';
const DEFAULT_CLIP_NAME = 'Clip';
export function clipsOf(asset: SoundAsset): readonly SoundClip[] {
/** The clips of a sound (#357) or a video (#402). */
export function clipsOf<C>(asset: { clips?: C[] }): readonly C[] {
return asset.clips ?? [];
}
/** What has clips: a sound and a video, each a file with named stretches of it. */
export type ClipOwner = SoundAsset | VideoAsset;
export function hasClips(asset: Asset): asset is ClipOwner {
return asset.type === 'sound' || asset.type === 'video';
}
/** What a sound field names (#357): a sound, the whole file, or one of its clips. */
export interface SoundRef {
asset: SoundAsset;
@@ -39,10 +47,12 @@ export function findSound(project: Project, id: string): SoundRef | null {
return null;
}
/** The asset among `assets` an id names: the asset itself, or the sound one of whose clips it is. */
/** The asset among `assets` an id names: the asset itself, or the one one of whose clips it is. */
export function assetNamedBy(assets: readonly Asset[], id: string): Asset | undefined {
if (id === '') return undefined;
return assets.find((a) => a.id === id || (a.type === 'sound' && clipsOf(a).some((c) => c.id === id)));
return assets.find(
(a) => a.id === id || (hasClips(a) && clipsOf<{ id: string }>(a).some((c) => c.id === id)),
);
}
/**
@@ -50,8 +60,8 @@ export function assetNamedBy(assets: readonly Asset[], id: string): Asset | unde
* each old id with its new one in `ids` so the places that named a clip can follow.
*/
export function renewClipIds(copy: Asset, ids: Map<string, string>): void {
if (copy.type !== 'sound') return;
for (const clip of clipsOf(copy)) {
if (!hasClips(copy)) return;
for (const clip of clipsOf<{ id: string }>(copy)) {
const id = newId();
ids.set(clip.id, id);
clip.id = id;
@@ -60,21 +70,26 @@ export function renewClipIds(copy: Asset, ids: Map<string, string>): void {
/** "Kick" for the sound, "Kick / Clip 1" for a clip of it. */
export function soundRefName(ref: SoundRef, soundName = ref.asset.name): string {
return ref.clip ? `${soundName}${CLIP_SEPARATOR}${ref.clip.name}` : soundName;
return ref.clip ? clipRefName(soundName, ref.clip.name) : soundName;
}
/** "Clip 1", 2, 3 …: the first number none of the sound's clips has. */
export function nextClipName(asset: SoundAsset): string {
/** "Kick / Clip 1": a clip of a sound or a video by name. */
export function clipRefName(assetName: string, clipName: string): string {
return `${assetName}${CLIP_SEPARATOR}${clipName}`;
}
/** "Clip 1", 2, 3 …: the first number none of the asset's clips has. */
export function nextClipName(asset: ClipOwner): string {
return nextNumberedName(
clipsOf(asset).map((c) => c.name),
clipsOf<{ name: string }>(asset).map((c) => c.name),
DEFAULT_CLIP_NAME,
);
}
/** A name free among the sound's clips: `name`, else "name 2", "name 3" … */
export function uniqueClipName(asset: SoundAsset, name: string, ignoreId = ''): string {
/** A name free among the asset's clips: `name`, else "name 2", "name 3" … */
export function uniqueClipName(asset: ClipOwner, name: string, ignoreId = ''): string {
return nextFreeName(
clipsOf(asset)
clipsOf<{ id: string; name: string }>(asset)
.filter((c) => c.id !== ignoreId)
.map((c) => c.name),
name,
src/core/model/texture.ts
+34 -3
View File
@@ -4,6 +4,17 @@ import { findPixelFrame, pixelFrameRect } from './pixelart';
import { type PropertyBag, textOf, type Vec2 } from './Properties';
import { findAsset, type Project } from './Project';
import { storedFilePattern } from './storedFile';
import { videoBusOf } from '../properties/videos';
import {
findVideo,
isVideoFile,
isVideoKey,
routedVideoKey,
videoKey,
videoMixBus,
videoPlayOf,
type VideoRef,
} from './video';
const FILE_PATTERN = storedFilePattern('png|jpg|webp|gif');
@@ -92,16 +103,23 @@ export function textureRefIds(asset: TextureAsset): string[] {
/** What a sprite wearing `props.texture` shows. */
export function resolveTexture(project: Project, props: PropertyBag): TextureRef | null {
return resolvePicture(project, textOf(props.texture ?? ''));
const ref = resolvePicture(project, textOf(props.texture ?? ''));
// A video heard on a bus the sprite picked plays in a player of its own (#418).
const bus = ref && isVideoKey(ref.file) ? videoMixBus(project, videoBusOf(props)) : null;
return bus ? { ...ref!, file: routedVideoKey(ref!.file, bus) } : ref;
}
/** A texture, a frame or a generating asset; any other asset's id is no picture. */
/** A texture, a frame, a video or a clip of one, or a generating asset; any other asset's id is no picture. */
export function resolvePicture(project: Project, id: string): TextureRef | null {
if (!id) return null;
const asset = findAsset(project, id);
if (asset) return asset.type === 'texture' ? refFor(asset, null) : generatedRef(asset, project);
if (asset?.type === 'texture') return refFor(asset, null);
if (asset?.type === 'video') return videoRef({ asset, clip: null });
if (asset) return generatedRef(asset, project);
const found = findFrame(project, id);
if (found) return refFor(found.asset, found.frame);
const video = findVideo(project, id);
if (video) return videoRef(video);
// A frame of pixel art (#130) is a cut of its generated sheet, as a texture frame is of its image.
const pixel = findPixelFrame(project, id);
return pixel ? generatedRef(pixel.asset, project, pixelFrameRect(pixel.asset, pixel.index)) : null;
@@ -128,6 +146,19 @@ function generatedRef(asset: Asset, project: Project, rect: FrameRect | null = n
};
}
/** A playing video (#402) reads like a texture of its size; the key says what plays, see `videoKey`. */
function videoRef(ref: VideoRef): TextureRef | null {
if (!isVideoFile(ref.asset.file)) return null;
return {
file: videoKey(videoPlayOf(ref)),
smooth: true,
rect: null,
sourceWidth: ref.asset.width,
sourceHeight: ref.asset.height,
slice: null,
};
}
function refFor(asset: TextureAsset, frame: TextureFrame | null): TextureRef | null {
if (!isTextureFile(asset.file)) return null;
return {
src/core/model/tilemap.ts
+222 -20
View File
@@ -13,7 +13,7 @@ import { type Neighbours, type TileQuadrants, tileQuadrants } from './autotile';
import { type TileKind, tileKind } from './tile';
import type { PropertyBag } from './Properties';
import { findAssetOf, type Project } from './Project';
import { cellsTileIds, cellsVersion, KeyedMemo } from './tilemapMemo';
import { cellsChangedSince, cellsTileIds, cellsVersion, identityToken, KeyedMemo } from './tilemapMemo';
export const TILEMAP_COLUMNS_KEY = 'columns';
export const TILEMAP_ROWS_KEY = 'rows';
@@ -145,33 +145,105 @@ function joinedBy(same: (dx: number, dy: number) => boolean): Neighbours {
export interface LayerCells {
columns: number;
rows: number;
list: readonly { x: number; y: number; tileId: string }[];
readonly list: readonly { x: number; y: number; tileId: string }[];
/** Tile id by `y * columns + x`. */
at: ReadonlyMap<number, string>;
}
const indexed = new WeakMap<Record<string, string>, LayerCells & { version: number }>();
/**
* Patched in place when cells are painted (#348), so it holds for the cells as they are now: whoever
* asks works from it at once and keeps it no longer.
*/
class CellIndex implements LayerCells {
readonly at = new Map<number, string>();
private ordered: LayerCells['list'] | null = null;
constructor(
readonly columns: number,
readonly rows: number,
public version: number,
) {}
get list(): LayerCells['list'] {
const { at, columns } = this;
this.ordered ??= [...at.keys()]
.sort((a, b) => a - b)
.map((i) => ({ x: i % columns, y: Math.floor(i / columns), tileId: at.get(i)! }));
return this.ordered;
}
read(key: string, tileId: string | undefined): void {
const cell = parseCellKey(key);
if (!cell || cell.x < 0 || cell.y < 0 || cell.x >= this.columns || cell.y >= this.rows) return;
const index = cell.y * this.columns + cell.x;
if (tileId) this.at.set(index, tileId);
else this.at.delete(index);
this.ordered = null;
}
}
const indexed = new WeakMap<Record<string, string>, CellIndex>();
export function layerCells(props: PropertyBag, layer: TilemapLayer): LayerCells {
const columns = tilemapColumns(props);
const rows = tilemapRows(props);
const version = cellsVersion(layer.cells);
const held = indexed.get(layer.cells);
if (held && held.version === version && held.columns === columns && held.rows === rows) return held;
const at = new Map<number, string>();
for (const [key, tileId] of Object.entries(layer.cells)) {
const cell = parseCellKey(key);
if (!tileId || !cell || cell.x < 0 || cell.y < 0 || cell.x >= columns || cell.y >= rows) continue;
at.set(cell.y * columns + cell.x, tileId);
if (held && held.columns === columns && held.rows === rows) {
if (held.version === version) return held;
const changed = cellsChangedSince(layer.cells, held.version);
if (changed) {
for (const key of changed) held.read(key, layer.cells[key]);
held.version = version;
return held;
}
}
const list = [...at.keys()]
.sort((a, b) => a - b)
.map((i) => ({ x: i % columns, y: Math.floor(i / columns), tileId: at.get(i)! }));
const fresh = { columns, rows, list, at, version };
const fresh = new CellIndex(columns, rows, version);
for (const [key, tileId] of Object.entries(layer.cells)) fresh.read(key, tileId);
indexed.set(layer.cells, fresh);
return fresh;
}
/**
* The cells that may show or block otherwise once `keys` are painted: those cells and their eight
* neighbours inside the grid, by `y * columns + x`, ascending. Further off nothing changes – a
* cell's quarters and its blocking shape follow its neighbours alone.
*/
export function cellsAround(keys: readonly string[], columns: number, rows: number): number[] {
const near = new Set<number>();
for (const key of keys) {
const cell = parseCellKey(key);
if (!cell) continue;
for (let y = Math.max(0, cell.y - 1); y <= Math.min(rows - 1, cell.y + 1); y++)
for (let x = Math.max(0, cell.x - 1); x <= Math.min(columns - 1, cell.x + 1); x++)
near.add(y * columns + x);
}
return [...near].sort((a, b) => a - b);
}
/**
* A list in cell order with the entries at `indices` (ascending) worked out again: `make` gives a
* cell's new entries, none or several, from the ones it had. A new list – the old one may still be
* held by identity.
*/
export function patchedInOrder<T>(
list: readonly T[],
indexOf: (entry: T) => number,
indices: readonly number[],
make: (index: number, before: readonly T[]) => readonly T[],
): T[] {
const out: T[] = [];
let i = 0;
for (const index of indices) {
while (i < list.length && indexOf(list[i]!) < index) out.push(list[i++]!);
const start = i;
while (i < list.length && indexOf(list[i]!) === index) i++;
for (const entry of make(index, list.slice(start, i))) out.push(entry);
}
while (i < list.length) out.push(list[i++]!);
return out;
}
/** `neighboursOf`, read from the index. */
export function neighboursIn(cells: LayerCells, x: number, y: number, tileId: string): Neighbours {
const { columns, rows, at } = cells;
@@ -202,16 +274,36 @@ export function layerTiles(project: Project, props: PropertyBag, layer: TilemapL
const tile: TileAsset | undefined = findAssetOf(project, id, 'tile');
if (tile) kinds.set(id, tileKind(tile));
}
const key = [
const kept = [
tilemapColumns(props),
tilemapRows(props),
cellsVersion(layer.cells),
[...kinds].map(([id, kind]) => `${id}:${kind}`).join(','),
].join('|');
return drawn.get(key, () => workOutLayerTiles(props, layer, kinds));
const version = cellsVersion(layer.cells);
return drawn.get(`${kept}|${version}`, () => {
// The last list for these cells and tiles, patched where they were painted since (#348).
const last = lastDrawn.get(`${kept}|${identityToken(layer.cells)}`, () => ({ version: 0, tiles: null }));
const changed = last.tiles && cellsChangedSince(layer.cells, last.version);
const tiles = changed
? patchedLayerTiles(props, layer, kinds, last.tiles!, changed)
: workOutLayerTiles(props, layer, kinds);
last.version = version;
last.tiles = tiles;
return tiles;
});
}
const drawn = new KeyedMemo<readonly TileDraw[]>();
const lastDrawn = new KeyedMemo<{ version: number; tiles: readonly TileDraw[] | null }>();
function tileDrawAt(cells: LayerCells, kinds: Map<string, TileKind>, index: number): TileDraw | null {
const tileId = cells.at.get(index);
const kind = tileId && kinds.get(tileId);
if (!kind) return null;
const x = index % cells.columns;
const y = Math.floor(index / cells.columns);
return { x, y, tileId, quadrants: tileQuadrants(kind, neighboursIn(cells, x, y, tileId)) };
}
function workOutLayerTiles(
props: PropertyBag,
@@ -220,14 +312,124 @@ function workOutLayerTiles(
): TileDraw[] {
const cells = layerCells(props, layer);
const tiles: TileDraw[] = [];
for (const { x, y, tileId } of cells.list) {
const kind = kinds.get(tileId);
if (!kind) continue;
tiles.push({ x, y, tileId, quadrants: tileQuadrants(kind, neighboursIn(cells, x, y, tileId)) });
for (const { x, y } of cells.list) {
const drawn = tileDrawAt(cells, kinds, y * cells.columns + x);
if (drawn) tiles.push(drawn);
}
return tiles;
}
/**
* A cell around the painted ones that still looks the same keeps its entry, so only what is drawn
* otherwise counts as changed (#424). Nothing drawn otherwise hands back the very same list.
*/
function patchedLayerTiles(
props: PropertyBag,
layer: TilemapLayer,
kinds: Map<string, TileKind>,
tiles: readonly TileDraw[],
changed: readonly string[],
): readonly TileDraw[] {
const cells = layerCells(props, layer);
const removed: TileDraw[] = [];
const added: TileDraw[] = [];
const patched = patchedInOrder(
tiles,
(tile) => tile.y * cells.columns + tile.x,
cellsAround(changed, cells.columns, cells.rows),
(index, [held]) => {
const drawn = tileDrawAt(cells, kinds, index);
if (held && drawn && sameTileDraw(held, drawn)) return [held];
if (held) removed.push(held);
if (drawn) added.push(drawn);
return drawn ? [drawn] : [];
},
);
if (removed.length === 0 && added.length === 0) return tiles;
noteTilesPatch(patched, { from: tiles, removed, added });
return patched;
}
function sameTileDraw(a: TileDraw, b: TileDraw): boolean {
return (
a.x === b.x &&
a.y === b.y &&
a.tileId === b.tileId &&
a.quadrants.every((q, i) => q.x === b.quadrants[i]!.x && q.y === b.quadrants[i]!.y)
);
}
/** How a list of tiles came out of the one before it (#424): the entries gone and the ones new. */
export interface TilesPatch {
from: readonly TileDraw[];
removed: readonly TileDraw[];
added: readonly TileDraw[];
}
const patches = new WeakMap<readonly TileDraw[], TilesPatch>();
/** Null for a list worked out whole, or one whose patch was passed on to a newer list. */
export function tilesPatch(tiles: readonly TileDraw[]): TilesPatch | null {
return patches.get(tiles) ?? null;
}
/** One step only: a chain of patched lists would otherwise keep every list before alive. */
function noteTilesPatch(tiles: readonly TileDraw[], patch: TilesPatch): void {
patches.delete(patch.from);
patches.set(tiles, patch);
}
/**
* `groups` of the list `patch` came from, grouped for the list it made: a group no changed entry
* falls into stays the very same array, the others are patched in cell order and note their own
* patch. In the order `order` gives, by default the one a fresh grouping has – by first cell.
*/
export function patchedGroups<K>(
groups: ReadonlyMap<K, readonly TileDraw[]>,
patch: TilesPatch,
keyOf: (tile: TileDraw) => K,
order: (a: [K, readonly TileDraw[]], b: [K, readonly TileDraw[]]) => number = ([, a], [, b]) =>
byCell(a[0]!, b[0]!),
): Map<K, readonly TileDraw[]> {
const touched = new Map<K, { removed: TileDraw[]; added: TileDraw[] }>();
const changeOf = (key: K) => {
let change = touched.get(key);
if (!change) touched.set(key, (change = { removed: [], added: [] }));
return change;
};
for (const tile of patch.removed) changeOf(keyOf(tile)).removed.push(tile);
for (const tile of patch.added) changeOf(keyOf(tile)).added.push(tile);
const out = new Map(groups);
for (const [key, { removed, added }] of touched) {
const from = groups.get(key) ?? [];
const gone = new Set(removed);
const merged = mergedInCellOrder(
from.filter((tile) => !gone.has(tile)),
[...added].sort(byCell),
);
if (merged.length === 0) out.delete(key);
else {
out.set(key, merged);
noteTilesPatch(merged, { from, removed, added });
}
}
return new Map([...out].sort(order));
}
function byCell(a: TileDraw, b: TileDraw): number {
return a.y - b.y || a.x - b.x;
}
function mergedInCellOrder(a: readonly TileDraw[], b: readonly TileDraw[]): TileDraw[] {
const out: TileDraw[] = [];
let i = 0;
let j = 0;
while (i < a.length && j < b.length) out.push(byCell(a[i]!, b[j]!) <= 0 ? a[i++]! : b[j++]!);
while (i < a.length) out.push(a[i++]!);
while (j < b.length) out.push(b[j++]!);
return out;
}
/**
* The layer names of every tilemap in the project, each once, in asset and layer order. A node
* picks a layer by name before it knows which map it will be given (#135), so it is offered them all.
src/core/model/tilemapCollision.ts
+214 -60
View File
@@ -45,14 +45,17 @@ import {
} from './tile';
import {
cellOrigin,
cellsAround,
layerCells,
type LayerCells,
neighboursIn,
patchedInOrder,
tilemapColumns,
tilemapRows,
tilemapSize,
tilemapTileSize,
} from './tilemap';
import { cellsTileIds, cellsVersion, identityToken, KeyedMemo } from './tilemapMemo';
import { cellsChangedSince, cellsTileIds, cellsVersion, identityToken, KeyedMemo } from './tilemapMemo';
import { type ColliderShape, oneWayAngle, type ShapeBounds } from '../properties/collider';
import { colliderMaterial } from '../properties/rigidbody';
@@ -258,7 +261,7 @@ function picks(tile: TileAsset, pick: TilePick): boolean {
return isTileSolid(tile) && passesFilter(tile, pick.oneWay, pick.material);
}
/** The solid tiles painted on the map, each once, in the order the layers first name them. */
/** The solid tiles painted on the map, each once: layer by layer, by id within one. */
function solidTiles(project: Project, asset: TilemapAsset): TileAsset[] {
const ids = new Set(asset.layers.flatMap((layer) => cellsTileIds(layer.cells)));
return [...ids].flatMap((id) => {
@@ -314,61 +317,123 @@ function blockingCells(
: null,
);
}
const key = [
const kept = [
tilemapColumns(props),
tilemapRows(props),
'trigger' in pick
? `trigger:${pick.trigger ?? ''}`
: `${pick.oneWay}|${pick.material ? `${pick.material.friction}/${pick.material.bounciness}` : ''}`,
asset.layers.map((layer) => cellsVersion(layer.cells)).join(','),
[...tiles]
.map(([id, t]) => (t ? `${id}:${t.scale}:${t.shape ? identityToken(t.shape) : 'box'}` : id))
.join(','),
].join('|');
return blocking.get(key, () => workOutBlockingCells(props, asset, tiles));
const versions = asset.layers.map((layer) => cellsVersion(layer.cells));
return blocking.get(`${kept}|${versions.join(',')}`, () => {
// The last cells for these layers and tiles, patched where they were painted since (#348).
const layersKey = asset.layers.map((layer) => identityToken(layer.cells)).join(',');
const last = lastBlocking.get(`${kept}|${layersKey}`, () => ({ versions: [], cells: null }));
const changed = last.cells
? asset.layers.map((layer, i) => cellsChangedSince(layer.cells, last.versions[i]!))
: [];
const cells =
last.cells && changed.every((keys) => keys !== null)
? patchedBlockingCells(props, asset, tiles, last.cells, changed.flat())
: workOutBlockingCells(props, asset, tiles);
last.versions = versions;
last.cells = cells;
return cells;
});
}
const lastBlocking = new KeyedMemo<{ versions: number[]; cells: readonly CellBlock[] | null }>();
/** What blocks in one cell, from what each layer holds there – null when nothing does. */
function blockAt(
layers: readonly LayerCells[],
tiles: Map<string, BlockingTile>,
index: number,
): CellBlock | null {
let cell: CellBlock | null = null;
for (const cells of layers) {
const tileId = cells.at.get(index);
const tile = tileId && tiles.get(tileId);
if (!tile) continue;
cell ??= { x: index % cells.columns, y: Math.floor(index / cells.columns), box: null, polygons: [] };
const { shape, scale } = tile;
if (!shape) {
cell.box = Math.max(cell.box ?? 0, scale);
} else if (shape.kind === 'plain') {
// The collision scale draws the outline in or out about the cell's centre (#140).
const at = 0.5 - scale / 2;
for (const outline of shape.outlines)
cell.polygons.push(placed(outline, cell.x + at, cell.y + at, scale));
} else {
const around = neighboursIn(cells, cell.x, cell.y, tileId);
const quarters = tileQuadrants(shape.tileKind, around).map(
(q) => shape.quarters.get(`${q.x},${q.y}`) ?? [],
);
if (quarters.every((q) => q === 'full')) {
cell.box = Math.max(cell.box ?? 0, 1);
continue;
}
for (const [corner, quarter] of quarters.entries()) {
const x = cell.x + (corner % 2) / 2;
const y = cell.y + Math.floor(corner / 2) / 2;
for (const outline of quarter === 'full' ? [SQUARE] : quarter)
cell.polygons.push(placed(outline, x, y, 0.5));
}
}
}
return cell;
}
/** In cell order, whichever layers block there. */
function workOutBlockingCells(
props: PropertyBag,
asset: TilemapAsset,
tiles: Map<string, BlockingTile>,
): CellBlock[] {
const found = new Map<number, CellBlock>();
for (const layer of asset.layers) {
const cells = layerCells(props, layer);
for (const { x, y, tileId } of cells.list) {
const tile = tiles.get(tileId);
if (!tile) continue;
const index = y * cells.columns + x;
let cell = found.get(index);
if (!cell) found.set(index, (cell = { x, y, box: null, polygons: [] }));
const { shape, scale } = tile;
if (!shape) {
cell.box = Math.max(cell.box ?? 0, scale);
} else if (shape.kind === 'plain') {
// The collision scale draws the outline in or out about the cell's centre (#140).
const at = 0.5 - scale / 2;
for (const outline of shape.outlines)
cell.polygons.push(placed(outline, cell.x + at, cell.y + at, scale));
} else {
const around = neighboursIn(cells, cell.x, cell.y, tileId);
const quarters = tileQuadrants(shape.tileKind, around).map(
(q) => shape.quarters.get(`${q.x},${q.y}`) ?? [],
);
if (quarters.every((q) => q === 'full')) {
cell.box = Math.max(cell.box ?? 0, 1);
continue;
}
for (const [corner, quarter] of quarters.entries()) {
const x = cell.x + (corner % 2) / 2;
const y = cell.y + Math.floor(corner / 2) / 2;
for (const outline of quarter === 'full' ? [SQUARE] : quarter)
cell.polygons.push(placed(outline, x, y, 0.5));
}
}
}
const layers = asset.layers.map((layer) => layerCells(props, layer));
const indices = new Set<number>();
for (const cells of layers) for (const index of cells.at.keys()) indices.add(index);
const found: CellBlock[] = [];
for (const index of [...indices].sort((a, b) => a - b)) {
const cell = blockAt(layers, tiles, index);
if (cell) found.push(cell);
}
return [...found.values()];
return found;
}
function patchedBlockingCells(
props: PropertyBag,
asset: TilemapAsset,
tiles: Map<string, BlockingTile>,
blocks: readonly CellBlock[],
changed: readonly string[],
): CellBlock[] {
const layers = asset.layers.map((layer) => layerCells(props, layer));
const columns = tilemapColumns(props);
const indices = cellsAround(changed, columns, tilemapRows(props));
const made = new Map<number, CellBlock>();
const patched = patchedInOrder(
blocks,
(cell) => cell.y * columns + cell.x,
indices,
(index) => {
const cell = blockAt(layers, tiles, index);
if (!cell) return [];
made.set(index, cell);
return [cell];
},
);
// A block worked out again may block otherwise, which cuts its neighbours' pieces otherwise too.
const rows = new Set<number>();
for (const index of indices) {
const y = Math.floor(index / columns);
for (let near = y - 1; near <= y + 1; near++) rows.add(near);
}
patchOf.set(patched, { from: blocks, rows: [...rows], indices, columns, made });
return patched;
}
/**
@@ -407,18 +472,33 @@ export function solidPolygons(
return polygonCells(blockingCells(project, props, asset, { oneWay, material }));
}
/**
* The blocks come in cell order, so the polygons do too: top-down, then left to right. A block a
* patch kept is the same object, and so are its polygons (#348).
*/
function polygonCells(blocks: readonly CellBlock[]): readonly CellPolygon[] {
return derived(polygonsOf, blocks, (cells) => {
const polygons: CellPolygon[] = [];
for (const cell of cells) {
if (cell.box !== null) continue;
for (const { outline, pieces } of cell.polygons)
polygons.push({ x: cell.x, y: cell.y, outline, pieces });
}
return polygons.sort((a, b) => a.y - b.y || a.x - b.x);
const patch = patchOf.get(cells);
const before = patch && polygonsOf.get(patch.from);
if (before)
return patchedInOrder(
before,
(polygon) => polygon.y * patch.columns + polygon.x,
patch.indices,
(index) => {
const cell = patch.made.get(index);
return cell ? polygonsOfBlock(cell) : [];
},
);
return cells.flatMap(polygonsOfBlock);
});
}
function polygonsOfBlock(cell: CellBlock): readonly CellPolygon[] {
if (cell.box !== null) return [];
return cell.polygons.map(({ outline, pieces }) => ({ x: cell.x, y: cell.y, outline, pieces }));
}
/**
* The same cells hand out the same rectangles and polygons, the very same arrays – so whoever builds
* on them can tell by identity that nothing changed (the physics does, every tick).
@@ -433,7 +513,6 @@ function derived<T>(
}
const polygonsOf = new WeakMap<readonly CellBlock[], readonly CellPolygon[]>();
const rectsOf = new WeakMap<readonly CellBlock[], readonly CellRect[]>();
/** A rectangle by its four sides, which is what the merge compares. */
interface Piece {
@@ -496,14 +575,17 @@ export function mergeCells(cells: readonly SolidCell[]): CellRect[] {
const unique = new Map<string, SolidCell>();
for (const cell of cells) unique.set(`${cell.x},${cell.y}`, cell);
const blocks = (x: number, y: number) => unique.has(`${x},${y}`);
return stacked(joinedStrips([...unique.values()].flatMap((cell) => cellPieces(cell, blocks))));
}
/** Pieces on the same strip – same top and bottom – that touch or overlap, joined into runs. */
function joinedStrips(pieces: readonly Piece[]): Piece[] {
const strips = new Map<string, Piece[]>();
for (const cell of unique.values()) {
for (const piece of cellPieces(cell, blocks)) {
const key = `${piece.y0},${piece.y1}`;
const strip = strips.get(key);
if (strip) strip.push(piece);
else strips.set(key, [piece]);
}
for (const piece of pieces) {
const key = `${piece.y0},${piece.y1}`;
const strip = strips.get(key);
if (strip) strip.push(piece);
else strips.set(key, [piece]);
}
const runs: Piece[] = [];
for (const strip of strips.values()) {
@@ -514,11 +596,15 @@ export function mergeCells(cells: readonly SolidCell[]): CellRect[] {
else runs.push((run = { ...piece }));
}
}
runs.sort((a, b) => a.y0 - b.y0 || a.x0 - b.x0);
return runs;
}
/** Runs of exactly the same left and right that meet top to bottom, joined into taller rectangles. */
function stacked(runs: readonly Piece[]): CellRect[] {
const done: Piece[] = [];
// Rectangles still open at the bottom, by the left and right they span.
const open = new Map<string, Piece>();
for (const run of runs) {
for (const run of [...runs].sort((a, b) => a.y0 - b.y0 || a.x0 - b.x0)) {
const key = `${run.x0},${run.x1}`;
const above = open.get(key);
if (above && above.y1 === run.y0) {
@@ -526,7 +612,7 @@ export function mergeCells(cells: readonly SolidCell[]): CellRect[] {
continue;
}
if (above) done.push(above);
open.set(key, run);
open.set(key, { ...run });
}
done.push(...open.values());
return done
@@ -534,6 +620,57 @@ export function mergeCells(cells: readonly SolidCell[]): CellRect[] {
.map((p) => ({ x: p.x0, y: p.y0, w: p.x1 - p.x0, h: p.y1 - p.y0 }));
}
/**
* A map's rectangles, merged row by row: a strip never spans two rows – its top and bottom name the
* row – so the runs of a row follow from that row and the two around it alone. A patched list of
* blocks (#348) joins only the rows near its patch again and stacks the rest as they were.
*/
interface MergedRows {
runs: ReadonlyMap<number, readonly Piece[]>;
rects: readonly CellRect[];
}
const mergedOf = new WeakMap<readonly CellBlock[], MergedRows>();
/**
* How a patched list of blocks came from the one before: the cells worked out again, what blocks
* there now, and the rows whose runs that may change.
*/
interface BlockPatch {
from: readonly CellBlock[];
rows: readonly number[];
indices: readonly number[];
columns: number;
made: ReadonlyMap<number, CellBlock>;
}
const patchOf = new WeakMap<readonly CellBlock[], BlockPatch>();
/** The index of the first block in row `y` or below; the blocks are in cell order. */
function firstInRow(blocks: readonly CellBlock[], y: number): number {
let low = 0;
let high = blocks.length;
while (low < high) {
const mid = (low + high) >> 1;
if (blocks[mid]!.y < y) low = mid + 1;
else high = mid;
}
return low;
}
function rowRuns(blocks: readonly CellBlock[], y: number): Piece[] {
const near = [new Set<number>(), new Set<number>(), new Set<number>()];
const row: Required<SolidCell>[] = [];
for (let i = firstInRow(blocks, y - 1); i < blocks.length && blocks[i]!.y <= y + 1; i++) {
const { x, y: at, box } = blocks[i]!;
if (box === null) continue;
near[at - y + 1]!.add(x);
if (at === y) row.push({ x, y, scale: box });
}
const blocksAt = (x: number, at: number) => near[at - y + 1]?.has(x) ?? false;
return joinedStrips(row.flatMap((cell) => cellPieces(cell, blocksAt)));
}
/** The blocking rectangles of a map, in cells. */
export function solidRects(
project: Project,
@@ -546,10 +683,27 @@ export function solidRects(
}
function rectCells(blocks: readonly CellBlock[]): readonly CellRect[] {
return derived(rectsOf, blocks, (cells) => mergeCells(boxCells(cells)));
let merged = mergedOf.get(blocks);
if (!merged) {
const patch = patchOf.get(blocks);
const before = patch && mergedOf.get(patch.from);
const runs = new Map(before?.runs);
const rows = before
? patch.rows
: new Set(blocks.filter((cell) => cell.box !== null).map((cell) => cell.y));
for (const y of rows) {
const found = rowRuns(blocks, y);
if (found.length > 0) runs.set(y, found);
else runs.delete(y);
}
const order = [...runs.keys()].sort((a, b) => a - b);
merged = { runs, rects: stacked(order.flatMap((y) => runs.get(y)!)) };
mergedOf.set(blocks, merged);
}
return merged.rects;
}
/** The trigger tiles painted on the map (#160), each once, in the order the layers first name them. */
/** The trigger tiles painted on the map (#160), each once: layer by layer, by id within one. */
function triggerTiles(project: Project, asset: TilemapAsset): TileAsset[] {
const ids = new Set(asset.layers.flatMap((layer) => cellsTileIds(layer.cells)));
return [...ids].flatMap((id) => {
src/core/model/tilemapMemo.ts
+98 -21
View File
@@ -8,52 +8,115 @@
* with what it saw last and hands out a new number only when it differs – a number that stands for
* exactly that content in a cache key. Without that word nothing is compared: a map is asked some
* sixty times a frame, and walking every cell each time was the largest single cost of a run (#284).
*
* Each new version also keeps which cells it changed, so whoever worked out the one before can
* patch those cells instead of the whole map (#348, `cellsChangedSince`).
*/
interface Seen {
keys: string[];
values: string[];
/** The content as it was last looked at. */
held: Map<string, string>;
/** How many cells name each tile. */
counts: Map<string, number>;
version: number;
tileIds: string[];
/** Written to since it was looked at. */
stale: boolean;
/** Written to since it was looked at: the keys the painter named, or every key when it named none. */
stale: Set<string> | 'all' | null;
/** The cells each of the last versions changed, oldest first. */
log: { from: number; to: number; keys: string[] }[];
}
/** How many versions back a change can be asked for; further back is worked out afresh. */
const LOG_LENGTH = 16;
const seen = new WeakMap<Record<string, string>, Seen>();
let versions = 0;
function count(counts: Map<string, number>, tileId: string | undefined, by: number): boolean {
if (!tileId) return false;
const now = (counts.get(tileId) ?? 0) + by;
if (now > 0) counts.set(tileId, now);
else counts.delete(tileId);
return now === 0 || now === by;
}
/**
* Walked in the object's own order: the same content painted the same way comes back in the same
* order, and a cell taken out and put back only costs a new version, never a wrong answer.
* Sorted, so the same content gives the same list whichever way it was painted: the order of an
* object's keys follows its history.
*/
function unchanged(cells: Record<string, string>, held: Seen): boolean {
let i = 0;
function idsOf(counts: Map<string, number>): string[] {
return [...counts.keys()].sort();
}
/** The keys whose tile differs from what was held: the ones named, or every key of both. */
function differing(cells: Record<string, string>, held: Seen): string[] {
const own = (key: string) => (Object.hasOwn(cells, key) ? cells[key] : undefined);
if (held.stale !== 'all') return [...held.stale!].filter((key) => held.held.get(key) !== own(key));
const keys: string[] = [];
let kept = 0;
for (const key in cells) {
if (held.keys[i] !== key || held.values[i] !== cells[key]) return false;
i++;
const before = held.held.get(key);
if (before === undefined) keys.push(key);
else {
kept++;
if (before !== cells[key]) keys.push(key);
}
}
return i === held.keys.length;
if (kept < held.held.size)
for (const key of held.held.keys()) if (!Object.hasOwn(cells, key)) keys.push(key);
return keys;
}
function look(cells: Record<string, string>): Seen {
const held = seen.get(cells);
if (held && !held.stale) return held;
if (held && unchanged(cells, held)) {
held.stale = false;
if (held) {
const keys = differing(cells, held);
held.stale = null;
if (keys.length === 0) return held;
let idsChanged = false;
for (const key of keys) {
const now = Object.hasOwn(cells, key) ? cells[key] : undefined;
idsChanged = count(held.counts, held.held.get(key), -1) || idsChanged;
idsChanged = count(held.counts, now, 1) || idsChanged;
if (now === undefined) held.held.delete(key);
else held.held.set(key, now);
}
if (idsChanged) held.tileIds = idsOf(held.counts);
const to = ++versions;
held.log.push({ from: held.version, to, keys });
if (held.log.length > LOG_LENGTH) held.log.shift();
held.version = to;
return held;
}
const keys = Object.keys(cells);
const values = keys.map((key) => cells[key]!);
const tileIds = [...new Set(values)].filter(Boolean);
const fresh = { keys, values, version: ++versions, tileIds, stale: false };
const content = new Map(Object.entries(cells));
const counts = new Map<string, number>();
for (const tileId of content.values()) count(counts, tileId, 1);
const fresh: Seen = {
held: content,
counts,
version: ++versions,
tileIds: idsOf(counts),
stale: null,
log: [],
};
seen.set(cells, fresh);
return fresh;
}
/** Says that cells were written to in place; a replaced object needs no word, it is a new one. */
export function cellsChanged(cells: Record<string, string>): void {
/**
* Says that cells were written to in place; a replaced object needs no word, it is a new one.
* Naming the keys written saves walking all of them.
*/
export function cellsChanged(cells: Record<string, string>, keys?: Iterable<string>): void {
const held = seen.get(cells);
if (held) held.stale = true;
if (!held || held.stale === 'all') return;
if (!keys) {
held.stale = 'all';
return;
}
held.stale ??= new Set();
for (const key of keys) held.stale.add(key);
}
/** A number that is the same exactly as long as the cells hold the same tiles in the same places. */
@@ -61,11 +124,25 @@ export function cellsVersion(cells: Record<string, string>): number {
return look(cells).version;
}
/** The tiles the cells name, each once. */
/** The tiles the cells name, each once, sorted. */
export function cellsTileIds(cells: Record<string, string>): readonly string[] {
return look(cells).tileIds;
}
/**
* The keys whose tile changed since `version` of these cells, each once – none when it is the
* current one. Null when that is not known: another object's version, or one too far back.
*/
export function cellsChangedSince(cells: Record<string, string>, version: number): readonly string[] | null {
const held = look(cells);
if (version === held.version) return [];
const start = held.log.findIndex((entry) => entry.from === version);
if (start < 0) return null;
const keys = new Set<string>();
for (const entry of held.log.slice(start)) for (const key of entry.keys) keys.add(key);
return [...keys];
}
const tokens = new WeakMap<object, number>();
let tokenCount = 0;
src/core/model/usages.ts
+75
View File
@@ -7,7 +7,9 @@ import { isKindWith } from '../properties/kinds';
import { schemaForAsset } from '../properties/PropertySchema';
import { referencesAny, trackDef } from './animation';
import type { AnimationAsset, Asset, TileAsset } from './Asset';
import { partsOf } from './assetParts';
import { overrideSites } from './instance';
import { pixelArtRefIds } from './pixelart';
import { type ObjectHolder, objectHolders, type Project } from './Project';
import type { SceneObject } from './SceneObject';
@@ -84,6 +86,79 @@ export function cellsPainted(project: Project, tileId: string): { tilemaps: numb
return { tilemaps, cells };
}
/** The nearest thing with an id of its own a use sits in: an object, a node, an item, a key. */
export interface UsePlace {
id: string;
name?: string;
type?: string;
kind?: string;
}
/** An asset naming another, and where in it; no places for a use on the asset itself, like a property. */
export interface AssetUse {
asset: Asset;
at: UsePlace[];
}
export interface Uses {
assets: AssetUse[];
/** The project's own settings naming it: its start scene, the master's FX, the Master routing. */
project: ('startScene' | 'masterFx' | 'routing')[];
}
/**
* Everything naming an asset or one of its parts – a frame, a clip (#433). Read as data, not field
* by field, as `namedStrings` reads it for sounds, so no place is missed: a song item, a Sample, a
* key, a cell of a tilemap and kinds of asset still to come alike. Ids, the owner a song keeps an
* asset for and the nodes an edge joins are no uses: the Master routing has a node per asset.
*/
export function usesOf(project: Project, assetId: string): Uses {
const target = project.assets.find((asset) => asset.id === assetId);
if (!target) return { assets: [], project: [] };
const wanted = new Set(
target.type === 'pixelart' ? pixelArtRefIds(target) : [target.id, ...partsOf(target).map((p) => p.id)],
);
const assets: AssetUse[] = [];
for (const asset of project.assets) {
if (asset.id === target.id) continue;
const found = placesNaming(asset, wanted);
if (found) assets.push({ asset, at: found });
}
const settings: Uses['project'] = [];
if (wanted.has(project.startSceneId)) settings.push('startScene');
if (wanted.has(project.masterFx)) settings.push('masterFx');
if (placesNaming(project.routing, wanted)) settings.push('routing');
return { assets, project: settings };
}
const NO_USE: ReadonlySet<string> = new Set(['id', 'owner', 'node']);
/** Where in `value` one of the ids is named; null for nowhere, no places for only outside anything with an id. */
function placesNaming(value: unknown, wanted: ReadonlySet<string>): UsePlace[] | null {
const places = new Map<string, UsePlace | null>();
const walk = (inner: unknown, place: UsePlace | null): void => {
if (typeof inner === 'string') {
if (wanted.has(inner)) places.set(place?.id ?? '', place);
} else if (Array.isArray(inner)) {
for (const item of inner) walk(item, place);
} else if (typeof inner === 'object' && inner !== null && !ArrayBuffer.isView(inner)) {
const record = inner as Record<string, unknown>;
const here = typeof record.id === 'string' && inner !== value ? placeOf(record) : place;
for (const [key, field] of Object.entries(record)) if (!NO_USE.has(key)) walk(field, here);
}
};
walk(value, null);
if (places.size === 0) return null;
return [...places.values()].filter((place): place is UsePlace => place !== null);
}
function placeOf(record: Record<string, unknown>): UsePlace {
const place: UsePlace = { id: record.id as string };
for (const key of ['name', 'type', 'kind'] as const)
if (typeof record[key] === 'string' && record[key] !== '') place[key] = record[key];
return place;
}
/** The animations keying any of these ids – intrinsic data, which `findPropertyUsages` does not see. */
export function animationsUsing(project: Project, ids: readonly string[]): AnimationAsset[] {
const wanted = new Set(ids);
src/core/model/video.ts
+190
View File
@@ -0,0 +1,190 @@
import { FROM_ASSET_BUS, MASTER_MIX, mixBusName, pickedComposition } from '../properties/buses';
import { videoLoopsOf, videoSpanOf, videoVolumeOf } from '../properties/videos';
import type { VideoAsset, VideoClip } from './Asset';
import { findAsset, findAssetOf, type Project } from './Project';
import { clipsOf } from './sound';
import { storedFilePattern } from './storedFile';
const FILE_PATTERN = storedFilePattern('mp4');
/**
* How long a run waits for a driven video (#404) it shows to have its first picture before it goes
* anyway: at its start, and a composition (#417) at each item that brings one in.
*/
export const VIDEO_WAIT_MS = 3000;
export function isVideoFile(name: string): boolean {
return FILE_PATTERN.test(name);
}
/** What a texture field names when it names a video: the whole video, or one of its clips. */
export interface VideoRef {
asset: VideoAsset;
clip: VideoClip | null;
}
/** A video or a clip of one, by id; clip ids share the generator with asset ids, as frames do. */
export function findVideo(project: Project, id: string): VideoRef | null {
if (id === '') return null;
const asset = findAsset(project, id);
if (asset) return asset.type === 'video' ? { asset, clip: null } : null;
for (const video of project.assets) {
if (video.type !== 'video') continue;
const clip = clipsOf(video).find((c) => c.id === id);
if (clip) return { asset: video, clip };
}
return null;
}
/** Every id that means "this video": the asset itself and each of its clips. */
export function videoRefIds(asset: VideoAsset): string[] {
return [asset.id, ...clipsOf(asset).map((c) => c.id)];
}
/**
* The stretch a clip plays, inside the video: a start before its end, an end of 0 or past the
* video's length being its end. Without a length (not known yet) the asked values stand.
*/
export function videoSpan(duration: number, start: number, end: number): { start: number; end: number } {
const length = duration > 0 ? duration : Infinity;
const from = Math.min(Math.max(0, start), Number.isFinite(length) ? Math.max(0, length - 0.001) : start);
const to = end > 0 ? Math.min(end, length) : length;
return { start: from, end: to > from ? to : length };
}
/** How a video is played: which file, which stretch of it, whether it repeats, how loud. */
export interface VideoPlay {
file: string;
start: number;
/** Seconds; Infinity for the end of the video, wherever that is. */
end: number;
loop: boolean;
volume: number;
/** The bus of the mix its sound goes into, where a sprite picked one; along its routing where left out. */
bus?: string;
}
export function videoPlayOf(ref: VideoRef): VideoPlay {
const asked = ref.clip ? videoSpanOf(ref.clip.props) : { start: 0, end: 0 };
const span = videoSpan(ref.asset.duration, asked.start, asked.end);
return {
file: ref.asset.file,
start: span.start,
end: ref.asset.duration > 0 && span.end >= ref.asset.duration ? Infinity : span.end,
loop: videoLoopsOf(ref.asset.props),
volume: videoVolumeOf(ref.asset.props),
};
}
/** Seconds a video or clip lasts; 0 while the video's length is not known yet. */
export function videoLengthOf(ref: VideoRef): number {
const play = videoPlayOf(ref);
const end = Number.isFinite(play.end) ? play.end : ref.asset.duration;
return end > play.start ? end - play.start : 0;
}
/** A key of a `Video time` row: seconds into the clip on a frame of the animation. */
export interface FrameValue {
frame: number;
value: number;
}
/**
* Where a driven video playing at 1× from `value` seconds on `frame` reaches the end of a clip
* `length` seconds long (#407): the last whole frame at `fps` before it, and the clip's time there.
* Null when not one frame fits.
*/
export function realTimeEnd(frame: number, value: number, length: number, fps: number): FrameValue | null {
if (!(length > 0) || !(fps > 0) || !Number.isFinite(value)) return null;
const frames = Math.floor((length - value) * fps + 1e-6);
if (frames < 1) return null;
return { frame: frame + frames, value: Math.round((value + frames / fps) * 1e6) / 1e6 };
}
const PREFIX = 'video:';
/**
* The name a playing video goes by where a picture is named by `TextureRef.file` (see
* `generated.ts`): `video:<file>#<start>~<end>~<loop>~<volume>`, and `!<bus>` where its sound goes
* into a bus a sprite picked. No uploaded image looks like it, and one key is one player: two sprites
* wearing the same clip show the same picture.
*/
export function videoKey(play: VideoPlay): string {
const end = Number.isFinite(play.end) ? play.end : 0;
const bus = play.bus ? `!${play.bus}` : '';
return `${PREFIX}${play.file}#${play.start}~${end}~${play.loop ? 1 : 0}~${play.volume}${bus}`;
}
/**
* The bus of the mix the Video bus of a sprite names: null along the routing of its video, the
* composition's for one it picked for a sprite of its items. A bus that is gone is the master.
*/
export function videoMixBus(project: Project, picked: string): string | null {
if (picked === '' || picked === FROM_ASSET_BUS) return null;
const comp = pickedComposition(picked);
if (comp !== null) return findAssetOf(project, comp, 'composition') ? mixBusName(comp) : null;
return findAssetOf(project, picked, 'bus') ? mixBusName(picked) : MASTER_MIX;
}
/** The key of the same video heard on `bus` (#418); along its routing for null. */
export function routedVideoKey(key: string, bus: string | null): string {
const play = parseVideoKey(key);
if (!play || isDrivenVideoKey(key)) return key;
const { bus: _, ...rest } = play;
return videoKey(bus ? { ...rest, bus } : rest);
}
/**
* The key of a player one wearer drives on its own (#404): two sprites driving the same clip must
* not fight over one `<video>`, so the wearer's name goes after an `@`.
*/
export function drivenVideoKey(key: string, wearer: string): string {
return `${key}@${wearer}`;
}
export function isVideoKey(key: string): boolean {
return parseVideoKey(key) !== null;
}
/** Whether the key names a player its wearer drives, see `drivenVideoKey`. */
export function isDrivenVideoKey(key: string): boolean {
return isVideoKey(key) && key.includes('@');
}
export function parseVideoKey(key: string): VideoPlay | null {
if (!key.startsWith(PREFIX)) return null;
const hash = key.indexOf('#');
if (hash < 0) return null;
const file = key.slice(PREFIX.length, hash);
const at = key.indexOf('@', hash);
const body = key.slice(hash + 1, at < 0 ? undefined : at);
const bang = body.indexOf('!');
const parts = (bang < 0 ? body : body.slice(0, bang)).split('~');
const [start, end, loop, volume] = parts.map(Number);
if (!isVideoFile(file) || parts.length !== 4 || ![start, end, loop, volume].every(Number.isFinite))
return null;
const play = { file, start: start!, end: end! > 0 ? end! : Infinity, loop: loop === 1, volume: volume! };
return bang < 0 ? play : { ...play, bus: body.slice(bang + 1) };
}
/** How far before its end a held clip shows its last picture: the end itself is the next one's. */
const LAST_PICTURE = 0.001;
/**
* Where in its file a driven video stands at `time` seconds into its clip (#404). Before the start
* it shows the first picture, past the end it goes round with Loop and holds the last one without.
* `duration` is the file's, which a clip running to the end of the video needs; unknown (NaN, not
* loaded yet) the time is not bounded at the end.
*/
export function drivenTimeOf(
play: Pick<VideoPlay, 'start' | 'end' | 'loop'>,
time: number,
duration: number,
): number {
const end = Number.isFinite(play.end) ? play.end : duration;
const length = end - play.start;
const run = Math.max(0, Number.isFinite(time) ? time : 0);
if (!(length > 0)) return play.start + run;
if (play.loop) return play.start + (run % length);
return play.start + Math.min(run, Math.max(0, length - LAST_PICTURE));
}
src/core/model/videoClock.ts
+67
View File
@@ -0,0 +1,67 @@
/**
* The time of the video a sprite wears, as a run counts it (#410): what `On video finished` and
* `Video time` go by, and what `Play video`, `Pause video` and `Seek video` set.
*
* A free video counts from when its sprite appears, while it is switched on; a behavior that steers
* it takes it over from there. A driven one has the time its `Video time` says.
*/
export interface VideoClock {
/** The texture it counts for, a video or a clip of one; another one starts it over. */
ref: string;
/** Seconds into the clip: between 0 and its length, once that is known. */
time: number;
playing: boolean;
/** Set once a behavior steered it: the run drives the sprite's video by this time from then on. */
steered: boolean;
/** The behaviors that steered it: they hear its end wherever it stands. */
steeredBy: object[];
}
export type VideoSteer = { kind: 'play' } | { kind: 'pause' } | { kind: 'seek'; time: number };
/** What `Video time` reads. */
export interface VideoReading {
time: number;
length: number;
playing: boolean;
}
export function newVideoClock(ref: string): VideoClock {
return { ref, time: 0, playing: true, steered: false, steeredBy: [] };
}
/**
* Where `time` seconds into a clip `length` long stand, by the rule of `Video time`: round again with
* Loop, held at the end without. An unknown length (0) bounds only at the start.
*/
export function clipTimeOf(time: number, length: number, loop: boolean): number {
const run = Number.isFinite(time) ? Math.max(0, time) : 0;
if (!(length > 0)) return run;
return loop ? run % length : Math.min(run, length);
}
/**
* Moves a playing clock on by `dt`. Returns whether it reached the end of a clip without Loop: it
* stops there, so that happens once a pass.
*/
export function advanceVideoClock(clock: VideoClock, dt: number, length: number, loop: boolean): boolean {
if (!clock.playing) return false;
const time = clock.time + dt;
if (!(length > 0) || loop || time < length) {
clock.time = clipTimeOf(time, length, loop);
return false;
}
clock.time = length;
clock.playing = false;
return true;
}
/** What a steering node does to the clock; Play at the end of a clip without Loop starts it over. */
export function steerVideoClock(clock: VideoClock, steer: VideoSteer, length: number, loop: boolean): void {
if (steer.kind === 'pause') clock.playing = false;
else if (steer.kind === 'seek') clock.time = clipTimeOf(steer.time, length, loop);
else {
if (!loop && length > 0 && clock.time >= length) clock.time = 0;
clock.playing = true;
}
}
src/core/music/Scheduler.ts
+182 -8
View File
@@ -1,4 +1,16 @@
import { clipsIn, heldLoop, type LoopRange, notesIn, playheadAt, type Sequence } from './sequence';
import type { FadeCurve } from '../model/sound';
import type { FxParams } from '../soundfx/nodes';
import { AUTOMATION_STEP, type AutomationTarget, type LaneMark, type SequenceLane } from './automation';
import {
heldMarks,
heldParamsAt,
heldValueAt,
holdKey,
holdsAfter,
type LaneHold,
type LanePlaying,
} from './laneHolds';
import { clipsIn, heldLoop, levelsIn, type LoopRange, notesIn, playheadAt, type Sequence } from './sequence';
import {
COMMON_TIME,
DEFAULT_TEMPO,
@@ -29,6 +41,8 @@ export interface ScheduledNote {
/** A factor on its loudness, and the track of a song it is heard on (#334). */
volume?: number;
bus?: string;
/** The values the parameter lanes of its track have where it starts (#337). */
params?: FxParams;
}
/** One clip handed to the player (#334): when it starts, how long it plays and from where in the sound. */
@@ -42,6 +56,34 @@ export interface ScheduledClip {
volume: number;
bus: string;
tempo: number;
/** As a note's. */
params?: FxParams;
}
/**
* One ramp of a bus's level handed to the player (#418): from `at` for `duration` seconds, along the
* part of the ramp from `done` (0 to 1) on – a playhead that lands inside one takes it up there.
*/
export interface ScheduledRamp {
at: number;
duration: number;
bus: string;
gain: number;
rise: boolean;
curve: FadeCurve;
done: number;
}
/**
* Where a lane of automation (#337) takes a bus over one window: each mark a second of the audio
* clock with the value the bus is ramped to, or jumps to there. With `fresh` the playing has just
* started, and what was planned for the bus from the first mark on is to go.
*/
export interface ScheduledAutomation {
bus: string;
target: AutomationTarget;
marks: LaneMark[];
fresh: boolean;
}
/** What the scheduler needs of the audio side; a test hands in a clock of its own. */
@@ -60,6 +102,10 @@ export interface SchedulerHost {
play(note: ScheduledNote): number;
/** Starts a clip the same way; a host that plays no clips leaves it out. */
playClip?(clip: ScheduledClip): number;
/** Ramps the level of a bus; a host without levels leaves it out. */
ramp?(ramp: ScheduledRamp): void;
/** Takes a bus along its automation; a host without it leaves it out. */
automate?(automation: ScheduledAutomation): void;
playing(voice: number): boolean;
/** Ends these voices; with `letRing` a note falls away in the release of its Envelope. */
stop(voices: readonly number[], letRing?: boolean): void;
@@ -100,6 +146,8 @@ export class Scheduler {
private before: { anchor: number; from: number; tempo: number; loop: LoopRange | null } | null = null;
private voices: number[] = [];
private sleep: (() => void) | null = null;
/** The lanes a game holds on a value of its own (#338), by `holdKey`, each's holds in the order they land. */
private holds = new Map<string, LaneHold[]>();
constructor(
private readonly host: SchedulerHost,
@@ -121,16 +169,26 @@ export class Scheduler {
return this.anchor;
}
/** Played to its end, and none of its notes is heard any more. */
/**
* Played to its end, and none of its notes is heard any more. Planned to its end is not yet played
* to it: its notes are planned ahead, and a last bar may end in silence (#373).
*/
get done(): boolean {
return this.ended && this.voices.every((voice) => !this.host.playing(voice));
return (
this.ended &&
this.headAt(this.host.now()) >= this.end &&
this.voices.every((voice) => !this.host.playing(voice))
);
}
/** Plays from `from` ticks of the sequence, or from where it was last started or sent. */
start(from = this.from): void {
/**
* Plays from `from` ticks of the sequence, or from where it was last started or sent – heard from
* `at` on the audio clock where that is later than it can be (#338).
*/
start(from = this.from, at = -Infinity): void {
this.stop();
this.from = Math.max(0, Math.round(from));
this.anchor = this.host.now() + LEAD + this.ahead();
this.anchor = Math.max(this.host.now() + LEAD + this.ahead(), at);
this.until = 0;
this.before = null;
this.ended = false;
@@ -171,6 +229,96 @@ export class Scheduler {
this.meter = heldMeter(meter);
}
/**
* Holds the lane of `target` on the bus `bus` on `value`, in its units (#338): from the second `at`
* of the audio clock it goes there from where it is over `fade` seconds and stays, its own points
* no longer heard. A hold that lands before it rules until then; one that would land later, or a
* release still to come, gives way. What is planned for it past `at` is planned again. False where
* the sequence has no such lane.
*/
holdLane(bus: string, target: AutomationTarget, value: number, at: number, fade: number): boolean {
const lane = this.laneOf(bus, target);
if (!lane) return false;
const key = holdKey(bus, target);
const playing = this.lanePlaying();
const kept = (this.holds.get(key) ?? []).filter((hold) => hold.at < at);
const start = this.anchor === null ? value : heldValueAt(lane, kept, playing, playing.elapsed(at));
const last = kept.at(-1);
if (last?.release && last.release.at >= at) last.release = null;
this.holds.set(key, [...kept, { at, fade: Math.max(0, fade), start, value, release: null }]);
this.replan(lane, at);
return true;
}
/**
* Lets a held lane go (#338): from the second `at` it goes back over `fade` seconds to where its
* curve is by then, and follows it from there. A hold that would land later is never heard. False
* where the lane is not held.
*/
releaseLane(bus: string, target: AutomationTarget, at: number, fade: number): boolean {
const lane = this.laneOf(bus, target);
const key = holdKey(bus, target);
const holds = this.holds.get(key);
if (!lane || !holds?.length) return false;
const kept = holds.filter((hold) => hold.at < at);
const dropped = holds[kept.length]?.at ?? Infinity;
const last = kept.at(-1);
if (last && !(last.release && last.release.at <= at)) last.release = { at, fade: Math.max(0, fade) };
if (kept.length > 0) this.holds.set(key, kept);
else this.holds.delete(key);
this.replan(lane, Math.min(at, dropped));
return true;
}
/** Holds the lanes `other` holds (#338): a playing that takes over from it keeps them. */
keepHolds(other: Scheduler): void {
for (const [key, holds] of other.holds)
this.holds.set(
key,
holds.map((hold) => ({ ...hold, release: hold.release && { ...hold.release } })),
);
}
/** The lane of the sequence on that bus for that target; undefined for none. */
private laneOf(bus: string, target: AutomationTarget): SequenceLane | undefined {
return this.sequence().lanes?.find((lane) => lane.bus === bus && lane.target === target);
}
/** How the sequence plays for its lanes: the playing time is counted from the anchor. */
private lanePlaying(): LanePlaying {
const anchor = this.anchor ?? 0;
return {
from: this.from,
loop: this.loop,
elapsed: (at) => ticksOfSeconds(at - anchor, this.tempo),
};
}
/** What is planned for a lane from the second `at` on is planned again, from there to where planning got. */
private replan(lane: SequenceLane, at: number): void {
if (this.anchor === null) return;
const start = Math.max(0, this.lanePlaying().elapsed(at));
if (start < this.until) this.automate(lane, start, this.until, true, true);
}
/**
* Takes the bus of a lane along it over the playing time from `start` up to `end` (#337), along its
* holds where they rule (#338). With `restart` what was planned for it from the first mark on goes.
*/
private automate(lane: SequenceLane, start: number, end: number, fresh: boolean, restart: boolean): void {
if (!this.host.automate || this.anchor === null) return;
const anchor = this.anchor;
const step = ticksOfSeconds(AUTOMATION_STEP, this.tempo);
const holds = this.holds.get(holdKey(lane.bus, lane.target)) ?? [];
const marks = heldMarks(lane, holds, this.lanePlaying(), start, end, fresh, step);
this.host.automate({
bus: lane.bus,
target: lane.target,
marks: marks.map((mark) => ({ ...mark, at: anchor + secondsOfTicks(mark.at, this.tempo) })),
fresh: restart,
});
}
/**
* Where what is heard right now stands: the audio clock less the output latency, so a playhead
* drawn from it is not ahead of the sound. Before the first note it is where it starts.
@@ -189,21 +337,26 @@ export class Scheduler {
if (to <= this.until) return;
this.voices = this.voices.filter((voice) => this.host.playing(voice));
const sequence = this.sequence();
const lanes = sequence.lanes ?? [];
const playing = this.lanePlaying();
for (const due of notesIn(sequence, this.from, this.loop, this.until, to)) {
const params = heldParamsAt(lanes, this.holds, playing, due.bus, due.tick, due.at);
const voice = this.host.play({
at: this.anchor + secondsOfTicks(due.at, this.tempo),
duration: secondsOfTicks(due.length, this.tempo),
note: due.note,
velocity: due.velocity,
instrument: due.instrument,
tempo: this.tempo,
tempo: due.tempo ?? this.tempo,
...(due.volume !== undefined && { volume: due.volume }),
...(due.bus !== undefined && { bus: due.bus }),
...(params && { params }),
});
if (voice !== 0) this.voices.push(voice);
}
if (this.host.playClip)
for (const due of clipsIn(sequence, this.from, this.loop, this.until, to, this.fresh)) {
const params = heldParamsAt(lanes, this.holds, playing, due.bus, due.tick + due.skip, due.at);
const voice = this.host.playClip({
at: this.anchor + secondsOfTicks(due.at, this.tempo),
duration: secondsOfTicks(due.length, this.tempo),
@@ -212,12 +365,33 @@ export class Scheduler {
fx: due.fx,
volume: due.volume,
bus: due.bus,
tempo: this.tempo,
tempo: due.tempo ?? this.tempo,
...(params && { params }),
});
if (voice !== 0) this.voices.push(voice);
}
if (this.host.ramp)
for (const due of levelsIn(sequence, this.from, this.loop, this.until, to, this.fresh)) {
const whole = due.length + due.skip;
this.host.ramp({
at: this.anchor + secondsOfTicks(due.at, this.tempo),
duration: secondsOfTicks(due.length, this.tempo),
bus: due.bus,
gain: due.gain,
rise: due.rise,
curve: due.curve,
done: whole > 0 ? due.skip / whole : 0,
});
}
for (const lane of lanes) this.automate(lane, this.until, to, this.fresh, this.fresh);
this.fresh = false;
this.until = to;
// A hold let go and planned back to its curve is over.
for (const [key, holds] of this.holds) {
const left = holdsAfter(holds, playing, to);
if (left.length > 0) this.holds.set(key, left);
else this.holds.delete(key);
}
if (!this.loop && playheadAt(to, this.from, null) >= this.end) {
this.ended = true;
this.sleep?.();
src/core/music/automation.ts
+425
View File
@@ -0,0 +1,425 @@
/**
* Automation of a song (#337): the volume and the pan of a track changing over the song, as points
* on a lane. From each point the value runs to the next along that point's easing, as from one
* animation key to the next. While a lane has points it rules its value in place of the fader: before
* its first point it is that point's value, after its last the last one's. Two points on one tick
* are a jump there.
*
* The scheduler reads a lane window by window as marks on the audio clock, each ramped or jumped to.
* Where the value moves along a shape the clock cannot ramp, it is read every `AUTOMATION_STEP`.
*
* A lane can also move a parameter (`param:<name>`): of the track's sound FX, of the sound FX of its
* items, of the instruments of its notes and the sound FX of its clips – wherever one of them has a
* parameter of that name, as `Set sound FX parameters` reaches them. A note or clip starts at the
* value its lane has there, so a parameter a voice only reads when it starts is heard from the next one.
*/
import type { KeyInterpolation, VectorPoint } from '../model/Asset';
import { easedShare, easingOf, isKeyInterpolation } from '../model/animation';
import type { CurveLookup } from '../model/curve';
import type { FxParams } from '../soundfx/nodes';
import { dbToGain, TRACK_MAX_DB, TRACK_MIN_DB } from './level';
import { type LoopRange, playheadAt } from './sequence';
/** What a lane always moves on a track: its fader and its pan. */
export type TrackTarget = 'volume' | 'pan';
/** A parameter by its name (#337, stage 3); no name of a parameter has a colon. */
export type ParamTarget = `param:${string}`;
export type AutomationTarget = TrackTarget | ParamTarget;
export const AUTOMATION_TARGETS: readonly TrackTarget[] = ['volume', 'pan'];
const PARAM_PREFIX = 'param:';
export function paramTarget(name: string): ParamTarget {
return `${PARAM_PREFIX}${name}`;
}
/** The parameter a target moves; null for the volume and the pan. */
export function paramOf(target: AutomationTarget): string | null {
return target.startsWith(PARAM_PREFIX) ? target.slice(PARAM_PREFIX.length) : null;
}
/** Whether a text is a target a lane can have: the volume, the pan or a parameter with a name. */
export function isAutomationTarget(value: unknown): value is AutomationTarget {
if (value === 'volume' || value === 'pan') return true;
if (typeof value !== 'string' || !value.startsWith(PARAM_PREFIX)) return false;
const name = value.slice(PARAM_PREFIX.length);
return name !== '' && !name.includes(':');
}
/**
* The target a lane is named by in a game (#338): `volume`, `pan`, or a parameter by its name, with
* or without `param:` in front; null for a name no lane can have.
*/
export function targetNamed(name: string): AutomationTarget | null {
const text = name.trim();
if (isAutomationTarget(text)) return text;
const target = paramTarget(text);
return isAutomationTarget(target) ? target : null;
}
export interface AutomationPoint {
tick: number;
/** In the units of its target: decibels for the volume, -1 (left) to 1 (right) for the pan. */
value: number;
/** How it runs to the next point; linear where left out. */
ease?: KeyInterpolation;
}
export interface AutomationLane {
target: AutomationTarget;
/** In the order of their ticks. */
points: AutomationPoint[];
/**
* Only on a parameter's lane: the range its values are drawn and held in, `PARAM_RANGE` where left
* out. A parameter says nothing of its own range, so the lane does. Such a lane is kept without a
* point too, for its range; it rules its parameter once it has one.
*/
min?: number;
max?: number;
}
/** What says how a lane's values are held and drawn: its target, and a parameter's range. */
export type LaneKind = Pick<AutomationLane, 'target' | 'min' | 'max'>;
/** A lane's kind, or the volume or the pan by name: neither has a range of its own to name. */
export type LaneSpec = LaneKind | TrackTarget;
export function laneKind(spec: LaneSpec): LaneKind {
return typeof spec === 'string' ? { target: spec } : spec;
}
/** The range of a parameter's lane that sets none. */
export const PARAM_RANGE = { min: 0, max: 1 } as const;
/** Seconds between two readings of a value that moves along a shape the audio clock cannot ramp. */
export const AUTOMATION_STEP = 0.01;
/** The lowest and the highest value a lane of that kind holds. */
export function laneRange(spec: LaneSpec): { min: number; max: number } {
const { target, min, max } = laneKind(spec);
if (target === 'volume') return { min: TRACK_MIN_DB, max: TRACK_MAX_DB };
if (target === 'pan') return { min: -1, max: 1 };
return heldRange(min, max) ?? { ...PARAM_RANGE };
}
/** A range a parameter's lane may have: both ends finite, the top above the bottom; null otherwise. */
export function heldRange(min: unknown, max: unknown): { min: number; max: number } | null {
if (typeof min !== 'number' || typeof max !== 'number') return null;
if (!Number.isFinite(min) || !Number.isFinite(max) || !(max > min)) return null;
return { min, max };
}
/** A lane's value as its bus takes it: the volume as a gain, the pan and a parameter as they are. */
export function audioValueOf(lane: LaneKind, value: number): number {
const { min, max } = laneRange(lane);
const held = Math.max(min, Math.min(max, value));
return lane.target === 'volume' ? dbToGain(held) : held;
}
/**
* Held to what the lanes of a track may be: a target it knows, one lane for each, points in whole
* ticks from 0 with their values in range, in the order of their ticks. A lane of the volume or the
* pan without a point is dropped; a parameter's is kept for its range.
*/
export function heldLanes(raw: unknown): AutomationLane[] {
const lanes: AutomationLane[] = [];
for (const entry of Array.isArray(raw) ? (raw as unknown[]) : []) {
if (typeof entry !== 'object' || entry === null) continue;
const lane = entry as Record<string, unknown>;
const target = lane.target;
if (!isAutomationTarget(target) || lanes.some((l) => l.target === target)) continue;
const range = paramOf(target) !== null ? heldRange(lane.min, lane.max) : null;
const { min, max } = range ?? laneRange({ target });
const points = (Array.isArray(lane.points) ? (lane.points as unknown[]) : []).flatMap(
(point): AutomationPoint[] => {
if (typeof point !== 'object' || point === null) return [];
const { tick, value, ease } = point as Record<string, unknown>;
if (typeof tick !== 'number' || !Number.isFinite(tick) || !(tick >= 0)) return [];
if (typeof value !== 'number' || !Number.isFinite(value)) return [];
return [
{
tick: Math.round(tick),
value: Math.max(min, Math.min(max, value)),
...(isKeyInterpolation(ease) && ease !== 'linear' && { ease }),
},
];
},
);
points.sort((a, b) => a.tick - b.tick);
if (range) lanes.push({ target, points, ...range });
else if (points.length > 0 || paramOf(target) !== null) lanes.push({ target, points });
}
return lanes;
}
/** The parameters a track's lanes move, in their order, those without a point among them. */
export function laneParams(track: { automation?: readonly AutomationLane[] }): string[] {
return (track.automation ?? []).flatMap((lane) => paramOf(lane.target) ?? []);
}
/** The lane of a track for that target, while it has a point. */
export function laneOf(
track: { automation?: readonly AutomationLane[] },
target: AutomationTarget,
): AutomationLane | undefined {
return track.automation?.find((lane) => lane.target === target && lane.points.length > 0);
}
/** A lane on the bus it acts on, the Curve assets its points ease along looked up; one gone is linear. */
export function sequenceLaneOf(lane: AutomationLane, bus: string, curves: CurveLookup): SequenceLane {
return {
bus,
target: lane.target,
...(lane.min !== undefined && lane.max !== undefined && { min: lane.min, max: lane.max }),
points: lane.points.map((point) => {
const { shape, curve } = easingOf(point.ease ?? 'linear');
const found = shape === 'curve' ? curves(curve ?? '') : undefined;
return found ? { ...point, curve: found } : point;
}),
};
}
/** A point as a sequence plays it: a Curve asset it eases along already looked up. */
export interface SequenceLanePoint extends AutomationPoint {
curve?: readonly VectorPoint[];
}
/**
* A lane as a sequence plays it: on the bus it acts on, never without a point. A parameter's acts on
* that bus's sound FX and on everything heard on a bus inside it (`reaches`).
*/
export interface SequenceLane extends LaneKind {
bus: string;
points: readonly SequenceLanePoint[];
}
/** Whether a voice or a bus by that name is on the track of `bus` – it, or a bus inside it (`<bus>:…`). */
export function reaches(bus: string, name: string): boolean {
return name === bus || name.startsWith(`${bus}:`);
}
/**
* The values the parameter lanes of a sequence have at `tick` for what is heard on `bus`, a note's or
* a clip's; undefined where none reaches it.
*/
export function paramsAt(
lanes: readonly SequenceLane[],
bus: string | undefined,
tick: number,
): FxParams | undefined {
if (bus === undefined) return undefined;
let params: Record<string, number> | undefined;
for (const lane of lanes) {
const name = paramOf(lane.target);
if (name === null || !reaches(lane.bus, bus)) continue;
const { min, max } = laneRange(lane);
(params ??= {})[name] = Math.max(min, Math.min(max, laneValueAt(lane.points, tick)));
}
return params;
}
function between(a: SequenceLanePoint, b: SequenceLanePoint, t: number): number {
return a.value + (b.value - a.value) * easedShare(a.ease ?? 'linear', t, () => a.curve);
}
/** The index of the last point on or before `tick`, -1 for none. */
function lastAt(points: readonly SequenceLanePoint[], tick: number): number {
let i = -1;
while (i + 1 < points.length && points[i + 1]!.tick <= tick) i++;
return i;
}
/** The value at `tick`, a jump there already taken. */
export function laneValueAt(points: readonly SequenceLanePoint[], tick: number): number {
const i = lastAt(points, tick);
if (i < 0) return points[0]!.value;
const a = points[i]!;
const b = points[i + 1];
return b ? between(a, b, (tick - a.tick) / (b.tick - a.tick)) : a.value;
}
/** The value just before `tick`: what a jump there jumps from. */
export function laneValueBefore(points: readonly SequenceLanePoint[], tick: number): number {
let i = -1;
while (i + 1 < points.length && points[i + 1]!.tick < tick) i++;
if (i < 0) return points[0]!.value;
const a = points[i]!;
const b = points[i + 1];
if (!b) return a.value;
if (b.tick > tick) return between(a, b, (tick - a.tick) / (b.tick - a.tick));
return a.ease === 'step' ? a.value : between(a, b, 1);
}
/** Whether the way from `a` to `b` holds or is a straight line in lane units. */
function straight(a: SequenceLanePoint, b: SequenceLanePoint): boolean {
const ease = a.ease ?? 'linear';
return a.value === b.value || ease === 'step' || ease === 'linear';
}
/**
* Whether the bus takes it as one ramp: held, or a straight line also in its own units. A parameter
* becomes its knobs along ways of their own, so only one held is a ramp of them.
*/
function rampable(target: AutomationTarget, a: SequenceLanePoint, b: SequenceLanePoint): boolean {
const ease = a.ease ?? 'linear';
return a.value === b.value || ease === 'step' || (ease === 'linear' && target === 'pan');
}
/** One place on the way of a lane: `at` in ticks, the value as its bus takes it, ramped to or jumped to. */
export interface LaneMark {
at: number;
value: number;
jump: boolean;
}
/**
* Where a lane is over the playing time from `start` up to `end`, as `notesIn` lays out notes: a mark
* where the window starts, at every point and every reading in between, and where it ends. Where the
* playhead lands – at the start with `fresh`, after a jump back – the first mark of that stretch is a
* jump. `step` is `AUTOMATION_STEP` in ticks.
*/
export function laneMarks(
lane: SequenceLane,
from: number,
loop: LoopRange | null,
start: number,
end: number,
fresh: boolean,
step: number,
): LaneMark[] {
const { points } = lane;
const audio = (value: number) => audioValueOf(lane, value);
const marks: LaneMark[] = [];
const stretchEnd = loop !== null && from < loop.end ? loop.end : Infinity;
let elapsed = start;
let head = playheadAt(start, from, loop);
let landed = fresh;
while (elapsed < end) {
const span = Math.min(end - elapsed, stretchEnd - head);
if (!(span > 0)) break;
const to = head + span;
const first = laneValueAt(points, head);
marks.push({
at: elapsed,
value: audio(first),
jump: landed || laneValueBefore(points, head) !== first,
});
for (const tick of readings(lane, head, to, step)) {
const before = laneValueBefore(points, tick);
const value = laneValueAt(points, tick);
const at = elapsed + (tick - head);
marks.push({ at, value: audio(before), jump: false });
if (value !== before) marks.push({ at, value: audio(value), jump: true });
}
marks.push({ at: elapsed + span, value: audio(laneValueBefore(points, to)), jump: false });
elapsed += span;
landed = to >= stretchEnd;
head = landed ? loop!.start : to;
}
return thinned(marks);
}
/** The ticks strictly between `head` and `to` to mark: every point, and readings where the bus cannot ramp. */
function readings(lane: SequenceLane, head: number, to: number, step: number): number[] {
const { points, target } = lane;
const ticks: number[] = [];
points.forEach((a, i) => {
if (a.tick > head && a.tick < to) ticks.push(a.tick);
const b = points[i + 1];
if (b && b.tick > head && a.tick < to && !rampable(target, a, b))
ticks.push(...stepsOf(a.tick, head, Math.min(b.tick, to), step));
});
ticks.sort((x, y) => x - y);
return ticks.filter((tick, i) => i === 0 || tick !== ticks[i - 1]);
}
/** The ticks `step` apart from `origin` on, strictly between `after` and `before`; none without a step. */
function stepsOf(origin: number, after: number, before: number, step: number): number[] {
const ticks: number[] = [];
if (!(step > 0)) return ticks;
for (let k = Math.max(1, Math.floor((after - origin) / step) + 1); origin + k * step < before; k++)
if (origin + k * step > after) ticks.push(origin + k * step);
return ticks;
}
/** Without the marks a ramp would pass anyway: a value held from the mark before on. */
function thinned(marks: readonly LaneMark[]): LaneMark[] {
const kept: LaneMark[] = [];
marks.forEach((mark, i) => {
const last = kept.at(-1);
const next = marks[i + 1];
const held =
last !== undefined &&
next !== undefined &&
!mark.jump &&
last.value === mark.value &&
(next.jump || next.value === mark.value);
if (!held) kept.push(mark);
});
return kept;
}
/**
* A lane as it is heard over the playing time from `start` up to `end`, played from 0 round `loop` –
* a song in an item of a composition (#418) –, as a lane of its own in ticks of that playing time:
* each round with its points, cut at the loop end and jumping back there. A way cut in the middle
* that is no straight line is read every `step` ticks.
*/
export function unrolledLane(
points: readonly SequenceLanePoint[],
loop: LoopRange | null,
start: number,
end: number,
step: number,
): SequenceLanePoint[] {
const out: SequenceLanePoint[] = [];
const stretchEnd = loop !== null && loop.end > 0 ? loop.end : Infinity;
let elapsed = start;
let head = playheadAt(start, 0, loop);
while (elapsed < end) {
const span = Math.min(end - elapsed, stretchEnd - head);
if (!(span > 0)) break;
const shift = elapsed - head;
for (const point of stretchOf(points, head, head + span, step))
out.push({ ...point, tick: point.tick + shift });
elapsed += span;
head = head + span >= stretchEnd ? loop!.start : head + span;
}
return out;
}
/** The points of a lane from `head` to `to` as a lane by itself, ending on the value it reaches there. */
function stretchOf(
points: readonly SequenceLanePoint[],
head: number,
to: number,
step: number,
): SequenceLanePoint[] {
const out: SequenceLanePoint[] = [];
const at = lastAt(points, head);
const starts: { tick: number; own: number; whole: boolean }[] = [
{ tick: head, own: Math.max(0, at), whole: at >= 0 && points[at]!.tick === head },
];
points.forEach((point, i) => {
if (point.tick > head && point.tick < to) starts.push({ tick: point.tick, own: i, whole: true });
});
starts.forEach((s, k) => {
const next = starts[k + 1]?.tick ?? to;
const a = points[s.own]!;
const b = points[s.own + 1];
const value = s.whole ? a.value : laneValueAt(points, s.tick);
if (!b || (k === 0 && at < 0)) out.push({ tick: s.tick, value });
else if (s.whole && b.tick <= to) out.push({ ...a, tick: s.tick });
else if (straight(a, b)) out.push({ tick: s.tick, value, ...(a.ease === 'step' && { ease: 'step' }) });
else {
out.push({ tick: s.tick, value });
for (const tick of stepsOf(a.tick, s.tick, next, step))
out.push({ tick, value: laneValueAt(points, tick) });
}
});
out.push({ tick: to, value: laneValueBefore(points, to) });
return out;
}
src/core/music/bake.ts
+2
View File
@@ -233,6 +233,8 @@ export function bakePrint(project: Project, asset: BakedAsset): string {
clips,
buses,
instruments,
// Only where there is any, so a bake made before automation (#337) still holds.
...(sequence.lanes?.length ? [sequence.lanes] : []),
]);
return hashText(text);
}
src/core/music/beats.ts
+275
View File
@@ -0,0 +1,275 @@
/**
* Where a playing of a song or a pattern is heard, and the beats, bars and markers it has passed
* (#338). The audio side keeps each playing's time as an anchor on its clock; a game looks once a
* frame, so whatever it passed since the last look is told late, and too late is not told at all.
*/
import type { SongMarker } from './markers';
import { playheadAt, type LoopRange } from './sequence';
import { type Meter, positionOf, secondsOfTicks, ticksOfSeconds, ticksPerBar, ticksPerBeat } from './time';
/** The time of one playing: how fast, in what meter, from where, round what, to where. */
export interface Timeline {
tempo: number;
meter: Meter;
/** Where it started, in ticks. */
from: number;
/** What it runs round once it gets there; null plays it once to `length`. */
loop: LoopRange | null;
/** Where it ends when it does not loop, in ticks. */
length: number;
}
/** The most a spot passed is told late, and less where half a beat is shorter. */
export const LATE_MAX = 0.15;
/**
* How late a spot may be told: half a beat, at most `LATE_MAX`. A frame that slips loses no beat;
* after a long stall the old ones are gone, and no burst of them follows.
*/
export function lateLimit(timeline: Pick<Timeline, 'tempo' | 'meter'>): number {
return Math.min(LATE_MAX, secondsOfTicks(ticksPerBeat(timeline.meter), timeline.tempo) / 2);
}
/** What `Song position` hands on. */
export interface HeardPosition {
bar: number;
beat: number;
/** How far into the beat, 0 to below 1. */
fraction: number;
/** Seconds from the start of the song to where it stands; back to the loop's start with it. */
seconds: number;
tick: number;
/** Heard now: started, and not played to its end. */
playing: boolean;
}
/**
* Where a playing stands at `heard` seconds of its playing time, as heard: before it starts (null, or
* less than 0) where it starts, after its end at the end.
*/
export function heardPosition(timeline: Timeline, heard: number | null): HeardPosition {
const { tempo, meter, from, loop, length } = timeline;
let tick = from;
let playing = false;
if (heard !== null && heard >= 0) {
tick = playheadAt(ticksOfSeconds(heard, tempo), from, loop);
playing = !ends(timeline) || tick < length;
if (!playing) tick = length;
}
const beat = ticksPerBeat(meter);
const at = positionOf(tick, meter);
const inBeat = tick - Math.floor(tick / beat) * beat;
return {
bar: at.bar,
beat: at.beat,
fraction: Math.min(Math.max(inBeat / beat, 0), 1 - Number.EPSILON),
seconds: secondsOfTicks(tick, tempo),
tick,
playing,
};
}
/** A spot the playhead passed: the tick it is at, and how many seconds ago it was heard. */
export interface Passed {
tick: number;
late: number;
}
/**
* The ticks among the playhead's run from `low` to `high` that are spots of a kind. `lowOpen` leaves
* `low` itself out (it was told before), `highOpen` leaves `high` out (the loop's end is its start).
*/
export type SpotsIn = (low: number, high: number, lowOpen: boolean, highOpen: boolean) => number[];
/** Every multiple of `step` in the run: the beats or the bars. */
export function gridSpots(step: number): SpotsIn {
return (low, high, lowOpen, highOpen) => {
const spots: number[] = [];
let n = Math.ceil(low / step);
if (lowOpen && n * step <= low) n++;
for (; n * step < high || (!highOpen && n * step === high); n++) spots.push(n * step);
return spots;
};
}
/** Those of `ticks` in the run: the markers. */
export function tickSpots(ticks: readonly number[]): SpotsIn {
return (low, high, lowOpen, highOpen) =>
[...new Set(ticks)].filter((t) => (lowOpen ? t > low : t >= low) && (highOpen ? t < high : t <= high));
}
/** Whether it ends rather than going round. */
function ends(timeline: Timeline): boolean {
return timeline.loop === null;
}
/** One stretch of playing time the playhead runs straight through: from `start` on, at `head`, to `end`. */
interface Stretch {
start: number;
head: number;
end: number;
}
/** The stretch playing time `elapsed` (in ticks) lies in. */
function stretchAt({ from, loop }: Timeline, elapsed: number): Stretch {
if (!loop || from >= loop.end) return { start: 0, head: from, end: Infinity };
const toEnd = loop.end - from;
if (elapsed < toEnd) return { start: 0, head: from, end: toEnd };
const round = loop.end - loop.start;
const start = toEnd + Math.floor((elapsed - toEnd) / round) * round;
return { start, head: loop.start, end: start + round };
}
/** The stretch that runs up to where `stretch` begins; null for the first. */
function stretchBefore({ from, loop }: Timeline, stretch: Stretch): Stretch | null {
if (!loop || stretch.start <= 0) return null;
const toEnd = loop.end - from;
const round = loop.end - loop.start;
const start = stretch.start - round;
return start >= toEnd
? { start, head: loop.start, end: stretch.start }
: { start: 0, head: from, end: toEnd };
}
/** No more stretches than this are looked back through: a loop shorter than a frame is no tune. */
const MOST_STRETCHES = 16;
/**
* The spots the playhead passed after `before` up to `now` (seconds of playing time heard, `before`
* null where nothing was heard of it yet), newest first – only those no more than `lateLimit` ago.
* A loop's start is passed each time it comes round; nothing at or after the end of one that ends.
*/
export function passed(timeline: Timeline, before: number | null, now: number, spots: SpotsIn): Passed[] {
const low = Math.max(before ?? -Infinity, now - lateLimit(timeline));
if (!(now >= 0) || !(now > low)) return [];
const { tempo, length } = timeline;
const elapsedLow = ticksOfSeconds(low, tempo);
const high = ticksOfSeconds(now, tempo);
const found: Passed[] = [];
let stretch: Stretch | null = stretchAt(timeline, high);
let highOpen = false;
for (let n = 0; stretch && n < MOST_STRETCHES; n++) {
// What was heard at `low` was told before; where the run reaches back past a stretch's start,
// that start is among it – the very start of the playing too, before which nothing was heard.
const lowInside = elapsedLow >= stretch.start;
const headLow = stretch.head + ((lowInside ? elapsedLow : stretch.start) - stretch.start);
let headHigh = stretch.head + ((highOpen ? stretch.end : high) - stretch.start);
let open = highOpen;
if (ends(timeline) && headHigh >= length) {
headHigh = length;
open = true;
}
const ticks = spots(headLow, headHigh, lowInside, open).sort((a, b) => b - a);
for (const tick of ticks) {
const at = stretch.start + (tick - stretch.head);
found.push({ tick, late: Math.max(0, now - secondsOfTicks(at, tempo)) });
}
if (lowInside) break;
stretch = stretchBefore(timeline, stretch);
highOpen = true;
}
return found;
}
/** A beat passed, as `On beat` tells it. */
export interface PassedBeat extends Passed {
bar: number;
beat: number;
}
/** The newest beat passed, if one is still to be told. */
export function passedBeat(timeline: Timeline, before: number | null, now: number): PassedBeat | null {
const [newest] = passed(timeline, before, now, gridSpots(ticksPerBeat(timeline.meter)));
if (!newest) return null;
const at = positionOf(newest.tick, timeline.meter);
return { ...newest, bar: at.bar, beat: at.beat };
}
/** A marker passed, as `On song marker` tells it. */
export interface PassedMarker extends Passed {
id: string;
name: string;
}
/** Every marker on a tick passed, newest first; markers sharing a tick in their own order. */
export function passedMarkers(
timeline: Timeline,
before: number | null,
now: number,
markers: readonly SongMarker[],
): PassedMarker[] {
if (markers.length === 0) return [];
return passed(timeline, before, now, tickSpots(markers.map((m) => m.tick))).flatMap((spot) =>
markers.filter((m) => m.tick === spot.tick).map((m) => ({ ...spot, id: m.id, name: m.name })),
);
}
/** Of what passed, those passed last: one spot, but every marker on it. */
export function newestPassed<T extends Passed>(list: readonly T[]): T[] {
const [first] = list;
return first ? list.filter((p) => p.late === first.late) : [];
}
/** Where a change a game asks of its music lands (#338): at once, or on the next of these. */
export type Quantize = 'now' | 'beat' | 'bar' | 'marker';
export const QUANTIZE_OPTIONS: readonly { value: Quantize; label: string }[] = [
{ value: 'now', label: 'Now' },
{ value: 'beat', label: 'Next beat' },
{ value: 'bar', label: 'Next bar' },
{ value: 'marker', label: 'Next marker' },
];
export function heldQuantize(value: unknown): Quantize {
return QUANTIZE_OPTIONS.some((option) => option.value === value) ? (value as Quantize) : 'bar';
}
/** The spots a change waits for; a song without markers, or a pattern, waits for the next bar. */
export function quantizeSpots(
quantize: Exclude<Quantize, 'now'>,
meter: Meter,
markers: readonly SongMarker[],
): SpotsIn {
if (quantize === 'beat') return gridSpots(ticksPerBeat(meter));
if (quantize === 'marker' && markers.length > 0) return tickSpots(markers.map((m) => m.tick));
return gridSpots(ticksPerBar(meter));
}
/** The furthest ahead a spot is looked for on a stretch without end. */
const LOOK_AHEAD = 600;
/** The stretch that follows `stretch`; null after the last one of a playing that ends. */
function stretchAfter(timeline: Timeline, stretch: Stretch): Stretch | null {
return stretch.end === Infinity ? null : stretchAt(timeline, stretch.end);
}
/**
* The first spot at or after `elapsed` seconds of playing time, in seconds of playing time; null where
* none comes before a playing that ends does – a change then lands at once.
*/
export function nextSpot(timeline: Timeline, elapsed: number, spots: SpotsIn): number | null {
const { tempo, length } = timeline;
const low = ticksOfSeconds(Math.max(0, elapsed), tempo);
let stretch: Stretch | null = stretchAt(timeline, low);
for (let n = 0; stretch && n < MOST_STRETCHES; n++) {
const headLow = stretch.head + (Math.max(low, stretch.start) - stretch.start);
// One that runs on without end is looked through for `LOOK_AHEAD` seconds, not for ever.
let headHigh =
stretch.head + (Math.min(stretch.end, low + ticksOfSeconds(LOOK_AHEAD, tempo)) - stretch.start);
if (ends(timeline)) headHigh = Math.min(headHigh, length);
if (headHigh > headLow) {
const [first] = spots(headLow, headHigh, false, true).sort((a, b) => a - b);
if (first !== undefined) return secondsOfTicks(stretch.start + (first - stretch.head), tempo);
}
if (ends(timeline) && headHigh >= length) return null;
stretch = stretchAfter(timeline, stretch);
}
return null;
}
/** The newest bar line passed, if one is still to be told. */
export function passedBar(timeline: Timeline, before: number | null, now: number): PassedBeat | null {
const [newest] = passed(timeline, before, now, gridSpots(ticksPerBar(timeline.meter)));
if (!newest) return null;
return { ...newest, bar: positionOf(newest.tick, timeline.meter).bar, beat: 1 };
}
src/core/music/compositionAudio.ts
+244
View File
@@ -0,0 +1,244 @@
/**
* What a composition plays of its audio items (#418), as sequences the scheduler plays on the audio
* clock. Time is counted in milliseconds – ticks at `COMPOSITION_TEMPO` –, so a frame of any `fps`
* lands within a millisecond. A pattern or a song keeps its own tempo inside its item: its notes are
* laid out here in the composition's time, each with the tempo a Delay synced to beats counts by.
*
* Every item is heard on a bus of its own at its volume, its fades ramps of that bus' level, through
* the sound FX of its pattern or song; a song's tracks are buses of that item. One playing goes to
* one place of the mix, so the items are split into one sequence for each place they are sent to.
*/
import type { CompositionAsset, CompositionItem } from '../model/Asset';
import { compositionFps, compositionLength, compositionLoops, itemFades } from '../model/composition';
import { findAssetOf, type Project } from '../model/Project';
import { findSound } from '../model/sound';
import { pickedFx } from '../properties/assetFx';
import { AUTOMATION_STEP, type SequenceLane, unrolledLane } from './automation';
import { itemRoute, MASTER_MIX, type MixRoute } from './mixer';
import { fxVoiceOf } from './needs';
import { patternInstrument, patternLength, patternSequence, patternTempo } from './pattern';
import type { BusLevel } from './playbacks';
import {
clipsIn,
type LoopRange,
notesIn,
type Sequence,
type SequenceClip,
type SequenceLevel,
type SequenceNote,
} from './sequence';
import { dbToGain, songLoop, songSequence, songTempo } from './song';
import { songBuses } from './songBuses';
import { secondsOfTicks, ticksOfSeconds } from './time';
/** The tempo a composition's sequences count at: 1000 ticks to the second. */
export const COMPOSITION_TEMPO = 125;
/** Ticks of a composition's sequence: milliseconds. */
export function compositionTicks(seconds: number): number {
return Math.round(ticksOfSeconds(seconds, COMPOSITION_TEMPO));
}
/** One sequence of a composition and where it goes. */
export interface CompositionPlaying {
route: MixRoute;
sequence: Sequence;
buses: BusLevel[];
}
/** The bus an audio item is heard on, at its volume; a song's tracks go into it. */
export function audioItemBus(compId: string, itemId: string): string {
return `composition:${compId}:${itemId}`;
}
/**
* Every audio item of the composition on a track there is, as sequences – one for each place of the
* mix they go to, in the order the first item for it lies in the list. With `routed` off all go to
* the master, as one.
*/
export function compositionPlayings(
project: Project,
comp: CompositionAsset,
routed = true,
): CompositionPlaying[] {
const fps = compositionFps(comp);
const tracks = new Set(comp.tracks.map((t) => t.id));
const byRoute = new Map<string, CompositionPlaying>();
for (const item of comp.items) {
if (!item.audio || !tracks.has(item.track)) continue;
const laid = laidOut(project, comp.id, item, fps);
if (!laid) continue;
const route = routed ? itemRoute(project, comp.id, item.audio.bus, laid.routedAs) : MASTER_MIX;
const key = JSON.stringify(route);
let playing = byRoute.get(key);
if (!playing)
byRoute.set(key, (playing = { route, sequence: { notes: [], clips: [], levels: [] }, buses: [] }));
(playing.sequence.notes as SequenceNote[]).push(...laid.notes);
(playing.sequence.clips as SequenceClip[]).push(...laid.clips);
(playing.sequence.levels as SequenceLevel[]).push(...laid.levels);
if (laid.lanes?.length) playing.sequence.lanes = [...(playing.sequence.lanes ?? []), ...laid.lanes];
playing.buses.push(...laid.buses);
}
const playings = [...byRoute.values()];
for (const { sequence } of playings) {
(sequence.notes as SequenceNote[]).sort((a, b) => a.tick - b.tick || a.note - b.note);
(sequence.clips as SequenceClip[]).sort((a, b) => a.tick - b.tick);
(sequence.levels as SequenceLevel[]).sort((a, b) => a.tick - b.tick);
}
return playings;
}
/** How long one run of the composition is in ticks, and the loop it goes round where it loops. */
export function compositionSpan(
project: Project,
comp: CompositionAsset,
): { length: number; loop: LoopRange | null } {
const length = compositionTicks(compositionLength(project, comp) / compositionFps(comp));
return { length, loop: compositionLoops(comp) && length > 0 ? { start: 0, end: length } : null };
}
interface Laid {
notes: SequenceNote[];
clips: SequenceClip[];
/** Its fades, on its bus. */
levels: SequenceLevel[];
/** A song's automation (#337), round its loop as the song plays inside the item. */
lanes?: SequenceLane[];
buses: BusLevel[];
/** The asset its routing is read from: the sound of a clip, else what it plays. */
routedAs: string;
}
/**
* An audio item in the composition's ticks: where it begins and ends, and from how far into what it
* plays. An offset below 0 starts that much later in the item.
*/
function laidOut(project: Project, compId: string, item: CompositionItem, fps: number): Laid | null {
const audio = item.audio!;
const late = Math.max(0, -item.offset);
const begin = compositionTicks(item.start / fps + late);
const end = compositionTicks((item.start + item.length) / fps);
if (end <= begin) return null;
const into = Math.max(0, item.offset);
const seconds = secondsOfTicks(end - begin, COMPOSITION_TEMPO);
const bus = audioItemBus(compId, item.id);
const own: BusLevel = { name: bus, gain: dbToGain(audio.volume), pan: 0 };
const levels = fadesOf(item, fps, bus, own.gain);
if (audio.kind === 'sound') {
const ref = findSound(project, audio.asset);
if (!ref?.asset.file) return null;
const clip = {
tick: begin,
length: end - begin,
sound: audio.asset,
fx: pickedFx('', ref.asset.props),
offset: into,
volume: 1,
bus,
};
return { notes: [], clips: [clip], levels, buses: [own], routedAs: ref.asset.id };
}
const withFx = (fxId: string, tempo: number): BusLevel => {
const fx = fxVoiceOf(project, fxId, tempo);
return fx ? { ...own, fx } : own;
};
if (audio.kind === 'pattern') {
const pattern = findAssetOf(project, audio.asset, 'pattern');
if (!pattern || !patternInstrument(pattern.props)) return null;
const tempo = patternTempo(pattern.props);
const length = patternLength(pattern);
const heard = played(patternSequence(pattern), { start: 0, end: length }, tempo, into, seconds);
return {
notes: heard.notes.map((n) => ({ ...n, tick: begin + n.tick, bus })),
clips: [],
levels,
buses: [withFx(pickedFx('', pattern.props), tempo)],
routedAs: pattern.id,
};
}
const song = findAssetOf(project, audio.asset, 'song');
if (!song) return null;
const tempo = songTempo(song.props);
const heard = played(songSequence(project, song), songLoop(song), tempo, into, seconds);
// Its tracks are the item's: two items of one song do not share them.
const prefix = `song:${song.id}`;
const renamed = (name: string) => (name.startsWith(prefix) ? `${bus}${name.slice(prefix.length)}` : name);
const tracks = songBuses(project, song).map(({ name, into: to, sends, ...level }) => ({
...level,
name: renamed(name),
...(to !== undefined && { into: renamed(to) }),
...(sends && { sends: sends.map((send) => ({ ...send, to: send.to ? renamed(send.to) : bus })) }),
}));
return {
notes: heard.notes.map((n) => ({ ...n, tick: begin + n.tick, ...(n.bus && { bus: renamed(n.bus) }) })),
clips: heard.clips.map((c) => ({ ...c, tick: begin + c.tick, bus: renamed(c.bus) })),
lanes: heard.lanes.map((lane) => ({
...lane,
bus: renamed(lane.bus),
points: lane.points.map((p) => ({ ...p, tick: begin + p.tick })),
})),
levels,
buses: [withFx(pickedFx('', song.props), tempo), ...tracks],
routedAs: song.id,
};
}
/**
* The fades of an item as ramps of its bus, none without: up from silence at its start – at once
* where it has no fade in, as the round before may have faded it out – and down at its end.
*/
function fadesOf(item: CompositionItem, fps: number, bus: string, gain: number): SequenceLevel[] {
const { fadeIn, fadeOut, curveIn, curveOut } = itemFades(item);
if (fadeIn === 0 && fadeOut === 0) return [];
const tick = (frame: number) => compositionTicks(frame / fps);
const start = tick(item.start);
const end = tick(item.start + item.length);
const rise = {
tick: start,
length: tick(item.start + fadeIn) - start,
bus,
gain,
rise: true,
curve: curveIn,
};
if (fadeOut === 0) return [rise];
const fall = tick(item.start + item.length - fadeOut);
return [rise, { tick: fall, length: end - fall, bus, gain, rise: false, curve: curveOut }];
}
/**
* What a pattern or a song plays from `into` seconds on for `seconds`, round its loop, in ticks of
* the composition from 0: each note and clip with its own tempo, none ringing past the end.
*/
function played(
sequence: Sequence,
loop: LoopRange,
tempo: number,
into: number,
seconds: number,
): { notes: SequenceNote[]; clips: SequenceClip[]; lanes: SequenceLane[] } {
const from = Math.round(ticksOfSeconds(into, tempo));
const to = from + Math.ceil(ticksOfSeconds(seconds, tempo));
const span = compositionTicks(seconds);
const at = (ticks: number) => compositionTicks(secondsOfTicks(ticks - from, tempo));
const notes = notesIn(sequence, 0, loop, from, to).flatMap((n) => {
const tick = at(n.at);
const length = Math.min(compositionTicks(secondsOfTicks(n.length, tempo)), span - tick);
if (tick >= span || length <= 0) return [];
const { at: _, ...note } = n;
return [{ ...note, tick, length: Math.max(1, length), tempo }];
});
const clips = clipsIn(sequence, 0, loop, from, to, true).flatMap((c) => {
const tick = at(c.at);
const length = Math.min(compositionTicks(secondsOfTicks(c.length, tempo)), span - tick);
if (tick >= span || length <= 0) return [];
const { at: _, skip, ...clip } = c;
return [{ ...clip, tick, length, offset: c.offset + secondsOfTicks(skip, tempo), tempo }];
});
const step = ticksOfSeconds(AUTOMATION_STEP, tempo);
const lanes = (sequence.lanes ?? []).map((lane) => ({
...lane,
points: unrolledLane(lane.points, loop, from, to, step).map((p) => ({ ...p, tick: at(p.tick) })),
}));
return { notes, clips, lanes };
}
src/core/music/embedded.ts
+7 -8
View File
@@ -53,20 +53,19 @@ export function embeddedIn(project: Project, songId: string): Asset[] {
return project.assets.filter((a) => a.owner === songId);
}
/** The song an asset belongs to; undefined for a project asset. */
export function ownerSong(project: Project, asset: Asset): SongAsset | undefined {
const owner = asset.owner ? findAsset(project, asset.owner) : undefined;
return owner?.type === 'song' ? owner : undefined;
}
export function isRowHiddenWhenEmbedded(asset: Asset, key: string): boolean {
return isEmbedded(asset) && (HIDDEN_ROWS[asset.type]?.includes(key) ?? false);
}
/** The song or composition (#419) an asset belongs to; undefined for a project asset. */
export function ownerOf(project: Project, asset: Asset): Asset | undefined {
return isEmbedded(asset) ? findAsset(project, asset.owner!) : undefined;
}
/** "Song › Take 1" for an embedded asset, its own name for any other. */
export function assetTitle(project: Project, asset: Asset): string {
const song = ownerSong(project, asset);
return song ? `${song.name} › ${asset.name}` : asset.name;
const owner = ownerOf(project, asset);
return owner ? `${owner.name} › ${asset.name}` : asset.name;
}
/** The tempo and the meter of a song, as the props a pattern made in it starts with. */
src/core/music/laneHolds.ts
+165
View File
@@ -0,0 +1,165 @@
import type { FxParams } from '../soundfx/nodes';
import {
type AutomationTarget,
type LaneMark,
laneMarks,
laneRange,
laneValueAt,
paramOf,
paramsAt,
reaches,
type SequenceLane,
type SequenceLanePoint,
} from './automation';
import { type LoopRange, playheadAt } from './sequence';
/**
* A lane a game holds on a value of its own (#338): from `at` it goes from `start`, where the lane
* is then, to `value` over `fade` and stays there, its own points no longer heard – until `release`
* takes it back over its own `fade` to where its curve is by then, or a later hold lands. Seconds of
* the audio clock, values in the units of the lane.
*/
export interface LaneHold {
at: number;
fade: number;
start: number;
value: number;
release: { at: number; fade: number } | null;
}
/** How a sequence is played: from where, round what, and its playing time in ticks at a second of the audio clock. */
export interface LanePlaying {
from: number;
loop: LoopRange | null;
elapsed(at: number): number;
}
/** What the holds of a lane are kept by: the bus of the lane and its target. */
export function holdKey(bus: string, target: AutomationTarget): string {
return `${bus}\n${target}`;
}
/**
* The playing time each of the holds of a lane rules, in ticks, in the order they land: from where
* it lands to where its release is through or the next one lands; open while it holds.
*/
export function holdSpans(
holds: readonly LaneHold[],
playing: LanePlaying,
): { hold: LaneHold; start: number; end: number }[] {
return holds
.map((hold, i) => {
const { release } = hold;
const next = holds[i + 1];
const own = release ? playing.elapsed(release.at + release.fade) : Infinity;
return {
hold,
start: playing.elapsed(hold.at),
end: next ? Math.min(own, playing.elapsed(next.at)) : own,
};
})
.filter((span) => span.end > span.start);
}
/** The holds of a lane still to be planned past `until`, in ticks of the playing time. */
export function holdsAfter(holds: readonly LaneHold[], playing: LanePlaying, until: number): LaneHold[] {
const going = new Set(
holdSpans(holds, playing)
.filter((span) => span.end > until)
.map((s) => s.hold),
);
return holds.filter((hold) => going.has(hold));
}
/**
* The way of a hold as the points of a lane in ticks of the playing time: from where the lane was to
* the value, and from wherever that got to by the release back to the curve where the release ends.
*/
export function holdPoints(hold: LaneHold, lane: SequenceLane, playing: LanePlaying): SequenceLanePoint[] {
const held: SequenceLanePoint[] = [
{ tick: playing.elapsed(hold.at), value: hold.start },
{ tick: playing.elapsed(hold.at + hold.fade), value: hold.value },
];
const { release } = hold;
if (!release) return held;
const from = playing.elapsed(release.at);
const to = playing.elapsed(release.at + release.fade);
const back = laneValueAt(lane.points, playheadAt(Math.max(0, to), playing.from, playing.loop));
return [
...held.filter((point) => point.tick <= from),
{ tick: from, value: laneValueAt(held, from) },
{ tick: to, value: back },
];
}
/** Where a lane is at the playing time `elapsed`, in its units: along the hold that rules then, if one does. */
export function heldValueAt(
lane: SequenceLane,
holds: readonly LaneHold[],
playing: LanePlaying,
elapsed: number,
): number {
const span = holdSpans(holds, playing).find((s) => elapsed >= s.start && elapsed < s.end);
if (span) return laneValueAt(holdPoints(span.hold, lane, playing), elapsed);
return laneValueAt(lane.points, playheadAt(Math.max(0, elapsed), playing.from, playing.loop));
}
/**
* The marks of a lane over the playing time from `start` up to `end`, as `laneMarks` gives them, but
* along its holds where they rule: its own points between them, the way of each hold while it rules.
* Where a hold or the way back to the lane begins is a jump, so a bus is set there and does not ramp
* from whatever was planned for it last.
*/
export function heldMarks(
lane: SequenceLane,
holds: readonly LaneHold[],
playing: LanePlaying,
start: number,
end: number,
fresh: boolean,
step: number,
): LaneMark[] {
const { from, loop } = playing;
const marks: LaneMark[] = [];
let cursor = start;
let landed = fresh;
for (const span of holdSpans(holds, playing)) {
if (span.end <= cursor) continue;
if (span.start >= end) break;
if (cursor < span.start) marks.push(...laneMarks(lane, from, loop, cursor, span.start, landed, step));
const to = Math.min(end, span.end);
const way: SequenceLane = { ...lane, points: holdPoints(span.hold, lane, playing) };
marks.push(
...laneMarks(way, 0, null, Math.max(cursor, span.start), to, fresh || span.start >= start, step),
);
cursor = to;
landed = true;
}
if (cursor < end) marks.push(...laneMarks(lane, from, loop, cursor, end, landed, step));
return marks;
}
/**
* The values the parameter lanes have for what starts on `bus` at `tick` of the sequence, `elapsed`
* of the playing time, as `paramsAt` gives them: a held one along its hold.
*/
export function heldParamsAt(
lanes: readonly SequenceLane[],
holds: ReadonlyMap<string, readonly LaneHold[]>,
playing: LanePlaying,
bus: string | undefined,
tick: number,
elapsed: number,
): FxParams | undefined {
const params = paramsAt(lanes, bus, tick);
if (!params || bus === undefined || holds.size === 0) return params;
const held: Record<string, number> = { ...params };
for (const lane of lanes) {
const name = paramOf(lane.target);
const laneHolds = holds.get(holdKey(lane.bus, lane.target));
if (name === null || !laneHolds || !reaches(lane.bus, bus)) continue;
const { min, max } = laneRange(lane);
held[name] = Math.max(min, Math.min(max, heldValueAt(lane, laneHolds, playing, elapsed)));
}
return held;
}
src/core/music/level.ts
+7
View File
@@ -0,0 +1,7 @@
/** The volume of a track of a song (#334), in decibels; at the lowest it is silent. */
export const TRACK_MIN_DB = -60;
export const TRACK_MAX_DB = 12;
export function dbToGain(db: number): number {
return db <= TRACK_MIN_DB ? 0 : 10 ** (db / 20);
}
src/core/music/markers.ts
+57
View File
@@ -0,0 +1,57 @@
/**
* Markers of a song (#338): named spots on its time, where an event fires in the game and where a
* switch to the song can start. A marker sits on a whole tick; two may share one, and so may their
* names – a node names a marker by its id.
*/
import { newId } from '../ids';
export interface SongMarker {
id: string;
name: string;
tick: number;
}
const finite = (value: unknown) => (typeof value === 'number' && Number.isFinite(value) ? value : NaN);
/** Held to what a marker may be: a whole tick from 0, a name, an id no other has; in the order of their ticks. */
export function heldSongMarkers(raw: unknown): SongMarker[] {
const ids = new Set<string>();
const markers: SongMarker[] = [];
for (const entry of Array.isArray(raw) ? (raw as unknown[]) : []) {
if (typeof entry !== 'object' || entry === null) continue;
const m = entry as Record<string, unknown>;
const tick = Math.round(finite(m.tick));
if (!(tick >= 0)) continue;
let id = typeof m.id === 'string' && m.id ? m.id : newId();
while (ids.has(id)) id = newId();
ids.add(id);
const name = typeof m.name === 'string' ? m.name.trim() : '';
markers.push({ id, name: name || `Marker ${markers.length + 1}`, tick });
}
return markers.sort((a, b) => a.tick - b.tick);
}
/** "Marker 3": the first number no marker is called by yet. */
export function nextMarkerName(markers: readonly SongMarker[]): string {
const taken = new Set(markers.map((m) => m.name));
let n = 1;
while (taken.has(`Marker ${n}`)) n++;
return `Marker ${n}`;
}
export function newMarker(markers: readonly SongMarker[], tick: number, name?: string): SongMarker {
return { id: newId(), name: name?.trim() || nextMarkerName(markers), tick: Math.max(0, Math.round(tick)) };
}
/** The markers with one of them changed; the others as they are. */
export function changedMarker(
markers: readonly SongMarker[],
id: string,
change: Partial<Pick<SongMarker, 'name' | 'tick'>>,
): SongMarker[] {
return markers.map((m) => (m.id === id ? { ...m, ...change } : m));
}
export function withoutMarker(markers: readonly SongMarker[], id: string): SongMarker[] {
return markers.filter((m) => m.id !== id);
}
src/core/music/mixer.ts
+39 -12
View File
@@ -1,6 +1,13 @@
import type { SendTap } from '../graph/Graph';
import { findAssetOf, type Project } from '../model/Project';
import { BUS_FX_KEY, busVolumeOf, MASTER_BUS } from '../properties/buses';
import {
BUS_FX_KEY,
busVolumeOf,
MASTER_BUS,
MASTER_MIX,
mixBusName,
TO_COMPOSITION_BUS,
} from '../properties/buses';
import { coerceAs } from '../properties/kinds';
import type { FxVoice } from '../soundfx/nodes';
import { fxVoiceOf } from './needs';
@@ -9,8 +16,7 @@ import { sendOf } from './routing';
import { dbToGain } from './song';
import { DEFAULT_TEMPO } from './time';
/** What the sound library calls the master bus (#325); every other bus of the mix is `mix:<id>`. */
export const MASTER_MIX = 'mix';
export { MASTER_MIX, mixBusName };
/** One link of the master routing (#351) as the mix plays it: into that bus, at that level, from that tap. */
export interface MixSend {
@@ -36,11 +42,6 @@ export interface MixBus {
fx: FxVoice | null;
}
/** The name of a bus of the mix by the id of its asset; the master for '' and for `MASTER_BUS`. */
export function mixBusName(busId: string): string {
return busId === '' || busId === MASTER_BUS ? MASTER_MIX : `mix:${busId}`;
}
/**
* Where an asset goes by the master routing (#351). One the routing has no node for goes into the
* master, as every asset of a project written before there was a routing did. A link to what is
@@ -59,10 +60,13 @@ export function routeOf(project: Project, assetId: string): MixRoute {
return only && only.gain === 1 && only.tap === 'post' ? only.to : sends;
}
const OWN_BUS_TYPES: readonly string[] = ['sound', 'instrument', 'video'];
/**
* The master and every bus of the project (#325), each where the routing sends it (#351); and a bus
* of its own for each sound and instrument the routing sends anywhere but straight into one bus, for
* all its voices to go through. A sound FX that does not compile is none.
* The master and every bus of the project (#325), each where the routing sends it (#351); a bus of
* its own for each sound, instrument and video the routing sends anywhere but straight into one bus,
* for all its voices to go through; and one for each composition, for its items. A sound FX that
* does not compile is none.
*/
export function mixOf(project: Project): MixBus[] {
const fxOf = (id: string) => fxVoiceOf(project, id, DEFAULT_TEMPO);
@@ -78,7 +82,10 @@ export function mixOf(project: Project): MixBus[] {
fx: fxOf(coerceAs(asset.props[BUS_FX_KEY], 'asset', '')),
},
];
if ((asset.type !== 'sound' && asset.type !== 'instrument') || !isRouted(asset)) return [];
// A composition's bus is where its items go (#418), whatever its routing.
if (asset.type === 'composition')
return [{ name: mixBusName(asset.id), volume: 1, ...routed(routeOf(project, asset.id)), fx: null }];
if (!OWN_BUS_TYPES.includes(asset.type) || !isRouted(asset)) return [];
const route = routeOf(project, asset.id);
return typeof route === 'string'
? []
@@ -125,6 +132,18 @@ export function playedBus(project: Project, picked: string, assetId: string): st
return typeof route === 'string' ? route : mixBusName(assetId);
}
/**
* The bus the sound of each video file is heard on, by the routing of its asset: a player knows
* the file it plays, not the asset. Where two videos share a file, the first one says.
*/
export function videoBusesOf(project: Project): Map<string, string> {
const buses = new Map<string, string>();
for (const asset of project.assets)
if (asset.type === 'video' && asset.file !== '' && !buses.has(asset.file))
buses.set(asset.file, playedBus(project, '', asset.id));
return buses;
}
/** Where a playing of a pattern or a song goes: the bus a node picked, else along its routing (#351). */
export function playedRoute(project: Project, picked: string, assetId: string): MixRoute {
if (picked === MASTER_BUS) return MASTER_MIX;
@@ -132,6 +151,14 @@ export function playedRoute(project: Project, picked: string, assetId: string):
return routeOf(project, assetId);
}
/**
* Where an audio item of a composition goes: into the composition's bus for `TO_COMPOSITION_BUS`,
* else as a node's pick (`playedRoute`) – along the routing of what it plays where it picks none.
*/
export function itemRoute(project: Project, compId: string, picked: string, assetId: string): MixRoute {
return picked === TO_COMPOSITION_BUS ? mixBusName(compId) : playedRoute(project, picked, assetId);
}
/** The bus of the mix a node reaches where an empty pick is the master: Set bus volume, Bus volume. */
export function reachedBus(project: Project, picked: string): string | null {
if (picked === '' || picked === MASTER_BUS) return MASTER_MIX;
src/core/music/needs.ts
+3 -3
View File
@@ -22,9 +22,9 @@ export function fxVoiceOf(
return plan ? { plan, params, tempo } : null;
}
/** The sound FX a clip goes through. */
export function clipFx(project: Project, fxId: string, tempo: number): FxVoice {
return fxVoiceOf(project, fxId, tempo) ?? DRY_VOICE;
/** The sound FX a clip goes through, with the values its parameters start at. */
export function clipFx(project: Project, fxId: string, tempo: number, params: FxParams = {}): FxVoice {
return fxVoiceOf(project, fxId, tempo, params) ?? DRY_VOICE;
}
/** Every layer and FX of every note and every file and FX of every clip a sequence plays. */
src/core/music/ogg.ts
+6 -15
View File
@@ -4,6 +4,7 @@
* (`preSkip`) and the last page how many there are, so a decoder gives back exactly what went in,
* to the sample.
*/
import { concatBytes } from '../util/bytes';
/** Opus always counts at 48 kHz, whatever the rate of what went in. */
export const OPUS_RATE = 48000;
@@ -25,7 +26,7 @@ export interface OpusStream {
/** What a page holds at most: 255 lacing values, and so at most a packet of about 64 kB. */
const MAX_SEGMENTS = 255;
const SERIAL = 0x6d616b65;
const VENDOR = 'game-maker';
const VENDOR = 'feuerwerk';
const CRC_TABLE = (() => {
const table = new Uint32Array(256);
@@ -101,7 +102,7 @@ export function oggOpus(stream: OpusStream): Uint8Array {
if (i === stream.packets.length - 1) page(held, end, 0x04);
});
if (stream.packets.length === 0) page([], end, 0x04);
return concat(pages);
return concatBytes(pages);
}
function oggPage(
@@ -115,7 +116,7 @@ function oggPage(
for (let left = packet.length; left >= 255; left -= 255) lacing.push(255);
lacing.push(packet.length % 255);
}
const body = concat(packets);
const body = concatBytes(packets);
const page = new Uint8Array(27 + lacing.length + body.length);
const view = new DataView(page.buffer);
page.set(ascii('OggS'));
@@ -179,7 +180,7 @@ export class OggOpusReader {
private next = 0;
push(bytes: Uint8Array): void {
let data = this.held.length > 0 ? concat([this.held, bytes]) : bytes;
let data = this.held.length > 0 ? concatBytes([this.held, bytes]) : bytes;
let at = 0;
while (data.length - at >= 27) {
if (text(data.subarray(at, at + 4)) !== 'OggS') throw new Error(`no page at byte ${at}`);
@@ -209,7 +210,7 @@ export class OggOpusReader {
this.open.push(page.subarray(from, from + value));
from += value;
if (value < 255) {
this.packet(concat(this.open));
this.packet(concatBytes(this.open));
this.open = [];
}
}
@@ -254,13 +255,3 @@ function ascii(value: string): Uint8Array {
function text(bytes: Uint8Array): string {
return String.fromCharCode(...bytes);
}
function concat(parts: readonly Uint8Array[]): Uint8Array {
const out = new Uint8Array(parts.reduce((sum, part) => sum + part.length, 0));
let at = 0;
for (const part of parts) {
out.set(part, at);
at += part.length;
}
return out;
}
src/core/music/pattern.ts
+32 -1
View File
@@ -8,6 +8,7 @@ import { HIGHEST_NOTE, LOWEST_NOTE } from '../instrument/notes';
import type { PatternAsset } from '../model/Asset';
import { type PropertyBag, textOf } from '../model/Properties';
import { heldLoop, type LoopRange, type Sequence } from './sequence';
import { type Swing, swungNote } from './swing';
import { DEFAULT_SIGN, heldTempo, type Meter, signMeter, ticksPerBar } from './time';
export interface PatternNote {
@@ -26,6 +27,8 @@ export const PATTERN_BARS_KEY = 'bars';
export const PATTERN_GRID_KEY = 'grid';
export const PATTERN_SIGN_KEY = 'sign';
export const PATTERN_TEMPO_KEY = 'tempo';
export const PATTERN_SWING_KEY = 'swing';
export const PATTERN_SWING_GRID_KEY = 'swingGrid';
export const DEFAULT_BARS = 4;
export const MAX_BARS = 256;
@@ -49,6 +52,10 @@ const GRIDS: readonly (readonly [string, number])[] = [
export const GRID_OPTIONS = GRIDS.map(([value]) => ({ value, label: value }));
/** What swing (#372) pairs up: eighths or sixteenths. */
export const SWING_GRID_OPTIONS = ['1/8', '1/16'].map((value) => ({ value, label: value }));
export const DEFAULT_SWING_GRID = '1/8';
export function gridTicks(grid: string): number {
return (GRIDS.find(([value]) => value === grid) ?? GRIDS[4]!)[1];
}
@@ -84,6 +91,29 @@ export function patternInstrument(props: PropertyBag): string {
return textOf(props[PATTERN_INSTRUMENT_KEY] ?? '');
}
/** The swing it is played with (#372); null without, so a pattern with none plays as it always did. */
export function patternSwing(props: PropertyBag): Swing | null {
const percent = props[PATTERN_SWING_KEY];
const amount = typeof percent === 'number' && Number.isFinite(percent) ? Math.min(1, percent / 100) : 0;
if (!(amount > 0)) return null;
const grid = textOf(props[PATTERN_SWING_GRID_KEY] ?? DEFAULT_SWING_GRID) === '1/16' ? '1/16' : '1/8';
return { amount, pair: 2 * gridTicks(grid), bar: ticksPerBar(patternMeter(props)) };
}
/**
* A note of the pattern as the scheduler plays it: with swing heard `late` ticks after its place and
* held as long as its swung end says; without, as it is.
*/
export function playedNote(
tick: number,
length: number,
swing: Swing | null,
): { length: number; late?: number } {
if (!swing) return { length };
const swung = swungNote(tick, length, swing);
return swung.late > 0 ? swung : { length: swung.length };
}
/** Where the pattern ends, in ticks: a note starting there or later is kept, but not played. */
export function patternLength(asset: PatternAsset): number {
return patternBars(asset.props) * ticksPerBar(patternMeter(asset.props));
@@ -102,12 +132,13 @@ export function patternSequence(asset: PatternAsset): Sequence {
const instrument = patternInstrument(asset.props);
if (!instrument) return { notes: [] };
const length = patternLength(asset);
const swing = patternSwing(asset.props);
return {
notes: asset.notes
.filter((n) => n.tick < length)
.map((n) => ({
tick: n.tick,
length: Math.min(n.length, length - n.tick),
...playedNote(n.tick, Math.min(n.length, length - n.tick), swing),
note: n.note,
velocity: n.velocity,
instrument,
src/core/music/playbacks.ts
+300 -21
View File
@@ -1,9 +1,19 @@
import type { SendTap } from '../graph/Graph';
import type { FxPlan, FxVoice } from '../soundfx/nodes';
import type { AutomationTarget } from './automation';
import type { BakePlayback } from './bake';
import type { MixSend } from './mixer';
import { LEAD, type ScheduledClip, type ScheduledNote, Scheduler, type SchedulerHost } from './Scheduler';
import {
LEAD,
type ScheduledAutomation,
type ScheduledClip,
type ScheduledNote,
type ScheduledRamp,
Scheduler,
type SchedulerHost,
} from './Scheduler';
import type { LoopRange, Sequence } from './sequence';
import { ticksOfSeconds } from './time';
/** What a sequence is made of: loaded before the first of its notes and clips is planned. */
export interface SequenceNeeds {
@@ -22,6 +32,11 @@ export interface BusLevel {
sends?: readonly BusSend[];
/** The sound FX all of it goes through as one chain (#346), before its level and pan. */
fx?: FxVoice;
/**
* What its automation sets while it plays (#337): `gain`, `pan` and a parameter of `fx` are then
* only where it starts.
*/
automated?: readonly AutomationTarget[];
}
/** One send of a track (#350): into another track's bus, or the output of the song where `to` is left out. */
@@ -54,8 +69,22 @@ export interface SequencePlay {
/** Starts one note as the player was asked to, and gives back the number of its voice. */
strike(note: ScheduledNote): number;
strikeClip?(clip: ScheduledClip): number;
/** Ramps the level of one of its buses (#418). */
ramp?(ramp: ScheduledRamp): void;
/** Takes one of its buses along its automation (#337). */
automate?(automation: ScheduledAutomation): void;
/** Its bake (#340), played in place of its notes and clips where the host can. */
bake?: BakePlay;
/**
* Where it starts, in ticks, as heard `LEAD` from now: a composition's sound (#418). Where it first
* waits for its files, it starts as much further in, to keep to the clock it was asked for by.
*/
from?: number;
/**
* The second of the audio clock it is heard from (#338), where that is still to come once its
* files are there: a stinger in time with another playing (`Playbacks.landOn`).
*/
at?: number;
}
/** A bake as it is played: its file, the level it is heard at, and which stretches of it. */
@@ -72,27 +101,64 @@ export interface Runner {
readonly startedAt: number | null;
/** Played to its end, and nothing of it is heard any more. */
readonly done: boolean;
start(from?: number): void;
/** Plays from `from` ticks, heard from `at` on the audio clock where that is still to come (#338). */
start(from?: number, at?: number): void;
stop(letRing?: boolean): void;
}
export interface PlaybackHost extends Omit<SchedulerHost, 'play' | 'playClip' | 'ahead'> {
/** What a change a game asks of a playing goes by (#338): its time, where it started, the clock now. */
export interface PlayingClock {
time: PlayingTime;
/** The second of the audio clock its playing time 0 is heard at; null before it starts. */
startedAt: number | null;
now: number;
/** The soonest second of the audio clock the change can be heard from. */
soonest: number;
}
/** The second of the audio clock a change to a playing lands at, from its clock (#338). */
export type LandingAt = (clock: PlayingClock) => number;
/** Seconds a hard cut between two playings takes, so it does not click. */
export const CUT = 0.005;
/**
* The audio side of the playings. Each is reached here by its `key`, the number its buses go by: the
* same as its `id` but where it took over another one's (#338).
*/
export interface PlaybackHost extends Omit<
SchedulerHost,
'play' | 'playClip' | 'ahead' | 'ramp' | 'automate'
> {
/** Loads what that playing needs and makes its buses ready, their sound FX among them (#346). */
prepare(needs: SequenceNeeds, id: number): Promise<void>;
prepare(needs: SequenceNeeds, key: number): Promise<void>;
/** How far ahead of its time a note of that playing may be started (see `SchedulerHost.ahead`). */
ahead?(id: number): number;
ahead?(key: number): number;
/** The level of a playing, from where it is at `at` to `gain`, reached `seconds` later. */
level(id: number, gain: number, at: number, seconds: number): void;
level(key: number, gain: number, at: number, seconds: number): void;
/** Calls `then` once the audio clock reaches `at`. */
after(at: number, then: () => void): void;
/** A playing is gone: stopped, or played to its end. */
closed?(id: number): void;
closed?(key: number): void;
/**
* Nothing of the playings numbered `id` is left (#373): played to their end, or `stopped`. `name`
* is that of the one heard last, a Switch song's in place of the one it took over from.
*/
ended?(id: number, name: string, stopped: boolean): void;
/** Starts the bake of that playing to be heard from `at`, as one voice; 0 where nothing plays. */
playBaked?(id: number, bake: BakePlay, at: number): number;
playBaked?(key: number, bake: BakePlay, at: number): number;
}
export interface Playback {
id: number;
/** What its buses go by; its `id` but where it took over from another playing (#338). */
key: number;
/** The level of the whole of it, as it was asked for. */
volume: number;
/** The second of the audio clock another playing takes over from it at (#338); null while it is the one. */
until: number | null;
/** It takes over from another playing of its number, once it is heard (#338). */
successor: boolean;
/** What stands for the pattern or song, as a file stands for a sound: `pattern:<id>`, `song:<id>`. */
name: string;
owner: string;
@@ -104,6 +170,28 @@ export interface Playback {
fading: number | null;
/** Played from its bake (#340) rather than its notes. */
baked: boolean;
/** How fast, round what and to where it plays, and from where it started (#338). */
time: PlayingTime;
/** A stop reached it (#373), rather than it playing to its end. */
stopped: boolean;
}
/** The time of a playing (#338): its tempo, where it started, its loop and its length, in ticks. */
export interface PlayingTime {
tempo: number;
from: number;
loop: LoopRange | null;
length: number;
}
/** A playing and how far into its playing time it is heard now, null before it starts (#338). */
export interface HeardTime extends PlayingTime {
id: number;
/** Told apart from the playing of its number it took over from (`Playback.key`). */
key: number;
name: string;
heard: number | null;
baked: boolean;
}
/** Whether a playing goes by `name`: its own, every one of a kind for `kind:`, every one for `''`. */
@@ -124,6 +212,155 @@ export class Playbacks {
/** Starts once its files are there; a stop that came first wins. */
start(id: number, name: string, play: SequencePlay, owner: string): void {
const playback = this.make(id, id, name, play, owner, false);
const asked = this.host.now();
void this.host.prepare(play.needs, id).then(() => {
if (!this.list.includes(playback) || playback.fading !== null) return;
const from = play.from;
playback.time.from =
from === undefined ? 0 : Math.round(from + ticksOfSeconds(this.host.now() - asked, play.tempo));
playback.runner.start(playback.time.from, play.at);
// From the first note heard, which is later where the buses are (#346).
const at = playback.runner.startedAt ?? this.host.now() + LEAD;
this.host.level(id, playback.volume, at, Math.max(0, play.fadeIn ?? 0));
});
}
/**
* Plays `play` in place of the playing numbered `id` (#338), on its buses `key`, from `play.from`
* ticks: once its files are there, it is heard from the second `at` names for the playing it takes
* over from, which falls silent there – over `fade` seconds as this one rises over them, else in a
* cut. It keeps the number, so whatever knows the playing goes on reaching it. Asked again before
* that, the one still waiting is dropped and the playing heard now is the one to go. Where nothing
* of that number plays any more, it starts as soon as it can.
*/
switchTo(
id: number,
key: number,
name: string,
play: SequencePlay,
owner: string,
at: LandingAt,
fade: number,
): void {
const now = this.host.now();
const waiting = this.list.filter((p) => p.id === id && this.waits(p, now));
this.close(waiting, false);
for (const p of this.list)
if (p.id === id && p.until !== null && p.until > now) {
p.until = null;
p.fading = null;
this.host.level(p.key, p.volume, now, 0);
}
const playback = this.make(id, key, name, play, owner, true);
// What a game holds of the lanes stays held: a lane of the same song is there again.
const heard = this.list.find((p) => p.id === id && p !== playback && p.until === null);
if (heard?.runner instanceof Scheduler && playback.runner instanceof Scheduler)
playback.runner.keepHolds(heard.runner);
void this.host.prepare(play.needs, key).then(() => {
if (!this.list.includes(playback) || playback.fading !== null) return;
const from = this.heardOne(id, playback);
const now = this.host.now();
const ahead = this.host.ahead?.(key) ?? 0;
const soonest = now + LEAD + (Number.isFinite(ahead) && ahead > 0 ? ahead : 0);
const clock = from && { time: from.time, startedAt: from.runner.startedAt, now, soonest };
playback.time.from = Math.max(0, Math.round(play.from ?? 0));
playback.runner.start(playback.time.from, clock ? at(clock) : undefined);
const start = playback.runner.startedAt ?? soonest;
const over = fade > 0 ? fade : 0;
if (from) this.handOver(from, start, Math.max(over, CUT));
this.host.level(key, playback.volume, start, over);
});
}
/** The playing of that number heard now, but `besides`: the one a switch takes over from. */
private heardOne(id: number, besides: Playback): Playback | null {
return this.current().find((p) => p.id === id && p !== besides && p.until === null) ?? null;
}
/** Falls silent from `at` over `fade` seconds and goes then: the one taken over from (#338). */
private handOver(playback: Playback, at: number, fade: number): void {
const end = at + fade;
playback.until = at;
playback.fading = end;
this.host.level(playback.key, 0, at, fade);
this.host.after(end, () => {
if (this.list.includes(playback) && playback.fading === end) this.close([playback], false);
});
}
/** The clock of the playing of that number heard now (#338), and whether it is a bake; null where none is. */
clockOf(id: number): (PlayingClock & { baked: boolean }) | null {
const playing = this.current().find((p) => p.id === id && p.until === null && !this.waits(p));
if (!playing) return null;
const now = this.host.now();
const { time, baked } = playing;
return { time, startedAt: playing.runner.startedAt, now, soonest: now + CUT, baked };
}
/**
* The second of the audio clock `at` names on the clock of the playing of that number heard now
* (#338), no sooner than something started now can be heard: where a stinger in time with it
* starts. Null where none is heard.
*/
landOn(id: number, at: LandingAt): number | null {
const clock = this.clockOf(id);
if (!clock) return null;
const soonest = clock.now + LEAD;
return Math.max(soonest, at({ ...clock, soonest }));
}
/**
* Holds a lane of the playing of that number on `value` from the second `at` (#338), as
* `Scheduler.holdLane` does: the one heard now and one waiting to take over from it alike. Whether
* any of them has that lane; a bake has none.
*/
holdLane(
id: number,
bus: string,
target: AutomationTarget,
value: number,
at: number,
fade: number,
): boolean {
let found = false;
for (const p of this.named(id))
if (p.runner instanceof Scheduler && p.runner.holdLane(bus, target, value, at, fade)) found = true;
return found;
}
/** Lets a held lane of the playing of that number go from the second `at` (#338), see `Scheduler.releaseLane`. */
releaseLane(id: number, bus: string, target: AutomationTarget, at: number, fade: number): boolean {
let found = false;
for (const p of this.named(id))
if (p.runner instanceof Scheduler && p.runner.releaseLane(bus, target, at, fade)) found = true;
return found;
}
/** The playings of that number, the one heard now and one waiting to take over from it (#338). */
named(id: number): readonly Readonly<Playback>[] {
return this.current().filter((p) => p.id === id);
}
/** Whether the buses `key` belong to a playing still to be heard. */
holds(key: number): boolean {
return this.current().some((playback) => playback.key === key);
}
/** A successor not heard yet: waiting on its files, or on the second it takes over at. */
private waits(playback: Playback, now = this.host.now()): boolean {
const at = playback.runner.startedAt;
return playback.successor && (at === null || at > now);
}
private make(
id: number,
key: number,
name: string,
play: SequencePlay,
owner: string,
successor: boolean,
): Playback {
const planned: ScheduledNote[] = [];
const clips: ScheduledClip[] = [];
const keep = <T>(list: T[], item: T) => {
@@ -131,24 +368,27 @@ export class Playbacks {
if (list.length > KEPT) list.shift();
};
const strikeClip = play.strikeClip;
const ahead = () => this.host.ahead?.(id) ?? 0;
const ahead = () => this.host.ahead?.(key) ?? 0;
const { bake } = play;
const playBaked = this.host.playBaked?.bind(this.host);
const runner: Runner =
bake && playBaked
? new BakedRun(this.host, ahead, (at) => playBaked(id, bake, at), bake.playback.end !== null)
? new BakedRun(this.host, ahead, (at) => playBaked(key, bake, at), bake.playback.end !== null)
: new Scheduler(
{
...this.host,
ahead,
play: (note) => (keep(planned, note), play.strike(note)),
...(strikeClip && { playClip: (clip: ScheduledClip) => (keep(clips, clip), strikeClip(clip)) }),
...(play.ramp && { ramp: play.ramp }),
...(play.automate && { automate: play.automate }),
},
() => play.sequence,
{ tempo: play.tempo, loop: play.loop, end: play.length },
);
const playback: Playback = {
id,
key,
name,
owner,
runner,
@@ -156,15 +396,14 @@ export class Playbacks {
clips,
fading: null,
baked: runner instanceof BakedRun,
volume: play.volume ?? 1,
until: null,
successor,
time: { tempo: play.tempo, from: 0, loop: play.loop, length: play.length },
stopped: false,
};
this.list.push(playback);
void this.host.prepare(play.needs, id).then(() => {
if (!this.list.includes(playback) || playback.fading !== null) return;
runner.start(0);
// From the first note heard, which is later where the buses are (#346).
const at = runner.startedAt ?? this.host.now() + LEAD;
this.host.level(id, play.volume ?? 1, at, Math.max(0, play.fadeIn ?? 0));
});
return playback;
}
/** Whether that one is still to be heard: waiting on its files, playing, fading, or its last notes ringing. */
@@ -172,6 +411,27 @@ export class Playbacks {
return this.current().some((playback) => playback.id === id);
}
/**
* Every playing with how far into its playing time it is heard now: the audio clock less the
* output latency, from the moment its first note is heard (#338). With none playing the clock is
* not asked: asking for it makes the audio context, which a page may make only after a gesture.
*/
times(): HeardTime[] {
const list = this.current();
if (list.length === 0) return [];
const heard = this.host.now() - this.host.latency();
// Of a switch (#338) one is heard at a time: the old one up to where the new one is.
const told = list.filter(
({ until, successor, runner }) =>
(until === null || heard < until) &&
(!successor || (runner.startedAt !== null && heard >= runner.startedAt)),
);
return told.map(({ id, key, name, runner, time, baked }) => {
const at = runner.startedAt;
return { id, key, name, ...time, heard: at === null ? null : heard - at, baked };
});
}
/** Those that go by `name` (see `goesBy`); with `fade` they fall silent over that many seconds first. */
stopNamed(name: string, letRing = false, fade = 0): void {
this.stopWhere((playback) => goesBy(playback.name, name), letRing, fade);
@@ -195,13 +455,14 @@ export class Playbacks {
private stopWhere(going: (playback: Playback) => boolean, letRing: boolean, fade: number): void {
const stopping = this.list.filter(going);
for (const playback of stopping) playback.stopped = true;
if (!(fade > 0)) return this.close(stopping, letRing);
const now = this.host.now();
const end = now + fade;
for (const playback of stopping) {
if (playback.fading !== null && playback.fading <= end) continue;
playback.fading = end;
this.host.level(playback.id, 0, now, fade);
this.host.level(playback.key, 0, now, fade);
this.host.after(end, () => {
if (this.list.includes(playback) && playback.fading === end) this.close([playback], false);
});
@@ -212,10 +473,27 @@ export class Playbacks {
private close(going: readonly Playback[], letRing: boolean): void {
if (going.length === 0) return;
this.list = this.list.filter((playback) => !going.includes(playback));
const ended = this.endedOf(going);
for (const playback of going) {
playback.runner.stop(letRing);
this.host.closed?.(playback.id);
this.host.closed?.(playback.key);
}
for (const { id, name, stopped } of ended) this.host.ended?.(id, name, stopped);
}
/**
* The numbers nothing is left of once `going` goes (#373). Of a switch (#338) the one taken over
* from goes first and the number lives on in the other; where both go, it is named by the newer
* one if that had started.
*/
private endedOf(going: readonly Playback[]): { id: number; name: string; stopped: boolean }[] {
return [...new Set(going.map((playback) => playback.id))].flatMap((id) => {
if (this.list.some((playback) => playback.id === id)) return [];
const ones = going.filter((playback) => playback.id === id);
const started = ones.filter((playback) => !playback.successor || playback.runner.startedAt !== null);
const last = started[started.length - 1] ?? ones[0]!;
return [{ id, name: last.name, stopped: ones.some((playback) => playback.stopped) }];
});
}
}
@@ -246,10 +524,11 @@ class BakedRun implements Runner {
return this.anchor !== null && this.ends && !this.host.playing(this.voice);
}
start(): void {
/** From its start: a bake plays from nowhere else. Heard from `at` where that is still to come. */
start(_from?: number, at = -Infinity): void {
this.stop();
const ahead = this.ahead();
this.anchor = this.host.now() + LEAD + (Number.isFinite(ahead) && ahead > 0 ? ahead : 0);
this.anchor = Math.max(this.host.now() + LEAD + (Number.isFinite(ahead) && ahead > 0 ? ahead : 0), at);
this.voice = this.play(this.anchor);
}
src/core/music/projectRouting.ts
+26 -10
View File
@@ -1,8 +1,8 @@
/**
* The master routing of a project (#351): a graph with a node per sound, instrument, pattern, song and
* bus, and the master, its links the sends from one to the next. Its nodes follow the assets: one
* that comes gets a node of its own and the links most others of its type have, one that goes takes
* its node along.
* The master routing of a project (#351): a graph with a node per sound, instrument, pattern, song,
* video with sound, composition and bus, and the master, its links the sends from one to the next.
* Its nodes follow the assets: one that comes gets a node of its own and the links most others of
* its type have, one that goes takes its node along.
*/
import { createEmptyGraph, type Graph, type GraphEdge, type GraphNode } from '../graph/Graph';
import { newId } from '../ids';
@@ -16,7 +16,15 @@ import { isEmbedded } from './embedded';
import { addLink, heldSend } from './routing';
import { ROUTED_ASSET, ROUTED_KIND, ROUTED_MASTER, ROUTED_NAME } from './routingNodes';
const ROUTED_TYPES: readonly AssetType[] = ['sound', 'instrument', 'pattern', 'song', 'bus'];
const ROUTED_TYPES: readonly AssetType[] = [
'sound',
'instrument',
'pattern',
'song',
'video',
'composition',
'bus',
];
/** What has its own sound FX and level on a bus of its own, and so may send from before them. */
const TAPPED_TYPES: readonly AssetType[] = ['bus', 'song'];
@@ -29,8 +37,9 @@ const ROW_STEP = 80;
const COLUMN_OF = (type: string) => (type === 'bus' ? 1 : 0);
const MASTER_COLUMN = 2;
/** A video only where it has sound: one without has nothing to send. */
export function isRouted(asset: Asset): boolean {
return ROUTED_TYPES.includes(asset.type) && !isEmbedded(asset);
return ROUTED_TYPES.includes(asset.type) && !isEmbedded(asset) && (asset.type !== 'video' || asset.audio);
}
/** Whether links out of an asset of this type may take the sound before its sound FX or its level. */
@@ -132,9 +141,11 @@ export function heldRouting(graph: Graph | undefined, assets: readonly Asset[],
}
/**
* The links a new sound, instrument, pattern or song takes, so it is heard where the others are: those
* of the one whose targets most others of its type share too – the latest of them on a tie –, or of
* all that plays where nothing of its type is linked yet. Empty where nothing is, and for a bus.
* The links a new sound, instrument, pattern, song, video or composition takes, so it is heard where
* the others are: those of the one whose targets most others of its type share too – the latest of
* them on a tie –, or of all that plays where nothing of its type is linked yet. Videos, compositions
* and the music go by each other apart: a video where no other is linked goes to the master, not
* where the sounds go, and so does a composition, a mix of its own. Empty where nothing is, and for a bus.
*/
function linksLikeTheirs(
kind: string,
@@ -159,12 +170,17 @@ function linksLikeTheirs(
};
const playing = nodes.filter((node) => {
const of = routedKind(node);
return of !== '' && of !== 'bus' && out.has(node.id);
return of !== '' && of !== 'bus' && peersOf(of) === peersOf(kind) && out.has(node.id);
});
const own = playing.filter((node) => routedKind(node) === kind);
return pick(own.length > 0 ? own : playing);
}
/** Whose links a new node may take: those of its own kind, else of what plays alike. */
function peersOf(kind: string): string {
return kind === 'video' || kind === 'composition' ? kind : 'music';
}
/** A link like another, out of a new node: as loud, and after the sound FX where it cannot tap before. */
function copiedLink(edge: GraphEdge, from: string, kind: string): GraphEdge {
const raw = heldSend(edge.send);
Loaded 100 of 589 files, more files were not shown because too many files have changed in this diff. Show more