77ac9d1aafd21272e4b43b44f7b1da4af30c5aff
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  adminandClaude Opus 5.5
|
77ac9d1aaf |
Play sounds, patterns and songs on the tracks of a composition (#418)
A trailer needs its music and effects on the same timeline as its pictures. Until now a composition could only place objects, so sound had to be started by hand from a behavior and drifted against the picture. - Audio items: a sound, a clip of a sound, a pattern or a song dragged from the asset panel onto a track becomes an item as long as one pass of it. It may share a track with object and video items and overlap them. - Editing works as for other items: move, resize, trim on the left (moves the offset), fit to media, delete, copy and paste, box selection. Sounds draw their waveform, the colour tells the kind. - The inspector shows what it plays, start, length, offset, volume (-60 to +12 dB) and bus. The bus is "From asset", the master or one of the buses, as asked for in the issue. - An item longer than its material plays in rounds. Sounds and patterns start over, a song runs on in its loop. Loop marks on the item follow the same rule, a song's first mark sits at its loop end. - compositionAudio.ts turns the items into sequences, each with its own tempo. CompositionSound plays them, in the tab and in a game. - In the tab the items are heard from the playhead on, straight into the master like every tab. A scrub is silent. Playing a song or pattern in its own tab stops the composition, and the other way round. - In a game they are planned on the audio clock for the moment the composition clock reaches them, with the scheduler's LEAD made up. While the clock stands (waiting for a video, paused, placement off) they are silent and resume at the right spot. Found along the way: - The asset panel never let a song out of its list, so a song could not be dragged anywhere. Song and scene tab reject it as they reject a pattern. - The asset panel scrolled to the selected asset on every rebuild of its list, so folding a group far below the selection jumped back up. Only a new selection scrolls now. - At the end of the list, folding a group pulled every row down by the rows that went away. ScrollContainer has a new opt-in holdOffset, set only in the asset panel, that keeps the offset until the user scrolls up. Limits: a time scale other than 1 or a frame longer than 0.1 s moves the picture and not the sound. A baked song plays its notes here, not the bake. Tests: - tsc, eslint, Prettier clean. Gate on melektaus-atx green, 309 files, 4788 tests (4772 plus 16 new). - Unit: composition-audio.test.ts 13 new, scroll-container 3 new. Counter-checks: without catching up on load time or without the silence on pause the unit tests go red. - e2e 68h: the sound is planned 1.2 ms after the composition reaches 0.5 s. Without the LEAD make-up it is 54 ms late and red. - e2e 04 and 69 ALL PASSED, 86 checks. 69o-ii drags a pattern and a song onto a track, 20 frames each. - e2e 04: folding Sound at the end of the list keeps its header at y 234. With the old panel it jumps to 984 and the check is red. - Not run in a browser: the loop marks of a song item, copy and paste of audio items, the bus choice. - README and wiki count 2072 checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
b519cd192f |
Send a track, a pattern item or a whole song through a sound FX (#346)
A sound FX could only be put on a single sound. Now a track, a pattern item, a Play pattern and a Play song send everything they play through one chain of it, as an insert in a DAW. A Play note gets one of its own. - Track: an FX box in the head next to M and S opens a menu of the project's sound FX, and the track's inspector has the row too. Order: items, track FX, volume, pan. - Pattern item: Sound FX in its inspector. Its notes play on a bus of their own into the track's bus. - Play pattern and Play song take fx and fx params. The chain sits on the master of the playing, after the tracks and before the level and the fades. Set sound FX parameters reaches it by the playback. - Play note takes fx and fx params next to the instrument's params. The FX parameters go by "fx:" so the two sets stay apart. Set sound FX parameters on a note changes a name both have in both. - A Pitch on a bus is a new PitchShifter: the phase vocoder of the PitchPlayer, fed while it plays. It cannot look ahead and is exactly one frame late at every pitch, 2048 samples. - That latency is made up. The other way of a Mix is held back by a delay of the same length. Every note and clip starts as much earlier as the buses it passes are late. The scheduler plans that much further ahead, and the first note waits that much after Play. - The Song tab waits for the chains before it plays and lets its buses go a moment after Stop. A playing in a game lets them go after its echoes and what the shifters still hold. - songBuses moved to its own file. Kept in song.ts it pulled the FX compiler into the node registry through an import cycle. Decisions of the user: one chain for the whole (1a), FX everywhere asked for (2a), Set sound FX parameters on playbacks (3a), and a pitch shifter of its own with exact latency compensation for a Pitch on a bus. Tests: - Unit pitch-shifter (new): 11 green. At 0 semitones the output is the input 2048 samples later, off by at most 1.2e-7. - Unit sound-library: 84 green, 8 new. Counter-checks (no early start, no Mix delay, chain made every time, tune misses the master) each turned their test red. - Unit bus-fx (new): 6 green, 4 counter-checks red. Scheduler ahead: 1 new, its counter-check red. - e2e 65 (melektaus-atx): ALL PASSED, 35 checks, 4 new, first run. Chrome at 44.1 kHz: the bus was 0.046440 s late, the note started exactly that much early. Offline the shifter moved 1 kHz to 2000 Hz and passed a click 2048 samples later. - Gate (melektaus-atx): CHECK EXIT 0, 259 files, 4021 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
2fdf97f7de |
Record a take from the microphone in the Song tab (#342)
A song could only be filled with sounds that were made somewhere else. Record plays the song from the cursor and takes the input meanwhile, so a part played along lands on the bar it was played to. - An audio worklet takes the input's samples on the clock the song plays on. Each run carries the frame it was taken at, so the take is cut where the song was heard to start. The output's and the input's latency are taken off, as far as the browser reports them. - The take goes up as mono 16-bit WAV. Sound, a new track "Recording" under the track clicked last and the item at the cursor are one undo step. The sound is named "Song – Take 1", 2, 3 and so on. - While recording the take grows red with its waveform on its track, and a meter beside the input dropdown shows the level. The lanes take no edits then, only a middle-drag pans. - The loop is not used while recording. The song plays once through and on past its end until Stop, Play or Space keep the take. Esc throws it away and nothing goes up. - Echo cancelling, noise suppression and auto gain are off, nothing is played back. The input picked is kept in localStorage. - A take stops by itself at the size a sound may have. A failed upload offers Retry, Download WAV and Discard. - The API takes sounds up to 256 MB and bodies of the same size. nginx and PHP were raised to match outside the repo (body 256m, upload and post 256M, memory_limit 512M). - The pure rules (cut, cap, WAV header, name, track) live in core/music/take.ts. The scheduler tells where it started on the audio clock. - The e2e browser gets chromium's fake microphone and skips the permission prompt. Each run leaves its take in the sound store. Found on the way: - The header was too full with the new widgets and cut the track and item count on the right. Input dropdown and meter are narrower now. Tests: - Unit take.test.ts: 7 new, svg-icons adjusted to 8 toolbar icons - e2e 65 (melektaus-atx): ALL PASSED – 31 checks (+4) - Gate (melektaus-atx): CHECK EXIT 0, 252 files, 3923 tests - Probe on stein2: a 3 s take of the test tone, peak 32767, 285100 bytes, waveform seen in the screenshot Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
8f9c250eb6 |
Lay patterns and sounds out on tracks and play them as a song #334
Patterns could only be heard one at a time, in their own tab or from a behavior. A song lays patterns and sounds out on tracks along one time axis and plays them through the scheduler of #332, the third piece of the music tools of #330. - New asset type song under the heading Songs, with four empty tracks, 120 BPM in 4/4 and a grid of a bar. Tempo, meter and grid are rows in the inspector. Tracks, items and the loop are the asset's data, held to their rules on load and in every command. - A track has a name, volume, pan, mute and solo. Each is heard on a bus of its own, so a change reaches what is sounding at once. - A pattern item repeats its pattern for as long as it is, with a transpose and another instrument if wanted. A sound item plays its sound as a clip at its own speed, round and round, through a sound FX if wanted. Both have a volume on top of their track's. - Cut back at the left end or split, an item starts as far into what it plays as the cut is: ticks into a pattern, seconds into a sound. - The tab shows the heads with M, S, volume and pan, the ruler of the Pattern tab and the items, a pattern with its notes, a sound with its waveform, a missing asset red. Patterns and sounds are dragged in from the Assets panel. Body, right and left end move, lengthen and cut back, Alt leaves the grid. Ctrl+C/V/D/A, Ctrl+E splits where playing starts, Delete, arrows. A double-click opens a pattern item in its tab. - What is picked is a new selection of its own, so the inspector shows the item or the track, with a button to delete it. - The scheduler plans clips beside notes. Where the playhead starts or jumps back inside a clip, it is taken up that far into its sound. A clip fades out at the end of its item or of the loop. - The editor plays one piece at a time: a pattern started stops the song and the other way round. - The rules are pure functions in core/music/song, editor/song/ songView and songEdit. Findings: - The suites of |
||
|
adminandClaude Opus 5.5
|
2a3b90b017 |
Play and stop a pattern from a behavior in the game #336
A pattern could only be heard in its tab. Songs come with #334, and only then can #336 play them in a game. Until then a game plays a single pattern, looped or once, through two new nodes. - Play pattern plays the whole pattern through its instrument at its own tempo. The loop on the ruler is only for editing. Inputs are pattern, volume, loop (on by default) and detached. It gives out playback, 0 where there is no instrument or no note. - Stop pattern stops one pattern, every pattern for none, or one playback by its number. Let ring lets notes with an Envelope fall away in their release. - A playing is numbered and owned like a voice. Destroying the object, Stop sound with nothing named and the end of the run stop it. Stop sound with a voice takes a playback too. - Each playing runs on a scheduler of its own (core/music/playbacks). It waits for its files first, and a stop that came earlier wins. - The scheduler has an end for a sequence that does not loop. It stops waking there and is done once its last notes are. - The worker clock moved to viewport/wake.ts, so SoundLibrary can use it without importing music.ts. Tests: - pattern-play.test.ts (new): 8 tests, node catalog +7 cases - e2e 64 (melektaus-atx): ALL PASSED, 23 checks (+3) - Gate (melektaus-atx): CHECK EXIT 0, 248 files, 3854 tests - Counter-checks: no end in the scheduler, and no owner for the playing, each fail 1 test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
23bdc3f775 |
Play notes in time on the audio clock with a scheduler #332
Patterns and songs (#333, #334) have to play sample-accurate in time,
whatever the frame rate does. What is counted in game time runs away
from the sound. This lays the ground: musical time, a scheduler that
plans notes ahead on the audio clock, and a player that starts and ends
a note at a given second. Nothing in the editor plays a sequence yet,
the check scripts drive it through `__gm.music`.
- Ticks, 480 to the quarter as in MIDI files (decided in #332).
Triplets and quintuplets land on whole ticks. Any meter, a beat being
the note value under the line. The tempo always counts quarters.
- `core/music/sequence.ts` is a pure rule: the notes starting in a
window of playing time. Across the end of a loop it looks into the
next round, so the seam has no gap. A note past the loop end ends
there. Windows are whole ticks, and the seam is stepped exactly, so
no rounding can drop the first note of a round.
- `core/music/Scheduler.ts` plans the next 100 ms every 25 ms through a
host, so it is tested with a clock of its own. The first note sounds
50 ms after the start. Changes to the sequence are heard from the
next window on. Stop and seek fade what it started. A new tempo or
loop counts from what is planned, and the playhead keeps its old
pace until the clock gets there.
- The position is the audio clock less `outputLatency`, as bar, beat
and tick.
- The beat comes from a Web Worker (`viewport/musicTimer.ts`), which
keeps its pace in a hidden tab.
- The sound library starts a note at `timing.at`: a buffer source by
`start(when)`, the pitch worklet by counting down inside the render
block it falls in. A note's end is set on the clock at once, its
fade to the sample. Leaving a loop still waits for the page.
- Delay synced to beats (decided in #332, noted from #327): a setting
`Sync` (Seconds / Beats), `time` then in quarter notes, held to the
2 s of the line. Play sound and Play note have an input `tempo`,
120 BPM where nothing is connected. A scheduled note carries the
tempo of its sequence.
- Swing is left for #333/#334, as decided.
Also fixes the red suite of
|