b50c7dbc35421cc340c04cd6099eda64bafc2d49
8
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  adminandClaude Opus 5.5
|
b5e071ca68 |
Let the agent read what is heard in the running game (#436)
The agent has no ears. It could start music and sounds, but not tell whether anything reached the speakers: a song without an instrument, a bus at volume 0 or a browser that holds the audio clock all looked the same from outside. - get_runtime has a section audio. Under levels it gives the peak and RMS in dB of the master, each bus of the mix, each pattern, song or composition playing and the tracks of a song. Below -90 dB it says silent. - The levels come from a tap of their own after the level and pan of each bus, an AnalyserNode per side as for the meters of the Song tab. It looks back about 0.7 s, so the gap between two notes is not silence. A pause holds the values, which fits play with until. - Under voices it names what sounds now: a sound by its asset, a note of an instrument with its note name, each with its bus and object. Next to them planned, playing, waiting (bytes not here yet) and failed. - Under clock it gives the state of the audio clock, with a hint when it stands still while the game runs. The state is told as it was before a pause, else the pause would hide that case. - The buses are tapped only while the agent is switched on. - The note now travels with a voice from the Start of a pattern or song to the SoundLibrary, so a voice can be named by it. - tools.json and the relay's instructions tell the agent to check under audio that music and sounds are really heard. Found on the user's project: started right after F5, before a click in the page, the browser kept the audio clock suspended for the whole run. Nothing was heard and the notes stayed planned. That is what clock now shows. Tests: - Unit: sound-library (taps, levels, voices, clock), agent-play (audioView). Pattern and song tests expect the note in their timing. - 9 counter-checks, 8 red. Not dropping a tap with its bus stays green, as the bus is cut off and gone anyway. - e2e 71 (melektaus-atx): 28/28. New 71h4: a sound on the master and one on a bus at volume 0, both voices named, the master loud, the bus silent, the values held in the pause. - Gate (melektaus-atx): CHECK EXIT 0, 327 files, 5037 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
84cd36f902 |
Bake a pattern or song to Opus, played in its place by a game (#340)
A pattern or song played by a game runs every note through the scheduler, the instruments and their FX, live. Bake works that out once, keeps it as a file on the asset, and Play pattern and Play song play the file instead, as one voice, for as long as nothing it is made of has changed. The tabs always play live. - Bake is an icon in the header of the Pattern and the Song tab, lit while the bake holds. A click bakes, cancels a bake that runs, or offers Bake again and Remove bake. The toolbar shows the progress. The info of the tab says "baked" or "bake outdated". - The bake holds everything up to the output of the pattern or song. The sound FX of the node, fx params, volume, fade in, bus, Master routing and stopping with a fade stay live. - A print of the worked-out plan decides whether it holds: notes and clips, tempo, length, loop, tracks, the build of each instrument for each note, sound FX, files, cuts and levels. A rename keeps it, an undo back to the baked state makes it hold again. - The file holds the straight run and then one more round of the loop as heard the second time, tails included. Looped, it plays to the loop end and then round that round, to the sample. Once, it fades out over its last 10 ms. Let ring fades it out over 0.3 s. - It is worked out by a second SoundLibrary on an OfflineAudioContext, its scheduler woken by suspends of the offline clock, and what reaches the master is taken. A worklet probe tells whether that was mono, so a mono bake goes on exactly as the notes did. - Opus in Ogg, 160 kbit/s stereo or 96 kbit/s mono, encoded with WebCodecs, the Ogg pages written by hand. Normalized to full scale, the peak kept and played back. - The file goes up into the sounds of the account as "[bake] name", which the sound picker leaves out. An old bake file is deleted once the project is saved without it. Only a bake that holds is loaded. - Ogg files are decoded at 48 kHz: resampled to 44.1 kHz, the loop of a bake lost a sample a round. - Baking and removing a bake are no undo steps, like the EQ's zoom. Findings: - Chrome's AudioEncoder refuses opus.format 'ogg' (NotSupportedError) but hands out the OpusHead with a pre-skip of 312 for 'opus'. - The Song tab's header is full at 1280 px: a captioned Bake pushed Fit off the tab. Hence the icon. Tests: - Unit: tests/core/ogg.test.ts (6), tests/core/bake.test.ts (16), four new in sound-library.test.ts, one in song-play.test.ts. One test cuts, reverses and fades a clip on a song's track: each change outdates the bake, each undo makes it hold again. - e2e 64 (melektaus-atx): ALL PASSED, 36 checks (6 new). Live and baked rendered offline differ by at most 0.06 dB over 114 windows of 20 ms, two loop ends among them. e2e 65 had no red check with the icon in its header. - Gate (melektaus-atx): CHECK EXIT 0, 277 files, 4281 tests. - Counter-checks: print without instrument builds, print taking the sound's file instead of the clip's cut, round taken from the first pass, bake voice without its looping source, Play pattern or Play song ignoring the bake: one unit test red each. Pre-skip set to 0: the e2e comparison 4.8 dB off, only that check red. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
25084f8576 |
Refactoring round 10: split SoundLibrary, fold sound duplicates (#85)
Refactoring round 10 covers everything since round 9 ( |
||
|
adminandClaude Opus 5.5
|
daeffe0f9d |
Route the tracks of a song through a graph (#350)
Every track of a song went straight into its output. Now the Song tab has a second view, Routing: a graph with a node per track and one Output, the links between them sends. A track can go into other tracks, into several at once, and a link has its own volume and the point of the track it taps. - The song asset has a graph like an instrument (SongTrack, SongOutput). heldSongGraph keeps it in step with the tracks on load and in SetSongCommand: a node per track named after it, one output, a new track and every track of an old song linked to the output. Links of gone tracks, doubled links and links closing a loop are dropped. - A link carries an optional send (volume in dB, tap). The inspector shows Volume and Mode for a picked link of a song: Post volume (Post pan), Pre volume (Post FX), Pre volume (Pre FX). SetSendCommand. - NodeDef has two new flags. `fixed` nodes are never added or deleted by hand. Links between `acyclic` nodes may not close a loop, checked in canConnect, dropTargetPort and ConnectCommand. - songBuses gives each track its sends instead of one `into`, and a bus `song:<song>:<track>:#items` for its own items. That bus is what solo closes on a track that only passes a soloed one on. - Mute and solo now act on the sends (trackGates). A muted track passes nothing on, on any tap. A soloed track is heard with what goes into it, through the tracks it goes through, their own items silent. Only sends on a way through a soloed track stay open, so a track going into it and into the output is heard through it only. - SoundLibrary: a bus has either `into` or sends. Each send is tap, delay, level, target. busLatency takes the latest way (arrivalOf), and each send is held back so all ways of a track arrive together (sendDelays). Taps are wired again whenever a chain comes or goes. - The Song tab has Timeline and Routing in its header. The routing uses GraphCanvas with an empty palette. A track linked nowhere shows "Goes nowhere". Delete takes links, never a track node. - Game playings route the same way, the output being the master of the playing. Decided without asking: a send ranges -60 to +12 dB like a track, default 0 dB Post. Mute closes the sends, the track's own level stays. Found on the way: the inspector built the Mode row with the send as it was when the rows were made. Setting Volume and then Mode wrote the old volume back. Both rows now read the current send when they change. Tests: - Unit song-routing (new): 13 green. Sound library: 4 new, 94 green. 11 counter-probes (loop on load, canConnect, solo, solo past the soloed track, mute, preFx latency, graph sync, delete guard, delays, taps, glide), each red. - Existing tests adjusted to the new bus names and send shape: song, bus-fx, song-play, graph registry, edge description. - e2e 65 on melektaus-atx: ALL PASSED, 43 checks (5 new). The first run caught the inspector bug. Counter-probe with the old closure: 2 FAILED. - Gate on melektaus-atx: typecheck, lint, format clean, 2 tests red (registry rule, edge shape), fixed and rerun locally: 56 green. - Not rerun after the per-send solo fix: e2e 65 and the gate. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
b519cd192f |
Send a track, a pattern item or a whole song through a sound FX (#346)
A sound FX could only be put on a single sound. Now a track, a pattern item, a Play pattern and a Play song send everything they play through one chain of it, as an insert in a DAW. A Play note gets one of its own. - Track: an FX box in the head next to M and S opens a menu of the project's sound FX, and the track's inspector has the row too. Order: items, track FX, volume, pan. - Pattern item: Sound FX in its inspector. Its notes play on a bus of their own into the track's bus. - Play pattern and Play song take fx and fx params. The chain sits on the master of the playing, after the tracks and before the level and the fades. Set sound FX parameters reaches it by the playback. - Play note takes fx and fx params next to the instrument's params. The FX parameters go by "fx:" so the two sets stay apart. Set sound FX parameters on a note changes a name both have in both. - A Pitch on a bus is a new PitchShifter: the phase vocoder of the PitchPlayer, fed while it plays. It cannot look ahead and is exactly one frame late at every pitch, 2048 samples. - That latency is made up. The other way of a Mix is held back by a delay of the same length. Every note and clip starts as much earlier as the buses it passes are late. The scheduler plans that much further ahead, and the first note waits that much after Play. - The Song tab waits for the chains before it plays and lets its buses go a moment after Stop. A playing in a game lets them go after its echoes and what the shifters still hold. - songBuses moved to its own file. Kept in song.ts it pulled the FX compiler into the node registry through an import cycle. Decisions of the user: one chain for the whole (1a), FX everywhere asked for (2a), Set sound FX parameters on playbacks (3a), and a pitch shifter of its own with exact latency compensation for a Pitch on a bus. Tests: - Unit pitch-shifter (new): 11 green. At 0 semitones the output is the input 2048 samples later, off by at most 1.2e-7. - Unit sound-library: 84 green, 8 new. Counter-checks (no early start, no Mix delay, chain made every time, tune misses the master) each turned their test red. - Unit bus-fx (new): 6 green, 4 counter-checks red. Scheduler ahead: 1 new, its counter-check red. - e2e 65 (melektaus-atx): ALL PASSED, 35 checks, 4 new, first run. Chrome at 44.1 kHz: the bus was 0.046440 s late, the note started exactly that much early. Offline the shifter moved 1 kHz to 2000 Hz and passed a click 2048 samples later. - Gate (melektaus-atx): CHECK EXIT 0, 259 files, 4021 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
a3101c590c |
Size notes by either end and pan before bar one in music tabs
A round of small fixes to the Pattern and Song tabs: what the ruler and a pan do in both, and a few things the tabs carried that were in the way. - A middle-drag can take bar 1 up to half the grid or lanes in. The time before it is drawn dark on the ruler, the grid and the lanes. Nothing lands there: a click draws no note, a drop shows no frame. - A press or drag on the ruler puts the start on the nearest grid line, no longer on the one before the pointer. - A new song has one track instead of four. The "+ Track" row under the heads is gone, a head's menu adds tracks above or below. A song still keeps at least one, so that menu is always there. - A note can be sized by its left end too: the start goes to the nearest grid line and the end stays put, never before tick 0. - An end of a note under the pointer lights up and shows a resize cursor, as on an item of the Song tab. Tests: - Unit: piano-roll, song-edit, song, song-play 52 tests. Two new cases for the left end and the pan, the song fixtures add their tracks themselves now. - e2e 64+65 on melektaus-atx: ALL PASSED - 50 checks (+2). e2e 65 adds its tracks through the head's menu and clicks the ruler three quarters into a bar, which only the nearest line passes. - Gate on melektaus-atx: 251 files, 3909 tests, CHECK EXIT 0. - A probe shot of the piano roll panned in: the time before bar 1 dark, the left end of a note lit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
aa097bd076 |
Zoom with the wheel and set the loop on its own strip in music tabs
The ruler of the Pattern and Song tabs did two things with one drag,
and the wheel did three things with Alt and Shift. Both now do one
thing each, and the Song tab gets a few edits it was missing.
- The ruler has a loop strip along its top. A drag there sets the
loop, a press or drag below it moves only where playing starts.
- The wheel only zooms: the time over the grid, lanes and ruler, the
rows over the keys, the tracks over their heads. Alt and Shift no
longer change it. A middle-drag still pans.
- Tracks can be zoomed from 28 to 160 px. Below 46 px a head shows
only its name, M and S. An old saved view gets the old height.
- A click on nothing in the lanes picks the song itself.
- A right-click on an item opens a menu with Delete instead of
deleting it. The inspector has no delete buttons any more.
- A sound item goes up to +12 dB, a track up to +12 dB. A pattern
item stays at 0 dB, since a note is held to unity.
- An end of an item under the pointer lights up and shows a resize
cursor. A long item marks every point where it starts over.
- The Pattern tab has an Audition switch in its header, off at first.
Only with it on does a note sound when drawn, grabbed or moved.
The keys down the left sound either way.
- A looped sound item draws every round alike. Each round is drawn
from its own start, and each column shows the loudest part of the
sound it covers instead of one sampled bucket.
- A song is at least one bar long instead of four. An empty song
fits that bar across, and Play song ends after the last item.
- The ghost notes held back in #334 got their own issue, #339.
Finding: suite 542 for
|
||
|
adminandClaude Opus 5.5
|
f97961c8ff |
Play and stop a song from a behavior in the game #336
A song could only be heard in its tab. Now a game plays it through Play song and ends it through Stop song, with its tracks as they are set. A playing pattern or song also no longer undoes the pause of the toolbar. - Play song plays from the start of the song and then round the loop of its ruler, so an intro is heard once. Without loop it plays once to its end. Inputs are song, volume, fade in, loop (on by default) and detached (off by default). It gives out playback. - Stop song stops one song, every song for none, or one playback by its number, over its fade out. Stop sound with no sound named stops songs too. Stop pattern with no pattern named stops only patterns. - Songs play side by side, as patterns do. They are owned like a sound, so leaving the scene ends one unless it is detached. Pause game and the time scale leave the music alone. - Each playing is heard through a master of its own, silent until its first note. The fades move it, and a song's tracks sit inside it on buses of their own. They go once the playing and its voices have. - Notes and clips are struck the same way in the editor and the game (strikeNote, playClip). What a sequence needs comes from one place (core/music/needs). Findings: - Reading the audio clock resumed a suspended context. A playing pattern read it every 25 ms and so undid the toolbar's pause. The library now leaves a context alone while the toolbar holds it. - A pattern played once to its end asked the list of playings about its voices, and that list asked the pattern again. Once its last voice was gone this ran until the stack overflowed. The scheduler of a playing now asks only after voices. - e2e 64 reads the playings through __gm.sound.playbacks() now. Tests: - song-play.test.ts (new): 8 tests, node catalog +8 cases - sound-library.test.ts: 66 tests, 4 new - e2e 64 (melektaus-atx): 23 of 23 checks passed - e2e 65 (melektaus-atx): ALL PASSED, 22 checks (+4) - Gate (melektaus-atx): CHECK EXIT 0, 251 files, 3903 tests - Counter-checks: the old way of asking after voices, a clock read that wakes a held context, and a start after a fade out began each fail 1 test Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |