Let the agent read what is heard in the running game (#436)
The agent has no ears. It could start music and sounds, but not tell whether anything reached the speakers: a song without an instrument, a bus at volume 0 or a browser that holds the audio clock all looked the same from outside. - get_runtime has a section audio. Under levels it gives the peak and RMS in dB of the master, each bus of the mix, each pattern, song or composition playing and the tracks of a song. Below -90 dB it says silent. - The levels come from a tap of their own after the level and pan of each bus, an AnalyserNode per side as for the meters of the Song tab. It looks back about 0.7 s, so the gap between two notes is not silence. A pause holds the values, which fits play with until. - Under voices it names what sounds now: a sound by its asset, a note of an instrument with its note name, each with its bus and object. Next to them planned, playing, waiting (bytes not here yet) and failed. - Under clock it gives the state of the audio clock, with a hint when it stands still while the game runs. The state is told as it was before a pause, else the pause would hide that case. - The buses are tapped only while the agent is switched on. - The note now travels with a voice from the Start of a pattern or song to the SoundLibrary, so a voice can be named by it. - tools.json and the relay's instructions tell the agent to check under audio that music and sounds are really heard. Found on the user's project: started right after F5, before a click in the page, the browser kept the audio clock suspended for the whole run. Nothing was heard and the notes stayed planned. That is what clock now shows. Tests: - Unit: sound-library (taps, levels, voices, clock), agent-play (audioView). Pattern and song tests expect the note in their timing. - 9 counter-checks, 8 red. Not dropping a tap with its bus stays green, as the bus is cut off and gone anyway. - e2e 71 (melektaus-atx): 28/28. New 71h4: a sound on the master and one on a bus at volume 0, both voices named, the master loud, the bus silent, the values held in the pause. - Gate (melektaus-atx): CHECK EXIT 0, 327 files, 5037 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
dd2096e0df
commit
b5e071ca68
14 files changed
+607
-16
No files matched your search
+3
-1
@@ -56,7 +56,9 @@ const INSTRUCTIONS = [
|
||||
'The editor follows your edits: after run_commands the user sees the graph, scene or asset it worked',
|
||||
'on, with what it touched selected. Use open and select to show anything else.',
|
||||
'Test what you built yourself: play, then play_input sends keys and mouse clicks into the game,',
|
||||
'and get_runtime reads where the objects are, their variables and the console. To look at one',
|
||||
'and get_runtime reads where the objects are, their variables and the console, and under audio',
|
||||
'what you would hear: the voices sounding and the level of each bus, silent where nothing reaches',
|
||||
'it. Check there that music and sounds are really heard. To look at one',
|
||||
'moment, play with until holds the game at that game time, as does a last play_input step',
|
||||
'{"pause": true}; a call that only reads comes too late for it, the game runs on meanwhile. A game needs a',
|
||||
'camera in its scene to show anything; get_view says when it has none.',
|
||||
|
||||
Reference in new issue
Block a user