The agent has no ears. It could start music and sounds, but not tell
whether anything reached the speakers: a song without an instrument, a
bus at volume 0 or a browser that holds the audio clock all looked the
same from outside.
- get_runtime has a section audio. Under levels it gives the peak and
RMS in dB of the master, each bus of the mix, each pattern, song or
composition playing and the tracks of a song. Below -90 dB it says
silent.
- The levels come from a tap of their own after the level and pan of
each bus, an AnalyserNode per side as for the meters of the Song tab.
It looks back about 0.7 s, so the gap between two notes is not
silence. A pause holds the values, which fits play with until.
- Under voices it names what sounds now: a sound by its asset, a note
of an instrument with its note name, each with its bus and object.
Next to them planned, playing, waiting (bytes not here yet) and
failed.
- Under clock it gives the state of the audio clock, with a hint when
it stands still while the game runs. The state is told as it was
before a pause, else the pause would hide that case.
- The buses are tapped only while the agent is switched on.
- The note now travels with a voice from the Start of a pattern or song
to the SoundLibrary, so a voice can be named by it.
- tools.json and the relay's instructions tell the agent to check under
audio that music and sounds are really heard.
Found on the user's project: started right after F5, before a click in
the page, the browser kept the audio clock suspended for the whole run.
Nothing was heard and the notes stayed planned. That is what clock now
shows.
Tests:
- Unit: sound-library (taps, levels, voices, clock), agent-play
(audioView). Pattern and song tests expect the note in their timing.
- 9 counter-checks, 8 red. Not dropping a tap with its bus stays green,
as the bus is cut off and gone anyway.
- e2e 71 (melektaus-atx): 28/28. New 71h4: a sound on the master and
one on a bus at volume 0, both voices named, the master loud, the bus
silent, the values held in the pause.
- Gate (melektaus-atx): CHECK EXIT 0, 327 files, 5037 tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The agent could not look at a short moment of a running game. Its
calls took longer than the moment lasted, so an intro window from
0.5 to 2 s slipped past twice in the 5d test. Now the editor itself
stops the clock at the time the agent names.
- play takes until: play, resume or pause let the game run on to
that game time and hold it there. The call answers once the game
is paused, so get_runtime and screenshot show that moment.
- The frame that would step past the time is cut short
(frameWithin in src/core/time.ts, used in PlaySession). The game
stands at 0.5 s, not one frame later.
- play_input takes a last step {"pause": true}. After a wait it
holds the game at the time of the wait, after a key or click at
once. Its answer names the game time and whether it is paused.
- A pause that comes first (toolbar or a Break node), a Load scene
or a Restart game call the until off, and the answer says so.
- When the clock stands still, the call gives up after 25 s with
the time it is at. A tab in the background is refused at once.
- A recording that does not run in real time is not cut. There the
game stops in the first frame that reaches the time.
- get_runtime and play_input give times to the millisecond, not to
the hundredth, so 0.505 no longer reads as 0.51.
- The relay's instructions name both ways.
Tests:
- Unit agent-play (6 new) and time (2 new): 19/19, related editor
and relay tests 1158/1158.
- 6 counter-checks, all compile, all red. One stayed green at
first: 0.3 s is exactly 18 frames. The test now takes 0.31 s.
- e2e 71 (atx): ALL PASSED, 27 checks. New 71h3 holds at 0.505 s
and 0.8 s and after a wait of 0.31 s. Without the cut in
PlaySession it went red at 0.5166 s.
- Gate (atx): CHECK EXIT 0, 327 files, 5033 tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An agent could build a game object but not see whether it works. It
had to ask the user to press the keys. Now it can play the game in the
editor itself, send keys and mouse input and read what happened.
- play_input sends steps into the running game: hold or tap a key,
keep it down or let it go, click, press, release or move the mouse,
turn the wheel, wait. Holding and waiting count game seconds.
- A mouse target is a point in world units or an object by id or
name. A click on an object reports whether it landed on it.
- The answer names the game time used, the keys still held, the mouse
and the new console lines.
- One tool instead of press_keys and pointer from the plan. A step
list can then mix both, like holding Shift during a click.
- get_runtime reads the running game: objects with id, type and
position, the own variables of their behaviors, the Variables
assets, the camera, keys, mouse and the console. Named objects come
with all their properties.
- A hidden editor tab gets no frames. play_input says so at once. If
game time stands still for 25 s it stops, names the step and lets
go of every key it held.
- get_view and play say when the scene has no active camera. The
first agent test missed that and only saw it on a screenshot.
- The relay instructions tell the agent to test what it built.
Found in the live test on a real project:
- A click on an object by its name reported hit false. The hit test
looked the target up by id only.
- get_runtime listed the built-in switches of a behavior (enabled,
runInEditor) as its variables.
Tests:
- agent-play (new), agent-edit, agent-view, relay green.
- 5 counter-checks red: hold counts one frame, key never released,
click ignores the object, hidden tab not refused, no variable
filter.
- e2e 71: 21/21. The agent holds ArrowLeft 0.5 s (x -120 to -270)
and clicks the object (hit true).
- Gate: CHECK EXIT 0, 319 files, 4944 tests.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
adminandClaude Opus 5.5