Let the agent read what is heard in the running game (#436)
The agent has no ears. It could start music and sounds, but not tell whether anything reached the speakers: a song without an instrument, a bus at volume 0 or a browser that holds the audio clock all looked the same from outside. - get_runtime has a section audio. Under levels it gives the peak and RMS in dB of the master, each bus of the mix, each pattern, song or composition playing and the tracks of a song. Below -90 dB it says silent. - The levels come from a tap of their own after the level and pan of each bus, an AnalyserNode per side as for the meters of the Song tab. It looks back about 0.7 s, so the gap between two notes is not silence. A pause holds the values, which fits play with until. - Under voices it names what sounds now: a sound by its asset, a note of an instrument with its note name, each with its bus and object. Next to them planned, playing, waiting (bytes not here yet) and failed. - Under clock it gives the state of the audio clock, with a hint when it stands still while the game runs. The state is told as it was before a pause, else the pause would hide that case. - The buses are tapped only while the agent is switched on. - The note now travels with a voice from the Start of a pattern or song to the SoundLibrary, so a voice can be named by it. - tools.json and the relay's instructions tell the agent to check under audio that music and sounds are really heard. Found on the user's project: started right after F5, before a click in the page, the browser kept the audio clock suspended for the whole run. Nothing was heard and the notes stayed planned. That is what clock now shows. Tests: - Unit: sound-library (taps, levels, voices, clock), agent-play (audioView). Pattern and song tests expect the note in their timing. - 9 counter-checks, 8 red. Not dropping a tap with its bus stays green, as the bus is cut off and gone anyway. - e2e 71 (melektaus-atx): 28/28. New 71h4: a sound on the master and one on a bus at volume 0, both voices named, the master loud, the bus silent, the values held in the pause. - Gate (melektaus-atx): CHECK EXIT 0, 327 files, 5037 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
dd2096e0df
commit
b5e071ca68
14 files changed
+607
-16
No files matched your search
@@ -349,14 +349,18 @@ export function playNote(
|
||||
): number {
|
||||
const asset = findAssetOf(project, instrumentId, 'instrument');
|
||||
if (!asset) return 0;
|
||||
const built = instrumentBuildFor(asset, heldNote(struck.note), instrumentSourcesOf(project)).layers;
|
||||
const note = heldNote(struck.note);
|
||||
const built = instrumentBuildFor(asset, note, instrumentSourcesOf(project)).layers;
|
||||
if (built.length === 0) return 0;
|
||||
const layers = fx ? built.map((layer) => throughScopedFx(fx.plan, layer, FX_SCOPE)) : built;
|
||||
const velocity = Number.isFinite(struck.velocity) ? Math.max(0, Math.min(1, struck.velocity)) : 1;
|
||||
const level = Math.max(0, Math.min(1, struck.volume * velocity));
|
||||
const duration = Number.isFinite(struck.duration) ? Math.max(0, struck.duration) : 0;
|
||||
const values = { ...params, [VELOCITY_PARAM]: velocity, ...scoped(fx?.params ?? {}) };
|
||||
return audio.playNote(instrumentVoiceName(instrumentId), layers, values, level, owner, duration, timing);
|
||||
return audio.playNote(instrumentVoiceName(instrumentId), layers, values, level, owner, duration, {
|
||||
...timing,
|
||||
note,
|
||||
});
|
||||
}
|
||||
|
||||
/** Where the parameters of a note's own sound FX go by (#346): no name of a parameter has a colon. */
|
||||
|
||||
Reference in new issue
Block a user