Let the agent hold the game at a given game time (#435)
feuerwerk / gate (push) Successful in 5m31s
feuerwerk / suite (push) Successful in 19m9s

The agent could not look at a short moment of a running game. Its
calls took longer than the moment lasted, so an intro window from
0.5 to 2 s slipped past twice in the 5d test. Now the editor itself
stops the clock at the time the agent names.

- play takes until: play, resume or pause let the game run on to
  that game time and hold it there. The call answers once the game
  is paused, so get_runtime and screenshot show that moment.
- The frame that would step past the time is cut short
  (frameWithin in src/core/time.ts, used in PlaySession). The game
  stands at 0.5 s, not one frame later.
- play_input takes a last step {"pause": true}. After a wait it
  holds the game at the time of the wait, after a key or click at
  once. Its answer names the game time and whether it is paused.
- A pause that comes first (toolbar or a Break node), a Load scene
  or a Restart game call the until off, and the answer says so.
- When the clock stands still, the call gives up after 25 s with
  the time it is at. A tab in the background is refused at once.
- A recording that does not run in real time is not cut. There the
  game stops in the first frame that reaches the time.
- get_runtime and play_input give times to the millisecond, not to
  the hundredth, so 0.505 no longer reads as 0.51.
- The relay's instructions name both ways.

Tests:
- Unit agent-play (6 new) and time (2 new): 19/19, related editor
  and relay tests 1158/1158.
- 6 counter-checks, all compile, all red. One stayed green at
  first: 0.3 s is exactly 18 frames. The test now takes 0.31 s.
- e2e 71 (atx): ALL PASSED, 27 checks. New 71h3 holds at 0.505 s
  and 0.8 s and after a wait of 0.31 s. Without the cut in
  PlaySession it went red at 0.5166 s.
- Gate (atx): CHECK EXIT 0, 327 files, 5033 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-18 21:26:53 +02:00
1 parent e4ccf5f72f
commit 70402aa19a
12 files changed
+351 -26

No files matched your search

+12 -4
View File
@@ -187,10 +187,17 @@
{
"name": "play",
"title": "Play the game",
"description": "Plays the game in the editor, pauses, resumes or stops it. Edits made while it runs change the running copy only and are gone when it stops.",
"description": "Plays the game in the editor, pauses, resumes or stops it. With until, play, resume or pause let the game run on to that game time, as get_runtime gives it, and hold it there, exactly: the call answers once it is paused, and get_runtime and screenshot then show that moment. Edits made while it runs change the running copy only and are gone when it stops.",
"inputSchema": {
"type": "object",
"properties": { "action": { "type": "string", "enum": ["play", "pause", "resume", "stop"] } },
"properties": {
"action": { "type": "string", "enum": ["play", "pause", "resume", "stop"] },
"until": {
"type": "number",
"minimum": 0,
"description": "Game seconds to pause at, at most 30 ahead. A Load scene or Restart game starts the time over and calls it off."
}
},
"required": ["action"],
"additionalProperties": false
},
@@ -199,7 +206,7 @@
{
"name": "play_input",
"title": "Keys and mouse in the running game",
"description": "Plays keys and the mouse into the running game like a player, step by step; each step takes at least one frame, and hold and wait count game seconds. A step does one thing: {\"key\": \"ArrowLeft\", \"hold\": 0.5} holds a key; {\"key\": \"Space\"} taps it for one frame; {\"key\": \"Shift\", \"keep\": \"down\"} presses it and leaves it held after the call, \"keep\": \"up\" lets it go; {\"mouse\": \"click\", \"at\": {\"object\": \"<object id>\"}} or {\"mouse\": \"click\", \"at\": {\"x\": 10, \"y\": -20}} clicks the left button there (world units, as get_runtime gives them); \"down\", \"up\" and \"move\" do just that; {\"wheel\": -1} turns the wheel; {\"wait\": 1} lets the game run on. Keys go by their KeyboardEvent.key name: \"a\", \"ArrowUp\", \"Enter\", \"Space\". Answers the game time it took, the keys still held, where the mouse is, whether a click on an object landed on it, and the new console lines; read positions and variables with get_runtime. Needs the game playing, not paused, and the editor tab in front.",
"description": "Plays keys and the mouse into the running game like a player, step by step; each step takes at least one frame, and hold and wait count game seconds. A step does one thing: {\"key\": \"ArrowLeft\", \"hold\": 0.5} holds a key; {\"key\": \"Space\"} taps it for one frame; {\"key\": \"Shift\", \"keep\": \"down\"} presses it and leaves it held after the call, \"keep\": \"up\" lets it go; {\"mouse\": \"click\", \"at\": {\"object\": \"<object id>\"}} or {\"mouse\": \"click\", \"at\": {\"x\": 10, \"y\": -20}} clicks the left button there (world units, as get_runtime gives them); \"down\", \"up\" and \"move\" do just that; {\"wheel\": -1} turns the wheel; {\"wait\": 1} lets the game run on; {\"pause\": true}, the last step, holds the game right after the step before, a wait to its time exactly, so get_runtime and screenshot show that moment. Keys go by their KeyboardEvent.key name: \"a\", \"ArrowUp\", \"Enter\", \"Space\". Answers the game time it took and the game time it is at, whether it is paused, the keys still held, where the mouse is, whether a click on an object landed on it, and the new console lines; read positions and variables with get_runtime. Needs the game playing, not paused, and the editor tab in front.",
"inputSchema": {
"type": "object",
"properties": {
@@ -228,7 +235,8 @@
"additionalProperties": false
},
"wheel": { "type": "integer", "minimum": -10, "maximum": 10 },
"wait": { "type": "number", "minimum": 0, "maximum": 10, "description": "Game seconds." }
"wait": { "type": "number", "minimum": 0, "maximum": 10, "description": "Game seconds." },
"pause": { "type": "boolean", "description": "true: holds the game here; the last step." }
},
"additionalProperties": false
}