A test that waits for a deadline somebody guessed is waiting for the wrong thing, so it waits for the editor (#93)

The suite spent 88% of its step time asleep: 564 s of steps, 496 s of
them in sleep(n) - fixed spans estimated by hand and then copied from
step to step. That is why it was slow, and why it flickered.

The signal is afterFrames(count, timeout) in ui/render.ts, exposed as
__gm.afterFrames(). It wakes the loop itself, because an editor that
draws nothing would never redeem a promise of a frame, and it gives up
after a timeout instead of hanging the run. settle() in the harness
waits two drawn frames (~32 ms) instead of 150-300 ms and falls back to
sleep(150) when the hook is missing (an old build): silently not
waiting would make every later check a coin toss.

Every action goes through it now - click, doubleClick, wheel, drag,
dragAsset, press, typeText, select, mutate, undo, redo - and with that
about 550 sleep() calls behind those actions became redundant, helpers
that are nothing but a mutate in a coat included. What a frame cannot
answer got its own condition instead: waitUntil(expr), play()/stop()
waiting on __gm.state.mode rather than on 300 ms, saved() for the
autosave (the two hand-written polling loops sampled at 300 ms for
something that arrives after a second), and showTab() as a real tool -
47 places called evaluate("__gm.showTab(...)") raw, and 24 of them
waited for nothing at all and read the old panel.

564 s -> 243 s of step time, the waiting share 88% -> 76%. Three kinds
of wait were kept, and each one bit when it was taken away: loading
bytes (textures, fonts), game time that a repaint cannot stand in for
(time scale, shader and particle clocks, the live values a running
graph carries), and the inertia the scroll step measures on purpose.

Two things the new speed uncovered, both real: 25-animation read the
frame menu unfiltered and had frozen an incomplete list (the CI failure
on c990dbc), and 33-render-on-change measured idleness while the
autosave of the step before was still painting - it waits for
saveStatus to leave dirty/saving now, so it measures that nothing
happens any more instead of that it waited long enough.

tools/lib/harness.mjs

@@ -6,7 +6,17 @@
 import { writeFileSync } from 'node:fs';
 import { launchBrowser } from './browser.mjs';
 
-const sleep = (ms) => new Promise((r) => setTimeout(r, ms));
+/**
+ * Every wait the checks do goes through here, and the time is booked. A run that takes ten
+ * minutes needs to say how much of that was the editor working and how much was the harness
+ * waiting for a deadline it made up – the two call for opposite fixes.
+ */
+export const waited = { ms: 0, calls: 0 };
+const sleep = (ms) => {
+  waited.ms += ms;
+  waited.calls++;
+  return new Promise((r) => setTimeout(r, ms));
+};
 
 /** Generous on purpose: a slow navigation still finishes, a wedged chromium never does. */
 const CDP_TIMEOUT_MS = 30_000;
@@ -131,6 +141,41 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
   };
   /** Evaluates an expression in the page and returns it as plain JSON; `undefined` comes back as null. */
   const json = async (expression) => JSON.parse(await evaluate(`JSON.stringify((${expression}) ?? null)`));
+  /**
+   * Waits for the editor to have drawn, instead of for a deadline someone guessed. A frame costs
+   * ~16 ms where the fixed waits this replaces cost 150–300, and it is the editor itself that
+   * says when it has caught up – so the same call is both faster and less flaky than the number
+   * it replaces. Two frames by default: one for the handler, one for the layout it invalidated.
+   *
+   * Falls back to a fixed wait when the page has no such hook (an old build, or before the first
+   * navigation), because a silent no-wait would turn every later check into a coin toss.
+   */
+  const settle = async (count = 2) => {
+    const at = Date.now();
+    const ok = await evaluate(
+      `window.__gm?.afterFrames ? __gm.afterFrames(${count}).then(() => true) : false`,
+    );
+    if (!ok) return sleep(150);
+    waited.ms += Date.now() - at;
+    waited.calls++;
+  };
+  /**
+   * Waits until `expression` is true in the page – for the things a frame cannot answer, such as a
+   * save that is still in flight. Polls, because the conditions live all over the model and none
+   * of them has an event worth inventing one for.
+   */
+  const waitUntil = async (expression, { timeout = 5000, label = expression } = {}) => {
+    const at = Date.now();
+    while (Date.now() - at < timeout) {
+      if (await json(`Boolean(${expression})`)) {
+        waited.ms += Date.now() - at;
+        waited.calls++;
+        return;
+      }
+      await sleep(25);
+    }
+    throw new Error(`waited ${timeout} ms and ${label} never came true`);
+  };
   /** Like `locate`, but null instead of throwing – which a check for absence needs. */
   const find = async (text, last = false) =>
     (await evaluate(`window.__gm && window.__gm.locate(${JSON.stringify(text)}, ${last})`)) || null;
@@ -163,7 +208,7 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
     await mouse('mouseMoved', x, y);
     await mouse('mousePressed', x, y, button, extra);
     await mouse('mouseReleased', x, y, button, extra);
-    await sleep(150);
+    await settle();
   };
   /**
    * A real double-click: two press/release pairs in quick succession, the second carrying
@@ -176,13 +221,13 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
       await mouse('mousePressed', x, y, 'left', { clickCount });
       await mouse('mouseReleased', x, y, 'left', { clickCount });
     }
-    await sleep(150);
+    await settle();
   };
   /** Mouse wheel over (x, y); positive `dy` scrolls down. */
   const wheel = async (x, y, dy) => {
     await mouse('mouseMoved', x, y);
     await mouse('mouseWheel', x, y, 'none', { deltaX: 0, deltaY: dy });
-    await sleep(150);
+    await settle();
   };
   const clickText = async (text, button = 'left') => {
     const p = await locate(text);
@@ -206,7 +251,7 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
       await sleep(16);
     }
     await mouse('mouseReleased', x2, y2, button, extra);
-    await sleep(150);
+    await settle();
   };
 
   /**
@@ -231,7 +276,7 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
       await sleep(16);
     }
     await mouse('mouseReleased', x, y);
-    await sleep(200);
+    await settle();
   };
 
   const keyEvent = (type, name, modifiers) => {
@@ -243,6 +288,7 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
   const press = async (name, modifiers = {}) => {
     await keyEvent('keyDown', name, modifiers);
     await keyEvent('keyUp', name, modifiers);
+    await settle();
   };
   /** Hold a key for `ms` milliseconds (arrow keys while playing). */
   const holdKey = async (name, ms) => {
@@ -267,14 +313,13 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
       throw new Error(
         `typing "${text}" did not arrive: focus is ${typed.tag}, it holds ${JSON.stringify(typed.value)}`,
       );
+    await settle();
   };
   const undo = async () => {
     await press('z', { ctrl: true });
-    await sleep(200);
   };
   const redo = async () => {
     await press('z', { ctrl: true, shift: true });
-    await sleep(200);
   };
 
   /** Selection, mode, save status and notice – the facts most checks look at. */
@@ -294,22 +339,47 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
   const asset = (id) => json(`__gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)})`);
   const world = () => json('__gm.world()');
   const entity = async (objectId) => (await world())?.entities.find((e) => e.objectId === objectId);
-  const select = (selection) => evaluate(`__gm.state.select(${JSON.stringify(selection)})`);
+  const select = async (selection) => {
+    await evaluate(`__gm.state.select(${JSON.stringify(selection)})`);
+    await settle();
+  };
+  /**
+   * Switching the centre tab rebuilds a whole panel, so it needs the same wait an action gets.
+   * It is a helper rather than a bare `evaluate` in every step because a tab change without one
+   * reads the old panel – which is a check that passes on stale content, the worst kind.
+   */
+  const showTab = async (tab) => {
+    await evaluate(`__gm.showTab(${JSON.stringify(tab)})`);
+    await settle();
+  };
+  /**
+   * Waits for the autosave to have reached the server. The status goes `dirty` → `saving` →
+   * `saved`, and only the last one means the document a reload would fetch holds the change.
+   */
+  const saved = () =>
+    waitUntil("__gm.state.saveStatus === 'saved'", { timeout: 9000, label: 'the save finished' });
   /**
    * Runs `body` (JavaScript source) against the page's model and previews the result, the way a
    * scrub or a command would – for putting the scene into a known state between checks.
    */
   const mutate = async (body) => {
     await evaluate(`(() => { ${body}; __gm.state.preview(); })()`);
-    await sleep(200);
+    await settle();
   };
+  /**
+   * Starting and stopping a run build or tear down a whole world, which is more than the two frames
+   * `settle()` grants – so these wait for the mode itself to have changed, then for a drawn frame so
+   * the panels that follow the mode are on screen too.
+   */
   const play = async () => {
     await clickText('▶  Play');
-    await sleep(300);
+    await waitUntil("__gm.state.mode === 'play'", { label: 'the run started' });
+    await settle();
   };
   const stop = async () => {
     await press('Escape');
-    await sleep(300);
+    await waitUntil("__gm.state.mode === 'edit'", { label: 'the run stopped' });
+    await settle();
   };
 
   // Timeout, not convenience: a stalled response keeps its socket open, so node stays alive and
@@ -342,6 +412,8 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
     sleep,
     evaluate,
     json,
+    settle,
+    waitUntil,
     find,
     has,
     locate,
@@ -368,6 +440,8 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
     world,
     entity,
     select,
+    showTab,
+    saved,
     mutate,
     play,
     stop,