Wait for driven videos before a run starts so no sprite flashes (#404)

The game view opens a player for each driven video only when it draws,
and it draws only while its tab is visible. Without a detour over the
Game tab the players came up with the click on Play. Until a video had
loaded, its sprite showed its base colour, white in the user's cut.
Sharing the players of the scene view would not have helped. From the
Animations tab those are discarded after two to four seconds.

- Play holds the run at second 0 until every driven video in the game
  view has its first picture, at most 3 s. The status line says
  "Loading the videos… · Esc stops".
- The game clock stands while it waits. Behaviors and sounds start
  only after it, so picture and sound stay together.
- Load scene in a running game waits the same way.
- A driven video without a picture is not drawn at all instead of
  white. That holds everywhere, also on a switch to the Game tab
  without Play and at a cut whose video is not ready.
- The exported player waits the same way.
- __gm.playInfo() reports waiting for the e2e check.

Tests:
- e2e 66p puts the sprite in front of the camera and waits 4.5 s so
  the game view has no player left. Then it reads every frame of the
  run: 14 frames of waiting on #22252c at time 0, then blue, no white.
- Against the old code 66p fails: 14 frames #ffffff, then blue.
- Unit video-library: 7 passed, 1 new for loading().
- e2e 66 on melektaus-atx: ALL PASSED, 27 checks.
- Gate on melektaus-atx: exit 0, 297 files, 4558 tests.
- tsc, eslint, Prettier clean.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -47,7 +47,7 @@ npm run build        # writes dist/ and regenerates index.html; the folder is wh
 The browser suite and the three example games need a machine with a GPU:
 
 ```bash
-node tools/seed-e2e.mjs && node tools/e2e.mjs      # the editor, 2003 checks (GM_STEPS=20 for one step)
+node tools/seed-e2e.mjs && node tools/e2e.mjs      # the editor, 2004 checks (GM_STEPS=20 for one step)
                                                    #   every tool signs in first – see the Accounts page
 node tools/seed-jumprun.mjs && node tools/smoke-jumprun.mjs
 node tools/seed-tetris.mjs && node tools/smoke-tetris.mjs

src/editor/Editor.ts

@@ -1334,8 +1334,9 @@ export class Editor implements PlayHost {
     this.measureFps(dt);
     // One step for everything below: a world gets exactly the seconds the particle, shader and
     // animation clocks moved, so the global time scale cannot pull them apart.
-    // The game's own pause holds the clock as well, so particles and shaders freeze with the world.
+    // The game's own pause holds the clock as well, so particles and shaders freeze with the world,
-    const step = gameClock.advance(dt, this.state.paused || this.play.isGamePaused);
+    // and so does a run that waits for its videos at the start (#404).
+    const step = gameClock.advance(dt, this.state.paused || this.play.isGamePaused || this.play.isWaiting);
     this.play.tick(step, timeStep(dt, gameClock.scale, this.state.paused));
     this.editorRun.tick(step);
     this.viewport.animate(sceneDraws);
@@ -1461,6 +1462,8 @@ export class Editor implements PlayHost {
     playing: boolean;
     paused: boolean;
     gamePaused: boolean;
+    /** Held at the start until the Game tab's driven videos have a picture (#404). */
+    waiting: boolean;
     time: number;
     sceneId: string;
     panelUpdates: { structure: number; values: number };
@@ -1470,6 +1473,7 @@ export class Editor implements PlayHost {
       playing: this.play.isPlaying,
       paused: this.play.isPaused,
       gamePaused: this.play.isGamePaused,
+      waiting: this.play.isWaiting,
       time: this.play.time,
       sceneId: this.play.sceneId,
       panelUpdates: this.play.panelUpdates(),

src/editor/PlaySession.ts

@@ -16,7 +16,7 @@ import { Input } from '../runtime/systems/Input';
 import type { World } from '../runtime/World';
 import { ui } from '../ui';
 import { soundLibrary } from '../viewport/SoundLibrary';
-import { videoLibrary } from '../viewport/VideoLibrary';
+import { VIDEO_WAIT_MS, videoLibrary } from '../viewport/VideoLibrary';
 import { FrameMeter, type FrameTimes } from './frameMeter';
 import { saveArea } from './saves';
 import { PanelClock, panelPolicy, type PanelUpdate } from './playThrottle';
@@ -68,6 +68,12 @@ export class PlaySession {
   private updates = { structure: 0, values: 0 };
   /** Where the frames of this run go (#275). */
   private readonly meter = new FrameMeter();
+  /**
+   * Until when the run waits at its start for the Game tab's driven videos (#404), by
+   * `performance.now()`; null once it goes. Its players are opened only at Play, and the first
+   * frames of a cut showed white without them.
+   */
+  private videoWait: number | null = null;
 
   constructor(
     private readonly state: EditorState,
@@ -100,6 +106,11 @@ export class PlaySession {
     return this.world?.paused ?? false;
   }
 
+  /** Held at its start until the Game tab has a picture of its driven videos (#404). */
+  get isWaiting(): boolean {
+    return this.videoWait !== null;
+  }
+
   /** Pause or resume; does nothing while not playing. */
   togglePause(): void {
     if (!this.world) return;
@@ -171,6 +182,7 @@ export class PlaySession {
     this.input.attach();
     this.host.gameView.setWorld(this.run.world);
     this.host.showGame();
+    this.waitForVideos();
     this.state.setNotice(this.playNotice(), 'quiet');
     if (this.run.world.stopRequested) this.stop();
   }
@@ -179,6 +191,7 @@ export class PlaySession {
   private sceneLoaded(run: GameRun): void {
     this.timeSystems(run);
     this.host.gameView.setWorld(run.world);
+    this.waitForVideos();
     this.state.setNotice(this.playNotice(), 'quiet');
     if (run.world.stopRequested) this.stop();
   }
@@ -201,7 +214,22 @@ export class PlaySession {
     );
   }
 
+  /** The Game tab has drawn the new world once by now, which opened the players it shows. */
+  private waitForVideos(): void {
+    this.videoWait = this.host.gameView.videosLoading() ? performance.now() + VIDEO_WAIT_MS : null;
+  }
+
+  /** Whether the run still waits for its videos; the frame it stops waiting, the notice says so. */
+  private waitsForVideos(): boolean {
+    if (this.videoWait === null) return false;
+    if (performance.now() < this.videoWait && this.host.gameView.videosLoading()) return true;
+    this.videoWait = null;
+    this.state.setNotice(this.playNotice(), 'quiet');
+    return false;
+  }
+
   private playNotice(): string {
+    if (this.videoWait !== null) return 'Loading the videos… · Esc stops';
     if (this.state.paused) return 'Paused · Ctrl+Enter resumes · Esc stops';
     if (this.world?.paused) return 'The game paused itself · Esc stops';
     return this.world?.activeCamera()
@@ -216,6 +244,7 @@ export class PlaySession {
     soundLibrary.setSuspended(false);
     soundLibrary.stopAll();
     videoLibrary.endRun();
+    this.videoWait = null;
     const world = this.world;
     this.run = null;
     this.input.detach();
@@ -237,7 +266,7 @@ export class PlaySession {
     const run = this.run;
     if (!run) return;
     const world = run.world;
-    if (this.state.paused) {
+    if (this.state.paused || this.waitsForVideos()) {
       // Keys and clicks that happen while paused are dropped instead of piling up – otherwise
       // resuming would deliver every one of them in a single frame.
       this.input.endFrame();

src/player/Player.ts

@@ -7,7 +7,7 @@ import { behaviorError } from '../runtime/systems/BehaviorSystem';
 import { Input } from '../runtime/systems/Input';
 import { requestRender } from '../ui';
 import { soundLibrary } from '../viewport/SoundLibrary';
-import { videoLibrary } from '../viewport/VideoLibrary';
+import { VIDEO_WAIT_MS, videoLibrary } from '../viewport/VideoLibrary';
 import type { GameScreen } from './GameScreen';
 
 /**
@@ -18,6 +18,8 @@ import type { GameScreen } from './GameScreen';
 export class Player {
   private run: GameRun | null = null;
   private readonly input = new Input();
+  /** Until when a run waits at its start for its driven videos (#404); see `PlaySession`. */
+  private videoWait: number | null = null;
 
   constructor(
     private readonly project: Project,
@@ -61,6 +63,7 @@ export class Player {
     videoLibrary.startRun();
     this.input.attach();
     this.screen.setWorld(this.run.world);
+    this.waitForVideos();
     if (this.run.world.stopRequested) this.stop();
   }
 
@@ -68,6 +71,7 @@ export class Player {
     const run = this.run;
     if (!run) return;
     this.run = null;
+    this.videoWait = null;
     this.input.detach();
     soundLibrary.stopAll();
     videoLibrary.endRun();
@@ -83,6 +87,13 @@ export class Player {
   tick(dt: number): void {
     const run = this.run;
     if (!run) return;
+    if (this.waitsForVideos()) {
+      this.input.endFrame();
+      run.world.dropPointerEdges();
+      this.screen.redraw();
+      requestRender();
+      return;
+    }
     const step = gameClock.advance(dt, run.world.paused);
     const request = run.tick(step, timeStep(dt, gameClock.scale, false));
     if (request === 'stop') this.stop();
@@ -91,9 +102,22 @@ export class Player {
       this.start();
     } else if (request === 'scene') {
       this.screen.setWorld(run.world);
+      this.waitForVideos();
       if (run.world.stopRequested) this.stop();
     }
     if (this.run) this.screen.redraw();
     requestRender();
   }
+
+  /** The screen has drawn the new world once by now, which opened the players it shows. */
+  private waitForVideos(): void {
+    this.videoWait = this.screen.videosLoading() ? performance.now() + VIDEO_WAIT_MS : null;
+  }
+
+  private waitsForVideos(): boolean {
+    if (this.videoWait === null) return false;
+    if (performance.now() < this.videoWait && this.screen.videosLoading()) return true;
+    this.videoWait = null;
+    return false;
+  }
 }

src/viewport/VideoLibrary.ts

@@ -33,6 +33,8 @@ const END_SLACK = 1 / 60;
 const IDLE_MS = 2000;
 /** Closer than this a seek that waited for the one before it is not worth another. */
 const SEEK_SLACK = 1 / 120;
+/** How long a run waits at its start for the driven videos it shows (#404) before it goes anyway. */
+export const VIDEO_WAIT_MS = 3000;
 
 /** What the wearer of a driven player (#404) asks of it, see `wear`. */
 interface Drive {
@@ -111,6 +113,21 @@ export class VideoLibrary {
     return { ...ref, file: key };
   }
 
+  /** Whether the player behind `key` has its first picture; it does not count as an ask. */
+  hasPicture(key: string): boolean {
+    return this.players.get(key)?.texture != null;
+  }
+
+  /**
+   * Whether a driven player (#404) the view `view` wears (see `wear`) is still without its first
+   * picture. A run waits for them at its start: the Game tab opens players of its own at Play.
+   */
+  loading(view: string): boolean {
+    const mark = `@${view}:`;
+    for (const [key, player] of this.players) if (!player.texture && key.includes(mark)) return true;
+    return false;
+  }
+
   /** A run begins: every player from its start, heard. A driven one goes where its wearer says. */
   startRun(): void {
     this.heard = true;

src/viewport/WorldViewport.ts

@@ -769,8 +769,16 @@ export abstract class WorldViewport<S extends ViewSource = ViewSource> extends W
     if (wornShader(item.props, shaders)?.time || (item.texture && isVideoKey(item.texture.file)))
       this.animated = true;
     const drawn = this.wearVideo(item);
-    syncSpriteMesh(sprite, drawn, size, textureLibrary, shaders);
     let ghost = this.ghostSprites.get(item.id);
+    // A driven video (#404) is left out until it has its first picture: its plain colour flashed
+    // white at the start of a run and at a cut.
+    if (drawn !== item && !videoLibrary.hasPicture(drawn.texture?.file ?? '')) {
+      sprite.visible = false;
+      if (ghost) disposeSpriteMesh(ghost);
+      this.ghostSprites.delete(item.id);
+      return;
+    }
+    syncSpriteMesh(sprite, drawn, size, textureLibrary, shaders);
     if (!this.ghosted(item, onScreen)) {
       if (ghost) disposeSpriteMesh(ghost);
       this.ghostSprites.delete(item.id);
@@ -799,6 +807,11 @@ export abstract class WorldViewport<S extends ViewSource = ViewSource> extends W
     return texture === item.texture ? item : { ...item, texture };
   }
 
+  /** Whether a driven video (#404) this view wears is still without its first picture. */
+  videosLoading(): boolean {
+    return videoLibrary.loading(this.wearer);
+  }
+
   /** Whether a driven video (#404) drawn here may be heard: in a run, and in the animation preview. */
   protected hearsVideo(): boolean {
     return false;

tests/viewport/video-library.test.ts

@@ -127,4 +127,18 @@ describe('VideoLibrary driven players (#404)', () => {
       [`${KEY}@view1:b`, 1.25],
     ]);
   });
+
+  it('tells a view whether one of its driven players still waits for its first picture', async () => {
+    const videos = await freshLibrary();
+    videos.get(KEY);
+    expect(videos.loading('view12')).toBe(false);
+    const worn = videos.wear(REF, { videoClock: 'driven', videoTime: 0 }, 'view12:a', false);
+    expect(videos.hasPicture(worn.file)).toBe(false);
+    // A run waits only for its own view: view12 is neither view1 nor view2.
+    expect([videos.loading('view12'), videos.loading('view1'), videos.loading('view2')]).toEqual([
+      true,
+      false,
+      false,
+    ]);
+  });
 });

tools/e2e/66-video.mjs

@@ -455,6 +455,41 @@ export async function video(t) {
     JSON.stringify(inRun),
   );
 
+  // 66p. Play opened the Game tab's players only then: unless that tab had been shown just before,
+  //      a driven sprite showed its plain white for the first frames of a run. Now the run waits
+  //      for their first pictures, the clock held, and a sprite without one is not drawn. LATE goes
+  //      in front of the camera (and of what earlier steps left there); every frame of the run is
+  //      read in the page. 4.5 s first, so the players 66n opened are let go.
+  await mutate(`
+    const scene = __gm.state.activeScene;
+    const cam = scene.objects.find((o) => o.id === 'o3').position;
+    Object.assign(scene.objects.find((o) => o.id === '${LATE}').position, { x: cam.x + 120, y: cam.y + 80, z: 1000 });`);
+  await sleep(4500);
+  await t.evaluate(`window.__vd66p = [];
+    (function read() {
+      const info = __gm.playInfo();
+      const rect = info.playing && __gm.gameRect('${LATE}');
+      if (rect) window.__vd66p.push({ color: __gm.pixelAt(Math.round(rect.x), Math.round(rect.y)), waiting: info.waiting, time: info.time });
+      if (window.__vd66p.length < 90) requestAnimationFrame(read);
+    })();`);
+  await play();
+  await sleep(1500);
+  await stop();
+  const runFrames = await json('window.__vd66p.splice(0)');
+  const waited = runFrames.filter((f) => f.waiting);
+  check(
+    'a run waits for the pictures of its driven videos and never shows a driven sprite white',
+    runFrames.length >= 20 &&
+      waited.length > 0 &&
+      waited.every((f) => f.time === 0) &&
+      runFrames.every((f) => f.color !== COLORS.white) &&
+      runFrames.at(-1).color === COLORS.blue,
+    JSON.stringify({
+      waited: waited.length,
+      frames: runFrames.map((f) => `${f.color}${f.waiting ? '*' : ''}`),
+    }),
+  );
+
   await mutate(`
     const p = __gm.state.project;
     const gone = ['${OBJECT}', '${EARLY}', '${LATE}'];