Silence a free video once its sprite is destroyed (#420)

A sprite whose Video clock is free played its video on after it was
destroyed, heard in a game, until the player was let go 2 to 4 s
later. A driven one was already held and quiet after 100 ms.

- The library holds and quiets a free player nothing asked for in
  HUSH_MS, as it did a driven one. That covers Destroy, a scene load,
  the end of a run and a sprite switched off.
- Asked for again before it is let go, it plays on from where it was
  held, heard only in a run. A hidden page leaves it alone, as the
  sweep does.
- A sprite only out of view keeps its free video playing and heard.
  Whether a video is heard should not hang on the camera. Before, it
  ran 2 to 4 s and was then let go. Now it plays on, which opens its
  player even where it was never in view.

Tests:
- Unit: video-library 1 new. A free player losing its wearer is held
  and quiet 200 ms later, stays so through a new run, plays on heard
  when asked again with the same element, and runs on on a hidden
  page.
- e2e 66 on melektaus-atx: ALL PASSED, 36 checks, 66g-ii new. Heard
  31 frames before Destroy, held and muted in every frame read after.
- Counter-probe without hushing free players: the unit test and only
  66g-ii fail, the player plays on heard (paused false, muted false).
- Gate on melektaus-atx: 309 files, 4797 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -47,7 +47,7 @@ npm run build        # writes dist/ and regenerates index.html; the folder is wh
 The browser suite and the three example games need a machine with a GPU:
 
 ```bash
-node tools/seed-e2e.mjs && node tools/e2e.mjs      # the editor, 2073 checks (GM_STEPS=20 for one step)
+node tools/seed-e2e.mjs && node tools/e2e.mjs      # the editor, 2074 checks (GM_STEPS=20 for one step)
                                                    #   every tool signs in first – see the Accounts page
 node tools/seed-jumprun.mjs && node tools/smoke-jumprun.mjs
 node tools/seed-tetris.mjs && node tools/smoke-tetris.mjs

src/viewport/VideoLibrary.ts

@@ -39,9 +39,9 @@ const END_SLACK = 1 / 60;
 /** A player nothing has asked for in this long is let go: no sprite wears it any more. */
 const IDLE_MS = 2000;
 /**
- * A driven player (#404) nothing has asked for in this long is held and quiet: its wearer is gone,
- * like the sprite of a composition item that has ended, or no longer drawn. It runs on by itself
- * otherwise, heard, until it is let go.
+ * A player nothing has asked for in this long is held and quiet: its wearer is gone, like a sprite
+ * destroyed or the sprite of a composition item that has ended, or no longer drawn. It runs on by
+ * itself otherwise, heard, until it is let go. A free one (#420) plays on from there once asked again.
  */
 const HUSH_MS = 100;
 /** Seconds a driven video's level takes to get most of the way to a new one, see `hear`. */
@@ -90,6 +90,8 @@ interface Player {
   asked: number;
   /** Set when its wearer sets the time rather than the video itself. */
   drive: Drive | null;
+  /** Whether a free player is held and quiet because nothing wears it, see `hush`. */
+  hushed: boolean;
 }
 
 export class VideoLibrary {
@@ -120,6 +122,7 @@ export class VideoLibrary {
     const player = this.players.get(key) ?? this.start(key);
     if (!player) return null;
     player.asked = performance.now();
+    if (player.hushed) this.unhush(player);
     return player.texture;
   }
 
@@ -170,12 +173,15 @@ export class VideoLibrary {
     return false;
   }
 
-  /** A run begins: every player from its start, heard. A driven one goes where its wearer says. */
+  /**
+   * A run begins: every player from its start, heard; a hushed one once it is worn again. A driven
+   * one goes where its wearer says.
+   */
   startRun(): void {
     this.heard = true;
     this.paused = false;
     for (const player of this.freePlayers()) {
-      this.hear(player);
+      if (!player.hushed) this.hear(player);
       this.rewind(player);
     }
   }
@@ -194,7 +200,7 @@ export class VideoLibrary {
     this.paused = paused;
     for (const player of this.freePlayers())
       if (paused) player.element.pause();
-      else void player.element.play().catch(() => {});
+      else if (!player.hushed) void player.element.play().catch(() => {});
   }
 
   private freePlayers(): Player[] {
@@ -246,18 +252,30 @@ export class VideoLibrary {
   }
 
   /**
-   * Holds and quiets the driven players nobody asked for lately, see `HUSH_MS`. One still without
-   * its first picture plays on, without sound: it only gets one by playing (see `openVideo`).
+   * Holds and quiets the players nobody asked for lately, see `HUSH_MS`. One still without its first
+   * picture plays on, without sound: it only gets one by playing (see `openVideo`). A free one is
+   * left alone while the page is hidden, as by `sweep`.
    */
   private hush(): void {
     const now = performance.now();
     for (const player of this.players.values()) {
-      if (!player.drive || !player.texture || now - player.asked <= HUSH_MS) continue;
+      if (player.drive) {
+        if (!player.texture || now - player.asked <= HUSH_MS) continue;
+      } else if (player.hushed || document.hidden || now - Math.max(player.asked, this.shown) <= HUSH_MS)
+        continue;
+      player.hushed = !player.drive;
       player.element.muted = true;
-      if (!player.element.paused) player.element.pause();
+      if (player.texture && !player.element.paused) player.element.pause();
     }
   }
 
+  /** A free player worn again plays on from where it was held, heard in a run. */
+  private unhush(player: Player): void {
+    player.hushed = false;
+    if (this.heard) this.hear(player);
+    if (!this.paused) void player.element.play().catch(() => {});
+  }
+
   private stopSweeping(): void {
     if (this.sweeper !== null) clearInterval(this.sweeper);
     if (this.husher !== null) clearInterval(this.husher);
@@ -306,8 +324,8 @@ export class VideoLibrary {
         if (player.gone) return texture.destroy(true);
         player.texture = texture;
         if (drive) this.follow(player, drive);
-        // Started even in a paused run, and held once there is a picture.
-        else if (this.paused) player.element.pause();
+        // Started even in a paused run or when nothing wears it, and held once there is a picture.
+        else if (this.paused || player.hushed) player.element.pause();
         this.notify();
       }),
       texture: null,
@@ -316,10 +334,11 @@ export class VideoLibrary {
       gone: false,
       asked: performance.now(),
       drive,
+      hushed: false,
     };
     this.players.set(key, player);
     this.sweeper ??= setInterval(() => this.sweep(), IDLE_MS);
-    if (drive) this.husher ??= setInterval(() => this.hush(), HUSH_MS);
+    this.husher ??= setInterval(() => this.hush(), HUSH_MS);
     if (drive)
       chaseSeeks(
         player.element,
@@ -376,7 +395,7 @@ export class VideoLibrary {
 
   private rewind(player: Player): void {
     player.element.currentTime = player.play.start;
-    if (!this.paused) void player.element.play().catch(() => {});
+    if (!this.paused && !player.hushed) void player.element.play().catch(() => {});
   }
 
   private drop(player: Player): void {

src/viewport/WorldViewport.ts

@@ -861,8 +861,14 @@ export abstract class WorldViewport<S extends ViewSource = ViewSource> extends W
       this.ghostSprites.delete(item.id);
       this.videosShown.delete(item.id);
       // A driven video (#404) is kept ready at its Video time while its sprite is switched off or out
-      // of view: switched on, it showed its plain colour until the video had loaded.
-      if (item.type === 'sprite' && this.wearVideo(item, false) !== item) this.animated = true;
+      // of view: switched on, it showed its plain colour until the video had loaded. A free one plays
+      // on, heard in a run, while its sprite is only out of view; switched off, it is held (#420).
+      if (item.type !== 'sprite') return;
+      if (this.wearVideo(item, false) !== item) this.animated = true;
+      else if (item.enabled && item.texture && isVideoKey(item.texture.file)) {
+        videoLibrary.get(item.texture.file);
+        this.animated = true;
+      }
       return;
     }
     if (!sprite) {

tests/viewport/video-library.test.ts

@@ -258,6 +258,64 @@ describe('VideoLibrary driven players (#404)', () => {
     vi.useRealTimers();
   });
 
+  it('holds a free player quiet once no sprite wears it, and plays it on when one does (#420)', async () => {
+    vi.useFakeTimers({ toFake: ['setInterval', 'clearInterval', 'performance'] });
+    const page = { hidden: false };
+    const events = new Map<string, () => void>();
+    let frames: (() => void)[] = [];
+    const element = {
+      ...fakeVideo(),
+      play() {
+        this.paused = false;
+        return Promise.resolve();
+      },
+      pause() {
+        this.paused = true;
+      },
+      addEventListener: (type: string, listener: () => void) => events.set(type, listener),
+      requestVideoFrameCallback: (callback: () => void) => frames.push(callback),
+    };
+    vi.stubGlobal('document', {
+      get hidden() {
+        return page.hidden;
+      },
+      addEventListener: () => {},
+      createElement: () => element,
+    });
+    const { soundLibrary } = await import('../../src/viewport/SoundLibrary');
+    vi.spyOn(soundLibrary, 'mediaInput').mockReturnValue(null);
+    const videos = await freshLibrary();
+    const drawFor = (ms: number) => {
+      for (let at = 0; at < ms; at += 1000 / 60) {
+        videos.get(KEY);
+        vi.advanceTimersByTime(1000 / 60);
+      }
+    };
+    videos.get(KEY);
+    events.get('loadeddata')!();
+    [frames] = [[], ...frames.map((frame) => frame())];
+    videos.startRun();
+    drawFor(1000);
+    const state = () => [element.paused, element.muted];
+    expect(state()).toEqual([false, false]);
+    // Its sprite is destroyed: held and quiet at once, not heard until it is let go 2-4 s later.
+    vi.advanceTimersByTime(200);
+    expect(state()).toEqual([true, true]);
+    // A run started meanwhile takes it back to its start, still quiet.
+    videos.startRun();
+    vi.advanceTimersByTime(200);
+    expect(state()).toEqual([true, true]);
+    // Worn again before it is let go: it plays on, heard, the same video.
+    videosMade = 0;
+    drawFor(500);
+    expect([...state(), videosMade]).toEqual([false, false, 0]);
+    // A hidden page draws nothing, and asks for nothing: it plays on as before.
+    page.hidden = true;
+    vi.advanceTimersByTime(1000);
+    expect(state()).toEqual([false, false]);
+    vi.useRealTimers();
+  });
+
   it('tells a view whether one of its driven players still waits for its first picture', async () => {
     const videos = await freshLibrary();
     videos.get(KEY);

tools/e2e/66-video.mjs

@@ -1,4 +1,5 @@
 import { readFileSync } from 'node:fs';
+import { edge, node } from '../lib/builders.mjs';
 import { uploadVideo } from '../lib/projects.mjs';
 import { LANES, MAKE_OBJECT, pickAssetType } from './lib/scaffold.mjs';
 import { sampleZoom } from './lib/zoomTransition.mjs';
@@ -17,6 +18,8 @@ import { sampleZoom } from './lib/zoomTransition.mjs';
  */
 const SPRITE = 'vd66-sprite';
 const OBJECT = 'vd66-object';
+const LOGIC = 'vd66-logic';
+const BRAIN = 'vd66-brain';
 const DRIVEN = 'vd66-driven';
 const EARLY = 'vd66-early';
 const LATE = 'vd66-late';
@@ -26,7 +29,7 @@ const LANE = LANES.video;
 const COLORS = { red: '#fe0000', green: '#00ff01', blue: '#0000ff', white: '#ffffff' };
 
 export async function video(t) {
-  const { check, sleep, json, mutate, click, clickText, drag, shot, showTab, play, stop } = t;
+  const { check, sleep, json, mutate, click, clickText, drag, press, shot, showTab, play, stop } = t;
   const waitFor = async (read, done, tries = 40) => {
     let value = await read();
     for (let i = 0; i < tries && !done(value); i++) {
@@ -323,6 +326,66 @@ export async function video(t) {
       runRoutes.some((r) => r.file === uploaded.file && r.routed && r.bus === `mix:${BUS}`),
     JSON.stringify({ runPeaks, runRoutes }),
   );
+
+  // 66g-ii (#420). A behavior destroys the sprite: its free video stops and goes quiet with it. It
+  //      played on, heard, until it was let go 2-4 s later. Every frame is read in the page.
+  await stop();
+  await mutate(`
+    const p = __gm.state.project;
+    const scene = __gm.state.activeScene;
+    const logic = structuredClone(p.assets.find((a) => a.id === 'mover'));
+    Object.assign(logic, {
+      id: '${LOGIC}',
+      name: '${LOGIC}',
+      props: { enabled: true, runInEditor: false },
+      variables: [],
+      graph: ${JSON.stringify({
+        nodes: [
+          node('vd66-key', 'OnKeyDown', 0, 0, { key: 'd' }),
+          node('vd66-owner', 'Owner', 0, 180),
+          node('vd66-destroy', 'Destroy', 260, 0),
+        ],
+        edges: [
+          edge('vd66-e1', 'vd66-key', 'exec', 'vd66-destroy', 'exec'),
+          edge('vd66-e2', 'vd66-owner', 'object', 'vd66-destroy', 'object'),
+        ],
+      })},
+    });
+    p.assets.unshift(logic);
+    ${MAKE_OBJECT}
+    scene.objects.push(make('${BRAIN}', '${LOGIC}', '${OBJECT}', 0, '${BRAIN}'));`);
+  await play();
+  await t.evaluate(`window.__vd66g = new Promise((done) => {
+    const rows = [];
+    let gone = -1;
+    const step = () => {
+      const exists = (__gm.world()?.entities ?? []).some((e) => e.objectId === '${OBJECT}');
+      const players = __gm.videos().filter((v) => !v.key.includes('@'));
+      rows.push({ exists, players: players.length, paused: players[0]?.paused ?? null, muted: players[0]?.muted ?? null });
+      if (gone < 0 && rows.some((row) => row.exists) && !exists) gone = rows.length - 1;
+      if ((gone >= 0 && rows.length > gone + 30) || rows.length >= 600 || !__gm.world())
+        return done(JSON.stringify({ gone, heard: rows.slice(0, Math.max(gone, 0)).filter((row) => row.paused === false && row.muted === false).length, after: rows.slice(gone + 12) }));
+      requestAnimationFrame(step);
+    };
+    step();
+  });
+  0;`);
+  await sleep(500);
+  await press('d');
+  const destroyed = JSON.parse(await t.evaluate('window.__vd66g'));
+  check(
+    'a free video stops and goes quiet as soon as its sprite is destroyed',
+    destroyed.gone > 0 &&
+      destroyed.heard > 0 &&
+      destroyed.after.length > 0 &&
+      destroyed.after.every((row) => row.players === 1 && row.paused === true && row.muted === true),
+    JSON.stringify({
+      gone: destroyed.gone,
+      heard: destroyed.heard,
+      after: destroyed.after.slice(0, 3),
+      last: destroyed.after.at(-1),
+    }),
+  );
   await stop();
   await sleep(200);
   const quiet = (await json('__gm.videos()'))[0];
@@ -333,7 +396,8 @@ export async function video(t) {
   //      leave the preview drawing a destroyed one, and the batcher threw on every frame.
   await mutate(`
     const p = __gm.state.project;
-    for (const holder of p.assets) if (holder.objects) holder.objects = holder.objects.filter((o) => o.id !== '${OBJECT}');
+    for (const holder of p.assets) if (holder.objects) holder.objects = holder.objects.filter((o) => o.id !== '${OBJECT}' && o.id !== '${BRAIN}');
+    p.assets = p.assets.filter((a) => a.id !== '${LOGIC}');
     __gm.state.select({ kind: 'asset', assetId: '${SPRITE}' });`);
   const logFrom = t.consoleLog.length;
   await sleep(4500);