Let an animation drive a sprite's video picture by picture (#404)

A video used to run on its own clock as soon as a sprite wore it. An
animation could switch the sprite on and off, but never say which
picture shows at which frame. Cutting a video in an animation needs
the opposite: the animation sets the time and the video follows.

- Sprites wearing a video or a clip get two rows, Video clock and
  Video time. They show only while the texture holds a video.
- With Video clock on Driven the video does not play by itself. It
  shows the picture at Video time, counted from the clip's start.
  Past the end it goes round with Loop and holds the last picture
  without.
- Video time is a number like any other, so keys, step, easing and
  curves give stills, slow motion and ramps.
- A driven sprite gets a player of its own, keyed by the wearer's
  name. The same clip can run twice at different times.
- Following has two ways (`videoFollow.ts`). Running forward at 0.25x
  to 4x the video plays at that rate and is nudged. A still, a scrub
  or a step backwards seeks, and only the newest seek counts.
- Sound is heard only near 1x. The animation tab now plays it too, so
  a cut can follow the sound. Scrubbing is silent.
- Scene, game view, the inspector's PREVIEW and the shader preview
  show the driven picture. In a running game Play animation sets it.

Found on the way:

- After a scrub the video in the animation tab kept running. The
  preview redrew only while the playhead moved, so nobody asked for
  the video once it stood. `AnimationEditor.tick` now redraws it every
  frame while it shows a video.
- Every sprite has two more defaults (`videoClock: 'free'`,
  `videoTime: 0`). Eight tests in five files pinned the old list and
  are updated.

Tests:
- Unit: video-clock and video-follow new, 4 files with 42 tests green.
- e2e 66 (melektaus-atx): ALL PASSED, 22 checks, 3 new. 66i: two
  sprites on one clip stand at 0.2 s red and 0.7 s green. 66j: a
  scrub to frames 2, 7, 12 and 3 shows red, green, blue, red, and
  each colour still stands 0.7 s later. 66k: playing runs 10-30 ms
  behind at rate 1.00-1.03 and is not muted.
- Counterchecks: a seek that keeps the rate turns the seek test red.
  Not seeing Driven turns 66i, 66j and 66k red.
- Gate (melektaus-atx): exit 0, 297 files, 4546 tests.
- Not tested in the browser: Play animation in a running game.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

README.md

@@ -47,7 +47,7 @@ npm run build        # writes dist/ and regenerates index.html; the folder is wh
 The browser suite and the three example games need a machine with a GPU:
 
 ```bash
-node tools/seed-e2e.mjs && node tools/e2e.mjs      # the editor, 1996 checks (GM_STEPS=20 for one step)
+node tools/seed-e2e.mjs && node tools/e2e.mjs      # the editor, 1999 checks (GM_STEPS=20 for one step)
                                                    #   every tool signs in first – see the Accounts page
 node tools/seed-jumprun.mjs && node tools/smoke-jumprun.mjs
 node tools/seed-tetris.mjs && node tools/smoke-tetris.mjs

src/core/model/video.ts

@@ -84,22 +84,58 @@ export function videoKey(play: VideoPlay): string {
   return `${PREFIX}${play.file}#${play.start}~${end}~${play.loop ? 1 : 0}~${play.volume}`;
 }
 
+/**
+ * The key of a player one wearer drives on its own (#404): two sprites driving the same clip must
+ * not fight over one `<video>`, so the wearer's name goes after an `@`.
+ */
+export function drivenVideoKey(key: string, wearer: string): string {
+  return `${key}@${wearer}`;
+}
+
 export function isVideoKey(key: string): boolean {
   return parseVideoKey(key) !== null;
 }
 
+/** Whether the key names a player its wearer drives, see `drivenVideoKey`. */
+export function isDrivenVideoKey(key: string): boolean {
+  return isVideoKey(key) && key.includes('@');
+}
+
 export function parseVideoKey(key: string): VideoPlay | null {
   if (!key.startsWith(PREFIX)) return null;
   const hash = key.indexOf('#');
   if (hash < 0) return null;
   const file = key.slice(PREFIX.length, hash);
-  const parts = key.slice(hash + 1).split('~');
+  const at = key.indexOf('@', hash);
+  const parts = key.slice(hash + 1, at < 0 ? undefined : at).split('~');
   const [start, end, loop, volume] = parts.map(Number);
   if (!isVideoFile(file) || parts.length !== 4 || ![start, end, loop, volume].every(Number.isFinite))
     return null;
   return { file, start: start!, end: end! > 0 ? end! : Infinity, loop: loop === 1, volume: volume! };
 }
 
+/** How far before its end a held clip shows its last picture: the end itself is the next one's. */
+const LAST_PICTURE = 0.001;
+
+/**
+ * Where in its file a driven video stands at `time` seconds into its clip (#404). Before the start
+ * it shows the first picture, past the end it goes round with Loop and holds the last one without.
+ * `duration` is the file's, which a clip running to the end of the video needs; unknown (NaN, not
+ * loaded yet) the time is not bounded at the end.
+ */
+export function drivenTimeOf(
+  play: Pick<VideoPlay, 'start' | 'end' | 'loop'>,
+  time: number,
+  duration: number,
+): number {
+  const end = Number.isFinite(play.end) ? play.end : duration;
+  const length = end - play.start;
+  const run = Math.max(0, Number.isFinite(time) ? time : 0);
+  if (!(length > 0)) return play.start + run;
+  if (play.loop) return play.start + (run % length);
+  return play.start + Math.min(run, Math.max(0, length - LAST_PICTURE));
+}
+
 /** Where in its file a player is after `elapsed` seconds of playing (#402). */
 export function videoTimeAt(play: Pick<VideoPlay, 'start' | 'end' | 'loop'>, elapsed: number): number {
   const length = play.end - play.start;

src/core/properties/PropertySchema.ts

@@ -129,7 +129,8 @@ import {
 import { DEFAULT_SIGN, DEFAULT_TEMPO, MAX_TEMPO, MIN_TEMPO, SIGN_OPTIONS } from '../music/time';
 import { ASSET_FX_KEY } from './assetFx';
 import { BUS_FX_KEY, BUS_VOLUME_KEY } from './buses';
-import { VIDEO_ROWS } from './videos';
+import { SPRITE_VIDEO_ROWS, VIDEO_ROWS } from './videos';
+import { findVideo } from '../model/video';
 import { Theme } from '../../ui';
 
 /**
@@ -166,7 +167,8 @@ type Extras<K extends FlatKind> = K extends 'number'
  * (see `isRowShown`), everything else – graph, defaults, files – always has it. `shownFor` does the
  * same by a choice: the row is shown while the enum under `key` holds one of `values` (#154); a
  * list of such choices must all hold. `shownWhenSet` names a text or asset row the row belongs to:
- * shown while that one is not empty (#170).
+ * shown while that one is not empty (#170). `shownForVideo` names a texture row: shown while it
+ * holds a video or a clip of one (#404), which only a check that knows the project can tell.
  */
 export type FlatDef<K extends FlatKind = FlatKind> = {
   [P in Exclude<K, 'object'>]: {
@@ -179,6 +181,7 @@ export type FlatDef<K extends FlatKind = FlatKind> = {
     shownWith?: string;
     shownFor?: ShownFor | readonly ShownFor[];
     shownWhenSet?: string;
+    shownForVideo?: string;
   } & Extras<P>;
 }[Exclude<K, 'object'>];
 
@@ -193,8 +196,10 @@ interface ShownFor {
 }
 
 /** Whether the inspector shows the row, given the values of the bag it is on. */
-export function isRowShown(def: PropertyDef, values: PropertyBag): boolean {
+export function isRowShown(def: PropertyDef, values: PropertyBag, project?: Project): boolean {
   if ('noRow' in def && def.noRow) return false;
+  if ('shownForVideo' in def && def.shownForVideo && project)
+    if (!findVideo(project, textOf(values[def.shownForVideo] ?? ''))) return false;
   const choices: readonly ShownFor[] =
     'shownFor' in def && def.shownFor ? ('key' in def.shownFor ? [def.shownFor] : def.shownFor) : [];
   if (choices.some(({ key, values: wanted }) => !wanted.includes(textOf(values[key] ?? '')))) return false;
@@ -532,6 +537,7 @@ const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
     { key: 'color', label: 'Color', kind: 'color', default: 0xffffff },
     // A video or a clip of one plays on the sprite (#402).
     { key: 'texture', label: 'Texture', kind: 'texture', default: '', moving: true },
+    ...SPRITE_VIDEO_ROWS,
     // The shader's own parameters follow this row – see `schemaForAsset`. Empty is no shader at all.
     { key: SHADER_KEY, label: 'Shader', kind: 'asset', default: '', assetTypes: ['shader'] },
   ],

src/core/properties/videos.ts

@@ -15,6 +15,40 @@ export const VIDEO_ROWS: readonly PropertyDef[] = [
   { key: VIDEO_LOOP_KEY, label: 'Loop', kind: 'boolean', default: true },
 ];
 
+/**
+ * Who sets the time of the video a sprite wears (#404): `free` lets it play by itself, `driven`
+ * shows the picture at `videoTime`, seconds into the clip, which an animation keys like any number.
+ */
+export const VIDEO_CLOCK_KEY = 'videoClock';
+export const VIDEO_TIME_KEY = 'videoTime';
+
+export const VIDEO_CLOCK_OPTIONS = [
+  { value: 'free', label: 'Free' },
+  { value: 'driven', label: 'Driven' },
+];
+
+/** On a sprite, below its texture; shown only while the texture is a video or a clip of one. */
+export const SPRITE_VIDEO_ROWS: readonly PropertyDef[] = [
+  {
+    key: VIDEO_CLOCK_KEY,
+    label: 'Video clock',
+    kind: 'enum',
+    default: 'free',
+    options: VIDEO_CLOCK_OPTIONS,
+    shownForVideo: 'texture',
+  },
+  {
+    key: VIDEO_TIME_KEY,
+    label: 'Video time',
+    kind: 'number',
+    default: 0,
+    step: 0.01,
+    decimals: 3,
+    shownFor: { key: VIDEO_CLOCK_KEY, values: ['driven'] },
+    shownForVideo: 'texture',
+  },
+];
+
 /** The rows of a clip of a video, set in the Video tab. */
 export const VIDEO_CLIP_ROWS: readonly PropertyDef[] = [
   { key: VIDEO_START_KEY, label: 'Start (s)', kind: 'number', default: 0, min: 0, step: 0.01, decimals: 3 },
@@ -29,6 +63,14 @@ export function videoLoopsOf(bag: PropertyBag): boolean {
   return bag[VIDEO_LOOP_KEY] === undefined ? true : booleanOf(bag[VIDEO_LOOP_KEY]);
 }
 
+export function videoDrivenOf(bag: PropertyBag): boolean {
+  return bag[VIDEO_CLOCK_KEY] === 'driven';
+}
+
+export function videoTimeOf(bag: PropertyBag): number {
+  return coerceAs(bag[VIDEO_TIME_KEY], 'number', 0);
+}
+
 /** Start and end of a clip in seconds, `end` 0 for the end of the video. */
 export function videoSpanOf(bag: PropertyBag): { start: number; end: number } {
   const seconds = (key: string) => Math.max(0, coerceAs(bag[key], 'number', 0));

src/editor/animation/AnimationEditor.ts

@@ -66,10 +66,15 @@ export class AnimationEditor extends AssetTabEditor<AnimationAsset, TimelineCanv
 
   /** Only called while the tab is visible. */
   tick(dt: number): void {
-    if (!this.canvas.isPlaying) return;
-    this.canvas.tick(dt);
-    // A finished `once` pass pauses the canvas; the button has to say so.
-    this.syncPlayButton();
+    const drawn = this.preview.redrawCount;
+    if (this.canvas.isPlaying) {
+      this.canvas.tick(dt);
+      // A finished `once` pass pauses the canvas; the button has to say so.
+      this.syncPlayButton();
+    }
+    // A video moves by itself, and a driven one (#404) has to be told every frame that the
+    // playhead stands; the playhead redraws only when it moves.
+    this.preview.animate(drawn);
   }
 
   /** Whether the preview is running – a moving picture needs a frame without an event. */

src/editor/animation/PreviewCanvas.ts

@@ -87,6 +87,11 @@ export class PreviewCanvas extends FreeViewport {
     return Theme.colors.viewport;
   }
 
+  /** Heard while it plays at about 1x, so a cut can be timed by its sound (#404). */
+  protected override hearsVideo(): boolean {
+    return true;
+  }
+
   private asset(): AnimationAsset | undefined {
     return this.assetId ? findAssetOf(this.state.project, this.assetId, 'animation') : undefined;
   }

src/editor/inspector/SpritePreview.ts

@@ -8,6 +8,7 @@ import { shaderLibrary } from '../../viewport/ShaderLibrary';
 import { wornShader } from '../../viewport/shaders';
 import { disposeSpriteMesh, newSpriteMesh, syncSpriteMesh } from '../../viewport/symbols';
 import { textureLibrary } from '../../viewport/TextureLibrary';
+import { videoLibrary } from '../../viewport/VideoLibrary';
 import { type PictureSubject, previewPicture } from '../shader/previewPicture';
 import type { EditorState } from '../state';
 import { addTitle, type InspectorHost } from './host';
@@ -73,7 +74,9 @@ export class SpritePreview extends Widget {
     if (!picture) return;
     const shaders = shaderLibrary.source(this.state.project);
     const worn = wornShader(picture.props, shaders);
-    const texture = picture.texture && textureLibrary.get(picture.texture, worn?.edge ?? SHEET_EDGE);
+    // A driven video (#404) stands at its Video time here, as in the Scene tab.
+    const ref = picture.texture && videoLibrary.wear(picture.texture, picture.props, this.frameKey, false);
+    const texture = ref && textureLibrary.get(ref, worn?.edge ?? SHEET_EDGE);
     const pad = quadPadding(worn?.padding ?? 0, texture?.width ?? picture.w, texture?.height ?? picture.h);
     const ratio = ui().app.renderer.resolution;
     const { zoom, x, y } = previewFit(picture.w, picture.h, this, {
@@ -84,13 +87,7 @@ export class SpritePreview extends Widget {
     });
     const [w, h] = [picture.w * zoom, picture.h * zoom];
     const m = { a: zoom, b: 0, c: 0, d: zoom, tx: x + w / 2, ty: y + h / 2 };
-    syncSpriteMesh(
-      this.mesh,
-      { m, props: picture.props, texture: picture.texture },
-      picture,
-      textureLibrary,
-      shaders,
-    );
+    syncSpriteMesh(this.mesh, { m, props: picture.props, texture: ref }, picture, textureLibrary, shaders);
     this.drawn = { x, y, w, h };
     // A video (#402) moves by itself, and its player is let go once nobody asks for its picture.
     const video = picture.texture !== null && isVideoKey(picture.texture.file);

src/editor/inspector/assetSection.ts

@@ -97,7 +97,7 @@ export function assetSignature(project: Project, assetId: string): string {
   const shapes = asset.type === 'vectorart' ? `|${shapeListSignature(asset)}` : '';
   const values = resolveAssetProps(project, asset);
   const rows = schemaForAsset(project, asset).filter(
-    (def) => isRowShown(def, values) && !isRowHiddenWhenEmbedded(asset, def.key),
+    (def) => isRowShown(def, values, project) && !isRowHiddenWhenEmbedded(asset, def.key),
   );
   // A Type row shows the name of its Struct asset, or "Own fields" once that asset is gone (#253).
   const types = `|types:${typeNames(project, rows)}`;
@@ -207,7 +207,9 @@ const isSwitch = (key: string): boolean =>
 function buildProps(host: InspectorHost, asset: Asset, schema: readonly PropertyDef[], title: string): void {
   const project = () => host.state.project;
   const values = resolveAssetProps(project(), asset);
-  const shown = schema.filter((d) => isRowShown(d, values) && !isRowHiddenWhenEmbedded(asset, d.key));
+  const shown = schema.filter(
+    (d) => isRowShown(d, values, project()) && !isRowHiddenWhenEmbedded(asset, d.key),
+  );
   if (shown.length > 0) addTitle(host, title);
   for (const def of shown) {
     const row = createFieldRow(

src/editor/inspector/objectSection.ts

@@ -107,7 +107,7 @@ export function objectSignature(
       ? 'instance'
       : '';
   const values = resolveSceneObjectProps(project, asset, object);
-  const rows = schemaForObject(project, asset, object).filter((def) => isRowShown(def, values));
+  const rows = schemaForObject(project, asset, object).filter((def) => isRowShown(def, values, project));
   // A tilemap warns when it is placed more than once, so the number of placements shapes it too.
   const placed = asset.type === 'tilemap' ? `|placed:${findUsages(project, asset.id).length}` : '';
   // A variable turned private leaves the rows, with nothing else about it changed (#257).
@@ -257,7 +257,7 @@ export function buildOwnRows(host: InspectorHost, holderId: string, objectId: st
   if (!shown) return;
   const { object, asset } = shown;
   const values = resolveSceneObjectProps(project, asset, object);
-  const own = objectRows(project, asset, object).filter((def) => isRowShown(def, values));
+  const own = objectRows(project, asset, object).filter((def) => isRowShown(def, values, project));
   host.objectChoices = () => pickableObjects(host.state.project, holderId, objectId);
   host.objectPicker = objectPicker(host.state, holderId, objectId, host.objectChoices);
   const behavior = asset.type === 'behavior';

src/editor/shader/ShaderPreview.ts

@@ -10,6 +10,7 @@ import { syncShader } from '../../viewport/shaders';
 import { disposeSpriteMesh, newSpriteMesh, syncQuadPadding } from '../../viewport/symbols';
 import { quadPadding } from '../../viewport/quadPadding';
 import { textureLibrary } from '../../viewport/TextureLibrary';
+import { videoLibrary } from '../../viewport/VideoLibrary';
 import type { EditorState } from '../state';
 import { type PreviewPicture, TEST_IMAGE_SIZE, testImagePixels } from './previewPicture';
 
@@ -18,6 +19,8 @@ const MARGIN = 12;
 /** A 16-px sprite fills a big panel only blurred by the zoom; past this it stops growing. */
 const MAX_ZOOM = 8;
 
+let previews = 0;
+
 /**
  * One picture wearing the shown shader, with the sliders' values instead of anyone's rows (#168).
  * The shader is the scene's own (`syncShader`), so what the panel shows is what a sprite would.
@@ -35,6 +38,7 @@ export class ShaderPreview extends Widget {
   /** Null: the clock runs. A number: the preview stands at that second. */
   private held: number | null = null;
   private readonly stopPictures: () => void;
+  private readonly wearer = `shader-preview:${(previews += 1)}`;
 
   constructor(private readonly state: EditorState) {
     super();
@@ -106,9 +110,9 @@ export class ShaderPreview extends Widget {
     const w = picture?.w ?? TEST_IMAGE_SIZE;
     const h = picture?.h ?? TEST_IMAGE_SIZE;
     // A sprite whose picture is still loading shows its plain colour, like the scene does.
-    const texture = picture
-      ? ((picture.texture && textureLibrary.get(picture.texture, edge)) ?? null)
-      : testTexture(edge);
+    // A driven video (#404) stands at its Video time here, as in the Scene tab.
+    const ref = picture?.texture && videoLibrary.wear(picture.texture, picture.props, this.wearer, false);
+    const texture = picture ? ((ref && textureLibrary.get(ref, edge)) ?? null) : testTexture(edge);
     // The shader's padding in pixels of the picture, as the scene counts it; the panel keeps room for it.
     const pad = quadPadding(worn?.padding ?? 0, texture?.width ?? w, texture?.height ?? h);
     const zoom = Math.max(

src/editor/viewport/GameViewport.ts

@@ -77,6 +77,10 @@ export class GameViewport extends WorldViewport<EditorState> {
     return this.performanceShown;
   }
 
+  protected override hearsVideo(): boolean {
+    return this.playWorld !== null;
+  }
+
   setPerformanceShown(on: boolean): void {
     this.performanceShown = on;
     this.syncPerformance();

src/player/GameScreen.ts

@@ -61,6 +61,10 @@ export class GameScreen extends WorldViewport {
     return this.startable && !this.running;
   }
 
+  protected override hearsVideo(): boolean {
+    return this.running !== null;
+  }
+
   /** null = not running. */
   setWorld(world: World | null): void {
     this.running = world;

src/viewport/VideoLibrary.ts

@@ -6,17 +6,44 @@
  * In the editor a video plays without sound, as a preview. A run starts every player over at its
  * clip's start and lets it be heard through the master bus; a pause holds them, and the end of the
  * run makes them quiet previews again.
+ *
+ * A sprite whose Video clock is driven (#404) has a player of its own instead, which shows the
+ * picture at its Video time and leaves runs and pauses to it (see `wear` and `videoFollow.ts`).
  */
 import { Assets, Texture, VideoSource } from 'pixi.js';
-import { parseVideoKey, type VideoPlay } from '../core/model/video';
+import type { PropertyBag } from '../core/model/Properties';
+import type { TextureRef } from '../core/model/texture';
+import {
+  drivenTimeOf,
+  drivenVideoKey,
+  isDrivenVideoKey,
+  isVideoKey,
+  parseVideoKey,
+  type VideoPlay,
+} from '../core/model/video';
+import { videoDrivenOf, videoTimeOf } from '../core/properties/videos';
 import { FileLibrary } from './FileLibrary';
 import { soundLibrary } from './SoundLibrary';
+import { followVideo, readSpeed, type SpeedReading } from './videoFollow';
 
 const VIDEO_BASE = '/game-maker/data/videos';
 /** How close to its end a clip counts as there: half a frame at 30 fps. */
 const END_SLACK = 1 / 60;
 /** A player nothing has asked for in this long is let go: no sprite wears it any more. */
 const IDLE_MS = 2000;
+/** Closer than this a seek that waited for the one before it is not worth another. */
+const SEEK_SLACK = 1 / 120;
+
+/** What the wearer of a driven player (#404) asks of it, see `wear`. */
+interface Drive {
+  /** Seconds into the clip. */
+  time: number;
+  /** Whether the wearer lets it be heard, when it plays at about 1x. */
+  heard: boolean;
+  reading: SpeedReading | null;
+  /** The newest seek asked for while one was still running: only that one follows. */
+  wanted: number | null;
+}
 
 interface Player {
   play: VideoPlay;
@@ -28,6 +55,8 @@ interface Player {
   gone: boolean;
   /** When a sprite last asked for its picture, by `performance.now()`. */
   asked: number;
+  /** Set when its wearer sets the time rather than the video itself. */
+  drive: Drive | null;
 }
 
 export class VideoLibrary {
@@ -60,11 +89,30 @@ export class VideoLibrary {
     return player.texture;
   }
 
-  /** A run begins: every player from its start, heard. */
+  /**
+   * What a sprite wears (#404): with its clock driven, a player of its own that shows the picture
+   * at its Video time; otherwise the ref as it is. `wearer` names the sprite where it is drawn, so
+   * the same object in two views drives two players. Asked every frame, like `get`.
+   */
+  wear(ref: TextureRef, props: PropertyBag, wearer: string, heard: boolean): TextureRef {
+    if (!videoDrivenOf(props) || !isVideoKey(ref.file)) return ref;
+    const key = drivenVideoKey(ref.file, wearer);
+    const time = videoTimeOf(props);
+    const player = this.players.get(key) ?? this.start(key, time);
+    if (!player?.drive) return ref;
+    const drive = player.drive;
+    drive.time = time;
+    drive.heard = heard;
+    drive.reading = readSpeed(drive.reading, time, performance.now());
+    if (player.texture) this.follow(player, drive);
+    return { ...ref, file: key };
+  }
+
+  /** A run begins: every player from its start, heard. A driven one goes where its wearer says. */
   startRun(): void {
     this.heard = true;
     this.paused = false;
-    for (const player of this.players.values()) {
+    for (const player of this.freePlayers()) {
       this.hear(player);
       this.rewind(player);
     }
@@ -74,7 +122,7 @@ export class VideoLibrary {
   endRun(): void {
     this.heard = false;
     this.paused = false;
-    for (const player of this.players.values()) {
+    for (const player of this.freePlayers()) {
       player.element.muted = true;
       this.rewind(player);
     }
@@ -82,11 +130,41 @@ export class VideoLibrary {
 
   setPaused(paused: boolean): void {
     this.paused = paused;
-    for (const player of this.players.values())
+    for (const player of this.freePlayers())
       if (paused) player.element.pause();
       else void player.element.play().catch(() => {});
   }
 
+  private freePlayers(): Player[] {
+    return [...this.players.values()].filter((player) => !player.drive);
+  }
+
+  private follow(player: Player, drive: Drive): void {
+    const { element, play } = player;
+    const step = followVideo({
+      target: drivenTimeOf(play, drive.time, element.duration),
+      speed: drive.reading?.speed ?? 0,
+      current: element.currentTime,
+      end: play.end,
+    });
+    if (step.seek !== null) this.seek(element, drive, step.seek);
+    if (step.kind === 'hold') {
+      element.muted = true;
+      if (!element.paused) element.pause();
+      return;
+    }
+    if (Math.abs(element.playbackRate - step.rate) > 0.001) element.playbackRate = step.rate;
+    if (step.heard && drive.heard) this.hear(player);
+    else element.muted = true;
+    if (element.paused) void element.play().catch(() => {});
+  }
+
+  /** Seeks, or, while a seek is still running, remembers the newest wish for when it is done. */
+  private seek(element: HTMLVideoElement, drive: Drive, time: number): void {
+    if (element.seeking) drive.wanted = time;
+    else element.currentTime = time;
+  }
+
   /** Lets every player go: a session that ends takes its videos along. */
   clear(): void {
     for (const player of this.players.values()) this.drop(player);
@@ -116,40 +194,71 @@ export class VideoLibrary {
   }
 
   /** What each player is doing (test hook). */
-  debugInfo(): { key: string; time: number; paused: boolean; muted: boolean; ready: boolean }[] {
+  debugInfo(): {
+    key: string;
+    time: number;
+    paused: boolean;
+    muted: boolean;
+    ready: boolean;
+    rate: number;
+    seeking: boolean;
+  }[] {
     return [...this.players].map(([key, { element, texture }]) => ({
       key,
       time: element.currentTime,
       paused: element.paused,
       muted: element.muted,
       ready: texture !== null,
+      rate: element.playbackRate,
+      seeking: element.seeking,
     }));
   }
 
-  private start(key: string): Player | null {
+  /** A driven player (#404) opens at the picture its wearer asks for, `time` seconds into the clip. */
+  private start(key: string, time = 0): Player | null {
     const play = parseVideoKey(key);
     if (!play) return null;
+    const drive: Drive | null = isDrivenVideoKey(key)
+      ? { time, heard: false, reading: null, wanted: null }
+      : null;
+    const at = drive ? drivenTimeOf(play, time, NaN) : play.start;
     const player: Player = {
       play,
-      element: openVideo(`${VIDEO_BASE}/${play.file}`, play.start, (texture) => {
+      element: openVideo(`${VIDEO_BASE}/${play.file}`, at, (texture) => {
         if (player.gone) return texture.destroy(true);
         player.texture = texture;
+        if (drive) this.follow(player, drive);
         // Started even in a paused run, and held once there is a picture.
-        if (this.paused) player.element.pause();
+        else if (this.paused) player.element.pause();
         this.notify();
       }),
       texture: null,
       gain: null,
       gone: false,
       asked: performance.now(),
+      drive,
     };
     this.players.set(key, player);
     this.sweeper ??= setInterval(() => this.sweep(), IDLE_MS);
-    this.watch(player);
-    if (this.heard) this.hear(player);
+    if (drive) this.chase(player, drive);
+    else {
+      this.watch(player);
+      if (this.heard) this.hear(player);
+    }
     return player;
   }
 
+  /** Once a seek is done, the newest one asked for meanwhile follows. */
+  private chase(player: Player, drive: Drive): void {
+    const { element } = player;
+    element.addEventListener('seeked', () => {
+      const wanted = drive.wanted;
+      drive.wanted = null;
+      if (!player.gone && wanted !== null && Math.abs(wanted - element.currentTime) > SEEK_SLACK)
+        element.currentTime = wanted;
+    });
+  }
+
   /**
    * Sends a clip back to its start at its end, or holds it there when it does not repeat. A clip
    * that runs to the end of the video is sent back when the video ends; the whole video loops by

src/viewport/WorldViewport.ts

@@ -27,6 +27,7 @@ import { CameraShade, shadedPicture } from './cameraShade';
 import { snapPlacement } from './pixelGrid';
 import { cameraSees } from '../core/model/renderLayers';
 import { isVideoKey } from '../core/model/video';
+import { videoLibrary } from './VideoLibrary';
 
 /** How strongly the cut-off part of an object shows while the scene view opens its clip area (#260). */
 const CLIP_GHOST_ALPHA = 0.3;
@@ -97,6 +98,8 @@ export abstract class WorldViewport<S extends ViewSource = ViewSource> extends W
   /** Full redraws and camera-only moves since start (test hook). */
   private redraws = 0;
   private readonly animateKey = `animate:${(views += 1)}`;
+  /** What a driven video (#404) knows this view by: each view drives its own players. */
+  private readonly wearer = `view${views}`;
   private cameraMoves = 0;
   private readonly stopFiles: (() => void)[];
   /** One view per overlay camera (#302), by camera id, laid over this one in camera order. */
@@ -762,7 +765,8 @@ export abstract class WorldViewport<S extends ViewSource = ViewSource> extends W
     // A video (#402) moves by itself like a Time shader does.
     if (wornShader(item.props, shaders)?.time || (item.texture && isVideoKey(item.texture.file)))
       this.animated = true;
-    syncSpriteMesh(sprite, item, size, textureLibrary, shaders);
+    const drawn = this.wearVideo(item);
+    syncSpriteMesh(sprite, drawn, size, textureLibrary, shaders);
     let ghost = this.ghostSprites.get(item.id);
     if (!this.ghosted(item, onScreen)) {
       if (ghost) disposeSpriteMesh(ghost);
@@ -774,7 +778,24 @@ export abstract class WorldViewport<S extends ViewSource = ViewSource> extends W
       ghost.alpha = CLIP_GHOST_ALPHA;
       this.ghostSprites.set(item.id, ghost);
     }
-    syncSpriteMesh(ghost, item, size, textureLibrary, shaders);
+    syncSpriteMesh(ghost, drawn, size, textureLibrary, shaders);
+  }
+
+  /** The item with the player of its own a driven video (#404) has, per view and object. */
+  private wearVideo(item: DrawItem): DrawItem {
+    if (!item.texture) return item;
+    const texture = videoLibrary.wear(
+      item.texture,
+      item.props,
+      `${this.wearer}:${item.id}`,
+      this.hearsVideo(),
+    );
+    return texture === item.texture ? item : { ...item, texture };
+  }
+
+  /** Whether a driven video (#404) drawn here may be heard: in a run, and in the animation preview. */
+  protected hearsVideo(): boolean {
+    return false;
   }
 
   /**

src/viewport/videoFollow.ts

@@ -0,0 +1,81 @@
+/**
+ * How a driven video keeps up with the time its wearer asks for (#404). Seeking every frame costs
+ * too much: Chrome decodes from the last keyframe, up to 14 pictures at our distance of 15. So
+ * while the asked time runs forward at a speed a video can play, the video plays at that speed and
+ * is nudged; only when it stands, jumps or runs back does it hold and seek.
+ */
+
+/** The speeds a driven video plays at; outside them it holds and seeks to every asked time. */
+export const MIN_PLAY_SPEED = 0.25;
+export const MAX_PLAY_SPEED = 4;
+/** How far a playing video may be off before it seeks instead of being nudged, in seconds. */
+export const SEEK_DRIFT = 0.25;
+/** How much the speed is nudged per second off, and at most, as a share of it. */
+const NUDGE_GAIN = 0.5;
+const MAX_NUDGE = 0.1;
+/** Closer than half a frame at 60 fps a held video shows the asked picture already. */
+const HELD_SLACK = 1 / 120;
+/** Around 1x a driven video is heard; slowed or sped up it is quiet. */
+const HEARD_SPEED = 0.1;
+/** The stretch of wall time the speed of the asked time is measured over, in milliseconds. */
+export const SPEED_WINDOW_MS = 100;
+
+/**
+ * The speed of the asked time, measured over `SPEED_WINDOW_MS` rather than from frame to frame: a
+ * game steps its animations at its own rate, so between two drawn frames the time may not move at
+ * all and then move twice.
+ */
+export interface SpeedReading {
+  /** Where the asked time stood when the window began, and when that was. */
+  from: number;
+  since: number;
+  last: number;
+  speed: number;
+}
+
+/**
+ * The next reading after the wearer asked for `time` at `now`. A jump (a loop going round, a key
+ * set to another time) starts the window over and keeps the speed: playing on through a loop is
+ * right, and a still picture stays still.
+ */
+export function readSpeed(reading: SpeedReading | null, time: number, now: number): SpeedReading {
+  if (!reading) return { from: time, since: now, last: time, speed: 0 };
+  if (Math.abs(time - reading.last) > SEEK_DRIFT)
+    return { from: time, since: now, last: time, speed: reading.speed };
+  const span = now - reading.since;
+  if (span < SPEED_WINDOW_MS) return { ...reading, last: time };
+  return { from: time, since: now, last: time, speed: ((time - reading.from) * 1000) / span };
+}
+
+export interface FollowState {
+  /** Where in its file the video should stand now. */
+  target: number;
+  /** How fast the asked time runs, in seconds per second. */
+  speed: number;
+  /** Where the video stands. */
+  current: number;
+  /** Where the stretch it plays ends in its file; Infinity for the end of the video. */
+  end: number;
+}
+
+export type FollowStep =
+  /** Play at `rate`, after seeking to `seek` when it is set. */
+  | { kind: 'play'; rate: number; seek: number | null; heard: boolean }
+  /** Hold still, after seeking to `seek` when it is set. */
+  | { kind: 'hold'; seek: number | null };
+
+export function followVideo({ target, speed, current, end }: FollowState): FollowStep {
+  const plays = speed >= MIN_PLAY_SPEED && speed <= MAX_PLAY_SPEED;
+  // Past the end of its stretch a playing video would show the next one until the asked time
+  // turns round or stops; it waits at the asked picture instead.
+  if (!plays || current >= end)
+    return { kind: 'hold', seek: Math.abs(target - current) > HELD_SLACK ? target : null };
+  const drift = target - current;
+  if (Math.abs(drift) > SEEK_DRIFT) return { kind: 'play', rate: speed, seek: target, heard: heardAt(speed) };
+  const nudge = Math.max(-MAX_NUDGE, Math.min(MAX_NUDGE, drift * NUDGE_GAIN));
+  return { kind: 'play', rate: speed * (1 + nudge), seek: null, heard: heardAt(speed) };
+}
+
+function heardAt(speed: number): boolean {
+  return Math.abs(speed - 1) <= HEARD_SPEED;
+}

tests/core/commands.test.ts

@@ -35,6 +35,8 @@ describe('asset commands', () => {
       height: 100,
       color: 0xffffff,
       texture: '',
+      videoClock: 'free',
+      videoTime: 0,
       shader: '',
     });
   });

tests/core/resolve.test.ts

@@ -36,6 +36,8 @@ describe('PropertySchema', () => {
       height: 100,
       color: 0xffffff,
       texture: '',
+      videoClock: 'free',
+      videoTime: 0,
       shader: '',
     });
   });
@@ -69,6 +71,8 @@ describe('resolve', () => {
       height: 100,
       color: 0xffffff,
       texture: '',
+      videoClock: 'free',
+      videoTime: 0,
       shader: '',
     });
   });
@@ -90,6 +94,8 @@ describe('resolve', () => {
       height: 100,
       color: 0x00ff00,
       texture: '',
+      videoClock: 'free',
+      videoTime: 0,
       shader: '',
     });
 

tests/core/serialize.test.ts

@@ -183,6 +183,8 @@ describe('serialize', () => {
       height: 100,
       color: 0xffffff,
       texture: '',
+      videoClock: 'free',
+      videoTime: 0,
       shader: '',
     });
   });

tests/core/shaders.test.ts

@@ -727,6 +727,8 @@ describe('a sprite wearing a shader', () => {
       'height',
       'color',
       'texture',
+      'videoClock',
+      'videoTime',
       'shader',
     ]);
     expect(schemaFor('sprite').find((d) => d.key === SHADER_KEY)).toMatchObject({
@@ -748,6 +750,8 @@ describe('a sprite wearing a shader', () => {
       'height',
       'color',
       'texture',
+      'videoClock',
+      'videoTime',
       'shader',
       'brightness',
       'contrast',
@@ -756,14 +760,14 @@ describe('a sprite wearing a shader', () => {
     expect(assetDeviates(f.project, f.sprite())).toBe(true);
     // A dangling id adds no rows and draws nothing – the picker still shows it.
     f.sprite().props[SHADER_KEY] = 'gone';
-    expect(keys()).toHaveLength(13);
+    expect(keys()).toHaveLength(15);
   });
 
   it('gives a placed object the rows of the shader it overrides, filled with the node defaults', () => {
     const f = wearing();
     const id = place(f, f.spriteId, { x: 0, y: 0 });
     const object = () => f.object(id);
-    expect(schemaForObject(f.project, f.sprite(), object())).toHaveLength(13);
+    expect(schemaForObject(f.project, f.sprite(), object())).toHaveLength(15);
     f.stack.execute(new SetOverrideCommand(f.sceneId, id, SHADER_KEY, f.s.id));
     expect(schemaForObject(f.project, f.sprite(), object()).map((d) => d.key)).toContain('saturation');
     expect(resolveSceneObjectProps(f.project, f.sprite(), object())).toMatchObject({

tests/core/texture-commands.test.ts

@@ -74,6 +74,8 @@ describe('CreateAssetCommand with starting props', () => {
       height: 32,
       color: 0xffffff,
       texture: texture.asset.id,
+      videoClock: 'free',
+      videoTime: 0,
       shader: '',
     });
     stack.undo();

tests/core/video-clock.test.ts

@@ -0,0 +1,94 @@
+/**
+ * A video that follows the animation (#404): a sprite whose Video clock is driven shows the picture
+ * at its Video time, a number an animation keys like any other.
+ */
+import { describe, expect, it } from 'vitest';
+import { CreateVideoCommand } from '../../src/core/commands/assetCommands';
+import { AddClipCommand } from '../../src/core/commands/clipCommands';
+import { animatableDefs } from '../../src/core/model/animation';
+import { drivenTimeOf, drivenVideoKey, isDrivenVideoKey, parseVideoKey } from '../../src/core/model/video';
+import { isRowShown, schemaFor } from '../../src/core/properties/PropertySchema';
+import {
+  VIDEO_CLOCK_KEY,
+  VIDEO_END_KEY,
+  VIDEO_START_KEY,
+  VIDEO_TIME_KEY,
+} from '../../src/core/properties/videos';
+import { makeProject } from '../helpers';
+
+const FILE = `a1b2c3d4e5f6/${'c7'.repeat(32)}.mp4`;
+const KEY = `video:${FILE}#2~4~1~1`;
+
+describe('where a driven video stands', () => {
+  const clip = { start: 2, end: 4, loop: true };
+
+  it('counts from the start of its clip', () => {
+    expect(drivenTimeOf(clip, 0.5, 10)).toBe(2.5);
+  });
+
+  it('goes round past the end with Loop', () => {
+    expect(drivenTimeOf(clip, 5, 10)).toBe(3);
+  });
+
+  it('holds the last picture without Loop, and the first one before the start', () => {
+    expect(drivenTimeOf({ ...clip, loop: false }, 9, 10)).toBe(3.999);
+    expect(drivenTimeOf(clip, -1, 10)).toBe(2);
+  });
+
+  it('measures a clip running to the end of the video by the file', () => {
+    const tail = { start: 6, end: Infinity, loop: true };
+    expect(drivenTimeOf(tail, 5, 10)).toBe(7);
+    expect(drivenTimeOf({ ...tail, loop: false }, 5, 10)).toBe(9.999);
+    expect(drivenTimeOf(tail, 5, NaN)).toBe(11);
+  });
+});
+
+describe('the player a wearer drives', () => {
+  it('goes by the clip key and the name of its wearer', () => {
+    const key = drivenVideoKey(KEY, 'view3:obj1');
+    expect(key).toBe(`${KEY}@view3:obj1`);
+    expect(isDrivenVideoKey(key)).toBe(true);
+    expect(isDrivenVideoKey(KEY)).toBe(false);
+    expect(parseVideoKey(key)).toEqual(parseVideoKey(KEY));
+  });
+});
+
+describe('the Video clock and Video time rows', () => {
+  function setup() {
+    const f = makeProject();
+    const create = new CreateVideoCommand('Intro', FILE, {
+      width: 320,
+      height: 180,
+      duration: 2,
+      fps: 30,
+      audio: false,
+    });
+    f.stack.execute(create);
+    const clip = new AddClipCommand(create.asset.id, { [VIDEO_START_KEY]: 0.5, [VIDEO_END_KEY]: 1 });
+    f.stack.execute(clip);
+    return { project: f.project, videoId: create.asset.id, clipId: clip.clipId };
+  }
+  const row = (key: string) => schemaFor('sprite').find((d) => d.key === key)!;
+
+  it('are shown only while the texture is a video or a clip of one', () => {
+    const { project, videoId, clipId } = setup();
+    const driven = { [VIDEO_CLOCK_KEY]: 'driven' };
+    for (const texture of [videoId, clipId]) {
+      expect(isRowShown(row(VIDEO_CLOCK_KEY), { texture }, project)).toBe(true);
+      expect(isRowShown(row(VIDEO_TIME_KEY), { texture, ...driven }, project)).toBe(true);
+    }
+    expect(isRowShown(row(VIDEO_CLOCK_KEY), { texture: '' }, project)).toBe(false);
+    expect(isRowShown(row(VIDEO_TIME_KEY), { texture: '', ...driven }, project)).toBe(false);
+  });
+
+  it('show the time only while the clock is driven', () => {
+    const { project, videoId } = setup();
+    expect(isRowShown(row(VIDEO_TIME_KEY), { texture: videoId }, project)).toBe(false);
+  });
+
+  it('can be keyed by an animation', () => {
+    const keys = animatableDefs('sprite').map((d) => d.key);
+    expect(keys).toContain(VIDEO_TIME_KEY);
+    expect(keys).toContain(VIDEO_CLOCK_KEY);
+  });
+});

tests/viewport/video-follow.test.ts

@@ -0,0 +1,74 @@
+import { describe, expect, it } from 'vitest';
+import { followVideo, readSpeed, SEEK_DRIFT, SPEED_WINDOW_MS } from '../../src/viewport/videoFollow';
+
+/**
+ * How a driven video keeps up with the time its wearer asks for (#404): it plays along while that
+ * time runs forward at a playable speed, and holds and seeks when it stands, jumps or runs back.
+ */
+describe('the speed of the asked time', () => {
+  it('is measured over a window, not from frame to frame', () => {
+    let reading = readSpeed(null, 0, 0);
+    // A game steps its animation at its own rate: here a frame without a step, then two in one.
+    for (const [time, now] of [
+      [0, 16],
+      [0.033, 33],
+      [0.05, 50],
+      [0.05, 66],
+      [0.083, 83],
+    ] as const)
+      reading = readSpeed(reading, time, now);
+    expect(reading.speed).toBe(0);
+    reading = readSpeed(reading, 0.1, SPEED_WINDOW_MS);
+    expect(reading.speed).toBeCloseTo(1, 5);
+  });
+
+  it('keeps the speed through a jump and measures again from there', () => {
+    const playing = { from: 3.9, since: 900, last: 3.98, speed: 1 };
+    const turned = readSpeed(playing, 0.01, 990);
+    expect(turned).toEqual({ from: 0.01, since: 990, last: 0.01, speed: 1 });
+  });
+
+  it('reads a still time as standing', () => {
+    const playing = { from: 2, since: 0, last: 2, speed: 1 };
+    expect(readSpeed(playing, 2, SPEED_WINDOW_MS).speed).toBe(0);
+  });
+});
+
+describe('following the asked time', () => {
+  const at = { target: 1, speed: 1, current: 1, end: Infinity };
+
+  it('plays along at the speed of the asked time', () => {
+    expect(followVideo(at)).toEqual({ kind: 'play', rate: 1, seek: null, heard: true });
+    expect(followVideo({ ...at, speed: 0.5 })).toEqual({ kind: 'play', rate: 0.5, seek: null, heard: false });
+    expect(followVideo({ ...at, speed: 4 })).toMatchObject({ kind: 'play', rate: 4 });
+  });
+
+  it('nudges a small lag away by a few percent, never more than a tenth', () => {
+    const behind = followVideo({ ...at, current: 0.9 });
+    expect(behind).toMatchObject({ kind: 'play', seek: null });
+    expect(behind.kind === 'play' && behind.rate).toBeCloseTo(1.05, 5);
+    const ahead = followVideo({ ...at, current: 1 + SEEK_DRIFT });
+    expect(ahead.kind === 'play' && ahead.rate).toBeCloseTo(0.9, 5);
+  });
+
+  it('seeks once it is further off than the drift allows', () => {
+    expect(followVideo({ ...at, current: 0.7 })).toEqual({ kind: 'play', rate: 1, seek: 1, heard: true });
+  });
+
+  it('holds and seeks for a still picture, a time running back or too fast', () => {
+    expect(followVideo({ ...at, speed: 0, current: 0.2 })).toEqual({ kind: 'hold', seek: 1 });
+    expect(followVideo({ ...at, speed: -1, current: 1.2 })).toEqual({ kind: 'hold', seek: 1 });
+    expect(followVideo({ ...at, speed: 8, current: 0.2 })).toEqual({ kind: 'hold', seek: 1 });
+  });
+
+  it('does not seek a held video that shows the asked picture already', () => {
+    expect(followVideo({ ...at, speed: 0, current: 1.004 })).toEqual({ kind: 'hold', seek: null });
+  });
+
+  it('waits at the asked picture once it played past the end of its stretch', () => {
+    expect(followVideo({ target: 2.999, speed: 1, current: 3.02, end: 3 })).toEqual({
+      kind: 'hold',
+      seek: 2.999,
+    });
+  });
+});

tests/viewport/video-library.test.ts

@@ -84,3 +84,30 @@ describe('VideoLibrary sweeping', () => {
     expect(videos.debugInfo()).toEqual([]);
   });
 });
+
+describe('VideoLibrary driven players (#404)', () => {
+  beforeEach(() => {
+    vi.stubGlobal('document', { hidden: false, addEventListener: () => {}, createElement: fakeVideo });
+  });
+
+  afterEach(() => vi.unstubAllGlobals());
+
+  const REF = { file: KEY, smooth: true, rect: null, sourceWidth: 320, sourceHeight: 180, slice: null };
+
+  it('leaves a sprite with a free clock on the shared player', async () => {
+    const videos = await freshLibrary();
+    expect(videos.wear(REF, { videoTime: 1 }, 'view1:a', false)).toBe(REF);
+    expect(videos.debugInfo()).toEqual([]);
+  });
+
+  it('gives every wearer of a driven video a player of its own, opened at its time', async () => {
+    const videos = await freshLibrary();
+    const a = videos.wear(REF, { videoClock: 'driven', videoTime: 0.5 }, 'view1:a', false);
+    const b = videos.wear(REF, { videoClock: 'driven', videoTime: 1.25 }, 'view1:b', false);
+    expect([a.file, b.file]).toEqual([`${KEY}@view1:a`, `${KEY}@view1:b`]);
+    expect(videos.debugInfo().map(({ key, time }) => [key, time])).toEqual([
+      [`${KEY}@view1:a`, 0.5],
+      [`${KEY}@view1:b`, 1.25],
+    ]);
+  });
+});

tools/e2e/66-video.mjs

@@ -16,6 +16,10 @@ import { LANES, MAKE_OBJECT, pickAssetType } from './lib/scaffold.mjs';
  */
 const SPRITE = 'vd66-sprite';
 const OBJECT = 'vd66-object';
+const DRIVEN = 'vd66-driven';
+const EARLY = 'vd66-early';
+const LATE = 'vd66-late';
+const CUT = 'vd66-cut';
 const LANE = LANES.video;
 const COLORS = { red: '#fe0000', green: '#00ff01', blue: '#0000ff', white: '#ffffff' };
 
@@ -265,10 +269,97 @@ export async function video(t) {
     JSON.stringify({ thrown, inspected, kept, inPreview }),
   );
 
+  // 66i. Driven by its Video time (#404): two sprites wearing the same video stand still in the
+  //      scene, each at its own time (0.2 s red, 0.7 s green), each on a player of its own.
   await mutate(`
     const p = __gm.state.project;
-    for (const holder of p.assets) if (holder.objects) holder.objects = holder.objects.filter((o) => o.id !== '${OBJECT}');
-    p.assets = p.assets.filter((a) => a.id !== '${SPRITE}' && a.id !== ${JSON.stringify(assetId)});
+    const scene = __gm.state.activeScene;
+    ${MAKE_OBJECT}
+    p.assets.unshift({ id: '${DRIVEN}', type: 'sprite', name: '${DRIVEN}', props: { width: 320, height: 180, color: 0xffffff, texture: ${JSON.stringify(assetId)}, shader: '', videoClock: 'driven', videoTime: 0 } });
+    const early = make('${EARLY}', '${DRIVEN}', '', ${LANE}, 'at 0.2');
+    const late = make('${LATE}', '${DRIVEN}', '', ${LANE}, 'at 0.7');
+    early.overrides = { videoTime: 0.2 };
+    late.position.x = 400;
+    late.overrides = { videoTime: 0.7 };
+    scene.objects.push(early, late);
+    __gm.state.select(null);
+    __gm.lookAt(200, ${LANE});`);
+  const colorAt = (r) => r && json(`__gm.pixelAt(${Math.round(r.x)}, ${Math.round(r.y)})`);
+  const pair = async () => [
+    await colorAt(await json(`__gm.objectRect('${EARLY}')`)),
+    await colorAt(await json(`__gm.objectRect('${LATE}')`)),
+  ];
+  const stills = await waitFor(pair, ([a, b]) => a === COLORS.red && b === COLORS.green);
+  await sleep(700);
+  const later = await pair();
+  const drivenPlayers = (await json('__gm.videos()')).filter((v) => v.key.includes('@'));
+  check(
+    'driven sprites stand still at their Video time, two of the same video on players of their own',
+    stills[0] === COLORS.red &&
+      stills[1] === COLORS.green &&
+      later[0] === COLORS.red &&
+      later[1] === COLORS.green &&
+      drivenPlayers.length === 2 &&
+      drivenPlayers.every((v) => v.paused),
+    JSON.stringify({ stills, later, drivenPlayers }),
+  );
+  await shot('66-video-driven');
+
+  // 66j. Keyed in an animation, the preview shows the picture of the time at the playhead, back and
+  //      forth: frame 2 = 0.2 s red, 7 = green, 12 = blue, and back to 3 = red.
+  await mutate(`
+    __gm.state.project.assets.push({ id: '${CUT}', type: 'animation', name: '${CUT}', props: { fps: 10, playback: 'loop' },
+      root: '${DRIVEN}',
+      tracks: [{ id: 'vd66-row', target: '', property: 'videoTime', keys: [{ frame: 0, value: 0 }, { frame: 20, value: 2 }] }] });`);
+  await t.evaluate(`__gm.openDocument('asset:${CUT}')`);
+  await sleep(300);
+  const rootId = await json("__gm.animation.preview.objectId('')");
+  // Each picture must also stay: a video playing by itself shows every colour within 2 s.
+  const previewColor = async () =>
+    colorAt(await json(`__gm.animation.preview.rect(${JSON.stringify(rootId)})`));
+  const scrubbed = [];
+  const wanted = [COLORS.red, COLORS.green, COLORS.blue, COLORS.red];
+  for (const [i, frame] of [2, 7, 12, 3].entries()) {
+    await t.evaluate(`__gm.animation.scrub(${frame})`);
+    const first = await waitFor(previewColor, (color) => color === wanted[i], 10);
+    await sleep(700);
+    scrubbed.push([first, await previewColor()]);
+  }
+  check(
+    'scrubbing the animation shows the picture of the keyed Video time and holds it, back and forth',
+    scrubbed.every(([first, held], i) => first === wanted[i] && held === wanted[i]),
+    JSON.stringify(scrubbed),
+  );
+  await shot('66-video-scrubbed');
+
+  // 66k. Played in the tab, the video plays along at 1x, heard, at the time the playhead asks for
+  //      (the clip and the animation both loop at 2 s, so the gap is measured round the loop).
+  const playCut = await json('__gm.animation.playAt()');
+  await click(playCut.x, playCut.y);
+  await sleep(600);
+  const along = [];
+  for (let i = 0; i < 5; i++) {
+    const asked = (await json(`__gm.animation.preview.item(${JSON.stringify(rootId)})`))?.props.videoTime;
+    const player = (await json('__gm.videos()')).find(
+      (v) => v.key.includes(`@`) && !v.key.includes(EARLY) && !v.key.includes(LATE),
+    );
+    along.push({ asked, ...player });
+    await sleep(200);
+  }
+  await click(playCut.x, playCut.y);
+  const gap = (a, b) => Math.min(Math.abs(a - b), 2 - Math.abs(a - b));
+  check(
+    'playing the animation plays the video along at 1x and lets it be heard',
+    along.every((p) => !p.paused && !p.muted && Math.abs(p.rate - 1) <= 0.1 && gap(p.time, p.asked) < 0.15),
+    JSON.stringify(along),
+  );
+  await showTab('scene');
+
+  await mutate(`
+    const p = __gm.state.project;
+    const gone = ['${OBJECT}', '${EARLY}', '${LATE}'];
+    for (const holder of p.assets) if (holder.objects) holder.objects = holder.objects.filter((o) => !gone.includes(o.id));
+    p.assets = p.assets.filter((a) => ![ '${SPRITE}', '${DRIVEN}', '${CUT}', ${JSON.stringify(assetId)} ].includes(a.id));
     __gm.state.select(null);
     __gm.lookAt(0, 0);`);
 }