Files
feuerwerk/tests/helpers.ts
T
adminandClaude Opus 5.5 e191dc4b2e
game-maker / gate (push) Successful in 4m14s
game-maker / suite (push) Successful in 17m7s
Record the game view from a behavior into a video asset (#405)
A game can now film itself. A behavior starts and stops a take, and
the editor turns it into a video asset with the run's picture and the
sound the player hears. The exported game hands the file over as a
download, since there is no project to keep it in.

- New action nodes `Start recording` (Name, Resolution, FPS) and
  `Stop recording`. An empty name becomes `Recording <timestamp>`.
- Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes
  the height is set and the width follows the camera.
- The picture is the camera only, without hints, overlays or the
  letterbox. It is drawn into a render texture of the target size,
  so a smaller window still gives full 1080p.
- `PictureReader` reads the texture back through pixel buffers with
  a fence. The editor never waits on the GPU, and a frame with all
  three buffers busy is dropped and counted.
- Nothing of the take reaches the visible canvas. A first version
  drew tiles into it and copied them out, and with the low-latency
  canvas that midway state flickered on screen.
- The sound is tapped behind the master limiter in the FX worklet
  and encoded as Opus.
- H.264 is chosen when the browser has it, VP9 otherwise. A small
  own Matroska writer packs both tracks, no new dependency. It names
  the frame rate and BT.709 so ffmpeg keeps both.
- A second `Start recording` is ignored with a warning in the log.
  A take goes on through `Load scene` and a restart, pauses with the
  editor and is saved when the run ends. It stops at 2 GB.
- The editor uploads the take on the existing video route and makes
  the asset once the run is over. The video tab gets a Download
  button for the converted MP4.
- `downloadBlob` in `src/ui` replaces the song editor's own copy.

Findings:
- `On start` runs before the game screen knows the world. A take
  therefore fixes its size on the first frame that has a camera.
- Without a frame rate in the file, ffmpeg turned 30 fps into 59.94.
- The worker's poster JPEG is off in colour for every upload (red
  255 comes out as 229, BT.601). Not touched here.

Tests:
- Unit recording + matroska: 21 green, plus the node catalog cases.
  The Matroska test reads the file back with its own EBML reader.
- e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a
  1280x800 window gives a 1920x1080 MP4 with sound and no dropped
  frame. Every 4th pixel is checked, a green quarter catches an
  upside-down picture. A take ended by Esc is saved with the
  timestamp name.
- e2e 60j: the exported game downloads an `.mkv` with EBML magic.
- The flicker is gone on the user's machine without `?vsync`.
- Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:11:05 +02:00

547 lines
20 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import { CreateAssetCommand } from '../src/core/commands/assetCommands';
import { CommandStack } from '../src/core/commands/CommandStack';
import { AddNodeCommand, ConnectCommand, DisconnectCommand } from '../src/core/commands/graphCommands';
import { AddPortsCommand } from '../src/core/commands/portCommands';
import { PlaceAssetCommand } from '../src/core/commands/sceneObjectCommands';
import { requireNodeDef } from '../src/core/commands/require';
import { AddVariableCommand } from '../src/core/commands/variableCommands';
import {
type DeclaredPort,
emptySharedState,
ENTITY_HANDLE,
type EntityHandle,
type GraphLookup,
type GraphNode,
NO_LOOKUP,
type NodeContext,
type PointerState,
type Value,
type RunResult,
portKindOf,
PROPERTIES,
VARIABLES,
VARIABLES_PARAM,
} from '../src/core/graph';
import type { GraphInterpreter } from '../src/core/graph/GraphInterpreter';
import type { EaseShare } from '../src/core/model/animation';
import type { BehaviorAsset, SceneAsset, VariablesAsset } from '../src/core/model/Asset';
import { findObject } from '../src/core/model/hierarchy';
import { worldPosition } from '../src/core/properties/transform';
import { resolvedHolder } from '../src/core/model/instance';
import type { Position, PropertyBag, Vec2 } from '../src/core/model/Properties';
import { createEmptyProject, findScene, type Project, ROUTING_ID } from '../src/core/model/Project';
import { settleRouting } from '../src/core/music/projectRouting';
import { MASTER_BUS } from '../src/core/properties/buses';
import type { RecordingSettings } from '../src/core/recording/recording';
import type { SceneObject } from '../src/core/model/SceneObject';
import { deserializeProject, serializeProject } from '../src/core/store/serialize';
import type { AudioSink } from '../src/runtime/systems/Audio';
import { BehaviorSystem, type TimeScaled } from '../src/runtime/systems/BehaviorSystem';
import type { SaveArea } from '../src/core/graph/savedValues';
import { readyPhysics } from '../src/runtime/physics/engine';
import { PhysicsSystem } from '../src/runtime/physics/PhysicsSystem';
import { CollisionSystem } from '../src/runtime/systems/CollisionSystem';
import type { CollisionSource } from '../src/runtime/systems/solids';
import { AnimationSystem } from '../src/runtime/systems/AnimationSystem';
import { Input } from '../src/runtime/systems/Input';
import { World } from '../src/runtime/World';
export interface ProjectFixture {
project: Project;
stack: CommandStack;
}
export interface SceneFixture extends ProjectFixture {
sceneId: string;
spriteId: string;
/** The scene asset, looked up fresh (undo may replace it). */
sceneOf(): SceneAsset;
/** The scene with the content of placed containers resolved in – what the editor shows. */
viewOf(): SceneAsset;
object(id: string): SceneObject;
/** World position of a scene object, ancestors included. */
world(id: string): Position;
}
export function makeProject(): ProjectFixture {
const project = createEmptyProject('p', 'P');
return { project, stack: new CommandStack(project) };
}
/**
* The master routing of a fixture (#351), held to its assets before each change: `link` adds a link
* from an asset to a bus or the master, `only` makes that its one link.
*/
export function routingOf({ project, stack }: ProjectFixture) {
const link = (from: string, to = MASTER_BUS) => {
settleRouting(project);
const command = new ConnectCommand(ROUTING_ID, { node: from, port: 'out' }, { node: to, port: 'in' });
stack.execute(command);
return command.edge;
};
return {
link,
only(from: string, to = MASTER_BUS) {
settleRouting(project);
for (const edge of project.routing.edges.filter((e) => e.from.node === from))
stack.execute(new DisconnectCommand(ROUTING_ID, edge.id));
return link(from, to);
},
};
}
/** A project with a scene "Level" and a 100×100 sprite (named "Box" unless told otherwise), plus lookups into the scene. */
export function makeSpriteScene(spriteName = 'Box'): SceneFixture {
const fixture = makeProject();
const scene = new CreateAssetCommand('scene', 'Level');
const sprite = new CreateAssetCommand('sprite', spriteName);
fixture.stack.execute(scene);
fixture.stack.execute(sprite);
const sceneOf = () => findScene(fixture.project, scene.asset.id)!;
const viewOf = () => resolvedHolder(fixture.project, sceneOf()) as SceneAsset;
const object = (id: string) => findObject(viewOf(), id)!;
return {
...fixture,
sceneId: scene.asset.id,
spriteId: sprite.asset.id,
sceneOf,
viewOf,
object,
world: (id) => worldPosition(fixture.project, viewOf(), object(id)),
};
}
/** Places an asset into the fixture's scene at a world position (optionally below a parent) and returns the new object's id. */
export function place(
fixture: SceneFixture,
assetId: string,
position: Vec2 | Position,
parentId = '',
): string {
const command = new PlaceAssetCommand(fixture.sceneId, assetId, position, parentId);
fixture.stack.execute(command);
return command.objectId;
}
/** Adds a Variables asset declaring the given number variables – what a run shares. */
export function makeVariables(
fixture: ProjectFixture,
name: string,
keys: readonly string[],
): VariablesAsset {
const command = new CreateAssetCommand('variables', name);
fixture.stack.execute(command);
for (const key of keys) fixture.stack.execute(new AddVariableCommand(command.asset.id, key, 'number'));
return command.asset as VariablesAsset;
}
/** Adds a behavior asset declaring the given number variables. */
export function makeBehavior(
fixture: ProjectFixture,
name = 'B',
variables: readonly string[] = [],
): BehaviorAsset {
const command = new CreateAssetCommand('behavior', name);
fixture.stack.execute(command);
for (const key of variables) fixture.stack.execute(new AddVariableCommand(command.asset.id, key, 'number'));
return command.asset as BehaviorAsset;
}
export type Edge = readonly [from: GraphNode, fromPort: string, to: GraphNode, toPort: string];
export interface GraphEditor {
/** Adds a node with inline params (graph coordinates do not matter to tests). */
addNode(type: string, params?: PropertyBag): GraphNode;
/** A Variables node naming `keys`, typed by their declaration; `write` sets all of them or the listed. */
variables(
keys: readonly string[],
options?: { write?: boolean | readonly string[]; params?: PropertyBag },
): GraphNode;
/** A Properties node with `params` (its type) declaring `ports`. */
properties(ports: readonly DeclaredPort[], params?: PropertyBag): GraphNode;
connect(from: GraphNode, fromPort: string, to: GraphNode, toPort: string): void;
wire(edges: readonly Edge[]): void;
}
/** Command-driven editing of one behavior's graph, the way the editor does it. */
export function graphEditor(fixture: ProjectFixture, behavior: BehaviorAsset): GraphEditor {
const { stack } = fixture;
const connect: GraphEditor['connect'] = (from, fromPort, to, toPort) => {
stack.execute(
new ConnectCommand(behavior.id, { node: from.id, port: fromPort }, { node: to.id, port: toPort }),
);
};
return {
addNode(type, params = {}) {
const command = new AddNodeCommand(behavior.id, type, 0, 0, params);
stack.execute(command);
return command.node;
},
variables(keys, { write = false, params = {} } = {}) {
const command = new AddNodeCommand(behavior.id, VARIABLES, 0, 0, params);
stack.execute(command);
const writes = (key: string) => write === true || (Array.isArray(write) && write.includes(key));
const ports = keys.map((key): DeclaredPort => {
// A node pointed at a Variables asset declares that asset's variables, not the behavior's.
const source = fixture.project.assets.find((a) => a.id === params[VARIABLES_PARAM]);
const declarations = source?.type === 'variables' ? source.variables : behavior.variables;
const declared = declarations.find((v) => v.key === key);
const kind = declared ? portKindOf(declared) : 'number';
return writes(key) ? { key, kind, write: true } : { key, kind };
});
if (ports.length > 0) stack.execute(new AddPortsCommand(behavior.id, command.node.id, ports));
return command.node;
},
properties(ports, params = {}) {
const command = new AddNodeCommand(behavior.id, PROPERTIES, 0, 0, params);
stack.execute(command);
if (ports.length > 0) stack.execute(new AddPortsCommand(behavior.id, command.node.id, ports));
return command.node;
},
connect,
wire(edges) {
for (const [from, fromPort, to, toPort] of edges) connect(from, fromPort, to, toPort);
},
};
}
export const NO_POINTER: PointerState = {
x: 0,
y: 0,
inside: false,
down: false,
pressed: false,
released: false,
wheel: 0,
};
/** A top-level entity handle named after itself, with empty props. */
export function fakeEntity(name: string, x = 0, assetId = `asset-${name}`): EntityHandle {
return { [ENTITY_HANDLE]: true, id: name, assetId, name, x, y: 0, z: 0, props: {}, parent: null };
}
/** An entity handle "h" carrying the given props, optionally below a parent. */
export function fakeHandle(props: PropertyBag, parent: EntityHandle | null = null): EntityHandle {
return { [ENTITY_HANDLE]: true, id: 'h', assetId: 'a', name: 'h', x: 0, y: 0, z: 0, props, parent };
}
/** `changes` records the state moves the context was told about, in the order they were reported. */
export type FakeContext = NodeContext & {
logs: string[];
stopped: boolean;
changes: { machineId: string; from: string; to: string }[];
/** What the context was asked to play and to silence, in order – nothing here makes a noise. */
played: {
assetId: string;
volume: number;
loop: boolean | undefined;
detached: boolean;
fx: string;
fxParams: Readonly<Record<string, number>>;
tempo: number;
/** The bus picked (#325); left out for none. */
bus?: string;
}[];
/** The notes the context was asked to play (#331); a voice's number is how many have been played. */
notes: {
instrumentId: string;
note: number;
velocity: number;
volume: number;
duration: number;
detached: boolean;
instrumentParams: Readonly<Record<string, number>>;
tempo: number;
fx: string;
fxParams: Readonly<Record<string, number>>;
bus?: string;
}[];
/** The patterns the context was asked to play; a playing's number is how many have been played. */
patterns: {
patternId: string;
volume: number;
loop: boolean;
detached: boolean;
fx: string;
fxParams: Readonly<Record<string, number>>;
bus?: string;
}[];
/** The songs the context was asked to play, numbered the same way. */
songs: {
songId: string;
volume: number;
fadeIn: number;
loop: boolean;
detached: boolean;
fx: string;
fxParams: Readonly<Record<string, number>>;
bus?: string;
}[];
silenced: string[];
tuned: { voice: number; fxParams: Readonly<Record<string, number>>; time: number; ease: EaseShare }[];
/** What buses the context was told to change (#325): a volume or FX parameters, in order. */
buses: {
bus: string;
volume?: number;
fxParams?: Readonly<Record<string, number>>;
time: number;
ease: EaseShare;
}[];
/** The scenes the context was asked to load, in order. */
loaded: string[];
/** The time scale and the pause the context holds, as the game nodes left them. */
scale: number;
paused: boolean;
/** What the recording nodes asked for (#405), in order: the settings of a start, `'stop'` for a stop. */
recordings: (RecordingSettings | 'stop')[];
};
/** A NodeContext double: self/owner handles, `speed = 100`, dt 0.5, moves applied directly, `log` and `stop` recorded. */
export function fakeContext(overrides: Partial<NodeContext> = {}): FakeContext {
const logs: string[] = [];
const ctx: FakeContext = {
self: fakeEntity('self'),
owner: fakeEntity('owner'),
variables: { speed: 100 },
states: {},
shared: emptySharedState(),
changes: [],
stateChanged: (machineId, from, to) => ctx.changes.push({ machineId, from, to }),
time: 0,
dt: 0.5,
logs,
stopped: false,
findByName: () => null,
isAlive: () => true,
findReferenced: () => null,
childByName: () => null,
objects: () => [],
isKeyDown: () => false,
wasKeyPressed: () => false,
wasKeyReleased: () => false,
pointer: { ...NO_POINTER },
pointerThrough: () => ({ x: 0, y: 0, inside: false }),
hitTest: () => false,
isTouching: () => false,
raycast: () => null,
raycastAll: () => [],
collisionLayerOf: () => null,
renderLayerOf: () => null,
ignoreOneWay: () => {},
isTilemap: () => false,
tileAt: () => null,
setTile: () => false,
moveAndSlide: (target, dx, dy) => {
target.x += dx;
target.y += dy;
return { blockedX: false, blockedY: false, hit: false, normal: { x: 0, y: 0 } };
},
log: (message) => logs.push(message),
played: [],
silenced: [],
loaded: [],
// The number of a voice is how many have been played: unique, and never 0.
playSound: (assetId, volume, loop, detached, fx, fxParams, tempo, bus) =>
ctx.played.push({ assetId, volume, loop, detached, fx, fxParams, tempo, ...(bus && { bus }) }),
notes: [],
playNote: (
instrumentId,
note,
velocity,
volume,
duration,
detached,
instrumentParams,
tempo,
fx,
fxParams,
bus,
) =>
ctx.notes.push({
instrumentId,
note,
velocity,
volume,
duration,
detached,
instrumentParams,
tempo,
fx,
fxParams,
...(bus && { bus }),
}),
patterns: [],
playPattern: (patternId, volume, loop, detached, fx, fxParams, bus) =>
ctx.patterns.push({ patternId, volume, loop, detached, fx, fxParams, ...(bus && { bus }) }),
stopPattern: (patternId, letRing) =>
ctx.silenced.push(`pattern ${patternId}${letRing ? ' (let ring)' : ''}`),
songs: [],
playSong: (songId, volume, fadeIn, loop, detached, fx, fxParams, bus) =>
ctx.songs.push({ songId, volume, fadeIn, loop, detached, fx, fxParams, ...(bus && { bus }) }),
stopSong: (songId, fadeOut) => ctx.silenced.push(`song ${songId} (fade ${fadeOut})`),
stopPlayback: (playback, fadeOut) => ctx.silenced.push(`playback ${playback} (fade ${fadeOut})`),
// Every sound lasts a quarter of a second, as if it had been loaded.
soundLength: () => 0.25,
// The tail of a voice is its number in tenths of a second, so a test can tell one from another.
soundTail: (voice) => voice / 10,
// A voice by its number, and letting ring, are written into the line, so one list says it all.
stopSound: (assetId, letRing) => ctx.silenced.push(letRing ? `${assetId} (let ring)` : assetId),
stopVoice: (voice, letRing) => ctx.silenced.push(`voice ${voice}${letRing ? ' (let ring)' : ''}`),
tuned: [],
tuneSound: (voice, fxParams, time, ease) => ctx.tuned.push({ voice, fxParams, time, ease }),
buses: [],
tuneBus: (bus, fxParams, time, ease) => ctx.buses.push({ bus, fxParams, time, ease }),
setBusVolume: (bus, volume, time, ease) => ctx.buses.push({ bus, volume, time, ease }),
// The last volume set, else that of an asset nobody touched.
busVolume: (bus) =>
[...ctx.buses].reverse().find((change) => change.bus === bus && change.volume !== undefined)?.volume ??
1,
loadScene: (sceneId) => ctx.loaded.push(sceneId),
spawn: () => null,
addJoint: () => null,
destroy: () => [],
stop: () => {
ctx.stopped = true;
},
restart: () => {},
scale: 1,
paused: false,
timeScale: () => ctx.scale,
setTimeScale: (scale) => {
ctx.scale = scale;
},
isPaused: () => ctx.paused,
setPaused: (paused) => {
ctx.paused = paused;
},
recordings: [],
startRecording: (settings) => ctx.recordings.push(settings),
stopRecording: () => ctx.recordings.push('stop'),
...overrides,
};
return ctx;
}
/** Runs a node on its registered definition: the shape it has while sitting in no graph. */
export function runDef(
type: string,
ctx: NodeContext,
inputs: Record<string, Value>,
params: PropertyBag = {},
entry = 'exec',
lookup: GraphLookup = NO_LOOKUP,
): RunResult {
const def = requireNodeDef(type);
return def.run!(ctx, inputs, params, def, entry, lookup);
}
/**
* Runs every event node of `type` in the graph, in the order the graph holds them, and answers how
* many ran. A game orders its events across all behaviors instead (#264); a test of one graph does not.
*/
export function fire(
interpreter: GraphInterpreter,
type: string,
ctx: NodeContext,
outputs: Record<string, Value> = {},
filter?: (node: GraphNode) => boolean,
): number {
const nodes = interpreter.eventNodes(type).filter((node) => !filter || filter(node));
for (const node of nodes) interpreter.pulse(node, ctx, outputs);
return nodes.length;
}
/** Runs one node definition in isolation and returns its data outputs. */
export function runNode(
type: string,
inputs: Record<string, Value>,
params: PropertyBag = {},
ctx: NodeContext = fakeContext(),
): Record<string, Value> {
return runDef(type, ctx, inputs, params).outputs ?? {};
}
export interface WorldOptions {
collisions?: boolean;
/**
* With `collisions`: the played game's physics engine rather than the editor run's detection
* (#146). The engine has to be loaded first – `await loadPhysics()` in a `beforeAll`.
*/
physics?: boolean;
animations?: boolean;
behaviors?: boolean;
/** Where `Play sound` ends up; without one the run plays to nobody (see `SILENT_AUDIO`). */
audio?: AudioSink;
/** The clock `Set time scale` turns; without one the run is treated like the editor run. */
clock?: TimeScaled;
/** Where persistent Variables assets keep their values; without one nothing outlives the run. */
saves?: SaveArea;
}
export interface StartedWorld {
world: World;
collisions: CollisionSource | null;
behaviors: BehaviorSystem | null;
/** Feed keys with `input.press('a')`; `frame()` clears the edges the way the editor does. */
input: Input;
/** One tick plus the end-of-frame bookkeeping the editor's play loop performs. */
frame(dt?: number): void;
}
/** Instantiates the fixture's scene, optionally with collisions, animations and a BehaviorSystem fed by both. */
export function startWorld(
fixture: SceneFixture,
options: WorldOptions & { collisions: true },
): StartedWorld & { collisions: CollisionSource };
export function startWorld(fixture: SceneFixture, options?: WorldOptions): StartedWorld;
export function startWorld(fixture: SceneFixture, options: WorldOptions = {}): StartedWorld {
const world = World.instantiate(fixture.project, fixture.sceneId);
const collisions = options.collisions
? options.physics
? new PhysicsSystem(readyPhysics() ?? fail('load the physics engine before starting a world with it'))
: new CollisionSystem()
: null;
const input = new Input();
const animations = options.animations ? new AnimationSystem() : null;
if (collisions) world.addSystem(collisions);
if (animations) world.addSystem(animations);
const behaviors = options.behaviors
? new BehaviorSystem(fixture.project, input, {
collisions,
animations,
audio: options.audio,
clock: options.clock,
saves: options.saves,
})
: null;
if (behaviors) world.addSystem(behaviors);
return {
world,
collisions,
behaviors,
input,
frame(dt = 1 / 60) {
world.tick(dt);
input.endFrame();
},
};
}
/**
* Both ways a run detects collisions: the editor run's own and the played game's engine (#146).
* Everything without a rigidbody must come out the same on either.
*/
export const COLLISION_SYSTEMS = [
{ system: 'CollisionSystem', physics: false },
{ system: 'PhysicsSystem', physics: true },
] as const;
function fail(message: string): never {
throw new Error(message);
}
/** The project as it comes back after being saved and loaded again. */
export function roundTrip(project: Project): Project {
return deserializeProject(serializeProject(project));
}