Record the game view from a behavior into a video asset (#405)
game-maker / gate (push) Successful in 4m14s
game-maker / suite (push) Successful in 17m7s

A game can now film itself. A behavior starts and stops a take, and
the editor turns it into a video asset with the run's picture and the
sound the player hears. The exported game hands the file over as a
download, since there is no project to keep it in.

- New action nodes `Start recording` (Name, Resolution, FPS) and
  `Stop recording`. An empty name becomes `Recording <timestamp>`.
- Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes
  the height is set and the width follows the camera.
- The picture is the camera only, without hints, overlays or the
  letterbox. It is drawn into a render texture of the target size,
  so a smaller window still gives full 1080p.
- `PictureReader` reads the texture back through pixel buffers with
  a fence. The editor never waits on the GPU, and a frame with all
  three buffers busy is dropped and counted.
- Nothing of the take reaches the visible canvas. A first version
  drew tiles into it and copied them out, and with the low-latency
  canvas that midway state flickered on screen.
- The sound is tapped behind the master limiter in the FX worklet
  and encoded as Opus.
- H.264 is chosen when the browser has it, VP9 otherwise. A small
  own Matroska writer packs both tracks, no new dependency. It names
  the frame rate and BT.709 so ffmpeg keeps both.
- A second `Start recording` is ignored with a warning in the log.
  A take goes on through `Load scene` and a restart, pauses with the
  editor and is saved when the run ends. It stops at 2 GB.
- The editor uploads the take on the existing video route and makes
  the asset once the run is over. The video tab gets a Download
  button for the converted MP4.
- `downloadBlob` in `src/ui` replaces the song editor's own copy.

Findings:
- `On start` runs before the game screen knows the world. A take
  therefore fixes its size on the first frame that has a camera.
- Without a frame rate in the file, ffmpeg turned 30 fps into 59.94.
- The worker's poster JPEG is off in colour for every upload (red
  255 comes out as 229, BT.601). Not touched here.

Tests:
- Unit recording + matroska: 21 green, plus the node catalog cases.
  The Matroska test reads the file back with its own EBML reader.
- e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a
  1280x800 window gives a 1920x1080 MP4 with sound and no dropped
  frame. Every 4th pixel is checked, a green quarter catches an
  upside-down picture. A take ended by Esc is saved with the
  timestamp name.
- e2e 60j: the exported game downloads an `.mkv` with EBML magic.
- The flicker is gone on the user's machine without `?vsync`.
- Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-11 21:11:05 +02:00
1 parent 36ffd9a61a
commit e191dc4b2e
35 files changed
+2211 -26

No files matched your search

+14
View File
@@ -76,6 +76,8 @@ import { HierarchyPanel } from './panels/HierarchyPanel';
import { TilemapPanel } from './panels/TilemapPanel';
import { Toolbar } from './panels/Toolbar';
import { type PlayHost, PlaySession, type WorldSnapshot } from './PlaySession';
import { type RecordingUpload, RecordingUploads } from './recordingUploads';
import type { MadeRecording } from '../viewport/GameRecorder';
import { ProjectSession } from './ProjectSession';
import { ParticlesEditor } from './particles/ParticlesEditor';
import { ShaderEditor } from './shader/ShaderEditor';
@@ -313,6 +315,7 @@ export class Editor implements PlayHost {
/** Frame rate, refreshed twice a second; null until the first measurement. */
fps: number | null = null;
private readonly play: PlaySession;
private readonly recordings: RecordingUploads;
private readonly editorRun: EditorRun;
private readonly projects: ProjectSession;
private readonly leftColumn: VStack;
@@ -400,6 +403,7 @@ export class Editor implements PlayHost {
this.hierarchy = new HierarchyPanel(state);
this.inspector = new Inspector(state);
this.tilemapPanel = new TilemapPanel(state);
this.recordings = new RecordingUploads({ state, videos });
this.play = new PlaySession(state, this);
this.editorRun = new EditorRun(state, {
// The game view is the one place where no key means anything to the editor, so that is where
@@ -1341,6 +1345,7 @@ export class Editor implements PlayHost {
this.editorRun.tick(step);
this.viewport.animate(sceneDraws);
this.gameView.animate(gameDraws);
this.play.recordFrame();
this.console.refresh();
if (this.shows('animation')) this.animationEditor.tick(step);
if (this.shows('tile')) this.tileEditor.tick(step);
@@ -1480,4 +1485,13 @@ export class Editor implements PlayHost {
performance: this.play.performance(),
};
}
/** The recordings of the game view (#405) and what became of them on the way into the project. */
recordingInfo(): ReturnType<PlaySession['recordingInfo']> & { uploads: RecordingUpload[] } {
return { ...this.play.recordingInfo(), uploads: this.recordings.info() };
}
recordingSaved(recording: MadeRecording): void {
this.recordings.add(recording);
}
}