Files
feuerwerk/src/editor/video/VideoEditor.ts
T
adminandClaude Opus 5.5 e191dc4b2e
game-maker / gate (push) Successful in 4m14s
game-maker / suite (push) Successful in 17m7s
Record the game view from a behavior into a video asset (#405)
A game can now film itself. A behavior starts and stops a take, and
the editor turns it into a video asset with the run's picture and the
sound the player hears. The exported game hands the file over as a
download, since there is no project to keep it in.

- New action nodes `Start recording` (Name, Resolution, FPS) and
  `Stop recording`. An empty name becomes `Recording <timestamp>`.
- Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes
  the height is set and the width follows the camera.
- The picture is the camera only, without hints, overlays or the
  letterbox. It is drawn into a render texture of the target size,
  so a smaller window still gives full 1080p.
- `PictureReader` reads the texture back through pixel buffers with
  a fence. The editor never waits on the GPU, and a frame with all
  three buffers busy is dropped and counted.
- Nothing of the take reaches the visible canvas. A first version
  drew tiles into it and copied them out, and with the low-latency
  canvas that midway state flickered on screen.
- The sound is tapped behind the master limiter in the FX worklet
  and encoded as Opus.
- H.264 is chosen when the browser has it, VP9 otherwise. A small
  own Matroska writer packs both tracks, no new dependency. It names
  the frame rate and BT.709 so ffmpeg keeps both.
- A second `Start recording` is ignored with a warning in the log.
  A take goes on through `Load scene` and a restart, pauses with the
  editor and is saved when the run ends. It stops at 2 GB.
- The editor uploads the take on the existing video route and makes
  the asset once the run is over. The video tab gets a Download
  button for the converted MP4.
- `downloadBlob` in `src/ui` replaces the song editor's own copy.

Findings:
- `On start` runs before the game screen knows the world. A take
  therefore fixes its size on the first frame that has a camera.
- Without a frame rate in the file, ffmpeg turned 30 fps into 59.94.
- The worker's poster JPEG is off in colour for every upload (red
  255 comes out as 229, BT.601). Not touched here.

Tests:
- Unit recording + matroska: 21 green, plus the node catalog cases.
  The Matroska test reads the file back with its own EBML reader.
- e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a
  1280x800 window gives a 1920x1080 MP4 with sound and no dropped
  frame. Every 4th pixel is checked, a green quarter catches an
  upside-down picture. A take ended by Esc is saved with the
  timestamp name.
- e2e 60j: the exported game downloads an `.mkv` with EBML magic.
- The flicker is gone on the user's machine without `?vsync`.
- Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:11:05 +02:00

98 lines
3.3 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import type { Asset, VideoAsset } from '../../core/model/Asset';
import { Button, downloadFile } from '../../ui';
import { videoFileUrl } from '../../viewport/VideoLibrary';
import { AssetTabEditor } from '../AssetTabEditor';
import { TOOLBAR_ICON_SVG } from '../panels/toolbarIcons';
import type { EditorState } from '../state';
import { VideoCanvas } from './VideoCanvas';
import { formatDuration } from './videoInfo';
const INFO_WIDTH = 220;
/**
* The Video tab (#402): the video with its clips, the stretches of it a sprite can wear instead of
* the whole video. The header plays the picked clip, else the whole video (Space does too).
*/
export class VideoEditor extends AssetTabEditor<VideoAsset, VideoCanvas> {
/** Public for the check scripts. */
readonly playButton: Button;
/** Saves the MP4 the server made (#405): a recording of the game view is had this way. */
readonly downloadButton: Button;
constructor(state: EditorState) {
super(state, { infoWidth: INFO_WIDTH });
this.playButton = new Button('Play', {
icon: TOOLBAR_ICON_SVG.play,
minWidth: 72,
tooltip: 'Play the picked clip, else the whole video (Space)',
onClick: () => this.togglePlay(),
});
this.downloadButton = new Button('Download', {
tooltip: 'Save the video as an MP4 file',
onClick: () => this.download(),
});
const canvas = new VideoCanvas(state);
canvas.onTogglePlay = () => this.togglePlay();
canvas.onStopped = () => this.syncPlayButton();
this.mount(canvas, [this.playButton, this.downloadButton]);
state.on('load', () => this.canvas.stop());
state.on('selection', () => this.refresh());
}
override selectedId(): string {
return this.state.selectedVideoId ?? '';
}
override isMine(asset: Asset): asset is VideoAsset {
return asset.type === 'video';
}
get isPlaying(): boolean {
return this.canvas.isPlaying;
}
togglePlay(): void {
if (this.canvas.isPlaying) this.canvas.stop();
else this.canvas.play();
this.syncPlayButton();
}
private download(): void {
const asset = this.asset();
if (asset?.file) downloadFile(videoFileUrl(asset.file), `${asset.name}.mp4`);
}
/** Only called while the tab is visible. */
tick(): void {
this.canvas.tick();
}
protected override closed(assetId: string): void {
if (this.canvas.currentAssetId === assetId) this.canvas.stop();
}
protected override infoText(asset: VideoAsset): string {
if (!asset.file) return 'Not uploaded';
const span = this.canvas.shownSpan();
const whole = formatDuration(asset.duration);
const length =
span && span.end - span.start < asset.duration
? `${formatDuration(span.end - span.start)} of ${whole}`
: whole;
return `${asset.width} × ${asset.height} · ${length} · ${Number(asset.fps.toFixed(2))} fps`;
}
protected override syncExtras(asset: VideoAsset | null): void {
this.fitButton.setEnabled(Boolean(asset?.file));
this.downloadButton.setEnabled(Boolean(asset?.file));
this.syncPlayButton();
}
private syncPlayButton(): void {
const playing = this.canvas.isPlaying;
this.playButton.setIcon(playing ? TOOLBAR_ICON_SVG.stop : TOOLBAR_ICON_SVG.play);
this.playButton.setText(playing ? 'Stop' : 'Play');
this.playButton.setEnabled(Boolean(this.asset()?.file));
}
}