Record the game view from a behavior into a video asset (#405)
A game can now film itself. A behavior starts and stops a take, and the editor turns it into a video asset with the run's picture and the sound the player hears. The exported game hands the file over as a download, since there is no project to keep it in. - New action nodes `Start recording` (Name, Resolution, FPS) and `Stop recording`. An empty name becomes `Recording <timestamp>`. - Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes the height is set and the width follows the camera. - The picture is the camera only, without hints, overlays or the letterbox. It is drawn into a render texture of the target size, so a smaller window still gives full 1080p. - `PictureReader` reads the texture back through pixel buffers with a fence. The editor never waits on the GPU, and a frame with all three buffers busy is dropped and counted. - Nothing of the take reaches the visible canvas. A first version drew tiles into it and copied them out, and with the low-latency canvas that midway state flickered on screen. - The sound is tapped behind the master limiter in the FX worklet and encoded as Opus. - H.264 is chosen when the browser has it, VP9 otherwise. A small own Matroska writer packs both tracks, no new dependency. It names the frame rate and BT.709 so ffmpeg keeps both. - A second `Start recording` is ignored with a warning in the log. A take goes on through `Load scene` and a restart, pauses with the editor and is saved when the run ends. It stops at 2 GB. - The editor uploads the take on the existing video route and makes the asset once the run is over. The video tab gets a Download button for the converted MP4. - `downloadBlob` in `src/ui` replaces the song editor's own copy. Findings: - `On start` runs before the game screen knows the world. A take therefore fixes its size on the first frame that has a camera. - Without a frame rate in the file, ffmpeg turned 30 fps into 59.94. - The worker's poster JPEG is off in colour for every upload (red 255 comes out as 229, BT.601). Not touched here. Tests: - Unit recording + matroska: 21 green, plus the node catalog cases. The Matroska test reads the file back with its own EBML reader. - e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a 1280x800 window gives a 1920x1080 MP4 with sound and no dropped frame. Every 4th pixel is checked, a green quarter catches an upside-down picture. A take ended by Esc is saved with the timestamp name. - e2e 60j: the exported game downloads an `.mkv` with EBML magic. - The flicker is gone on the user's machine without `?vsync`. - Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
36ffd9a61a
commit
e191dc4b2e
35 files changed
+2211
-26
No files matched your search
@@ -0,0 +1,66 @@
|
||||
import { RECORDING_FPS, RECORDING_RESOLUTIONS, recordingSettingsOf } from '../../../recording/recording';
|
||||
import type { NodeDef, PortDef } from '../../NodeRegistry';
|
||||
import { asString } from '../../values';
|
||||
import { exec } from '../shared';
|
||||
|
||||
const choicePort = (
|
||||
key: string,
|
||||
label: string,
|
||||
options: readonly { value: string; label: string }[],
|
||||
): PortDef => ({
|
||||
key,
|
||||
label: key,
|
||||
type: 'string',
|
||||
default: options[0]!.value,
|
||||
field: { key, label, kind: 'enum', default: options[0]!.value, options: [...options] },
|
||||
});
|
||||
|
||||
export const StartRecording: NodeDef = {
|
||||
type: 'StartRecording',
|
||||
label: 'Start recording',
|
||||
category: 'action',
|
||||
description:
|
||||
'Records the game view as a video, with what the master plays: only the picture of the camera, ' +
|
||||
'none of what lies over it. "name" is what the recording is saved as; left empty, it is named ' +
|
||||
'after the moment it starts. "resolution" Game view records at the size the picture has on ' +
|
||||
'screen when it starts, 720p and 1080p at that height with the width of the camera. The recording ' +
|
||||
'goes on through Load scene and Restart game, holds while the editor pauses the run, and ends with ' +
|
||||
'Stop recording or with the run. In the editor it becomes a video asset of the project, once the ' +
|
||||
'server has converted it; a released game downloads it as an MKV file. A second start while one ' +
|
||||
'records is ignored and says so in the log.',
|
||||
inputs: [
|
||||
exec(),
|
||||
{
|
||||
key: 'name',
|
||||
label: 'name',
|
||||
type: 'string',
|
||||
field: { key: 'name', label: 'Name', kind: 'text', default: '' },
|
||||
},
|
||||
choicePort('resolution', 'Resolution', RECORDING_RESOLUTIONS),
|
||||
choicePort('fps', 'FPS', RECORDING_FPS),
|
||||
],
|
||||
outputs: [exec()],
|
||||
params: [],
|
||||
run: (ctx, inputs) => {
|
||||
ctx.startRecording(
|
||||
recordingSettingsOf(asString(inputs.name), asString(inputs.resolution), asString(inputs.fps)),
|
||||
);
|
||||
return { next: 'exec' };
|
||||
},
|
||||
};
|
||||
|
||||
export const StopRecording: NodeDef = {
|
||||
type: 'StopRecording',
|
||||
label: 'Stop recording',
|
||||
category: 'action',
|
||||
description:
|
||||
'Ends the recording Start recording began and saves it. Without one running it does nothing but ' +
|
||||
'say so in the log.',
|
||||
inputs: [exec()],
|
||||
outputs: [exec()],
|
||||
params: [],
|
||||
run: (ctx) => {
|
||||
ctx.stopRecording();
|
||||
return { next: 'exec' };
|
||||
},
|
||||
};
|
||||
@@ -33,6 +33,7 @@ import { StopSong } from './action/StopSong';
|
||||
import { SetZ } from './action/SetZ';
|
||||
import { Spawn } from './action/Spawn';
|
||||
import { StopGame } from './action/StopGame';
|
||||
import { StartRecording, StopRecording } from './action/recordingNodes';
|
||||
import { StopSound } from './action/StopSound';
|
||||
import { CellToWorld, SetTile, TileAt, TileLiteral, WorldToCell } from './action/tileNodes';
|
||||
import { Abs } from './data/Abs';
|
||||
@@ -323,6 +324,8 @@ export const BUILT_IN_NODES: readonly NodeDef[] = [
|
||||
Log,
|
||||
StopGame,
|
||||
RestartGame,
|
||||
StartRecording,
|
||||
StopRecording,
|
||||
PauseGame,
|
||||
ResumeGame,
|
||||
IsGamePaused,
|
||||
|
||||
Reference in new issue
Block a user