Record the game view from a behavior into a video asset (#405)
A game can now film itself. A behavior starts and stops a take, and the editor turns it into a video asset with the run's picture and the sound the player hears. The exported game hands the file over as a download, since there is no project to keep it in. - New action nodes `Start recording` (Name, Resolution, FPS) and `Stop recording`. An empty name becomes `Recording <timestamp>`. - Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes the height is set and the width follows the camera. - The picture is the camera only, without hints, overlays or the letterbox. It is drawn into a render texture of the target size, so a smaller window still gives full 1080p. - `PictureReader` reads the texture back through pixel buffers with a fence. The editor never waits on the GPU, and a frame with all three buffers busy is dropped and counted. - Nothing of the take reaches the visible canvas. A first version drew tiles into it and copied them out, and with the low-latency canvas that midway state flickered on screen. - The sound is tapped behind the master limiter in the FX worklet and encoded as Opus. - H.264 is chosen when the browser has it, VP9 otherwise. A small own Matroska writer packs both tracks, no new dependency. It names the frame rate and BT.709 so ffmpeg keeps both. - A second `Start recording` is ignored with a warning in the log. A take goes on through `Load scene` and a restart, pauses with the editor and is saved when the run ends. It stops at 2 GB. - The editor uploads the take on the existing video route and makes the asset once the run is over. The video tab gets a Download button for the converted MP4. - `downloadBlob` in `src/ui` replaces the song editor's own copy. Findings: - `On start` runs before the game screen knows the world. A take therefore fixes its size on the first frame that has a camera. - Without a frame rate in the file, ffmpeg turned 30 fps into 59.94. - The worker's poster JPEG is off in colour for every upload (red 255 comes out as 229, BT.601). Not touched here. Tests: - Unit recording + matroska: 21 green, plus the node catalog cases. The Matroska test reads the file back with its own EBML reader. - e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a 1280x800 window gives a 1920x1080 MP4 with sound and no dropped frame. Every 4th pixel is checked, a green quarter catches an upside-down picture. A take ended by Esc is saved with the timestamp name. - e2e 60j: the exported game downloads an `.mkv` with EBML magic. - The flicker is gone on the user's machine without `?vsync`. - Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
36ffd9a61a
commit
e191dc4b2e
35 files changed
+2211
-26
No files matched your search
@@ -391,3 +391,52 @@ class ChannelProbe extends AudioWorkletProcessor {
|
||||
}
|
||||
|
||||
registerProcessor('channel-probe', ChannelProbe);
|
||||
|
||||
/** Frames a recording's sound is handed over in: about a tenth of a second. */
|
||||
const RECORD_BLOCK = 4096;
|
||||
|
||||
/**
|
||||
* Hands what reaches it to the page, both sides, `RECORD_BLOCK` frames at a time (#405): the master
|
||||
* as it leaves the limiter, for a recording of the game view. Told to stop, it sends what it holds
|
||||
* and an empty piece after it, the sign that nothing more comes.
|
||||
*/
|
||||
class RecordTap extends Stoppable {
|
||||
private readonly sides = [new Float32Array(RECORD_BLOCK), new Float32Array(RECORD_BLOCK)];
|
||||
private filled = 0;
|
||||
|
||||
constructor() {
|
||||
super();
|
||||
this.port.onmessage = () => {
|
||||
this.send();
|
||||
this.port.postMessage([new Float32Array(0), new Float32Array(0)]);
|
||||
this.stop();
|
||||
};
|
||||
}
|
||||
|
||||
process(inputs: Float32Array[][]): boolean {
|
||||
if (this.stopped) return false;
|
||||
const input = inputs[0] ?? [];
|
||||
const left = input[0];
|
||||
const frames = left?.length ?? 128;
|
||||
this.sides.forEach((side, c) => {
|
||||
const data = input[c] ?? left;
|
||||
if (data) side.set(data, this.filled);
|
||||
else side.fill(0, this.filled, this.filled + frames);
|
||||
});
|
||||
this.filled += frames;
|
||||
if (this.filled + frames > RECORD_BLOCK) this.send();
|
||||
return true;
|
||||
}
|
||||
|
||||
private send(): void {
|
||||
if (this.filled === 0) return;
|
||||
const out = this.sides.map((side) => side.slice(0, this.filled));
|
||||
this.filled = 0;
|
||||
this.port.postMessage(
|
||||
out,
|
||||
out.map((side) => side.buffer),
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
registerProcessor('record-tap', RecordTap);
|
||||
Reference in new issue
Block a user