Record the game view from a behavior into a video asset (#405)
A game can now film itself. A behavior starts and stops a take, and the editor turns it into a video asset with the run's picture and the sound the player hears. The exported game hands the file over as a download, since there is no project to keep it in. - New action nodes `Start recording` (Name, Resolution, FPS) and `Stop recording`. An empty name becomes `Recording <timestamp>`. - Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes the height is set and the width follows the camera. - The picture is the camera only, without hints, overlays or the letterbox. It is drawn into a render texture of the target size, so a smaller window still gives full 1080p. - `PictureReader` reads the texture back through pixel buffers with a fence. The editor never waits on the GPU, and a frame with all three buffers busy is dropped and counted. - Nothing of the take reaches the visible canvas. A first version drew tiles into it and copied them out, and with the low-latency canvas that midway state flickered on screen. - The sound is tapped behind the master limiter in the FX worklet and encoded as Opus. - H.264 is chosen when the browser has it, VP9 otherwise. A small own Matroska writer packs both tracks, no new dependency. It names the frame rate and BT.709 so ffmpeg keeps both. - A second `Start recording` is ignored with a warning in the log. A take goes on through `Load scene` and a restart, pauses with the editor and is saved when the run ends. It stops at 2 GB. - The editor uploads the take on the existing video route and makes the asset once the run is over. The video tab gets a Download button for the converted MP4. - `downloadBlob` in `src/ui` replaces the song editor's own copy. Findings: - `On start` runs before the game screen knows the world. A take therefore fixes its size on the first frame that has a camera. - Without a frame rate in the file, ffmpeg turned 30 fps into 59.94. - The worker's poster JPEG is off in colour for every upload (red 255 comes out as 229, BT.601). Not touched here. Tests: - Unit recording + matroska: 21 green, plus the node catalog cases. The Matroska test reads the file back with its own EBML reader. - e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a 1280x800 window gives a 1920x1080 MP4 with sound and no dropped frame. Every 4th pixel is checked, a green quarter catches an upside-down picture. A take ended by Esc is saved with the timestamp name. - e2e 60j: the exported game downloads an `.mkv` with EBML magic. - The flicker is gone on the user's machine without `?vsync`. - Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
36ffd9a61a
commit
e191dc4b2e
35 files changed
+2211
-26
No files matched your search
@@ -13,7 +13,7 @@ const BACKGROUND = '#22252c';
|
||||
|
||||
/**
|
||||
* A small game of its own: a red crate in the middle of the camera that the arrow keys move, a
|
||||
* `Log` on start, and S stopping the game.
|
||||
* `Log` and a recording on start, and S stopping the game.
|
||||
*/
|
||||
function game(id) {
|
||||
const mover = {
|
||||
@@ -37,10 +37,15 @@ function game(id) {
|
||||
nodes: [
|
||||
node('c-start', 'OnStart', 0, 0),
|
||||
node('c-log', 'Log', 260, 0, { message: 'released game started' }),
|
||||
node('c-record', 'StartRecording', 520, 0, { name: 'Released 60', fps: '30' }),
|
||||
node('c-key', 'OnKeyDown', 0, 160, { key: 's' }),
|
||||
node('c-stop', 'StopGame', 260, 160),
|
||||
],
|
||||
edges: [edge('c1', 'c-start', 'exec', 'c-log', 'exec'), edge('c2', 'c-key', 'exec', 'c-stop', 'exec')],
|
||||
edges: [
|
||||
edge('c1', 'c-start', 'exec', 'c-log', 'exec'),
|
||||
edge('c2', 'c-key', 'exec', 'c-stop', 'exec'),
|
||||
edge('c3', 'c-log', 'exec', 'c-record', 'exec'),
|
||||
],
|
||||
};
|
||||
return {
|
||||
formatVersion: 2,
|
||||
@@ -226,6 +231,9 @@ export async function release(t) {
|
||||
);
|
||||
|
||||
// 60d. A click starts it in the start scene, zoomed to Auto: a whole number of pixels per unit.
|
||||
// What the page hands the browser to download is caught here instead (60j).
|
||||
await t.evaluate(`window.__downloads = [];
|
||||
HTMLAnchorElement.prototype.click = function () { window.__downloads.push({ name: this.download, href: this.href }); };`);
|
||||
const size = { w: await json('innerWidth'), h: await json('innerHeight') };
|
||||
await click(size.w / 2, size.h / 2);
|
||||
await waitUntil('__gameMakerPlay.running()', { label: 'the game runs' });
|
||||
@@ -286,6 +294,24 @@ export async function release(t) {
|
||||
JSON.stringify({ moved, screen: await player('screen()') }),
|
||||
);
|
||||
|
||||
// 60j (#405). The game recorded itself from its start: ending the run saved the recording, and a
|
||||
// released game, with no project to put it in, hands it to the browser as a Matroska file.
|
||||
await waitUntil('window.__downloads.length > 0', { label: 'the recording is downloaded' });
|
||||
const downloads = await json('window.__downloads');
|
||||
const file = JSON.parse(
|
||||
await t.evaluate(`fetch(${JSON.stringify(downloads[0]?.href ?? '')})
|
||||
.then((r) => r.arrayBuffer())
|
||||
.then((b) => JSON.stringify({ bytes: b.byteLength, head: [...new Uint8Array(b.slice(0, 4))] }))`),
|
||||
);
|
||||
check(
|
||||
'a released game downloads its recording as an MKV file once the run ends',
|
||||
downloads.length === 1 &&
|
||||
downloads[0].name === 'Released 60.mkv' &&
|
||||
file.bytes > 1000 &&
|
||||
JSON.stringify(file.head) === JSON.stringify([0x1a, 0x45, 0xdf, 0xa3]),
|
||||
JSON.stringify({ downloads: downloads.map((d) => d.name), file }),
|
||||
);
|
||||
|
||||
// 60h (#267). A click on the corner button makes the page fullscreen and does not reach the
|
||||
// game – it would start it again; a second click leaves fullscreen.
|
||||
const corner = (await player('fullscreen()')).button;
|
||||
|
||||
Reference in new issue
Block a user