From 59eda02688f20beeb1615587d1ebd2e69fc75a9d Mon Sep 17 00:00:00 2001 From: MelekTaus Date: Sun, 11 Oct 2026 21:11:08 +0200 Subject: [PATCH] Videos, Behaviors: record the game view (#405) - Videos: new section on recording the game view, and the Download button in the video tab. - Behaviors: the nodes Start recording and Stop recording. - Development: 2016 editor checks. Co-Authored-By: Claude Opus 5.5 --- Behaviors.md | 1 + Development.md | 2 +- Videos.md | 49 ++++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 50 insertions(+), 2 deletions(-) diff --git a/Behaviors.md b/Behaviors.md index ebeed8f..210e74a 100644 --- a/Behaviors.md +++ b/Behaviors.md @@ -267,6 +267,7 @@ the inspector). `Set bus volume` sets how loud a bus or the master is and the data node `Bus volume` reads it (see [Sounds](Sounds#buses-and-the-master)), `Load scene` (swaps the running scene once the frame is over — see [Scenes](Scenes)), `Log`, `Stop game`, `Restart game`, + `Start recording` / `Stop recording` (record the game view with its sound as a video, see [Videos](Videos#recording-the-game-view)), `Pause game`, `Resume game`, `Set time scale` with the data nodes `Is game paused` and `Time scale` (see [Concepts](Concepts), *The game pauses itself*), `Apply force`, `Apply impulse`, `Set velocity`, `Set angular velocity`, `Set gravity` (see [Physics](Physics)), `Raycast` (what a ray meets first on the layers it looks at: object, collider, point, normal, distance — see [Colliders](Colliders#raycast)), diff --git a/Development.md b/Development.md index b14e25a..b7b1f3a 100644 --- a/Development.md +++ b/Development.md @@ -21,7 +21,7 @@ php tests/php/releases.test.php # the release stor php tests/php/files.test.php # uploaded files in a folder per account (#344), the same way php tests/php/videos.test.php # the video store, its queue and the worker (#402), with the # real ffmpeg: the CI image and game-maker-video have it -node tools/seed-e2e.mjs && node tools/e2e.mjs # 2010 editor checks, screenshots in /tmp/gm-e2e-*.png +node tools/seed-e2e.mjs && node tools/e2e.mjs # 2016 editor checks, screenshots in /tmp/gm-e2e-*.png GM_STEPS=20 node tools/e2e.mjs # or 1-12, or 6,20 — a subset while working on one step ./e2e.sh start [GM_STEPS] # the same run detached, see below ./e2e.sh status # live progress, or the last run's result diff --git a/Videos.md b/Videos.md index 3c61700..ca58cf1 100644 --- a/Videos.md +++ b/Videos.md @@ -55,6 +55,7 @@ The clip gestures are the same as in the [Sound tab](Sounds), with the track in - **Play** (or **Space**) plays the picked clip, else the whole video, with its sound and its Loop. The playhead runs along the ruler. - The wheel zooms the time, the middle button pans, **Fit** shows the whole video. +- **Download** saves the MP4 the server made, under the asset's name. A clip's **Start (s)** and **End (s)** are in the inspector as well; an end of 0 is the end of the video. A clip is a frame long at least, its edges are stored to the millisecond. @@ -128,7 +129,8 @@ shader's parameters. Or one sprite whose Texture is keyed from clip to clip. measured from where the time began to move. - **Heard** only at about 1x (0.9x to 1.1x); a still, scrubbing, slow and fast motion are quiet. - **Exact while it runs live in the browser**, as far as Chrome keeps up: a picture that comes too - late leaves the one before it on screen. Exporting a cut as a video file is #405. + late leaves the one before it on screen. To have a cut as a file, record the game view while it + plays (see *Recording the game view* below). - **Ready for the cut:** a driven sprite that is switched off or out of view keeps its player, quiet, at its Video time, in the editor as in a game (#404). Switched on, the first frame shows its picture, not its plain colour @@ -143,6 +145,51 @@ shader's parameters. Or one sprite whose Texture is keyed from clip to clip. videos..."). A released game waits the same way. A driven sprite without its picture is not drawn at all, rather than in its plain colour. +## Recording the game view + +Two action nodes record what the game view shows, with what the master plays (#405): + +| Node | | +|---|---| +| **Start recording** | **name** (empty: `Recording 2026-10-11 16:42:05`, when it started), **resolution** (*Game view*, *720p*, *1080p*), **fps** (*60*, *30*) | +| **Stop recording** | ends it and saves it | + +- **The picture** is the camera's, nothing of what lies over it: no notice, no fullscreen button, + no performance overlay, no letterbox, no menu or window of the editor. *Game view* records at the + size the picture has on screen when the first frame is taken (at most 3840x2160), *720p* and + *1080p* at that height, the width from the camera's shape. A window smaller than that is no + loss: the frame is drawn into a picture of its own at its full size. Text and a camera shader + are drawn at the size of the screen, though, and are scaled up with it, so they come out softer. + The size holds for the whole recording. +- **Nothing of it shows on screen.** The frame is never drawn into the editor's canvas: that + canvas is low latency (without `?vsync`) and shows what is drawn into it at once, so a frame + drawn there flickered in its corner. The picture comes back from the graphics card a frame or + two later, without the editor waiting for it. +- **The sound** is what leaves the master behind its limiter, what you hear. Opus, 160 kbit/s. +- **It goes on** through `Load scene` and `Restart game`. The editor's pause holds it and leaves the + paused time out; a game that pauses itself is still seen, so that is recorded. Another tab in + front of the Game tab does not stop it. Ending the run (Esc, `Stop game`) saves it. +- **One at a time:** a second `Start recording` while one records is ignored and says so in the + log, and so does a `Stop recording` with none. In the editor run (*Run in editor*) the nodes only + write to the log. +- **In the editor** the recording goes up to the server like any upload and is converted to MP4. + Once that is done, and the run is over, it is a video asset of the project with its name, ready + for a sprite or for **Download** in its tab. The notice line says how far it is. +- **In a released game** there is no project to put it in: the browser downloads it as an `.mkv` + when it ends. That file is the one the browser made (H.264 and Opus), not converted. +- **A file of 2 GB**, the most an upload may be, ends the recording and saves it, with a line in + the log. At 1080p60 that is roughly half an hour. +- **When the encoder cannot keep up**, a frame is left out rather than everything falling behind; + the converted MP4 has a constant frame rate again. + +How it is made: H.264 from Chrome's `VideoEncoder` (VP9 where there is no H.264), hardware where +the computer has it, 10 Mbit/s at 1080p60, a keyframe every 2 s. The file is Matroska, written by +the editor itself (`src/core/recording/matroska.ts`) in clusters as the frames come, with the +frame rate and the colour (BT.709) in its head, so nothing has to guess them. The rules (size, +codec level, which frames are taken, where the picture goes in a frame) are plain functions in +`src/core/recording/recording.ts`, the browser's side is `src/viewport/GameRecorder.ts`, and +`src/viewport/PictureReader.ts` reads the frames back through pixel buffers with a fence. + ## The worker Converting is a job, not a request. The API puts the upload into `data/videos/queue/` (hidden