Videos, Behaviors: record the game view (#405)

- Videos: new section on recording the game view, and the Download
  button in the video tab.
- Behaviors: the nodes Start recording and Stop recording.
- Development: 2016 editor checks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
adminandClaude Opus 5.5 committed 2026-10-11 21:11:08 +02:00
1 parent 4fdb69ba9e
commit 59eda02688
3 files changed
+50 -2

No files matched your search

+1
@@ -267,6 +267,7 @@ the inspector).
`Set bus volume` sets how loud a bus or the master is and the data node `Bus volume` reads it
(see [Sounds](Sounds#buses-and-the-master)),
`Load scene` (swaps the running scene once the frame is over — see [Scenes](Scenes)), `Log`, `Stop game`, `Restart game`,
`Start recording` / `Stop recording` (record the game view with its sound as a video, see [Videos](Videos#recording-the-game-view)),
`Pause game`, `Resume game`, `Set time scale` with the data nodes `Is game paused` and `Time scale` (see [Concepts](Concepts), *The game pauses itself*),
`Apply force`, `Apply impulse`, `Set velocity`, `Set angular velocity`, `Set gravity` (see [Physics](Physics)),
`Raycast` (what a ray meets first on the layers it looks at: object, collider, point, normal, distance — see [Colliders](Colliders#raycast)),
+1 -1
@@ -21,7 +21,7 @@ php tests/php/releases.test.php # the release stor
php tests/php/files.test.php # uploaded files in a folder per account (#344), the same way
php tests/php/videos.test.php # the video store, its queue and the worker (#402), with the
# real ffmpeg: the CI image and game-maker-video have it
node tools/seed-e2e.mjs && node tools/e2e.mjs # 2010 editor checks, screenshots in /tmp/gm-e2e-*.png
node tools/seed-e2e.mjs && node tools/e2e.mjs # 2016 editor checks, screenshots in /tmp/gm-e2e-*.png
GM_STEPS=20 node tools/e2e.mjs # or 1-12, or 6,20 — a subset while working on one step
./e2e.sh start [GM_STEPS] # the same run detached, see below
./e2e.sh status # live progress, or the last run's result
+48 -1
@@ -55,6 +55,7 @@ The clip gestures are the same as in the [Sound tab](Sounds), with the track in
- **Play** (or **Space**) plays the picked clip, else the whole video, with its sound and its
Loop. The playhead runs along the ruler.
- The wheel zooms the time, the middle button pans, **Fit** shows the whole video.
- **Download** saves the MP4 the server made, under the asset's name.
A clip's **Start (s)** and **End (s)** are in the inspector as well; an end of 0 is the end of
the video. A clip is a frame long at least, its edges are stored to the millisecond.
@@ -128,7 +129,8 @@ shader's parameters. Or one sprite whose Texture is keyed from clip to clip.
measured from where the time began to move.
- **Heard** only at about 1x (0.9x to 1.1x); a still, scrubbing, slow and fast motion are quiet.
- **Exact while it runs live in the browser**, as far as Chrome keeps up: a picture that comes too
late leaves the one before it on screen. Exporting a cut as a video file is #405.
late leaves the one before it on screen. To have a cut as a file, record the game view while it
plays (see *Recording the game view* below).
- **Ready for the cut:** a driven sprite that is switched off or out of view keeps its player,
quiet, at its Video time, in the editor as in a game (#404). Switched on, the first frame shows
its picture, not its plain colour
@@ -143,6 +145,51 @@ shader's parameters. Or one sprite whose Texture is keyed from clip to clip.
videos..."). A released game waits the same way. A driven sprite without its picture is not
drawn at all, rather than in its plain colour.
## Recording the game view
Two action nodes record what the game view shows, with what the master plays (#405):
| Node | |
|---|---|
| **Start recording** | **name** (empty: `Recording 2026-10-11 16:42:05`, when it started), **resolution** (*Game view*, *720p*, *1080p*), **fps** (*60*, *30*) |
| **Stop recording** | ends it and saves it |
- **The picture** is the camera's, nothing of what lies over it: no notice, no fullscreen button,
no performance overlay, no letterbox, no menu or window of the editor. *Game view* records at the
size the picture has on screen when the first frame is taken (at most 3840x2160), *720p* and
*1080p* at that height, the width from the camera's shape. A window smaller than that is no
loss: the frame is drawn into a picture of its own at its full size. Text and a camera shader
are drawn at the size of the screen, though, and are scaled up with it, so they come out softer.
The size holds for the whole recording.
- **Nothing of it shows on screen.** The frame is never drawn into the editor's canvas: that
canvas is low latency (without `?vsync`) and shows what is drawn into it at once, so a frame
drawn there flickered in its corner. The picture comes back from the graphics card a frame or
two later, without the editor waiting for it.
- **The sound** is what leaves the master behind its limiter, what you hear. Opus, 160 kbit/s.
- **It goes on** through `Load scene` and `Restart game`. The editor's pause holds it and leaves the
paused time out; a game that pauses itself is still seen, so that is recorded. Another tab in
front of the Game tab does not stop it. Ending the run (Esc, `Stop game`) saves it.
- **One at a time:** a second `Start recording` while one records is ignored and says so in the
log, and so does a `Stop recording` with none. In the editor run (*Run in editor*) the nodes only
write to the log.
- **In the editor** the recording goes up to the server like any upload and is converted to MP4.
Once that is done, and the run is over, it is a video asset of the project with its name, ready
for a sprite or for **Download** in its tab. The notice line says how far it is.
- **In a released game** there is no project to put it in: the browser downloads it as an `.mkv`
when it ends. That file is the one the browser made (H.264 and Opus), not converted.
- **A file of 2 GB**, the most an upload may be, ends the recording and saves it, with a line in
the log. At 1080p60 that is roughly half an hour.
- **When the encoder cannot keep up**, a frame is left out rather than everything falling behind;
the converted MP4 has a constant frame rate again.
How it is made: H.264 from Chrome's `VideoEncoder` (VP9 where there is no H.264), hardware where
the computer has it, 10 Mbit/s at 1080p60, a keyframe every 2 s. The file is Matroska, written by
the editor itself (`src/core/recording/matroska.ts`) in clusters as the frames come, with the
frame rate and the colour (BT.709) in its head, so nothing has to guess them. The rules (size,
codec level, which frames are taken, where the picture goes in a frame) are plain functions in
`src/core/recording/recording.ts`, the browser's side is `src/viewport/GameRecorder.ts`, and
`src/viewport/PictureReader.ts` reads the frames back through pixel buffers with a fence.
## The worker
Converting is a job, not a request. The API puts the upload into `data/videos/queue/` (hidden