Show game view recordings in their true colours in other players

A game view recording looked darker and its colours off in VLC, and
each round of recording a recording made it worse. Chrome's encoder,
fed RGB frames, wrote full-range H.264 and left the colour out of the
stream. Only our Matroska head said "full range", and VLC reads only
the stream. Chrome itself plays the converted MP4, which was right.

- The GPU now turns the game view into I420, limited range BT.709,
  before it is read back (PictureReader). The encoder gets the planes
  with that colour, as it does for a recording of the whole editor.
- Where the planes lie in the read-back picture is a plain function,
  i420Layout in recording.ts.
- Only 1.5 instead of 4 bytes a pixel come back from the GPU.
- avcColour.ts writes the colour into the SPS of the encoder's avcC as
  well, so a player that reads only the stream knows it. An SPS with a
  VUI only gets its colour block replaced. One without gets a VUI that
  holds the colour. An SPS it cannot read stays as the encoder made it.
- A first try sent the frames through a 2D canvas. That gave limited
  range too, but only 18 to 20 of 33 frames at 1080p30.

Found on the way:

- With the colour in the stream alone, VLC 3.0.8 still showed the
  full-range picture too dark. It ignores the range.
- The one old full-range original on the server got the same SPS fix
  without loss: the clusters are byte-equal, the decoded frames are
  equal (framemd5), backup in /root/config-backup-20261013/videos/.
- Older game view recordings stay full range. Fixing that in VLC would
  mean encoding them again.

Tests:
- tsc, eslint, Prettier clean.
- Gate on melektaus-atx: CHECK EXIT 0, 308 files, 4772 tests.
- avc-colour.test.ts new (9), expectations read back with ffmpeg
  trace_headers. Counter-check: the old bug in the no-VUI path turns
  it red.
- e2e 67: ALL PASSED, 12 checks. 67d-ii reads tv/bt709 from the file
  and from the bare H.264 stream, red with the old code.
- 1080p30 34 frames, 640x360@60 68 frames, none dropped (was 33/47).
- Checked by the user: a new recording looks right in VLC 3.0.8.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-13 19:48:44 +02:00
1 parent 31ce6480b2
commit 30b618d9a9
9 files changed
+666 -15

No files matched your search

+56
View File
@@ -1,3 +1,7 @@
import { execFileSync } from 'node:child_process';
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
import { tmpdir } from 'node:os';
import { join } from 'node:path';
import { MAKE_OBJECT, pickAssetType } from './lib/scaffold.mjs';
/**
@@ -196,6 +200,58 @@ export async function recording(t) {
colors,
);
// 67d-ii. The original is limited range and says so in the H.264 stream too, not only in the
// Matroska head: VLC 3 showed a full-range recording too dark, even with the range in the
// stream. ffprobe reads the file, then the stream taken out of it as raw H.264.
const original = named
? await fetch(`${t.baseUrl}data/videos/${named.file.replace(/\.mp4$/, '.source.mkv')}`)
: null;
const streamColour = { file: null, stream: null };
if (original?.ok) {
const dir = mkdtempSync(join(tmpdir(), 'rc67-'));
const mkv = join(dir, 'take.mkv');
const raw = join(dir, 'take.h264');
writeFileSync(mkv, new Uint8Array(await original.arrayBuffer()));
execFileSync('ffmpeg', [
'-v',
'error',
'-y',
'-i',
mkv,
'-map',
'0:v',
'-c',
'copy',
'-bsf:v',
'h264_mp4toannexb',
'-f',
'h264',
raw,
]);
const probe = (path) =>
execFileSync('ffprobe', [
'-v',
'error',
'-select_streams',
'v:0',
'-of',
'json',
'-show_entries',
'stream=color_range,color_space,color_transfer,color_primaries',
path,
]).toString();
streamColour.file = JSON.parse(probe(mkv)).streams[0];
streamColour.stream = JSON.parse(probe(raw)).streams[0];
rmSync(dir, { recursive: true, force: true });
}
check(
'the recording is limited range and says its colour in the H.264 stream as in its Matroska head',
streamColour.file !== null &&
streamColour.file.color_range === 'tv' &&
JSON.stringify(streamColour.stream) === JSON.stringify(streamColour.file),
JSON.stringify({ status: original?.status ?? null, ...streamColour }),
);
// 67e. The editor records itself from the main menu: the whole canvas at 60 fps, the mouse pointer
// drawn in where the system draws it over the canvas, and the red dot beside the menu on
// screen only. The frame is read where the pointer rested on the empty middle of the top bar.