406839baeaad6fd9590d31f5eb6f025657fbc7d7
9
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  adminandClaude Opus 5.5
|
87b2ee9402 |
Record every frame with Real time off on Start recording (#423)
At 4K and 60 fps the user's computer reaches about 39 fps in real time.
Reading each frame back from the GPU takes longer than a frame, and
neither reading back later nor the canvas route helped. With Real time
off the run moves on by one frame of the recording at a time, however
long the computer takes, so the file has every frame.
- Start recording has a new check Real time, on by default. Only the
node records so, the editor recording stays in real time.
- SteadyPace in recording.ts is the pace as a pure class. A step is
1/fps once the frame before is taken and the sound up to it is made,
otherwise 0 and the run holds. A frame's time is its number.
- The sound is made apart, as decided in the issue. AudioSwitch routes
what the run plays to the live library, or to the offline library of
a SteadySound while such a recording lasts. That library has the same
mix, buses, FX and limiter and is rendered up to each next frame.
- A driven video holds at every picture asked for, and the frame waits
for it. Its sound is decoded from the MP4 and played in step through
its bus.
- Two Chrome quirks measured and worked around. A held video hands
over the picture of a seek before seeked, so the frame callback is
asked for when the seek begins. A seek right onto where a picture
begins shows that one or the one before, so such a seek lands half a
millisecond further. Together 30 s became 10.7 s for 2 s of video.
- The notice tells how far the recording has got, and the line at the
end how long it took.
Deviations and limits:
- Input is not ignored, it still reaches the game, which runs slower.
- At most ten minutes per recording. Chrome holds the whole buffer of
the OfflineAudioContext while rendering, 230 MB for ten minutes.
- A free video plays on in real time and does not fit. A sound playing
before the start stays at the speakers.
Found along the way:
- A check in e2e 60 was red since
|
||
|
adminandClaude Opus 5.5
|
77365f17d3 |
Say when a recording starts and when it cannot keep up
A 4K60 game view recording on the user's machine had 517 of 827 frames. The rest were skipped without a word: when the editor's tick came late, FrameGate let the due slots pass, and only frames lost to a busy encoder or read back counted as dropped. The file kept its time but stuttered, and nothing said so. - FrameGate counts the frames it took and the slots the recording's time has room for. The difference is skipped, next to dropped in __gm.recording(). - FramePace judges the missed frames over spans of 2 s. The first time more than 10 % are missed, the log warns once with the frames a second the recording really got. - At the end, when more than 5 % are missed, a warning gives the frames of the whole file and what helps (30 fps or a smaller resolution). It comes after the upload's notice, which would cover it at once in the editor. - The first second counts for neither warning. Without that, a short e2e recording warned about 3 frames lost while starting. - A recording that starts logs an info line with its size, fps, codec and sound. TakeHost.log takes a level for it. The editor recording keeps the line to the console, its notice would be in the video. - consoleLine is the one place with an eslint exception for console.info, so the console panel shows the line plain. Found along the way: headless Chrome in WSL records the editor with real stutter in e2e 67 (1280x800@60 27 %, 2304x1440@30 12-14 %). That is the software encoder there, and only now shown. Tests: - Gate on melektaus-atx: green, 309 files, 4810 tests. - Unit recording.test.ts: 36 green, 6 new for FrameGate.skipped and FramePace. With skipped stuck at 0 the test turns red. - e2e 67 on melektaus-atx: ALL PASSED, 15 checks, run twice. The new 67b-ii slows the editor to about 20 fps: a 640x360@60 recording says "about 19 of 60 fps" and saves with 95 of 192 frames. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
30b618d9a9 |
Show game view recordings in their true colours in other players
A game view recording looked darker and its colours off in VLC, and each round of recording a recording made it worse. Chrome's encoder, fed RGB frames, wrote full-range H.264 and left the colour out of the stream. Only our Matroska head said "full range", and VLC reads only the stream. Chrome itself plays the converted MP4, which was right. - The GPU now turns the game view into I420, limited range BT.709, before it is read back (PictureReader). The encoder gets the planes with that colour, as it does for a recording of the whole editor. - Where the planes lie in the read-back picture is a plain function, i420Layout in recording.ts. - Only 1.5 instead of 4 bytes a pixel come back from the GPU. - avcColour.ts writes the colour into the SPS of the encoder's avcC as well, so a player that reads only the stream knows it. An SPS with a VUI only gets its colour block replaced. One without gets a VUI that holds the colour. An SPS it cannot read stays as the encoder made it. - A first try sent the frames through a 2D canvas. That gave limited range too, but only 18 to 20 of 33 frames at 1080p30. Found on the way: - With the colour in the stream alone, VLC 3.0.8 still showed the full-range picture too dark. It ignores the range. - The one old full-range original on the server got the same SPS fix without loss: the clusters are byte-equal, the decoded frames are equal (framemd5), backup in /root/config-backup-20261013/videos/. - Older game view recordings stay full range. Fixing that in VLC would mean encoding them again. Tests: - tsc, eslint, Prettier clean. - Gate on melektaus-atx: CHECK EXIT 0, 308 files, 4772 tests. - avc-colour.test.ts new (9), expectations read back with ffmpeg trace_headers. Counter-check: the old bug in the no-VUI path turns it red. - e2e 67: ALL PASSED, 12 checks. 67d-ii reads tv/bt709 from the file and from the bare H.264 stream, red with the old code. - 1080p30 34 frames, 640x360@60 68 frames, none dropped (was 33/47). - Checked by the user: a new recording looks right in VLC 3.0.8. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
2ba41c1bd0 |
Record 1440p and 4K, and choose sound and conversion in the node
The Start recording node only offered the game view, 720p and 1080p, and always recorded with sound, converted and imported. The recording from the main menu could already leave those out. - Resolution has 1440p and 4K. As with 720p and 1080p the height is fixed and the width follows the camera, so 16:9 gives 2560x1440 and 3840x2160. - Every resolution is capped at 3840x2160, as the game view already was. A 21:9 camera in 4K would be 5040 px wide, which the encoder levels are not made for. It is scaled down to 3840x1646. - The node has Sound, Convert and Import after conversion, all on by default as in "Start recording...". Import only counts with Convert. An old graph without the inputs records as before. - A released game ignores Convert and Import and downloads the MKV. - Both recordings now hand their upload options the same way, from GameRecorder over PlaySession to RecordingUploads. RecordingUploadOptions moved to core/recording/recording.ts. Tests: - tsc, eslint, Prettier clean - Unit recording and node-catalog: 451 green, recording 29 with 3 new tests and 1 extended - e2e 67 on melektaus-atx: ALL PASSED, 10 checks - Not checked in the browser: a real 4K recording, and the node with its switches off. - The gate ran once on this commit and the next one together. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
d4fc425233 |
Draw the editor's mouse pointer from SVG files, on screen too
The pointer drawn into an editor recording lived as paths in code, so it could not be looked at or edited by hand. The shapes are now SVG files, and the editor can show the same pointer on screen, so what is recorded is what was seen. - src/core/recording/cursors/ holds one file per shape: arrow, hand, open-hand, fist, move, crosshair, text and resize. Resize is turned for the vertical and diagonal directions. - data-hot="x y" on the <svg> is the hotspot in viewBox units. Width and height in the file are ignored, and anything before <svg>, such as <?xml ...> from a drawing program, is cut off. - The open hand had six fingers. It has a thumb and four now, the middle one longest. - CURSOR_SIZE = 0.8 scales every pointer the same on screen and in a recording, so the files keep their units. - General settings has "Custom mouse pointer", on by default. It hangs the files into Pixi's cursorStyles as CSS cursors with the keyword as fallback, so the system still draws the pointer and the canvas does not. Off gives Pixi's own values back at once. - A recording draws the pointer from the same files at its own size, so it stays sharp in 4K. It still reads the shape from the canvas when that holds a url(...) cursor. Tests: - Unit recording and view-tween: 3 new, green. They read viewBox and data-hot in an edited form, the boxes of turned shapes, the default size and old settings without the switch. Red without data-hot and without cutting before <svg>. - e2e 46i-ii: the canvas cursor is a url(...) ending in default and the image loads. Off gives inherit and stores false, on brings the file back. Red without useCursorFiles. - e2e 67e: the recording is still black at the mouse position. - The README names 2046 e2e checks. - Gate on melektaus-atx: 304 files, 4702 tests green. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
ba01d146e9 |
Choose size, rate and what follows when recording the editor
The editor recording had fixed settings: the canvas size at 60 fps,
always converted and imported. Recording in 4K needs a choice, and the
server shrank every video to 1080p anyway. Start recording... now opens
a dialog, and a recording always goes to the server as it is.
- The dialog holds name, resolution (Screen, 1080p, 1440p, 4K), fps
(60 or 30), mouse pointer, sound, Convert and Import after
conversion. Import is greyed out while Convert is off. The browser
keeps the last choice, not the name.
- A larger resolution makes the editor draw finer while it records,
so 4K is sharp, not stretched. It is never below the screen's own
resolution and goes back after Stop.
- Stop always uploads. Without Convert the file only stays on the
server. Without Import it is converted but no asset is added. Both
are found in the video picker.
- The server keeps the original next to each MP4 for every upload, so
Download in the video tab and the picker gives the original. Videos
from before have none, they still download their MP4.
- A file left unconverted shows in the picker as not converted, with
a Convert button. POST videos/{file}/convert queues it, and upload
takes convert=0.
- The worker now caps a video at 3840x2160 instead of 1920x1080. Videos
converted before stay as they are.
Tests:
- PHP videos.test.php: 81 checks green, with the original, convert=0,
a later Convert and the 4K cap.
- Unit recording: 5 new, green. Red without the rule that a recording
is never below the screen's resolution.
- e2e 67e-ii: the video tab downloads the .source.mkv, a Matroska
file. 67f: 1440p from a 1280x800 window gives 2304x1440 at 30 fps
without sound, kept and not imported, then converted in the picker.
Both red with the MP4 download and without the finer drawing.
- The README names 2045 e2e checks.
- Gate on melektaus-atx: 304 files, 4699 tests green.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
adminandClaude Opus 5.5
|
f177ee90a4 |
Record the whole editor from the main menu
The game view could already record itself through nodes. Making a video of the editor itself, to show how something is done, needed an outside screen recorder. The main menu now starts and stops one, with no options to set. - "Start recording" sits in its own group before General settings, and reads "Stop recording" while one runs. The name is "Recording <time>", the size is the whole canvas in device pixels (an odd row or column is cut), 60 fps always. Sound is the master, saving goes the same way as for the game view. - The encoder, the sound and the Matroska writing of a take moved from GameRecorder.ts to RecordingTake.ts. A take now reads its pictures from a FrameSource, so the game view and the editor share it. - The editor's picture is copied from the canvas after each render, in the same frame, so nothing is drawn twice. A resting editor does not render, and the recording repeats its last picture. - The mouse pointer is drawn into the recording in the shape the editor shows (arrow, hand, resize arrows, I-beam, crosshair, move). - A red dot with the running time stands beside the menu button on screen only. It is drawn after the copy, and the toolbar makes room for it. - Drawing that dot made Pixi take it as the root for every hit test, and the menu would not open any more. The stage is set back as the last rendered object after it. Tests: - Gate on melektaus-atx green: 303 files, 4686 tests. - Unit recording: 20 green, 5 new. - e2e 67: 8 checks, 67e new. It records through the menu and decodes the MP4: 1280x800 at 60 fps, the pointer black at the mouse, the dot missing, 238 plain spots alike with the canvas. Without the pointer and with the dot drawn before the copy, 67e is red. - e2e 46: the main menu lists "Start recording". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
b2b4cbad28 |
Keep a video's loading out of recordings and first frames (#417)
A Start recording right after Play composition still began with empty frames. The composition's clock waited for its video, but the recording went on and took every frame while the video loaded. A clip that starts later in the video also slipped through, because its player already had one picture while it still sought to the clip's start. - The view now counts every video sprite it leaves out as missing, not only one without any first picture. - GameRecorder takes no frame while the game view misses a video, for at most VIDEO_WAIT_MS (3 s) at a time. The recorded time stands and the sound of that time is dropped, so it stays in sync. - RecordingClock.start forgets a hold from before the first frame. Without that, the first real frames all got the time 0. - A followed player starts at 1x until the first speed reading. Before, it paused and sought every frame for about 100 ms, so the first pictures after a wait stuck and then jumped. A step faster than 4x and a time running backwards do not count as a start. - Test hooks: __gm.gameVideo(id) reads the game view's own player, TakeInfo.lastFrameAt shows when the last frame was taken. Tests: - Unit recording (clock after an early hold), video-follow (2 new, 1 adapted): 449 green in tests/core/recording, tests/viewport and tests/core/composition* - e2e 68e (a clip from 0.8 s shows that picture first and then plays), 68f (a recording that already runs takes no frame while the composition's video loads) - 68e is red without the start at 1x, 68f without the hold - README: 2038 editor checks Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
adminandClaude Opus 5.5
|
e191dc4b2e |
Record the game view from a behavior into a video asset (#405)
A game can now film itself. A behavior starts and stops a take, and the editor turns it into a video asset with the run's picture and the sound the player hears. The exported game hands the file over as a download, since there is no project to keep it in. - New action nodes `Start recording` (Name, Resolution, FPS) and `Stop recording`. An empty name becomes `Recording <timestamp>`. - Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes the height is set and the width follows the camera. - The picture is the camera only, without hints, overlays or the letterbox. It is drawn into a render texture of the target size, so a smaller window still gives full 1080p. - `PictureReader` reads the texture back through pixel buffers with a fence. The editor never waits on the GPU, and a frame with all three buffers busy is dropped and counted. - Nothing of the take reaches the visible canvas. A first version drew tiles into it and copied them out, and with the low-latency canvas that midway state flickered on screen. - The sound is tapped behind the master limiter in the FX worklet and encoded as Opus. - H.264 is chosen when the browser has it, VP9 otherwise. A small own Matroska writer packs both tracks, no new dependency. It names the frame rate and BT.709 so ffmpeg keeps both. - A second `Start recording` is ignored with a warning in the log. A take goes on through `Load scene` and a restart, pauses with the editor and is saved when the run ends. It stops at 2 GB. - The editor uploads the take on the existing video route and makes the asset once the run is over. The video tab gets a Download button for the converted MP4. - `downloadBlob` in `src/ui` replaces the song editor's own copy. Findings: - `On start` runs before the game screen knows the world. A take therefore fixes its size on the first frame that has a camera. - Without a frame rate in the file, ffmpeg turned 30 fps into 59.94. - The worker's poster JPEG is off in colour for every upload (red 255 comes out as 229, BT.601). Not touched here. Tests: - Unit recording + matroska: 21 green, plus the node catalog cases. The Matroska test reads the file back with its own EBML reader. - e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a 1280x800 window gives a 1920x1080 MP4 with sound and no dropped frame. Every 4th pixel is checked, a green quarter catches an upside-down picture. A take ended by Esc is saved with the timestamp name. - e2e 60j: the exported game downloads an `.mkv` with EBML magic. - The flicker is gone on the user's machine without `?vsync`. - Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |