Record the whole editor from the main menu

The game view could already record itself through nodes. Making a
video of the editor itself, to show how something is done, needed an
outside screen recorder. The main menu now starts and stops one, with
no options to set.

- "Start recording" sits in its own group before General settings, and
  reads "Stop recording" while one runs. The name is
  "Recording <time>", the size is the whole canvas in device pixels
  (an odd row or column is cut), 60 fps always. Sound is the master,
  saving goes the same way as for the game view.
- The encoder, the sound and the Matroska writing of a take moved from
  GameRecorder.ts to RecordingTake.ts. A take now reads its pictures
  from a FrameSource, so the game view and the editor share it.
- The editor's picture is copied from the canvas after each render, in
  the same frame, so nothing is drawn twice. A resting editor does not
  render, and the recording repeats its last picture.
- The mouse pointer is drawn into the recording in the shape the editor
  shows (arrow, hand, resize arrows, I-beam, crosshair, move).
- A red dot with the running time stands beside the menu button on
  screen only. It is drawn after the copy, and the toolbar makes room
  for it.
- Drawing that dot made Pixi take it as the root for every hit test,
  and the menu would not open any more. The stage is set back as the
  last rendered object after it.

Tests:
- Gate on melektaus-atx green: 303 files, 4686 tests.
- Unit recording: 20 green, 5 new.
- e2e 67: 8 checks, 67e new. It records through the menu and decodes
  the MP4: 1280x800 at 60 fps, the pointer black at the mouse, the dot
  missing, 238 plain spots alike with the canvas. Without the pointer
  and with the dot drawn before the copy, 67e is red.
- e2e 46: the main menu lists "Start recording".

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-12 13:40:47 +02:00
1 parent c0ba2673ba
commit f177ee90a4
13 files changed
+1190 -478

No files matched your search

+35 -6
View File
@@ -78,7 +78,8 @@ import { TilemapPanel } from './panels/TilemapPanel';
import { Toolbar } from './panels/Toolbar';
import { type PlayHost, PlaySession, type WorldSnapshot } from './PlaySession';
import { type RecordingUpload, RecordingUploads } from './recordingUploads';
import type { MadeRecording } from '../viewport/GameRecorder';
import { EditorRecorder } from './EditorRecorder';
import type { MadeRecording } from '../viewport/RecordingTake';
import { ProjectSession } from './ProjectSession';
import { ParticlesEditor } from './particles/ParticlesEditor';
import { ShaderEditor } from './shader/ShaderEditor';
@@ -321,6 +322,8 @@ export class Editor implements PlayHost {
fps: number | null = null;
private readonly play: PlaySession;
private readonly recordings: RecordingUploads;
/** Records the whole editor from the main menu. */
private readonly editorRecorder: EditorRecorder;
private readonly editorRun: EditorRun;
private readonly projects: ProjectSession;
private readonly leftColumn: VStack;
@@ -413,6 +416,15 @@ export class Editor implements PlayHost {
this.inspector = new Inspector(state);
this.tilemapPanel = new TilemapPanel(state);
this.recordings = new RecordingUploads({ state, videos });
this.editorRecorder = new EditorRecorder({
log: (line) => {
state.setNotice(line);
console.warn('[recording]', line);
},
saved: (recording) => this.recordings.add(recording),
showBadge: (on) => this.toolbar.showRecording(on),
badgeRoom: () => this.toolbar.recordingRoom(),
});
this.play = new PlaySession(state, this);
this.editorRun = new EditorRun(state, {
// The game view is the one place where no key means anything to the editor, so that is where
@@ -792,13 +804,19 @@ export class Editor implements PlayHost {
}
/**
* The main menu's last group (#179): the user management for an administrator, and the plugins
* for everybody – both open the one tab, which holds the plugins below the accounts.
* The main menu's last groups: recording the editor, then the user management for an
* administrator and the plugins for everybody (#179) – both open the one tab, which holds the
* plugins below the accounts.
*/
private managementEntries(): MenuEntry[] {
const open = () => this.openDocument('management');
const recorder = this.editorRecorder;
return [
{ separator: true },
recorder.isRecording
? { label: 'Stop recording', onSelect: () => recorder.stop() }
: { label: 'Start recording', onSelect: () => recorder.start() },
{ separator: true },
{ label: 'General settings…', onSelect: () => this.openGeneralSettings() },
...(this.usersPane ? [{ label: 'User management', onSelect: open }] : []),
{ label: 'Plugins', onSelect: open },
@@ -1323,6 +1341,7 @@ export class Editor implements PlayHost {
await this.autosave.flush();
this.autosave.stop();
if (this.play.isPlaying) this.play.stop();
this.editorRecorder.stop();
this.editorRun.shutDown();
await this.plugins?.stopAll();
for (const cleanup of this.cleanups.splice(0).reverse()) cleanup();
@@ -1511,9 +1530,19 @@ export class Editor implements PlayHost {
};
}
/** The recordings of the game view (#405) and what became of them on the way into the project. */
recordingInfo(): ReturnType<PlaySession['recordingInfo']> & { uploads: RecordingUpload[] } {
return { ...this.play.recordingInfo(), uploads: this.recordings.info() };
/**
* The recordings of the game view (#405), those of the editor, and what became of them on the way
* into the project.
*/
recordingInfo(): ReturnType<PlaySession['recordingInfo']> & {
editor: ReturnType<EditorRecorder['info']>;
uploads: RecordingUpload[];
} {
return {
...this.play.recordingInfo(),
editor: this.editorRecorder.info(),
uploads: this.recordings.info(),
};
}
recordingSaved(recording: MadeRecording): void {
+281
View File
@@ -0,0 +1,281 @@
import { Container, Graphics, UPDATE_PRIORITY } from 'pixi.js';
import { cursorShape, type CursorShape } from '../core/recording/cursors';
import {
EDITOR_RECORDING_FPS,
editorRecordingSize,
elapsedText,
fitPicture,
recordingName,
} from '../core/recording/recording';
import { crispText, renderCount, requestRender, textStyle, Theme, ui } from '../ui';
import { type FrameSource, Take, type TakeHost, type TakeInfo } from '../viewport/RecordingTake';
/** What the editor recorder needs of the editor. */
export interface EditorRecorderHost extends TakeHost {
/** Makes room in the top bar for the badge while recording, or takes it away. */
showBadge(on: boolean): void;
/** Where the badge goes, in CSS pixels. */
badgeRoom(): { x: number; y: number; w: number; h: number };
}
/**
* Records the whole editor (main menu → Start recording): the canvas as it is on screen, at its size
* in device pixels and 60 fps, with the mouse pointer drawn in, since the system draws that over the
* canvas. What the editor draws is copied from the canvas right after each render, so recording costs
* no second drawing. The badge beside the menu is drawn onto the screen after that copy, so it is
* never in the recording.
*/
export class EditorRecorder {
private take: Take | null = null;
private source: CanvasSource | null = null;
/** Every recording of this page, newest last (test hook). */
private readonly takes: Take[] = [];
private seen = -1;
private startedAt = 0;
private shownText = '';
private readonly badge = new RecordingBadge();
constructor(private readonly host: EditorRecorderHost) {}
get isRecording(): boolean {
return this.take !== null;
}
start(): void {
if (this.take) return;
const app = ui().app;
const source = new CanvasSource(app.canvas);
const take = new Take(recordingName('', new Date()), EDITOR_RECORDING_FPS, source, this.host, () => {
if (this.take === take) this.end();
});
this.take = take;
this.source = source;
this.takes.push(take);
this.startedAt = performance.now();
this.shownText = '';
this.seen = -1;
this.host.showBadge(true);
// After the render loop (`UPDATE_PRIORITY.LOW`): the canvas holds this tick's picture then.
app.ticker.add(this.tick, this, UPDATE_PRIORITY.UTILITY);
requestRender();
}
stop(): void {
const take = this.take;
if (!take) return;
this.end();
take.finish('');
}
private end(): void {
this.take = null;
this.source?.detach();
this.source = null;
ui().app.ticker.remove(this.tick, this);
this.host.showBadge(false);
requestRender();
}
private tick(): void {
const take = this.take;
const source = this.source;
if (!take || !source) return;
const drawn = renderCount();
if (drawn !== this.seen) {
this.seen = drawn;
source.copy();
this.drawBadge();
}
take.capture();
// An idle editor draws nothing, and the time beside the menu has to move on.
if (elapsedText(performance.now() - this.startedAt) !== this.shownText) requestRender();
}
private drawBadge(): void {
this.shownText = elapsedText(performance.now() - this.startedAt);
this.badge.show(this.host.badgeRoom(), this.shownText);
const app = ui().app;
app.renderer.render({ container: this.badge, clear: false });
// Pixi hit-tests the pointer against what it drew on screen last: that has to be the stage.
(app.renderer as unknown as { _lastObjectRendered: Container })._lastObjectRendered = app.stage;
}
info(): {
recording: boolean;
takes: TakeInfo[];
badge: { x: number; y: number; w: number; h: number } | null;
} {
return {
recording: this.isRecording,
takes: this.takes.map((take) => take.info()),
badge: this.isRecording ? this.host.badgeRoom() : null,
};
}
}
/** The red dot and the time beside the menu while the editor records. */
class RecordingBadge extends Container {
private readonly dot = new Graphics();
private readonly time = crispText({ text: '', style: textStyle('body') });
constructor() {
super();
this.addChild(this.dot, this.time);
}
show(room: { x: number; y: number; w: number; h: number }, text: string): void {
const radius = 5;
this.position.set(room.x, room.y);
this.dot
.clear()
.circle(radius + 2, room.h / 2, radius)
.fill(Theme.colors.danger);
this.time.text = text;
this.time.position.set(radius * 2 + 7, Math.round((room.h - this.time.height) / 2));
}
}
/**
* The canvas, copied after each render into a picture of the recording's size; a frame is that
* picture with the mouse pointer drawn over it.
*/
class CanvasSource implements FrameSource {
private readonly size: { w: number; h: number };
private readonly picture: OffscreenCanvas;
private readonly picturePen: OffscreenCanvasRenderingContext2D;
private readonly frame: OffscreenCanvas;
private readonly framePen: OffscreenCanvasRenderingContext2D;
private readonly pointer = new PointerTrack();
private readonly paths = new Map<CursorShape, Path2D>();
private copied = false;
private ready: VideoFrame[] = [];
constructor(private readonly canvas: HTMLCanvasElement) {
this.size = editorRecordingSize({ w: canvas.width, h: canvas.height });
this.picture = new OffscreenCanvas(this.size.w, this.size.h);
this.picturePen = this.picture.getContext('2d', { alpha: false })!;
this.frame = new OffscreenCanvas(this.size.w, this.size.h);
this.framePen = this.frame.getContext('2d', { alpha: false })!;
this.pointer.attach(canvas);
}
open(): { w: number; h: number } {
return this.size;
}
/** The canvas holds what the editor just drew: it becomes the picture. */
copy(): void {
const { canvas, picturePen: pen } = this;
const at = fitPicture({ w: canvas.width, h: canvas.height }, this.size);
pen.fillStyle = '#000';
pen.fillRect(0, 0, this.size.w, this.size.h);
pen.drawImage(canvas, 0, 0, canvas.width, canvas.height, at.x, at.y, at.w, at.h);
this.copied = true;
}
holds(): boolean {
return !this.copied;
}
get busy(): boolean {
return false;
}
get pending(): number {
return this.ready.length;
}
take(ms: number): boolean {
if (!this.copied) return false;
this.framePen.drawImage(this.picture, 0, 0);
this.drawPointer();
this.ready.push(
new VideoFrame(this.frame, {
timestamp: Math.round(ms * 1000),
duration: Math.round(1e6 / EDITOR_RECORDING_FPS),
}),
);
return true;
}
private drawPointer(): void {
const { canvas, pointer } = this;
const shape = cursorShape(canvas.style.cursor);
if (!pointer.inside || !shape) return;
const rect = canvas.getBoundingClientRect();
if (rect.width === 0 || rect.height === 0) return;
const at = fitPicture({ w: canvas.width, h: canvas.height }, this.size);
const x = at.x + ((pointer.x - rect.left) / rect.width) * at.w;
const y = at.y + ((pointer.y - rect.top) / rect.height) * at.h;
const scale = at.w / rect.width;
let path = this.paths.get(shape);
if (!path) this.paths.set(shape, (path = new Path2D(shape.path)));
const pen = this.framePen;
pen.save();
pen.translate(x, y);
pen.scale(scale, scale);
pen.rotate((shape.turn * Math.PI) / 180);
pen.translate(-shape.hot.x, -shape.hot.y);
pen.lineJoin = 'round';
pen.lineWidth = 2;
pen.strokeStyle = '#fff';
pen.stroke(path);
pen.fillStyle = '#000';
pen.fill(path);
pen.restore();
}
collect(encode: (frame: VideoFrame) => void): void {
for (const frame of this.ready.splice(0)) encode(frame);
}
/** The recording ends: the pointer is no longer followed, the frames taken still go out. */
detach(): void {
this.pointer.detach();
}
close(): void {
this.pointer.detach();
for (const frame of this.ready.splice(0)) frame.close();
}
}
/** Where the mouse is, in client pixels, and whether it is over the page at all. */
class PointerTrack {
x = 0;
y = 0;
inside = false;
private attached = false;
private readonly moved = (event: PointerEvent): void => {
this.x = event.clientX;
this.y = event.clientY;
this.inside = true;
};
private readonly left = (): void => {
this.inside = false;
};
/** Starts where Pixi last saw the pointer: the click on Start recording put it there. */
attach(canvas: HTMLCanvasElement): void {
if (this.attached) return;
this.attached = true;
const rect = canvas.getBoundingClientRect();
const at = ui().app.renderer.events.pointer.global;
this.x = rect.left + at.x;
this.y = rect.top + at.y;
this.inside = at.x > 0 || at.y > 0;
window.addEventListener('pointermove', this.moved, { capture: true, passive: true });
window.addEventListener('pointerdown', this.moved, { capture: true, passive: true });
document.documentElement.addEventListener('pointerleave', this.left);
}
detach(): void {
if (!this.attached) return;
this.attached = false;
window.removeEventListener('pointermove', this.moved, { capture: true });
window.removeEventListener('pointerdown', this.moved, { capture: true });
document.documentElement.removeEventListener('pointerleave', this.left);
}
}
+2 -1
View File
@@ -17,7 +17,8 @@ import { Input } from '../runtime/systems/Input';
import type { World } from '../runtime/World';
import { onFrame, ui } from '../ui';
import { soundLibrary } from '../viewport/SoundLibrary';
import { GameRecorder, type MadeRecording } from '../viewport/GameRecorder';
import { GameRecorder } from '../viewport/GameRecorder';
import type { MadeRecording } from '../viewport/RecordingTake';
import { videoLibrary } from '../viewport/VideoLibrary';
import { FrameMeter, type FrameTimes } from './frameMeter';
import { saveArea } from './saves';
+21 -1
View File
@@ -27,6 +27,9 @@ const SAVE_LABELS: Record<SaveStatus, string> = {
/** Square, because an icon has no reading direction that would ask for a wider box. */
const ICON_BUTTON = Theme.controlHeight;
/** Width beside the menu for the recording's dot and time. */
const RECORDING_ROOM = 52;
type IconName = keyof typeof TOOLBAR_ICON_SVG;
/**
@@ -46,6 +49,8 @@ function iconButton(name: IconName, onClick: () => void, tooltip: () => string):
export class Toolbar extends HStack {
private readonly bg = new Graphics();
private readonly menuButton: Button;
/** The room between the menu and undo; it widens while the editor records (`showRecording`). */
private readonly lead = new Spacer(Theme.padding);
private readonly undoButton: Button;
private readonly redoButton: Button;
private readonly playButton: Button;
@@ -74,7 +79,7 @@ export class Toolbar extends HStack {
});
this.menuButton = iconButton('menu', () => actions.openMainMenu(this.menuButton), tip('menu'));
this.add(this.menuButton);
this.add(new Spacer(Theme.padding));
this.add(this.lead);
this.undoButton = iconButton('undo', () => state.undo(), tip('undo'));
this.redoButton = iconButton('redo', () => state.redo(), tip('redo'));
this.add(this.undoButton);
@@ -132,6 +137,21 @@ export class Toolbar extends HStack {
};
}
/**
* Makes room beside the menu for what shows that the editor records, or takes it away again. The
* room is left empty here: what fills it is drawn over the screen only, not into the recording.
*/
showRecording(on: boolean): void {
this.lead.minWidth = Theme.padding + (on ? RECORDING_ROOM : 0);
this.relayout();
}
/** Where that room is on screen, in CSS pixels. */
recordingRoom(): { x: number; y: number; w: number; h: number } {
const at = this.lead.getGlobalPosition();
return { x: at.x, y: this.menuButton.getGlobalPosition().y, w: RECORDING_ROOM, h: this.menuButton.h };
}
/** Frame rate readout; `null` while no measurement is available yet. */
setFps(fps: number | null): void {
this.fps.setText(fps === null ? '– fps' : `${fps} fps`);
+2 -2
View File
@@ -1,5 +1,5 @@
import type { UploadedVideo } from '../core/store/VideoStore';
import type { MadeRecording } from '../viewport/GameRecorder';
import type { MadeRecording } from '../viewport/RecordingTake';
import { adoptVideo, type VideoImportDeps } from './videoImport';
/** How often the server is asked whether a recording is converted, as the video picker does. */
@@ -19,7 +19,7 @@ export interface RecordingUpload {
}
/**
* Brings a recording of the game view into the project (#405): up to the server, which converts it
* Brings a recording of the game view (#405) or of the editor into the project: up to the server, which converts it
* like any video, then an asset named after the recording once it is ready. Only while the editor
* is not playing: a run plays a copy of the project that is thrown away, and so would the asset be.
*/