Commit Graph
4 Commits
Author SHA1 Message Date
adminandClaude Opus 5.5 e191dc4b2e Record the game view from a behavior into a video asset (#405)
game-maker / gate (push) Successful in 4m14s
game-maker / suite (push) Successful in 17m7s
A game can now film itself. A behavior starts and stops a take, and
the editor turns it into a video asset with the run's picture and the
sound the player hears. The exported game hands the file over as a
download, since there is no project to keep it in.

- New action nodes `Start recording` (Name, Resolution, FPS) and
  `Stop recording`. An empty name becomes `Recording <timestamp>`.
- Resolution is `Game view`, `720p` or `1080p`. For the fixed sizes
  the height is set and the width follows the camera.
- The picture is the camera only, without hints, overlays or the
  letterbox. It is drawn into a render texture of the target size,
  so a smaller window still gives full 1080p.
- `PictureReader` reads the texture back through pixel buffers with
  a fence. The editor never waits on the GPU, and a frame with all
  three buffers busy is dropped and counted.
- Nothing of the take reaches the visible canvas. A first version
  drew tiles into it and copied them out, and with the low-latency
  canvas that midway state flickered on screen.
- The sound is tapped behind the master limiter in the FX worklet
  and encoded as Opus.
- H.264 is chosen when the browser has it, VP9 otherwise. A small
  own Matroska writer packs both tracks, no new dependency. It names
  the frame rate and BT.709 so ffmpeg keeps both.
- A second `Start recording` is ignored with a warning in the log.
  A take goes on through `Load scene` and a restart, pauses with the
  editor and is saved when the run ends. It stops at 2 GB.
- The editor uploads the take on the existing video route and makes
  the asset once the run is over. The video tab gets a Download
  button for the converted MP4.
- `downloadBlob` in `src/ui` replaces the song editor's own copy.

Findings:
- `On start` runs before the game screen knows the world. A take
  therefore fixes its size on the first frame that has a camera.
- Without a frame rate in the file, ffmpeg turned 30 fps into 59.94.
- The worker's poster JPEG is off in colour for every upload (red
  255 comes out as 229, BT.601). Not touched here.

Tests:
- Unit recording + matroska: 21 green, plus the node catalog cases.
  The Matroska test reads the file back with its own EBML reader.
- e2e 67 (new) on melektaus-atx: ALL PASSED. 1080p30 from a
  1280x800 window gives a 1920x1080 MP4 with sound and no dropped
  frame. Every 4th pixel is checked, a green quarter catches an
  upside-down picture. A take ended by Esc is saved with the
  timestamp name.
- e2e 60j: the exported game downloads an `.mkv` with EBML magic.
- The flicker is gone on the user's machine without `?vsync`.
- Gate on melektaus-atx: CHECK EXIT 0, 299 files, 4586 tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 21:11:05 +02:00
adminandClaude Opus 5.5 d48f174caf Play videos on sprites, converted on the server with clips (#402)
game-maker / gate (push) Successful in 4m15s
game-maker / suite (push) Failing after 15m49s
A video becomes an asset like a sound: uploaded once, converted to one
format by a worker of its own, cut into clips in its tab, and worn by a
sprite through its Texture row. Only Chrome is a target, without alpha.

- Server: POST ?r=videos queues the upload under the hash of its bytes,
  so the MP4's name is known at once. The worker (VideoWorker over
  VideoFormat) writes H.264/AAC, yuv420p, constant frame rate, a
  keyframe every 15 frames, at most 1920x1080, with a poster and the
  details beside it. A failed job says why and is tried again on the
  next upload.
- An upload may be up to 2 GB. It is streamed from the request to disk
  and hashed there, so php-fpm holds none of it in memory. The other
  routes keep their 256 MB.
- The worker runs in the container game-maker-video on stein2, a
  compose project of its own (tools/video/), and in CI beside the PHP
  server. The CI image has ffmpeg for it.
- Model: asset type video with size, length, frame rate and sound, and
  clips like a sound's. The clip helpers and commands serve both.
- A sprite's Texture row offers videos and clips after the pictures.
  resolvePicture names a playing video by a key, and the texture library
  hands it to a VideoLibrary with one player per key.
- The editor shows a placed video as a quiet preview. Play starts it
  over at its clip's start and lets it be heard through the master bus,
  a pause holds it, Esc makes it quiet again.
- Video tab: the picture, a ruler, the clip strip and a track on the
  Sound tab's time axis. A drag spans a clip, a handle moves an edge,
  Play plays the picked clip.
- The picker shows a video on its way with its progress and adopts an
  upload from it once it is ready. Dropping a video file does the same.

Decided on my own:

- Two sprites wearing the same clip share one player and show the same
  picture. Control per instance belongs to the follow-up with nodes.
- A video plays in real time, the game's time scale does not slow it.
- The container may use 3 of the 4 cores of stein2.

Findings:

- The first real upload (279 MB) got 413: the API's own ceiling of
  256 MB for every request, not nginx or the Brandmauer.
- Chrome has no picture of a video that never played, and a Pixi
  VideoSource drawn before the first frame stays empty for good. The
  texture is made in the first frame callback (openVideo).
- A player nobody asks for in 2 s is let go. The inspector's PREVIEW
  asked only on an edit, so with the sprite not on screen its texture
  was destroyed under it and the batcher threw on every frame. The
  preview now redraws every frame while it shows a video, like a Time
  shader. A let-go texture keeps an empty source instead of none.
- VideoSource.load() reloads the element and cuts off an earlier
  play(). Playing starts after loadeddata.

Tests:

- PHP videos.test.php: 68 checks, with the real ffmpeg
- 300 MB upload through Brandmauer and nginx: 201 in 21 s, php-fpm at
  13.7 MB RSS at most
- Unit videos.test.ts: 20 tests
- e2e 66 (stein2, atx was off): ALL PASSED, 18 checks; e2e 04 with the
  new asset count green
- Gate (atx): 294 files, 4520 tests. serialize.test.ts used video as
  its example of an unknown type and now uses a made-up one.
- Counter-checks: a keyframe gap of 30 fails the PHP check, a key that
  keeps the video's end fails 2 unit tests, a clip without its end
  fails "plays only its stretch" in e2e 66

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-11 12:12:39 +02:00
adminandClaude Opus 5.5 bc688f3efc Refactoring round 7: split BehaviorSystem, fewer copies (#85)
game-maker / gate (push) Successful in 3m50s
game-maker / suite (push) Successful in 12m3s
Recurring cleanup round over the code since round 6 (056a622), 105
commits. Behaviour is unchanged. Duplicated code in src went from 29
clones with 229 lines (0.32 %) to 14 clones with 109 lines (0.15 %),
measured with jscpd and the same settings as round 6.

- BehaviorSystem.ts shrinks from 941 to 589 lines. Collecting the
  pulses (keys, timers, collisions, animations) moved to
  behaviorPulses.ts. The NodeContext the nodes receive moved to
  behaviorContext.ts. Imports from outside stay the same, TimeScaled
  is still exported from BehaviorSystem.ts.
- PointKindButton and drawPointGrip replace the "Point:" button with
  its menu and the point and handle drawing that the curve and the
  vector art tab each had on their own.
- lerpVec, cubicAt and splitCubic in src/core/util/vec.ts replace four
  copies of the cubic curve and two de Casteljau splits.
- mapNested in Properties.ts is the one walk over lists and structs
  that liveObjects and objectRefs did separately. withFieldRenamed
  and withoutField replace three copies of renaming a struct field
  and two of removing one.
- jsonBody and staleRevision build the JSON body and turn a 409 into
  StaleRevisionError for all stores. The auth store turns a 401 into
  its error in one place.
- Smaller helpers: structAssetFrom for the two "Make struct type"
  commands, shownObject in the inspector, selectedItem for the size
  gizmo and the polygon corners, solidTiles in the tilemap collision,
  requireVariablesAsset and requireBehaviorVariable.
- Left as is on purpose: the plain forwarding from the collision and
  physics systems to Solids, the pointer wiring of the canvases and
  the identical last lines of the two make commands.
- Removed unused code: tilemapTileIds and tilemapsUsing were never
  called since #128 and #131. EditorState.runtimeProject became unused
  with GameRun in #266. The assetId getter of the make commands was
  read only by a test.
- 60 values used only in their own file lost their export. The shader
  and particle node catalogues stay exported as in round 6.
- Five comments that told history now describe the current state. The
  doc comment of BehaviorSystem sits on the class again.
- README lists the struct, clip area and curve assets and mentions the
  game link from #266.

Findings:
- Deleting a tile that is painted on a tilemap leaves its cells in
  place, and they draw nothing. The delete dialog does not mention the
  tilemaps. That was probably the job of the unused tilemapsUsing.
  Left as is, because fixing it changes behaviour.
- The solid filter for tile surfaces was untested. A bent filter
  stayed green. A new test in tests/core/surface.test.ts now fails
  against it.

Tests:
- ./check.sh: 195 files, 2815 tests, exit 0 (one test removed with
  tilemapsUsing, one added for tile surfaces).
- Full e2e run: ALL PASSED, 1580 checks.
- Smokes jumprun, tetris, actor: 24, 19 and 21 checks, all passed.
  The example project files were restored byte for byte afterwards.
- Counter-checks, 12, all fail as expected: 10 in unit tests (among
  them mapNested, splitCubic, cubicAt, staleRevision, the stay pulse,
  the late timer and the time scale), 2 in the browser (Point button
  always enabled fails 47, selectedItem without a hit fails 29).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-29 16:16:37 +02:00
adminandClaude Fable 5.1 cc6db5f350 Three stores, three importers and three pickers that differ only in their type are one of each (#85)
game-maker / gate (push) Successful in 1m33s
game-maker / suite (push) Successful in 5m46s
Refactoring round one: HttpFontStore/HttpSoundStore/HttpTextureStore collapse into a generic
HttpFileStore over a FileStore contract; fontImport/soundImport/textureImport share assetImport
with an ImportKind per sort; ImagePicker/FontPicker/SoundPicker sit on FilePicker and describe
only their row. The group/comment drag logic leaves GraphCanvas for annotationDrag.ts (with its
own unit test), declaredRows.ts replaces the duplicated row build in particles.ts and shaders.ts,
and dead code in font/sound/compile/emission/transform/actions is gone. README: asset list,
check count 958, the two-job CI, the Scenes wiki page.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-16 17:38:54 +02:00