Run CI on a WSL2 runner with the RTX 4070 (#283)
stein2 should stop carrying the full e2e run. The new runner lives in the WSL2 distro game-maker-ci on melektaus-atx and uses the same image and fonts. Its GPU sits behind /dev/dxg and Mesa's d3d12 driver, which headless chromium cannot reach, so the harness gets a second way to start the browser. - GAME_MAKER_BROWSER=wsl starts an invisible Xvfb and an X11 chromium on ANGLE/GL. Without the variable, stein2 keeps the headless Vulkan launch exactly as before. - The harness logs the renderer and stops on software rendering. Under WSL a missing d3d12 driver falls back to llvmpipe, which still offers WebGL 2 and would pass unnoticed. - The CI image gains xvfb. The font set is unchanged: fc-list gives the same 149 fonts with the same hash. - ci.yml runs both jobs on the new label game-maker-wsl. The stein2 runner keeps game-maker-ci, so moving back is one line per job. Findings: - Wayland on weston reached the GPU too, but drew at about 40 fps and let the editor clock stall in long runs (GM_STEPS=1-17: 2 failed). - Vulkan through a self-built dzn caps ANGLE at ES 2.0 because VK_EXT_provoking_vertex is missing, so chromium drops the GPU. - The flaky check in e2e 41 (#282) fails more often under WSL: two of three solo runs were red. Tests: - ./check.sh: 195 files, 2823 tests passed - WSL container, GM_STEPS=1-17: ALL PASSED, 553 checks - WSL container, full run: 1589 of 1590, the one FAIL is #282 - WSL container, smokes: 24 / 19 / 21 checks - stein2 headless, e2e 39+40: ALL PASSED, 100 checks Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9b5eeaab57
commit
857e23c69b
4 files changed
+125
-21
No files matched your search
+11
-8
@@ -1,13 +1,16 @@
|
||||
# Runs the whole proof on the Gitea runner: gate, the editor suite and the three example games.
|
||||
#
|
||||
# `runs-on: game-maker-ci` picks the image built from tools/ci/Dockerfile. The render device and
|
||||
# /dev/shm come from the runner's own container.options (/_stacks/git/runner/config.yaml), not
|
||||
# from here – without them chromium falls back to software WebGL.
|
||||
# `runs-on: game-maker-wsl` is the runner in the WSL2 distro game-maker-ci on melektaus-atx
|
||||
# (#283). Its label picks the image built from tools/ci/Dockerfile. The GPU (/dev/dxg and the
|
||||
# Windows driver libraries), /dev/shm and GAME_MAKER_BROWSER=wsl come from that runner's own
|
||||
# config.yaml, not from here – without them chromium falls back to software WebGL and the harness
|
||||
# stops. The runner on stein2 still carries `game-maker-ci` with the same image and its Intel
|
||||
# GPU; pointing both jobs back at that label moves the runs there again.
|
||||
#
|
||||
# Two jobs without `needs:`, so they start together and use both slots of the runner
|
||||
# (`capacity: 2`, four cores). The gate no longer stands in front of the suite: a lint error used
|
||||
# to cost the full e2e run before it was reported, and the run reported it after six minutes
|
||||
# instead of one. Both still have to pass for the push to be green.
|
||||
# (`capacity: 2`, four cores on either host). The gate no longer stands in front of the suite:
|
||||
# a lint error used to cost the full e2e run before it was reported, and the run reported it
|
||||
# after six minutes instead of one. Both still have to pass for the push to be green.
|
||||
#
|
||||
# The price is contention: while the gate runs, vitest and chromium share the four cores, and the
|
||||
# suite measures text widths and frame counts. That is why the gate caps vitest at two workers
|
||||
@@ -20,7 +23,7 @@ on:
|
||||
|
||||
jobs:
|
||||
gate:
|
||||
runs-on: game-maker-ci
|
||||
runs-on: game-maker-wsl
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
@@ -42,7 +45,7 @@ jobs:
|
||||
php tests/php/releases.test.php
|
||||
|
||||
suite:
|
||||
runs-on: game-maker-ci
|
||||
runs-on: game-maker-wsl
|
||||
timeout-minutes: 45
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
@@ -10,11 +10,14 @@ FROM node:22-bookworm
|
||||
# chromium + the Vulkan drivers: the e2e harness starts chromium with --use-angle=vulkan and has
|
||||
# no software fallback. The render device itself comes from the runner's container.options;
|
||||
# without it chromium falls back to llvmpipe and gets WebGL 1.0 only.
|
||||
# xvfb serves the other GPU path, GAME_MAKER_BROWSER=wsl (tools/lib/browser.mjs): under WSL2 the
|
||||
# GPU is /dev/dxg behind Mesa's d3d12 driver, which headless chromium cannot reach, an X11 one can.
|
||||
# php-cli serves api/index.php, which every tool loads and saves its project through.
|
||||
# The fonts are the host's set on purpose: the checks measure text widths to find click points,
|
||||
# so a different set moves them.
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
chromium mesa-vulkan-drivers libvulkan1 \
|
||||
xvfb \
|
||||
php-cli php-mbstring \
|
||||
fonts-dejavu-core fonts-droid-fallback fonts-noto-color-emoji fonts-noto-mono \
|
||||
fonts-texgyre fonts-urw-base35 \
|
||||
|
||||
+88
-12
@@ -1,16 +1,9 @@
|
||||
import { spawn } from 'node:child_process';
|
||||
|
||||
const FLAGS = [
|
||||
'--headless',
|
||||
const COMMON = [
|
||||
'--hide-scrollbars',
|
||||
/**
|
||||
* Measured (idle editor, 20 s): software WebGL 373% of a core at 8 fps, HD 630 via ANGLE/Vulkan 69% at
|
||||
* 56 fps; no software fallback on purpose. `--use-angle=gl` and `--use-gl=egl` get no WebGL context at all.
|
||||
*/
|
||||
'--use-angle=vulkan',
|
||||
'--ignore-gpu-blocklist',
|
||||
'--enable-gpu',
|
||||
'--enable-features=Vulkan',
|
||||
// None of this serves a headless check, and its stderr was full of GCM sign-in retries.
|
||||
'--disable-background-networking',
|
||||
'--disable-sync',
|
||||
@@ -23,19 +16,102 @@ const FLAGS = [
|
||||
'--autoplay-policy=no-user-gesture-required',
|
||||
];
|
||||
|
||||
const HEADLESS = [
|
||||
'--headless',
|
||||
/**
|
||||
* Measured (idle editor, 20 s): software WebGL 373% of a core at 8 fps, HD 630 via ANGLE/Vulkan 69% at
|
||||
* 56 fps; no software fallback on purpose. `--use-angle=gl` and `--use-gl=egl` get no WebGL context at all.
|
||||
*/
|
||||
'--use-angle=vulkan',
|
||||
'--enable-features=Vulkan',
|
||||
];
|
||||
|
||||
/**
|
||||
* GAME_MAKER_BROWSER=wsl, for a runner inside WSL2 (#283). There the GPU is /dev/dxg behind Mesa's
|
||||
* d3d12 driver: no DRM render node, no Vulkan. Headless chromium finds no surface and gets no WebGL,
|
||||
* an X11 chromium on ANGLE/GL does. So this mode brings its own invisible X server.
|
||||
*
|
||||
* Measured on an RTX 4070: WebGL 2.0 on "D3D12 (NVIDIA …), OpenGL 4.2", 60 frames a second while
|
||||
* drawing. A Wayland chromium on weston reached the GPU too, but drew at ~40 and let the editor's
|
||||
* clock stall for good in a long run – its frames hang on the compositor's callbacks.
|
||||
* A real window can be backgrounded, headless never is, hence the three flags that keep it drawing.
|
||||
*/
|
||||
const WSL = [
|
||||
'--ozone-platform=x11',
|
||||
'--use-angle=gl',
|
||||
'--disable-renderer-backgrounding',
|
||||
'--disable-backgrounding-occluded-windows',
|
||||
'--disable-background-timer-throttling',
|
||||
];
|
||||
|
||||
/** Starts chromium with the CDP port open; `proc.stderr` carries its last words. */
|
||||
export function launchBrowser({ port, width, height }) {
|
||||
export async function launchBrowser({ port, width, height }) {
|
||||
const wsl = process.env.GAME_MAKER_BROWSER === 'wsl';
|
||||
const display = wsl ? await startXvfb({ width, height }) : null;
|
||||
const proc = spawn(
|
||||
'chromium',
|
||||
// --no-sandbox: this container runs as root, where chromium refuses to start otherwise.
|
||||
[
|
||||
...FLAGS,
|
||||
...COMMON,
|
||||
...(wsl ? WSL : HEADLESS),
|
||||
'--no-sandbox',
|
||||
`--remote-debugging-port=${port}`,
|
||||
`--window-size=${width},${height}`,
|
||||
'about:blank',
|
||||
],
|
||||
{ stdio: ['ignore', 'ignore', 'pipe'] },
|
||||
{ stdio: ['ignore', 'ignore', 'pipe'], env: display ? display.env : process.env },
|
||||
);
|
||||
return { proc, stop: () => proc.kill('SIGTERM'), where: 'local chromium' };
|
||||
if (display) proc.once('exit', display.stop);
|
||||
return {
|
||||
proc,
|
||||
stop: () => proc.kill('SIGTERM'),
|
||||
where: wsl ? `local chromium on Xvfb ${display.env.DISPLAY} (WSL, d3d12)` : 'local chromium',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* An X server that shows nothing. It only has to hold the window: the drawing happens in Mesa's
|
||||
* d3d12 driver, which hands finished frames over by copy. `-displayfd` lets it pick a free display
|
||||
* and report it, so two runs never share one.
|
||||
*/
|
||||
async function startXvfb({ width, height }) {
|
||||
const xvfb = spawn(
|
||||
'Xvfb',
|
||||
['-displayfd', '3', '-screen', '0', `${width}x${height}x24`, '-nolisten', 'tcp'],
|
||||
{
|
||||
stdio: ['ignore', 'ignore', 'pipe', 'pipe'],
|
||||
},
|
||||
);
|
||||
const log = [];
|
||||
xvfb.stderr.on('data', (b) => log.push(String(b)));
|
||||
const stop = () => xvfb.kill('SIGTERM');
|
||||
const number = await new Promise((resolve) => {
|
||||
let got = '';
|
||||
const timer = setTimeout(() => resolve(null), 10_000);
|
||||
xvfb.stdio[3].on('data', (b) => {
|
||||
got += String(b);
|
||||
if (got.includes('\n')) {
|
||||
clearTimeout(timer);
|
||||
resolve(got.trim());
|
||||
}
|
||||
});
|
||||
xvfb.once('exit', () => {
|
||||
clearTimeout(timer);
|
||||
resolve(null);
|
||||
});
|
||||
});
|
||||
if (!number) {
|
||||
stop();
|
||||
throw new Error(`Xvfb never reported a display:\n${log.join('').trim().slice(-2000)}`);
|
||||
}
|
||||
const env = {
|
||||
...process.env,
|
||||
DISPLAY: `:${number}`,
|
||||
// Without both, Mesa picks llvmpipe: the d3d12 driver is not its default, and it needs the
|
||||
// Windows side's libd3d12/libdxcore, which WSL mounts there.
|
||||
GALLIUM_DRIVER: 'd3d12',
|
||||
LD_LIBRARY_PATH: ['/usr/lib/wsl/lib', process.env.LD_LIBRARY_PATH].filter(Boolean).join(':'),
|
||||
};
|
||||
delete env.WAYLAND_DISPLAY;
|
||||
return { env, stop };
|
||||
}
|
||||
+23
-1
@@ -129,6 +129,27 @@ async function waitForChrome(port) {
|
||||
throw new Error('chromium did not start');
|
||||
}
|
||||
|
||||
/**
|
||||
* Names the GPU in the log and refuses software rendering. Under WSL (#283) a missing d3d12 driver
|
||||
* does not fail: Mesa falls back to llvmpipe, which still offers WebGL 2, and every check would pass
|
||||
* at a fraction of the speed the timing-sensitive ones assume.
|
||||
*/
|
||||
async function requireGpu(cdp) {
|
||||
const { result } = await cdp.send('Runtime.evaluate', {
|
||||
expression: `(() => {
|
||||
const gl = document.createElement('canvas').getContext('webgl2');
|
||||
if (!gl) return null;
|
||||
const info = gl.getExtension('WEBGL_debug_renderer_info');
|
||||
return info ? gl.getParameter(info.UNMASKED_RENDERER_WEBGL) : gl.getParameter(gl.RENDERER);
|
||||
})()`,
|
||||
returnByValue: true,
|
||||
});
|
||||
const renderer = result.value;
|
||||
console.log(`renderer: ${renderer ?? 'no WebGL 2'}`);
|
||||
if (!renderer || /llvmpipe|softpipe|lavapipe|swiftshader/i.test(renderer))
|
||||
throw new Error(`no GPU for WebGL 2 (${renderer ?? 'no context'}) – the checks would run in software`);
|
||||
}
|
||||
|
||||
/** The toolbox handed to a check body – everything a step needs to drive and inspect the editor. */
|
||||
function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
|
||||
// A throwing expression is reported, never swallowed: a mutate that dies half-way leaves the
|
||||
@@ -520,7 +541,7 @@ function makeTools({ cdp, projectId, baseUrl, consoleLog, check, shotPrefix }) {
|
||||
|
||||
export async function runChecks(options, body) {
|
||||
const { projectId, baseUrl, port = 9333, shotPrefix = 'gm-e2e', width = 1280, height = 800 } = options;
|
||||
const { proc: chrome, stop: stopBrowser, where } = launchBrowser({ port, width, height });
|
||||
const { proc: chrome, stop: stopBrowser, where } = await launchBrowser({ port, width, height });
|
||||
console.log(`browser: ${where}`);
|
||||
|
||||
const consoleLog = [];
|
||||
@@ -585,6 +606,7 @@ export async function runChecks(options, body) {
|
||||
deviceScaleFactor: 1,
|
||||
mobile: false,
|
||||
});
|
||||
await requireGpu(cdp);
|
||||
cdp.on((msg) => {
|
||||
if (msg.method === 'Runtime.consoleAPICalled')
|
||||
consoleLog.push(msg.params.args.map((a) => a.value ?? a.description).join(' '));
|
||||
|
||||
Reference in new issue
Block a user