Let the editor follow the agent's edits on screen (#429)
feuerwerk / gate (push) Successful in 4m45s
feuerwerk / suite (push) Failing after 18m0s

The user asked to see what the agent is working on through MCP. Until
now the agent had to call open for that, and mostly did not.

- After run_commands the editor shows what the call worked on last. A
  graph opens its tab with the touched nodes selected, deleted ones
  left out. A scene or container shows with its objects selected. Any
  other asset shows in the inspector.
- Objects in another scene switch the scene inside the same undo step.
  A switch of its own would be its own step, and Ctrl+Z would take back
  the switch before the agent's edit.
- A graph tab that is open already moves its camera after the nodes.
  It stays put while they are in view, moves only as far as needed with
  the zoom kept, and frames them like Fit only when they do not fit.
  This is revealBoxes in framing.ts and the state's reveal event.
- A tab that has just opened frames itself first and moves after that.
- The account menu has "Follow agent" below "Agent", on by default and
  kept in the browser. While the game plays the editor never follows.
- The run_commands description and the relay's instructions tell the
  agent about it.

Tests:
- agent-edit.test.ts: 3 new tests, each red with its bent-back fix
  (scene switch outside the undo step, deleted nodes selected, switch
  ignored)
- graph-framing.test.ts 29/29, 6 of them new
- e2e 71 on melektaus-atx: ALL PASSED, 20 checks. 71g4 adds a node
  3000 px to the right of an open tab: it ends 24 px inside the edge,
  zoom still 1
- Tried live on a test project through a real MCP session: the tab
  opened, the nodes and the object were selected. With the old build
  a far node stayed out of view, after the reload the camera went there
- Gate on melektaus-atx: CHECK EXIT 0, 318 files, 4931 tests

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-18 12:48:35 +02:00
1 parent 732989a452
commit 97e0d93168
13 files changed
+383 -14

No files matched your search

+13
View File
@@ -191,6 +191,17 @@ const FIXED_TABS = ['scene', 'game'] as const;
/** `sessionStorage` key: the agent was on in this browser tab (#429). */
const AGENT_KEY = 'feuerwerk.agent';
/** `localStorage` key: the user switched "Follow agent" off; the editor follows the agent unless it is set. */
const FOLLOW_OFF_KEY = 'feuerwerk.agent-follow-off';
function followsAgent(): boolean {
return localStorage.getItem(FOLLOW_OFF_KEY) !== '1';
}
function setFollowsAgent(on: boolean): void {
if (on) localStorage.removeItem(FOLLOW_OFF_KEY);
else localStorage.setItem(FOLLOW_OFF_KEY, '1');
}
/** A dialog's bold first line, or a menu's first entry: what tells one popup from another. */
function popupTitle(popup: Popup): string {
@@ -939,6 +950,7 @@ export class Editor implements PlayHost {
return [
{ separator: true },
{ label: 'Agent', checked: () => link.on, onSelect: () => this.switchAgent(!link.on) },
{ label: 'Follow agent', checked: followsAgent, onSelect: () => setFollowsAgent(!followsAgent()) },
{ label: 'Agent tokens…', onSelect: () => void this.openAgentTokens(agents) },
];
}
@@ -962,6 +974,7 @@ export class Editor implements PlayHost {
view: () => this.agentView(),
showTab: (tab) => this.showTab(tab),
play: this.play,
follows: followsAgent,
},
wait,
),
+8 -1
View File
@@ -5,6 +5,7 @@ import { layoutOf, type MeasureNode } from '../graph/graphLayout';
import { type ArgSchema, argProblem } from './argCheck';
import type { AgentView } from './agentView';
import { AgentEdit, deletedAssets, type EditStep } from './editPlan';
import type { StepTarget } from './commandCatalog';
import { catalog, type CatalogTopic, getAsset, getGraph, listAssets } from './readTools';
import TOOLS from './tools.json';
@@ -45,6 +46,10 @@ export interface ToolHost {
redo(): string | null;
/** How the nodes of a graph are measured: drawn off screen in the editor, worked out without one. */
measure?(assetId: string): MeasureNode;
/** Whether the editor follows the agent: shows what each edit worked on (the user can switch it off). */
follows?(): boolean;
/** Brings up what an edit worked on, with its objects or nodes selected. */
show?(focus: StepTarget): void;
}
const DEFAULT_SCREENSHOT_WIDTH = 1280;
@@ -156,9 +161,11 @@ async function runCommands(args: Record<string, unknown>, host: ToolHost): Promi
if (refusal !== null) return failure(refusal);
}
const label = `Agent: ${(args.label as string | undefined) ?? steps.map((step) => step.command).join(', ')}`;
const edit = new AgentEdit(label, steps, host.measure && ((assetId) => host.measure!(assetId)));
const follows = host.follows?.() === true;
const edit = new AgentEdit(label, steps, host.measure && ((assetId) => host.measure!(assetId)), follows);
const refusal = host.execute(edit);
if (refusal !== null) return failure(`Nothing was changed. ${refusal}`);
if (follows && edit.focus) host.show?.(edit.focus);
const made = edit.made.flatMap((ids, index) => (ids ? [{ step: index + 1, ...ids }] : []));
const playing = host.view().play !== 'editing';
const lines = [
+55
View File
@@ -59,6 +59,12 @@ export interface Built {
type Args = Record<string, unknown>;
/** What a step worked on, for the editor to show the user: an asset, objects of a holder, nodes of a graph. */
export type StepTarget =
| { kind: 'asset'; asset: string }
| { kind: 'objects'; in: string; ids: string[] }
| { kind: 'nodes'; asset: string; ids: string[] };
/** One command the agent can run (#429): what it does, its arguments, and how it is made. */
export interface CatalogCommand {
name: string;
@@ -67,6 +73,8 @@ export interface CatalogCommand {
/** Deletes something the user has to agree to first – decision 4 of #429. */
confirms?: true;
build(args: Args, context: BuildContext): Built;
/** What it worked on, from its arguments and what it made; none for a step there is nothing to show of. */
shows?(args: Args, made: Record<string, string>): StepTarget;
}
/** Assets the agent cannot make: each needs a file, and there is no upload for agents. */
@@ -97,6 +105,23 @@ const PORT_END = object({ node: NODE, port: id('Key of the port, as get_graph li
const text = (args: Args, key: string): string => args[key] as string;
const numberOf = (args: Args, key: string): number => args[key] as number;
const theAsset = (args: Args): StepTarget => ({ kind: 'asset', asset: text(args, 'asset') });
const madeAsset = (_args: Args, made: Record<string, string>): StepTarget => ({
kind: 'asset',
asset: made.id!,
});
const theObject = (args: Args): StepTarget => ({
kind: 'objects',
in: text(args, 'in'),
ids: [text(args, 'object')],
});
const theNode = (args: Args): StepTarget => ({
kind: 'nodes',
asset: text(args, 'asset'),
ids: [text(args, 'node')],
});
const theGraph = (args: Args): StepTarget => ({ kind: 'nodes', asset: text(args, 'asset'), ids: [] });
export const COMMANDS: readonly CatalogCommand[] = [
{
name: 'create_asset',
@@ -126,12 +151,14 @@ export const COMMANDS: readonly CatalogCommand[] = [
);
return { command, made: { id: command.asset.id } };
},
shows: madeAsset,
},
{
name: 'rename_asset',
description: 'Renames an asset.',
args: object({ asset: ASSET, name: { type: 'string', minLength: 1 } }, ['asset', 'name']),
build: (args) => ({ command: new RenameAssetCommand(text(args, 'asset'), text(args, 'name')) }),
shows: theAsset,
},
{
name: 'duplicate_asset',
@@ -141,6 +168,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
const command = new DuplicateAssetCommand(text(args, 'asset'));
return { command, made: { id: command.copyId } };
},
shows: madeAsset,
},
{
name: 'delete_asset',
@@ -165,12 +193,14 @@ export const COMMANDS: readonly CatalogCommand[] = [
args.value as PropertyValue,
),
}),
shows: theAsset,
},
{
name: 'reset_asset_property',
description: 'Puts a property of an asset back to what a new asset starts with.',
args: object({ asset: ASSET, key: id('Key of the property.') }, ['asset', 'key']),
build: (args) => ({ command: new ResetAssetPropertyCommand(text(args, 'asset'), text(args, 'key')) }),
shows: theAsset,
},
{
name: 'set_start_scene',
@@ -196,12 +226,14 @@ export const COMMANDS: readonly CatalogCommand[] = [
text(args, 'kind') as VariableKind,
),
}),
shows: theAsset,
},
{
name: 'remove_variable',
description: 'Removes a variable, its value and every override of it on placed objects.',
args: object({ asset: ASSET, key: id('Name of the variable.') }, ['asset', 'key']),
build: (args) => ({ command: new RemoveVariableCommand(text(args, 'asset'), text(args, 'key')) }),
shows: theAsset,
},
{
name: 'rename_variable',
@@ -214,6 +246,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
build: (args) => ({
command: new RenameVariableCommand(text(args, 'asset'), text(args, 'from'), text(args, 'to')),
}),
shows: theAsset,
},
{
name: 'place_asset',
@@ -242,12 +275,14 @@ export const COMMANDS: readonly CatalogCommand[] = [
);
return { command, made: { id: command.objectId } };
},
shows: (args, made) => ({ kind: 'objects', in: text(args, 'in'), ids: [made.id!] }),
},
{
name: 'remove_objects',
description: 'Removes placed objects together with everything placed under them.',
args: object({ in: HOLDER, objects: { type: 'array', items: OBJECT, minItems: 1 } }, ['in', 'objects']),
build: (args) => ({ command: new RemoveSceneObjectsCommand(text(args, 'in'), args.objects as string[]) }),
shows: (args) => ({ kind: 'objects', in: text(args, 'in'), ids: [] }),
},
{
name: 'move_object',
@@ -269,6 +304,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
}),
};
},
shows: theObject,
},
{
name: 'rename_object',
@@ -277,6 +313,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
build: (args) => ({
command: new RenameSceneObjectCommand(text(args, 'in'), text(args, 'object'), text(args, 'name')),
}),
shows: theObject,
},
{
name: 'set_override',
@@ -295,6 +332,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
args.value as PropertyValue,
),
}),
shows: theObject,
},
{
name: 'clear_override',
@@ -303,6 +341,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
build: (args) => ({
command: new ClearOverrideCommand(text(args, 'in'), text(args, 'object'), text(args, 'key')),
}),
shows: theObject,
},
{
name: 'set_parent',
@@ -312,6 +351,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
build: (args) => ({
command: new SetParentCommand(text(args, 'in'), text(args, 'object'), text(args, 'parent')),
}),
shows: theObject,
},
{
name: 'add_node',
@@ -342,6 +382,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
? { command, made }
: { command, made, unplaced: { asset: assetId, nodes: Object.values(made) } };
},
shows: (args, made) => ({ kind: 'nodes', asset: text(args, 'asset'), ids: Object.values(made) }),
},
{
name: 'arrange_graph',
@@ -371,12 +412,18 @@ export const COMMANDS: readonly CatalogCommand[] = [
const command = arrangeCommand(context.project, assetId, nodes, context.measure(assetId));
return { command: command ?? new CompoundCommand('Arrange nodes', []) };
},
shows: (args) => ({
kind: 'nodes',
asset: text(args, 'asset'),
ids: (args.nodes as string[] | undefined) ?? [],
}),
},
{
name: 'remove_node',
description: 'Removes a node and every edge on it.',
args: object({ asset: GRAPH, node: NODE }, ['asset', 'node']),
build: (args) => ({ command: new RemoveNodeCommand(text(args, 'asset'), text(args, 'node')) }),
shows: theGraph,
},
{
name: 'move_node',
@@ -390,6 +437,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
numberOf(args, 'y'),
),
}),
shows: theNode,
},
{
name: 'set_node_param',
@@ -408,6 +456,7 @@ export const COMMANDS: readonly CatalogCommand[] = [
args.value as PropertyValue,
),
}),
shows: theNode,
},
{
name: 'connect',
@@ -420,12 +469,18 @@ export const COMMANDS: readonly CatalogCommand[] = [
const command = new ConnectCommand(text(args, 'asset'), end('from'), end('to'));
return { command, made: { id: command.edge.id } };
},
shows: (args) => ({
kind: 'nodes',
asset: text(args, 'asset'),
ids: (['from', 'to'] as const).map((key) => (args[key] as { node: string }).node),
}),
},
{
name: 'disconnect',
description: 'Removes an edge.',
args: object({ asset: GRAPH, edge: id('Id of the edge.') }, ['asset', 'edge']),
build: (args) => ({ command: new DisconnectCommand(text(args, 'asset'), text(args, 'edge')) }),
shows: theGraph,
},
];
+48 -6
View File
@@ -1,9 +1,10 @@
import { SetActiveSceneCommand } from '../../core/commands/assetCommands';
import { type Command, CommandError } from '../../core/commands/Command';
import { findGraphAsset, type Project } from '../../core/model/Project';
import { findAsset, findGraphAsset, findObjectHolder, type Project } from '../../core/model/Project';
import { arrangeCommand, type MeasureNode } from '../graph/graphLayout';
import { estimatedShape } from '../graph/nodeBox';
import { argProblem } from './argCheck';
import { type BuildContext, type Built, catalogCommand } from './commandCatalog';
import { type BuildContext, type Built, catalogCommand, type StepTarget } from './commandCatalog';
/** One step of an edit as the agent sends it. */
export interface EditStep {
@@ -28,12 +29,19 @@ const REF = /^\$([A-Za-z_][\w-]*)(?:\.(\w+))?$/;
export class AgentEdit implements Command {
/** What each step made, by the names of `Built.made`; filled by the first run. */
readonly made: (Record<string, string> | null)[] = [];
/** What the edit worked on last, for the editor to show; filled by the first run. */
focus: StepTarget | null = null;
private built: Command[] | null = null;
/**
* `showScene`: the editor follows the agent, so an edit that ends on objects of another scene
* switches to it – as part of the edit, so one undo takes the switch back with it.
*/
constructor(
readonly label: string,
private readonly steps: readonly EditStep[],
private readonly measure: (assetId: string) => MeasureNode = () => estimatedShape,
private readonly showScene = false,
) {}
execute(project: Project): void {
@@ -42,6 +50,7 @@ export class AgentEdit implements Command {
return;
}
const done: Command[] = [];
const targets: StepTarget[] = [];
const names = new Map<string, Record<string, string>>();
const unplaced = new Map<string, string[]>();
const context: BuildContext = {
@@ -53,10 +62,11 @@ export class AgentEdit implements Command {
for (const [index, step] of this.steps.entries()) {
const where = `Step ${index + 1} (${step.command})`;
try {
const built = buildStep(step, names, context);
const { built, target } = buildStep(step, names, context);
built.command.execute(project);
done.push(built.command);
this.made.push(built.made ?? null);
if (target) targets.push(target);
if (built.unplaced) {
const { asset, nodes } = built.unplaced;
unplaced.set(asset, [...(unplaced.get(asset) ?? []), ...nodes]);
@@ -77,9 +87,17 @@ export class AgentEdit implements Command {
arranged.execute(project);
done.push(arranged);
}
this.focus = focusOf(project, targets);
const scene = this.focus?.kind === 'objects' ? findAsset(project, this.focus.in) : undefined;
if (this.showScene && scene?.type === 'scene' && project.activeSceneId !== scene.id) {
const show = new SetActiveSceneCommand(scene.id);
show.execute(project);
done.push(show);
}
} catch (error) {
for (const command of done.reverse()) command.undo(project);
this.made.length = 0;
this.focus = null;
throw error;
}
this.built = done;
@@ -103,13 +121,37 @@ function buildStep(
step: EditStep,
names: ReadonlyMap<string, Record<string, string>>,
context: BuildContext,
): Built {
): { built: Built; target: StepTarget | null } {
const entry = catalogCommand(step.command);
if (!entry) throw new CommandError('no such command – catalog topic "commands" lists them');
const args = resolve(step.args ?? {}, names);
const args = resolve(step.args ?? {}, names) as Record<string, unknown>;
const problem = argProblem(entry.args, args, 'args');
if (problem) throw new CommandError(problem);
return entry.build(args as Record<string, unknown>, context);
const built = entry.build(args, context);
return { built, target: entry.shows?.(args, built.made ?? {}) ?? null };
}
/**
* What an edit is shown by: what its last step with something to show worked on, together with all
* the objects or nodes the edit touched there – a graph built in one call comes up with every node
* it made selected. What the edit removed again is left out; a target gone altogether shows nothing.
*/
export function focusOf(project: Project, targets: readonly StepTarget[]): StepTarget | null {
const last = targets.at(-1);
if (!last) return null;
if (last.kind === 'asset') return findAsset(project, last.asset) ? last : null;
const holderId = last.kind === 'objects' ? last.in : last.asset;
const present =
last.kind === 'objects'
? findObjectHolder(project, holderId)?.objects.map((object) => object.id)
: findGraphAsset(project, holderId)?.graph.nodes.map((node) => node.id);
if (!present) return null;
const touched = targets.flatMap((target) =>
target.kind === last.kind && (target.kind === 'objects' ? target.in : target.asset) === holderId
? target.ids
: [],
);
return { ...last, ids: [...new Set(touched)].filter((id) => present.includes(id)) };
}
function runAll(project: Project, commands: readonly Command[]): void {
+28
View File
@@ -6,6 +6,7 @@ import { measuredNodes } from '../graph/measureNodes';
import { type EditorState, nodeSelection, objectSelection } from '../state';
import type { AgentSelection, PlayAction, ToolHost } from './agentTools';
import type { AgentView } from './agentView';
import type { StepTarget } from './commandCatalog';
import { screenshot } from './screenshot';
/** What the agent's tools reach of the editor beyond its state. */
@@ -15,6 +16,8 @@ export interface EditorParts {
view(): AgentView;
showTab(tab: 'scene' | 'game'): void;
play: { readonly isPlaying: boolean; play(paused: boolean): void; stop(): void; togglePause(): void };
/** The user's "Follow agent" switch. */
follows(): boolean;
}
/** How long the user has to answer an agent's question before it counts as a no. */
@@ -57,9 +60,34 @@ export function editorToolHost(parts: EditorParts, wait: () => void): ToolHost {
return label;
},
measure: (assetId) => measuredNodes(state, assetId),
// A running game keeps the user's eyes: an edit then goes to the running copy anyway.
follows: () => parts.follows() && !parts.play.isPlaying,
show: (focus) => show(parts, focus),
};
}
/** Brings up what an edit worked on: a graph's tab, the scene or container, the asset in the inspector. */
function show(parts: EditorParts, focus: StepTarget): void {
const { state } = parts;
if (focus.kind === 'asset') {
state.select({ kind: 'asset', assetId: focus.asset });
return;
}
const holder = focus.kind === 'objects' ? focus.in : focus.asset;
if (focus.ids.length === 0) {
state.openAsset(holder);
return;
}
if (focus.kind === 'nodes') {
state.select(nodeSelection(holder, focus.ids, focus.ids[0], [], []));
// A tab that was open already keeps its camera; what was added off to one side comes into it.
state.revealNodes(holder, focus.ids);
return;
}
if (findAsset(state.project, holder)?.type === 'scene') parts.showTab('scene');
state.select(objectSelection(holder, focus.ids));
}
function select(state: EditorState, selection: AgentSelection): string | null {
switch (selection.kind) {
case 'none':
+2 -2
View File
@@ -100,7 +100,7 @@
{
"name": "run_commands",
"title": "Change the project",
"description": "Runs commands (catalog topic \"commands\") in order as ONE undo step: if one fails, none of them stays, and the answer names the step. A step with \"as\": \"name\" lets later steps use what it made: \"$name\" is its id, \"$name.closing\" another id it names. Each step is built after the one before has run, so a step can add nodes to a behavior made in the same call. Deleting an asset asks the user first.",
"description": "Runs commands (catalog topic \"commands\") in order as ONE undo step: if one fails, none of them stays, and the answer names the step. A step with \"as\": \"name\" lets later steps use what it made: \"$name\" is its id, \"$name.closing\" another id it names. Each step is built after the one before has run, so a step can add nodes to a behavior made in the same call. Deleting an asset asks the user first. Unless the user switched it off, the editor then shows what the call worked on last: the graph's tab with the nodes it touched selected, the scene (switching to it within the same undo step) or container with its objects selected, or the asset in the inspector.",
"inputSchema": {
"type": "object",
"properties": {
@@ -137,7 +137,7 @@
{
"name": "open",
"title": "Show something",
"description": "Brings up an asset's tab (a scene becomes the scene being edited), or the scene or game tab.",
"description": "Brings up an asset's tab (a scene becomes the scene being edited), or the scene or game tab. run_commands shows what it changed by itself; this is for anything else the user should see.",
"inputSchema": {
"type": "object",
"properties": {
+34
View File
@@ -170,3 +170,37 @@ export function frameBoxes(
panY: view.h / 2 - ((minY + maxY) / 2) * zoom,
};
}
/**
* The view that brings every box into the picture with the least change (#429): the same view
* when they are all in it, else moved by just as much as it takes, keeping the zoom. Only boxes
* too large for the picture at this zoom are framed afresh, which zooms out. What the user had
* in sight stays in sight wherever it can.
*/
export function revealBoxes(
boxes: readonly Box[],
current: Framing,
view: { w: number; h: number },
margin: number,
limits: { min: number; max: number },
): Framing {
const bounds = boundsOf(boxes);
if (!bounds) return current;
const { zoom } = current;
const room = { w: view.w - margin * 2, h: view.h - margin * 2 };
if ((bounds.maxX - bounds.minX) * zoom > room.w || (bounds.maxY - bounds.minY) * zoom > room.h)
return frameBoxes(boxes, view, margin, limits);
const x = shiftInto(bounds.minX * zoom + current.panX, bounds.maxX * zoom + current.panX, view.w, margin);
const y = shiftInto(bounds.minY * zoom + current.panY, bounds.maxY * zoom + current.panY, view.h, margin);
return { zoom, panX: current.panX + x, panY: current.panY + y };
}
/**
* How far a span on screen has to move into `[0, size]`: not at all when it lies there already, even
* against an edge, else until it is `margin` clear of the edge it crossed.
*/
function shiftInto(start: number, end: number, size: number, margin: number): number {
if (start >= 0 && end <= size) return 0;
if (start < 0) return margin - start;
return size - margin - end;
}
+33 -1
View File
@@ -67,7 +67,7 @@ import {
nodeSelection,
} from '../state';
import { canPasteGraph, copySelection, pasteGraph } from '../clipboard';
import { type Box, boundsOf, boxesInBand, boxesWithinBand, frameBoxes } from '../framing';
import { type Box, boundsOf, boxesInBand, boxesWithinBand, frameBoxes, revealBoxes } from '../framing';
import { annotationBoxes, boxOf } from './annotationDrag';
import { AnnotationLayer } from './annotationLayer';
import { ANNOTATION_TITLE_HEIGHT, type AnnotationView, RESIZE_HANDLE } from './AnnotationView';
@@ -152,6 +152,8 @@ export class GraphCanvas extends Widget {
private incoming: { assetId: string; at: { x: number; y: number }; rejection: string } | null = null;
/** Fit the view once the editor has a real size (it may be hidden behind the Scene tab). */
private needsFit = true;
/** Nodes to bring into the picture once the fit is done (#429). */
private pendingReveal: readonly string[] | null = null;
private valuesShownAt = 0;
private readonly beads = new BeadClock();
/** Last place the pointer was seen, in global coordinates – where a paste lands. */
@@ -240,6 +242,7 @@ export class GraphCanvas extends Widget {
state.on('selection', () => this.redraw());
state.on('project', () => this.sync());
state.on('mode', () => this.redraw());
state.on('reveal', () => this.reveal());
}
get currentAssetId(): string {
@@ -252,11 +255,39 @@ export class GraphCanvas extends Widget {
// Its source node is in the graph being left, so the click that finished it would land nowhere.
this.wiring.cancel();
this.assetId = assetId;
this.pendingReveal = null;
this.sync();
this.needsFit = true;
this.fitView();
}
/** Brings the nodes `revealNodes` asked for into the picture, if they are this graph's. */
private reveal(): void {
const asked = this.state.revealedNodes;
if (!asked || asked.assetId !== this.assetId) return;
this.pendingReveal = asked.nodeIds;
// A tab that has only just opened frames itself first, once it knows its size.
if (this.w > 0 && this.h > 0 && !this.needsFit) this.revealPending();
}
private revealPending(): void {
const ids = new Set(this.pendingReveal);
this.pendingReveal = null;
const boxes = this.nodeBoxes().filter((box) => ids.has(box.id));
this.tween.change(() => {
const framing = revealBoxes(
boxes,
{ zoom: this.zoom, panX: this.pan.x, panY: this.pan.y },
{ w: this.w, h: this.h },
FIT_MARGIN,
ZOOM_LIMITS,
);
this.zoom = framing.zoom;
this.pan = { x: framing.panX, y: framing.panY };
this.applyView();
});
}
fit(): void {
this.tween.change(() => {
this.needsFit = true;
@@ -1012,6 +1043,7 @@ export class GraphCanvas extends Widget {
override layout(): void {
this.hint.position.set(Theme.padding, this.h - Theme.padding - 16);
if (this.needsFit) this.fitView();
if (this.pendingReveal && !this.needsFit) this.revealPending();
}
override redraw(): void {
+14 -1
View File
@@ -248,7 +248,9 @@ export type EditorEvent =
/** The eyedropper of an object's place started, moved its candidate or ended (`objectPick`). */
| 'pick'
/** The running world advanced a frame (play mode): values may have changed without a command. */
| 'runtime';
| 'runtime'
/** Nodes should come into the picture of the graph showing them (`revealedNodes`). */
| 'reveal';
type Listener = () => void;
@@ -268,6 +270,8 @@ export class EditorState {
openedValues: ValuesTarget | null = null;
/** The last node asked for by `openNodeTab`. */
openedNode: { assetId: string; nodeId: string; type: string } | null = null;
/** The last nodes asked for by `revealNodes`. */
revealedNodes: { assetId: string; nodeIds: readonly string[] } | null = null;
mode: EditorMode = 'edit';
/**
* A sub-state of play rather than a third mode: the runtime copy, the keyboard and unsaved edits all
@@ -581,6 +585,15 @@ export class EditorState {
this.emit('open');
}
/**
* Asks the graph showing `assetId` to bring these nodes into its picture (#429): what the agent
* just added far to one side is otherwise selected out of sight.
*/
revealNodes(assetId: string, nodeIds: readonly string[]): void {
this.revealedNodes = { assetId, nodeIds };
this.emit('reveal');
}
/**
* Shows where an asset or a clip of one (#425) is listed without selecting it – what one click on a
* link in the inspector does (#221), so the inspector stays on the object the link belongs to.
+63 -3
View File
@@ -4,7 +4,7 @@ import { findGraphAsset } from '../../src/core/model/Project';
import { argProblem } from '../../src/editor/agent/argCheck';
import { runTool, type ToolHost, type ToolResult } from '../../src/editor/agent/agentTools';
import type { AgentView } from '../../src/editor/agent/agentView';
import { COMMANDS } from '../../src/editor/agent/commandCatalog';
import { COMMANDS, type StepTarget } from '../../src/editor/agent/commandCatalog';
import { NODE_STEP } from '../../src/editor/agent/editPlan';
import { COLUMN_GAP, type GraphLint } from '../../src/editor/graph/arrange';
import { NODE_WIDTH } from '../../src/editor/graph/nodeBox';
@@ -13,8 +13,9 @@ import { makeBehavior, makeSpriteScene, type SceneFixture } from '../helpers';
/** What an agent changes and reads through the tools (#429, stage 2). */
function hostFor(fixture: SceneFixture, deleteAnswer: string | null = null) {
function hostFor(fixture: SceneFixture, deleteAnswer: string | null = null, follows = false) {
const asked: string[][] = [];
const shown: StepTarget[] = [];
const host: ToolHost = {
view: () => ({ play: 'editing' }) as AgentView,
screenshot: () => Promise.resolve({ data: '', width: 0, height: 0 }),
@@ -45,8 +46,10 @@ function hostFor(fixture: SceneFixture, deleteAnswer: string | null = null) {
if (label !== null) fixture.stack.redo();
return label;
},
follows: () => follows,
show: (focus) => shown.push(focus),
};
return { host, asked };
return { host, asked, shown };
}
const text = (result: ToolResult) =>
@@ -332,6 +335,63 @@ describe('run_commands', () => {
});
});
describe('the editor following the agent', () => {
it('shows the graph a call built, with every node it touched selected and the removed ones left out', async () => {
const fixture = makeSpriteScene();
const { host, shown } = hostFor(fixture, null, true);
await runTool('run_commands', { steps: walkLeft(fixture) }, host);
const walk = fixture.project.assets.find((asset) => asset.name === 'Walk')!;
const nodes = findGraphAsset(fixture.project, walk.id)!.graph.nodes.map((node) => node.id);
expect(shown).toEqual([{ kind: 'nodes', asset: walk.id, ids: nodes }]);
const steps = [
{ command: 'add_node', args: { asset: walk.id, node: 'OnStart' }, as: 'start' },
{ command: 'set_node_param', args: { asset: walk.id, node: nodes[1], key: 'dx', value: -100 } },
{ command: 'remove_node', args: { asset: walk.id, node: '$start' } },
];
await runTool('run_commands', { steps }, host);
expect(shown[1]).toEqual({ kind: 'nodes', asset: walk.id, ids: [nodes[1]] });
});
it('switches to the scene of the objects a call ends on, inside the same undo step', async () => {
const fixture = makeSpriteScene();
const { host, shown } = hostFor(fixture, null, true);
const made = await runTool(
'run_commands',
{ steps: [{ command: 'create_asset', args: { type: 'scene', name: 'Cave' } }] },
host,
);
const cave = (made.structuredContent as { made: { id: string }[] }).made[0]!.id;
expect(shown).toEqual([{ kind: 'asset', asset: cave }]);
expect(fixture.project.activeSceneId).toBe(fixture.sceneId);
const place = {
command: 'place_asset',
args: { in: cave, asset: fixture.spriteId, x: 0, y: 0 },
as: 'bat',
};
const placed = await runTool('run_commands', { steps: [place], label: 'bat' }, host);
const bat = (placed.structuredContent as { made: { id: string }[] }).made[0]!.id;
expect(shown[1]).toEqual({ kind: 'objects', in: cave, ids: [bat] });
expect(fixture.project.activeSceneId).toBe(cave);
expect(text(await runTool('undo', {}, host))).toBe('Undone: Agent: bat.');
expect(fixture.project.activeSceneId).toBe(fixture.sceneId);
});
it('leaves the view and the scene alone when the user switched following off', async () => {
const fixture = makeSpriteScene();
const { host, shown } = hostFor(fixture);
const steps = [
{ command: 'create_asset', args: { type: 'scene', name: 'Cave' }, as: 'cave' },
{ command: 'place_asset', args: { in: '$cave', asset: fixture.spriteId, x: 0, y: 0 } },
];
expect((await runTool('run_commands', { steps }, host)).isError).toBeFalsy();
expect(shown).toEqual([]);
expect(fixture.project.activeSceneId).toBe(fixture.sceneId);
});
});
describe('reading', () => {
it('get_graph lists each node’s ports as they are, and the edges', async () => {
const fixture = makeSpriteScene();
+41
View File
@@ -6,6 +6,7 @@ import {
dragOffset,
frameBoxes,
resizeSize,
revealBoxes,
} from '../../src/editor/framing';
const LIMITS = { min: 0.25, max: 2.5 };
@@ -155,6 +156,46 @@ describe('fit', () => {
});
});
describe('reveal (#429)', () => {
const view = { w: 400, h: 300 };
const at = { zoom: 1, panX: 0, panY: 0 };
it('leaves the view alone when the nodes are in it already', () => {
expect(revealBoxes(boxes, at, view, 24, LIMITS)).toEqual(at);
});
it('moves just far enough to bring a node off to the right in, keeping the zoom', () => {
// Right edge at 700, the picture ends at 400 - 24: moved left by 324, not centred.
const f = revealBoxes([{ id: 'c', x: 600, y: 100, w: 100, h: 60 }], at, view, 24, LIMITS);
expect(f).toEqual({ zoom: 1, panX: -324, panY: 0 });
});
it('moves up and left alike', () => {
const f = revealBoxes([{ id: 'c', x: -200, y: -100, w: 100, h: 60 }], at, view, 24, LIMITS);
expect(f).toEqual({ zoom: 1, panX: 224, panY: 124 });
});
it('counts the zoom: what fits at 1 can stick out at 2', () => {
const f = revealBoxes(boxes, { zoom: 2, panX: 0, panY: 0 }, { w: 800, h: 600 }, 24, LIMITS);
// At zoom 2 the two nodes end at 600 on screen and are 240 tall: inside, untouched.
expect(f).toEqual({ zoom: 2, panX: 0, panY: 0 });
const g = revealBoxes(boxes, { zoom: 2, panX: 0, panY: 0 }, view, 24, LIMITS);
expect(g.zoom).toBeLessThan(2);
});
it('zooms out to frame them only when they are too large for the picture', () => {
const wide: Box[] = [
{ id: 'a', x: 0, y: 0, w: 100, h: 60 },
{ id: 'b', x: 900, y: 0, w: 100, h: 60 },
];
expect(revealBoxes(wide, at, view, 24, LIMITS)).toEqual(frameBoxes(wide, view, 24, LIMITS));
});
it('keeps the view for no nodes at all', () => {
expect(revealBoxes([], at, view, 24, LIMITS)).toEqual(at);
});
});
describe('boundsOf', () => {
it('encloses every box', () => {
const boxes: Box[] = [
+42
View File
@@ -261,6 +261,18 @@ export async function mcp(t) {
JSON.stringify({ built: built?.content?.[0]?.text?.slice(0, 300), graph: graph?.edges?.length }),
);
// 71g1. The editor follows the agent: the user sees the graph it built, its nodes selected.
await settle();
const followed = await viewNow();
check(
'the editor follows the agent: the behavior it built is the open tab, with its three nodes selected',
followed?.document?.asset?.id === walkId &&
followed.selection?.kind === 'node' &&
followed.selection.in?.id === walkId &&
followed.selection.nodes?.length === 3,
JSON.stringify({ document: followed?.document, selection: followed?.selection }),
);
// 71g2. The nodes the agent gave no place come out arranged, as the drawn nodes measure.
const lint = walkId ? (await callTool('lint_graph', { asset: walkId }))?.structuredContent : null;
check(
@@ -302,6 +314,36 @@ export async function mcp(t) {
JSON.stringify({ tangled, tidied, arrangedLabel, backTo }),
);
// 71g4. In a graph tab that is open already, the camera goes after a node added far to one side,
// keeping its zoom: the user sees it rather than a selection out of sight.
const zoomBefore = await json('__gm.graph.zoom()');
const far = await callTool('run_commands', {
label: 'far node',
steps: [
{ command: 'add_node', args: { asset: walkId, node: 'Owner', x: keyNode.x + 3000, y: keyNode.y } },
],
});
const farId = far?.structuredContent?.made?.[0]?.id;
const inSight = `(() => {
const node = __gm.graph.nodeRect(${JSON.stringify(farId)});
const view = __gm.graph.canvasRect();
return Boolean(node) && node.x >= view.x && node.x + node.w <= view.x + view.w &&
node.y >= view.y && node.y + node.h <= view.y + view.h;
})()`;
const revealed = await waitUntil(inSight, { timeout: 3000, label: 'the far node in sight' }).then(
() => true,
() => false,
);
const farView = await json(
`({ node: __gm.graph.nodeRect(${JSON.stringify(farId)}), view: __gm.graph.canvasRect(), zoom: __gm.graph.zoom() })`,
);
await callTool('undo');
check(
'the open graph tab moves its camera to a node the agent added far away, at the same zoom',
revealed && farView.zoom === zoomBefore,
JSON.stringify({ zoomBefore, ...farView }),
);
// 71h. Played by the agent, the object walks left while the user holds the arrow.
const started = await callTool('play', { action: 'play' });
await waitUntil('__gm.state.mode === "play"', { timeout: 10000, label: 'the game runs' });
+2
View File
@@ -41,6 +41,8 @@ const INSTRUCTIONS = [
'Give new nodes no x and y: the editor arranges them when the call ends, and leaves the nodes',
'already there where they are. Keep a graph readable: a value needed far away comes from a second',
'Get or Owner node next to its user, not over a long link; check with lint_graph.',
'The editor follows your edits: after run_commands the user sees the graph, scene or asset it worked',
'on, with what it touched selected. Use open and select to show anything else.',
'When a tool answers that no editor is connected, ask the user to open the editor and switch on',
'"Agent" in its account menu.',
].join(' ');