Files
feuerwerk/src/core/graph/NodeContext.ts
T
adminandClaude Opus 5.5 8a4e4a6762
game-maker / suite (push) Failing after 2m40s
game-maker / gate (push) Successful in 3m14s
Play sounds as notes of an instrument asset #331
A sound could only be played as it was, one at a time. An instrument
is a node graph that makes a note a sound: Notes nodes bring the note
in, Samples play sound assets for it, the effects of a sound FX shape
it and the Output sums the layers. Its tab plays it from a keyboard,
a behavior with Play note. It is what patterns and songs (#330) will
play.

- New asset type instrument, under the heading Music, not placeable.
  A new one starts as Notes into a Sample into the Output.
- Notes are counted as MIDI does: 60 is C4, names like C#4, sharps
  only, the lowest C-1. Notes has From and To, outputs any, note,
  frequency and velocity, and with Single notes a trigger per note,
  each with an optional name for a drum kit.
- New port type trigger, only into the trigger of a source. Inputs
  can take any number of edges now (PortDef.many): a trigger, and
  the Output, which plays everything into it side by side.
- Sample plays a sound asset. Follow note shifts it by note - root
  through the pitch worklet of #317, so it keeps its length. Gate
  fades it out at the end of the note, One-shot plays through. Loop
  runs between Loop start and Loop end, also in the worklet.
- Parameters declares the instrument's own parameters with a
  default. Sound FX sends the sound through a sound FX asset, its
  parameters as inputs, the sound set in where its Input was.
- compileInstrument walks the graph once per note. A trigger is on
  only for the notes it stands for, a silent source silences what it
  alone feeds, a Mix with a silent side is the other side. velocity
  and the parameters stay open for the voice. Builds are cached per
  note until the graph or a sound or FX it names changes.
- The player takes a note as one voice of several layers, each way
  with its own file and volume. release fades the gated ways out,
  one-shots play on. playNote can release after a duration.
- Play note plays one note with velocity, volume, duration and
  detached, and hands out voice for Set sound FX parameters and
  Stop sound. Velocity is always loudness. Instrument parameters
  picks values for the instrument's parameters like Sound FX
  parameters does for an FX, renames follow.
- The tab has the graph, a header field per parameter, Stop, and
  128 keys below. Keys without sound are grey. A key plays while it
  is held. Focused, the computer keyboard plays as in a tracker by
  KeyboardEvent.code, the arrows move the octave.

Findings:
- A label of the keys set fill on the shared small text style and
  darkened every small label of the editor. Caught in a screenshot
  before the commit, the keys have a style of their own now.
- Suite task 523 on cf6799a failed two checks of e2e 34 that timed
  audio by the page's clock: a 2.95 s echo tail was gone after 1 s,
  and Quad in was further along than 0.7 s allow. Both time on the
  audio clock now through a new hook __gm.sound.clock().

Tests:
- instrument.test.ts (new, 22): notes, asset, palette, round trip,
  compile per note, drum kit, zones, Mix, velocity and parameters,
  Sound FX node with rename, problems, cache, Play note, the keys
- sound-library +7 (layers, release, duration, early release, loop
  region in buffer and worklet), pitch-player +2 (loop region),
  node-catalog +4, graph adjusted. 582 tests in the touched files.
- e2e 4 and 34 on melektaus-atx: ALL PASSED, 34 with 44 checks (+7)
- Gate on melektaus-atx: CHECK EXIT 0, 243 test files, 3750 tests
- Counter-checks: any always on, release ending one-shots, Sound FX
  inputs ignored, loop region ignored. 3, 2, 1 and 1 tests failed.
- Browser probe on stein2: keys, held key, Single notes names and
  the header looked at in screenshots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-06 10:07:40 +02:00

331 lines
14 KiB
TypeScript
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import type { AnimationPlayer, EaseShare } from '../model/animation';
import type { Mat } from '../model/matrix';
import type { Position, PropertyBag } from '../model/Properties';
import type { FxParams } from '../soundfx/nodes';
import { type ChainTransform, composeAlongChain } from '../model/walk';
import { enabledAlongChain } from '../properties/enabled';
import { localMatrixOf, positionOf, transformOf } from '../properties/transform';
/**
* Marks a live entity apart from a struct value: both reach a data edge as an object. A symbol
* rather than a field, so no struct field name can impersonate it – and it never serializes,
* which is right for a thing that exists only while the game runs.
*/
export const ENTITY_HANDLE: unique symbol = Symbol('entity handle');
/**
* What a graph sees of a running entity; the runtime's Entity satisfies this shape. Mutating
* `x`/`y`/`props` is how actions take effect.
*/
export interface EntityHandle {
readonly [ENTITY_HANDLE]: true;
id: string;
assetId: string;
name: string;
/** Position local to `parent` (world position for top-level entities). */
x: number;
y: number;
z: number;
props: PropertyBag;
parent: EntityHandle | null;
/**
* Counts the Emit pulses a particle source has been given. Kept off `props` for the reason the
* animation players are: what the world writes back to the model must not grow an override per frame.
*/
emits?: number;
/** What `Play animation` started on this object (see `model/animation.ts`). */
animations?: AnimationPlayer[];
}
/** Position, rotation, scale and mirror of every ancestor applied, nearest last. */
export function worldTransformOf(handle: EntityHandle): ChainTransform {
return composeAlongChain(
handle,
(h) => h.parent,
(h) => ({ m: localMatrixOf(h, transformOf(h.props)), z: h.z }),
);
}
export function worldMatrixOf(handle: EntityHandle): Mat {
return worldTransformOf(handle).m;
}
export function worldPositionOf(handle: EntityHandle): Position {
return positionOf(worldTransformOf(handle));
}
/** Whether a handle and every ancestor of it are switched on. */
export function isHandleEnabled(handle: EntityHandle): boolean {
return enabledAlongChain(
handle,
(h) => h.props,
(h) => h.parent,
);
}
/** Position in world units. */
export interface PointerState {
x: number;
y: number;
inside: boolean;
down: boolean;
pressed: boolean;
released: boolean;
/** How far the wheel turned this tick, in notches, positive downwards (see `wheelNotches`). */
wheel: number;
}
/**
* What is shared for the length of one run: the values of each Variables asset, keyed by its id,
* and the current state of each `global` machine. Two Variables assets may declare the same name
* without meeting, since a node names the asset it reads (see `graph/sharedState.ts`).
*/
export interface SharedState {
variables: Record<string, PropertyBag>;
states: Record<string, string>;
}
/**
* What a graph can do to the played game's physics (#147). The object is the one a rigidbody sits
* on, or the rigidbody itself; only a dynamic body is acted on or read, and each call answers
* whether there was one. Vectors are world units, turning is degrees the way rotation counts.
*/
export interface PhysicsControl {
velocityOf(target: EntityHandle): { x: number; y: number; angular: number } | null;
setVelocity(target: EntityHandle, x: number, y: number): boolean;
setAngularVelocity(target: EntityHandle, degrees: number): boolean;
/** World units × kg per second: the velocity changes by impulse ÷ mass. */
applyImpulse(target: EntityHandle, x: number, y: number): boolean;
/** World units per second squared, for the rest of the run. */
setGravity(x: number, y: number): void;
/**
* How far a hinge has turned (degrees) or a slider has glided (world units) from how it was placed,
* and how fast; null for any other joint, for one that holds nothing, and for anything else (#154).
*/
jointStateOf(target: EntityHandle): { value: number; speed: number } | null;
}
/**
* What a move with collision met: which of the object's own axes something stood against, whether
* it hit anything, and the world normal of the first contact (pointing away from what was hit).
*/
export interface MoveOutcome {
blockedX: boolean;
blockedY: boolean;
hit: boolean;
normal: { x: number; y: number };
}
/** What a ray met (#153): the object and the collider (or tilemap), where, the face and how far. */
export interface RayOutcome {
object: EntityHandle;
collider: EntityHandle;
point: { x: number; y: number };
normal: { x: number; y: number };
distance: number;
}
/**
* Everything the graph does to the world goes through here, so the core stays free of runtime
* imports. Implemented by the runtime (BehaviorSystem) and by test doubles.
*/
export interface NodeContext {
self: EntityHandle;
/** The behavior's parent object – the one the graph acts on. */
owner: EntityHandle | null;
variables: PropertyBag;
/**
* The current state of each state machine this instance has set, keyed by machine asset id,
* as the state node's id. A machine not in here is in no state: `State branch` follows nothing.
*/
states: Record<string, string>;
shared: SharedState;
/**
* A machine moved from one state to another – what `On state exited` / `On state entered` are
* fired from. Reported rather than fired on the spot: the events belong after the exec flow that
* caused them, never in the middle of it (see `BehaviorSystem`).
*/
stateChanged(machineId: string, from: string, to: string): void;
/** Seconds since the world started and since the last tick. */
time: number;
dt: number;
findByName(name: string): EntityHandle | null;
/** Whether the object is still in the game – false once destroyed, or once its world has ended. */
isAlive(target: EntityHandle): boolean;
/** The object a reference on `from` names (#258, see `model/objectRefs.ts`); null when there is none. */
findReferenced(from: EntityHandle, ref: string): EntityHandle | null;
/** The nearest object called `name` below `parent` – a child before a grandchild; null for none. */
childByName(parent: EntityHandle, name: string): EntityHandle | null;
/** Every object in the game right now, in scene order – what the plural lookups filter (#259). */
objects(): readonly EntityHandle[];
isKeyDown(key: string): boolean;
/** Whether the key went down / came up in this frame – the edges `On key down` and `On key up` fire on. */
wasKeyPressed(key: string): boolean;
wasKeyReleased(key: string): boolean;
pointer: PointerState;
/**
* The mouse as `camera` sees it (#304): world units through that camera, and whether the mouse is
* on its picture. The main camera gives `pointer`; a camera not drawn is never under the mouse.
*/
pointerThrough(camera: EntityHandle): { x: number; y: number; inside: boolean };
/**
* Whether a world point lies on `target`: on its colliders when it has any, otherwise on its own
* footprint or that of anything parented below it.
*/
hitTest(target: EntityHandle, x: number, y: number): boolean;
/** A collider entity stands for itself; any other entity for its direct collider children. */
isTouching(a: EntityHandle, b: EntityHandle): boolean;
/**
* Moves `target` by (dx, dy) in local units; its solid colliders stop flush at other solid ones
* and the rest of the move goes on along the surface, less what friction takes.
*/
moveAndSlide(target: EntityHandle, dx: number, dy: number): MoveOutcome;
/**
* The first solid a ray from (x, y) along (dx, dy) meets within `length`, all in world units;
* null for none. `ignore` and whatever is parented below it are not met, and neither is anything
* off the layers of `layers` (a 16-bit mask, #158).
*/
raycast(
from: { x: number; y: number },
direction: { x: number; y: number },
length: number,
options: { ignore: EntityHandle | null; triggers: boolean; layers: number },
): RayOutcome | null;
/**
* Everything that ray meets, nearest first, rather than only the first of them (#157). One entry
* per collider and one per tilemap cell; `maxHits` above 0 keeps that many of the nearest.
*/
raycastAll(
from: { x: number; y: number },
direction: { x: number; y: number },
length: number,
options: { ignore: EntityHandle | null; triggers: boolean; layers: number; maxHits: number },
): RayOutcome[];
/**
* The number of the collision layer `target` is on (#211): a collider's or tilemap's own, any
* other object's that of its first collider child; null for an object without a collider.
*/
collisionLayerOf(target: EntityHandle): number | null;
/**
* The number of the render layer `target` is drawn on (#305): its own choice, else its parent's,
* at the top 0; null for an object that is not drawn.
*/
renderLayerOf(target: EntityHandle): number | null;
/** Lets `target` fall through one-way colliders and tiles for `seconds` (#155). */
ignoreOneWay(target: EntityHandle, seconds: number): void;
/** Whether `target` is a placed tilemap – what the tile nodes act on (#135). */
isTilemap(target: EntityHandle): boolean;
/**
* The tile id in a cell of a placed tilemap, `''` for an empty one; null when `target` is no
* tilemap, has no layer of that name (`''` = its first) or the cell lies outside its grid.
*/
tileAt(target: EntityHandle, layer: string, cell: { x: number; y: number }): string | null;
/**
* Puts a tile into that cell, `''` empties it; false where `tileAt` answers null or the id is no
* tile. Only this placement changes, and only for the run (see `model/liveTilemap.ts`).
*/
setTile(target: EntityHandle, layer: string, cell: { x: number; y: number }, tileId: string): boolean;
/** Absent where nothing is simulated: the editor run, and a test double that does not ask for it. */
physics?: PhysicsControl;
log(message: string): void;
/**
* Starts a sound asset, named by its id. `volume` multiplies the asset's own (1 leaves it alone);
* `loop` is the caller's answer to whether it repeats, and `undefined` leaves that to the asset,
* because the asset says what a sound *is* and the node only colours it. Every call is a voice of
* its own: two plays overlap, whether they loop or not, which is what makes one sound effect
* fired twice in a row audible twice.
*
* The voice belongs to the behavior that started it and ends when that behavior is destroyed –
* a sound is something a thing in the world is doing. `detached` gives that up on purpose, for
* the one case where the rule is in the way: the death sound of an object that destroys itself.
* `fx` names a sound FX asset the voice is played through (#311); `''` plays it as it is.
* `fxParams` are values for its parameters; one not given is the default the FX declares.
* Returns the number of the voice for `tuneSound`, or 0 where nothing is played.
*/
playSound(
assetId: string,
volume: number,
loop: boolean | undefined,
detached: boolean,
fx: string,
fxParams: FxParams,
): number;
/**
* Plays one note of an instrument asset (#331): `note` as MIDI counts it (60 is C4), struck with
* `velocity` (0 to 1, how loud it is next to `volume`) and released after `duration` seconds, or
* held until it is stopped where that is 0. Owned like a `playSound` voice, `instrumentParams` are
* values for the instrument's parameters. Returns the number of the voice, 0 where nothing plays.
*/
playNote(
instrumentId: string,
note: number,
velocity: number,
volume: number,
duration: number,
detached: boolean,
instrumentParams: FxParams,
): number;
/**
* How many seconds one run of that sound asset lasts, so a change can span it; 0 where it does not
* exist or has not been loaded yet.
*/
soundLength(assetId: string): number;
/** How many seconds that voice rings on after its sound ends: its echoes (#324); 0 for none. */
soundTail(voice: number): number;
/**
* Silences every voice of that sound; `''` silences everything this run started. `letRing` stops
* the sound and lets its echoes fade (#324).
*/
stopSound(assetId: string, letRing: boolean): void;
/** Silences one voice, by the number `playSound` gave it. */
stopVoice(voice: number, letRing: boolean): void;
/**
* Hands new values to the sound FX parameters of one voice, by the number `playSound` gave it
* (#315), so a running sound follows the game. A voice that has ended is not reached. `time` is
* how many seconds the values take to get there, on the audio clock and along `ease`; 0 is a
* quick glide.
*/
tuneSound(voice: number, fxParams: FxParams, time: number, ease: EaseShare): void;
/**
* Asks for another scene, by asset id. The swap happens once the frame is over – a graph must be
* able to finish the flow it is in, and half a frame in the old world and half in the new one is
* the kind of thing nobody can debug. The last request of a frame is the one that happens.
*/
loadScene(sceneId: string): void;
/**
* The position is local to `parent` (world when none); null when the asset does not exist or
* cannot be placed. A spawned behavior runs its `On start` on the next tick.
*/
spawn(
assetId: string,
position: { x: number; y: number; z: number },
parent: EntityHandle | null,
): EntityHandle | null;
/**
* Places the joint asset on `owner` at `at` of that object and connects it to `connected`, the
* scene when null (#154). Null when the asset is no joint or the object is gone.
*/
addJoint(
assetId: string,
owner: EntityHandle,
connected: EntityHandle | null,
at: { x: number; y: number },
): EntityHandle | null;
/**
* Removes `target` with everything below it. With `keepChildren` only the object goes: its
* children move up to its parent where they stand, and are answered, nearest level only.
*/
destroy(target: EntityHandle, keepChildren: boolean): EntityHandle[];
stop(): void;
/** Ends the run and starts it again from the saved scene (everything spawned is lost). */
restart(): void;
/** The global time scale the run goes at (#167): 1 is real time, 0 frozen. */
timeScale(): number;
/** For the rest of the run, the next scene included; Play and Stop set it back to 1. */
setTimeScale(scale: number): void;
/** Whether a behavior paused the game – not the editor's own pause, which runs no graph at all. */
isPaused(): boolean;
/** From the next frame on, only behaviors that run while paused are ticked. */
setPaused(paused: boolean): void;
}