A sound could only be played as it was, one at a time. An instrument
is a node graph that makes a note a sound: Notes nodes bring the note
in, Samples play sound assets for it, the effects of a sound FX shape
it and the Output sums the layers. Its tab plays it from a keyboard,
a behavior with Play note. It is what patterns and songs (#330) will
play.
- New asset type instrument, under the heading Music, not placeable.
A new one starts as Notes into a Sample into the Output.
- Notes are counted as MIDI does: 60 is C4, names like C#4, sharps
only, the lowest C-1. Notes has From and To, outputs any, note,
frequency and velocity, and with Single notes a trigger per note,
each with an optional name for a drum kit.
- New port type trigger, only into the trigger of a source. Inputs
can take any number of edges now (PortDef.many): a trigger, and
the Output, which plays everything into it side by side.
- Sample plays a sound asset. Follow note shifts it by note - root
through the pitch worklet of #317, so it keeps its length. Gate
fades it out at the end of the note, One-shot plays through. Loop
runs between Loop start and Loop end, also in the worklet.
- Parameters declares the instrument's own parameters with a
default. Sound FX sends the sound through a sound FX asset, its
parameters as inputs, the sound set in where its Input was.
- compileInstrument walks the graph once per note. A trigger is on
only for the notes it stands for, a silent source silences what it
alone feeds, a Mix with a silent side is the other side. velocity
and the parameters stay open for the voice. Builds are cached per
note until the graph or a sound or FX it names changes.
- The player takes a note as one voice of several layers, each way
with its own file and volume. release fades the gated ways out,
one-shots play on. playNote can release after a duration.
- Play note plays one note with velocity, volume, duration and
detached, and hands out voice for Set sound FX parameters and
Stop sound. Velocity is always loudness. Instrument parameters
picks values for the instrument's parameters like Sound FX
parameters does for an FX, renames follow.
- The tab has the graph, a header field per parameter, Stop, and
128 keys below. Keys without sound are grey. A key plays while it
is held. Focused, the computer keyboard plays as in a tracker by
KeyboardEvent.code, the arrows move the octave.
Findings:
- A label of the keys set fill on the shared small text style and
darkened every small label of the editor. Caught in a screenshot
before the commit, the keys have a style of their own now.
- Suite task 523 on cf6799a failed two checks of e2e 34 that timed
audio by the page's clock: a 2.95 s echo tail was gone after 1 s,
and Quad in was further along than 0.7 s allow. Both time on the
audio clock now through a new hook __gm.sound.clock().
Tests:
- instrument.test.ts (new, 22): notes, asset, palette, round trip,
compile per note, drum kit, zones, Mix, velocity and parameters,
Sound FX node with rename, problems, cache, Play note, the keys
- sound-library +7 (layers, release, duration, early release, loop
region in buffer and worklet), pitch-player +2 (loop region),
node-catalog +4, graph adjusted. 582 tests in the touched files.
- e2e 4 and 34 on melektaus-atx: ALL PASSED, 34 with 44 checks (+7)
- Gate on melektaus-atx: CHECK EXIT 0, 243 test files, 3750 tests
- Counter-checks: any always on, release ending one-shots, Sound FX
inputs ignored, loop region ignored. 3, 2, 1 and 1 tests failed.
- Browser probe on stein2: keys, held key, Single notes names and
the header looked at in screenshots.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
271 lines
11 KiB
TypeScript
271 lines
11 KiB
TypeScript
import { isList, type ListValue, type PropertyBag, type PropertyValue } from '../model/Properties';
|
||
import { type DeclaredPortKind, isKindWith, plainKindOf } from '../properties/kinds';
|
||
import { flatDef, type PropertyDef } from '../properties/PropertySchema';
|
||
import type { DeclaredPort, Graph, GraphNode, PortType } from './Graph';
|
||
import type { GraphLookup, ValueTyping } from './lookup';
|
||
import type { NodeContext } from './NodeContext';
|
||
import type { SharedPortGroup } from './portTypes';
|
||
import { asList, namedValue, type Value } from './values';
|
||
|
||
export interface PortDef {
|
||
key: string;
|
||
label: string;
|
||
type: PortType;
|
||
/** Inline value used while the input is unconnected and the node has no own value (else 0/''/false). */
|
||
default?: PropertyValue;
|
||
/** The property this port carries; its typed-in value is edited as that property (color, enum, …). */
|
||
field?: PropertyDef;
|
||
/** `false`: nothing to type in even for a number – unconnected, the node's own rule applies. */
|
||
inline?: boolean;
|
||
/** Not a port of the definition: a key only an edge still names, drawn so that edge can be seen. */
|
||
ghost?: true;
|
||
/**
|
||
* Drawing only: the colour an `any` input is painted in, taken from the edge that feeds it (#207).
|
||
* Its `type` stays `any`, so the port keeps taking everything and can be swapped at any time –
|
||
* only the pin says what currently flows in. Set by `drawnDef`, read by `NodeView`, nowhere else.
|
||
*/
|
||
paintType?: PortType;
|
||
/** On a port writing a declared variable: what the written value is held to (#271). */
|
||
typing?: ValueTyping;
|
||
/** An input taking any number of edges, like an exec input: the layers of an instrument (#331). */
|
||
many?: true;
|
||
}
|
||
|
||
/**
|
||
* `state`, `shader`, `particles` and `sound` are the categories no behavior graph offers: their nodes
|
||
* are the states of a state machine, the GLSL of a shader asset, the motion of one particle and the
|
||
* effects of a sound FX asset.
|
||
*/
|
||
export type NodeCategory =
|
||
'event' | 'flow' | 'data' | 'variable' | 'action' | 'state' | 'shader' | 'particles' | 'sound';
|
||
|
||
/**
|
||
* How a loop node asks for one of its exec outputs to be run repeatedly: `port` carries the body,
|
||
* `index` is the number output seeded with the round number.
|
||
*/
|
||
export interface RepeatResult {
|
||
port: string;
|
||
index: string;
|
||
times: number;
|
||
/** Round i also sets output `port` to `values[i]` – the element a `For each` is at. */
|
||
items?: { port: string; values: readonly Value[] };
|
||
/**
|
||
* Boolean input read anew before every round, after the round's outputs are set. True ends the
|
||
* loop there: that round's body does not run, and its outputs stay for what follows.
|
||
*/
|
||
stopWhen?: string;
|
||
/** Boolean output telling whether `stopWhen` ended the loop. */
|
||
stopped?: string;
|
||
/** Outputs standing after a loop that ran through all rounds without being stopped. */
|
||
exhausted?: Record<string, Value>;
|
||
}
|
||
|
||
/** How a node asks for another behavior's graph to be run in place, `inputs` seeding its start node. */
|
||
export interface CallResult {
|
||
behaviorId: string;
|
||
inputs: Record<string, Value>;
|
||
/** The flow input the call came in by, which is the output its start node leaves through (#294). */
|
||
entry: string;
|
||
}
|
||
|
||
export interface RunResult {
|
||
/** Cached for the rest of the pulse. */
|
||
outputs?: Record<string, Value>;
|
||
/** Exec output(s) to continue along; omitted = the node ends the flow. */
|
||
next?: string | string[];
|
||
/** Runs `port` that many times before continuing along `next`. */
|
||
repeat?: RepeatResult;
|
||
/** Runs another behavior before continuing along `next`; its results join `outputs`. */
|
||
call?: CallResult;
|
||
/**
|
||
* The elements a `Find` tests one by one against the `Match` closing its block. The flow goes on
|
||
* from that Match, not from the Find.
|
||
*/
|
||
find?: readonly Value[];
|
||
}
|
||
|
||
/** What a node may compute its ports from: itself, the graph it sits in, and the project around it. */
|
||
export interface PortSource {
|
||
node: GraphNode;
|
||
graph: Graph;
|
||
lookup: GraphLookup;
|
||
}
|
||
|
||
/** Ports a single node has on top of the fixed ones of its type, and a label naming what it does. */
|
||
export interface DerivedShape {
|
||
label?: string;
|
||
inputs?: PortDef[];
|
||
outputs?: PortDef[];
|
||
/** The ports stand in the order given instead of exec pins first – a node laying out a whole side. */
|
||
inOrder?: true;
|
||
}
|
||
|
||
/** Palette, port rendering, connection validation and inspector fields are all derived from this. */
|
||
export interface NodeDef {
|
||
type: string;
|
||
label: string;
|
||
category: NodeCategory;
|
||
description: string;
|
||
inputs: PortDef[];
|
||
outputs: PortDef[];
|
||
/** Node settings that are not ports (key name, operator, variable). */
|
||
params: PropertyDef[];
|
||
/**
|
||
* Settings that say nothing at this value set, so the node body leaves them off; the inspector
|
||
* still offers them. Without it a switched-off repeat is two rows of "0" on every key node.
|
||
*/
|
||
hiddenParams?(params: PropertyBag): string[];
|
||
/**
|
||
* Settings that do nothing at this value set, so neither the node body nor the inspector shows
|
||
* them – like the Direction of a linear key. Unlike `hiddenParams`, nothing is left to pick there.
|
||
*/
|
||
idleParams?(params: PropertyBag): string[];
|
||
/**
|
||
* Which side the ports this node declares for itself (`GraphNode.ports`) appear on. Setting it
|
||
* is what gives a node the port editor in the inspector. `both`: every port is an output, and an
|
||
* input as well where it is marked `write`.
|
||
*/
|
||
declaresPorts?: 'inputs' | 'outputs' | 'both';
|
||
/** The kinds a declared port may have here; without it, every declarable kind. */
|
||
portKinds?: readonly DeclaredPortKind[];
|
||
/** How a declared port becomes a port; without it, named after its key and typed by its kind. */
|
||
declaredPort?(port: DeclaredPort, node: GraphNode, graph: Graph, lookup: GraphLookup): PortDef;
|
||
/**
|
||
* Edges between two such nodes are transitions: drawn as a straight arrow instead of the usual
|
||
* curve, and hung on ports that are never shown. A transition graph is read as directed steps
|
||
* between whole states, not as signal flowing left to right into a particular socket.
|
||
*/
|
||
transitionEdges?: true;
|
||
/** The node that comes along when this one is added, `dx` to the right: the end of its block. */
|
||
opens?: { type: string; dx: number };
|
||
/**
|
||
* Ports this node computes beyond the fixed and the declared ones – how many outputs a Sequence
|
||
* offers, which values the behavior a call node names takes. `resolveDef` applies it.
|
||
*/
|
||
derivePorts?(source: PortSource): DerivedShape;
|
||
/**
|
||
* Ports carrying one and the same value: the outputs take the type the inputs agree on, so an
|
||
* `any` output is drawn, validated and edited as what actually flows through it (#207).
|
||
* `resolveDef` applies it; only the outputs change, the inputs stay `any` and take everything.
|
||
*/
|
||
sharedType?: SharedPortGroup;
|
||
/**
|
||
* Outputs readable at any moment, even while the node's exec input has not fired in this pulse.
|
||
* Never cached, so a read after a write sees the new value. `input` pulls one input on demand:
|
||
* pulling all of them would loop through an edge that feeds the node's own output back in.
|
||
* `def` and `lookup` are as in `run`: the definition resolved for this very node, and the project
|
||
* around it – what a reader with one output per state needs to answer for all of them.
|
||
*/
|
||
read?(
|
||
ctx: NodeContext,
|
||
params: PropertyBag,
|
||
input: (key: string) => Value,
|
||
def: NodeDef,
|
||
lookup: GraphLookup,
|
||
): Record<string, Value>;
|
||
/**
|
||
* Event nodes have no `run` – the interpreter seeds their outputs. `def` is resolved for the node at
|
||
* hand, the only way a node with derived ports knows how many it has. `entry` is the exec input
|
||
* the pulse arrived through – what a node with one exec input per meaning reads. `lookup` is the
|
||
* project around the graph, for a node whose step is written in another asset.
|
||
*/
|
||
run?(
|
||
ctx: NodeContext,
|
||
inputs: Record<string, Value>,
|
||
params: PropertyBag,
|
||
def: NodeDef,
|
||
entry: string,
|
||
lookup: GraphLookup,
|
||
): RunResult;
|
||
}
|
||
|
||
const defs = new Map<string, NodeDef>();
|
||
|
||
export function registerNode(def: NodeDef): NodeDef {
|
||
if (defs.has(def.type)) throw new Error(`Node type "${def.type}" registered twice`);
|
||
defs.set(def.type, def);
|
||
return def;
|
||
}
|
||
|
||
export function nodeDef(type: string): NodeDef | undefined {
|
||
return defs.get(type);
|
||
}
|
||
|
||
export function allNodeDefs(): NodeDef[] {
|
||
return [...defs.values()];
|
||
}
|
||
|
||
export function isEventNode(def: NodeDef): boolean {
|
||
return def.category === 'event';
|
||
}
|
||
|
||
/** Whether an input keeps the edges it has when another one is connected to it. */
|
||
export function takesManyEdges(port: PortDef): boolean {
|
||
return port.type === 'exec' || port.many === true;
|
||
}
|
||
|
||
export function hasExecInput(def: NodeDef): boolean {
|
||
return def.inputs.some((p) => p.type === 'exec');
|
||
}
|
||
|
||
/**
|
||
* Data inputs that may carry an inline value while unconnected. An object is a live handle and a
|
||
* struct is assembled from its parts, so neither has a value that could be typed on one line; a
|
||
* shader colour has one only where the node gives it a colour field.
|
||
*/
|
||
export function inlineInputs(def: NodeDef): PortDef[] {
|
||
return def.inputs.filter(isInlinePort);
|
||
}
|
||
|
||
function isInlinePort(port: PortDef): boolean {
|
||
if (port.inline === false) return false;
|
||
if (port.type === 'vec3') return port.field !== undefined;
|
||
return (
|
||
port.type !== 'exec' &&
|
||
port.type !== 'object' &&
|
||
port.type !== 'struct' &&
|
||
port.type !== 'picture' &&
|
||
port.type !== 'audio' &&
|
||
port.type !== 'fxparams' &&
|
||
port.type !== 'trigger'
|
||
);
|
||
}
|
||
|
||
/** Untyped, an input writes the zero of its kind; as its default the node body shows it, not just the name. */
|
||
export function withInlineDefault(port: PortDef): PortDef {
|
||
return isInlinePort(port) ? { ...port, default: inlineInputDef(port).default } : port;
|
||
}
|
||
|
||
export function inlineInputDef(port: PortDef): PropertyDef {
|
||
if (port.field) return { ...port.field, key: port.key, label: port.label };
|
||
const kind = port.type === 'any' ? 'text' : plainKindOf(port.type);
|
||
// A declared default is a stored value, never a live handle.
|
||
if (kind === 'list')
|
||
return { key: port.key, label: port.label, kind, default: namedValue(asList(port.default)) as ListValue };
|
||
const flat = kind !== undefined && isKindWith('field', kind) ? kind : 'number';
|
||
const def = flatDef(port.key, flat, { label: port.label, value: port.default });
|
||
return def.kind === 'number' ? { ...def, step: 1 } : def;
|
||
}
|
||
|
||
/**
|
||
* The value an unconnected input carries: the node's own inline value, else the default the
|
||
* inspector shows for it – `false`, `0`, `""`, an empty list –, so what the node reads is what the
|
||
* editor says it reads. Only an input with nothing to type in (object, struct, exec) is `null`.
|
||
*/
|
||
export function inlineValue(node: { params: PropertyBag }, port: PortDef): Value {
|
||
return node.params[port.key] ?? port.default ?? shownDefault(port);
|
||
}
|
||
|
||
/** Cached per port: the interpreter asks for it on every read of an unconnected input. */
|
||
const shownDefaults = new WeakMap<PortDef, PropertyValue | null>();
|
||
|
||
function shownDefault(port: PortDef): Value {
|
||
let value = shownDefaults.get(port);
|
||
if (value === undefined) {
|
||
value = isInlinePort(port) ? inlineInputDef(port).default : null;
|
||
shownDefaults.set(port, value);
|
||
}
|
||
// A list is handed out fresh: a node that edits its input in place must not edit every other one.
|
||
return isList(value) ? [...value] : value;
|
||
}
|