Play sounds as notes of an instrument asset #331

A sound could only be played as it was, one at a time. An instrument
is a node graph that makes a note a sound: Notes nodes bring the note
in, Samples play sound assets for it, the effects of a sound FX shape
it and the Output sums the layers. Its tab plays it from a keyboard,
a behavior with Play note. It is what patterns and songs (#330) will
play.

- New asset type instrument, under the heading Music, not placeable.
  A new one starts as Notes into a Sample into the Output.
- Notes are counted as MIDI does: 60 is C4, names like C#4, sharps
  only, the lowest C-1. Notes has From and To, outputs any, note,
  frequency and velocity, and with Single notes a trigger per note,
  each with an optional name for a drum kit.
- New port type trigger, only into the trigger of a source. Inputs
  can take any number of edges now (PortDef.many): a trigger, and
  the Output, which plays everything into it side by side.
- Sample plays a sound asset. Follow note shifts it by note - root
  through the pitch worklet of #317, so it keeps its length. Gate
  fades it out at the end of the note, One-shot plays through. Loop
  runs between Loop start and Loop end, also in the worklet.
- Parameters declares the instrument's own parameters with a
  default. Sound FX sends the sound through a sound FX asset, its
  parameters as inputs, the sound set in where its Input was.
- compileInstrument walks the graph once per note. A trigger is on
  only for the notes it stands for, a silent source silences what it
  alone feeds, a Mix with a silent side is the other side. velocity
  and the parameters stay open for the voice. Builds are cached per
  note until the graph or a sound or FX it names changes.
- The player takes a note as one voice of several layers, each way
  with its own file and volume. release fades the gated ways out,
  one-shots play on. playNote can release after a duration.
- Play note plays one note with velocity, volume, duration and
  detached, and hands out voice for Set sound FX parameters and
  Stop sound. Velocity is always loudness. Instrument parameters
  picks values for the instrument's parameters like Sound FX
  parameters does for an FX, renames follow.
- The tab has the graph, a header field per parameter, Stop, and
  128 keys below. Keys without sound are grey. A key plays while it
  is held. Focused, the computer keyboard plays as in a tracker by
  KeyboardEvent.code, the arrows move the octave.

Findings:
- A label of the keys set fill on the shared small text style and
  darkened every small label of the editor. Caught in a screenshot
  before the commit, the keys have a style of their own now.
- Suite task 523 on cf6799a failed two checks of e2e 34 that timed
  audio by the page's clock: a 2.95 s echo tail was gone after 1 s,
  and Quad in was further along than 0.7 s allow. Both time on the
  audio clock now through a new hook __gm.sound.clock().

Tests:
- instrument.test.ts (new, 22): notes, asset, palette, round trip,
  compile per note, drum kit, zones, Mix, velocity and parameters,
  Sound FX node with rename, problems, cache, Play note, the keys
- sound-library +7 (layers, release, duration, early release, loop
  region in buffer and worklet), pitch-player +2 (loop region),
  node-catalog +4, graph adjusted. 582 tests in the touched files.
- e2e 4 and 34 on melektaus-atx: ALL PASSED, 34 with 44 checks (+7)
- Gate on melektaus-atx: CHECK EXIT 0, 243 test files, 3750 tests
- Counter-checks: any always on, release ending one-shots, Sound FX
  inputs ignored, loop region ignored. 3, 2, 1 and 1 tests failed.
- Browser probe on stein2: keys, held key, Single notes names and
  the header looked at in screenshots.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

src/editor/Editor.ts

@@ -73,6 +73,7 @@ import { ProjectSession } from './ProjectSession';
 import { ParticlesEditor } from './particles/ParticlesEditor';
 import { ShaderEditor } from './shader/ShaderEditor';
 import { SoundFxEditor } from './soundfx/SoundFxEditor';
+import { InstrumentEditor } from './instrument/InstrumentEditor';
 import { StateMachineEditor } from './statemachine/StateMachineEditor';
 import { installShortcuts } from './shortcuts';
 import type { PlayFrom } from '../core/model/playSettings';
@@ -95,6 +96,7 @@ export type CentreTab =
   | 'shader'
   | 'particles'
   | 'soundfx'
+  | 'instrument'
   | 'game'
   | 'texture'
   | 'vectorart'
@@ -145,6 +147,7 @@ const PANE_ORDER = [
   'shader',
   'particles',
   'soundfx',
+  'instrument',
   'texture',
   'vectorart',
   'pixelart',
@@ -165,6 +168,7 @@ const TAB_LABELS: Record<CentreTab, string> = {
   shader: 'Shader',
   particles: 'Particles',
   soundfx: 'Sound FX',
+  instrument: 'Instrument',
   texture: 'Texture',
   vectorart: 'Vector art',
   pixelart: 'Pixel art',
@@ -214,6 +218,7 @@ export class Editor implements PlayHost {
   readonly shaderEditor: ShaderEditor;
   readonly particlesEditor: ParticlesEditor;
   readonly soundFxEditor: SoundFxEditor;
+  readonly instrumentEditor: InstrumentEditor;
   readonly textureEditor: TextureEditor;
   readonly vectorArtEditor: VectorArtEditor;
   readonly pixelArtEditor: PixelArtEditor;
@@ -300,6 +305,7 @@ export class Editor implements PlayHost {
     this.shaderEditor = new ShaderEditor(state);
     this.particlesEditor = new ParticlesEditor(state);
     this.soundFxEditor = new SoundFxEditor(state);
+    this.instrumentEditor = new InstrumentEditor(state);
     this.textureEditor = new TextureEditor(state);
     this.vectorArtEditor = new VectorArtEditor(state);
     this.pixelArtEditor = new PixelArtEditor(state);
@@ -409,6 +415,7 @@ export class Editor implements PlayHost {
       shader: this.shaderEditor,
       particles: this.particlesEditor,
       soundfx: this.soundFxEditor,
+      instrument: this.instrumentEditor,
       texture: this.textureEditor,
       vectorart: this.vectorArtEditor,
       pixelart: this.pixelArtEditor,
@@ -600,6 +607,8 @@ export class Editor implements PlayHost {
         return this.particlesEditor.canvas.paste();
       case 'soundfx':
         return this.soundFxEditor.canvas.paste();
+      case 'instrument':
+        return this.instrumentEditor.canvas.paste();
       default:
         return false;
     }

src/editor/assetTabs.ts

@@ -10,6 +10,7 @@ export const TAB_OF_ASSET: Partial<Record<AssetType, CentreTab>> = {
   shaderfunction: 'shader',
   particles: 'particles',
   soundfx: 'soundfx',
+  instrument: 'instrument',
   texture: 'texture',
   vectorart: 'vectorart',
   pixelart: 'pixelart',

src/editor/debug/graphHooks.ts

@@ -113,6 +113,27 @@ export function graphHooks({ state, editor }: HookContext) {
       params: () => editor.soundFxEditor.previewValues,
       setParam: (key: string, value: number) => editor.soundFxEditor.setPreviewValue(key, value),
     },
+    instrument: {
+      current: () => editor.instrumentEditor.currentAssetId,
+      info: () => editor.instrumentEditor.shownInfo,
+      nodeAt: (nodeId: string) => editor.instrumentEditor.canvas.nodeGlobal(nodeId),
+      nodeLabel: (nodeId: string) => editor.instrumentEditor.canvas.nodeLabel(nodeId),
+      nodeOutputs: (nodeId: string) => editor.instrumentEditor.canvas.nodeOutputs(nodeId),
+      nodeProblem: (nodeId: string) => editor.instrumentEditor.canvas.nodeProblem(nodeId),
+      portAt: (nodeId: string, port: string, isOutput: boolean) =>
+        editor.instrumentEditor.canvas.portGlobal(nodeId, port, isOutput),
+      canvasRect: () => editor.instrumentEditor.canvas.canvasRect(),
+      /** The keys: which make a sound, which are held, where one is, and the computer keyboard's octave. */
+      keys: () => ({
+        sounding: editor.instrumentEditor.soundingNotes(),
+        held: editor.instrumentEditor.keyboard.heldNotes,
+        octave: editor.instrumentEditor.keyboard.bottomOctave,
+      }),
+      keyAt: (note: number) => editor.instrumentEditor.keyboard.keyCentre(note),
+      stopAt: () => centreOf(editor.instrumentEditor.stopButton),
+      params: () => editor.instrumentEditor.previewValues,
+      setParam: (key: string, value: number) => editor.instrumentEditor.setPreviewValue(key, value),
+    },
     particles: {
       current: () => editor.particlesEditor.currentAssetId,
       info: () => editor.particlesEditor.shownInfo,

src/editor/debugHook.ts

@@ -45,6 +45,8 @@ export function installDebugHook(app: Application, state: EditorState, editor: E
       /** Where the knobs of each voice of that file are now, oldest first, a pitch as its speed (#315). */
       levels: (file: string) => soundLibrary.levels(file),
       pitchWorkletUrl: () => soundLibrary.pitchWorkletUrl(),
+      /** Seconds on the audio clock: the time a voice really runs on. */
+      clock: () => soundLibrary.clock(),
       /** Decodes ahead of a check, so a play can be asserted without racing the network. */
       load: (file: string) => soundLibrary.load(file).then((buffer) => buffer?.duration ?? null),
       stopAll: () => soundLibrary.stopAll(),

src/editor/graph/GraphCanvas.ts

@@ -21,6 +21,7 @@ import {
   namesBehavior,
   RUN_BEHAVIOR,
   SHADER_CALL,
+  INSTRUMENT_SOUND_FX,
   callsFunction,
   soundFxParamsMismatch,
   startStateOf,
@@ -30,6 +31,7 @@ import { graphLookup } from '../../core/model/graphLookup';
 import { particleLibrary } from '../../viewport/ParticleLibrary';
 import { shaderLibrary } from '../../viewport/ShaderLibrary';
 import { soundFxBuildFor } from '../../core/soundfx/compile';
+import { instrumentCheck, instrumentSourcesOf } from '../../core/instrument/compile';
 import { findAsset, findGraphAsset } from '../../core/model/Project';
 import { type Trace, TRACE_FADE } from '../../runtime/systems/BehaviorSystem';
 import { ViewTween } from '../viewTween';
@@ -309,7 +311,9 @@ export class GraphCanvas extends Widget {
           ? particleLibrary.buildFor(asset).problems
           : asset.type === 'soundfx'
             ? soundFxBuildFor(asset).problems
-            : null;
+            : asset.type === 'instrument'
+              ? instrumentCheck(asset, instrumentSourcesOf(this.state.project)).problems
+              : null;
     for (const node of asset.graph.nodes) {
       const resolved = resolveDef(node, asset.graph, lookup);
       if (!resolved) continue;
@@ -325,7 +329,9 @@ export class GraphCanvas extends Widget {
       }
       const problem =
         compilerProblems?.[node.id] ??
-        (!compilerProblems || node.type === SHADER_CALL ? callProblem(node, lookup, asset.id) : null) ??
+        (!compilerProblems || node.type === SHADER_CALL || node.type === INSTRUMENT_SOUND_FX
+          ? callProblem(node, lookup, asset.id)
+          : null) ??
         soundFxParamsMismatch(node, asset.graph) ??
         ghostProblem(node, asset.graph, resolved) ??
         blockProblem(asset.graph, node);
@@ -656,9 +662,10 @@ export class GraphCanvas extends Widget {
   }
 
   private openCalledBehavior(node: GraphNode): boolean {
-    if (node.type === SHADER_CALL) {
-      const id = textOf(node.params[FUNCTION_PARAM] ?? '');
-      if (findAsset(this.state.project, id)?.type !== 'shaderfunction') return false;
+    if (node.type === SHADER_CALL || node.type === INSTRUMENT_SOUND_FX) {
+      const shader = node.type === SHADER_CALL;
+      const id = textOf(node.params[shader ? FUNCTION_PARAM : 'fx'] ?? '');
+      if (findAsset(this.state.project, id)?.type !== (shader ? 'shaderfunction' : 'soundfx')) return false;
       this.state.openAsset(id);
       return true;
     }

src/editor/graph/NodeView.ts

@@ -97,6 +97,8 @@ export function portColor(type: PortType): number {
       return c.portAudio;
     case 'fxparams':
       return c.portFxParams;
+    case 'trigger':
+      return c.portTrigger;
     case 'any':
       return c.portAny;
   }

src/editor/graph/paletteEntries.ts

@@ -42,6 +42,7 @@ import { nextParticlePropertyName, PARTICLE_NODES, PARTICLE_OUTPUT } from '../..
 import { offeredIn, ShaderFunctionInput, ShaderFunctionOutput } from '../../core/shader/functions';
 import { SHADER_NODES, SHADER_OUTPUT, SHADER_VERTEX_OUTPUT } from '../../core/shader/nodes';
 import { SOUND_FX_INPUT, SOUND_FX_NODES, SOUND_FX_OUTPUT } from '../../core/soundfx/nodes';
+import { INSTRUMENT_NODES } from '../../core/instrument/nodes';
 
 /**
  * One row of the palette. `def` is the definition the node would actually have once placed, so the
@@ -127,7 +128,7 @@ function shaderEntries(project: Project, asset: ShaderAsset | ShaderFunctionAsse
 /**
  * The node types offerable in `asset`'s graph. A state machine holds states and nothing else, each
  * new one named by its number; a shader holds shader nodes (`shaderEntries`), a sound FX its effects
- * with one Input and one Output. In a behavior, every other behavior appears under its own name instead of behind a
+ * with one Input and one Output, an instrument its own nodes and the same effects. In a behavior, every other behavior appears under its own name instead of behind a
  * generic "Run behavior" – and once more as the event "On <name>" if it has an End to be heard
  * through (#294) –, and the two terminals drop out once the graph has them.
  */
@@ -143,6 +144,15 @@ export function paletteEntries(project: Project, asset: GraphAsset): PaletteEntr
       (def) => !(def.type === SOUND_FX_INPUT || def.type === SOUND_FX_OUTPUT) || !placed.has(def.type),
     ).map((def) => ({ id: def.type, hint: def.category, def, params: {} }));
   }
+  if (asset.type === 'instrument') {
+    const placed = new Set(asset.graph.nodes.map((n) => n.type));
+    const effects = SOUND_FX_NODES.filter(
+      (def) => def.type !== SOUND_FX_INPUT && def.type !== SOUND_FX_OUTPUT,
+    );
+    return [...INSTRUMENT_NODES, ...effects]
+      .filter((def) => !isTerminalType(def.type) || !placed.has(def.type))
+      .map((def) => ({ id: def.type, hint: def.category, def, params: {} }));
+  }
   if (asset.type === 'particles') {
     const hasOutput = asset.graph.nodes.some((n) => n.type === PARTICLE_OUTPUT);
     const taken = new Set(asset.graph.nodes.filter(isParticlePropertyNode).map(propertyNameOf));

src/editor/inspector/portSection.ts

@@ -13,7 +13,10 @@ import {
 import {
   type DeclaredPort,
   type GraphNode,
+  declaresParameters,
   declaresSoundFxPorts,
+  INSTRUMENT_PARAMETER_VALUES,
+  INSTRUMENT_SETTING,
   isEntryTerminal,
   isPortOf,
   isTerminalNode,
@@ -28,7 +31,6 @@ import {
   SOUND_FX_INPUT,
   SOUND_FX_SETTING,
   soundFxSourceOf,
-  terminalNode,
   VARIABLES_SETTING,
   variablesSourceOf,
   propertyTargetOf,
@@ -37,6 +39,7 @@ import {
   targetProperties,
 } from '../../core/graph';
 import { isPrivateVariable } from '../../core/model/Asset';
+import { graphLookup } from '../../core/model/graphLookup';
 import { findAssetOf, findGraphAsset, type Project } from '../../core/model/Project';
 import { textOf } from '../../core/model/Properties';
 import { assetRefChoices, NO_ASSET_LABEL } from '../../core/properties/assetRef';
@@ -98,10 +101,12 @@ export function buildVariablePortSection(
   });
 }
 
-/** The parameters the Input of the sound FX a `Sound FX parameters` node names declares (#311). */
+/**
+ * The parameters the sound FX a `Sound FX parameters` node names declares at its Input (#311), or
+ * the instrument an `Instrument parameters` node names at its Parameters node (#331).
+ */
 function soundFxChoices(project: Project, node: GraphNode): PropertyDef[] {
-  const fx = findAssetOf(project, soundFxSourceOf(node.params), 'soundfx');
-  const ports = fx ? (terminalNode(fx.graph, SOUND_FX_INPUT)?.ports ?? []) : [];
+  const ports = graphLookup(project).soundFx(soundFxSourceOf(node.params))?.params ?? [];
   return ports.map((p) => ({ key: p.key, label: p.key, kind: 'number', default: p.default ?? 0 }));
 }
 
@@ -112,18 +117,20 @@ export function buildSoundFxPortSection(
   node: GraphNode,
   addSetting: (def: PropertyDef) => void,
 ): void {
-  addSetting(SOUND_FX_SETTING);
+  const ofInstrument = node.type === INSTRUMENT_PARAMETER_VALUES;
+  addSetting(ofInstrument ? INSTRUMENT_SETTING : SOUND_FX_SETTING);
+  const noun = ofInstrument ? 'instrument' : 'sound FX';
   const id = soundFxSourceOf(node.params);
-  const fx = id ? findAssetOf(host.state.project, id, 'soundfx') : undefined;
+  const fx = id ? graphLookup(host.state.project).soundFx(id) : null;
   buildPickedPortSection(host, assetId, node, {
     noun: 'Parameter',
     choices: soundFxChoices(host.state.project, node),
-    owner: fx ? `"${fx.name}"` : 'a sound FX',
+    owner: fx ? `"${fx.name}"` : `a ${noun}`,
     empty: id
       ? fx
-        ? `"${fx.name}" declares no parameters yet. Add them at its Input.`
-        : 'That sound FX is gone.'
-      : 'Pick a sound FX first.',
+        ? `"${fx.name}" declares no parameters yet. Add them at its ${ofInstrument ? 'Parameters node' : 'Input'}.`
+        : `That ${noun} is gone.`
+      : `Pick ${ofInstrument ? 'an instrument' : 'a sound FX'} first.`,
   });
 }
 
@@ -276,7 +283,7 @@ export function buildPortSection(
   typeName: string | null = null,
 ): void {
   const onStart = isEntryTerminal(node.type);
-  const ofSoundFx = node.type === SOUND_FX_INPUT;
+  const ofSoundFx = declaresParameters(node.type);
   const noun = ofSoundFx ? 'Parameter' : isTerminalNode(node) ? (onStart ? 'Input' : 'Output') : 'Field';
   const ofFunction = node.type === SHADER_FUNCTION_INPUT || node.type === SHADER_FUNCTION_OUTPUT;
   const base = ofSoundFx ? 'param' : noun === 'Field' ? 'field' : onStart ? 'in' : 'out';
@@ -317,7 +324,11 @@ export function buildPortSection(
       noun === 'Field'
         ? 'No fields yet. Every field added here becomes a port of the struct.'
         : ofSoundFx
-          ? 'No parameters yet. Every parameter added here is a number with a default, and a Sound FX parameters node can set it when the sound is played.'
+          ? `No parameters yet. Every parameter added here is a number with a default, and ${
+              node.type === SOUND_FX_INPUT
+                ? 'a Sound FX parameters node can set it when the sound'
+                : 'an Instrument parameters node can set it when a note'
+            } is played.`
           : ofFunction
             ? `No ${plural(noun)} yet. Every ${noun.toLowerCase()} added here, a number or a color, becomes a port on the nodes calling this function.`
             : onStart

src/editor/instrument/InstrumentEditor.ts

@@ -0,0 +1,185 @@
+import {
+  instrumentBuildFor,
+  instrumentCheck,
+  instrumentParams,
+  instrumentSourcesOf,
+  playedNotes,
+} from '../../core/instrument/compile';
+import { noteName } from '../../core/instrument/notes';
+import type { Asset, InstrumentAsset } from '../../core/model/Asset';
+import { playNote } from '../../runtime/systems/behaviorContext';
+import { Button, HStack, Label, NumberField, Stack, Theme, VStack, type Widget } from '../../ui';
+import { soundLibrary } from '../../viewport/SoundLibrary';
+import { AssetTabEditor } from '../AssetTabEditor';
+import { GraphCanvas } from '../graph/GraphCanvas';
+import { SnapToggle } from '../graph/SnapToggle';
+import { TOOLBAR_ICON_SVG } from '../panels/toolbarIcons';
+import type { EditorState } from '../state';
+import { PianoKeyboard } from './PianoKeyboard';
+
+/** Room kept for "N notes", or for the reason the graph does not compile. */
+const INFO_WIDTH = 220;
+const PARAM_WIDTH = 48;
+/** Whose voices the tab's Stop silences: its own, never a running game's. */
+const PREVIEW_OWNER = 'instrument-preview';
+
+/**
+ * The nodes of one instrument (#331), and below them its keys: a note is heard while a key is held,
+ * by the pointer or by the computer keyboard. The header holds a value for each of its parameters –
+ * the tab's own, never written into the asset – and a Stop for everything the tab still plays.
+ */
+export class InstrumentEditor extends AssetTabEditor<InstrumentAsset, GraphCanvas> {
+  readonly snap = new SnapToggle(() => this.canvas);
+  /** Public for the check scripts. */
+  readonly keyboard = new PianoKeyboard();
+  readonly stopButton: Button;
+  private readonly octaveLabel = new Label('', { style: 'small' });
+  private readonly paramBar = new HStack({ gap: Theme.gap, align: 'center' });
+  private readonly paramFields = new Map<string, NumberField>();
+  private values: Record<string, number> = {};
+  private defaults: Record<string, number> = {};
+  private paramsKey = '';
+  private valuesFor = '';
+  /** The voices the tab started that may still be playing – what a moved field reaches. */
+  private voices: number[] = [];
+
+  constructor(state: EditorState) {
+    super(state, { infoWidth: INFO_WIDTH });
+    this.stopButton = new Button('Stop', { icon: TOOLBAR_ICON_SVG.stop, onClick: () => this.stop() });
+    this.keyboard.onNoteOn = (note) => this.playNote(note);
+    this.keyboard.onNoteOff = (voice) => soundLibrary.release(voice);
+    this.keyboard.onOctave = () => this.syncOctave();
+    this.mount(new GraphCanvas(state, 'instrument'), [
+      ...this.snap.widgets,
+      this.stopButton,
+      this.octaveLabel,
+      this.paramBar,
+    ]);
+    this.syncOctave();
+    state.on('load', () => {
+      this.valuesFor = '';
+    });
+  }
+
+  override selectedId(): string {
+    return this.state.selectedInstrumentId ?? '';
+  }
+
+  override isMine(asset: Asset): asset is InstrumentAsset {
+    return asset.type === 'instrument';
+  }
+
+  protected override body(canvas: GraphCanvas): Widget {
+    const column = new VStack();
+    column.add(canvas, 1);
+    column.add(this.keyboard);
+    return column;
+  }
+
+  /** What the header's parameter fields are set to – the check scripts read this. */
+  get previewValues(): Record<string, number> {
+    return { ...this.values };
+  }
+
+  setPreviewValue(key: string, value: number): void {
+    if (!(key in this.values)) return;
+    this.paramFields.get(key)?.setNumber(value);
+    this.setValue(key, value);
+  }
+
+  /** Starts one note as the keys do, held until it is released; gives back its voice. */
+  playNote(note: number): number {
+    const asset = this.asset();
+    if (!asset) return 0;
+    const struck = { note, velocity: 1, volume: 1, duration: 0 };
+    const voice = playNote(
+      { project: this.state.project, audio: soundLibrary },
+      asset.id,
+      struck,
+      PREVIEW_OWNER,
+      { ...this.values },
+    );
+    if (voice) this.voices.push(voice);
+    return voice;
+  }
+
+  stop(): void {
+    this.keyboard.releaseAll();
+    soundLibrary.stopOwner(PREVIEW_OWNER);
+    this.voices = [];
+  }
+
+  /** The notes the keys show as making a sound (test hook). */
+  soundingNotes(): number[] {
+    const asset = this.asset();
+    if (!asset) return [];
+    const sources = instrumentSourcesOf(this.state.project);
+    return [...playedNotes(asset.graph)]
+      .filter((note) => instrumentBuildFor(asset, note, sources).layers.length > 0)
+      .sort((a, b) => a - b);
+  }
+
+  private setValue(key: string, value: number): void {
+    this.values[key] = value;
+    this.voices = this.voices.filter((voice) => soundLibrary.tune(voice, { [key]: value }));
+  }
+
+  protected override infoText(asset: InstrumentAsset): string {
+    const build = instrumentCheck(asset, instrumentSourcesOf(this.state.project));
+    if (build.reason) return build.reason;
+    const count = this.soundingNotes().length;
+    return `${count} ${count === 1 ? 'note' : 'notes'}`;
+  }
+
+  protected override syncExtras(asset: InstrumentAsset | null): void {
+    this.snap.setEnabled(Boolean(asset));
+    this.stopButton.setEnabled(Boolean(asset));
+    this.keyboard.setEnabled(Boolean(asset));
+    if (!asset) this.keyboard.releaseAll();
+    this.keyboard.setSounding(new Set(this.soundingNotes()));
+    this.syncParams(asset);
+  }
+
+  private syncOctave(): void {
+    const octave = this.keyboard.bottomOctave;
+    this.octaveLabel.setText(`Keys ${noteName((octave + 1) * 12)} to ${noteName((octave + 3) * 12 + 4)}`);
+  }
+
+  /** As in the Sound FX tab: another asset starts from its defaults, an edit keeps a moved value. */
+  private syncParams(asset: InstrumentAsset | null): void {
+    const params = asset ? instrumentParams(asset.graph) : [];
+    const same = (asset?.id ?? '') === this.valuesFor;
+    this.valuesFor = asset?.id ?? '';
+    const values: Record<string, number> = {};
+    const defaults: Record<string, number> = {};
+    for (const port of params) {
+      const fallback = port.default ?? 0;
+      const kept = same && this.defaults[port.key] === fallback ? this.values[port.key] : undefined;
+      values[port.key] = kept ?? fallback;
+      defaults[port.key] = fallback;
+    }
+    this.values = values;
+    this.defaults = defaults;
+
+    const key = params.map((p) => p.key).join('\u0000');
+    if (key !== this.paramsKey) {
+      this.paramsKey = key;
+      for (const child of this.paramBar.removeChildren()) child.destroy({ children: true });
+      this.paramFields.clear();
+      for (const port of params) {
+        this.paramBar.add(new Label(port.key, { style: 'small' }));
+        const field = new NumberField(values[port.key], {
+          width: PARAM_WIDTH,
+          step: 0.1,
+          onInput: (value) => this.setValue(port.key, value),
+          onChange: (value) => this.setValue(port.key, value),
+        });
+        this.paramFields.set(port.key, field);
+        this.paramBar.add(field);
+      }
+      if (this.paramBar.parent instanceof Stack) this.paramBar.parent.relayout();
+      this.paramBar.relayout();
+    }
+    for (const [name, field] of this.paramFields) field.setNumber(this.values[name] ?? 0);
+  }
+}

src/editor/instrument/PianoKeyboard.ts

@@ -0,0 +1,185 @@
+import { type FederatedPointerEvent, Graphics, Text } from 'pixi.js';
+import { noteName } from '../../core/instrument/notes';
+import { textStyle, Theme, ui, Widget } from '../../ui';
+import {
+  DEFAULT_OCTAVE,
+  HIGHEST_OCTAVE,
+  type KeyRect,
+  keyboardLayout,
+  LOWEST_OCTAVE,
+  noteAt,
+  octaveStep,
+  trackerNote,
+} from './keys';
+
+const HEIGHT = 64;
+const WHITE = 0xe8e8e8;
+const BLACK = 0x22252b;
+const MUTED_WHITE = 0x6b6f78;
+const MUTED_BLACK = 0x3a3e46;
+/** Dark, to stand on a white key – its own style, since the shared ones are cached for everyone. */
+const KEY_LABEL = textStyle('small').clone();
+KEY_LABEL.fill = 0x30343c;
+
+/**
+ * The keys of an instrument's tab (#331): every note there is, the ones the instrument plays nothing
+ * for greyed out, a pressed one lit. A key sounds while the pointer holds it; with the keyboard
+ * focused, the computer keyboard plays it as a tracker does, and the arrows move its octave.
+ */
+export class PianoKeyboard extends Widget {
+  private readonly keysLayer = new Graphics();
+  private readonly labels: Text[] = [];
+  private keys: KeyRect[] = [];
+  private sounding: ReadonlySet<number> = new Set();
+  /** What each held note is held by – the pointer, or a key by its code – and the voice it started. */
+  private readonly held = new Map<string, { note: number; voice: number }>();
+  private octave = DEFAULT_OCTAVE;
+  private readonly onKeyUp = (event: KeyboardEvent) => this.up(event.code);
+  /** Starts a note and gives back its voice, 0 for none. */
+  onNoteOn: (note: number) => number = () => 0;
+  onNoteOff: (voice: number) => void = () => {};
+  onOctave: (octave: number) => void = () => {};
+
+  constructor() {
+    super();
+    this.focusable = true;
+    this.minHeight = HEIGHT;
+    this.cursor = Theme.hoverCursor;
+    this.addChild(this.keysLayer);
+    this.on('pointerdown', (e: FederatedPointerEvent) => {
+      ui().focus.focus(this);
+      this.press('pointer', this.noteUnder(e));
+      e.stopPropagation();
+    });
+    this.on('globalpointermove', (e: FederatedPointerEvent) => {
+      const holding = this.held.get('pointer');
+      if (!holding) return;
+      const note = this.noteUnder(e);
+      if (note !== holding.note) this.press('pointer', note);
+    });
+    this.onPointerRelease(() => this.up('pointer'));
+  }
+
+  /** The notes the instrument makes a sound for; the others are drawn greyed out. */
+  setSounding(notes: ReadonlySet<number>): void {
+    this.sounding = notes;
+    this.invalidate();
+  }
+
+  /** The notes held right now, lowest first (test hook). */
+  get heldNotes(): number[] {
+    return [...this.held.values()].map((h) => h.note).sort((a, b) => a - b);
+  }
+
+  get bottomOctave(): number {
+    return this.octave;
+  }
+
+  /** Where a note's key is on screen, its middle (test hook). */
+  keyCentre(note: number): { x: number; y: number } | null {
+    const key = this.keys.find((k) => k.note === note);
+    if (!key) return null;
+    const at = this.getGlobalPosition();
+    return { x: at.x + key.x + key.w / 2, y: at.y + key.y + key.h * (key.black ? 0.5 : 0.85) };
+  }
+
+  /** Lets go of every note, as losing the focus or leaving the tab does. */
+  releaseAll(): void {
+    for (const holder of [...this.held.keys()]) this.up(holder);
+  }
+
+  override onFocus(): void {
+    super.onFocus();
+    window.addEventListener('keyup', this.onKeyUp);
+  }
+
+  override onBlur(): void {
+    super.onBlur();
+    window.removeEventListener('keyup', this.onKeyUp);
+    this.releaseAll();
+  }
+
+  override onKeyDown(event: KeyboardEvent): boolean {
+    if (event.ctrlKey || event.metaKey || event.altKey) return false;
+    const step = octaveStep(event.code);
+    if (step !== 0) {
+      this.octave = Math.max(LOWEST_OCTAVE, Math.min(HIGHEST_OCTAVE, this.octave + step));
+      this.onOctave(this.octave);
+      this.invalidate();
+      return true;
+    }
+    const note = trackerNote(event.code, this.octave);
+    if (note === null) return false;
+    // A held key repeats; the note it started goes on until it is let go.
+    if (!event.repeat && !this.held.has(event.code)) this.press(event.code, note);
+    return true;
+  }
+
+  override destroy(options?: Parameters<Widget['destroy']>[0]): void {
+    window.removeEventListener('keyup', this.onKeyUp);
+    super.destroy(options);
+  }
+
+  override layout(): void {
+    this.keys = keyboardLayout(this.w, this.h);
+  }
+
+  override redraw(): void {
+    const g = this.keysLayer;
+    g.clear();
+    const lit = new Set(this.heldNotes);
+    const c = Theme.colors;
+    for (const key of this.keys) {
+      const sounds = this.sounding.has(key.note);
+      const fill = lit.has(key.note)
+        ? c.accent
+        : key.black
+          ? sounds
+            ? BLACK
+            : MUTED_BLACK
+          : sounds
+            ? WHITE
+            : MUTED_WHITE;
+      g.rect(key.x, key.y, key.w, key.h).fill(fill).stroke({ width: 1, color: c.border });
+    }
+    // The octave the bottom row of the computer keyboard plays, underlined.
+    const from = this.keys.find((k) => k.note === (this.octave + 1) * 12);
+    const to = this.keys.find((k) => k.note === Math.min(127, (this.octave + 2) * 12 + 16));
+    if (from && to && this.focused) g.rect(from.x, this.h - 3, to.x + to.w - from.x, 3).fill(c.accent);
+    this.drawLabels();
+  }
+
+  /** A name on every C, where there is room for one. */
+  private drawLabels(): void {
+    for (const label of this.labels.splice(0)) label.destroy();
+    const cs = this.keys.filter((k) => !k.black && k.note % 12 === 0);
+    if (cs.length === 0 || (cs[0]?.w ?? 0) * 7 < 28) return;
+    for (const key of cs) {
+      const label = new Text({ text: noteName(key.note), style: KEY_LABEL });
+      label.x = key.x + 1;
+      label.y = key.h - label.height - 2;
+      this.labels.push(label);
+      this.addChild(label);
+    }
+  }
+
+  private noteUnder(e: FederatedPointerEvent): number | null {
+    const at = this.toLocal(e.global);
+    return noteAt(this.keys, at.x, Math.min(at.y, this.h - 1));
+  }
+
+  private press(holder: string, note: number | null): void {
+    this.up(holder);
+    if (note === null) return;
+    this.held.set(holder, { note, voice: this.onNoteOn(note) });
+    this.invalidate();
+  }
+
+  private up(holder: string): void {
+    const holding = this.held.get(holder);
+    if (!holding) return;
+    this.held.delete(holder);
+    if (holding.voice) this.onNoteOff(holding.voice);
+    this.invalidate();
+  }
+}

src/editor/instrument/keys.ts

@@ -0,0 +1,106 @@
+import { HIGHEST_NOTE, isSharp, LOWEST_NOTE } from '../../core/instrument/notes';
+
+/** One key as drawn: where it is, and whether it is a black one lying over the white ones. */
+export interface KeyRect {
+  note: number;
+  x: number;
+  y: number;
+  w: number;
+  h: number;
+  black: boolean;
+}
+
+/** Which white key in its octave a natural note is: C 0, D 1 … B 6. */
+const WHITE_STEP = [0, -1, 1, -1, 2, 3, -1, 4, -1, 5, -1, 6];
+const BLACK_WIDTH = 0.6;
+const BLACK_HEIGHT = 0.62;
+
+function whiteIndex(note: number): number {
+  return Math.floor(note / 12) * 7 + WHITE_STEP[note % 12]!;
+}
+
+const WHITE_KEYS = whiteIndex(HIGHEST_NOTE) + 1;
+
+/**
+ * Every key of the 128 notes across `w` by `h`: the white keys side by side, each black key over the
+ * gap after the white one below it. White first, so a black one is drawn over them.
+ */
+export function keyboardLayout(w: number, h: number): KeyRect[] {
+  const white = w / WHITE_KEYS;
+  const keys: KeyRect[] = [];
+  for (let note = LOWEST_NOTE; note <= HIGHEST_NOTE; note++)
+    if (!isSharp(note)) keys.push({ note, x: whiteIndex(note) * white, y: 0, w: white, h, black: false });
+  for (let note = LOWEST_NOTE; note <= HIGHEST_NOTE; note++) {
+    if (!isSharp(note)) continue;
+    const gap = (whiteIndex(note - 1) + 1) * white;
+    const width = white * BLACK_WIDTH;
+    keys.push({ note, x: gap - width / 2, y: 0, w: width, h: h * BLACK_HEIGHT, black: true });
+  }
+  return keys;
+}
+
+/** The note under a point, a black key before the white one it lies on; null off the keys. */
+export function noteAt(keys: readonly KeyRect[], x: number, y: number): number | null {
+  const inside = (k: KeyRect) => x >= k.x && x < k.x + k.w && y >= k.y && y < k.y + k.h;
+  return (keys.find((k) => k.black && inside(k)) ?? keys.find((k) => !k.black && inside(k)))?.note ?? null;
+}
+
+/**
+ * The computer keyboard as a tracker has it, by where a key is rather than what it says, so QWERTZ and
+ * QWERTY play alike: the bottom row is one octave from C, the row above it the black keys, the row
+ * with Q the octave after, its black keys on the digits.
+ */
+const TRACKER_KEYS: Readonly<Record<string, number>> = {
+  KeyZ: 0,
+  KeyS: 1,
+  KeyX: 2,
+  KeyD: 3,
+  KeyC: 4,
+  KeyV: 5,
+  KeyG: 6,
+  KeyB: 7,
+  KeyH: 8,
+  KeyN: 9,
+  KeyJ: 10,
+  KeyM: 11,
+  Comma: 12,
+  KeyL: 13,
+  Period: 14,
+  Semicolon: 15,
+  Slash: 16,
+  KeyQ: 12,
+  Digit2: 13,
+  KeyW: 14,
+  Digit3: 15,
+  KeyE: 16,
+  KeyR: 17,
+  Digit5: 18,
+  KeyT: 19,
+  Digit6: 20,
+  KeyY: 21,
+  Digit7: 22,
+  KeyU: 23,
+  KeyI: 24,
+  Digit9: 25,
+  KeyO: 26,
+  Digit0: 27,
+  KeyP: 28,
+};
+
+/** The octave the bottom row starts in at first: Z is C3, Q is C4. */
+export const DEFAULT_OCTAVE = 3;
+export const LOWEST_OCTAVE = -1;
+export const HIGHEST_OCTAVE = 8;
+
+/** The note a key plays with the bottom row starting at C of `octave`; null for no note key. */
+export function trackerNote(code: string, octave: number): number | null {
+  const step = TRACKER_KEYS[code];
+  if (step === undefined) return null;
+  const note = (octave + 1) * 12 + step;
+  return note >= LOWEST_NOTE && note <= HIGHEST_NOTE ? note : null;
+}
+
+/** Which way a key moves the octave: the arrows left and right. */
+export function octaveStep(code: string): -1 | 1 | 0 {
+  return code === 'ArrowLeft' ? -1 : code === 'ArrowRight' ? 1 : 0;
+}

src/editor/panels/assetGroups.ts

@@ -36,6 +36,8 @@ export const ASSET_GROUP_LABELS: Record<AssetType, string> = {
   sound: 'Sounds',
   // Not "Sound FX": that is the `+` menu entry (see Vector art).
   soundfx: 'Sound effects',
+  // Where patterns and songs join it (#330).
+  instrument: 'Music',
   particles: 'Particles',
 };
 

src/editor/panels/assetIcons.ts

@@ -14,6 +14,7 @@ import font from './icons/font.svg?raw';
 import game from './icons/game.svg?raw';
 import group from './icons/group.svg?raw';
 import list from './icons/list.svg?raw';
+import instrument from './icons/instrument.svg?raw';
 import joint from './icons/joint.svg?raw';
 import palette from './icons/palette.svg?raw';
 import particles from './icons/particles.svg?raw';
@@ -45,6 +46,7 @@ export const ASSET_ICON_SVG: Record<AssetType, string> = {
   container,
   curve,
   font,
+  instrument,
   joint,
   palette,
   particles,

src/editor/panels/icons/instrument.svg

@@ -0,0 +1,7 @@
+<svg xmlns="http://www.w3.org/2000/svg" viewBox="-8 -8 16 16" width="16" height="16">
+  <!-- Keys of a keyboard: what a note is played on. -->
+  <rect x="-6.5" y="-5.5" width="13" height="11" rx="1.2" fill="none" stroke="currentColor" stroke-width="1.25" />
+  <path d="M-2.2 -5.5 V5.5 M2.2 -5.5 V5.5" fill="none" stroke="currentColor" stroke-width="1" />
+  <rect x="-3.6" y="-5.5" width="2.8" height="5.5" fill="currentColor" />
+  <rect x="0.8" y="-5.5" width="2.8" height="5.5" fill="currentColor" />
+</svg>

src/editor/state.ts

@@ -815,6 +815,11 @@ export class EditorState {
     return this.selectedGraphId('soundfx');
   }
 
+  /** The instrument the selection refers to: the asset, or one of its nodes. */
+  get selectedInstrumentId(): string | null {
+    return this.selectedGraphId('instrument');
+  }
+
   private selectedGraphId(type: GraphAsset['type']): string | null {
     const s = this.selection;
     if (!s) return null;