Play sounds as notes of an instrument asset #331
A sound could only be played as it was, one at a time. An instrument
is a node graph that makes a note a sound: Notes nodes bring the note
in, Samples play sound assets for it, the effects of a sound FX shape
it and the Output sums the layers. Its tab plays it from a keyboard,
a behavior with Play note. It is what patterns and songs (#330) will
play.
- New asset type instrument, under the heading Music, not placeable.
A new one starts as Notes into a Sample into the Output.
- Notes are counted as MIDI does: 60 is C4, names like C#4, sharps
only, the lowest C-1. Notes has From and To, outputs any, note,
frequency and velocity, and with Single notes a trigger per note,
each with an optional name for a drum kit.
- New port type trigger, only into the trigger of a source. Inputs
can take any number of edges now (PortDef.many): a trigger, and
the Output, which plays everything into it side by side.
- Sample plays a sound asset. Follow note shifts it by note - root
through the pitch worklet of #317, so it keeps its length. Gate
fades it out at the end of the note, One-shot plays through. Loop
runs between Loop start and Loop end, also in the worklet.
- Parameters declares the instrument's own parameters with a
default. Sound FX sends the sound through a sound FX asset, its
parameters as inputs, the sound set in where its Input was.
- compileInstrument walks the graph once per note. A trigger is on
only for the notes it stands for, a silent source silences what it
alone feeds, a Mix with a silent side is the other side. velocity
and the parameters stay open for the voice. Builds are cached per
note until the graph or a sound or FX it names changes.
- The player takes a note as one voice of several layers, each way
with its own file and volume. release fades the gated ways out,
one-shots play on. playNote can release after a duration.
- Play note plays one note with velocity, volume, duration and
detached, and hands out voice for Set sound FX parameters and
Stop sound. Velocity is always loudness. Instrument parameters
picks values for the instrument's parameters like Sound FX
parameters does for an FX, renames follow.
- The tab has the graph, a header field per parameter, Stop, and
128 keys below. Keys without sound are grey. A key plays while it
is held. Focused, the computer keyboard plays as in a tracker by
KeyboardEvent.code, the arrows move the octave.
Findings:
- A label of the keys set fill on the shared small text style and
darkened every small label of the editor. Caught in a screenshot
before the commit, the keys have a style of their own now.
- Suite task 523 on cf6799a failed two checks of e2e 34 that timed
audio by the page's clock: a 2.95 s echo tail was gone after 1 s,
and Quad in was further along than 0.7 s allow. Both time on the
audio clock now through a new hook __gm.sound.clock().
Tests:
- instrument.test.ts (new, 22): notes, asset, palette, round trip,
compile per note, drum kit, zones, Mix, velocity and parameters,
Sound FX node with rename, problems, cache, Play note, the keys
- sound-library +7 (layers, release, duration, early release, loop
region in buffer and worklet), pitch-player +2 (loop region),
node-catalog +4, graph adjusted. 582 tests in the touched files.
- e2e 4 and 34 on melektaus-atx: ALL PASSED, 34 with 44 checks (+7)
- Gate on melektaus-atx: CHECK EXIT 0, 243 test files, 3750 tests
- Counter-checks: any always on, release ending one-shots, Sound FX
inputs ignored, loop region ignored. 3, 2, 1 and 1 tests failed.
- Browser probe on stein2: keys, held key, Single notes names and
the header looked at in screenshots.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
cf6799a784
commit
8a4e4a6762
54 files changed
+2561
-143
No files matched your search
@@ -33,6 +33,17 @@ export const SOUND_FX_INPUT = 'SoundFxInput';
|
||||
export const SOUND_FX_OUTPUT = 'SoundFxOutput';
|
||||
export const SOUND_FX_PARAMETERS = 'SoundFxParameters';
|
||||
const PLAY_SOUND = 'PlaySound';
|
||||
/**
|
||||
* The nodes of an instrument's graph that are its own (#331), and the behavior node naming its
|
||||
* parameters. The effects it shares with a sound FX keep their types.
|
||||
*/
|
||||
export const INSTRUMENT_NOTES = 'InstrumentNotes';
|
||||
export const INSTRUMENT_SAMPLE = 'InstrumentSample';
|
||||
export const INSTRUMENT_SOUND_FX = 'InstrumentSoundFx';
|
||||
export const INSTRUMENT_PARAMETERS = 'InstrumentParameters';
|
||||
export const INSTRUMENT_OUTPUT = 'InstrumentOutput';
|
||||
export const INSTRUMENT_PARAMETER_VALUES = 'InstrumentParameterValues';
|
||||
const PLAY_NOTE = 'PlayNote';
|
||||
|
||||
export const BEHAVIOR_PARAM = 'behavior';
|
||||
export const FUNCTION_PARAM = 'function';
|
||||
@@ -48,6 +59,14 @@ export const SOUND_FX_SETTING: PropertyDef = {
|
||||
assetTypes: ['soundfx'],
|
||||
};
|
||||
|
||||
/** Whose parameters an `Instrument parameters` node sets: the same key, another kind of asset. */
|
||||
export const INSTRUMENT_SETTING: PropertyDef = {
|
||||
...SOUND_FX_SETTING,
|
||||
label: 'Instrument',
|
||||
assetTypes: ['instrument'],
|
||||
};
|
||||
|
||||
/** The sound FX – or the instrument (#331) – whose parameters a node names. */
|
||||
export function soundFxSourceOf(params: PropertyBag): string {
|
||||
return textOf(params[SOUND_FX_PARAM] ?? '');
|
||||
}
|
||||
@@ -90,9 +109,14 @@ export function declaresPropertyPorts(type: string): boolean {
|
||||
return type === PROPERTIES;
|
||||
}
|
||||
|
||||
/** Nodes whose declared ports are parameters of the sound FX they name. */
|
||||
/** Nodes whose declared ports are parameters of the sound FX or the instrument they name. */
|
||||
export function declaresSoundFxPorts(type: string): boolean {
|
||||
return type === SOUND_FX_PARAMETERS;
|
||||
return type === SOUND_FX_PARAMETERS || type === INSTRUMENT_PARAMETER_VALUES;
|
||||
}
|
||||
|
||||
/** The node declaring the parameters of a sound FX or an instrument, each with a default. */
|
||||
export function declaresParameters(type: string): boolean {
|
||||
return type === SOUND_FX_INPUT || type === INSTRUMENT_PARAMETERS;
|
||||
}
|
||||
|
||||
/** Identifier-like, and not `exec`: that key belongs to the fixed flow port of every terminal node. */
|
||||
@@ -153,7 +177,9 @@ export function isTerminalType(type: string): boolean {
|
||||
type === SHADER_FUNCTION_INPUT ||
|
||||
type === SHADER_FUNCTION_OUTPUT ||
|
||||
type === SOUND_FX_INPUT ||
|
||||
type === SOUND_FX_OUTPUT
|
||||
type === SOUND_FX_OUTPUT ||
|
||||
type === INSTRUMENT_PARAMETERS ||
|
||||
type === INSTRUMENT_OUTPUT
|
||||
);
|
||||
}
|
||||
|
||||
@@ -381,7 +407,8 @@ export function ghostProblem(node: GraphNode, graph: Graph, def: NodeDef): strin
|
||||
export function callProblem(node: GraphNode, lookup: GraphLookup, ownerId = ''): string | null {
|
||||
if (node.type === ON_BEHAVIOR) return listenProblem(node, lookup, ownerId);
|
||||
if (node.type === SHADER_CALL) return functionCallProblem(node, lookup, ownerId);
|
||||
if (node.type === SOUND_FX_PARAMETERS) return soundFxParametersProblem(node, lookup);
|
||||
if (declaresSoundFxPorts(node.type)) return soundFxParametersProblem(node, lookup);
|
||||
if (node.type === INSTRUMENT_SOUND_FX) return nestedFxProblem(node, lookup);
|
||||
if (node.type !== RUN_BEHAVIOR) return easingProblem(node, lookup);
|
||||
const id = textOf(node.params[BEHAVIOR_PARAM] ?? '');
|
||||
if (!id) return 'No behavior picked';
|
||||
@@ -427,7 +454,15 @@ function functionCallProblem(node: GraphNode, lookup: GraphLookup, ownerId: stri
|
||||
}
|
||||
|
||||
function soundFxParametersProblem(node: GraphNode, lookup: GraphLookup): string | null {
|
||||
const noun = node.type === INSTRUMENT_PARAMETER_VALUES ? 'instrument' : 'sound FX';
|
||||
const id = soundFxSourceOf(node.params);
|
||||
if (!id) return `No ${noun} picked`;
|
||||
return lookup.soundFx(id) ? null : `That ${noun} is gone`;
|
||||
}
|
||||
|
||||
/** A Sound FX node in an instrument (#331) without its FX lets the sound through untouched. */
|
||||
function nestedFxProblem(node: GraphNode, lookup: GraphLookup): string | null {
|
||||
const id = textOf(node.params.fx ?? '');
|
||||
if (!id) return 'No sound FX picked';
|
||||
return lookup.soundFx(id) ? null : 'That sound FX is gone';
|
||||
}
|
||||
@@ -438,13 +473,17 @@ function soundFxParametersProblem(node: GraphNode, lookup: GraphLookup): string
|
||||
* are picked in place – one coming over an edge may be either.
|
||||
*/
|
||||
export function soundFxParamsMismatch(node: GraphNode, graph: Graph): string | null {
|
||||
if (node.type !== PLAY_SOUND || edgesInto(graph, node.id, 'fx').length > 0) return null;
|
||||
const played = textOf(node.params.fx ?? '');
|
||||
const playsNote = node.type === PLAY_NOTE;
|
||||
if (node.type !== PLAY_SOUND && !playsNote) return null;
|
||||
const pick = playsNote ? 'instrument' : 'fx';
|
||||
if (edgesInto(graph, node.id, pick).length > 0) return null;
|
||||
const played = textOf(node.params[pick] ?? '');
|
||||
const edge = edgesInto(graph, node.id, 'fxParams')[0];
|
||||
const from = edge && findNode(graph, edge.from.node);
|
||||
if (!played || from?.type !== SOUND_FX_PARAMETERS) return null;
|
||||
if (!played || !from || !declaresSoundFxPorts(from.type)) return null;
|
||||
const named = soundFxSourceOf(from.params);
|
||||
return named && named !== played ? 'The parameters are for another sound FX' : null;
|
||||
if (!named || named === played) return null;
|
||||
return playsNote ? 'The parameters are for another instrument' : 'The parameters are for another sound FX';
|
||||
}
|
||||
|
||||
/** The ids the nodes of one type in that graph name in `param`, picked or not found alike. */
|
||||
|
||||
Reference in new issue
Block a user