Play sounds as notes of an instrument asset #331
A sound could only be played as it was, one at a time. An instrument
is a node graph that makes a note a sound: Notes nodes bring the note
in, Samples play sound assets for it, the effects of a sound FX shape
it and the Output sums the layers. Its tab plays it from a keyboard,
a behavior with Play note. It is what patterns and songs (#330) will
play.
- New asset type instrument, under the heading Music, not placeable.
A new one starts as Notes into a Sample into the Output.
- Notes are counted as MIDI does: 60 is C4, names like C#4, sharps
only, the lowest C-1. Notes has From and To, outputs any, note,
frequency and velocity, and with Single notes a trigger per note,
each with an optional name for a drum kit.
- New port type trigger, only into the trigger of a source. Inputs
can take any number of edges now (PortDef.many): a trigger, and
the Output, which plays everything into it side by side.
- Sample plays a sound asset. Follow note shifts it by note - root
through the pitch worklet of #317, so it keeps its length. Gate
fades it out at the end of the note, One-shot plays through. Loop
runs between Loop start and Loop end, also in the worklet.
- Parameters declares the instrument's own parameters with a
default. Sound FX sends the sound through a sound FX asset, its
parameters as inputs, the sound set in where its Input was.
- compileInstrument walks the graph once per note. A trigger is on
only for the notes it stands for, a silent source silences what it
alone feeds, a Mix with a silent side is the other side. velocity
and the parameters stay open for the voice. Builds are cached per
note until the graph or a sound or FX it names changes.
- The player takes a note as one voice of several layers, each way
with its own file and volume. release fades the gated ways out,
one-shots play on. playNote can release after a duration.
- Play note plays one note with velocity, volume, duration and
detached, and hands out voice for Set sound FX parameters and
Stop sound. Velocity is always loudness. Instrument parameters
picks values for the instrument's parameters like Sound FX
parameters does for an FX, renames follow.
- The tab has the graph, a header field per parameter, Stop, and
128 keys below. Keys without sound are grey. A key plays while it
is held. Focused, the computer keyboard plays as in a tracker by
KeyboardEvent.code, the arrows move the octave.
Findings:
- A label of the keys set fill on the shared small text style and
darkened every small label of the editor. Caught in a screenshot
before the commit, the keys have a style of their own now.
- Suite task 523 on cf6799a failed two checks of e2e 34 that timed
audio by the page's clock: a 2.95 s echo tail was gone after 1 s,
and Quad in was further along than 0.7 s allow. Both time on the
audio clock now through a new hook __gm.sound.clock().
Tests:
- instrument.test.ts (new, 22): notes, asset, palette, round trip,
compile per note, drum kit, zones, Mix, velocity and parameters,
Sound FX node with rename, problems, cache, Play note, the keys
- sound-library +7 (layers, release, duration, early release, loop
region in buffer and worklet), pitch-player +2 (loop region),
node-catalog +4, graph adjusted. 582 tests in the touched files.
- e2e 4 and 34 on melektaus-atx: ALL PASSED, 34 with 44 checks (+7)
- Gate on melektaus-atx: CHECK EXIT 0, 243 test files, 3750 tests
- Counter-checks: any always on, release ending one-shots, Sound FX
inputs ignored, loop region ignored. 3, 2, 1 and 1 tests failed.
- Browser probe on stein2: keys, held key, Single notes names and
the header looked at in screenshots.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed
1 parent
cf6799a784
commit
8a4e4a6762
54 files changed
+2561
-143
No files matched your search
@@ -79,10 +79,10 @@ describe('node registry', () => {
|
||||
if (def.category === 'event') expect(def.inputs.filter((p) => p.type === 'exec')).toHaveLength(0);
|
||||
// A state is a name in a machine; its ports only anchor transitions, so it never runs.
|
||||
else if (def.category === 'state') expect(def.run).toBeUndefined();
|
||||
// A shader node is compiled to GLSL, a particle node to a function and a sound FX node to a
|
||||
// stage of a voice: none is ever run.
|
||||
// A shader node is compiled to GLSL, a particle node to a function, a sound FX node to a
|
||||
// stage of a voice and an instrument's own node to part of a note: none is ever run.
|
||||
else if (def.category === 'shader' || def.category === 'particles' || def.category === 'sound')
|
||||
expect('emit' in def).toBe(true);
|
||||
expect('emit' in def || 'instrument' in def).toBe(true);
|
||||
// The end of a block runs as part of the node that opens it (`Find` … `Match`).
|
||||
else if (defs.some((d) => d.opens?.type === def.type)) expect(def.run).toBeUndefined();
|
||||
// Everything else does one of the two: a step (`run`), or an answer whenever asked (`read`).
|
||||
|
||||
@@ -0,0 +1,485 @@
|
||||
/**
|
||||
* An instrument (#331) is a graph that makes a note a sound: Notes nodes bring the note in, Samples
|
||||
* play sounds for it, the effects of a sound FX shape them and the Output sums them. It is compiled
|
||||
* once per note; a trigger is on only for the notes it stands for.
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { CreateAssetCommand, CreateSoundCommand } from '../../src/core/commands/assetCommands';
|
||||
import { ConnectCommand } from '../../src/core/commands/graphCommands';
|
||||
import {
|
||||
AddPortCommand,
|
||||
RenamePortCommand,
|
||||
SetPortDefaultCommand,
|
||||
} from '../../src/core/commands/portCommands';
|
||||
import { callProblem, resolveDef, soundFxParamsMismatch } from '../../src/core/graph';
|
||||
import { createEmptyGraph, portsCompatible } from '../../src/core/graph/Graph';
|
||||
import {
|
||||
compileInstrument,
|
||||
instrumentBuildFor,
|
||||
instrumentSourcesOf,
|
||||
} from '../../src/core/instrument/compile';
|
||||
import { VELOCITY_PARAM } from '../../src/core/instrument/nodes';
|
||||
import { heldNote, noteFrequency, noteName } from '../../src/core/instrument/notes';
|
||||
import type { BehaviorAsset, InstrumentAsset, SoundFxAsset } from '../../src/core/model/Asset';
|
||||
import { isPlaceableType } from '../../src/core/model/AssetTypes';
|
||||
import { graphLookup } from '../../src/core/model/graphLookup';
|
||||
import { fxChainOf, type FxPlan } from '../../src/core/soundfx/nodes';
|
||||
import { paletteEntries } from '../../src/editor/graph/paletteEntries';
|
||||
import { keyboardLayout, noteAt, trackerNote } from '../../src/editor/instrument/keys';
|
||||
import type { AudioSink } from '../../src/runtime/systems/Audio';
|
||||
import { playNote } from '../../src/runtime/systems/behaviorContext';
|
||||
import { graphEditor, makeProject, type ProjectFixture, roundTrip } from '../helpers';
|
||||
|
||||
const KICK = `${'a1'.repeat(32)}.wav`;
|
||||
const SNARE = `${'b2'.repeat(32)}.wav`;
|
||||
|
||||
function makeInstrument(f: ProjectFixture = makeProject()) {
|
||||
const command = new CreateAssetCommand('instrument', 'Piano');
|
||||
f.stack.execute(command);
|
||||
const instrument = command.asset as InstrumentAsset;
|
||||
const sound = (name: string, file: string) => {
|
||||
const made = new CreateSoundCommand(name, file);
|
||||
f.stack.execute(made);
|
||||
return made.asset;
|
||||
};
|
||||
return { ...f, instrument, sound, edit: graphEditor(f, instrument as unknown as BehaviorAsset) };
|
||||
}
|
||||
|
||||
/** An instrument with nothing in it yet, for a test that builds its own graph. */
|
||||
function emptyInstrument() {
|
||||
const f = makeInstrument();
|
||||
f.instrument.graph = createEmptyGraph();
|
||||
return f;
|
||||
}
|
||||
|
||||
const nodeOf = (f: { instrument: InstrumentAsset }, type: string) =>
|
||||
f.instrument.graph.nodes.find((n) => n.type === type)!;
|
||||
|
||||
const compile = (f: ProjectFixture & { instrument: InstrumentAsset }, note: number) =>
|
||||
compileInstrument(f.instrument.graph, note, instrumentSourcesOf(f.project));
|
||||
|
||||
const way = (plan: FxPlan | undefined) => (plan && 'pitch' in plan ? plan : undefined);
|
||||
|
||||
describe('notes', () => {
|
||||
it('count as MIDI does: C4 is 60, A4 is 440 Hz, the lowest is C-1, sharps only', () => {
|
||||
expect(noteName(60)).toBe('C4');
|
||||
expect(noteName(61)).toBe('C#4');
|
||||
expect(noteName(0)).toBe('C-1');
|
||||
expect(noteName(127)).toBe('G9');
|
||||
expect(noteFrequency(69)).toBe(440);
|
||||
expect(noteFrequency(81)).toBeCloseTo(880, 9);
|
||||
expect(heldNote(200)).toBe(127);
|
||||
expect(heldNote(Number.NaN)).toBe(60);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the instrument asset', () => {
|
||||
it('is not placeable and starts as Notes into a Sample into the Output', () => {
|
||||
expect(isPlaceableType('instrument')).toBe(false);
|
||||
const f = makeInstrument();
|
||||
expect(f.instrument.graph.nodes.map((n) => n.type)).toEqual([
|
||||
'InstrumentNotes',
|
||||
'InstrumentSample',
|
||||
'InstrumentOutput',
|
||||
]);
|
||||
expect(f.instrument.graph.edges.map((e) => `${e.from.port}>${e.to.port}`)).toEqual([
|
||||
'any>trigger',
|
||||
'out>in',
|
||||
]);
|
||||
// Its Sample has no sound yet, and says so.
|
||||
expect(compile(f, 60)).toMatchObject({ layers: [], reason: 'No sound picked' });
|
||||
});
|
||||
|
||||
it('survives a round trip, keeping its own nodes and the effects it shares with a sound FX', () => {
|
||||
const f = makeInstrument();
|
||||
f.edit.addNode('SoundFxGain', { gain: -6 });
|
||||
const back = roundTrip(f.project).assets.find((a) => a.id === f.instrument.id) as InstrumentAsset;
|
||||
expect(back.graph.nodes.map((n) => n.type)).toEqual([
|
||||
'InstrumentNotes',
|
||||
'InstrumentSample',
|
||||
'InstrumentOutput',
|
||||
'SoundFxGain',
|
||||
]);
|
||||
expect(back.graph.edges).toHaveLength(2);
|
||||
});
|
||||
|
||||
it('offers its own nodes and the effects, but no Input and no second Output', () => {
|
||||
const f = makeInstrument();
|
||||
const offered = paletteEntries(f.project, f.instrument).map((e) => e.id);
|
||||
expect(offered).toEqual(
|
||||
expect.arrayContaining(['InstrumentNotes', 'InstrumentSample', 'InstrumentParameters', 'SoundFxDelay']),
|
||||
);
|
||||
expect(offered).not.toContain('InstrumentOutput');
|
||||
expect(offered).not.toContain('SoundFxInput');
|
||||
expect(offered).not.toContain('SoundFxOutput');
|
||||
});
|
||||
});
|
||||
|
||||
describe('compiling a note', () => {
|
||||
it('plays a Sample at its root as it is, and other notes shifted by the distance', () => {
|
||||
const f = makeInstrument();
|
||||
const piano = f.sound('Piano C4', KICK);
|
||||
nodeOf(f, 'InstrumentSample').params.sound = piano.id;
|
||||
const root = compile(f, 60);
|
||||
expect(root.reason).toBe('');
|
||||
expect(root.layers).toHaveLength(1);
|
||||
expect(way(root.layers[0])).toMatchObject({
|
||||
pitch: [],
|
||||
source: { file: KICK, volume: 1, loop: false, gate: true },
|
||||
});
|
||||
expect(way(compile(f, 64).layers[0])?.pitch).toEqual([{ value: 4 }]);
|
||||
expect(way(compile(f, 67).layers[0])?.pitch).toEqual([{ value: 7 }]);
|
||||
});
|
||||
|
||||
it('plays nothing for a note outside the range of its Notes node – silence, not a problem', () => {
|
||||
const f = makeInstrument();
|
||||
nodeOf(f, 'InstrumentSample').params.sound = f.sound('Piano', KICK).id;
|
||||
expect(compile(f, 72)).toEqual({ layers: [], problems: {}, reason: '' });
|
||||
expect(compile(f, 59).layers).toEqual([]);
|
||||
expect(compile(f, 71).layers).toHaveLength(1);
|
||||
});
|
||||
|
||||
it('plays a Sample that does not follow the note the same for every note', () => {
|
||||
const f = makeInstrument();
|
||||
Object.assign(nodeOf(f, 'InstrumentSample').params, {
|
||||
sound: f.sound('Kick', KICK).id,
|
||||
followNote: false,
|
||||
});
|
||||
expect(way(compile(f, 67).layers[0])?.pitch).toEqual([]);
|
||||
});
|
||||
|
||||
it('a drum kit: one Notes node with single notes, a Sample on each, both into the Output', () => {
|
||||
const f = emptyInstrument();
|
||||
const notes = f.edit.addNode('InstrumentNotes', {
|
||||
from: '36',
|
||||
to: '47',
|
||||
singleNotes: true,
|
||||
name36: 'Kick',
|
||||
});
|
||||
const kick = f.edit.addNode('InstrumentSample', { sound: f.sound('Kick', KICK).id, followNote: false });
|
||||
const snare = f.edit.addNode('InstrumentSample', { sound: f.sound('Snare', SNARE).id, mode: 'oneshot' });
|
||||
const output = f.edit.addNode('InstrumentOutput');
|
||||
f.edit.connect(notes, 'n36', kick, 'trigger');
|
||||
f.edit.connect(notes, 'n38', snare, 'trigger');
|
||||
f.edit.connect(kick, 'out', output, 'in');
|
||||
f.edit.connect(snare, 'out', output, 'in');
|
||||
// The Output takes both: a second edge into it does not replace the first.
|
||||
expect(f.instrument.graph.edges.filter((e) => e.to.node === output.id)).toHaveLength(2);
|
||||
const files = (note: number) => compile(f, note).layers.map((plan) => way(plan)?.source?.file);
|
||||
expect(files(36)).toEqual([KICK]);
|
||||
expect(files(38)).toEqual([SNARE]);
|
||||
expect(files(37)).toEqual([]);
|
||||
expect(way(compile(f, 38).layers[0])?.source?.gate).toBe(false);
|
||||
// The node names its output after the note's own name, the others after the note.
|
||||
const def = resolveDef(notes, f.instrument.graph, graphLookup(f.project))!;
|
||||
expect(def.outputs.map((p) => p.label).slice(0, 7)).toEqual([
|
||||
'any',
|
||||
'note',
|
||||
'frequency',
|
||||
'velocity',
|
||||
'Kick',
|
||||
'C#2',
|
||||
'D2',
|
||||
]);
|
||||
expect(def.outputs).toHaveLength(4 + 12);
|
||||
});
|
||||
|
||||
it('two Notes nodes over one Sample: a trigger takes more than one edge, either one plays it', () => {
|
||||
const f = emptyInstrument();
|
||||
const low = f.edit.addNode('InstrumentNotes', { from: '48', to: '52' });
|
||||
const high = f.edit.addNode('InstrumentNotes', { from: '70', to: '72' });
|
||||
const sample = f.edit.addNode('InstrumentSample', { sound: f.sound('Piano', KICK).id });
|
||||
const output = f.edit.addNode('InstrumentOutput');
|
||||
f.edit.wire([
|
||||
[low, 'any', sample, 'trigger'],
|
||||
[high, 'any', sample, 'trigger'],
|
||||
[sample, 'out', output, 'in'],
|
||||
]);
|
||||
expect(compile(f, 50).layers).toHaveLength(1);
|
||||
expect(compile(f, 71).layers).toHaveLength(1);
|
||||
expect(compile(f, 60).layers).toHaveLength(0);
|
||||
});
|
||||
|
||||
it('a Mix with one silent side is its other side', () => {
|
||||
const f = emptyInstrument();
|
||||
const notes = f.edit.addNode('InstrumentNotes', { from: '60', to: '62', singleNotes: true });
|
||||
const a = f.edit.addNode('InstrumentSample', { sound: f.sound('A', KICK).id });
|
||||
const b = f.edit.addNode('InstrumentSample', { sound: f.sound('B', SNARE).id });
|
||||
const mix = f.edit.addNode('SoundFxMix', { mix: 0.5 });
|
||||
const output = f.edit.addNode('InstrumentOutput');
|
||||
f.edit.wire([
|
||||
[notes, 'n60', a, 'trigger'],
|
||||
[notes, 'any', b, 'trigger'],
|
||||
[a, 'out', mix, 'a'],
|
||||
[b, 'out', mix, 'b'],
|
||||
[mix, 'out', output, 'in'],
|
||||
]);
|
||||
const both = compile(f, 60).layers[0]!;
|
||||
expect('mix' in both).toBe(true);
|
||||
const only = compile(f, 61).layers[0];
|
||||
expect(way(only)?.source?.file).toBe(SNARE);
|
||||
});
|
||||
|
||||
it('leaves velocity and the parameters open for the voice, and fills in note and frequency', () => {
|
||||
const f = makeInstrument();
|
||||
nodeOf(f, 'InstrumentSample').params.sound = f.sound('Piano', KICK).id;
|
||||
const params = f.edit.addNode('InstrumentParameters');
|
||||
f.stack.execute(new AddPortCommand(f.instrument.id, params.id, { key: 'level', kind: 'number' }));
|
||||
f.stack.execute(new SetPortDefaultCommand(f.instrument.id, params.id, 'level', -3));
|
||||
const sample = nodeOf(f, 'InstrumentSample');
|
||||
const output = nodeOf(f, 'InstrumentOutput');
|
||||
const notes = nodeOf(f, 'InstrumentNotes');
|
||||
f.instrument.graph.edges = f.instrument.graph.edges.filter((e) => e.to.node !== output.id);
|
||||
const byVelocity = f.edit.addNode('SoundFxGain');
|
||||
const byParam = f.edit.addNode('SoundFxGain');
|
||||
const byNote = f.edit.addNode('SoundFxEq');
|
||||
f.stack.execute(
|
||||
new ConnectCommand(
|
||||
f.instrument.id,
|
||||
{ node: sample.id, port: 'out' },
|
||||
{ node: byVelocity.id, port: 'in' },
|
||||
),
|
||||
);
|
||||
f.edit.wire([
|
||||
[notes, 'velocity', byVelocity, 'gain'],
|
||||
[byVelocity, 'out', byParam, 'in'],
|
||||
[params, 'level', byParam, 'gain'],
|
||||
[byParam, 'out', byNote, 'in'],
|
||||
[notes, 'note', byNote, 'high'],
|
||||
[byNote, 'out', output, 'in'],
|
||||
]);
|
||||
const plan = compile(f, 62).layers[0]!;
|
||||
expect(plan.stages).toEqual([
|
||||
{ kind: 'gain', db: { value: 1, param: VELOCITY_PARAM, min: -120, max: 24 } },
|
||||
{ kind: 'gain', db: { value: -3, param: 'level', min: -120, max: 24 } },
|
||||
// The note is a number by now: 62, held to the band's row.
|
||||
{ kind: 'filter', filter: 'highshelf', frequency: 12000, gain: { value: 24 }, q: 1 },
|
||||
]);
|
||||
const chain = fxChainOf(plan, { [VELOCITY_PARAM]: -6, level: -9 });
|
||||
expect(chain.stages.map((s) => ('gain' in s ? s.gain : 0))).toEqual([
|
||||
10 ** (-6 / 20),
|
||||
10 ** (-9 / 20),
|
||||
24,
|
||||
]);
|
||||
});
|
||||
|
||||
it('sends a sound through a sound FX, its parameters bound to the inputs of the node', () => {
|
||||
const f = makeInstrument();
|
||||
nodeOf(f, 'InstrumentSample').params.sound = f.sound('Piano', KICK).id;
|
||||
// The FX: Input, a Gain fed by its parameter "depth" (default -6), a Pitch of 12, Output.
|
||||
const made = new CreateAssetCommand('soundfx', 'Cave');
|
||||
f.stack.execute(made);
|
||||
const fx = made.asset as SoundFxAsset;
|
||||
const fxEdit = graphEditor(f, fx as unknown as BehaviorAsset);
|
||||
const input = fxEdit.addNode('SoundFxInput');
|
||||
f.stack.execute(new AddPortCommand(fx.id, input.id, { key: 'depth', kind: 'number' }));
|
||||
f.stack.execute(new SetPortDefaultCommand(fx.id, input.id, 'depth', -6));
|
||||
const gain = fxEdit.addNode('SoundFxGain');
|
||||
const pitch = fxEdit.addNode('SoundFxPitch', { semitones: 12 });
|
||||
const fxOut = fxEdit.addNode('SoundFxOutput');
|
||||
fxEdit.wire([
|
||||
[input, 'out', gain, 'in'],
|
||||
[input, 'depth', gain, 'gain'],
|
||||
[gain, 'out', pitch, 'in'],
|
||||
[pitch, 'out', fxOut, 'in'],
|
||||
]);
|
||||
// In the instrument: Sample → Sound FX → Output.
|
||||
const sample = nodeOf(f, 'InstrumentSample');
|
||||
const output = nodeOf(f, 'InstrumentOutput');
|
||||
f.instrument.graph.edges = f.instrument.graph.edges.filter((e) => e.to.node !== output.id);
|
||||
const through = f.edit.addNode('InstrumentSoundFx', { fx: fx.id });
|
||||
f.edit.wire([
|
||||
[sample, 'out', through, 'in'],
|
||||
[through, 'out', output, 'in'],
|
||||
]);
|
||||
const lookup = graphLookup(f.project);
|
||||
const def = resolveDef(through, f.instrument.graph, lookup)!;
|
||||
expect(def.label).toBe('Cave');
|
||||
expect(def.inputs.map((p) => [p.key, p.default])).toEqual([
|
||||
['in', undefined],
|
||||
['depth', -6],
|
||||
]);
|
||||
|
||||
// Left open, the parameter is the FX's default.
|
||||
const open = way(compile(f, 64).layers[0])!;
|
||||
expect(open.pitch).toEqual([{ value: 4 }, { value: 12 }]);
|
||||
expect(open.stages).toEqual([{ kind: 'gain', db: { value: -6 } }]);
|
||||
expect(open.source?.file).toBe(KICK);
|
||||
|
||||
// Typed in at the node, it is a number – held to the FX's row.
|
||||
through.params.depth = -500;
|
||||
expect(way(compile(f, 64).layers[0])!.stages).toEqual([{ kind: 'gain', db: { value: -120 } }]);
|
||||
|
||||
// Fed from the instrument's own parameter, it is that one.
|
||||
const params = f.edit.addNode('InstrumentParameters');
|
||||
f.stack.execute(new AddPortCommand(f.instrument.id, params.id, { key: 'dark', kind: 'number' }));
|
||||
f.edit.connect(params, 'dark', through, 'depth');
|
||||
expect(way(compile(f, 64).layers[0])!.stages).toEqual([
|
||||
{ kind: 'gain', db: { value: 0, param: 'dark', min: -120, max: 24 } },
|
||||
]);
|
||||
|
||||
// Renaming the FX's parameter takes the edge into the node along.
|
||||
f.stack.execute(new RenamePortCommand(fx.id, input.id, 'depth', 'floor'));
|
||||
expect(f.instrument.graph.edges.some((e) => e.to.node === through.id && e.to.port === 'floor')).toBe(
|
||||
true,
|
||||
);
|
||||
expect(callProblem(through, lookup, f.instrument.id)).toBeNull();
|
||||
through.params.fx = '';
|
||||
expect(callProblem(through, lookup, f.instrument.id)).toBe('No sound FX picked');
|
||||
});
|
||||
|
||||
it('says what keeps it from compiling: a Sample nothing triggers, a sound that is gone', () => {
|
||||
const f = makeInstrument();
|
||||
const sample = nodeOf(f, 'InstrumentSample');
|
||||
sample.params.sound = 'gone';
|
||||
expect(compile(f, 60).problems[sample.id]).toBe('That sound is gone');
|
||||
sample.params.sound = f.sound('Piano', KICK).id;
|
||||
f.instrument.graph.edges = f.instrument.graph.edges.filter((e) => e.to.node !== sample.id);
|
||||
expect(compile(f, 60).problems[sample.id]).toBe('Nothing triggers it: connect it to a Notes node');
|
||||
});
|
||||
|
||||
it('is compiled again when a sound it plays changes, not on every note', () => {
|
||||
const f = makeInstrument();
|
||||
const piano = f.sound('Piano', KICK);
|
||||
nodeOf(f, 'InstrumentSample').params.sound = piano.id;
|
||||
const sources = instrumentSourcesOf(f.project);
|
||||
const first = instrumentBuildFor(f.instrument, 60, sources);
|
||||
expect(instrumentBuildFor(f.instrument, 60, sources)).toBe(first);
|
||||
piano.file = SNARE;
|
||||
const again = instrumentBuildFor(f.instrument, 60, instrumentSourcesOf(f.project));
|
||||
expect(way(again.layers[0])?.source?.file).toBe(SNARE);
|
||||
});
|
||||
});
|
||||
|
||||
describe('ports of an instrument', () => {
|
||||
it('a trigger fits only a trigger', () => {
|
||||
expect(portsCompatible('trigger', 'trigger')).toBe(true);
|
||||
expect(portsCompatible('trigger', 'boolean')).toBe(false);
|
||||
expect(portsCompatible('trigger', 'any')).toBe(false);
|
||||
expect(portsCompatible('audio', 'trigger')).toBe(false);
|
||||
});
|
||||
|
||||
it('a Sample keeps every trigger connected to it, a Gain still only one sound', () => {
|
||||
const f = makeInstrument();
|
||||
const notes = nodeOf(f, 'InstrumentNotes');
|
||||
notes.params.singleNotes = true;
|
||||
const sample = nodeOf(f, 'InstrumentSample');
|
||||
f.edit.connect(notes, 'n61', sample, 'trigger');
|
||||
expect(f.instrument.graph.edges.filter((e) => e.to.node === sample.id)).toHaveLength(2);
|
||||
const gain = f.edit.addNode('SoundFxGain');
|
||||
f.edit.connect(sample, 'out', gain, 'in');
|
||||
const other = f.edit.addNode('InstrumentSample');
|
||||
f.edit.connect(other, 'out', gain, 'in');
|
||||
expect(f.instrument.graph.edges.filter((e) => e.to.node === gain.id)).toHaveLength(1);
|
||||
});
|
||||
});
|
||||
|
||||
describe('Play note', () => {
|
||||
function sink() {
|
||||
const notes: {
|
||||
name: string;
|
||||
layers: readonly FxPlan[];
|
||||
params: object;
|
||||
volume: number;
|
||||
duration: number;
|
||||
}[] = [];
|
||||
const audio = {
|
||||
playNote: (
|
||||
name: string,
|
||||
layers: readonly FxPlan[],
|
||||
params: object,
|
||||
volume: number,
|
||||
_o: string,
|
||||
duration: number,
|
||||
) => notes.push({ name, layers, params, volume, duration }),
|
||||
} as unknown as AudioSink;
|
||||
return { audio, notes };
|
||||
}
|
||||
|
||||
it('hands the sink the layers of the note, its velocity as loudness and as a value, held to 0..1', () => {
|
||||
const f = makeInstrument();
|
||||
nodeOf(f, 'InstrumentSample').params.sound = f.sound('Piano', KICK).id;
|
||||
const { audio, notes } = sink();
|
||||
const struck = { note: 64, velocity: 0.5, volume: 0.8, duration: 0.25 };
|
||||
expect(playNote({ project: f.project, audio }, f.instrument.id, struck, '', { cut: 1 })).toBe(1);
|
||||
expect(notes[0]).toMatchObject({
|
||||
name: `instrument:${f.instrument.id}`,
|
||||
params: { cut: 1, [VELOCITY_PARAM]: 0.5 },
|
||||
volume: 0.4,
|
||||
duration: 0.25,
|
||||
});
|
||||
expect(way(notes[0]!.layers[0])?.pitch).toEqual([{ value: 4 }]);
|
||||
playNote({ project: f.project, audio }, f.instrument.id, { ...struck, velocity: 3, volume: 2 }, '', {});
|
||||
expect(notes[1]).toMatchObject({ volume: 1, params: { [VELOCITY_PARAM]: 1 } });
|
||||
});
|
||||
|
||||
it('plays nothing for a note the instrument has nothing for, or an instrument that is gone', () => {
|
||||
const f = makeInstrument();
|
||||
nodeOf(f, 'InstrumentSample').params.sound = f.sound('Piano', KICK).id;
|
||||
const { audio, notes } = sink();
|
||||
const struck = { note: 30, velocity: 1, volume: 1, duration: 0 };
|
||||
expect(playNote({ project: f.project, audio }, f.instrument.id, struck, '', {})).toBe(0);
|
||||
expect(playNote({ project: f.project, audio }, 'gone', { ...struck, note: 60 }, '', {})).toBe(0);
|
||||
expect(notes).toEqual([]);
|
||||
});
|
||||
|
||||
it('warns where its parameters come from a node naming another instrument', () => {
|
||||
const f = makeInstrument();
|
||||
const behavior = new CreateAssetCommand('behavior', 'B');
|
||||
f.stack.execute(behavior);
|
||||
const asset = behavior.asset as BehaviorAsset;
|
||||
const edit = graphEditor(f, asset);
|
||||
const play = edit.addNode('PlayNote', { instrument: f.instrument.id });
|
||||
const values = edit.addNode('InstrumentParameterValues', { soundFxAsset: 'other' });
|
||||
edit.connect(values, 'params', play, 'fxParams');
|
||||
expect(soundFxParamsMismatch(play, asset.graph)).toBe('The parameters are for another instrument');
|
||||
values.params.soundFxAsset = f.instrument.id;
|
||||
expect(soundFxParamsMismatch(play, asset.graph)).toBeNull();
|
||||
expect(callProblem(values, graphLookup(f.project))).toBeNull();
|
||||
});
|
||||
|
||||
it('an Instrument parameters node offers what the Parameters node declares, and follows a rename', () => {
|
||||
const f = makeInstrument();
|
||||
const params = f.edit.addNode('InstrumentParameters');
|
||||
f.stack.execute(new AddPortCommand(f.instrument.id, params.id, { key: 'cut', kind: 'number' }));
|
||||
f.stack.execute(new SetPortDefaultCommand(f.instrument.id, params.id, 'cut', 0.7));
|
||||
expect(graphLookup(f.project).soundFx(f.instrument.id)?.params).toEqual([
|
||||
{ key: 'cut', kind: 'number', default: 0.7 },
|
||||
]);
|
||||
const behavior = new CreateAssetCommand('behavior', 'B');
|
||||
f.stack.execute(behavior);
|
||||
const asset = behavior.asset as BehaviorAsset;
|
||||
const values = graphEditor(f, asset).addNode('InstrumentParameterValues', {
|
||||
soundFxAsset: f.instrument.id,
|
||||
});
|
||||
f.stack.execute(new AddPortCommand(asset.id, values.id, { key: 'cut', kind: 'number' }));
|
||||
const def = resolveDef(values, asset.graph, graphLookup(f.project))!;
|
||||
expect(def.inputs.map((p) => [p.key, p.default])).toEqual([['cut', 0.7]]);
|
||||
f.stack.execute(new RenamePortCommand(f.instrument.id, params.id, 'cut', 'bright'));
|
||||
expect(values.ports?.map((p) => p.key)).toEqual(['bright']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('the keys of the tab', () => {
|
||||
it('lays out all 128 notes, black over white, and finds the note under a point', () => {
|
||||
const keys = keyboardLayout(750, 60);
|
||||
expect(keys).toHaveLength(128);
|
||||
expect(keys.filter((k) => k.black)).toHaveLength(53);
|
||||
const c4 = keys.find((k) => k.note === 60)!;
|
||||
const cSharp4 = keys.find((k) => k.note === 61)!;
|
||||
expect(noteAt(keys, c4.x + 1, 55)).toBe(60);
|
||||
// Up top, the black key lies over the edge of the white one.
|
||||
expect(noteAt(keys, cSharp4.x + cSharp4.w / 2, 5)).toBe(61);
|
||||
expect(noteAt(keys, 800, 10)).toBeNull();
|
||||
});
|
||||
|
||||
it('plays the computer keyboard by where a key is, as a tracker does', () => {
|
||||
expect(trackerNote('KeyZ', 3)).toBe(48);
|
||||
expect(trackerNote('KeyS', 3)).toBe(49);
|
||||
expect(trackerNote('KeyQ', 3)).toBe(60);
|
||||
expect(trackerNote('Digit2', 3)).toBe(61);
|
||||
expect(trackerNote('KeyP', 3)).toBe(76);
|
||||
expect(trackerNote('KeyA', 3)).toBeNull();
|
||||
expect(trackerNote('KeyP', 8)).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -55,6 +55,8 @@ function start(f: Fixture, audio?: AudioSink) {
|
||||
const stoppedOwners: string[] = [];
|
||||
const sink: AudioSink = audio ?? {
|
||||
play: () => 0,
|
||||
playNote: () => 0,
|
||||
release: () => {},
|
||||
tune: () => {},
|
||||
length: () => 0,
|
||||
stop: () => {},
|
||||
|
||||
@@ -2028,6 +2028,49 @@ const CASES: Record<string, Case[]> = {
|
||||
after: (ctx) => expect(ctx.played[0]?.fxParams).toEqual({ depth: -12 }),
|
||||
},
|
||||
],
|
||||
PlayNote: [
|
||||
{
|
||||
name: 'no instrument is a no-op that still continues the flow',
|
||||
inputs: { instrument: '', note: 60 },
|
||||
next: 'exec',
|
||||
outputs: { voice: 0 },
|
||||
after: (ctx) => expect(ctx.notes).toEqual([]),
|
||||
},
|
||||
{
|
||||
name: 'hands the instrument on with its note, velocity, volume and duration, and gives out the voice (#331)',
|
||||
inputs: {
|
||||
instrument: 'ins-1',
|
||||
note: 64,
|
||||
velocity: 0.5,
|
||||
volume: 0.8,
|
||||
duration: 0.25,
|
||||
fxParams: { cut: 2 },
|
||||
},
|
||||
next: 'exec',
|
||||
outputs: { voice: 1 },
|
||||
after: (ctx) =>
|
||||
expect(ctx.notes).toEqual([
|
||||
{
|
||||
instrumentId: 'ins-1',
|
||||
note: 64,
|
||||
velocity: 0.5,
|
||||
volume: 0.8,
|
||||
duration: 0.25,
|
||||
detached: false,
|
||||
instrumentParams: { cut: 2 },
|
||||
},
|
||||
]),
|
||||
},
|
||||
],
|
||||
InstrumentParameterValues: [
|
||||
{
|
||||
name: 'bundles its picked parameters as numbers, as for a sound FX',
|
||||
ports: [{ key: 'cut', kind: 'number' }],
|
||||
inputs: { cut: '0.5' },
|
||||
outputs: { params: { cut: 0.5 } },
|
||||
},
|
||||
{ name: 'with none picked the bundle is empty', outputs: { params: {} } },
|
||||
],
|
||||
SoundFxParameters: [
|
||||
{
|
||||
name: 'bundles its picked parameters as numbers',
|
||||
|
||||
@@ -21,7 +21,11 @@ function sine(frequency: number, seconds: number, rate = RATE): Float32Array {
|
||||
function render(
|
||||
sound: Float32Array,
|
||||
semitones: number | ((at: number) => number),
|
||||
{ step = 1, loop = false, blocks = Infinity } = {},
|
||||
{
|
||||
step = 1,
|
||||
loop = false,
|
||||
blocks = Infinity,
|
||||
}: { step?: number; loop?: boolean | { start: number; end: number }; blocks?: number } = {},
|
||||
): Float32Array {
|
||||
const player = new PitchPlayer([sound], step, RATE, loop);
|
||||
const out: number[] = [];
|
||||
@@ -138,3 +142,22 @@ describe('the pitch player', () => {
|
||||
expect(off).toBeLessThan(1e-5);
|
||||
});
|
||||
});
|
||||
|
||||
describe('a loop between two points (#331)', () => {
|
||||
it('plays up to the loop end, then on from the loop start, never ending', () => {
|
||||
// A ramp, so every sample says where in the sound it was read.
|
||||
const ramp = Float32Array.from({ length: 48000 }, (_, i) => i / 48000);
|
||||
const out = render(ramp, 0, { loop: { start: 12000, end: 24000 }, blocks: 400 });
|
||||
expect(out.length).toBe(400 * BLOCK);
|
||||
expect(out[20000]).toBeCloseTo(20000 / 48000, 3);
|
||||
// 30000 is 6000 past the end: 6000 past the start.
|
||||
expect(out[30000]).toBeCloseTo(18000 / 48000, 3);
|
||||
expect(out[40000]).toBeCloseTo(16000 / 48000, 3);
|
||||
});
|
||||
|
||||
it('takes a loop that does not fit the sound as the whole sound', () => {
|
||||
const ramp = Float32Array.from({ length: 4800 }, (_, i) => i / 4800);
|
||||
const out = render(ramp, 0, { loop: { start: 3000, end: 9000 }, blocks: 60 });
|
||||
expect(out[6000]).toBeCloseTo(1200 / 4800, 3);
|
||||
});
|
||||
});
|
||||
@@ -372,6 +372,8 @@ function fakeAudio(): AudioSink & {
|
||||
sink.tuned.push({ voice, params });
|
||||
sink.times.push(time);
|
||||
},
|
||||
playNote: () => 0,
|
||||
release: () => {},
|
||||
length: () => 0.25,
|
||||
stop: () => {},
|
||||
tail: () => 0,
|
||||
|
||||
@@ -42,6 +42,8 @@ function fakeAudio(): AudioSink & {
|
||||
stoppedOwners: [] as string[],
|
||||
play: (file: string, volume: number, loop: boolean, owner: string) =>
|
||||
sink.played.push({ file, volume, loop, owner }),
|
||||
playNote: () => 0,
|
||||
release: () => {},
|
||||
tune: () => {},
|
||||
length: () => 0,
|
||||
tail: () => 0,
|
||||
|
||||
@@ -214,6 +214,16 @@ export type FakeContext = NodeContext & {
|
||||
fx: string;
|
||||
fxParams: Readonly<Record<string, number>>;
|
||||
}[];
|
||||
/** The notes the context was asked to play (#331); a voice's number is how many have been played. */
|
||||
notes: {
|
||||
instrumentId: string;
|
||||
note: number;
|
||||
velocity: number;
|
||||
volume: number;
|
||||
duration: number;
|
||||
detached: boolean;
|
||||
instrumentParams: Readonly<Record<string, number>>;
|
||||
}[];
|
||||
silenced: string[];
|
||||
tuned: { voice: number; fxParams: Readonly<Record<string, number>>; time: number; ease: EaseShare }[];
|
||||
/** The scenes the context was asked to load, in order. */
|
||||
@@ -270,6 +280,9 @@ export function fakeContext(overrides: Partial<NodeContext> = {}): FakeContext {
|
||||
// The number of a voice is how many have been played: unique, and never 0.
|
||||
playSound: (assetId, volume, loop, detached, fx, fxParams) =>
|
||||
ctx.played.push({ assetId, volume, loop, detached, fx, fxParams }),
|
||||
notes: [],
|
||||
playNote: (instrumentId, note, velocity, volume, duration, detached, instrumentParams) =>
|
||||
ctx.notes.push({ instrumentId, note, velocity, volume, duration, detached, instrumentParams }),
|
||||
// Every sound lasts a quarter of a second, as if it had been loaded.
|
||||
soundLength: () => 0.25,
|
||||
// The tail of a voice is its number in tenths of a second, so a test can tell one from another.
|
||||
|
||||
@@ -383,6 +383,7 @@ describe('the sound library', () => {
|
||||
file: FILE,
|
||||
volume: 0.25,
|
||||
loop: true,
|
||||
layers: 1,
|
||||
pitches: [0],
|
||||
stages: 0,
|
||||
duration: 1.5,
|
||||
@@ -890,3 +891,108 @@ describe('the sound library', () => {
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('notes of an instrument (#331)', () => {
|
||||
/** A way that plays its own file, as a Sample makes it. */
|
||||
const sample = (
|
||||
file: string,
|
||||
extra: Partial<NonNullable<Extract<FxPlan, { pitch: unknown }>['source']>> = {},
|
||||
) =>
|
||||
({
|
||||
pitch: [],
|
||||
stages: [],
|
||||
source: { file, volume: 0.5, loop: false, loopStart: 0, loopEnd: 0, gate: true, ...extra },
|
||||
}) as FxPlan;
|
||||
|
||||
async function note(library: SoundLibrary, layers: FxPlan[], duration = 0): Promise<number> {
|
||||
const voice = library.playNote('instrument:i', layers, {}, 1, '', duration);
|
||||
await settle();
|
||||
await arrive();
|
||||
return voice;
|
||||
}
|
||||
|
||||
it('plays its layers side by side as one voice, each way its own file behind its own volume', async () => {
|
||||
const library = new SoundLibrary();
|
||||
const voice = await note(library, [sample(FILE), sample(OTHER, { gate: false })]);
|
||||
expect(voice).toBe(1);
|
||||
expect(library.stats()).toMatchObject({ started: 1, playing: 1, decoded: 2 });
|
||||
expect(library.stats().last).toMatchObject({ file: 'instrument:i', layers: 2, duration: 1.5 });
|
||||
expect(sources.map((source) => source.started)).toEqual([true, true]);
|
||||
// Voice gain first, then a fade per way at the sound's volume.
|
||||
expect(gains.map((gain) => gain.value)).toEqual([1, 0.5, 0.5]);
|
||||
expect(wires).toEqual([
|
||||
'source > gain',
|
||||
'gain > gain',
|
||||
'source > gain',
|
||||
'gain > gain',
|
||||
'gain > destination',
|
||||
]);
|
||||
});
|
||||
|
||||
it('a release fades out the ways that end with the note and lets a one-shot play on', async () => {
|
||||
const library = new SoundLibrary();
|
||||
const voice = await note(library, [sample(FILE), sample(OTHER, { gate: false })]);
|
||||
library.release(voice);
|
||||
expect(gains[1]!.ramps).toEqual([['linear', 0, 0.01]]);
|
||||
expect(gains[2]!.ramps).toEqual([]);
|
||||
clockTo();
|
||||
expect(sources.map((source) => source.stopped)).toEqual([true, false]);
|
||||
expect(library.stats().playing).toBe(1);
|
||||
// The one-shot ends by itself: now the voice is gone.
|
||||
sources[1]!.onended?.();
|
||||
expect(library.stats().playing).toBe(0);
|
||||
});
|
||||
|
||||
it('a note with a duration is released that long after it starts, on the audio clock', async () => {
|
||||
const library = new SoundLibrary();
|
||||
await note(library, [sample(FILE)], 0.5);
|
||||
expect(clocks.map((clock) => clock.stopAt)).toEqual([0.5]);
|
||||
clockTo(0.5);
|
||||
expect(gains[1]!.ramps).toEqual([['linear', 0, 0.01]]);
|
||||
clockTo();
|
||||
expect(sources[0]!.stopped).toBe(true);
|
||||
expect(library.stats().playing).toBe(0);
|
||||
});
|
||||
|
||||
it('a note released before its bytes arrived still starts, and is released at once', async () => {
|
||||
const library = new SoundLibrary();
|
||||
const voice = library.playNote(
|
||||
'instrument:i',
|
||||
[sample(FILE), sample(OTHER, { gate: false })],
|
||||
{},
|
||||
1,
|
||||
'',
|
||||
0,
|
||||
);
|
||||
library.release(voice);
|
||||
await settle();
|
||||
await arrive();
|
||||
expect(library.stats().started).toBe(1);
|
||||
clockTo();
|
||||
expect(sources.map((source) => source.stopped)).toEqual([true, false]);
|
||||
});
|
||||
|
||||
it('loops a way between its loop start and end', async () => {
|
||||
const library = new SoundLibrary();
|
||||
await note(library, [sample(FILE, { loop: true, loopStart: 0.25, loopEnd: 1 })]);
|
||||
expect(sources[0]).toMatchObject({ loop: true, loopStart: 0.25, loopEnd: 1 });
|
||||
});
|
||||
|
||||
it('hands a pitched way its loop in samples of the sound', async () => {
|
||||
const library = new SoundLibrary();
|
||||
const pitched = { ...sample(FILE, { loop: true, loopStart: 0.25, loopEnd: 0 }), pitch: [{ value: 3 }] };
|
||||
await note(library, [pitched]);
|
||||
// 800 samples a second, 1200 of them: from 200 to the end.
|
||||
expect(worklets[0]!.options.processorOptions).toMatchObject({
|
||||
loop: true,
|
||||
loopStart: 200,
|
||||
loopEnd: 1200,
|
||||
});
|
||||
});
|
||||
|
||||
it('plays nothing where no way has a file', () => {
|
||||
const library = new SoundLibrary();
|
||||
expect(library.playNote('instrument:i', [], {}, 1, '', 0)).toBe(0);
|
||||
expect(library.playNote('instrument:i', [{ pitch: [], stages: [] }], {}, 1, '', 0)).toBe(0);
|
||||
});
|
||||
});
|
||||
Reference in new issue
Block a user