Give sounds, instruments, patterns and songs a sound FX (#352)

A sound FX could only be picked where something was played: on a Play
node, a track or an item of a song. A sound that should always sound
through the same effect needed that pick everywhere it was used. Now
the asset says it once, and every place that plays it follows.

- New row Sound FX on a sound, an instrument, a pattern and a song,
  right before Bus. A sound goes through it on every voice, an
  instrument on every note after its graph, a pattern on all its
  notes together and a song on its output.
- The fx input of Play sound, Play note, Play pattern and Play song
  and the Sound FX of a song item now read "From the asset" while
  empty. None is a pick of its own, stored as `#none` like `#master`.
  A pick replaces the FX of the asset, it does not come on top.
- A pattern item's Instrument reads "From the asset" instead of None.
  The meaning is unchanged.
- A double-click on a sound item opens the sound, as one on a pattern
  item opens the pattern.
- The tabs play through the FX of their asset. The Pattern and Song
  tab put it on a bus of their own between the output and the bus of
  the asset. The keys of the Pattern tab play there too.
- Notes of a pattern or song go through the FX of their instrument.
  Its chain is loaded with the other needs of the sequence.

Decided without asking:
- Old items and Play nodes with an empty fx now mean "From the
  asset". No asset has an FX yet, so nothing sounds different.
- The label is "From the asset" as on Loop and Bus, not "From asset".
- A sound used as a Sample, as the impulse of a Reverb or as the test
  sound of the Sound FX tab plays without its own FX.

Findings:
- In e2e, `__gm.inspectorField('Sound FX')` and `('Instrument')` find
  the group of the Assets panel first. The check asks for the last
  match.

Tests:
- Unit asset-fx (new): 8 green. Six counter-checks each turned red.
- Field lists of sound, pattern and song: Sound FX added.
- e2e 65 (melektaus-atx): ALL PASSED, 46 checks (3 new). Counter-check
  without the FX bus of the tab: that check red.
- Gate (melektaus-atx): CHECK EXIT 0, 268 files, 4142 tests.
- Not checked: how it sounds, the Sound, Instrument and Pattern tab
  in the browser.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-07 22:02:26 +02:00
1 parent d349251a01
commit a295d233ed
26 files changed
+521 -60

No files matched your search

+6 -3
View File
@@ -238,7 +238,8 @@ export interface NodeContext {
* The voice belongs to the behavior that started it and ends when that behavior is destroyed – * The voice belongs to the behavior that started it and ends when that behavior is destroyed –
* a sound is something a thing in the world is doing. `detached` gives that up on purpose, for * a sound is something a thing in the world is doing. `detached` gives that up on purpose, for
* the one case where the rule is in the way: the death sound of an object that destroys itself. * the one case where the rule is in the way: the death sound of an object that destroys itself.
* `fx` names a sound FX asset the voice is played through (#311); `''` plays it as it is. * `fx` names a sound FX asset the voice is played through (#311); `''` the one the sound asset
* names (#352), `NO_FX` none.
* `fxParams` are values for its parameters; one not given is the default the FX declares. * `fxParams` are values for its parameters; one not given is the default the FX declares.
* `tempo` is what a Delay synced to beats counts by (#332). `bus` is the bus it is heard on * `tempo` is what a Delay synced to beats counts by (#332). `bus` is the bus it is heard on
* (#325): a bus asset or `MASTER_BUS`, `''` for the one the sound asset names. * (#325): a bus asset or `MASTER_BUS`, `''` for the one the sound asset names.
@@ -259,7 +260,8 @@ export interface NodeContext {
* `velocity` (0 to 1, how loud it is next to `volume`) and released after `duration` seconds, or * `velocity` (0 to 1, how loud it is next to `volume`) and released after `duration` seconds, or
* held until it is stopped where that is 0. Owned like a `playSound` voice, `instrumentParams` are * held until it is stopped where that is 0. Owned like a `playSound` voice, `instrumentParams` are
* values for the instrument's parameters, `tempo` as for `playSound`. `fx` sends the note through a * values for the instrument's parameters, `tempo` as for `playSound`. `fx` sends the note through a
* sound FX of its own (#346), `fxParams` with values for its parameters, apart from the instrument's. * sound FX of its own (#346), `fxParams` with values for its parameters, apart from the instrument's;
* `''` the one the instrument names, `NO_FX` none (#352).
* `bus` as for `playSound`, `''` for the one the instrument names. * `bus` as for `playSound`, `''` for the one the instrument names.
* Returns the number of the voice, 0 where nothing plays. * Returns the number of the voice, 0 where nothing plays.
*/ */
@@ -279,7 +281,8 @@ export interface NodeContext {
/** /**
* Plays a pattern asset through its instrument, at its own tempo, once or round and round, every * Plays a pattern asset through its instrument, at its own tempo, once or round and round, every
* note at `volume`. Owned like a `playSound` voice. `fx` is a sound FX all of it goes through as * note at `volume`. Owned like a `playSound` voice. `fx` is a sound FX all of it goes through as
* one chain (#346), `fxParams` values for its parameters, `bus` as for `playSound`. Returns its * one chain (#346), `''` the one the pattern names, `NO_FX` none (#352); `fxParams` values for its
* parameters, `bus` as for `playSound`. Returns its
* number, for `stopVoice` and `tuneSound`; 0 where there is nothing to play. * number, for `stopVoice` and `tuneSound`; 0 where there is nothing to play.
*/ */
playPattern( playPattern(
+1 -1
View File
@@ -10,7 +10,7 @@ export const PlayNote: NodeDef = {
label: 'Play note', label: 'Play note',
category: 'action', category: 'action',
description: description:
'Plays one note of an instrument asset. "note" is its number as MIDI counts it: 60 is C4, 61 C#4, 72 C5. "velocity" is how hard it is struck, 0 to 1 – it makes the note quieter and is handed to what the instrument does with it. "duration" is how many seconds until the note is released; at 0 it is held until a Stop sound ends it. A note belongs to this object like a sound and stops when the object is destroyed, unless "detached" is on. "params" takes values for the parameters of the instrument from an Instrument parameters node. "fx" sends the note through a sound FX on top of what the instrument does, "fx params" takes values for its parameters from a Sound FX parameters node – the two sets stay apart. "voice" is the number of this note, for Set sound FX parameters and Stop sound; 0 where the instrument plays nothing for it. "tempo" is what a Delay in the instrument synced to beats counts by, in quarter notes a minute. "bus" is the bus it is heard on; left empty, the one the instrument names.', 'Plays one note of an instrument asset. "note" is its number as MIDI counts it: 60 is C4, 61 C#4, 72 C5. "velocity" is how hard it is struck, 0 to 1 – it makes the note quieter and is handed to what the instrument does with it. "duration" is how many seconds until the note is released; at 0 it is held until a Stop sound ends it. A note belongs to this object like a sound and stops when the object is destroyed, unless "detached" is on. "params" takes values for the parameters of the instrument from an Instrument parameters node. "fx" sends the note through a sound FX on top of what the instrument does; left empty, the one the instrument names, and None through none. "fx params" takes values for its parameters from a Sound FX parameters node – the two sets stay apart. "voice" is the number of this note, for Set sound FX parameters and Stop sound; 0 where the instrument plays nothing for it. "tempo" is what a Delay in the instrument synced to beats counts by, in quarter notes a minute. "bus" is the bus it is heard on; left empty, the one the instrument names.',
inputs: [ inputs: [
exec(), exec(),
{ {
+2 -1
View File
@@ -21,7 +21,8 @@ export const PlayPattern: NodeDef = {
'it quieter. Each play is a playing of its own: firing this twice plays it twice, overlapping. ' + 'it quieter. Each play is a playing of its own: firing this twice plays it twice, overlapping. ' +
'It belongs to this object like a sound and stops when the object is destroyed, unless ' + 'It belongs to this object like a sound and stops when the object is destroyed, unless ' +
'"detached" is on. "fx" sends all of it through one sound FX, its notes together: an LFO runs on ' + '"detached" is on. "fx" sends all of it through one sound FX, its notes together: an LFO runs on ' +
'across them, an echo repeats the whole. "fx params" takes values for its parameters from a Sound ' + 'across them, an echo repeats the whole. Left empty, it is the one the pattern names; None is ' +
'none. "fx params" takes values for its parameters from a Sound ' +
'FX parameters node. "playback" is the number of this playing, for Stop pattern and for Set sound ' + 'FX parameters node. "playback" is the number of this playing, for Stop pattern and for Set sound ' +
'FX parameters; 0 where the pattern has no instrument or no notes. "bus" is the bus all of it is ' + 'FX parameters; 0 where the pattern has no instrument or no notes. "bus" is the bus all of it is ' +
'heard on; left empty, the one the pattern names.', 'heard on; left empty, the one the pattern names.',
+1
View File
@@ -22,6 +22,7 @@ export const PlaySong: NodeDef = {
'playing of its own: firing this twice plays it twice, side by side. It belongs to this object ' + 'playing of its own: firing this twice plays it twice, side by side. It belongs to this object ' +
'like a sound and stops when the object is destroyed or the scene is left, unless "detached" ' + 'like a sound and stops when the object is destroyed or the scene is left, unless "detached" ' +
'is on. "fx" sends the whole song through one sound FX, after its tracks and before "volume"; ' + 'is on. "fx" sends the whole song through one sound FX, after its tracks and before "volume"; ' +
'left empty, the one the song names, and None is none. ' +
'"fx params" takes values for its parameters from a Sound FX parameters node. "playback" is the ' + '"fx params" takes values for its parameters from a Sound FX parameters node. "playback" is the ' +
'number of this playing, for Stop song and for Set sound FX parameters; 0 where the song has ' + 'number of this playing, for Stop song and for Set sound FX parameters; 0 where the song has ' +
'nothing to play. "bus" is the bus all of it is heard on; left empty, the one the song names.', 'nothing to play. "bus" is the bus all of it is heard on; left empty, the one the song names.',
+2 -1
View File
@@ -59,7 +59,8 @@ export const PlaySound: NodeDef = {
"leaves the asset's own setting alone or overrules it, so the asset says what the sound is and " + "leaves the asset's own setting alone or overrules it, so the asset says what the sound is and " +
'the node only colors it. Each play is a voice of its own: firing this twice in a row is heard ' + 'the node only colors it. Each play is a voice of its own: firing this twice in a row is heard ' +
'twice, overlapping. The sound belongs to this object and stops when the object is destroyed, ' + 'twice, overlapping. The sound belongs to this object and stops when the object is destroyed, ' +
'unless "detached" is on. "fx" sends it through a sound FX asset; left empty, it plays as it is. ' + 'unless "detached" is on. "fx" sends it through a sound FX asset instead of the one the sound ' +
'names; left empty, it takes that one, and None plays it as it is. ' +
'"fx params" takes values for its parameters from a Sound FX parameters node. "voice" is the ' + '"fx params" takes values for its parameters from a Sound FX parameters node. "voice" is the ' +
'number of this one play, for Set sound FX parameters to change it while it plays; 0 where ' + 'number of this one play, for Set sound FX parameters to change it while it plays; 0 where ' +
'nothing was played. "length" is how many seconds one run of the sound lasts, to hand to the ' + 'nothing was played. "length" is how many seconds one run of the sound lasts, to hand to the ' +
+13 -2
View File
@@ -94,12 +94,23 @@ export const soundPort = (): PortDef => ({
field: { key: 'sound', label: 'Sound', kind: 'sound', default: '' }, field: { key: 'sound', label: 'Sound', kind: 'sound', default: '' },
}); });
/** The `fx` input of `Play sound` (#311); unconnected, it shows the picker of sound FX assets. */ /**
* The `fx` input of a Play node (#311); unconnected, it shows the picker of sound FX assets. Empty is
* the one its asset names, `NO_FX` none (#352).
*/
export const soundFxPort = (): PortDef => ({ export const soundFxPort = (): PortDef => ({
key: 'fx', key: 'fx',
label: 'fx', label: 'fx',
type: 'string', type: 'string',
field: { key: 'fx', label: 'FX', kind: 'asset', default: '', assetTypes: ['soundfx'] }, field: {
key: 'fx',
label: 'FX',
kind: 'asset',
default: '',
assetTypes: ['soundfx'],
none: 'From the asset',
noFx: true,
},
}); });
/** /**
+7 -1
View File
@@ -1,5 +1,6 @@
import { instrumentBuildFor, instrumentSourcesOf } from '../instrument/compile'; import { instrumentBuildFor, instrumentSourcesOf } from '../instrument/compile';
import { findAssetOf, type Project } from '../model/Project'; import { findAssetOf, type Project } from '../model/Project';
import { assetFxOf } from '../properties/assetFx';
import { playedSoundById } from '../properties/playedSound'; import { playedSoundById } from '../properties/playedSound';
import { soundFxBuildFor } from '../soundfx/compile'; import { soundFxBuildFor } from '../soundfx/compile';
import { DRY_VOICE, type FxParams, type FxPlan, type FxVoice } from '../soundfx/nodes'; import { DRY_VOICE, type FxParams, type FxPlan, type FxVoice } from '../soundfx/nodes';
@@ -26,7 +27,7 @@ export function clipFx(project: Project, fxId: string, tempo: number): FxVoice {
return fxVoiceOf(project, fxId, tempo) ?? DRY_VOICE; return fxVoiceOf(project, fxId, tempo) ?? DRY_VOICE;
} }
/** Every layer of every note and every file and FX of every clip a sequence plays. */ /** Every layer and FX of every note and every file and FX of every clip a sequence plays. */
export function sequenceNeeds(project: Project, sequence: Sequence): SequenceNeeds { export function sequenceNeeds(project: Project, sequence: Sequence): SequenceNeeds {
const sources = instrumentSourcesOf(project); const sources = instrumentSourcesOf(project);
const seen = new Set<string>(); const seen = new Set<string>();
@@ -35,6 +36,11 @@ export function sequenceNeeds(project: Project, sequence: Sequence): SequenceNee
const key = `${instrument}\u0000${note}`; const key = `${instrument}\u0000${note}`;
const asset = findAssetOf(project, instrument, 'instrument'); const asset = findAssetOf(project, instrument, 'instrument');
if (seen.has(key) || !asset) continue; if (seen.has(key) || !asset) continue;
if (!seen.has(instrument)) {
seen.add(instrument);
const fx = fxVoiceOf(project, assetFxOf(asset.props), 120);
if (fx) layers.push(fx.plan);
}
seen.add(key); seen.add(key);
layers.push(...instrumentBuildFor(asset, note, sources).layers); layers.push(...instrumentBuildFor(asset, note, sources).layers);
} }
+15 -6
View File
@@ -9,7 +9,8 @@ import { newId } from '../ids';
import { HIGHEST_NOTE, LOWEST_NOTE } from '../instrument/notes'; import { HIGHEST_NOTE, LOWEST_NOTE } from '../instrument/notes';
import type { PatternAsset, SongAsset } from '../model/Asset'; import type { PatternAsset, SongAsset } from '../model/Asset';
import { type PropertyBag, textOf } from '../model/Properties'; import { type PropertyBag, textOf } from '../model/Properties';
import { findAssetOf, type Project } from '../model/Project'; import { findAsset, findAssetOf, type Project } from '../model/Project';
import { pickedFx } from '../properties/assetFx';
import { gridTicks, GRID_OPTIONS, patternInstrument, patternLength } from './pattern'; import { gridTicks, GRID_OPTIONS, patternInstrument, patternLength } from './pattern';
import { heldLoop, type LoopRange, type Sequence, type SequenceClip, type SequenceNote } from './sequence'; import { heldLoop, type LoopRange, type Sequence, type SequenceClip, type SequenceNote } from './sequence';
import { heldMeter, heldTempo, type Meter, secondsOfTicks, ticksPerBar } from './time'; import { heldMeter, heldTempo, type Meter, secondsOfTicks, ticksPerBar } from './time';
@@ -47,8 +48,8 @@ export interface SongItem {
/** The instrument a pattern is played with instead of its own, `''` for its own. */ /** The instrument a pattern is played with instead of its own, `''` for its own. */
instrument: string; instrument: string;
/** /**
* The sound FX it goes through, `''` for none: a sound by itself, a pattern all its notes together * The sound FX it goes through: a sound by itself, a pattern all its notes together on a bus of
* on a bus of their own (#346). * their own (#346). `''` is the one its asset names, `NO_FX` none (#352).
*/ */
fx: string; fx: string;
} }
@@ -145,9 +146,17 @@ export function dbToGain(db: number): number {
return db <= TRACK_MIN_DB ? 0 : 10 ** (db / 20); return db <= TRACK_MIN_DB ? 0 : 10 ** (db / 20);
} }
/** Whether a pattern item is heard on a bus of its own (#346): it names a sound FX that is there. */ /** The sound FX an item goes through (#352): its own pick, else the one of what it plays; '' for none. */
export function itemFx(project: Project, item: SongItem): string {
const played = findAsset(project, item.asset);
return pickedFx(item.fx, played?.type === item.kind ? played.props : {});
}
/** Whether a pattern item is heard on a bus of its own (#346): it goes through a sound FX that is there. */
export function hasItemBus(project: Project, item: SongItem): boolean { export function hasItemBus(project: Project, item: SongItem): boolean {
return item.kind === 'pattern' && item.fx !== '' && findAssetOf(project, item.fx, 'soundfx') !== undefined; if (item.kind !== 'pattern') return false;
const fx = itemFx(project, item);
return fx !== '' && findAssetOf(project, fx, 'soundfx') !== undefined;
} }
/** /**
@@ -203,7 +212,7 @@ export function songSequence(project: Project, song: SongAsset): Sequence {
tick: item.tick, tick: item.tick,
length: item.length, length: item.length,
sound: item.asset, sound: item.asset,
fx: item.fx, fx: itemFx(project, item),
offset: item.offset, offset: item.offset,
volume, volume,
bus, bus,
+2 -1
View File
@@ -8,6 +8,7 @@ import {
dbToGain, dbToGain,
hasItemBus, hasItemBus,
itemBus, itemBus,
itemFx,
type SongItem, type SongItem,
songTempo, songTempo,
type SongTrack, type SongTrack,
@@ -68,7 +69,7 @@ export function songBuses(
gain: 1, gain: 1,
pan: 0, pan: 0,
into: trackItemsBus(song.id, item.track), into: trackItemsBus(song.id, item.track),
...fxOf(item.fx), ...fxOf(itemFx(project, item)),
})); }));
return [...tracks, ...own, ...items]; return [...tracks, ...own, ...items];
} }
+23 -3
View File
@@ -139,6 +139,7 @@ import {
SONG_TEMPO_KEY, SONG_TEMPO_KEY,
} from '../music/song'; } from '../music/song';
import { DEFAULT_TEMPO, MAX_TEMPO, MIN_TEMPO } from '../music/time'; import { DEFAULT_TEMPO, MAX_TEMPO, MIN_TEMPO } from '../music/time';
import { ASSET_FX_KEY } from './assetFx';
import { BUS_FX_KEY, BUS_INTO_KEY, BUS_KEY, BUS_VOLUME_KEY } from './buses'; import { BUS_FX_KEY, BUS_INTO_KEY, BUS_KEY, BUS_VOLUME_KEY } from './buses';
import { Theme } from '../../ui'; import { Theme } from '../../ui';
@@ -147,7 +148,8 @@ import { Theme } from '../../ui';
* properties that are integral by meaning (counts, sizes in whole pixels), not on geometry: dragging * properties that are integral by meaning (counts, sizes in whole pixels), not on geometry: dragging
* produces fractions, and the field rounds the value away rather than hiding it. Without * produces fractions, and the field rounds the value away rather than hiding it. Without
* `assetTypes` an asset picker offers every placeable asset; `none` is what it calls an empty pick, * `assetTypes` an asset picker offers every placeable asset; `none` is what it calls an empty pick,
* `master` offers the master bus as a choice of its own, and `intoFor` names the bus the row is * `master` offers the master bus as a choice of its own, `noFx` "None" as one where empty means
* "from the asset" (#352), and `intoFor` names the bus the row is
* the Into of, which leaves out that bus and every one going into it when the picker opens (#325); * the Into of, which leaves out that bus and every one going into it when the picker opens (#325);
* `uploadedOnly` keeps drawn pictures out * `uploadedOnly` keeps drawn pictures out
* of a texture picker (#186). `renderLayers` makes a layer row pick from the project's render layers * of a texture picker (#186). `renderLayers` makes a layer row pick from the project's render layers
@@ -158,7 +160,13 @@ type Extras<K extends FlatKind> = K extends 'number'
: K extends 'enum' : K extends 'enum'
? { options: { value: string; label: string }[] } ? { options: { value: string; label: string }[] }
: K extends 'asset' : K extends 'asset'
? { assetTypes?: readonly AssetType[]; none?: string; master?: boolean; intoFor?: string } ? {
assetTypes?: readonly AssetType[];
none?: string;
master?: boolean;
noFx?: boolean;
intoFor?: string;
}
: K extends 'texture' : K extends 'texture'
? { uploadedOnly?: boolean } ? { uploadedOnly?: boolean }
: K extends 'layer' | 'layers' : K extends 'layer' | 'layers'
@@ -503,6 +511,15 @@ const BUS_ROW: PropertyDef = {
none: 'Master', none: 'Master',
}; };
/** The sound FX a sound, an instrument, a pattern or a song goes through, before its bus (#352). */
const FX_ROW: PropertyDef = {
key: ASSET_FX_KEY,
label: 'Sound FX',
kind: 'asset',
default: '',
assetTypes: ['soundfx'],
};
/** Position is deliberately absent: it belongs to scene objects, not to asset types. */ /** Position is deliberately absent: it belongs to scene objects, not to asset types. */
const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = { const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
scene: [ scene: [
@@ -733,12 +750,13 @@ const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
options: FADE_CURVES, options: FADE_CURVES,
}, },
]), ]),
FX_ROW,
BUS_ROW, BUS_ROW,
], ],
// The effects are the asset's graph (see SoundFxAsset), not a property. // The effects are the asset's graph (see SoundFxAsset), not a property.
soundfx: [], soundfx: [],
// What a note sounds like is the asset's graph (see InstrumentAsset), not a property. // What a note sounds like is the asset's graph (see InstrumentAsset), not a property.
instrument: [BUS_ROW], instrument: [FX_ROW, BUS_ROW],
// The notes and the loop are intrinsic data on the asset (see PatternAsset). The meter and the // The notes and the loop are intrinsic data on the asset (see PatternAsset). The meter and the
// tempo are for drawing and hearing it on its own; a song plays it at its own tempo (#334). // tempo are for drawing and hearing it on its own; a song plays it at its own tempo (#334).
pattern: [ pattern: [
@@ -780,6 +798,7 @@ const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
max: MAX_TEMPO, max: MAX_TEMPO,
step: 1, step: 1,
}, },
FX_ROW,
BUS_ROW, BUS_ROW,
], ],
// The tracks, items and loop are intrinsic data on the asset (see SongAsset). // The tracks, items and loop are intrinsic data on the asset (see SongAsset).
@@ -811,6 +830,7 @@ const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
default: DEFAULT_SONG_GRID, default: DEFAULT_SONG_GRID,
options: SONG_GRID_OPTIONS, options: SONG_GRID_OPTIONS,
}, },
FX_ROW,
BUS_ROW, BUS_ROW,
], ],
// Where sounds are mixed (#325): how loud, through which sound FX, and on into which bus. // Where sounds are mixed (#325): how loud, through which sound FX, and on into which bus.
+21
View File
@@ -0,0 +1,21 @@
import type { PropertyBag } from '../model/Properties';
import { coerceAs } from './kinds';
/** The sound FX a sound, an instrument, a pattern or a song goes through (#352); empty is none. */
export const ASSET_FX_KEY = 'fx';
/**
* What a pick says for "no sound FX" where empty means "the one of the asset": on a Play node and on
* an item of a song. No asset id has a `#`.
*/
export const NO_FX = '#none';
export function assetFxOf(bag: PropertyBag): string {
return coerceAs(bag[ASSET_FX_KEY], 'asset', '');
}
/** The sound FX something plays through: the one picked, else the one its asset names; '' for none. */
export function pickedFx(picked: string, assetProps: PropertyBag): string {
if (picked === NO_FX) return '';
return picked !== '' ? picked : assetFxOf(assetProps);
}
+2 -1
View File
@@ -1,6 +1,7 @@
import type { Asset } from '../model/Asset'; import type { Asset } from '../model/Asset';
import { type AssetType, isPlaceableType } from '../model/AssetTypes'; import { type AssetType, isPlaceableType } from '../model/AssetTypes';
import { findAsset, type Project } from '../model/Project'; import { findAsset, type Project } from '../model/Project';
import { NO_FX } from './assetFx';
import { MASTER_BUS } from './buses'; import { MASTER_BUS } from './buses';
export const NO_ASSET_LABEL = 'None'; export const NO_ASSET_LABEL = 'None';
@@ -11,7 +12,7 @@ export const MASTER_LABEL = 'Master';
* reference shows its id rather than passing for an empty field. * reference shows its id rather than passing for an empty field.
*/ */
export function assetRefLabel(project: Project | null, id: string): string { export function assetRefLabel(project: Project | null, id: string): string {
if (id === '') return NO_ASSET_LABEL; if (id === '' || id === NO_FX) return NO_ASSET_LABEL;
if (id === MASTER_BUS) return MASTER_LABEL; if (id === MASTER_BUS) return MASTER_LABEL;
return (project ? findAsset(project, id) : undefined)?.name ?? `${id}?`; return (project ? findAsset(project, id) : undefined)?.name ?? `${id}?`;
} }
+2
View File
@@ -17,6 +17,7 @@ import {
MASTER_LABEL, MASTER_LABEL,
NO_ASSET_LABEL, NO_ASSET_LABEL,
} from '../../core/properties/assetRef'; } from '../../core/properties/assetRef';
import { NO_FX } from '../../core/properties/assetFx';
import { busesInto, MASTER_BUS } from '../../core/properties/buses'; import { busesInto, MASTER_BUS } from '../../core/properties/buses';
import { keyName } from '../../core/keys'; import { keyName } from '../../core/keys';
import { FIRST_TILE_LAYER } from '../../core/model/liveTilemap'; import { FIRST_TILE_LAYER } from '../../core/model/liveTilemap';
@@ -398,6 +399,7 @@ const REFERENCES: Record<ReferenceKind, Reference> = {
const left = new Set(extras?.intoFor ? busesInto(project, extras.intoFor) : []); const left = new Set(extras?.intoFor ? busesInto(project, extras.intoFor) : []);
return [ return [
...(extras?.master ? [{ id: MASTER_BUS, label: MASTER_LABEL }] : []), ...(extras?.master ? [{ id: MASTER_BUS, label: MASTER_LABEL }] : []),
...(extras?.noFx ? [{ id: NO_FX, label: NO_ASSET_LABEL }] : []),
...assetRefChoices(project, extras?.assetTypes) ...assetRefChoices(project, extras?.assetTypes)
.filter((asset) => !left.has(asset.id)) .filter((asset) => !left.has(asset.id))
.map((asset) => ({ id: asset.id, label: assetChoiceLabel(asset) })), .map((asset) => ({ id: asset.id, label: assetChoiceLabel(asset) })),
+21 -10
View File
@@ -16,13 +16,16 @@ import { createFieldRow } from './fieldFactory';
import { addNameField, addNote, addTitle, type InspectorHost } from './host'; import { addNameField, addNote, addTitle, type InspectorHost } from './host';
import { previewingHandlers } from './previewHandlers'; import { previewingHandlers } from './previewHandlers';
const FX_ROW: PropertyDef = { const FX_ROW = {
key: 'fx', key: 'fx',
label: 'Sound FX', label: 'Sound FX',
kind: 'asset', kind: 'asset',
default: '', default: '',
assetTypes: ['soundfx'], assetTypes: ['soundfx'],
}; } as const satisfies PropertyDef;
/** What an item's pick says while it is empty (#352): what its asset names. */
const FROM_ASSET = 'From the asset';
const TRACK_ROWS: readonly PropertyDef[] = [ const TRACK_ROWS: readonly PropertyDef[] = [
{ {
@@ -72,9 +75,16 @@ function itemRows(item: SongItem): PropertyDef[] {
step: 1, step: 1,
decimals: 0, decimals: 0,
}, },
{ key: 'instrument', label: 'Instrument', kind: 'asset', default: '', assetTypes: ['instrument'] }, {
key: 'instrument',
label: 'Instrument',
kind: 'asset',
default: '',
assetTypes: ['instrument'],
none: FROM_ASSET,
},
); );
rows.push(FX_ROW); rows.push({ ...FX_ROW, none: FROM_ASSET, noFx: true });
return rows; return rows;
} }
@@ -185,12 +195,13 @@ function buildItem(host: InspectorHost, asset: SongAsset, item: SongItem, picked
host.fields.set(def.key, row); host.fields.set(def.key, row);
host.body.add(row.row); host.body.add(row.row);
} }
if (item.kind === 'pattern') addNote(
addNote( host,
host, item.kind === 'pattern'
'An empty Instrument plays the pattern with its own. A Sound FX takes all its notes together, before the one of the track.', ? 'A Sound FX takes all its notes together, instead of the one of the pattern and before the one of the track.'
'small', : 'A Sound FX takes the place of the one of the sound.',
); 'small',
);
} }
function itemOf(asset: SongAsset | undefined, itemIds: readonly string[]): SongItem | undefined { function itemOf(asset: SongAsset | undefined, itemIds: readonly string[]): SongItem | undefined {
+4 -1
View File
@@ -20,7 +20,9 @@ import {
} from '../../core/instrument/compile'; } from '../../core/instrument/compile';
import { MIDDLE_C, noteName } from '../../core/instrument/notes'; import { MIDDLE_C, noteName } from '../../core/instrument/notes';
import type { Asset, InstrumentAsset } from '../../core/model/Asset'; import type { Asset, InstrumentAsset } from '../../core/model/Asset';
import { instrumentBus, playNote } from '../../runtime/systems/behaviorContext'; import { fxVoiceOf } from '../../core/music/needs';
import { DEFAULT_TEMPO } from '../../core/music/time';
import { instrumentBus, instrumentFx, playNote } from '../../runtime/systems/behaviorContext';
import { import {
Button, Button,
Dropdown, Dropdown,
@@ -165,6 +167,7 @@ export class InstrumentEditor extends AssetTabEditor<InstrumentAsset, GraphCanva
PREVIEW_OWNER, PREVIEW_OWNER,
{ ...this.values }, { ...this.values },
{ bus: instrumentBus(this.state.project, asset.id, '') }, { bus: instrumentBus(this.state.project, asset.id, '') },
fxVoiceOf(this.state.project, instrumentFx(this.state.project, asset.id, ''), DEFAULT_TEMPO),
); );
if (voice) this.voices.push(voice); if (voice) this.voices.push(voice);
return voice; return voice;
+16 -4
View File
@@ -18,7 +18,10 @@ import {
import type { Asset, PatternAsset } from '../../core/model/Asset'; import type { Asset, PatternAsset } from '../../core/model/Asset';
import { textOf } from '../../core/model/Properties'; import { textOf } from '../../core/model/Properties';
import { findAssetOf } from '../../core/model/Project'; import { findAssetOf } from '../../core/model/Project';
import { playNote } from '../../runtime/systems/behaviorContext'; import { fxVoiceOf } from '../../core/music/needs';
import { assetFxOf } from '../../core/properties/assetFx';
import { DEFAULT_TEMPO } from '../../core/music/time';
import { instrumentFx, playNote } from '../../runtime/systems/behaviorContext';
import { Button, Dropdown, Label, Menu, type Widget } from '../../ui'; import { Button, Dropdown, Label, Menu, type Widget } from '../../ui';
import { Music } from '../../viewport/music'; import { Music } from '../../viewport/music';
import { playedBus } from '../../core/music/mixer'; import { playedBus } from '../../core/music/mixer';
@@ -51,10 +54,14 @@ export class PatternEditor extends AssetTabEditor<PatternAsset, PianoRoll> {
readonly ghostsButton: Button; readonly ghostsButton: Button;
/** Where the ghost notes come from: song, track and bar. */ /** Where the ghost notes come from: song, track and bar. */
readonly ghostSource = new Label('', { style: 'muted' }); readonly ghostSource = new Label('', { style: 'muted' });
/** Heard on the bus of the asset and the master, as in the game (#325). */ /** Heard through the sound FX (#352) and on the bus of the asset and the master, as in the game (#325). */
readonly music = new Music( readonly music = new Music(
() => this.state.project, () => this.state.project,
() => playedBus(this.state.project, '', this.asset()?.props ?? {}), () => playedBus(this.state.project, '', this.asset()?.props ?? {}),
() => {
const asset = this.asset();
return asset && fxVoiceOf(this.state.project, assetFxOf(asset.props), patternTempo(asset.props));
},
); );
/** The pattern playing, `''` while none is. */ /** The pattern playing, `''` while none is. */
private playingId = ''; private playingId = '';
@@ -90,7 +97,10 @@ export class PatternEditor extends AssetTabEditor<PatternAsset, PianoRoll> {
this.ghostsButton.setToggled(true); this.ghostsButton.setToggled(true);
const roll = new PianoRoll(state); const roll = new PianoRoll(state);
roll.onPreview = (note, held) => this.preview(note, held); roll.onPreview = (note, held) => this.preview(note, held);
roll.onRelease = (voice) => soundLibrary.release(voice); roll.onRelease = (voice) => {
soundLibrary.release(voice);
this.music.idle();
};
roll.onTogglePlay = () => this.togglePlay(); roll.onTogglePlay = () => this.togglePlay();
roll.onStart = (tick) => { roll.onStart = (tick) => {
if (this.isPlaying) this.music.current?.seek(tick); if (this.isPlaying) this.music.current?.seek(tick);
@@ -167,13 +177,15 @@ export class PatternEditor extends AssetTabEditor<PatternAsset, PianoRoll> {
const instrument = asset ? patternInstrument(asset.props) : ''; const instrument = asset ? patternInstrument(asset.props) : '';
if (!instrument) return 0; if (!instrument) return 0;
const struck = { note, velocity: 1, volume: 1, duration: held ? 0 : PREVIEW_SECONDS }; const struck = { note, velocity: 1, volume: 1, duration: held ? 0 : PREVIEW_SECONDS };
if (!held) setTimeout(() => this.music.idle(), PREVIEW_SECONDS * 1000);
return playNote( return playNote(
{ project: this.state.project, audio: soundLibrary }, { project: this.state.project, audio: soundLibrary },
instrument, instrument,
struck, struck,
PREVIEW_OWNER, PREVIEW_OWNER,
{}, {},
{ bus: playedBus(this.state.project, '', asset?.props ?? {}) }, { bus: this.music.heardOn() },
fxVoiceOf(this.state.project, instrumentFx(this.state.project, instrument, ''), DEFAULT_TEMPO),
); );
} }
+9 -3
View File
@@ -12,12 +12,14 @@ import {
} from '../../core/music/song'; } from '../../core/music/song';
import { roundAt } from '../../core/music/ghosts'; import { roundAt } from '../../core/music/ghosts';
import { patternLength } from '../../core/music/pattern'; import { patternLength } from '../../core/music/pattern';
import { fxVoiceOf } from '../../core/music/needs';
import { songBuses } from '../../core/music/songBuses'; import { songBuses } from '../../core/music/songBuses';
import { sampleCount, takeName, takeStart, takeTrack, takeTrackIndex, wavBytes } from '../../core/music/take'; import { sampleCount, takeName, takeStart, takeTrack, takeTrackIndex, wavBytes } from '../../core/music/take';
import { ticksOfSeconds } from '../../core/music/time'; import { ticksOfSeconds } from '../../core/music/time';
import type { Asset, SongAsset } from '../../core/model/Asset'; import type { Asset, SongAsset } from '../../core/model/Asset';
import { findAssetOf } from '../../core/model/Project'; import { findAssetOf } from '../../core/model/Project';
import { textOf } from '../../core/model/Properties'; import { textOf } from '../../core/model/Properties';
import { assetFxOf } from '../../core/properties/assetFx';
import type { SoundStore } from '../../core/store/SoundStore'; import type { SoundStore } from '../../core/store/SoundStore';
import { Button, Dialog, Dropdown, Label, Menu, ui, VStack, type Widget } from '../../ui'; import { Button, Dialog, Dropdown, Label, Menu, ui, VStack, type Widget } from '../../ui';
import { Music } from '../../viewport/music'; import { Music } from '../../viewport/music';
@@ -82,10 +84,14 @@ export class SongEditor extends AssetTabEditor<SongAsset, SongTimeline> {
readonly routing: GraphCanvas; readonly routing: GraphCanvas;
private shown: SongView = 'timeline'; private shown: SongView = 'timeline';
private readonly pane = new VStack(); private readonly pane = new VStack();
/** Heard on the bus of the asset and the master, as in the game (#325). */ /** Heard through the sound FX (#352) and on the bus of the asset and the master, as in the game (#325). */
readonly music = new Music( readonly music = new Music(
() => this.state.project, () => this.state.project,
() => playedBus(this.state.project, '', this.asset()?.props ?? {}), () => playedBus(this.state.project, '', this.asset()?.props ?? {}),
() => {
const asset = this.asset();
return asset && fxVoiceOf(this.state.project, assetFxOf(asset.props), songTempo(asset.props));
},
); );
/** The song playing, `''` while none is. */ /** The song playing, `''` while none is. */
private playingId = ''; private playingId = '';
@@ -137,9 +143,9 @@ export class SongEditor extends AssetTabEditor<SongAsset, SongTimeline> {
const asset = this.asset(); const asset = this.asset();
if (asset && this.isPlaying) this.music.setBuses(songBuses(this.state.project, { ...asset, tracks })); if (asset && this.isPlaying) this.music.setBuses(songBuses(this.state.project, { ...asset, tracks }));
}; };
timeline.onOpenPattern = (item, tick) => { timeline.onOpenItem = (item, tick) => {
const song = this.asset(); const song = this.asset();
const pattern = findAssetOf(state.project, item.asset, 'pattern'); const pattern = item.kind === 'pattern' ? findAssetOf(state.project, item.asset, 'pattern') : undefined;
if (song && pattern) if (song && pattern)
state.patternPlaces.set(pattern.id, { state.patternPlaces.set(pattern.id, {
songId: song.id, songId: song.id,
+4 -4
View File
@@ -233,8 +233,8 @@ export class SongTimeline extends Widget {
onStart: (tick: number) => void = () => {}; onStart: (tick: number) => void = () => {};
/** The tracks as a slider in a head would leave them: heard at once, written when it lets go. */ /** The tracks as a slider in a head would leave them: heard at once, written when it lets go. */
onTracksPreview: (tracks: readonly SongTrack[]) => void = () => {}; onTracksPreview: (tracks: readonly SongTrack[]) => void = () => {};
/** A pattern item was double-clicked, at a tick of the song. */ /** An item was double-clicked, at a tick of the song. */
onOpenPattern: (item: SongItem, tick: number) => void = () => {}; onOpenItem: (item: SongItem, tick: number) => void = () => {};
onDiscardTake: () => void = () => {}; onDiscardTake: () => void = () => {};
constructor(private readonly state: EditorState) { constructor(private readonly state: EditorState) {
@@ -722,8 +722,8 @@ export class SongTimeline extends Widget {
this.gesture = { kind: 'empty', start: g, current: g, active: false, add: e.shiftKey }; this.gesture = { kind: 'empty', start: g, current: g, active: false, add: e.shiftKey };
return; return;
} }
if (pressCount(e, `song-item:${anchor.id}`) === 2 && anchor.kind === 'pattern') { if (pressCount(e, `song-item:${anchor.id}`) === 2) {
this.onOpenPattern(anchor, xToTick(this.view, g.x)); this.onOpenItem(anchor, xToTick(this.view, g.x));
return; return;
} }
const picked = this.pickedIds; const picked = this.pickedIds;
+6 -2
View File
@@ -17,6 +17,9 @@ import { Button, Dropdown, Label, Menu, NumberField } from '../../ui';
import { soundLibrary } from '../../viewport/SoundLibrary'; import { soundLibrary } from '../../viewport/SoundLibrary';
import { playedBus } from '../../core/music/mixer'; import { playedBus } from '../../core/music/mixer';
import { DRY_VOICE } from '../../core/soundfx/nodes'; import { DRY_VOICE } from '../../core/soundfx/nodes';
import { fxVoiceOf } from '../../core/music/needs';
import { DEFAULT_TEMPO } from '../../core/music/time';
import { assetFxOf } from '../../core/properties/assetFx';
import { AssetTabEditor } from '../AssetTabEditor'; import { AssetTabEditor } from '../AssetTabEditor';
import { claimTransport, releaseTransport } from '../music/transport'; import { claimTransport, releaseTransport } from '../music/transport';
import { HEADER_ICON_SVG } from '../panels/headerIcons'; import { HEADER_ICON_SVG } from '../panels/headerIcons';
@@ -151,10 +154,11 @@ export class SoundEditor extends AssetTabEditor<SoundAsset, WaveCanvas> {
this.stop(); this.stop();
claimTransport(this.halt); claimTransport(this.halt);
const props = resolveAssetProps(this.state.project, asset); const props = resolveAssetProps(this.state.project, asset);
// Through its bus and the master, as the game hears it (#325). // Through its sound FX (#352), its bus and the master, as the game hears it (#325).
const bus = playedBus(this.state.project, '', props); const bus = playedBus(this.state.project, '', props);
const fx = fxVoiceOf(this.state.project, assetFxOf(props), DEFAULT_TEMPO) ?? DRY_VOICE;
const file = playedSoundOf(this.state.project, asset); const file = playedSoundOf(this.state.project, asset);
this.voice = soundLibrary.play(file, volumeOf(props), false, PREVIEW_OWNER, DRY_VOICE, bus); this.voice = soundLibrary.play(file, volumeOf(props), false, PREVIEW_OWNER, fx, bus);
this.playingId = this.voice ? asset.id : ''; this.playingId = this.voice ? asset.id : '';
this.syncPlayButton(); this.syncPlayButton();
} }
+17 -9
View File
@@ -11,11 +11,11 @@ import { JOINT_CONNECTED_KEY } from '../../core/model/joint';
import { objectReference } from '../../core/model/objectRefs'; import { objectReference } from '../../core/model/objectRefs';
import { findAsset, findAssetOf, findScene, type Project } from '../../core/model/Project'; import { findAsset, findAssetOf, findScene, type Project } from '../../core/model/Project';
import { renderLayerAlong } from '../../core/model/renderLayers'; import { renderLayerAlong } from '../../core/model/renderLayers';
import { pickedFx } from '../../core/properties/assetFx';
import { playedSoundById, playedSoundOf } from '../../core/properties/playedSound'; import { playedSoundById, playedSoundOf } from '../../core/properties/playedSound';
import { hasRenderLayer } from '../../core/properties/PropertySchema'; import { hasRenderLayer } from '../../core/properties/PropertySchema';
import { resolveAssetProps } from '../../core/properties/resolve'; import { resolveAssetProps } from '../../core/properties/resolve';
import { loopsOf, volumeOf } from '../../core/properties/sounds'; import { loopsOf, volumeOf } from '../../core/properties/sounds';
import { soundFxBuildFor } from '../../core/soundfx/compile';
import { instrumentBuildFor, instrumentSourcesOf, throughScopedFx } from '../../core/instrument/compile'; import { instrumentBuildFor, instrumentSourcesOf, throughScopedFx } from '../../core/instrument/compile';
import { VELOCITY_PARAM } from '../../core/instrument/nodes'; import { VELOCITY_PARAM } from '../../core/instrument/nodes';
import { heldNote } from '../../core/instrument/notes'; import { heldNote } from '../../core/instrument/notes';
@@ -190,7 +190,7 @@ export function behaviorContext(
detached ? '' : entity.id, detached ? '' : entity.id,
params, params,
{ tempo, bus: instrumentBus(project, instrumentId, bus) }, { tempo, bus: instrumentBus(project, instrumentId, bus) },
fxVoiceOf(project, fx, tempo, fxParams), fxVoiceOf(project, instrumentFx(project, instrumentId, fx), tempo, fxParams),
), ),
playPattern: (patternId, volume, loop, detached, fx, fxParams, bus) => playPattern: (patternId, volume, loop, detached, fx, fxParams, bus) =>
playPattern(run, patternId, volume, loop, detached ? '' : entity.id, fx, fxParams, bus), playPattern(run, patternId, volume, loop, detached ? '' : entity.id, fx, fxParams, bus),
@@ -287,18 +287,22 @@ function playSound(
if (!asset?.file) return 0; if (!asset?.file) return 0;
const props = resolveAssetProps(project, asset); const props = resolveAssetProps(project, asset);
const level = Math.max(0, Math.min(1, volumeOf(props) * volume)); const level = Math.max(0, Math.min(1, volumeOf(props) * volume));
const fx = fxId ? findAssetOf(project, fxId, 'soundfx') : undefined; const fx = fxVoiceOf(project, pickedFx(fxId, props), tempo, fxParams);
const plan = fx && soundFxBuildFor(fx, project).compiled;
return audio.play( return audio.play(
playedSoundOf(project, asset), playedSoundOf(project, asset),
level, level,
loop ?? loopsOf(props), loop ?? loopsOf(props),
owner, owner,
plan ? { plan, params: fxParams, tempo } : DRY_VOICE, fx ?? DRY_VOICE,
playedBus(project, bus, props), playedBus(project, bus, props),
); );
} }
/** The sound FX a note of that instrument goes through, by the pick of a node (#352); '' for none. */
export function instrumentFx(project: Project, instrumentId: string, picked: string): string {
return pickedFx(picked, findAssetOf(project, instrumentId, 'instrument')?.props ?? {});
}
/** The bus of the mix a note of that instrument is heard on, by the pick of a node (#325). */ /** The bus of the mix a note of that instrument is heard on, by the pick of a node (#325). */
export function instrumentBus(project: Project, instrumentId: string, picked: string): string { export function instrumentBus(project: Project, instrumentId: string, picked: string): string {
return playedBus(project, picked, findAssetOf(project, instrumentId, 'instrument')?.props ?? {}); return playedBus(project, picked, findAssetOf(project, instrumentId, 'instrument')?.props ?? {});
@@ -375,7 +379,7 @@ function playPattern(
const level = heldLevel(volume); const level = heldLevel(volume);
const length = patternLength(asset); const length = patternLength(asset);
const tempo = patternTempo(asset.props); const tempo = patternTempo(asset.props);
const fx = fxVoiceOf(project, fxId, tempo, fxParams); const fx = fxVoiceOf(project, pickedFx(fxId, asset.props), tempo, fxParams);
return audio.playSequence( return audio.playSequence(
patternVoiceName(patternId), patternVoiceName(patternId),
{ {
@@ -413,7 +417,7 @@ function playSong(
if (!song) return 0; if (!song) return 0;
const sequence = songSequence(project, song); const sequence = songSequence(project, song);
const tempo = songTempo(song.props); const tempo = songTempo(song.props);
const fx = fxVoiceOf(project, fxId, tempo, fxParams); const fx = fxVoiceOf(project, pickedFx(fxId, song.props), tempo, fxParams);
return audio.playSequence( return audio.playSequence(
songVoiceName(songId), songVoiceName(songId),
{ {
@@ -434,7 +438,10 @@ function playSong(
); );
} }
/** A note the scheduler planned (#332), struck through its instrument at its volume, on its bus. */ /**
* A note the scheduler planned (#332), struck through its instrument and the sound FX that names
* (#352) at its volume, on its bus.
*/
export function strikeNote( export function strikeNote(
sources: Pick<ContextSources, 'project' | 'audio'>, sources: Pick<ContextSources, 'project' | 'audio'>,
note: ScheduledNote, note: ScheduledNote,
@@ -447,7 +454,8 @@ export function strikeNote(
duration: note.duration, duration: note.duration,
}; };
const timing = { at: note.at, tempo: note.tempo, ...(note.bus && { bus: note.bus }) }; const timing = { at: note.at, tempo: note.tempo, ...(note.bus && { bus: note.bus }) };
return playNote(sources, note.instrument, struck, owner, {}, timing); const fx = fxVoiceOf(sources.project, instrumentFx(sources.project, note.instrument, ''), note.tempo);
return playNote(sources, note.instrument, struck, owner, {}, timing, fx);
} }
/** A clip of a song (#334): its sound at its own volume times the item's, through its sound FX. */ /** A clip of a song (#334): its sound at its own volume times the item's, through its sound FX. */
+38 -7
View File
@@ -9,10 +9,13 @@ import { MASTER_MIX } from '../core/music/mixer';
import type { BusLevel } from '../core/music/playbacks'; import type { BusLevel } from '../core/music/playbacks';
import type { Sequence } from '../core/music/sequence'; import type { Sequence } from '../core/music/sequence';
import type { Project } from '../core/model/Project'; import type { Project } from '../core/model/Project';
import type { FxVoice } from '../core/soundfx/nodes';
import { playClip, strikeNote } from '../runtime/systems/behaviorContext'; import { playClip, strikeNote } from '../runtime/systems/behaviorContext';
import { soundLibrary } from './SoundLibrary'; import { soundLibrary } from './SoundLibrary';
import { wake } from './wake'; import { wake } from './wake';
let musicCount = 0;
/** One planned note as the player was given it, and where the audio clock stood then (test hook). */ /** One planned note as the player was given it, and where the audio clock stood then (test hook). */
export interface PlannedNote extends ScheduledNote { export interface PlannedNote extends ScheduledNote {
voice: number; voice: number;
@@ -38,17 +41,24 @@ export class Music {
/** The tracks last set, and when the chains of their sound FX are there (#346). */ /** The tracks last set, and when the chains of their sound FX are there (#346). */
private buses: readonly BusLevel[] = []; private buses: readonly BusLevel[] = [];
private busesReady: Promise<void> = Promise.resolve(); private busesReady: Promise<void> = Promise.resolve();
/** The bus the sound FX of the pattern or song is on (#352), into `output`; one per tab. */
private readonly fxBus = `music:${++musicCount}`;
/** Where what goes to the output lands: `fxBus` while there is a sound FX, else `output`. */
private landing = '';
private fxReady: Promise<void> = Promise.resolve();
/** The last notes and clips planned, newest last, for a check to hold against the clock. */ /** The last notes and clips planned, newest last, for a check to hold against the clock. */
readonly planned: PlannedNote[] = []; readonly planned: PlannedNote[] = [];
readonly clips: PlannedClip[] = []; readonly clips: PlannedClip[] = [];
/** /**
* `output` is the bus of the mix what it plays is heard on (#325): that of the pattern or song in * `output` is the bus of the mix what it plays is heard on (#325): that of the pattern or song in
* the tab, so it sounds as it would in the game. The master where left out. * the tab, so it sounds as it would in the game. The master where left out. `fx` is the sound FX of
* that pattern or song (#352), on the way there.
*/ */
constructor( constructor(
private readonly project: () => Project, private readonly project: () => Project,
private readonly output: () => string = () => MASTER_MIX, private readonly output: () => string = () => MASTER_MIX,
private readonly fx: () => FxVoice | null = () => null,
) {} ) {}
/** Loads what the notes and clips need, then plays from `from` ticks. */ /** Loads what the notes and clips need, then plays from `from` ticks. */
@@ -58,7 +68,8 @@ export class Music {
this.sequence = sequence; this.sequence = sequence;
this.planned.length = 0; this.planned.length = 0;
this.clips.length = 0; this.clips.length = 0;
await Promise.all([this.prepare(sequence), this.busesReady]); this.landing = this.heardOn();
await Promise.all([this.prepare(sequence), this.busesReady, this.fxReady]);
if (run !== this.run) return; if (run !== this.run) return;
this.scheduler = new Scheduler( this.scheduler = new Scheduler(
{ {
@@ -66,7 +77,7 @@ export class Music {
latency: () => soundLibrary.latency(), latency: () => soundLibrary.latency(),
ahead: () => ahead: () =>
Math.max( Math.max(
soundLibrary.busLatency(this.output()), soundLibrary.busLatency(this.landing),
...this.buses.map((bus) => soundLibrary.busLatency(bus.name)), ...this.buses.map((bus) => soundLibrary.busLatency(bus.name)),
), ),
play: (note) => this.playOne(note), play: (note) => this.playOne(note),
@@ -86,15 +97,35 @@ export class Music {
/** A change heard from the next window on; a note new to it loads what it needs meanwhile. */ /** A change heard from the next window on; a note new to it loads what it needs meanwhile. */
edit(sequence: Sequence): void { edit(sequence: Sequence): void {
this.sequence = sequence; this.sequence = sequence;
this.landing = this.heardOn();
void this.prepare(sequence); void this.prepare(sequence);
} }
/**
* Where what goes to the output of the pattern or song is heard: through its sound FX on a bus of
* its own, set as it is now, else straight on `output`. A key of the roll is played there too.
*/
heardOn(): string {
const fx = this.fx();
if (!fx) {
soundLibrary.releaseBuses([this.fxBus]);
return this.output();
}
this.fxReady = soundLibrary.setBuses([{ name: this.fxBus, gain: 1, pan: 0, into: this.output(), fx }]);
return this.fxBus;
}
/** Lets the bus of its sound FX go while nothing plays: after a key of the roll. */
idle(): void {
if (!this.scheduler) soundLibrary.releaseBuses([this.fxBus]);
}
/** /**
* The tracks of a song set to their level and pan, heard at once (#334), and to their sound FX * The tracks of a song set to their level and pan, heard at once (#334), and to their sound FX
* (#346). What goes to the output of the song goes to `output` (#350). A start waits for the chains. * (#346). What goes to the output of the song goes to `output` (#350). A start waits for the chains.
*/ */
setBuses(levels: readonly BusLevel[]): void { setBuses(levels: readonly BusLevel[]): void {
const output = this.output(); const output = (this.landing = this.heardOn());
this.buses = levels.map(({ sends, ...level }) => this.buses = levels.map(({ sends, ...level }) =>
sends sends
? { ...level, sends: sends.map((send) => ({ ...send, to: send.to ?? output })) } ? { ...level, sends: sends.map((send) => ({ ...send, to: send.to ?? output })) }
@@ -106,7 +137,7 @@ export class Music {
/** Its buses go once what is on them has faded, unless the next start uses them again. */ /** Its buses go once what is on them has faded, unless the next start uses them again. */
stop(): void { stop(): void {
this.halt(); this.halt();
soundLibrary.releaseBuses(this.buses.map((bus) => bus.name)); soundLibrary.releaseBuses([...this.buses.map((bus) => bus.name), this.fxBus]);
} }
private halt(): void { private halt(): void {
@@ -120,7 +151,7 @@ export class Music {
} }
private playOne(note: ScheduledNote): number { private playOne(note: ScheduledNote): number {
const heard = { ...note, bus: note.bus || this.output() }; const heard = { ...note, bus: note.bus || this.landing };
const voice = strikeNote({ project: this.project(), audio: soundLibrary }, heard, ''); const voice = strikeNote({ project: this.project(), audio: soundLibrary }, heard, '');
this.planned.push({ ...note, voice, planned: soundLibrary.clock() }); this.planned.push({ ...note, voice, planned: soundLibrary.clock() });
if (this.planned.length > 256) this.planned.shift(); if (this.planned.length > 256) this.planned.shift();
@@ -128,7 +159,7 @@ export class Music {
} }
private playClip(clip: ScheduledClip): number { private playClip(clip: ScheduledClip): number {
const heard = { ...clip, bus: clip.bus || this.output() }; const heard = { ...clip, bus: clip.bus || this.landing };
const voice = playClip({ project: this.project(), audio: soundLibrary }, heard); const voice = playClip({ project: this.project(), audio: soundLibrary }, heard);
this.clips.push({ ...clip, voice, planned: soundLibrary.clock() }); this.clips.push({ ...clip, voice, planned: soundLibrary.clock() });
if (this.clips.length > 256) this.clips.shift(); if (this.clips.length > 256) this.clips.shift();
+231
View File
@@ -0,0 +1,231 @@
/**
* A sound FX on a sound, an instrument, a pattern and a song (#352): what plays it goes through it
* unless a Play node or an item of a song picks another, or None.
*/
import { describe, expect, it } from 'vitest';
import {
CreateAssetCommand,
CreateSoundCommand,
SetAssetPropertyCommand,
} from '../../src/core/commands/assetCommands';
import { SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
import { SetSongCommand } from '../../src/core/commands/songCommands';
import type { BehaviorAsset, InstrumentAsset, SongAsset, SoundFxAsset } from '../../src/core/model/Asset';
import type { AssetType } from '../../src/core/model/AssetTypes';
import type { PropertyBag } from '../../src/core/model/Properties';
import { sequenceNeeds } from '../../src/core/music/needs';
import type { SequencePlay } from '../../src/core/music/playbacks';
import { itemBus, newItem, songSequence, trackItemsBus } from '../../src/core/music/song';
import { songBuses } from '../../src/core/music/songBuses';
import { ASSET_FX_KEY, NO_FX, pickedFx } from '../../src/core/properties/assetFx';
import { schemaFor } from '../../src/core/properties/PropertySchema';
import { soundFxBuildFor } from '../../src/core/soundfx/compile';
import { DRY_VOICE, type FxParams, type FxPlan, type FxVoice } from '../../src/core/soundfx/nodes';
import { type AudioSink, SILENT_AUDIO } from '../../src/runtime/systems/Audio';
import { graphEditor, makeBehavior, makeSpriteScene, place, startWorld } from '../helpers';
const FILE = `${'c4'.repeat(32)}.wav`;
const Q = 480;
const BAR = 4 * Q;
/** A sound FX of one Gain of `db`. */
function gainFx(f: ReturnType<typeof makeSpriteScene>, name: string, db: number): SoundFxAsset {
const made = new CreateAssetCommand('soundfx', name);
f.stack.execute(made);
const fx = made.asset as SoundFxAsset;
const edit = graphEditor(f, fx as unknown as BehaviorAsset);
const input = edit.addNode('SoundFxInput');
const gain = edit.addNode('SoundFxGain', { gain: db });
const output = edit.addNode('SoundFxOutput');
edit.wire([
[input, 'out', gain, 'in'],
[gain, 'out', output, 'in'],
]);
return fx;
}
/**
* A sound, an instrument, a pattern and a song, each through a sound FX of its own: Sound -1 dB,
* Lead -2, Riff -3, Theme -4, and Other -5 for a pick. The song has a sound item and a pattern item.
*/
function scene() {
const f = makeSpriteScene('Box');
const [soundFx, leadFx, riffFx, themeFx, other] = [-1, -2, -3, -4, -5].map((db, i) =>
gainFx(f, ['Sound', 'Lead', 'Riff', 'Theme', 'Other'][i]!, db),
) as [SoundFxAsset, SoundFxAsset, SoundFxAsset, SoundFxAsset, SoundFxAsset];
const sound = new CreateSoundCommand('Blip', FILE);
f.stack.execute(sound);
const ins = new CreateAssetCommand('instrument', 'Lead');
f.stack.execute(ins);
const lead = ins.asset as InstrumentAsset;
lead.graph.nodes.find((n) => n.type === 'InstrumentSample')!.params.sound = sound.asset.id;
const pattern = new CreateAssetCommand('pattern', 'Riff');
f.stack.execute(pattern);
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'instrument', lead.id));
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'bars', 1));
f.stack.execute(
new SetPatternNotesCommand(pattern.asset.id, [{ id: 'n', tick: 0, length: Q, note: 60, velocity: 1 }]),
);
const created = new CreateAssetCommand('song', 'Theme');
f.stack.execute(created);
const songId = created.asset.id;
const song = () => f.project.assets.find((a) => a.id === songId) as SongAsset;
const track = song().tracks[0]!.id;
const clip = newItem({ track, kind: 'sound', asset: sound.asset.id, tick: 0, length: BAR });
const riff = newItem({ track, kind: 'pattern', asset: pattern.asset.id, tick: 0, length: BAR });
f.stack.execute(new SetSongCommand(songId, { items: [clip, riff] }));
for (const [id, fx] of [
[sound.asset.id, soundFx],
[lead.id, leadFx],
[pattern.asset.id, riffFx],
[songId, themeFx],
] as const)
f.stack.execute(new SetAssetPropertyCommand(id, ASSET_FX_KEY, fx.id));
const setItems = (clipFx: string, riffFx: string) =>
f.stack.execute(
new SetSongCommand(songId, {
items: song().items.map((i) => ({ ...i, fx: i.kind === 'sound' ? clipFx : riffFx })),
}),
);
const behavior = makeBehavior(f, 'Logic');
place(f, behavior.id, { x: 0, y: 0 }, place(f, f.spriteId, { x: 0, y: 0 }));
const edit = graphEditor(f, behavior);
const plan = (fx: SoundFxAsset) => soundFxBuildFor(fx).compiled!;
return {
...f,
fx: { soundFx, leadFx, riffFx, themeFx, other },
plan,
sound: sound.asset.id,
lead,
pattern: pattern.asset.id,
song,
songId,
track,
riff: riff.id,
setItems,
edit,
};
}
function recording() {
const sounds: FxVoice[] = [];
const notes: { layers: readonly FxPlan[]; params: FxParams }[] = [];
const sequences: { name: string; play: SequencePlay }[] = [];
const sink: AudioSink = {
...SILENT_AUDIO,
play: (_file, _volume, _loop, _owner, fx) => (sounds.push(fx ?? DRY_VOICE), 3),
playNote: (_name, layers, params) => (notes.push({ layers, params }), 7),
playSequence: (name, play) => (sequences.push({ name, play }), 500),
};
return Object.assign(sink, { sounds, notes, sequences });
}
/** The gains of a layer's stages, as its plan has them. */
function gains(layer: FxPlan | undefined): number[] {
return (layer?.stages ?? []).flatMap((stage) =>
stage.kind === 'gain' && 'db' in stage ? [stage.db.value] : [],
);
}
describe('the Sound FX row (#352)', () => {
it('stands right before Bus on a sound, an instrument, a pattern and a song', () => {
for (const type of ['sound', 'instrument', 'pattern', 'song'] as AssetType[]) {
const keys = schemaFor(type).map((def) => def.key);
expect(keys.indexOf(ASSET_FX_KEY), type).toBe(keys.indexOf('bus') - 1);
expect(schemaFor(type).find((def) => def.key === ASSET_FX_KEY)).toMatchObject({
kind: 'asset',
default: '',
assetTypes: ['soundfx'],
});
}
});
it('is the one a pick leaves empty, none for None, and the pick itself for any other', () => {
const props = { [ASSET_FX_KEY]: 'own' };
expect(pickedFx('', props)).toBe('own');
expect(pickedFx(NO_FX, props)).toBe('');
expect(pickedFx('picked', props)).toBe('picked');
expect(pickedFx('', {})).toBe('');
});
});
describe('the sound FX of what a song plays (#352)', () => {
it('takes the one of the sound and of the pattern for an item that picks none', () => {
const f = scene();
const { clips, notes } = songSequence(f.project, f.song());
expect(clips?.map((c) => c.fx)).toEqual([f.fx.soundFx.id]);
expect(notes.map((n) => n.bus)).toEqual([itemBus(f.songId, { id: f.riff, track: f.track })]);
const bus = songBuses(f.project, f.song()).find(
(b) => b.name === itemBus(f.songId, { id: f.riff, track: f.track }),
);
expect(bus?.fx?.plan).toBe(f.plan(f.fx.riffFx));
});
it('takes the one an item picks instead, and none for None', () => {
const f = scene();
f.setItems(f.fx.other.id, f.fx.other.id);
expect(songSequence(f.project, f.song()).clips?.map((c) => c.fx)).toEqual([f.fx.other.id]);
const picked = songBuses(f.project, f.song()).find((b) => b.name.endsWith(f.riff));
expect(picked?.fx?.plan).toBe(f.plan(f.fx.other));
f.setItems(NO_FX, NO_FX);
const { clips, notes } = songSequence(f.project, f.song());
expect(clips?.map((c) => c.fx)).toEqual(['']);
// Without a sound FX the pattern item has no bus of its own; its notes go in with the track's.
expect(notes.map((n) => n.bus)).toEqual([trackItemsBus(f.songId, f.track)]);
expect(songBuses(f.project, f.song()).some((b) => b.name.endsWith(f.riff))).toBe(false);
});
it('loads the sound FX of an instrument with the layers of its notes', () => {
const f = scene();
const { layers } = sequenceNeeds(f.project, songSequence(f.project, f.song()));
expect(layers).toContain(f.plan(f.fx.leadFx));
});
});
describe('the sound FX of what a Play node plays (#352)', () => {
/** What one Play node plays on a scene of its own, its params given the scene. */
function played(type: string, params: (f: ReturnType<typeof scene>) => PropertyBag) {
const f = scene();
const start = f.edit.addNode('OnStart');
const play = f.edit.addNode(type, params(f));
f.edit.wire([[start, 'exec', play, 'exec']]);
const audio = recording();
startWorld(f, { behaviors: true, audio }).frame();
return Object.assign(audio, { f });
}
const planOf = (f: ReturnType<typeof scene>, name: keyof ReturnType<typeof scene>['fx']) =>
f.plan(f.fx[name]);
it('plays a sound through its own unless the node picks another or None', () => {
const own = played('PlaySound', (f) => ({ sound: f.sound }));
expect(own.sounds[0]?.plan).toBe(planOf(own.f, 'soundFx'));
const other = played('PlaySound', (f) => ({ sound: f.sound, fx: f.fx.other.id }));
expect(other.sounds[0]?.plan).toBe(planOf(other.f, 'other'));
expect(played('PlaySound', (f) => ({ sound: f.sound, fx: NO_FX })).sounds[0]).toBe(DRY_VOICE);
});
it('plays a note through the sound FX of its instrument after what the instrument does', () => {
const own = played('PlayNote', (f) => ({ instrument: f.lead.id, note: 60 })).notes[0];
expect(gains(own?.layers[0])).toEqual([-2]);
const picked = played('PlayNote', (f) => ({ instrument: f.lead.id, note: 60, soundFx: f.fx.other.id }))
.notes[0];
expect(gains(picked?.layers[0])).toEqual([-5]);
const none = played('PlayNote', (f) => ({ instrument: f.lead.id, note: 60, soundFx: NO_FX })).notes[0];
expect(gains(none?.layers[0])).toEqual([]);
});
it('plays a pattern and a song through theirs, and every note of them through the instrument’s', () => {
const pattern = played('PlayPattern', (f) => ({ pattern: f.pattern }));
expect(pattern.sequences[0]?.play.fx?.plan).toBe(planOf(pattern.f, 'riffFx'));
expect(
played('PlayPattern', (f) => ({ pattern: f.pattern, fx: NO_FX })).sequences[0]?.play.fx,
).toBeUndefined();
const song = played('PlaySong', (f) => ({ song: f.songId }));
expect(song.sequences[0]?.play.fx?.plan).toBe(planOf(song.f, 'themeFx'));
const other = played('PlaySong', (f) => ({ song: f.songId, fx: f.fx.other.id }));
expect(other.sequences[0]?.play.fx?.plan).toBe(planOf(other.f, 'other'));
const asked = pattern.sequences[0]!.play;
asked.strike({ ...asked.sequence.notes[0]!, at: 0, duration: 0.1, tempo: 120 });
expect(gains(pattern.notes.at(-1)?.layers[0])).toEqual([-2]);
});
});
+1
View File
@@ -55,6 +55,7 @@ describe('pattern asset (#333)', () => {
'beats', 'beats',
'beat', 'beat',
'tempo', 'tempo',
'fx',
'bus', 'bus',
]); ]);
}); });
+1
View File
@@ -75,6 +75,7 @@ describe('the song asset (#334)', () => {
'beats', 'beats',
'beat', 'beat',
'grid', 'grid',
'fx',
'bus', 'bus',
]); ]);
expect(songGrid(f.song().props)).toBe(BAR); expect(songGrid(f.song().props)).toBe(BAR);
+1
View File
@@ -138,6 +138,7 @@ describe('the sound asset', () => {
FADE_IN_CURVE_KEY, FADE_IN_CURVE_KEY,
FADE_OUT_KEY, FADE_OUT_KEY,
FADE_OUT_CURVE_KEY, FADE_OUT_CURVE_KEY,
'fx',
BUS_KEY, BUS_KEY,
]); ]);
expect(cutOf(asset.props)).toEqual({ expect(cutOf(asset.props)).toEqual({
+75
View File
@@ -976,6 +976,81 @@ export async function song(t) {
await click(p.x, p.y); await click(p.x, p.y);
await sleep(150); await sleep(150);
// 65n. A sound FX on the song and on its items (#352). An item's Sound FX and a pattern item's
// Instrument are the ones of its asset while empty; None is a pick of its own. The song
// plays through its own on a bus of the tab, into the bus of the song. A double-click on a
// sound item opens the sound.
await evaluate('__gm.sound.stopAll()');
// The items as they are now: those of 65c went with the takes of 65l.
const soundItem = (await model()).items.find((item) => item.kind === 'sound');
const patternItem = (await model()).items.find((item) => item.kind === 'pattern');
p = await json(`__gm.song.itemAt(${JSON.stringify(soundItem.id)}, 'body')`);
await click(p.x, p.y);
await sleep(150);
p = await json("__gm.inspectorField('Sound FX', true)");
await click(p.x, p.y);
await sleep(150);
const fxRows = await json('__gm.menuRows()');
await clickText('None');
await sleep(150);
const clipFx = (await model()).items.find((item) => item.id === soundItem.id)?.fx;
p = await json(`__gm.song.itemAt(${JSON.stringify(patternItem.id)}, 'body')`);
await click(p.x, p.y);
await sleep(150);
p = await json("__gm.inspectorField('Instrument', true)");
await click(p.x, p.y);
await sleep(150);
const instrumentRows = await json('__gm.menuRows()');
await press('Escape');
check(
"an item's Sound FX offers From the asset, None and the sound FX, its Instrument From the asset",
fxRows?.[0] === 'From the asset' &&
fxRows[1] === 'None' &&
fxRows.includes('Octave 65 (soundfx)') &&
clipFx === '#none' &&
instrumentRows?.[0] === 'From the asset',
JSON.stringify({ fxRows, clipFx, instrumentRows }),
);
await evaluate(`__gm.state.select({ kind: 'asset', assetId: ${JSON.stringify(id)} })`);
await sleep(150);
p = await json("__gm.inspectorField('Sound FX', true)");
await click(p.x, p.y);
await sleep(150);
await clickText('Octave 65 (soundfx)');
await sleep(150);
const songFx = await json(`__gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)}).props.fx`);
p = await json('__gm.song.playAt()');
await click(p.x, p.y);
const throughSong = await waitFor(
() => json('__gm.song.buses()'),
(list) => list.some((b) => b.name.startsWith('music:') && b.fx),
);
const songFxBus = throughSong.find((b) => b.name.startsWith('music:') && b.fx);
await click(p.x, p.y);
check(
'the Sound FX of the song is on a bus of the tab into the bus of the song, the tracks to the output go there',
songFx === 'sg65-fx' &&
songFxBus?.into === 'mix' &&
Math.abs(songFxBus.latency - 2048 / (await json('__gm.sound.sampleRate()'))) < 1e-9 &&
throughSong.find((b) => b.name === busOf(ids[1]))?.sends?.[0]?.to === songFxBus.name,
JSON.stringify({ songFx, songFxBus, sends: throughSong.find((b) => b.name === busOf(ids[1]))?.sends }),
);
const fxGone = await waitFor(
() => json('__gm.song.buses()'),
(list) => !list.some((b) => b.name.startsWith('music:')),
);
p = await json(`__gm.song.itemAt(${JSON.stringify(soundItem.id)}, 'body')`);
await doubleClick(p.x, p.y);
await sleep(200);
const openedSound = await json('{ tab: __gm.tab(), sound: __gm.soundTab.current() }');
check(
'stopping lets the bus of the sound FX go, and a double-click on a sound item opens the sound',
!fxGone.some((b) => b.name.startsWith('music:')) &&
openedSound.tab === 'sound' &&
openedSound.sound === SOUND,
JSON.stringify({ fxGone: fxGone.map((b) => b.name), openedSound }),
);
// 65j. The step takes back everything it made. // 65j. The step takes back everything it made.
await evaluate('__gm.sound.stopAll()'); await evaluate('__gm.sound.stopAll()');
await mutate(` await mutate(`