Give sounds, instruments, patterns and songs a sound FX (#352)
A sound FX could only be picked where something was played: on a Play
node, a track or an item of a song. A sound that should always sound
through the same effect needed that pick everywhere it was used. Now
the asset says it once, and every place that plays it follows.
- New row Sound FX on a sound, an instrument, a pattern and a song,
right before Bus. A sound goes through it on every voice, an
instrument on every note after its graph, a pattern on all its
notes together and a song on its output.
- The fx input of Play sound, Play note, Play pattern and Play song
and the Sound FX of a song item now read "From the asset" while
empty. None is a pick of its own, stored as `#none` like `#master`.
A pick replaces the FX of the asset, it does not come on top.
- A pattern item's Instrument reads "From the asset" instead of None.
The meaning is unchanged.
- A double-click on a sound item opens the sound, as one on a pattern
item opens the pattern.
- The tabs play through the FX of their asset. The Pattern and Song
tab put it on a bus of their own between the output and the bus of
the asset. The keys of the Pattern tab play there too.
- Notes of a pattern or song go through the FX of their instrument.
Its chain is loaded with the other needs of the sequence.
Decided without asking:
- Old items and Play nodes with an empty fx now mean "From the
asset". No asset has an FX yet, so nothing sounds different.
- The label is "From the asset" as on Loop and Bus, not "From asset".
- A sound used as a Sample, as the impulse of a Reverb or as the test
sound of the Sound FX tab plays without its own FX.
Findings:
- In e2e, `__gm.inspectorField('Sound FX')` and `('Instrument')` find
the group of the Assets panel first. The check asks for the last
match.
Tests:
- Unit asset-fx (new): 8 green. Six counter-checks each turned red.
- Field lists of sound, pattern and song: Sound FX added.
- e2e 65 (melektaus-atx): ALL PASSED, 46 checks (3 new). Counter-check
without the FX bus of the tab: that check red.
- Gate (melektaus-atx): CHECK EXIT 0, 268 files, 4142 tests.
- Not checked: how it sounds, the Sound, Instrument and Pattern tab
in the browser.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
d349251a01
commit
a295d233ed
26 files changed
+521
-60
No files matched your search
@@ -238,7 +238,8 @@ export interface NodeContext {
|
||||
* The voice belongs to the behavior that started it and ends when that behavior is destroyed –
|
||||
* a sound is something a thing in the world is doing. `detached` gives that up on purpose, for
|
||||
* the one case where the rule is in the way: the death sound of an object that destroys itself.
|
||||
* `fx` names a sound FX asset the voice is played through (#311); `''` plays it as it is.
|
||||
* `fx` names a sound FX asset the voice is played through (#311); `''` the one the sound asset
|
||||
* names (#352), `NO_FX` none.
|
||||
* `fxParams` are values for its parameters; one not given is the default the FX declares.
|
||||
* `tempo` is what a Delay synced to beats counts by (#332). `bus` is the bus it is heard on
|
||||
* (#325): a bus asset or `MASTER_BUS`, `''` for the one the sound asset names.
|
||||
@@ -259,7 +260,8 @@ export interface NodeContext {
|
||||
* `velocity` (0 to 1, how loud it is next to `volume`) and released after `duration` seconds, or
|
||||
* held until it is stopped where that is 0. Owned like a `playSound` voice, `instrumentParams` are
|
||||
* values for the instrument's parameters, `tempo` as for `playSound`. `fx` sends the note through a
|
||||
* sound FX of its own (#346), `fxParams` with values for its parameters, apart from the instrument's.
|
||||
* sound FX of its own (#346), `fxParams` with values for its parameters, apart from the instrument's;
|
||||
* `''` the one the instrument names, `NO_FX` none (#352).
|
||||
* `bus` as for `playSound`, `''` for the one the instrument names.
|
||||
* Returns the number of the voice, 0 where nothing plays.
|
||||
*/
|
||||
@@ -279,7 +281,8 @@ export interface NodeContext {
|
||||
/**
|
||||
* Plays a pattern asset through its instrument, at its own tempo, once or round and round, every
|
||||
* note at `volume`. Owned like a `playSound` voice. `fx` is a sound FX all of it goes through as
|
||||
* one chain (#346), `fxParams` values for its parameters, `bus` as for `playSound`. Returns its
|
||||
* one chain (#346), `''` the one the pattern names, `NO_FX` none (#352); `fxParams` values for its
|
||||
* parameters, `bus` as for `playSound`. Returns its
|
||||
* number, for `stopVoice` and `tuneSound`; 0 where there is nothing to play.
|
||||
*/
|
||||
playPattern(
|
||||
|
||||
@@ -10,7 +10,7 @@ export const PlayNote: NodeDef = {
|
||||
label: 'Play note',
|
||||
category: 'action',
|
||||
description:
|
||||
'Plays one note of an instrument asset. "note" is its number as MIDI counts it: 60 is C4, 61 C#4, 72 C5. "velocity" is how hard it is struck, 0 to 1 – it makes the note quieter and is handed to what the instrument does with it. "duration" is how many seconds until the note is released; at 0 it is held until a Stop sound ends it. A note belongs to this object like a sound and stops when the object is destroyed, unless "detached" is on. "params" takes values for the parameters of the instrument from an Instrument parameters node. "fx" sends the note through a sound FX on top of what the instrument does, "fx params" takes values for its parameters from a Sound FX parameters node – the two sets stay apart. "voice" is the number of this note, for Set sound FX parameters and Stop sound; 0 where the instrument plays nothing for it. "tempo" is what a Delay in the instrument synced to beats counts by, in quarter notes a minute. "bus" is the bus it is heard on; left empty, the one the instrument names.',
|
||||
'Plays one note of an instrument asset. "note" is its number as MIDI counts it: 60 is C4, 61 C#4, 72 C5. "velocity" is how hard it is struck, 0 to 1 – it makes the note quieter and is handed to what the instrument does with it. "duration" is how many seconds until the note is released; at 0 it is held until a Stop sound ends it. A note belongs to this object like a sound and stops when the object is destroyed, unless "detached" is on. "params" takes values for the parameters of the instrument from an Instrument parameters node. "fx" sends the note through a sound FX on top of what the instrument does; left empty, the one the instrument names, and None through none. "fx params" takes values for its parameters from a Sound FX parameters node – the two sets stay apart. "voice" is the number of this note, for Set sound FX parameters and Stop sound; 0 where the instrument plays nothing for it. "tempo" is what a Delay in the instrument synced to beats counts by, in quarter notes a minute. "bus" is the bus it is heard on; left empty, the one the instrument names.',
|
||||
inputs: [
|
||||
exec(),
|
||||
{
|
||||
|
||||
@@ -21,7 +21,8 @@ export const PlayPattern: NodeDef = {
|
||||
'it quieter. Each play is a playing of its own: firing this twice plays it twice, overlapping. ' +
|
||||
'It belongs to this object like a sound and stops when the object is destroyed, unless ' +
|
||||
'"detached" is on. "fx" sends all of it through one sound FX, its notes together: an LFO runs on ' +
|
||||
'across them, an echo repeats the whole. "fx params" takes values for its parameters from a Sound ' +
|
||||
'across them, an echo repeats the whole. Left empty, it is the one the pattern names; None is ' +
|
||||
'none. "fx params" takes values for its parameters from a Sound ' +
|
||||
'FX parameters node. "playback" is the number of this playing, for Stop pattern and for Set sound ' +
|
||||
'FX parameters; 0 where the pattern has no instrument or no notes. "bus" is the bus all of it is ' +
|
||||
'heard on; left empty, the one the pattern names.',
|
||||
|
||||
@@ -22,6 +22,7 @@ export const PlaySong: NodeDef = {
|
||||
'playing of its own: firing this twice plays it twice, side by side. It belongs to this object ' +
|
||||
'like a sound and stops when the object is destroyed or the scene is left, unless "detached" ' +
|
||||
'is on. "fx" sends the whole song through one sound FX, after its tracks and before "volume"; ' +
|
||||
'left empty, the one the song names, and None is none. ' +
|
||||
'"fx params" takes values for its parameters from a Sound FX parameters node. "playback" is the ' +
|
||||
'number of this playing, for Stop song and for Set sound FX parameters; 0 where the song has ' +
|
||||
'nothing to play. "bus" is the bus all of it is heard on; left empty, the one the song names.',
|
||||
|
||||
@@ -59,7 +59,8 @@ export const PlaySound: NodeDef = {
|
||||
"leaves the asset's own setting alone or overrules it, so the asset says what the sound is and " +
|
||||
'the node only colors it. Each play is a voice of its own: firing this twice in a row is heard ' +
|
||||
'twice, overlapping. The sound belongs to this object and stops when the object is destroyed, ' +
|
||||
'unless "detached" is on. "fx" sends it through a sound FX asset; left empty, it plays as it is. ' +
|
||||
'unless "detached" is on. "fx" sends it through a sound FX asset instead of the one the sound ' +
|
||||
'names; left empty, it takes that one, and None plays it as it is. ' +
|
||||
'"fx params" takes values for its parameters from a Sound FX parameters node. "voice" is the ' +
|
||||
'number of this one play, for Set sound FX parameters to change it while it plays; 0 where ' +
|
||||
'nothing was played. "length" is how many seconds one run of the sound lasts, to hand to the ' +
|
||||
|
||||
@@ -94,12 +94,23 @@ export const soundPort = (): PortDef => ({
|
||||
field: { key: 'sound', label: 'Sound', kind: 'sound', default: '' },
|
||||
});
|
||||
|
||||
/** The `fx` input of `Play sound` (#311); unconnected, it shows the picker of sound FX assets. */
|
||||
/**
|
||||
* The `fx` input of a Play node (#311); unconnected, it shows the picker of sound FX assets. Empty is
|
||||
* the one its asset names, `NO_FX` none (#352).
|
||||
*/
|
||||
export const soundFxPort = (): PortDef => ({
|
||||
key: 'fx',
|
||||
label: 'fx',
|
||||
type: 'string',
|
||||
field: { key: 'fx', label: 'FX', kind: 'asset', default: '', assetTypes: ['soundfx'] },
|
||||
field: {
|
||||
key: 'fx',
|
||||
label: 'FX',
|
||||
kind: 'asset',
|
||||
default: '',
|
||||
assetTypes: ['soundfx'],
|
||||
none: 'From the asset',
|
||||
noFx: true,
|
||||
},
|
||||
});
|
||||
|
||||
/**
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { instrumentBuildFor, instrumentSourcesOf } from '../instrument/compile';
|
||||
import { findAssetOf, type Project } from '../model/Project';
|
||||
import { assetFxOf } from '../properties/assetFx';
|
||||
import { playedSoundById } from '../properties/playedSound';
|
||||
import { soundFxBuildFor } from '../soundfx/compile';
|
||||
import { DRY_VOICE, type FxParams, type FxPlan, type FxVoice } from '../soundfx/nodes';
|
||||
@@ -26,7 +27,7 @@ export function clipFx(project: Project, fxId: string, tempo: number): FxVoice {
|
||||
return fxVoiceOf(project, fxId, tempo) ?? DRY_VOICE;
|
||||
}
|
||||
|
||||
/** Every layer of every note and every file and FX of every clip a sequence plays. */
|
||||
/** Every layer and FX of every note and every file and FX of every clip a sequence plays. */
|
||||
export function sequenceNeeds(project: Project, sequence: Sequence): SequenceNeeds {
|
||||
const sources = instrumentSourcesOf(project);
|
||||
const seen = new Set<string>();
|
||||
@@ -35,6 +36,11 @@ export function sequenceNeeds(project: Project, sequence: Sequence): SequenceNee
|
||||
const key = `${instrument}\u0000${note}`;
|
||||
const asset = findAssetOf(project, instrument, 'instrument');
|
||||
if (seen.has(key) || !asset) continue;
|
||||
if (!seen.has(instrument)) {
|
||||
seen.add(instrument);
|
||||
const fx = fxVoiceOf(project, assetFxOf(asset.props), 120);
|
||||
if (fx) layers.push(fx.plan);
|
||||
}
|
||||
seen.add(key);
|
||||
layers.push(...instrumentBuildFor(asset, note, sources).layers);
|
||||
}
|
||||
|
||||
+15
-6
@@ -9,7 +9,8 @@ import { newId } from '../ids';
|
||||
import { HIGHEST_NOTE, LOWEST_NOTE } from '../instrument/notes';
|
||||
import type { PatternAsset, SongAsset } from '../model/Asset';
|
||||
import { type PropertyBag, textOf } from '../model/Properties';
|
||||
import { findAssetOf, type Project } from '../model/Project';
|
||||
import { findAsset, findAssetOf, type Project } from '../model/Project';
|
||||
import { pickedFx } from '../properties/assetFx';
|
||||
import { gridTicks, GRID_OPTIONS, patternInstrument, patternLength } from './pattern';
|
||||
import { heldLoop, type LoopRange, type Sequence, type SequenceClip, type SequenceNote } from './sequence';
|
||||
import { heldMeter, heldTempo, type Meter, secondsOfTicks, ticksPerBar } from './time';
|
||||
@@ -47,8 +48,8 @@ export interface SongItem {
|
||||
/** The instrument a pattern is played with instead of its own, `''` for its own. */
|
||||
instrument: string;
|
||||
/**
|
||||
* The sound FX it goes through, `''` for none: a sound by itself, a pattern all its notes together
|
||||
* on a bus of their own (#346).
|
||||
* The sound FX it goes through: a sound by itself, a pattern all its notes together on a bus of
|
||||
* their own (#346). `''` is the one its asset names, `NO_FX` none (#352).
|
||||
*/
|
||||
fx: string;
|
||||
}
|
||||
@@ -145,9 +146,17 @@ export function dbToGain(db: number): number {
|
||||
return db <= TRACK_MIN_DB ? 0 : 10 ** (db / 20);
|
||||
}
|
||||
|
||||
/** Whether a pattern item is heard on a bus of its own (#346): it names a sound FX that is there. */
|
||||
/** The sound FX an item goes through (#352): its own pick, else the one of what it plays; '' for none. */
|
||||
export function itemFx(project: Project, item: SongItem): string {
|
||||
const played = findAsset(project, item.asset);
|
||||
return pickedFx(item.fx, played?.type === item.kind ? played.props : {});
|
||||
}
|
||||
|
||||
/** Whether a pattern item is heard on a bus of its own (#346): it goes through a sound FX that is there. */
|
||||
export function hasItemBus(project: Project, item: SongItem): boolean {
|
||||
return item.kind === 'pattern' && item.fx !== '' && findAssetOf(project, item.fx, 'soundfx') !== undefined;
|
||||
if (item.kind !== 'pattern') return false;
|
||||
const fx = itemFx(project, item);
|
||||
return fx !== '' && findAssetOf(project, fx, 'soundfx') !== undefined;
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -203,7 +212,7 @@ export function songSequence(project: Project, song: SongAsset): Sequence {
|
||||
tick: item.tick,
|
||||
length: item.length,
|
||||
sound: item.asset,
|
||||
fx: item.fx,
|
||||
fx: itemFx(project, item),
|
||||
offset: item.offset,
|
||||
volume,
|
||||
bus,
|
||||
|
||||
@@ -8,6 +8,7 @@ import {
|
||||
dbToGain,
|
||||
hasItemBus,
|
||||
itemBus,
|
||||
itemFx,
|
||||
type SongItem,
|
||||
songTempo,
|
||||
type SongTrack,
|
||||
@@ -68,7 +69,7 @@ export function songBuses(
|
||||
gain: 1,
|
||||
pan: 0,
|
||||
into: trackItemsBus(song.id, item.track),
|
||||
...fxOf(item.fx),
|
||||
...fxOf(itemFx(project, item)),
|
||||
}));
|
||||
return [...tracks, ...own, ...items];
|
||||
}
|
||||
@@ -139,6 +139,7 @@ import {
|
||||
SONG_TEMPO_KEY,
|
||||
} from '../music/song';
|
||||
import { DEFAULT_TEMPO, MAX_TEMPO, MIN_TEMPO } from '../music/time';
|
||||
import { ASSET_FX_KEY } from './assetFx';
|
||||
import { BUS_FX_KEY, BUS_INTO_KEY, BUS_KEY, BUS_VOLUME_KEY } from './buses';
|
||||
import { Theme } from '../../ui';
|
||||
|
||||
@@ -147,7 +148,8 @@ import { Theme } from '../../ui';
|
||||
* properties that are integral by meaning (counts, sizes in whole pixels), not on geometry: dragging
|
||||
* produces fractions, and the field rounds the value away rather than hiding it. Without
|
||||
* `assetTypes` an asset picker offers every placeable asset; `none` is what it calls an empty pick,
|
||||
* `master` offers the master bus as a choice of its own, and `intoFor` names the bus the row is
|
||||
* `master` offers the master bus as a choice of its own, `noFx` "None" as one where empty means
|
||||
* "from the asset" (#352), and `intoFor` names the bus the row is
|
||||
* the Into of, which leaves out that bus and every one going into it when the picker opens (#325);
|
||||
* `uploadedOnly` keeps drawn pictures out
|
||||
* of a texture picker (#186). `renderLayers` makes a layer row pick from the project's render layers
|
||||
@@ -158,7 +160,13 @@ type Extras<K extends FlatKind> = K extends 'number'
|
||||
: K extends 'enum'
|
||||
? { options: { value: string; label: string }[] }
|
||||
: K extends 'asset'
|
||||
? { assetTypes?: readonly AssetType[]; none?: string; master?: boolean; intoFor?: string }
|
||||
? {
|
||||
assetTypes?: readonly AssetType[];
|
||||
none?: string;
|
||||
master?: boolean;
|
||||
noFx?: boolean;
|
||||
intoFor?: string;
|
||||
}
|
||||
: K extends 'texture'
|
||||
? { uploadedOnly?: boolean }
|
||||
: K extends 'layer' | 'layers'
|
||||
@@ -503,6 +511,15 @@ const BUS_ROW: PropertyDef = {
|
||||
none: 'Master',
|
||||
};
|
||||
|
||||
/** The sound FX a sound, an instrument, a pattern or a song goes through, before its bus (#352). */
|
||||
const FX_ROW: PropertyDef = {
|
||||
key: ASSET_FX_KEY,
|
||||
label: 'Sound FX',
|
||||
kind: 'asset',
|
||||
default: '',
|
||||
assetTypes: ['soundfx'],
|
||||
};
|
||||
|
||||
/** Position is deliberately absent: it belongs to scene objects, not to asset types. */
|
||||
const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
|
||||
scene: [
|
||||
@@ -733,12 +750,13 @@ const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
|
||||
options: FADE_CURVES,
|
||||
},
|
||||
]),
|
||||
FX_ROW,
|
||||
BUS_ROW,
|
||||
],
|
||||
// The effects are the asset's graph (see SoundFxAsset), not a property.
|
||||
soundfx: [],
|
||||
// What a note sounds like is the asset's graph (see InstrumentAsset), not a property.
|
||||
instrument: [BUS_ROW],
|
||||
instrument: [FX_ROW, BUS_ROW],
|
||||
// The notes and the loop are intrinsic data on the asset (see PatternAsset). The meter and the
|
||||
// tempo are for drawing and hearing it on its own; a song plays it at its own tempo (#334).
|
||||
pattern: [
|
||||
@@ -780,6 +798,7 @@ const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
|
||||
max: MAX_TEMPO,
|
||||
step: 1,
|
||||
},
|
||||
FX_ROW,
|
||||
BUS_ROW,
|
||||
],
|
||||
// The tracks, items and loop are intrinsic data on the asset (see SongAsset).
|
||||
@@ -811,6 +830,7 @@ const ASSET_SCHEMAS: Record<AssetType, PropertySchema> = {
|
||||
default: DEFAULT_SONG_GRID,
|
||||
options: SONG_GRID_OPTIONS,
|
||||
},
|
||||
FX_ROW,
|
||||
BUS_ROW,
|
||||
],
|
||||
// Where sounds are mixed (#325): how loud, through which sound FX, and on into which bus.
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
import type { PropertyBag } from '../model/Properties';
|
||||
import { coerceAs } from './kinds';
|
||||
|
||||
/** The sound FX a sound, an instrument, a pattern or a song goes through (#352); empty is none. */
|
||||
export const ASSET_FX_KEY = 'fx';
|
||||
|
||||
/**
|
||||
* What a pick says for "no sound FX" where empty means "the one of the asset": on a Play node and on
|
||||
* an item of a song. No asset id has a `#`.
|
||||
*/
|
||||
export const NO_FX = '#none';
|
||||
|
||||
export function assetFxOf(bag: PropertyBag): string {
|
||||
return coerceAs(bag[ASSET_FX_KEY], 'asset', '');
|
||||
}
|
||||
|
||||
/** The sound FX something plays through: the one picked, else the one its asset names; '' for none. */
|
||||
export function pickedFx(picked: string, assetProps: PropertyBag): string {
|
||||
if (picked === NO_FX) return '';
|
||||
return picked !== '' ? picked : assetFxOf(assetProps);
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
import type { Asset } from '../model/Asset';
|
||||
import { type AssetType, isPlaceableType } from '../model/AssetTypes';
|
||||
import { findAsset, type Project } from '../model/Project';
|
||||
import { NO_FX } from './assetFx';
|
||||
import { MASTER_BUS } from './buses';
|
||||
|
||||
export const NO_ASSET_LABEL = 'None';
|
||||
@@ -11,7 +12,7 @@ export const MASTER_LABEL = 'Master';
|
||||
* reference shows its id rather than passing for an empty field.
|
||||
*/
|
||||
export function assetRefLabel(project: Project | null, id: string): string {
|
||||
if (id === '') return NO_ASSET_LABEL;
|
||||
if (id === '' || id === NO_FX) return NO_ASSET_LABEL;
|
||||
if (id === MASTER_BUS) return MASTER_LABEL;
|
||||
return (project ? findAsset(project, id) : undefined)?.name ?? `${id}?`;
|
||||
}
|
||||
|
||||
@@ -17,6 +17,7 @@ import {
|
||||
MASTER_LABEL,
|
||||
NO_ASSET_LABEL,
|
||||
} from '../../core/properties/assetRef';
|
||||
import { NO_FX } from '../../core/properties/assetFx';
|
||||
import { busesInto, MASTER_BUS } from '../../core/properties/buses';
|
||||
import { keyName } from '../../core/keys';
|
||||
import { FIRST_TILE_LAYER } from '../../core/model/liveTilemap';
|
||||
@@ -398,6 +399,7 @@ const REFERENCES: Record<ReferenceKind, Reference> = {
|
||||
const left = new Set(extras?.intoFor ? busesInto(project, extras.intoFor) : []);
|
||||
return [
|
||||
...(extras?.master ? [{ id: MASTER_BUS, label: MASTER_LABEL }] : []),
|
||||
...(extras?.noFx ? [{ id: NO_FX, label: NO_ASSET_LABEL }] : []),
|
||||
...assetRefChoices(project, extras?.assetTypes)
|
||||
.filter((asset) => !left.has(asset.id))
|
||||
.map((asset) => ({ id: asset.id, label: assetChoiceLabel(asset) })),
|
||||
|
||||
@@ -16,13 +16,16 @@ import { createFieldRow } from './fieldFactory';
|
||||
import { addNameField, addNote, addTitle, type InspectorHost } from './host';
|
||||
import { previewingHandlers } from './previewHandlers';
|
||||
|
||||
const FX_ROW: PropertyDef = {
|
||||
const FX_ROW = {
|
||||
key: 'fx',
|
||||
label: 'Sound FX',
|
||||
kind: 'asset',
|
||||
default: '',
|
||||
assetTypes: ['soundfx'],
|
||||
};
|
||||
} as const satisfies PropertyDef;
|
||||
|
||||
/** What an item's pick says while it is empty (#352): what its asset names. */
|
||||
const FROM_ASSET = 'From the asset';
|
||||
|
||||
const TRACK_ROWS: readonly PropertyDef[] = [
|
||||
{
|
||||
@@ -72,9 +75,16 @@ function itemRows(item: SongItem): PropertyDef[] {
|
||||
step: 1,
|
||||
decimals: 0,
|
||||
},
|
||||
{ key: 'instrument', label: 'Instrument', kind: 'asset', default: '', assetTypes: ['instrument'] },
|
||||
{
|
||||
key: 'instrument',
|
||||
label: 'Instrument',
|
||||
kind: 'asset',
|
||||
default: '',
|
||||
assetTypes: ['instrument'],
|
||||
none: FROM_ASSET,
|
||||
},
|
||||
);
|
||||
rows.push(FX_ROW);
|
||||
rows.push({ ...FX_ROW, none: FROM_ASSET, noFx: true });
|
||||
return rows;
|
||||
}
|
||||
|
||||
@@ -185,12 +195,13 @@ function buildItem(host: InspectorHost, asset: SongAsset, item: SongItem, picked
|
||||
host.fields.set(def.key, row);
|
||||
host.body.add(row.row);
|
||||
}
|
||||
if (item.kind === 'pattern')
|
||||
addNote(
|
||||
host,
|
||||
'An empty Instrument plays the pattern with its own. A Sound FX takes all its notes together, before the one of the track.',
|
||||
'small',
|
||||
);
|
||||
addNote(
|
||||
host,
|
||||
item.kind === 'pattern'
|
||||
? 'A Sound FX takes all its notes together, instead of the one of the pattern and before the one of the track.'
|
||||
: 'A Sound FX takes the place of the one of the sound.',
|
||||
'small',
|
||||
);
|
||||
}
|
||||
|
||||
function itemOf(asset: SongAsset | undefined, itemIds: readonly string[]): SongItem | undefined {
|
||||
|
||||
@@ -20,7 +20,9 @@ import {
|
||||
} from '../../core/instrument/compile';
|
||||
import { MIDDLE_C, noteName } from '../../core/instrument/notes';
|
||||
import type { Asset, InstrumentAsset } from '../../core/model/Asset';
|
||||
import { instrumentBus, playNote } from '../../runtime/systems/behaviorContext';
|
||||
import { fxVoiceOf } from '../../core/music/needs';
|
||||
import { DEFAULT_TEMPO } from '../../core/music/time';
|
||||
import { instrumentBus, instrumentFx, playNote } from '../../runtime/systems/behaviorContext';
|
||||
import {
|
||||
Button,
|
||||
Dropdown,
|
||||
@@ -165,6 +167,7 @@ export class InstrumentEditor extends AssetTabEditor<InstrumentAsset, GraphCanva
|
||||
PREVIEW_OWNER,
|
||||
{ ...this.values },
|
||||
{ bus: instrumentBus(this.state.project, asset.id, '') },
|
||||
fxVoiceOf(this.state.project, instrumentFx(this.state.project, asset.id, ''), DEFAULT_TEMPO),
|
||||
);
|
||||
if (voice) this.voices.push(voice);
|
||||
return voice;
|
||||
|
||||
@@ -18,7 +18,10 @@ import {
|
||||
import type { Asset, PatternAsset } from '../../core/model/Asset';
|
||||
import { textOf } from '../../core/model/Properties';
|
||||
import { findAssetOf } from '../../core/model/Project';
|
||||
import { playNote } from '../../runtime/systems/behaviorContext';
|
||||
import { fxVoiceOf } from '../../core/music/needs';
|
||||
import { assetFxOf } from '../../core/properties/assetFx';
|
||||
import { DEFAULT_TEMPO } from '../../core/music/time';
|
||||
import { instrumentFx, playNote } from '../../runtime/systems/behaviorContext';
|
||||
import { Button, Dropdown, Label, Menu, type Widget } from '../../ui';
|
||||
import { Music } from '../../viewport/music';
|
||||
import { playedBus } from '../../core/music/mixer';
|
||||
@@ -51,10 +54,14 @@ export class PatternEditor extends AssetTabEditor<PatternAsset, PianoRoll> {
|
||||
readonly ghostsButton: Button;
|
||||
/** Where the ghost notes come from: song, track and bar. */
|
||||
readonly ghostSource = new Label('', { style: 'muted' });
|
||||
/** Heard on the bus of the asset and the master, as in the game (#325). */
|
||||
/** Heard through the sound FX (#352) and on the bus of the asset and the master, as in the game (#325). */
|
||||
readonly music = new Music(
|
||||
() => this.state.project,
|
||||
() => playedBus(this.state.project, '', this.asset()?.props ?? {}),
|
||||
() => {
|
||||
const asset = this.asset();
|
||||
return asset && fxVoiceOf(this.state.project, assetFxOf(asset.props), patternTempo(asset.props));
|
||||
},
|
||||
);
|
||||
/** The pattern playing, `''` while none is. */
|
||||
private playingId = '';
|
||||
@@ -90,7 +97,10 @@ export class PatternEditor extends AssetTabEditor<PatternAsset, PianoRoll> {
|
||||
this.ghostsButton.setToggled(true);
|
||||
const roll = new PianoRoll(state);
|
||||
roll.onPreview = (note, held) => this.preview(note, held);
|
||||
roll.onRelease = (voice) => soundLibrary.release(voice);
|
||||
roll.onRelease = (voice) => {
|
||||
soundLibrary.release(voice);
|
||||
this.music.idle();
|
||||
};
|
||||
roll.onTogglePlay = () => this.togglePlay();
|
||||
roll.onStart = (tick) => {
|
||||
if (this.isPlaying) this.music.current?.seek(tick);
|
||||
@@ -167,13 +177,15 @@ export class PatternEditor extends AssetTabEditor<PatternAsset, PianoRoll> {
|
||||
const instrument = asset ? patternInstrument(asset.props) : '';
|
||||
if (!instrument) return 0;
|
||||
const struck = { note, velocity: 1, volume: 1, duration: held ? 0 : PREVIEW_SECONDS };
|
||||
if (!held) setTimeout(() => this.music.idle(), PREVIEW_SECONDS * 1000);
|
||||
return playNote(
|
||||
{ project: this.state.project, audio: soundLibrary },
|
||||
instrument,
|
||||
struck,
|
||||
PREVIEW_OWNER,
|
||||
{},
|
||||
{ bus: playedBus(this.state.project, '', asset?.props ?? {}) },
|
||||
{ bus: this.music.heardOn() },
|
||||
fxVoiceOf(this.state.project, instrumentFx(this.state.project, instrument, ''), DEFAULT_TEMPO),
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
@@ -12,12 +12,14 @@ import {
|
||||
} from '../../core/music/song';
|
||||
import { roundAt } from '../../core/music/ghosts';
|
||||
import { patternLength } from '../../core/music/pattern';
|
||||
import { fxVoiceOf } from '../../core/music/needs';
|
||||
import { songBuses } from '../../core/music/songBuses';
|
||||
import { sampleCount, takeName, takeStart, takeTrack, takeTrackIndex, wavBytes } from '../../core/music/take';
|
||||
import { ticksOfSeconds } from '../../core/music/time';
|
||||
import type { Asset, SongAsset } from '../../core/model/Asset';
|
||||
import { findAssetOf } from '../../core/model/Project';
|
||||
import { textOf } from '../../core/model/Properties';
|
||||
import { assetFxOf } from '../../core/properties/assetFx';
|
||||
import type { SoundStore } from '../../core/store/SoundStore';
|
||||
import { Button, Dialog, Dropdown, Label, Menu, ui, VStack, type Widget } from '../../ui';
|
||||
import { Music } from '../../viewport/music';
|
||||
@@ -82,10 +84,14 @@ export class SongEditor extends AssetTabEditor<SongAsset, SongTimeline> {
|
||||
readonly routing: GraphCanvas;
|
||||
private shown: SongView = 'timeline';
|
||||
private readonly pane = new VStack();
|
||||
/** Heard on the bus of the asset and the master, as in the game (#325). */
|
||||
/** Heard through the sound FX (#352) and on the bus of the asset and the master, as in the game (#325). */
|
||||
readonly music = new Music(
|
||||
() => this.state.project,
|
||||
() => playedBus(this.state.project, '', this.asset()?.props ?? {}),
|
||||
() => {
|
||||
const asset = this.asset();
|
||||
return asset && fxVoiceOf(this.state.project, assetFxOf(asset.props), songTempo(asset.props));
|
||||
},
|
||||
);
|
||||
/** The song playing, `''` while none is. */
|
||||
private playingId = '';
|
||||
@@ -137,9 +143,9 @@ export class SongEditor extends AssetTabEditor<SongAsset, SongTimeline> {
|
||||
const asset = this.asset();
|
||||
if (asset && this.isPlaying) this.music.setBuses(songBuses(this.state.project, { ...asset, tracks }));
|
||||
};
|
||||
timeline.onOpenPattern = (item, tick) => {
|
||||
timeline.onOpenItem = (item, tick) => {
|
||||
const song = this.asset();
|
||||
const pattern = findAssetOf(state.project, item.asset, 'pattern');
|
||||
const pattern = item.kind === 'pattern' ? findAssetOf(state.project, item.asset, 'pattern') : undefined;
|
||||
if (song && pattern)
|
||||
state.patternPlaces.set(pattern.id, {
|
||||
songId: song.id,
|
||||
|
||||
@@ -233,8 +233,8 @@ export class SongTimeline extends Widget {
|
||||
onStart: (tick: number) => void = () => {};
|
||||
/** The tracks as a slider in a head would leave them: heard at once, written when it lets go. */
|
||||
onTracksPreview: (tracks: readonly SongTrack[]) => void = () => {};
|
||||
/** A pattern item was double-clicked, at a tick of the song. */
|
||||
onOpenPattern: (item: SongItem, tick: number) => void = () => {};
|
||||
/** An item was double-clicked, at a tick of the song. */
|
||||
onOpenItem: (item: SongItem, tick: number) => void = () => {};
|
||||
onDiscardTake: () => void = () => {};
|
||||
|
||||
constructor(private readonly state: EditorState) {
|
||||
@@ -722,8 +722,8 @@ export class SongTimeline extends Widget {
|
||||
this.gesture = { kind: 'empty', start: g, current: g, active: false, add: e.shiftKey };
|
||||
return;
|
||||
}
|
||||
if (pressCount(e, `song-item:${anchor.id}`) === 2 && anchor.kind === 'pattern') {
|
||||
this.onOpenPattern(anchor, xToTick(this.view, g.x));
|
||||
if (pressCount(e, `song-item:${anchor.id}`) === 2) {
|
||||
this.onOpenItem(anchor, xToTick(this.view, g.x));
|
||||
return;
|
||||
}
|
||||
const picked = this.pickedIds;
|
||||
|
||||
@@ -17,6 +17,9 @@ import { Button, Dropdown, Label, Menu, NumberField } from '../../ui';
|
||||
import { soundLibrary } from '../../viewport/SoundLibrary';
|
||||
import { playedBus } from '../../core/music/mixer';
|
||||
import { DRY_VOICE } from '../../core/soundfx/nodes';
|
||||
import { fxVoiceOf } from '../../core/music/needs';
|
||||
import { DEFAULT_TEMPO } from '../../core/music/time';
|
||||
import { assetFxOf } from '../../core/properties/assetFx';
|
||||
import { AssetTabEditor } from '../AssetTabEditor';
|
||||
import { claimTransport, releaseTransport } from '../music/transport';
|
||||
import { HEADER_ICON_SVG } from '../panels/headerIcons';
|
||||
@@ -151,10 +154,11 @@ export class SoundEditor extends AssetTabEditor<SoundAsset, WaveCanvas> {
|
||||
this.stop();
|
||||
claimTransport(this.halt);
|
||||
const props = resolveAssetProps(this.state.project, asset);
|
||||
// Through its bus and the master, as the game hears it (#325).
|
||||
// Through its sound FX (#352), its bus and the master, as the game hears it (#325).
|
||||
const bus = playedBus(this.state.project, '', props);
|
||||
const fx = fxVoiceOf(this.state.project, assetFxOf(props), DEFAULT_TEMPO) ?? DRY_VOICE;
|
||||
const file = playedSoundOf(this.state.project, asset);
|
||||
this.voice = soundLibrary.play(file, volumeOf(props), false, PREVIEW_OWNER, DRY_VOICE, bus);
|
||||
this.voice = soundLibrary.play(file, volumeOf(props), false, PREVIEW_OWNER, fx, bus);
|
||||
this.playingId = this.voice ? asset.id : '';
|
||||
this.syncPlayButton();
|
||||
}
|
||||
|
||||
@@ -11,11 +11,11 @@ import { JOINT_CONNECTED_KEY } from '../../core/model/joint';
|
||||
import { objectReference } from '../../core/model/objectRefs';
|
||||
import { findAsset, findAssetOf, findScene, type Project } from '../../core/model/Project';
|
||||
import { renderLayerAlong } from '../../core/model/renderLayers';
|
||||
import { pickedFx } from '../../core/properties/assetFx';
|
||||
import { playedSoundById, playedSoundOf } from '../../core/properties/playedSound';
|
||||
import { hasRenderLayer } from '../../core/properties/PropertySchema';
|
||||
import { resolveAssetProps } from '../../core/properties/resolve';
|
||||
import { loopsOf, volumeOf } from '../../core/properties/sounds';
|
||||
import { soundFxBuildFor } from '../../core/soundfx/compile';
|
||||
import { instrumentBuildFor, instrumentSourcesOf, throughScopedFx } from '../../core/instrument/compile';
|
||||
import { VELOCITY_PARAM } from '../../core/instrument/nodes';
|
||||
import { heldNote } from '../../core/instrument/notes';
|
||||
@@ -190,7 +190,7 @@ export function behaviorContext(
|
||||
detached ? '' : entity.id,
|
||||
params,
|
||||
{ tempo, bus: instrumentBus(project, instrumentId, bus) },
|
||||
fxVoiceOf(project, fx, tempo, fxParams),
|
||||
fxVoiceOf(project, instrumentFx(project, instrumentId, fx), tempo, fxParams),
|
||||
),
|
||||
playPattern: (patternId, volume, loop, detached, fx, fxParams, bus) =>
|
||||
playPattern(run, patternId, volume, loop, detached ? '' : entity.id, fx, fxParams, bus),
|
||||
@@ -287,18 +287,22 @@ function playSound(
|
||||
if (!asset?.file) return 0;
|
||||
const props = resolveAssetProps(project, asset);
|
||||
const level = Math.max(0, Math.min(1, volumeOf(props) * volume));
|
||||
const fx = fxId ? findAssetOf(project, fxId, 'soundfx') : undefined;
|
||||
const plan = fx && soundFxBuildFor(fx, project).compiled;
|
||||
const fx = fxVoiceOf(project, pickedFx(fxId, props), tempo, fxParams);
|
||||
return audio.play(
|
||||
playedSoundOf(project, asset),
|
||||
level,
|
||||
loop ?? loopsOf(props),
|
||||
owner,
|
||||
plan ? { plan, params: fxParams, tempo } : DRY_VOICE,
|
||||
fx ?? DRY_VOICE,
|
||||
playedBus(project, bus, props),
|
||||
);
|
||||
}
|
||||
|
||||
/** The sound FX a note of that instrument goes through, by the pick of a node (#352); '' for none. */
|
||||
export function instrumentFx(project: Project, instrumentId: string, picked: string): string {
|
||||
return pickedFx(picked, findAssetOf(project, instrumentId, 'instrument')?.props ?? {});
|
||||
}
|
||||
|
||||
/** The bus of the mix a note of that instrument is heard on, by the pick of a node (#325). */
|
||||
export function instrumentBus(project: Project, instrumentId: string, picked: string): string {
|
||||
return playedBus(project, picked, findAssetOf(project, instrumentId, 'instrument')?.props ?? {});
|
||||
@@ -375,7 +379,7 @@ function playPattern(
|
||||
const level = heldLevel(volume);
|
||||
const length = patternLength(asset);
|
||||
const tempo = patternTempo(asset.props);
|
||||
const fx = fxVoiceOf(project, fxId, tempo, fxParams);
|
||||
const fx = fxVoiceOf(project, pickedFx(fxId, asset.props), tempo, fxParams);
|
||||
return audio.playSequence(
|
||||
patternVoiceName(patternId),
|
||||
{
|
||||
@@ -413,7 +417,7 @@ function playSong(
|
||||
if (!song) return 0;
|
||||
const sequence = songSequence(project, song);
|
||||
const tempo = songTempo(song.props);
|
||||
const fx = fxVoiceOf(project, fxId, tempo, fxParams);
|
||||
const fx = fxVoiceOf(project, pickedFx(fxId, song.props), tempo, fxParams);
|
||||
return audio.playSequence(
|
||||
songVoiceName(songId),
|
||||
{
|
||||
@@ -434,7 +438,10 @@ function playSong(
|
||||
);
|
||||
}
|
||||
|
||||
/** A note the scheduler planned (#332), struck through its instrument at its volume, on its bus. */
|
||||
/**
|
||||
* A note the scheduler planned (#332), struck through its instrument and the sound FX that names
|
||||
* (#352) at its volume, on its bus.
|
||||
*/
|
||||
export function strikeNote(
|
||||
sources: Pick<ContextSources, 'project' | 'audio'>,
|
||||
note: ScheduledNote,
|
||||
@@ -447,7 +454,8 @@ export function strikeNote(
|
||||
duration: note.duration,
|
||||
};
|
||||
const timing = { at: note.at, tempo: note.tempo, ...(note.bus && { bus: note.bus }) };
|
||||
return playNote(sources, note.instrument, struck, owner, {}, timing);
|
||||
const fx = fxVoiceOf(sources.project, instrumentFx(sources.project, note.instrument, ''), note.tempo);
|
||||
return playNote(sources, note.instrument, struck, owner, {}, timing, fx);
|
||||
}
|
||||
|
||||
/** A clip of a song (#334): its sound at its own volume times the item's, through its sound FX. */
|
||||
|
||||
+38
-7
@@ -9,10 +9,13 @@ import { MASTER_MIX } from '../core/music/mixer';
|
||||
import type { BusLevel } from '../core/music/playbacks';
|
||||
import type { Sequence } from '../core/music/sequence';
|
||||
import type { Project } from '../core/model/Project';
|
||||
import type { FxVoice } from '../core/soundfx/nodes';
|
||||
import { playClip, strikeNote } from '../runtime/systems/behaviorContext';
|
||||
import { soundLibrary } from './SoundLibrary';
|
||||
import { wake } from './wake';
|
||||
|
||||
let musicCount = 0;
|
||||
|
||||
/** One planned note as the player was given it, and where the audio clock stood then (test hook). */
|
||||
export interface PlannedNote extends ScheduledNote {
|
||||
voice: number;
|
||||
@@ -38,17 +41,24 @@ export class Music {
|
||||
/** The tracks last set, and when the chains of their sound FX are there (#346). */
|
||||
private buses: readonly BusLevel[] = [];
|
||||
private busesReady: Promise<void> = Promise.resolve();
|
||||
/** The bus the sound FX of the pattern or song is on (#352), into `output`; one per tab. */
|
||||
private readonly fxBus = `music:${++musicCount}`;
|
||||
/** Where what goes to the output lands: `fxBus` while there is a sound FX, else `output`. */
|
||||
private landing = '';
|
||||
private fxReady: Promise<void> = Promise.resolve();
|
||||
/** The last notes and clips planned, newest last, for a check to hold against the clock. */
|
||||
readonly planned: PlannedNote[] = [];
|
||||
readonly clips: PlannedClip[] = [];
|
||||
|
||||
/**
|
||||
* `output` is the bus of the mix what it plays is heard on (#325): that of the pattern or song in
|
||||
* the tab, so it sounds as it would in the game. The master where left out.
|
||||
* the tab, so it sounds as it would in the game. The master where left out. `fx` is the sound FX of
|
||||
* that pattern or song (#352), on the way there.
|
||||
*/
|
||||
constructor(
|
||||
private readonly project: () => Project,
|
||||
private readonly output: () => string = () => MASTER_MIX,
|
||||
private readonly fx: () => FxVoice | null = () => null,
|
||||
) {}
|
||||
|
||||
/** Loads what the notes and clips need, then plays from `from` ticks. */
|
||||
@@ -58,7 +68,8 @@ export class Music {
|
||||
this.sequence = sequence;
|
||||
this.planned.length = 0;
|
||||
this.clips.length = 0;
|
||||
await Promise.all([this.prepare(sequence), this.busesReady]);
|
||||
this.landing = this.heardOn();
|
||||
await Promise.all([this.prepare(sequence), this.busesReady, this.fxReady]);
|
||||
if (run !== this.run) return;
|
||||
this.scheduler = new Scheduler(
|
||||
{
|
||||
@@ -66,7 +77,7 @@ export class Music {
|
||||
latency: () => soundLibrary.latency(),
|
||||
ahead: () =>
|
||||
Math.max(
|
||||
soundLibrary.busLatency(this.output()),
|
||||
soundLibrary.busLatency(this.landing),
|
||||
...this.buses.map((bus) => soundLibrary.busLatency(bus.name)),
|
||||
),
|
||||
play: (note) => this.playOne(note),
|
||||
@@ -86,15 +97,35 @@ export class Music {
|
||||
/** A change heard from the next window on; a note new to it loads what it needs meanwhile. */
|
||||
edit(sequence: Sequence): void {
|
||||
this.sequence = sequence;
|
||||
this.landing = this.heardOn();
|
||||
void this.prepare(sequence);
|
||||
}
|
||||
|
||||
/**
|
||||
* Where what goes to the output of the pattern or song is heard: through its sound FX on a bus of
|
||||
* its own, set as it is now, else straight on `output`. A key of the roll is played there too.
|
||||
*/
|
||||
heardOn(): string {
|
||||
const fx = this.fx();
|
||||
if (!fx) {
|
||||
soundLibrary.releaseBuses([this.fxBus]);
|
||||
return this.output();
|
||||
}
|
||||
this.fxReady = soundLibrary.setBuses([{ name: this.fxBus, gain: 1, pan: 0, into: this.output(), fx }]);
|
||||
return this.fxBus;
|
||||
}
|
||||
|
||||
/** Lets the bus of its sound FX go while nothing plays: after a key of the roll. */
|
||||
idle(): void {
|
||||
if (!this.scheduler) soundLibrary.releaseBuses([this.fxBus]);
|
||||
}
|
||||
|
||||
/**
|
||||
* The tracks of a song set to their level and pan, heard at once (#334), and to their sound FX
|
||||
* (#346). What goes to the output of the song goes to `output` (#350). A start waits for the chains.
|
||||
*/
|
||||
setBuses(levels: readonly BusLevel[]): void {
|
||||
const output = this.output();
|
||||
const output = (this.landing = this.heardOn());
|
||||
this.buses = levels.map(({ sends, ...level }) =>
|
||||
sends
|
||||
? { ...level, sends: sends.map((send) => ({ ...send, to: send.to ?? output })) }
|
||||
@@ -106,7 +137,7 @@ export class Music {
|
||||
/** Its buses go once what is on them has faded, unless the next start uses them again. */
|
||||
stop(): void {
|
||||
this.halt();
|
||||
soundLibrary.releaseBuses(this.buses.map((bus) => bus.name));
|
||||
soundLibrary.releaseBuses([...this.buses.map((bus) => bus.name), this.fxBus]);
|
||||
}
|
||||
|
||||
private halt(): void {
|
||||
@@ -120,7 +151,7 @@ export class Music {
|
||||
}
|
||||
|
||||
private playOne(note: ScheduledNote): number {
|
||||
const heard = { ...note, bus: note.bus || this.output() };
|
||||
const heard = { ...note, bus: note.bus || this.landing };
|
||||
const voice = strikeNote({ project: this.project(), audio: soundLibrary }, heard, '');
|
||||
this.planned.push({ ...note, voice, planned: soundLibrary.clock() });
|
||||
if (this.planned.length > 256) this.planned.shift();
|
||||
@@ -128,7 +159,7 @@ export class Music {
|
||||
}
|
||||
|
||||
private playClip(clip: ScheduledClip): number {
|
||||
const heard = { ...clip, bus: clip.bus || this.output() };
|
||||
const heard = { ...clip, bus: clip.bus || this.landing };
|
||||
const voice = playClip({ project: this.project(), audio: soundLibrary }, heard);
|
||||
this.clips.push({ ...clip, voice, planned: soundLibrary.clock() });
|
||||
if (this.clips.length > 256) this.clips.shift();
|
||||
|
||||
@@ -0,0 +1,231 @@
|
||||
/**
|
||||
* A sound FX on a sound, an instrument, a pattern and a song (#352): what plays it goes through it
|
||||
* unless a Play node or an item of a song picks another, or None.
|
||||
*/
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
CreateAssetCommand,
|
||||
CreateSoundCommand,
|
||||
SetAssetPropertyCommand,
|
||||
} from '../../src/core/commands/assetCommands';
|
||||
import { SetPatternNotesCommand } from '../../src/core/commands/patternCommands';
|
||||
import { SetSongCommand } from '../../src/core/commands/songCommands';
|
||||
import type { BehaviorAsset, InstrumentAsset, SongAsset, SoundFxAsset } from '../../src/core/model/Asset';
|
||||
import type { AssetType } from '../../src/core/model/AssetTypes';
|
||||
import type { PropertyBag } from '../../src/core/model/Properties';
|
||||
import { sequenceNeeds } from '../../src/core/music/needs';
|
||||
import type { SequencePlay } from '../../src/core/music/playbacks';
|
||||
import { itemBus, newItem, songSequence, trackItemsBus } from '../../src/core/music/song';
|
||||
import { songBuses } from '../../src/core/music/songBuses';
|
||||
import { ASSET_FX_KEY, NO_FX, pickedFx } from '../../src/core/properties/assetFx';
|
||||
import { schemaFor } from '../../src/core/properties/PropertySchema';
|
||||
import { soundFxBuildFor } from '../../src/core/soundfx/compile';
|
||||
import { DRY_VOICE, type FxParams, type FxPlan, type FxVoice } from '../../src/core/soundfx/nodes';
|
||||
import { type AudioSink, SILENT_AUDIO } from '../../src/runtime/systems/Audio';
|
||||
import { graphEditor, makeBehavior, makeSpriteScene, place, startWorld } from '../helpers';
|
||||
|
||||
const FILE = `${'c4'.repeat(32)}.wav`;
|
||||
const Q = 480;
|
||||
const BAR = 4 * Q;
|
||||
|
||||
/** A sound FX of one Gain of `db`. */
|
||||
function gainFx(f: ReturnType<typeof makeSpriteScene>, name: string, db: number): SoundFxAsset {
|
||||
const made = new CreateAssetCommand('soundfx', name);
|
||||
f.stack.execute(made);
|
||||
const fx = made.asset as SoundFxAsset;
|
||||
const edit = graphEditor(f, fx as unknown as BehaviorAsset);
|
||||
const input = edit.addNode('SoundFxInput');
|
||||
const gain = edit.addNode('SoundFxGain', { gain: db });
|
||||
const output = edit.addNode('SoundFxOutput');
|
||||
edit.wire([
|
||||
[input, 'out', gain, 'in'],
|
||||
[gain, 'out', output, 'in'],
|
||||
]);
|
||||
return fx;
|
||||
}
|
||||
|
||||
/**
|
||||
* A sound, an instrument, a pattern and a song, each through a sound FX of its own: Sound -1 dB,
|
||||
* Lead -2, Riff -3, Theme -4, and Other -5 for a pick. The song has a sound item and a pattern item.
|
||||
*/
|
||||
function scene() {
|
||||
const f = makeSpriteScene('Box');
|
||||
const [soundFx, leadFx, riffFx, themeFx, other] = [-1, -2, -3, -4, -5].map((db, i) =>
|
||||
gainFx(f, ['Sound', 'Lead', 'Riff', 'Theme', 'Other'][i]!, db),
|
||||
) as [SoundFxAsset, SoundFxAsset, SoundFxAsset, SoundFxAsset, SoundFxAsset];
|
||||
const sound = new CreateSoundCommand('Blip', FILE);
|
||||
f.stack.execute(sound);
|
||||
const ins = new CreateAssetCommand('instrument', 'Lead');
|
||||
f.stack.execute(ins);
|
||||
const lead = ins.asset as InstrumentAsset;
|
||||
lead.graph.nodes.find((n) => n.type === 'InstrumentSample')!.params.sound = sound.asset.id;
|
||||
const pattern = new CreateAssetCommand('pattern', 'Riff');
|
||||
f.stack.execute(pattern);
|
||||
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'instrument', lead.id));
|
||||
f.stack.execute(new SetAssetPropertyCommand(pattern.asset.id, 'bars', 1));
|
||||
f.stack.execute(
|
||||
new SetPatternNotesCommand(pattern.asset.id, [{ id: 'n', tick: 0, length: Q, note: 60, velocity: 1 }]),
|
||||
);
|
||||
const created = new CreateAssetCommand('song', 'Theme');
|
||||
f.stack.execute(created);
|
||||
const songId = created.asset.id;
|
||||
const song = () => f.project.assets.find((a) => a.id === songId) as SongAsset;
|
||||
const track = song().tracks[0]!.id;
|
||||
const clip = newItem({ track, kind: 'sound', asset: sound.asset.id, tick: 0, length: BAR });
|
||||
const riff = newItem({ track, kind: 'pattern', asset: pattern.asset.id, tick: 0, length: BAR });
|
||||
f.stack.execute(new SetSongCommand(songId, { items: [clip, riff] }));
|
||||
for (const [id, fx] of [
|
||||
[sound.asset.id, soundFx],
|
||||
[lead.id, leadFx],
|
||||
[pattern.asset.id, riffFx],
|
||||
[songId, themeFx],
|
||||
] as const)
|
||||
f.stack.execute(new SetAssetPropertyCommand(id, ASSET_FX_KEY, fx.id));
|
||||
const setItems = (clipFx: string, riffFx: string) =>
|
||||
f.stack.execute(
|
||||
new SetSongCommand(songId, {
|
||||
items: song().items.map((i) => ({ ...i, fx: i.kind === 'sound' ? clipFx : riffFx })),
|
||||
}),
|
||||
);
|
||||
const behavior = makeBehavior(f, 'Logic');
|
||||
place(f, behavior.id, { x: 0, y: 0 }, place(f, f.spriteId, { x: 0, y: 0 }));
|
||||
const edit = graphEditor(f, behavior);
|
||||
const plan = (fx: SoundFxAsset) => soundFxBuildFor(fx).compiled!;
|
||||
return {
|
||||
...f,
|
||||
fx: { soundFx, leadFx, riffFx, themeFx, other },
|
||||
plan,
|
||||
sound: sound.asset.id,
|
||||
lead,
|
||||
pattern: pattern.asset.id,
|
||||
song,
|
||||
songId,
|
||||
track,
|
||||
riff: riff.id,
|
||||
setItems,
|
||||
edit,
|
||||
};
|
||||
}
|
||||
|
||||
function recording() {
|
||||
const sounds: FxVoice[] = [];
|
||||
const notes: { layers: readonly FxPlan[]; params: FxParams }[] = [];
|
||||
const sequences: { name: string; play: SequencePlay }[] = [];
|
||||
const sink: AudioSink = {
|
||||
...SILENT_AUDIO,
|
||||
play: (_file, _volume, _loop, _owner, fx) => (sounds.push(fx ?? DRY_VOICE), 3),
|
||||
playNote: (_name, layers, params) => (notes.push({ layers, params }), 7),
|
||||
playSequence: (name, play) => (sequences.push({ name, play }), 500),
|
||||
};
|
||||
return Object.assign(sink, { sounds, notes, sequences });
|
||||
}
|
||||
|
||||
/** The gains of a layer's stages, as its plan has them. */
|
||||
function gains(layer: FxPlan | undefined): number[] {
|
||||
return (layer?.stages ?? []).flatMap((stage) =>
|
||||
stage.kind === 'gain' && 'db' in stage ? [stage.db.value] : [],
|
||||
);
|
||||
}
|
||||
|
||||
describe('the Sound FX row (#352)', () => {
|
||||
it('stands right before Bus on a sound, an instrument, a pattern and a song', () => {
|
||||
for (const type of ['sound', 'instrument', 'pattern', 'song'] as AssetType[]) {
|
||||
const keys = schemaFor(type).map((def) => def.key);
|
||||
expect(keys.indexOf(ASSET_FX_KEY), type).toBe(keys.indexOf('bus') - 1);
|
||||
expect(schemaFor(type).find((def) => def.key === ASSET_FX_KEY)).toMatchObject({
|
||||
kind: 'asset',
|
||||
default: '',
|
||||
assetTypes: ['soundfx'],
|
||||
});
|
||||
}
|
||||
});
|
||||
|
||||
it('is the one a pick leaves empty, none for None, and the pick itself for any other', () => {
|
||||
const props = { [ASSET_FX_KEY]: 'own' };
|
||||
expect(pickedFx('', props)).toBe('own');
|
||||
expect(pickedFx(NO_FX, props)).toBe('');
|
||||
expect(pickedFx('picked', props)).toBe('picked');
|
||||
expect(pickedFx('', {})).toBe('');
|
||||
});
|
||||
});
|
||||
|
||||
describe('the sound FX of what a song plays (#352)', () => {
|
||||
it('takes the one of the sound and of the pattern for an item that picks none', () => {
|
||||
const f = scene();
|
||||
const { clips, notes } = songSequence(f.project, f.song());
|
||||
expect(clips?.map((c) => c.fx)).toEqual([f.fx.soundFx.id]);
|
||||
expect(notes.map((n) => n.bus)).toEqual([itemBus(f.songId, { id: f.riff, track: f.track })]);
|
||||
const bus = songBuses(f.project, f.song()).find(
|
||||
(b) => b.name === itemBus(f.songId, { id: f.riff, track: f.track }),
|
||||
);
|
||||
expect(bus?.fx?.plan).toBe(f.plan(f.fx.riffFx));
|
||||
});
|
||||
|
||||
it('takes the one an item picks instead, and none for None', () => {
|
||||
const f = scene();
|
||||
f.setItems(f.fx.other.id, f.fx.other.id);
|
||||
expect(songSequence(f.project, f.song()).clips?.map((c) => c.fx)).toEqual([f.fx.other.id]);
|
||||
const picked = songBuses(f.project, f.song()).find((b) => b.name.endsWith(f.riff));
|
||||
expect(picked?.fx?.plan).toBe(f.plan(f.fx.other));
|
||||
f.setItems(NO_FX, NO_FX);
|
||||
const { clips, notes } = songSequence(f.project, f.song());
|
||||
expect(clips?.map((c) => c.fx)).toEqual(['']);
|
||||
// Without a sound FX the pattern item has no bus of its own; its notes go in with the track's.
|
||||
expect(notes.map((n) => n.bus)).toEqual([trackItemsBus(f.songId, f.track)]);
|
||||
expect(songBuses(f.project, f.song()).some((b) => b.name.endsWith(f.riff))).toBe(false);
|
||||
});
|
||||
|
||||
it('loads the sound FX of an instrument with the layers of its notes', () => {
|
||||
const f = scene();
|
||||
const { layers } = sequenceNeeds(f.project, songSequence(f.project, f.song()));
|
||||
expect(layers).toContain(f.plan(f.fx.leadFx));
|
||||
});
|
||||
});
|
||||
|
||||
describe('the sound FX of what a Play node plays (#352)', () => {
|
||||
/** What one Play node plays on a scene of its own, its params given the scene. */
|
||||
function played(type: string, params: (f: ReturnType<typeof scene>) => PropertyBag) {
|
||||
const f = scene();
|
||||
const start = f.edit.addNode('OnStart');
|
||||
const play = f.edit.addNode(type, params(f));
|
||||
f.edit.wire([[start, 'exec', play, 'exec']]);
|
||||
const audio = recording();
|
||||
startWorld(f, { behaviors: true, audio }).frame();
|
||||
return Object.assign(audio, { f });
|
||||
}
|
||||
const planOf = (f: ReturnType<typeof scene>, name: keyof ReturnType<typeof scene>['fx']) =>
|
||||
f.plan(f.fx[name]);
|
||||
|
||||
it('plays a sound through its own unless the node picks another or None', () => {
|
||||
const own = played('PlaySound', (f) => ({ sound: f.sound }));
|
||||
expect(own.sounds[0]?.plan).toBe(planOf(own.f, 'soundFx'));
|
||||
const other = played('PlaySound', (f) => ({ sound: f.sound, fx: f.fx.other.id }));
|
||||
expect(other.sounds[0]?.plan).toBe(planOf(other.f, 'other'));
|
||||
expect(played('PlaySound', (f) => ({ sound: f.sound, fx: NO_FX })).sounds[0]).toBe(DRY_VOICE);
|
||||
});
|
||||
|
||||
it('plays a note through the sound FX of its instrument after what the instrument does', () => {
|
||||
const own = played('PlayNote', (f) => ({ instrument: f.lead.id, note: 60 })).notes[0];
|
||||
expect(gains(own?.layers[0])).toEqual([-2]);
|
||||
const picked = played('PlayNote', (f) => ({ instrument: f.lead.id, note: 60, soundFx: f.fx.other.id }))
|
||||
.notes[0];
|
||||
expect(gains(picked?.layers[0])).toEqual([-5]);
|
||||
const none = played('PlayNote', (f) => ({ instrument: f.lead.id, note: 60, soundFx: NO_FX })).notes[0];
|
||||
expect(gains(none?.layers[0])).toEqual([]);
|
||||
});
|
||||
|
||||
it('plays a pattern and a song through theirs, and every note of them through the instrument’s', () => {
|
||||
const pattern = played('PlayPattern', (f) => ({ pattern: f.pattern }));
|
||||
expect(pattern.sequences[0]?.play.fx?.plan).toBe(planOf(pattern.f, 'riffFx'));
|
||||
expect(
|
||||
played('PlayPattern', (f) => ({ pattern: f.pattern, fx: NO_FX })).sequences[0]?.play.fx,
|
||||
).toBeUndefined();
|
||||
const song = played('PlaySong', (f) => ({ song: f.songId }));
|
||||
expect(song.sequences[0]?.play.fx?.plan).toBe(planOf(song.f, 'themeFx'));
|
||||
const other = played('PlaySong', (f) => ({ song: f.songId, fx: f.fx.other.id }));
|
||||
expect(other.sequences[0]?.play.fx?.plan).toBe(planOf(other.f, 'other'));
|
||||
const asked = pattern.sequences[0]!.play;
|
||||
asked.strike({ ...asked.sequence.notes[0]!, at: 0, duration: 0.1, tempo: 120 });
|
||||
expect(gains(pattern.notes.at(-1)?.layers[0])).toEqual([-2]);
|
||||
});
|
||||
});
|
||||
@@ -55,6 +55,7 @@ describe('pattern asset (#333)', () => {
|
||||
'beats',
|
||||
'beat',
|
||||
'tempo',
|
||||
'fx',
|
||||
'bus',
|
||||
]);
|
||||
});
|
||||
|
||||
@@ -75,6 +75,7 @@ describe('the song asset (#334)', () => {
|
||||
'beats',
|
||||
'beat',
|
||||
'grid',
|
||||
'fx',
|
||||
'bus',
|
||||
]);
|
||||
expect(songGrid(f.song().props)).toBe(BAR);
|
||||
|
||||
@@ -138,6 +138,7 @@ describe('the sound asset', () => {
|
||||
FADE_IN_CURVE_KEY,
|
||||
FADE_OUT_KEY,
|
||||
FADE_OUT_CURVE_KEY,
|
||||
'fx',
|
||||
BUS_KEY,
|
||||
]);
|
||||
expect(cutOf(asset.props)).toEqual({
|
||||
|
||||
@@ -976,6 +976,81 @@ export async function song(t) {
|
||||
await click(p.x, p.y);
|
||||
await sleep(150);
|
||||
|
||||
// 65n. A sound FX on the song and on its items (#352). An item's Sound FX and a pattern item's
|
||||
// Instrument are the ones of its asset while empty; None is a pick of its own. The song
|
||||
// plays through its own on a bus of the tab, into the bus of the song. A double-click on a
|
||||
// sound item opens the sound.
|
||||
await evaluate('__gm.sound.stopAll()');
|
||||
// The items as they are now: those of 65c went with the takes of 65l.
|
||||
const soundItem = (await model()).items.find((item) => item.kind === 'sound');
|
||||
const patternItem = (await model()).items.find((item) => item.kind === 'pattern');
|
||||
p = await json(`__gm.song.itemAt(${JSON.stringify(soundItem.id)}, 'body')`);
|
||||
await click(p.x, p.y);
|
||||
await sleep(150);
|
||||
p = await json("__gm.inspectorField('Sound FX', true)");
|
||||
await click(p.x, p.y);
|
||||
await sleep(150);
|
||||
const fxRows = await json('__gm.menuRows()');
|
||||
await clickText('None');
|
||||
await sleep(150);
|
||||
const clipFx = (await model()).items.find((item) => item.id === soundItem.id)?.fx;
|
||||
p = await json(`__gm.song.itemAt(${JSON.stringify(patternItem.id)}, 'body')`);
|
||||
await click(p.x, p.y);
|
||||
await sleep(150);
|
||||
p = await json("__gm.inspectorField('Instrument', true)");
|
||||
await click(p.x, p.y);
|
||||
await sleep(150);
|
||||
const instrumentRows = await json('__gm.menuRows()');
|
||||
await press('Escape');
|
||||
check(
|
||||
"an item's Sound FX offers From the asset, None and the sound FX, its Instrument From the asset",
|
||||
fxRows?.[0] === 'From the asset' &&
|
||||
fxRows[1] === 'None' &&
|
||||
fxRows.includes('Octave 65 (soundfx)') &&
|
||||
clipFx === '#none' &&
|
||||
instrumentRows?.[0] === 'From the asset',
|
||||
JSON.stringify({ fxRows, clipFx, instrumentRows }),
|
||||
);
|
||||
await evaluate(`__gm.state.select({ kind: 'asset', assetId: ${JSON.stringify(id)} })`);
|
||||
await sleep(150);
|
||||
p = await json("__gm.inspectorField('Sound FX', true)");
|
||||
await click(p.x, p.y);
|
||||
await sleep(150);
|
||||
await clickText('Octave 65 (soundfx)');
|
||||
await sleep(150);
|
||||
const songFx = await json(`__gm.state.project.assets.find((a) => a.id === ${JSON.stringify(id)}).props.fx`);
|
||||
p = await json('__gm.song.playAt()');
|
||||
await click(p.x, p.y);
|
||||
const throughSong = await waitFor(
|
||||
() => json('__gm.song.buses()'),
|
||||
(list) => list.some((b) => b.name.startsWith('music:') && b.fx),
|
||||
);
|
||||
const songFxBus = throughSong.find((b) => b.name.startsWith('music:') && b.fx);
|
||||
await click(p.x, p.y);
|
||||
check(
|
||||
'the Sound FX of the song is on a bus of the tab into the bus of the song, the tracks to the output go there',
|
||||
songFx === 'sg65-fx' &&
|
||||
songFxBus?.into === 'mix' &&
|
||||
Math.abs(songFxBus.latency - 2048 / (await json('__gm.sound.sampleRate()'))) < 1e-9 &&
|
||||
throughSong.find((b) => b.name === busOf(ids[1]))?.sends?.[0]?.to === songFxBus.name,
|
||||
JSON.stringify({ songFx, songFxBus, sends: throughSong.find((b) => b.name === busOf(ids[1]))?.sends }),
|
||||
);
|
||||
const fxGone = await waitFor(
|
||||
() => json('__gm.song.buses()'),
|
||||
(list) => !list.some((b) => b.name.startsWith('music:')),
|
||||
);
|
||||
p = await json(`__gm.song.itemAt(${JSON.stringify(soundItem.id)}, 'body')`);
|
||||
await doubleClick(p.x, p.y);
|
||||
await sleep(200);
|
||||
const openedSound = await json('{ tab: __gm.tab(), sound: __gm.soundTab.current() }');
|
||||
check(
|
||||
'stopping lets the bus of the sound FX go, and a double-click on a sound item opens the sound',
|
||||
!fxGone.some((b) => b.name.startsWith('music:')) &&
|
||||
openedSound.tab === 'sound' &&
|
||||
openedSound.sound === SOUND,
|
||||
JSON.stringify({ fxGone: fxGone.map((b) => b.name), openedSound }),
|
||||
);
|
||||
|
||||
// 65j. The step takes back everything it made.
|
||||
await evaluate('__gm.sound.stopAll()');
|
||||
await mutate(`
|
||||
|
||||
Reference in new issue
Block a user