Describe the instrument asset and Play note #331
A new page Instruments: the nodes, triggers, zones and layers, how a note is compiled and played, the keys of the tab and Play note. - _Sidebar and Home link the page, Concepts lists the asset type. - Sounds points to it for playing a sound as notes. - Behaviors lists Play note, Architecture says a note is one voice of several layers.
admin committed
1 parent
c11e3109b0
commit
fb136da29d
7 files changed
+127
-4
No files matched your search
+2
-1
@@ -199,7 +199,8 @@ src/viewport/ WorldViewport (the drawing of a world or a holder, the ba
|
||||
server lacks are uploaded before a save, #130) + pixelArtPicture (every frame composed on the
|
||||
CPU and laid side by side onto one canvas, so a frame is cut out like a texture frame),
|
||||
SoundLibrary + pitchWorklet (the AudioWorklet that plays a way with a Pitch through
|
||||
core/soundfx/pitchPlayer, a phase vocoder, #317), ShaderLibrary, ParticleLibrary,
|
||||
core/soundfx/pitchPlayer, a phase vocoder, #317; a note of an instrument is one voice
|
||||
of several layers, each way with its own file, #331), ShaderLibrary, ParticleLibrary,
|
||||
particles (one ParticleContainer per source), tilemap (one container per layer),
|
||||
shaderPicture (the picture a shader is laid over when it shades a camera's frame, #299, or
|
||||
a particle field, #170: drawn anew each time into a render texture kept while its size
|
||||
|
||||
+2
-1
@@ -258,7 +258,8 @@ the inspector).
|
||||
`Play sound` / `Stop sound` / `Set sound FX parameters` (see [Sounds](Sounds): the asset says how loud and whether it loops, `fx` sends it through a sound FX, whose parameters a `Sound FX parameters` node sets through `fx params`, on a playing sound too, the node's
|
||||
`volume` multiplies that and its `loop` either leaves it alone or overrules it in either
|
||||
direction; every play is a voice of its own, so two in a row overlap, and the voice ends with the
|
||||
behavior that started it unless `detached` says otherwise),
|
||||
behavior that started it unless `detached` says otherwise), `Play note` (one note of an
|
||||
instrument, its parameters from an `Instrument parameters` node, see [Instruments](Instruments)),
|
||||
`Load scene` (swaps the running scene once the frame is over — see [Scenes](Scenes)), `Log`, `Stop game`, `Restart game`,
|
||||
`Pause game`, `Resume game`, `Set time scale` with the data nodes `Is game paused` and `Time scale` (see [Concepts](Concepts), *The game pauses itself*),
|
||||
`Apply force`, `Apply impulse`, `Set velocity`, `Set angular velocity`, `Set gravity` (see [Physics](Physics)),
|
||||
|
||||
+2
-2
@@ -1,10 +1,10 @@
|
||||
| Term | Meaning |
|
||||
|---|---|
|
||||
| **Account** | Who you are signed in as. The editor asks for a user name and a password before it loads anything, and every project belongs to one account — see [Accounts](Accounts). The **Account** menu is at the far right of the toolbar and holds the name, *Change password* and *Logout*; signing in and out changes what the canvas draws, it never reloads the page. |
|
||||
| **Asset** | Named thing with a type (`scene`, `camera`, `clip`, `sprite`, `text`, `behavior`, `collider`, `container`, `texture`, `animation`, `statemachine`, `shader`, `shaderfunction`, `font`, `sound`, `soundfx`, `particles`, `variables`) and typed properties. Assets have **no position**. |
|
||||
| **Asset** | Named thing with a type (`scene`, `camera`, `clip`, `sprite`, `text`, `behavior`, `collider`, `container`, `texture`, `animation`, `statemachine`, `shader`, `shaderfunction`, `font`, `sound`, `soundfx`, `instrument`, `particles`, `variables`) and typed properties. Assets have **no position**. |
|
||||
| **Texture** | An uploaded image. Scenes, textures, animations, state machines, shaders, fonts, sounds and Variables assets are the eight types that cannot be placed: a texture has no position, it is *worn* by a sprite through the ordinary property `texture`. See [Textures](Textures). |
|
||||
| **Font** | An uploaded typeface, worn by a text through the ordinary property `font`; no font means the editor's own. Uploaded and picked exactly like a texture, and stored the same way. See [Fonts](Fonts). |
|
||||
| **Sound** | An uploaded audio file. Uploaded, picked and stored exactly like a typeface — but nothing *wears* it: it is neither on screen nor in the scene, and a graph plays it with `Play sound`. Its own rows say how loud it is and whether it loops; the node scales the one and can overrule the other either way. A **sound FX** asset is a graph of effects (gain, pitch, EQ, mix) a `Play sound` can send it through. See [Sounds](Sounds). |
|
||||
| **Sound** | An uploaded audio file. Uploaded, picked and stored exactly like a typeface — but nothing *wears* it: it is neither on screen nor in the scene, and a graph plays it with `Play sound`. Its own rows say how loud it is and whether it loops; the node scales the one and can overrule the other either way. A **sound FX** asset is a graph of effects (gain, pitch, EQ, mix) a `Play sound` can send it through. See [Sounds](Sounds). An **instrument** plays sounds as notes, see [Instruments](Instruments). |
|
||||
| **State machine** | A named set of **states**, edited as nodes in its own tab. A behavior puts it into a state with `Set state`, moves it along a named transition with `State transition` and branches on it with `State branch`. Its **Scope** says where the current state lives: **Local** gives every behavior instance one of its own, **Global** keeps a single one for the whole run. See [State machines](State-machines). |
|
||||
| **Shader** | A node graph that becomes the GLSL a sprite is drawn with, edited in its own tab. A sprite, a text or a particle source wears it through `shader`; the numbers the graph declares are rows below that dropdown. A **shader function** is a part of such a graph kept on its own and used as a node in other shaders. See [Shaders](Shaders). |
|
||||
| **Particles** | A node graph that says what *one* particle is as a function of its age and of a random number drawn once for it — no forces, no simulation. Unlike the other graph assets it **is** placeable: the object made from it is a source, whose rows say how many go out and for how long they live. Because a particle is a formula, the same second always draws the same picture and the tab's slider can jump to any of them exactly. See [Particles](Particles). |
|
||||
|
||||
+1
@@ -22,6 +22,7 @@ Live: <https://nockal.com/game-maker/> · Widget playground: <https://nockal.com
|
||||
| [Pixel art](Pixel-Art) | The pixel art asset and its palette, the tools, layers and frames, onion skin and the preview, resizing, how frames are worn and where the pixels are stored |
|
||||
| [Fonts](Fonts) | Uploading a typeface, what a text wears, what the editor falls back to |
|
||||
| [Sounds](Sounds) | Uploading a sound, what the asset says, `Play sound` and `Stop sound`, sound FX (gain, pitch, EQ, mix, delay) and how long a voice rings on, what a pause and a stop do to a voice |
|
||||
| [Instruments](Instruments) | The instrument asset: Notes, Sample, Parameters, Sound FX and Output, triggers, zones and layers, the keys of the Instrument tab, `Play note` |
|
||||
| [Animations](Animations) | Animation assets, the Animation tab, how they run |
|
||||
| [Curves](Curves) | Easings drawn by hand, the Curve tab, using one on a key or a `Lerp` |
|
||||
| [Tiles](Tiles) | The tile asset, plain tiles and the two autotile kinds, the pictures as animation steps, the Tile tab |
|
||||
|
||||
+116
@@ -0,0 +1,116 @@
|
||||
An **instrument** (**+ → Instrument**, #331) makes a note a sound. It is a node graph like a
|
||||
[sound FX](Sounds#sound-fx): **Notes** nodes bring the note in, a **Sample** plays a sound for it,
|
||||
the effects of a sound FX shape it and the **Output** is what is heard. It is the first piece of
|
||||
the music tools planned in #330: a pattern (#333) and a song (#334) will play instruments. Until
|
||||
then a behavior plays one with **Play note**, and the tab plays it from its keys.
|
||||
|
||||
Instruments are listed under **Music** in the asset panel, where patterns and songs will join them.
|
||||
A new one starts as *Notes* → *Sample* → *Output*: every note from C4 to B4 plays the Sample,
|
||||
which waits for its sound to be picked.
|
||||
|
||||
## Notes
|
||||
|
||||
Notes are numbered as MIDI numbers them: 0 is the lowest, **60 is C4** (the middle C), 69 is the
|
||||
A at 440 Hz, 127 the highest. A note is named with its octave after it and no dash, sharps only:
|
||||
`C4`, `C#4`, `D4` … The lowest is `C-1`, the highest `G9`.
|
||||
|
||||
## The nodes
|
||||
|
||||
| Node | What it does |
|
||||
| --- | --- |
|
||||
| **Notes** | Where the notes come in. `any` triggers what it is connected to for every note from **From** to **To**. `note` is the number of the note played, `frequency` its pitch in Hz (`440 * 2^((note - 69) / 12)`), `velocity` how hard it is struck, 0 to 1. With **Single notes** there is one trigger output for every note in the range as well, and each can have a **name** of its own (*Name of C2* in the inspector): the output shows it, and a piano roll will. |
|
||||
| **Sample** | Plays a sound asset when its `trigger` is on. With **Follow note** (on by default) it plays higher or lower by the distance from **Root**, the note it sounds as it is (C4 by default), and **keeps its length** — it is played by the same worklet as a Pitch (#317). Off, every note plays it as it is, as for a drum. **Mode**: *Gate* ends it with the note, faded out in 10 ms; *One-shot* plays it through whatever the note does. **Loop** repeats it, between **Loop start** and **Loop end** in seconds where they are set (0 is the end of the sound). The volume of the sound asset counts. |
|
||||
| **Parameters** | The instrument's own parameters, each a number output with a default — like the parameters of a sound FX's Input. One per instrument. |
|
||||
| **Sound FX** | Sends the sound through a sound FX asset, as if it were the sound at the FX's Input. Every parameter of that FX is an input here; one left open is the FX's default. The node is titled after the FX, and a double click opens it. |
|
||||
| **Output** | What is heard. Unlike every other sound input it takes **any number** of sounds and plays them together: the layers of the instrument. One per instrument. |
|
||||
| Gain, Pitch, 5-Band-EQ, Mix, Delay | The effects of a [sound FX](Sounds#sound-fx), the same nodes. |
|
||||
|
||||
**Triggers.** The outputs of a Notes node that say "this note" are of their own kind, a
|
||||
**trigger**, drawn in lime. A trigger goes only into the `trigger` input of a source — nothing of a
|
||||
behavior can get in, and it fits nothing else. A `trigger` input takes any number of edges, and
|
||||
the source plays when one of them is on: C1 and D1 can both play the same Sample.
|
||||
|
||||
**Zones and layers.** Several Notes nodes split the keyboard into zones: a bass sample for C1-B2,
|
||||
another one from C3. Where two ranges overlap, both are heard. A drum kit is one Notes node over
|
||||
C1-B1 with Single notes on, a Sample on each output (C1 Kick, D1 Snare, F#1 Hi-Hat …) and all of
|
||||
them into the Output: one instrument plays the whole kit.
|
||||
|
||||
## How a note is made
|
||||
|
||||
The graph is **compiled once per note** and kept until the instrument, or a sound or sound FX it
|
||||
names, changes. For a note, `any` of a Notes node is on when the note is in its range, a single
|
||||
output when it is that note; every other trigger is off. A Sample whose trigger is off is silence,
|
||||
and so is everything only it feeds; a **Mix** with one silent side is its other side. A note for
|
||||
which nothing sounds is silence, not a problem: the keys draw it grey and `Play note` gives out
|
||||
voice 0.
|
||||
|
||||
- `note` and `frequency` are numbers once the note is known. `velocity` and the instrument's
|
||||
parameters stay open, filled in for every voice, so `Set sound FX parameters` can still move
|
||||
them on a playing note.
|
||||
- **Velocity is always loudness**: the voice is played at `volume * velocity`. The output
|
||||
`velocity` is there for anything else as well, a filter that opens with a harder stroke.
|
||||
- A note is **one voice**, its layers side by side, each way reading its own file. It ends when all
|
||||
its ways have ended, or rings on first where it has an echo (see [Sounds](Sounds#delay-and-the-tail-of-a-voice)).
|
||||
- **The end of a note** (the key let go, the duration of `Play note` over): every *Gate* Sample
|
||||
fades out in 10 ms, *One-shot* ones play on. A looped one-shot plays until it is stopped.
|
||||
- **Pitch.** A Sample that follows the note is shifted by the phase vocoder of #317, so a note far
|
||||
from the root sounds rough, as there. A note at its root is the sound itself, played without the
|
||||
worklet.
|
||||
- **Sound FX node and a Mix.** The arriving sound is set in wherever the FX's Input was. A Mix in
|
||||
the FX therefore plays the sound twice, one source per way, as in the sound FX itself.
|
||||
|
||||
A node that keeps the instrument from compiling says so, and the tab's header says it too
|
||||
(otherwise it counts the notes that sound): a Sample with no sound picked or one that is gone, a
|
||||
source nothing triggers, an effect nothing comes into. A Sound FX node naming no FX, or one that is
|
||||
gone, warns and lets the sound through as it is.
|
||||
|
||||
## The Instrument tab
|
||||
|
||||
The graph, and below it **the keys**: all 128 notes, the ones that sound in white and black, the
|
||||
others grey. A key plays while the pointer holds it; dragging across the keys plays each one in
|
||||
turn. A click also gives the keys the **computer keyboard**, played as in a tracker by where a key
|
||||
is, not what it says, so QWERTZ and QWERTY play alike:
|
||||
|
||||
| Keys | Notes |
|
||||
| --- | --- |
|
||||
| `Z S X D C V G B H N J M` | C to B of the bottom octave (C3 at first) |
|
||||
| `, L . ; /` | C to E of the octave above |
|
||||
| `Q 2 W 3 E R 5 T 6 Y 7 U` | C to B of the octave above the bottom one |
|
||||
| `I 9 O 0 P` | C to E of the octave after that |
|
||||
| `←` `→` | The bottom octave down and up, from C-1 to C8 |
|
||||
|
||||
The header says which notes the keys play (*Keys C3 to E5*), and a line under the keys marks them
|
||||
while the keys have the focus. A held key does not repeat; letting it go, or the keys losing the
|
||||
focus, ends the note. **Stop** ends everything the tab started. Every parameter of the
|
||||
instrument has a number field in the header, starting at its default: what the keys play with,
|
||||
heard at once on what is playing. Those values are the tab's own and never written into the asset.
|
||||
|
||||
## Playing one in a game
|
||||
|
||||
**Play note** plays one note of an instrument:
|
||||
|
||||
| Input | |
|
||||
| --- | --- |
|
||||
| `instrument` | The instrument asset, picked on the node or arriving as an id. |
|
||||
| `note` | Its number, 60 is C4 (the default). |
|
||||
| `velocity` | 0 to 1, how hard it is struck: quieter, and handed to what the instrument does with it. Default 1. |
|
||||
| `volume` | A factor like the volume of `Play sound`. Default 1. |
|
||||
| `duration` | Seconds until the note is released, on the clock of the sound. **0 holds it** until a `Stop sound` ends it. |
|
||||
| `detached` | As at `Play sound`: the note outlives the object that played it. |
|
||||
| `params` | Values for the instrument's parameters, from an **Instrument parameters** node. |
|
||||
|
||||
Its output `voice` is the number of the note, like `Play sound`'s: `Set sound FX parameters` moves
|
||||
its parameters while it plays (the bundle of an Instrument parameters node goes into its
|
||||
`fx params`), `Stop sound` with `voice` ends it — with *Let ring* only the sound, so its echoes
|
||||
ring on. A note belongs to the object that played it and stops when that object is destroyed.
|
||||
|
||||
**Instrument parameters** works like `Sound FX parameters`: it names an instrument and picks the
|
||||
parameters to set, each a number input starting at the default its Parameters node gives it.
|
||||
Renaming a parameter there renames it on every such node; where `Play note` and the node name two
|
||||
different instruments, `Play note` warns.
|
||||
|
||||
## Not yet
|
||||
|
||||
No envelope with a release after the note (#335), no oscillators or 8-bit sources (#329), no
|
||||
patterns, songs or scheduler (#332-#334). Notes can not be played from an `.mid` file or a MIDI
|
||||
keyboard.
|
||||
+3
@@ -266,6 +266,9 @@ one that was deleted, plays the sound **dry** rather than not at all: a broken e
|
||||
silence the game. One asset can serve any number of sounds, so "everything in the cave sounds
|
||||
hollow" is one asset and one pick per node.
|
||||
|
||||
**Notes.** To play a sound as notes — higher and lower, held and let go, several sounds as one
|
||||
kit — make it the Sample of an [instrument](Instruments) and play that with `Play note`.
|
||||
|
||||
**Known limits.** There is no `Set volume` on a playing voice (a Gain fed by a parameter does the
|
||||
job), no `Is playing`, no fade and no positional audio. There is no mixer and no master volume — the browser's own is the one. And nothing
|
||||
collects sound files a project stopped referencing; they stay on the server, which is what makes
|
||||
|
||||
+1
@@ -13,6 +13,7 @@
|
||||
- [Pixel art](Pixel-Art)
|
||||
- [Fonts](Fonts)
|
||||
- [Sounds](Sounds)
|
||||
- [Instruments](Instruments)
|
||||
- [Animations](Animations)
|
||||
- [Curves](Curves)
|
||||
- [Tiles](Tiles)
|
||||
|
||||
Reference in new issue
Block a user