Compositions: sounds, patterns and songs as items on a track (#418)
- Compositions has a new section on audio items: how they are dragged in, what the inspector offers, how rounds and loop marks work, and how they sound in the tab and in a game. - Concepts says that folding in the asset panel keeps the scroll position. - Development counts 2072 editor checks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 parent
3311619147
commit
4203ca83ea
3 files changed
+60
-10
No files matched your search
+56
-9
@@ -1,7 +1,8 @@
|
||||
A **composition** lays objects out in time (#417): a cut of video clips, a cutscene, a title that
|
||||
comes in and goes again. It has **tracks**, and on each track **items** and **animation clips**. An
|
||||
item is one object that lives from its start to its end; a clip plays an animation on the items of
|
||||
its track while it runs. Placed in a scene, a composition plays when a behavior says so.
|
||||
item is one object that lives from its start to its end, or a sound, a pattern or a song heard while
|
||||
it lasts (#418); a clip plays an animation on the items of its track while it runs. Placed in a
|
||||
scene, a composition plays when a behavior says so.
|
||||
|
||||
## The asset
|
||||
|
||||
@@ -19,8 +20,30 @@ of objects beside scenes and containers (`ObjectHolder` in `src/core/model/Proje
|
||||
|
||||
What is stored (`CompositionAsset` in `src/core/model/Asset.ts`): `tracks` (id, name), `items` (id
|
||||
= the object's id, track, start and length in frames, offset in seconds into its video,
|
||||
`fitMedia`) and `clips` (id, track, animation, start in frames). An item whose object is gone stays
|
||||
in the list, so an undo finds it again, and plays as nothing.
|
||||
`fitMedia`, and for an audio item `audio`) and `clips` (id, track, animation, start in frames). An
|
||||
item whose object is gone stays in the list, so an undo finds it again, and plays as nothing.
|
||||
|
||||
### Audio items
|
||||
|
||||
A **sound**, a **clip of a sound**, a **pattern** or a **song** dragged onto a track is an audio item
|
||||
(#418). It has no object; `audio` says what it plays (`kind`, `asset`), its **volume** in dB
|
||||
(-60 to +12) and its **bus**. It shares a track with object items and may overlap them.
|
||||
|
||||
- **It plays for as long as it lasts**, and stops at its end. Longer than what it plays, it goes
|
||||
round and round: a sound and a pattern from their start, a song round the loop of its ruler, as
|
||||
`Play song` with Loop does. The **offset** is how far into it the item begins, in seconds; a trim
|
||||
of the left edge moves it. Below 0 the item is silent until then.
|
||||
- **A pattern or a song keeps its own tempo** inside the item. A Delay synced to beats counts by it.
|
||||
- **The bus.** *From asset* (the default) sends it where the [Master routing](Sounds#master-routing)
|
||||
sends its sound, pattern or song; *Master* or a bus sends it there instead. Each item is heard on
|
||||
a bus of its own at its volume, through the sound FX of its pattern or song; a song's tracks are
|
||||
buses of that item, so two items of one song do not share them.
|
||||
- **Animation clips do not belong to an audio item**: there is nothing on it they could move.
|
||||
- **What is gone plays nothing.** The item stays, drawn red, named *(missing)*.
|
||||
|
||||
The sequences are made in `src/core/music/compositionAudio.ts`: everything in milliseconds of the
|
||||
composition, one sequence for each place of the mix the items go to. `CompositionSound`
|
||||
(`src/runtime/compositionSound.ts`) plays them through the scheduler, in the tab and in a game.
|
||||
|
||||
### Sprites of its own
|
||||
|
||||
@@ -79,6 +102,10 @@ the clips as narrower bars above them. The header bar has **+ Track**, **Play**
|
||||
same way, wearing a sprite the composition keeps for that picture (see *Sprites of its own*
|
||||
below). A video's item is as long as the video. A clip of a video is chosen afterwards, on the
|
||||
Texture row of the item's inspector.
|
||||
- **Dragging a sound, a clip of one, a pattern or a song onto a track** (#418) makes an audio item
|
||||
there, as long as one pass of it, picked; a song's first pass ends where its loop does. A sound
|
||||
has to be loaded to know its length; the drag loads it, and until it is there the item is 3 s
|
||||
long.
|
||||
- **Dragging an animation onto a track** makes a clip at that frame, if it fits between the clips
|
||||
there.
|
||||
- **Dragging an item** moves it in time and from track to track, past and over other items; it
|
||||
@@ -97,13 +124,19 @@ the clips as narrower bars above them. The header bar has **+ Track**, **Play**
|
||||
item cuts into the video on that side: there would be more of it. A line with a triangle, like a
|
||||
loop in the Song tab, marks where the video starts or ends inside the item; with Loop on the
|
||||
video, one line per round. An overlap is darkened, and a line marks the end of the item below.
|
||||
An audio item shows the same marks for what it plays, a line at every round of it; for a song
|
||||
the first at the end of its loop, then one per time round the loop. A sound item draws its
|
||||
waveform; sounds are blue-green, patterns violet, songs plum.
|
||||
- **Click** picks an item or a clip, `Ctrl`-click adds or removes one. A click on a track head picks
|
||||
the track, a click in the ruler moves the playhead there, and dragging in it scrubs.
|
||||
- **A band** dragged from an empty spot picks the items it touches. With `Alt` held it picks the
|
||||
clips instead and may start anywhere, on an item too. With `Ctrl` it adds to what is picked. A
|
||||
pick holds items or clips, never both.
|
||||
pick holds items or clips, never both. Items that are objects only are picked as objects, so the
|
||||
hierarchy and the inspector show them; with an audio item among them the pick is the
|
||||
composition's, and the inspector shows the audio item, or the count for several.
|
||||
- **The right-click menu** on an item: *Fit to media*, *Make project asset* (only for an item
|
||||
wearing a sprite of the composition's own), *Remove item*. On a clip: *Open animation*,
|
||||
wearing a sprite of the composition's own), *Remove item*. On an audio item *Open sound*
|
||||
(*pattern*, *song*) instead of *Make project asset*. On a clip: *Open animation*,
|
||||
*Remove clip*. On a track head or an empty stretch of a track, as in the Song tab: *Add track
|
||||
above*, *Add track below*, *Move up*, *Move down*, *Delete track*.
|
||||
- **Copy and paste.** `Ctrl+C` on picked items keeps their objects and their timing, `Ctrl+V` in
|
||||
@@ -113,7 +146,8 @@ the clips as narrower bars above them. The header bar has **+ Track**, **Play**
|
||||
their distances and their tracks to each other. The same copy pastes into a scene as plain
|
||||
objects; objects copied from a scene do not paste into a composition, they have no timing.
|
||||
Copied items take their clips along; where one of those would lie over a clip, items and clips
|
||||
move on together to the first frame where they fit. **Picked clips** are copied and pasted the same way, into this composition or another. As clips
|
||||
move on together to the first frame where they fit. Audio items are copied with them, and alone
|
||||
the same way. **Picked clips** are copied and pasted the same way, into this composition or another. As clips
|
||||
of a track never overlap, a paste that would lie over a clip moves on, all its clips together,
|
||||
to the first frame where they fit.
|
||||
- `Delete` removes the picked items with their objects and their clips, the picked clips or the
|
||||
@@ -121,6 +155,10 @@ the clips as narrower bars above them. The header bar has **+ Track**, **Play**
|
||||
- **Play** plays from the playhead at the composition's frame rate, round with its Loop; from the
|
||||
end it starts at 0. The wheel zooms the time, the middle button or `Shift`-drag pans. Both glide
|
||||
like the other tabs (see [Concepts](Concepts)).
|
||||
- **Audio items are heard while it plays**, from where the playhead stands, straight into the
|
||||
master as in every tab. Their bus is heard in a game. A scrub is silent. While the playhead
|
||||
waits for a video, the sound waits too. An edit while it plays is heard at once. Playing a song
|
||||
or a pattern in its tab stops the composition, and the other way round.
|
||||
|
||||
**The picture** is a scene view whose objects are the items alive at the playhead. A click picks
|
||||
an item or its child, dragging moves the real object. It takes nothing from the asset panel and
|
||||
@@ -130,7 +168,9 @@ the links of their video in the [Master routing](Sounds#master-routing).
|
||||
|
||||
**The hierarchy** lists the items alive at the playhead; re-parenting is off there, because items
|
||||
stay at the top. **The inspector** of an item shows its object's rows and a block **TIMING**:
|
||||
*Start*, *Length* and, for a video item, *Offset*, all in seconds, and **Fit to media**. A track
|
||||
*Start*, *Length* and, for a video item, *Offset*, all in seconds, and **Fit to media**. An audio
|
||||
item shows **SOUND ITEM** (*PATTERN*, *SONG*): what it plays, the same timing rows with *Offset*,
|
||||
**Volume** in dB and **Bus** (*From asset*, *Master* or a bus). A track
|
||||
shows its name; a clip its animation and its start, with the length of one pass.
|
||||
|
||||
**Fit to media.** A new item's length follows its video: when an override gives its sprite another
|
||||
@@ -167,6 +207,13 @@ it picks the placement, its items cannot be picked one by one there.
|
||||
still seeks to it — the composition's clock stands, for at most 3 s at a time.
|
||||
The rest of the game runs on. The Composition tab holds its playhead the same way. A recording
|
||||
takes no frame meanwhile (see [Videos](Videos#recording-the-game-view)).
|
||||
- **Its audio items are heard while its clock runs** (#418), on the bus each one names. They are
|
||||
planned on the audio clock like the notes of a song, for the moment the clock reaches them, so
|
||||
sound and picture keep together. While the clock stands (waiting for a video, the placement
|
||||
switched off, the game paused), they are silent, and start again where it stands. `Stop
|
||||
composition` and its end stop them. A time scale other than 1 moves the picture and not the
|
||||
sound, and so does a frame longer than 0.1 s, which the game clock counts as 0.1 s: after one,
|
||||
the sound is ahead of the picture until it starts again after the clock stood.
|
||||
- **The editor run plays none** (*Run in editor*): it writes into the saved project, and the items
|
||||
would stay there. `Play composition` says so in the log there.
|
||||
|
||||
@@ -176,6 +223,6 @@ with the same row binding the `AnimationSystem` uses.
|
||||
|
||||
## Not yet
|
||||
|
||||
- Sounds and songs as items on a track (#418).
|
||||
- A baked song plays its notes inside a composition, not its bake.
|
||||
- Dragging a clip of a video straight out of the Video tab onto a track.
|
||||
- A composition inside a composition, and *Sort with parent* for a placement.
|
||||
+3
@@ -321,6 +321,9 @@ below); quick steps add up and go on at the pace the list had, and dragging the
|
||||
a row ends the glide where it is.
|
||||
Below the last row the asset panel always keeps room for six more, so its end can be scrolled up
|
||||
away from the bottom edge; clicks and the context menu there work as on any empty part of the list.
|
||||
Folding a group leaves the list where it stands: the heading clicked stays under the pointer, and
|
||||
rows that fold away near the end leave their room below until the list is scrolled up. Only a new
|
||||
selection scrolls the list to its row.
|
||||
|
||||
**Panel sizes are remembered.** Every divider stays where it was dragged, after a reload too: the
|
||||
ones left and right of the centre column, the one above the console, the one in the left column,
|
||||
|
||||
+1
-1
@@ -21,7 +21,7 @@ php tests/php/releases.test.php # the release stor
|
||||
php tests/php/files.test.php # uploaded files in a folder per account (#344), the same way
|
||||
php tests/php/videos.test.php # the video store, its queue and the worker (#402), with the
|
||||
# real ffmpeg: the CI image and feuerwerk-video have it
|
||||
node tools/seed-e2e.mjs && node tools/e2e.mjs # 2068 editor checks, screenshots in /tmp/gm-e2e-*.png
|
||||
node tools/seed-e2e.mjs && node tools/e2e.mjs # 2072 editor checks, screenshots in /tmp/gm-e2e-*.png
|
||||
GM_STEPS=20 node tools/e2e.mjs # or 1-12, or 6,20 — a subset while working on one step
|
||||
./e2e.sh start [GM_STEPS] # the same run detached, see below
|
||||
./e2e.sh status # live progress, or the last run's result
|
||||
|
||||
Reference in new issue
Block a user