feuerwerk

0 Fork 0

Table of Contents

The asset

Audio items
The composition's bus
Fades
Sprites of its own

The rules
The Composition tab
In a game
Not yet

A composition lays objects out in time (#417): a cut of video clips, a cutscene, a title that comes in and goes again. It has tracks, and on each track items and animation clips. An item is one object that lives from its start to its end, or a sound, a pattern or a song heard while it lasts (#418); a clip plays an animation on the items of its track while it runs. Placed in a scene, a composition plays when a behavior says so.

The asset

A composition holds its objects like a container holds its template: each item is one object, with its asset, its overrides and, for a container, its content and the edits to it. So everything that works on the objects of a scene works here — the inspector, overrides, Apply and Reset, the references to an object, deleting an asset (its items go with it, one undo brings them back) and duplicating the composition (the copy gets objects of its own). A composition is the third holder of objects beside scenes and containers (ObjectHolder in src/core/model/Project.ts).

Row
Frames per second	1 to 120, 60 by default, whole numbers, so a cut lands on a frame. Item starts and lengths are stored in these frames
Loop	off by default: after its last item it is over. On, it starts over at frame 0 and never finishes

What is stored (CompositionAsset in src/core/model/Asset.ts): tracks (id, name), items (id = the object's id, track, start and length in frames, offset in seconds into its video, fitMedia, for an audio item audio, and its fades fadeIn, fadeOut, curveIn, curveOut) and clips (id, track, animation, start in frames). An item whose object is gone stays in the list, so an undo finds it again, and plays as nothing.

Audio items

A sound, a clip of a sound, a pattern or a song dragged onto a track is an audio item (#418). It has no object; audio says what it plays (kind, asset), its volume in dB (-60 to +12) and its bus. It shares a track with object items and may overlap them.

It plays for as long as it lasts, and stops at its end. Longer than what it plays, it goes round and round: a sound and a pattern from their start, a song round the loop of its ruler, as Play song with Loop does. The offset is how far into it the item begins, in seconds; a trim of the left edge moves it. Below 0 the item is silent until then.
A pattern or a song keeps its own tempo inside the item. A Delay synced to beats counts by it.
The bus. To composition (what a new item starts with) sends it into the composition's bus. From asset sends it where the Master routing sends its sound, pattern or song; an item laid before there was a composition's bus reads so. Master or a bus sends it there instead. Each item is heard on a bus of its own at its volume, through the sound FX of its pattern or song; a song's tracks are buses of that item, so two items of one song do not share them.
Animation clips do not belong to an audio item: there is nothing on it they could move.
What is gone plays nothing. The item stays, drawn red, named (missing).

The composition's bus

Every composition is a bus of the mix and has a node in the Master routing (#418), named after it, so a cut is mixed as one: linked into a bus, its sound goes there; at its volume, through that bus' sound FX. A new composition goes where most other compositions go, or to the master where none is linked yet, never by where the sounds go.

Audio items go into it where their Bus is To composition.
Videos of its items go into it where their sprite's Video bus is empty: there it reads To composition. From asset sends one along the links of its video, Master or a bus there instead (see Videos). That holds for a sprite below the item too.
In the tab all of it is heard straight on the master, as in every tab; the bus counts in a game.

Fades

An item with a sound, an audio item or an item whose sprite wears a video, can fade in at its start and fade out at its end. Each fade is a number of frames and a curve, the curves of a sound's clip: Linear (the default), Equal power, S-curve, Exponential, Logarithmic. The fades hang on the edges of the item: trimmed or resized, the item takes them along. A fade is never longer than the item, and the fade out never longer than what the fade in leaves (itemFades in src/core/model/composition.ts).

An audio item fades on its own bus. The fades are ramps of that bus' level on the audio clock, planned by the scheduler with the notes (SequenceLevel in src/core/music/sequence.ts), so they land to the sample. Started in the middle of a fade, it takes the ramp up from there.
A video item fades its sprite's Video volume (see Videos): every frame the composition sets it to the sprite's own value times the fade, and the player glides to it over a few milliseconds, so the steps of the frames do not click. Video sprites below the item fade along.

The sequences are made in src/core/music/compositionAudio.ts: everything in milliseconds of the composition, one sequence for each place of the mix the items go to. CompositionSound (src/runtime/compositionSound.ts) plays them through the scheduler, in the tab and in a game.

Sprites of its own

A picture or a video dragged straight onto a track (#419) needs a sprite to wear it, and a composition keeps that sprite to itself, the way a song keeps its takes (see Songs): a real sprite asset whose owner is the composition. Everything that finds an asset by its id finds it, but the asset panel and the pickers leave it out. The inspector names it with a dot in front, as the Song tab does with a song's own sounds.

One sprite per picture. A second drop of the same video wears the same sprite. What one item should have differently goes on its object as an override.
It goes with the last item wearing it, in the same undo step. Deleting the composition takes its sprites along, duplicating it gives the copy sprites of its own.
Pasted into a scene or a container, the sprite turns into a project asset (named apart from the others), so deleting the composition cannot take the pasted object along. Pasted into another composition, that one gets a copy of its own. A cut keeps the sprite in the copy, so a paste after it wears it again.
Make project asset on the item turns the sprite into an ordinary sprite in the asset list.

This is held after every edit (KeepCompositionSpritesCommand in src/core/commands/compositionSprites.ts), so it is taken back and done again with it.

The rules

What lives at a frame is one plain function, compositionAt (src/core/model/composition.ts), read by the tab, the scene view and the running game alike:

The upper track lies in front. On one track the item that starts later lies in front.
Items may overlap, on one track as on two. Nothing is cut or moved: both live at once.
A clip plays on every item of its own track it overlaps, two at once in an overlap. Its rows find their targets in the item's object the way an animation does: by the template path, else by the names. A clip is one pass of its animation, whatever the animation's play mode says. After its end it holds its last value until the item ends.
Clips on one track do not overlap. A clip dragged towards another stops at it.
A clip belongs to the item it begins on: the item lying on its track at its first frame, of two there the one in front. A clip that begins before an item and only reaches into it belongs to the item before. A clip may be longer than its item. It goes where its item goes: dragged, copied, removed (see below).
An item with a video — a sprite wearing a video or one of its clips — runs driven (see Videos): its Video time is the item's offset plus the time since the item began. Video sprites below it run along unless they were set to Driven by hand. Before the video begins (a negative offset) it shows the first picture, past its end it goes round with the video's Loop or holds the last one.

The Composition tab

Each composition opens as a document in a tab of its own. On top the picture at the playhead, below it the timeline: a ruler in seconds, the track heads on the left, the items as bars and the clips as narrower bars above them. The header bar has + Track, Play and the size ("3 tracks · 5 items · 12.5 s · 60 fps"); Fit frames picture and timeline.

Dragging an asset from the panel onto a track makes an item there, picked. Anything that can be placed in a scene can go, a composition excepted: one inside another is refused while it hovers. It lands at the frame under the pointer, over an item there too, and is 3 s long, or as long as the video the sprite wears.
Dragging a texture, vector art, pixel art or a video onto a track (#419) makes an item the same way, wearing a sprite the composition keeps for that picture (see Sprites of its own below). A video's item is as long as the video.
Dragging a clip of a video onto a track (#425) makes an item as long as the clip. It wears the same sprite as its whole video, so a composition keeps one sprite per video, and the clip is the Texture of the item's own object. The clip rows sit under their video in the asset panel (see Concepts). A clip can also be chosen afterwards, on the Texture row.
Dragging a frame of a texture or of pixel art onto a track (#430) works the same way: the item wears the sprite of the whole picture, the frame is the Texture of its object, and it lasts 3 s. The frame rows sit under their texture or pixel art in the asset panel.
Dragging a sound, a clip of one, a pattern or a song onto a track (#418) makes an audio item there, as long as one pass of it, picked; a song's first pass ends where its loop does. A clip of a sound comes from its row under the sound in the asset panel (#425). A sound has to be loaded to know its length; the drag loads it, and until it is there the item is 3 s long.
Dragging an animation onto a track makes a clip at that frame, if it fits between the clips there.
Dragging an item moves it in time and from track to track, past and over other items; it stops at frame 0. Several picked items move together. Its clips come along, in the same undo step. As clips of a track never overlap, the item stops where one of them would come to lie over another clip. Press Alt once the drag runs and the item goes alone, its clips stay (Alt pressed before is the clip band). Typing a Start in the inspector moves the clips the same way. Dragging an edge leaves them where they are.
The edges. Over an item's left or right edge the cursor turns to ew-resize and the edge lights up, as in the Song tab. Dragging the right edge sets the length. Dragging the left edge of a video item trims it: the start moves and the offset with it, so the picture stays where it is in time. It may go past the video's start (the offset turns negative). An edge snaps to the video's start and end within 6 px. An edge has the first say over a body, so the hidden end of an item below another can still be taken.
The fade handles. An item with a sound has a small square on its top edge at each end, shown while the pointer is over the item or it is picked, and always where it has a fade. Dragging one inwards draws out the fade, back to the edge removes it. The handle comes before the edge, so the top corner fades and the rest of the edge trims. The fade is drawn as the curve its level follows from the bottom of the item to the top, the part not heard shaded.
Where the video lies. On an item with a video, a small triangle in a lower corner says the item cuts into the video on that side: there would be more of it. A line with a triangle, like a loop in the Song tab, marks where the video starts or ends inside the item; with Loop on the video, one line per round. An overlap is darkened, and a line marks the end of the item below. An audio item shows the same marks for what it plays, a line at every round of it; for a song the first at the end of its loop, then one per time round the loop. A sound item draws its waveform; sounds are blue-green, patterns violet, songs plum.
Click picks an item or a clip, Ctrl-click adds or removes one. A click on a track head picks the track, a click in the ruler moves the playhead there, and dragging in it scrubs.
A band dragged from an empty spot picks the items it touches. With Alt held it picks the clips instead and may start anywhere, on an item too. With Ctrl it adds to what is picked. A pick holds items or clips, never both. Items that are objects only are picked as objects, so the hierarchy and the inspector show them; with an audio item among them the pick is the composition's, and the inspector shows the audio item, or the count for several.
The right-click menu on an item: Fit to media, Make project asset (only for an item wearing a sprite of the composition's own), Remove item. On an audio item Open sound (pattern, song) instead of Make project asset. On a clip: Open animation, Remove clip. On a track head or an empty stretch of a track, as in the Song tab: Add track above, Add track below, Move up, Move down, Delete track.
Copy and paste. Ctrl+C on picked items keeps their objects and their timing, Ctrl+V in the tab pastes them as new items in one undo step. With the pointer over the tracks the earliest of them lands at the frame under the pointer, on the track there. Without it they land at the playhead, on the picked track, else on the top track of the picked items. Several items keep their distances and their tracks to each other. The same copy pastes into a scene as plain objects; objects copied from a scene do not paste into a composition, they have no timing. Copied items take their clips along; where one of those would lie over a clip, items and clips move on together to the first frame where they fit. Audio items are copied with them, and alone the same way. Picked clips are copied and pasted the same way, into this composition or another. As clips of a track never overlap, a paste that would lie over a clip moves on, all its clips together, to the first frame where they fit.
Delete removes the picked items with their objects and their clips, the picked clips or the picked track.
Play plays from the playhead at the composition's frame rate, round with its Loop; from the end it starts at 0. The wheel zooms the time, the middle button or Shift-drag pans. Both glide like the other tabs (see Concepts).
Audio items are heard while it plays, from where the playhead stands, straight into the master as in every tab. Their bus is heard in a game. A scrub is silent. While the playhead waits for a video, the sound waits too. An edit while it plays is heard at once. Playing a song or a pattern in its tab stops the composition, and the other way round.

The picture is a scene view whose objects are the items alive at the playhead. A click picks an item or its child, dragging moves the real object. It takes nothing from the asset panel and nothing from Ctrl+V: an object put there would have no item. Driven videos follow the playhead and are heard while it plays, straight into the master as in every tab. In a game they go into the composition's bus, or where their sprite's Video bus says.

The hierarchy lists the items alive at the playhead; re-parenting is off there, because items stay at the top. The inspector of an item shows its object's rows and a block TIMING: Start, Length and, for a video item, Offset, all in seconds, and Fit to media. An audio item shows SOUND ITEM (PATTERN, SONG): what it plays, the same timing rows with Offset, Volume in dB and Bus (To composition, From asset, Master or a bus). An item with a sound adds Fade in and Fade out in seconds and a curve for each to its TIMING. The sprite of a video item reads Driven (by composition) in its Video clock row, whatever it holds, and has no Video time row: the composition sets that. Its Video volume stays, the fades scale it. A track shows its name; a clip its animation and its start, with the length of one pass.

Fit to media. A new item's length follows its video: when an override gives its sprite another video or clip, the length follows in the same undo step (FitMediaCommand). Dragging an edge or typing a length makes the length the item's own from then on, and Fit to media hands it back to the video. The button is greyed out while the item already fits.

In a game

A composition is placed in a scene like a container: dragged in, it has a symbol, the transform rows and a render layer. In the editor the placement shows the picture of frame 0; a click on it picks the placement, its items cannot be picked one by one there.

Node
Play composition	object (the placement), restart (off): plays it from frame 0. One already playing plays on; with restart it starts over
Stop composition	object: the objects of its items are destroyed on the next frame. It does not count as finished
On composition finished	runs once when a composition at the owner or below it plays past its last item, and when one this behavior played itself does. Composition (None: every one), output object

Before Play composition nothing of it shows. Frame 0 is the editor's picture only.
Each item becomes an object below the placement when it begins, with the id <placement>~<item> (the content of a container below it <placement>~<item>/<child>), and is destroyed when it ends. Its behaviors start when it is made, so On start and the rest run as for a spawned object. The upper track lies in front, whatever began first.
After the last item it is over: nothing of it is left in the scene and On composition finished fires. With Loop it starts over instead; an item that lives across the jump back to 0 stays and is not made anew.
A pause holds it, and so does switching the placement off. It plays in game time.
Videos are opened ahead. The video of an item that begins within the next second is opened before, held at its first picture, unseen, so a cut from one video to the next shows no empty frame. With Loop the items at the start are opened near the end. The tab does the same.
It waits for its videos. While the game view does not show a video of one of its live items because the picture is not there yet — the player still loads, or, for a clip that starts later, still seeks to it — the composition's clock stands, for at most 3 s at a time. The rest of the game runs on. The Composition tab holds its playhead the same way. A recording takes no frame meanwhile (see Videos).
Its audio items are heard while its clock runs (#418), on the bus each one names. They are planned on the audio clock like the notes of a song, for the moment the clock reaches them, so sound and picture keep together. While the clock stands (waiting for a video, the placement switched off, the game paused), they are silent, and start again where it stands. Stop composition and its end stop them.
Sound and picture are held together. The sound runs on the audio clock, the picture on the frames, and a frame counts 0.1 s at most: after a hitch the sound would be ahead. Every frame the composition compares the two, and where they lie more than 40 ms apart, it starts the sound again where the picture is (CompositionSound.follow), with the short fade a stop has. A note that rings then starts anew. The tab does the same. At a time scale other than 1 that happens again and again: the sound keeps to the picture, but in pieces.
The editor run plays none (Run in editor): it writes into the saved project, and the items would stay there. Play composition says so in the log there.

The system is src/runtime/systems/CompositionSystem.ts, ticked before the animations and the behaviors. It spawns through World.place and orders through World.arrange, and its clips write with the same row binding the AnimationSystem uses.

Not yet

A baked song plays its notes inside a composition, not its bake.
A composition inside a composition, and Sort with parent for a placement.

Using the editor

Working on the code