Drag clips of videos and sounds straight onto a track (#425)

A clip of a video could not be laid on a composition track at all, and
neither could a clip of a sound, although #418 and the wiki said so.
The only place a drag starts is the asset panel, and it listed assets,
not clips. The texture label of an item playing a video clip was dead
too, because the reference lookup did not know video clips.

- The asset panel gives a sound or video with clips a chevron of its
  own. It unfolds one row per clip and starts closed.
- A clip row drags like an asset. One click shows the clip in the
  inspector without leaving the tab (`EditorState.glance`). A double
  click opens its sound or video with the clip picked.
- A video clip dropped on a composition track becomes an item as long
  as the clip. It wears the sprite of its whole video, so a composition
  keeps one sprite per video, and the clip is a texture override on
  the object. Sprite, item and override are one undo step
  (`droppedPicture`).
- A sound clip becomes an audio item as long as the clip. The song tab
  takes sound clips on the same way, through `findSound`.
- `referencedTarget` lets a reference label link to the clip itself.
  One click pings the clip row, a double click opens the asset with the
  clip picked. The Plays row of an audio item reads "Sound / Clip".
- `clipOwner` in `sound.ts` finds the sound or video a clip id belongs
  to. `activateAssetOrClip` uses it for the double clicks.

Found on the way: the Blip of the e2e seed is only 0.25 s long, so a
clip of 0.2 to 0.7 s on it is cut to one frame. 69s takes the Drone
(0.75 s) instead.

Tests:
- tsc, eslint, Prettier clean
- Unit on stein2 (composition, references, sound, video, editor):
  94 files, 1238 tests green
- e2e 69 on melektaus-atx: ALL PASSED, 25 checks, 2 of them new (69s)
- Gate on melektaus-atx with the fix for 68k on top: CHECK EXIT 0,
  313 files, 4841 tests
- Counter checks, all red: no clip rows in e2e 69 (no drag starts),
  no clip override in the unit test, the link aimed at the asset
  instead of the clip
- Not covered by its own e2e check: sound clips on the song tab

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
adminandClaude Opus 5.5 committed 2026-10-15 08:01:04 +02:00
1 parent 0701fe3856
commit 406839baea
20 files changed
+469 -94

No files matched your search

+37 -7
View File
@@ -12,12 +12,14 @@ import { generatesPictures } from '../model/generated';
import { pixelArtWornId } from '../model/pixelart';
import type { PropertyBag } from '../model/Properties';
import { findAsset, objectHolders, type Project } from '../model/Project';
import { findVideo } from '../model/video';
import { spriteFromTexture } from '../properties/aspect';
import { type Removed, removeWhere, restoreAt } from '../util/list';
import type { Command } from './Command';
import { addCompositionItem } from './compositionCommands';
import { oneStep } from './CompoundCommand';
import { nextAssetName } from './naming';
import { SetOverrideCommand } from './sceneObjectCommands';
/**
* What a sprite made of `asset` wears: a texture, a drawn picture (#185) or a video; a sequence of
@@ -30,6 +32,29 @@ export function wornPicture(asset: Asset): string | null {
: null;
}
/** What an item dropped on a track wears: the picture its sprite is made of, and a clip of it (#425). */
export interface DroppedPicture {
picture: string;
/** The clip of a video the item plays, as an override on its object; null for the whole picture. */
clip: string | null;
/** The name a new sprite for the picture gets. */
name: string;
}
/**
* What an asset or a clip of a video (#425) dragged onto a track wears. A clip shares the sprite of
* its video, so one composition keeps one sprite per video. Null for anything that is no picture.
*/
export function droppedPicture(project: Project, id: string): DroppedPicture | null {
const asset = findAsset(project, id);
if (asset) {
const picture = wornPicture(asset);
return picture === null ? null : { picture, clip: null, name: asset.name };
}
const video = findVideo(project, id);
return video?.clip ? { picture: video.asset.id, clip: video.clip.id, name: video.asset.name } : null;
}
/** The sprite the composition already keeps for this picture: one serves every item wearing it. */
function keptSprite(project: Project, compId: string, textureId: string): SpriteAsset | undefined {
return project.assets.find(
@@ -49,20 +74,25 @@ function newCompositionSprite(
}
/**
* A texture, a drawn picture or a video laid on a track: an item wearing the sprite the composition
* keeps for that picture, made first when it has none yet, in one undo step.
* A texture, a drawn picture, a video or a clip of one laid on a track: an item wearing the sprite the
* composition keeps for that picture, made first when it has none yet, in one undo step. A clip is
* the texture of the item's object.
*/
export function addPictureItem(
project: Project,
compId: string,
textureId: string,
name: string,
dropped: DroppedPicture,
item: Omit<CompositionItem, 'id'>,
): { command: Command; objectId: string } {
const kept = keptSprite(project, compId, textureId);
const sprite = kept ?? newCompositionSprite(project, compId, textureId, name);
const kept = keptSprite(project, compId, dropped.picture);
const sprite = kept ?? newCompositionSprite(project, compId, dropped.picture, dropped.name);
const { command, objectId } = addCompositionItem(project, compId, sprite.id, item);
return { command: kept ? command : oneStep('Add item', [new AddSpriteCommand(sprite), command]), objectId };
const steps = [
...(kept ? [] : [new AddSpriteCommand(sprite)]),
command,
...(dropped.clip ? [new SetOverrideCommand(compId, objectId, 'texture', dropped.clip)] : []),
];
return { command: steps.length === 1 ? command : oneStep('Add item', steps), objectId };
}
class AddSpriteCommand implements Command {
+8 -14
View File
@@ -5,13 +5,7 @@
* into.
*/
import { videoLoopsOf } from '../properties/videos';
import type {
Asset,
CompositionAsset,
CompositionAudioKind,
CompositionClip,
CompositionItem,
} from './Asset';
import type { CompositionAsset, CompositionAudioKind, CompositionClip, CompositionItem } from './Asset';
import {
audioRoundSeconds,
audioSeconds,
@@ -407,11 +401,11 @@ export function audioKindOf(project: Project, assetId: string): CompositionAudio
}
/**
* How long an item of `asset` is when it is dropped in: as long as the video it is (#419) or its
* sprite wears, or what it plays as an audio item (#418); else 3 s.
* How long an item of the asset `id` names is when it is dropped in: as long as the video or clip of
* one it is (#419, #425) or its sprite wears, or what it plays as an audio item (#418); else 3 s.
*/
export function droppedLength(project: Project, comp: CompositionAsset, asset: Asset): number {
const kind = audioKindOf(project, asset.id);
export function droppedLength(project: Project, comp: CompositionAsset, id: string): number {
const kind = audioKindOf(project, id);
if (kind)
return fittedLength(project, comp, {
id: '',
@@ -419,10 +413,10 @@ export function droppedLength(project: Project, comp: CompositionAsset, asset: A
start: 0,
length: 1,
offset: 0,
audio: { kind, asset: asset.id, volume: 0, bus: '' },
audio: { kind, asset: id, volume: 0, bus: '' },
});
const texture =
asset.type === 'sprite' ? asset.props.texture : asset.type === 'video' ? asset.id : undefined;
const asset = findAsset(project, id);
const texture = asset?.type === 'sprite' ? asset.props.texture : id;
const video = typeof texture === 'string' ? findVideo(project, texture) : null;
const fps = compositionFps(comp);
const seconds = video ? videoLengthOf(video) : 0;
+14 -1
View File
@@ -55,6 +55,14 @@ export function assetNamedBy(assets: readonly Asset[], id: string): Asset | unde
);
}
/** The sound or video `id` is a clip of; undefined for an asset's own id or anything else. */
export function clipOwner(project: Project, id: string): ClipOwner | undefined {
if (id === '' || findAsset(project, id)) return undefined;
return project.assets.find(
(a): a is ClipOwner => hasClips(a) && clipsOf<{ id: string }>(a).some((c) => c.id === id),
);
}
/**
* Gives the clips of a copy fresh ids, since a clip's id is named across the document, and notes
* each old id with its new one in `ids` so the places that named a clip can follow.
@@ -70,7 +78,12 @@ export function renewClipIds(copy: Asset, ids: Map<string, string>): void {
/** "Kick" for the sound, "Kick / Clip 1" for a clip of it. */
export function soundRefName(ref: SoundRef, soundName = ref.asset.name): string {
return ref.clip ? `${soundName}${CLIP_SEPARATOR}${ref.clip.name}` : soundName;
return ref.clip ? clipRefName(soundName, ref.clip.name) : soundName;
}
/** "Kick / Clip 1": a clip of a sound or a video by name. */
export function clipRefName(assetName: string, clipName: string): string {
return `${assetName}${CLIP_SEPARATOR}${clipName}`;
}
/** "Clip 1", 2, 3 …: the first number none of the asset's clips has. */
+19 -3
View File
@@ -1,15 +1,16 @@
import { type PropertyBag, type PropertyValue, textOf } from '../model/Properties';
import { findPixelFrame } from '../model/pixelart';
import { findAsset, type Project } from '../model/Project';
import { findSound } from '../model/sound';
import { clipOwner, findSound } from '../model/sound';
import { findFrame } from '../model/texture';
import { findVideo } from '../model/video';
import { isKindWith } from './kinds';
import type { PropertyDef } from './PropertySchema';
/**
* The asset a reference field points at, null when it names nothing or no longer resolves. A frame
* names the texture or pixel art it was cut from – that is the asset one can open (#221) – and a
* clip its sound.
* clip its sound or video.
*/
export function referencedAssetId(project: Project, def: PropertyDef, value: PropertyValue): string | null {
if (!isKindWith('reference', def.kind)) return null;
@@ -18,7 +19,22 @@ export function referencedAssetId(project: Project, def: PropertyDef, value: Pro
if (findAsset(project, id)) return id;
// A clip (#357) names the sound it was cut from, as a frame does.
if (def.kind !== 'texture') return findSound(project, id)?.asset.id ?? null;
return findFrame(project, id)?.asset.id ?? findPixelFrame(project, id)?.asset.id ?? null;
return (
findFrame(project, id)?.asset.id ??
findPixelFrame(project, id)?.asset.id ??
findVideo(project, id)?.asset.id ??
null
);
}
/**
* What the label of a reference row links to: the asset it names, or the clip of a sound or a video
* itself (#425) – a clip has a row of its own in the asset panel and is picked in its asset's tab.
*/
export function referencedTarget(project: Project, def: PropertyDef, value: PropertyValue): string | null {
const assetId = referencedAssetId(project, def, value);
const id = textOf(value);
return assetId && clipOwner(project, id)?.id === assetId ? id : assetId;
}
export interface Reference {