180 lines
9.1 KiB
Markdown
180 lines
9.1 KiB
Markdown
# Ref2VA Segment Format (generation from vague prompt)
|
||
|
||
This reference defines the exact format of ONE segment prompt produced by this skill.
|
||
It adapts the full-reference mode output format (`segment-guide.md` in this repo) to
|
||
the skill's use case: generating a ~10-second segment from scratch, where the only
|
||
reference assets are character images (user-provided or generated `Image-N.json`
|
||
sheets) and, optionally, one reused song audio file.
|
||
|
||
Write everything in English. Timestamps are relative to the segment start
|
||
(`00:00.000` = first frame of this segment), because the video creator generates each
|
||
segment independently and splits the total length evenly.
|
||
|
||
## 1. Segment skeleton
|
||
|
||
Each segment is one plain-text document with six sections in this exact order. Blank
|
||
line between sections; one line per subject / retention entry:
|
||
|
||
```text
|
||
subject_definitions:
|
||
<Subject 1> is the <role/appearance summary> in <Picture 1>, with <locked traits>.
|
||
<Subject 2> is the <role/appearance summary> in <Picture 2>, with <locked traits>.
|
||
<Audio 1> is the user-supplied song file "<filename>", reused as the target segment's
|
||
complete final audio track; <Subject N> performs its vocals.
|
||
|
||
summary:
|
||
[reference generation] <one short paragraph naming subjects, shot flow, and roles of
|
||
the reference assets>
|
||
|
||
retention_analysis:
|
||
<Subject 1> (appears in [Shot 1]): fully_preserved - <what exact traits are retained>.
|
||
<Subject 2> (appears in [Shot 1]): fully_preserved - <what exact traits are retained>.
|
||
<Audio 1>: fully_copy - <Audio 1> is reused 1:1 as the target video's complete final audio track.
|
||
|
||
detailed_description:
|
||
<one or two style sentences>
|
||
[Shot 1] <full description of the shot>
|
||
|
||
overall_soundscape:
|
||
<ambience and physical sounds>
|
||
|
||
non_diegetic_music:
|
||
<audience-only score, or N/A>
|
||
```
|
||
|
||
Omit the `<Audio 1>` lines in all four places when no audio file is referenced.
|
||
|
||
## 2. `subject_definitions`
|
||
|
||
- One line per subject **present in this segment**, plus the audio definition when the
|
||
segment uses the song. Never list subjects that do not appear in this segment.
|
||
- Cite each character's reference image inside its definition: `<Subject 1> is the ...
|
||
in <Picture 1>`. Do NOT create standalone `<Picture N>` entries: the images in this
|
||
skill define characters, they are never frame anchors or storyboards.
|
||
- Definition contents: identity, role, and the locked visible traits (hair, eyes,
|
||
build, wardrobe, signature props) that the video must preserve. Keep the wording
|
||
identical across segments — copy the trait phrases from the story bible.
|
||
- Audio definition: name the file, state the reuse role. Example:
|
||
`<Audio 1> is the user-supplied song file "midnight.mp3", reused as the target
|
||
segment's complete final audio track; <Subject 1> performs its vocals.`
|
||
- No `<Video N>` labels, ever.
|
||
|
||
## 3. `summary`
|
||
|
||
One short English paragraph. It MUST begin with the square-bracketed task-type prefix:
|
||
|
||
- Without audio: `[reference generation]`
|
||
- With the song referenced: `[reference generation + audio reuse]`
|
||
|
||
After the prefix, name the subjects with their labels, the shot flow, and the role of
|
||
each reference asset (`<Picture N>` as appearance/wardrobe source, `<Audio 1>` as the
|
||
reused soundtrack the subject performs to). Do not introduce new labels.
|
||
|
||
## 4. `retention_analysis`
|
||
|
||
One line per subject present in the segment (plus the audio line when used). Markers
|
||
are fixed English values: `fully_preserved`, `partially_preserved`, `attribute_transfer`,
|
||
`weak_reference`. This skill's default is `fully_preserved` for every subject — segments
|
||
flow into one continuous scene, so nothing about a character's locked look may drift:
|
||
|
||
```text
|
||
<Subject 1> (appears in [Shot 1]): fully_preserved - the matte-white EVA suit with
|
||
orange stripes, short black hair, and the wrist scanner are retained.
|
||
```
|
||
|
||
Audio line for song segments (exact marker and phrasing):
|
||
|
||
```text
|
||
<Audio 1>: fully_copy - <Audio 1> is reused 1:1 as the target video's complete final audio track.
|
||
```
|
||
|
||
## 5. `detailed_description`
|
||
|
||
The main body. Structure:
|
||
|
||
1. **Style opening** — one or two English sentences BEFORE `[Shot 1]` fixing style,
|
||
lighting, and palette (from the story bible).
|
||
2. **Shots** — `[Shot 1]` opens the segment and carries NO timestamp. Only split into
|
||
`[Shot N] At MM:SS.mmm, ...` (N ≥ 2) when the segment needs a time-synced change
|
||
(beat drop, lighting change, arrival event). Timestamps are relative to the segment
|
||
start. Default is a single continuous `[Shot 1]`.
|
||
|
||
Inside the shot description:
|
||
|
||
- At the first appearance of each `<Subject N>`, weave in the locked traits, frame
|
||
position, and current action. Reuse the same label later without redefining it.
|
||
- Describe composition, environment and lighting, action and state changes, camera
|
||
movement (type, amplitude, speed) as natural English.
|
||
- Target roughly 350–500 words. A single shot does not justify a thin description;
|
||
distribute detail across the full ~10 seconds. Never reduce it to a plot summary.
|
||
- **Singing:** at the point(s) where the character is visibly singing, include the
|
||
exact phrase `<Subject N> lips are synced to the audio in <Audio 1>` (substitute the
|
||
real subject number; no possessive apostrophe). Example:
|
||
`<Subject 1> lips are synced to the audio in <Audio 1> through the final chorus as she ...`
|
||
- **Never quote lyrics** — no `<d>` blocks for the song, no lyric words. The vocal
|
||
content lives in the reused audio only.
|
||
- Ordinary spoken dialogue (non-song) is allowed when the scene implies it: give the
|
||
speaker a stable `(S1)`, `(S2)` ID and write `<d>[English] ...</d>` per the source
|
||
guide. This does not apply to the song.
|
||
- No `<Video N>` labels.
|
||
|
||
## 6. `overall_soundscape` and `non_diegetic_music`
|
||
|
||
Both are written heading-on-its-own-line, content on the next line.
|
||
|
||
**Singing segments (audio file referenced) — use this EXACT template, verbatim, with
|
||
only the subject number substituted. One line per singing subject:**
|
||
|
||
```text
|
||
overall_soundscape:
|
||
<Subject N> is Singing the words of <Audio 1>
|
||
|
||
non_diegetic_music:
|
||
N/A
|
||
```
|
||
|
||
**Non-singing segments** — describe ambience and physical sounds diegetically:
|
||
|
||
```text
|
||
overall_soundscape:
|
||
Dry desert wind with fine grit against the helmet shell, low boots pressing into
|
||
cooling sand, and a faint sub-bass hum rising from the monolith.
|
||
|
||
non_diegetic_music:
|
||
N/A
|
||
```
|
||
|
||
`non_diegetic_music` is `N/A` when the scene lives on diegetic sound alone; when the
|
||
vague prompt calls for a score (epic, tense, tender...), describe an original
|
||
audience-only score with instrumentation, tempo, and dynamics. Never put the reused
|
||
song here — the song is diegetic performance audio covered by the template above.
|
||
Never repeat lyrics in either section.
|
||
|
||
## 7. Worked micro-example
|
||
|
||
One 10-second segment of a two-character desert scene (full end-to-end example in
|
||
`references/worked-example.md`):
|
||
|
||
```text
|
||
subject_definitions:
|
||
<Subject 1> is the astronaut in <Picture 1>, a woman in her thirties with short black hair, wearing a matte-white EVA suit with orange stripe accents and a wrist-mounted scanner.
|
||
<Subject 2> is the scout robot in <Picture 2>, a boxy machine on three wheels with a pale-grey shell and a single round blue lens eye.
|
||
|
||
summary:
|
||
[reference generation] The target video shows <Subject 1> and <Subject 2> cresting a red dune at dusk and stopping before a monolith of black glass. <Subject 1>'s appearance and suit come from <Picture 1> and <Subject 2>'s shell and lens come from <Picture 2>.
|
||
|
||
retention_analysis:
|
||
<Subject 1> (appears in [Shot 1]): fully_preserved - the matte-white EVA suit with orange stripe accents, short black hair, and the wrist-mounted scanner are retained.
|
||
<Subject 2> (appears in [Shot 1]): fully_preserved - the boxy pale-grey shell, three wheels, and the single round blue lens eye are retained.
|
||
|
||
detailed_description:
|
||
The target video uses a cinematic sci-fi realism style with low dusk light, long violet shadows, and a muted red-and-violet palette.
|
||
[Shot 1] A wide establishing shot holds on a red desert at dusk, the sky a deep gradient of burnt orange fading into indigo. <Subject 1>, the astronaut in the matte-white EVA suit with orange stripe accents, short black hair under a clear dome, and a wrist-mounted scanner on her left forearm, crests the dune on the left of frame, boots sinking into rippled sand with each step. <Subject 2>, the boxy scout robot with a pale-grey shell and a single round blue lens eye, wheels up the slope on her right, treads leaving a neat triple track. The camera tracks laterally with their descent at walking pace, then slows as the pair reaches the flat basin. Ahead, a monolith of black glass stands three stories tall, its surface drinking the last light; a faint sub-bass hum pushes grit outward from its base in slow pulses. <Subject 1> stops two meters short, raises the wrist scanner, and the device's amber readout reflects in her dome. <Subject 2>'s blue lens dilates with a soft mechanical whir, and its head tilts toward the monolith. The camera pushes in slowly past her shoulder, ending on the monolith's mirrored face holding both their reflections as the hum deepens.
|
||
|
||
overall_soundscape:
|
||
Dry desert wind with fine grit against the helmet shell, the crunch of boots in cooling sand, the soft whir of the robot's wheels, and the monolith's low pulsing hum.
|
||
|
||
non_diegetic_music:
|
||
N/A
|
||
```
|