Add Segment-img-N frame prompts and opt-in FL2VA first-last mode (ref2va stays default)
This commit is contained in:
parent
9e931a5c7c
commit
45b4c1c211
29
README.md
29
README.md
@ -3,11 +3,15 @@
|
||||
An agent skill that turns a vague prompt plus a total video length (seconds) into a
|
||||
ready-to-run generation package:
|
||||
|
||||
- `segments.txt` — N segment prompts in full-reference (ref2va) format, joined by `|`.
|
||||
The video creator node splits the total length evenly across segments
|
||||
(`N = max(1, round(total_seconds / 10))`) and generates them one by one.
|
||||
- `segments.txt` — N segment prompts, joined by `|`. The video creator node splits the
|
||||
total length evenly across segments (`N = max(1, round(total_seconds / 10))`) and
|
||||
generates them one by one. Default format is full-reference (ref2va); FL2VA
|
||||
(first–last frame) is an opt-in alternative.
|
||||
- `Image-N.json` — one 16:9 character-sheet text-to-image prompt per character that
|
||||
has no user-provided image reference, consumed one at a time by a ComfyUI workflow.
|
||||
- `Segment-img-N.json` — one 16:9 first-frame image prompt per segment, rendered by
|
||||
the image engine with the character sheets as reference inputs; FL2VA mode adds
|
||||
`Segment-img-N-last.json` per segment for the last frame.
|
||||
- `image-map.txt` — manifest mapping each `<Subject N>` to its image file
|
||||
(user-provided filename or generated `Image-N.json`).
|
||||
|
||||
@ -15,10 +19,12 @@ ready-to-run generation package:
|
||||
|
||||
| Path | Purpose |
|
||||
| --- | --- |
|
||||
| `SKILL.md` | Skill entry point: triggers, pipeline, output contract, validation checklist |
|
||||
| `references/ref2va-segment-format.md` | Authoritative per-segment format (sections, labels, shots, audio template) |
|
||||
| `references/t2i-character-sheet.md` | Character-sheet caption rules and the two-key JSON contract |
|
||||
| `references/worked-example.md` | Complete end-to-end example, including the audio (singing) variant |
|
||||
| `SKILL.md` | Skill entry point: triggers, mode selection, pipeline, output contract, validation checklist |
|
||||
| `references/ref2va-segment-format.md` | Authoritative per-segment format for the default mode (sections, labels, shots, audio template) |
|
||||
| `references/fl2va-segment-format.md` | The opt-in FL2VA first–last frame video prompt format |
|
||||
| `references/t2i-character-sheet.md` | Character-sheet caption rules (four-panel photorealistic turnaround) and the two-key JSON contract |
|
||||
| `references/frame-image-prompts.md` | Per-segment first/last frame image prompt rules (`Segment-img-N.json`) |
|
||||
| `references/worked-example.md` | Complete end-to-end example, including the audio (singing) and FL2VA variants |
|
||||
| `scripts/validate_outputs.py` | Optional validator the agent can run on its outputs |
|
||||
| `segment-guide.md`, `t2i-prompt-rules/` | Source rule documents the skill was adapted from |
|
||||
|
||||
@ -27,8 +33,11 @@ ready-to-run generation package:
|
||||
1. Pull this repo into your agent's skills folder so that `SKILL.md` is discoverable.
|
||||
2. Give the agent a vague scene prompt and the total video length in seconds, plus
|
||||
optionally: song lyrics, an audio file mention (e.g. "use midnight.mp3"),
|
||||
per-character image references, and style hints.
|
||||
per-character image references, style hints, and — only if wanted — an explicit
|
||||
request for FL2VA / first-last frame mode.
|
||||
3. The agent writes `segments.txt`, `Image-N.json` for characters lacking references,
|
||||
and `image-map.txt` into the project folder.
|
||||
`Segment-img-N.json` (and `Segment-img-N-last.json` in FL2VA mode), and
|
||||
`image-map.txt` into the project folder.
|
||||
4. Verify before use: `python3 scripts/validate_outputs.py --segments segments.txt
|
||||
--expected-n <N> [--dir .] [--lyrics-file lyrics.txt]`
|
||||
--expected-n <N> [--dir .] [--lyrics-file lyrics.txt]` — add
|
||||
`--mode fl2va --total-seconds <seconds>` for FL2VA runs.
|
||||
|
||||
113
SKILL.md
113
SKILL.md
@ -1,6 +1,6 @@
|
||||
---
|
||||
name: mm-prompter
|
||||
description: Writes multi-segment ref2va video prompts plus T2I character-sheet prompts from a vague scene idea and a total video length in seconds. Use when the user gives a rough video/scene concept (optionally with song lyrics or an audio file name) and wants numbered video-generation segments joined by "|", character sheets as Image-N.json for characters that lack an image reference, and an image-map.txt manifest.
|
||||
description: Writes multi-segment ref2va video prompts plus T2I character-sheet prompts and per-segment first-frame image prompts from a vague scene idea and a total video length in seconds. Use when the user gives a rough video/scene concept (optionally with song lyrics or an audio file name) and wants numbered video-generation segments joined by "|", character sheets as Image-N.json for characters that lack an image reference, Segment-img-N.json frame prompts, and an image-map.txt manifest. Optional opt-in FL2VA mode adds last-frame prompts and first-last video prompts.
|
||||
---
|
||||
|
||||
# MM Prompt Creator
|
||||
@ -8,16 +8,34 @@ description: Writes multi-segment ref2va video prompts plus T2I character-sheet
|
||||
Turn one vague prompt and one number (total seconds) into a complete, ready-to-run
|
||||
generation package:
|
||||
|
||||
1. `segments.txt` — N segment prompts in full-reference (ref2va) format, joined by `|`.
|
||||
The downstream video creator node splits the total length evenly across segments and
|
||||
generates them one by one.
|
||||
1. `segments.txt` — N segment prompts, joined by `|`. The downstream video creator
|
||||
node splits the total length evenly across segments and generates them one by one.
|
||||
Default format is full-reference (ref2va); FL2VA (first–last frame) is an opt-in
|
||||
alternative (see Mode selection).
|
||||
2. `Image-N.json` — one 16:9 character-sheet text-to-image prompt per character that has
|
||||
no user-provided image reference. Consumed one at a time by a ComfyUI workflow.
|
||||
3. `image-map.txt` — manifest that says which `<Subject N>` maps to which image file.
|
||||
3. `Segment-img-N.json` — one 16:9 first-frame image prompt per segment (both modes);
|
||||
the image engine renders it using the character sheets as reference images. In
|
||||
FL2VA mode also `Segment-img-N-last.json` per segment for the last frame.
|
||||
4. `image-map.txt` — manifest that says which `<Subject N>` maps to which image file.
|
||||
|
||||
Read `references/ref2va-segment-format.md` before drafting the first segment, and
|
||||
`references/t2i-character-sheet.md` before writing the first JSON. A complete end-to-end
|
||||
example (including the audio variant) is in `references/worked-example.md`.
|
||||
Read `references/ref2va-segment-format.md` before drafting the first segment and
|
||||
`references/t2i-character-sheet.md` before writing the first JSON. When the run uses
|
||||
FL2VA mode, also read `references/fl2va-segment-format.md`; always read
|
||||
`references/frame-image-prompts.md` before writing frame files. A complete end-to-end
|
||||
example (including the audio and FL2VA variants) is in `references/worked-example.md`.
|
||||
|
||||
## Mode selection
|
||||
|
||||
- **ref2va (default).** Every run uses the six-section ref2va segment format unless
|
||||
the user explicitly opts into first–last frames.
|
||||
- **FL2VA (opt-in).** Switch when the user asks for "FL2V", "FL2VA", "first-last",
|
||||
"first and last frame", or "start and end frame" per segment. In FL2VA mode the
|
||||
video side receives ONLY the segment's first-frame image, last-frame image, and the
|
||||
FL2VA-format prompt defined in `references/fl2va-segment-format.md` — never the
|
||||
character sheets. The sheets are still generated: they feed the image side, which
|
||||
renders the first/last frames with consistent characters. If the user's wording is
|
||||
ambiguous, stay in ref2va mode; never mix the two formats in one run.
|
||||
|
||||
## Inputs
|
||||
|
||||
@ -29,6 +47,7 @@ example (including the audio variant) is in `references/worked-example.md`.
|
||||
| Audio file mention | no | e.g. "use midnight.mp3" → enables `<Audio 1>` |
|
||||
| Per-character image references | no | User-supplied files (e.g. `mary.png`) |
|
||||
| Style hints | no | Honor if given; otherwise invent and lock |
|
||||
| Frame mode | no | Default ref2va; FL2VA (first–last) only when explicitly requested |
|
||||
|
||||
Make reasonable creative choices and proceed. Do not block on questions; this skill
|
||||
exists to expand vagueness into concrete, consistent detail.
|
||||
@ -67,20 +86,29 @@ every sheet must use the identical locked values.
|
||||
**Step 4 — Assign numbering.** Subject 1..M by first on-screen appearance across the
|
||||
whole video. Map references: user file (e.g. `mary.png`) or `Image-N.json`.
|
||||
|
||||
**Step 5 — Draft each segment** exactly per `references/ref2va-segment-format.md`:
|
||||
six sections in order, global labels, default one continuous `[Shot 1]` per segment,
|
||||
extra shots only for time-synced changes, timestamps relative to the segment start.
|
||||
When a subject sings to the provided audio, use the exact fixed audio template and the
|
||||
exact lip-sync phrase defined in that reference.
|
||||
**Step 5 — Draft each segment** in the selected mode. ref2va: exactly per
|
||||
`references/ref2va-segment-format.md` — six sections in order, global labels, default
|
||||
one continuous `[Shot 1]` per segment, extra shots only for time-synced changes,
|
||||
timestamps relative to the segment start; when a subject sings to the provided audio,
|
||||
use the exact fixed audio template and the exact lip-sync phrase defined in that
|
||||
reference. FL2VA: exactly per `references/fl2va-segment-format.md` — instruction line
|
||||
+ three fields, single-shot interpolation arc from first-frame state to last-frame
|
||||
state, no reference labels.
|
||||
|
||||
**Step 6 — Write the character sheets** per `references/t2i-character-sheet.md`, one
|
||||
**Step 6 — Write the frame image prompts** per `references/frame-image-prompts.md`:
|
||||
one `Segment-img-N.json` per segment (N = 1..n) describing that segment's opening
|
||||
state, and — FL2VA mode only — one `Segment-img-N-last.json` per segment describing
|
||||
its final state. Captions reuse the locked story-bible values verbatim and anchor
|
||||
each character to the supplied reference images.
|
||||
|
||||
**Step 7 — Write the character sheets** per `references/t2i-character-sheet.md`, one
|
||||
`Image-N.json` per subject lacking a user reference. Every caption describes the
|
||||
fixed four-panel photorealistic turnaround (face close-up, front, side facing 90°
|
||||
right, back) on a white background with no text, no labels, and no borders — studio
|
||||
photography of a real person, never a cartoon or designed card — and must reuse the
|
||||
locked story-bible values verbatim.
|
||||
|
||||
**Step 7 — Write the manifest** `image-map.txt`, one line per subject:
|
||||
**Step 8 — Write the manifest** `image-map.txt`, one line per subject:
|
||||
|
||||
```
|
||||
Subject 1 - mary.png
|
||||
@ -88,19 +116,23 @@ Subject 2 - Image-2.json
|
||||
```
|
||||
|
||||
User-supplied files keep their filenames; generated sheets are always `Image-N.json`.
|
||||
The manifest never lists frame files — `Segment-img-N` numbering follows segments,
|
||||
not subjects.
|
||||
|
||||
**Step 8 — Assemble `segments.txt`.** Strip each segment text, join with a single `|`
|
||||
**Step 9 — Assemble `segments.txt`.** Strip each segment text, join with a single `|`
|
||||
(no spaces added around it). Exactly n segments, exactly n−1 `|` characters, no
|
||||
trailing separator, no extra headers or numbering. The `|` character must never appear
|
||||
inside a segment's own text.
|
||||
|
||||
**Step 9 — Validate** against the checklist below. When the environment can run Python,
|
||||
also run `python scripts/validate_outputs.py --segments segments.txt --expected-n <n>`.
|
||||
**Step 10 — Validate** against the checklist below. When the environment can run Python,
|
||||
also run `python scripts/validate_outputs.py --segments segments.txt --expected-n <n>`
|
||||
(add `--mode fl2va --total-seconds <total>` in FL2VA mode).
|
||||
|
||||
## Output location
|
||||
|
||||
Write all outputs (`segments.txt`, `Image-N.json`, `image-map.txt`) into the current
|
||||
project folder of the project the skill is running on. If no project folder is
|
||||
Write all outputs (`segments.txt`, `Image-N.json`, `Segment-img-N.json` frames,
|
||||
`Segment-img-N-last.json` in FL2VA mode, `image-map.txt`) into the current project
|
||||
folder of the project the skill is running on. If no project folder is
|
||||
obvious, use the current working directory. Never write outputs into the skill's own
|
||||
folder.
|
||||
|
||||
@ -118,22 +150,31 @@ folder.
|
||||
5. `[Shot 1]` never carries a timestamp; later shots use `At MM:SS.mmm` relative to the
|
||||
segment start.
|
||||
6. Every subject appearing anywhere in the video must have a manifest line.
|
||||
7. Frame captions describe single full-bleed 16:9 film stills in the locked style —
|
||||
photorealistic photographic language, no sheet/panel vocabulary, no text or
|
||||
borders, no lyrics, no timestamps or labels, and every on-screen character anchored
|
||||
to the supplied reference images with verbatim story-bible traits.
|
||||
8. FL2VA mode: the instruction line of every segment is the exact fixed template with
|
||||
only the final shot index and the two-decimal duration substituted; the prompt
|
||||
contains no `<Subject N>` / `<Picture N>` / `<Video N>` / `<Audio N>` labels.
|
||||
|
||||
## Validation checklist (all must pass before delivery)
|
||||
|
||||
1. `segments.txt` has exactly n segments and exactly n−1 `|` separators.
|
||||
2. Each segment contains all six sections in order: `subject_definitions`, `summary`,
|
||||
`retention_analysis`, `detailed_description`, `overall_soundscape`,
|
||||
2. (ref2va mode) Each segment contains all six sections in order: `subject_definitions`,
|
||||
`summary`, `retention_analysis`, `detailed_description`, `overall_soundscape`,
|
||||
`non_diegetic_music`.
|
||||
3. Subject numbering is global and gapless (1..M); `<Picture N>` cited inside
|
||||
`<Subject N>` always uses the same N.
|
||||
3. (ref2va mode) Subject numbering is global and gapless (1..M); `<Picture N>` cited
|
||||
inside `<Subject N>` always uses the same N. In FL2VA mode numbering lives on the
|
||||
image side (manifest + sheets) only.
|
||||
4. Every subject in the prompts has a manifest line; every `Image-N.json` in the
|
||||
manifest exists, parses, has exactly the two keys `expanded_caption` and
|
||||
`aspect_ratio`, and `aspect_ratio` is `"16:9"`. No orphan `Image-*.json` files.
|
||||
5. No `<Video` anywhere; no lyric text anywhere.
|
||||
6. Singing segments: summary prefix `[reference generation + audio reuse]`, the exact
|
||||
fixed audio template in the sound sections, and the exact lip-sync phrase inside
|
||||
`detailed_description`. Non-singing segments use `[reference generation]`.
|
||||
6. (ref2va mode) Singing segments: summary prefix `[reference generation + audio
|
||||
reuse]`, the exact fixed audio template in the sound sections, and the exact
|
||||
lip-sync phrase inside `detailed_description`. Non-singing segments use
|
||||
`[reference generation]`.
|
||||
7. `[Shot 1]` has no timestamp; any `At MM:SS.mmm` appears only on `[Shot N]`, N≥2.
|
||||
8. Sheet captions and segment descriptions agree with the locked story bible.
|
||||
9. Each sheet caption describes the four-panel photorealistic turnaround (face
|
||||
@ -142,11 +183,23 @@ background with equal spacing and no text, no labels, no borders; it uses
|
||||
photorealistic/photographic language and contains no cartoon, illustration, or
|
||||
design-board vocabulary (cartoon, anime, mascot, name tag, swatch, palette strip,
|
||||
design board, CGI, 3D render...).
|
||||
10. One `Segment-img-N.json` exists per segment (plus `Segment-img-N-last.json` per
|
||||
segment in FL2VA mode, and none in ref2va mode); each is two-key JSON with
|
||||
`"aspect_ratio": "16:9"`, photorealistic language, no banned vocabulary.
|
||||
11. FL2VA mode: each segment's first line is the exact FL2VA instruction with the
|
||||
correct final shot index and two-decimal duration; then `integrated_multimodal_description:`,
|
||||
`overall_soundscape:`, `non_diegetic_music:` in order, and none of the six ref2va
|
||||
section labels appear.
|
||||
|
||||
## References
|
||||
|
||||
- `references/ref2va-segment-format.md` — authoritative per-segment format (sections,
|
||||
labels, shots, audio template, retention markers).
|
||||
- `references/ref2va-segment-format.md` — authoritative per-segment format for the
|
||||
default mode (sections, labels, shots, audio template, retention markers).
|
||||
- `references/fl2va-segment-format.md` — the opt-in FL2VA first–last frame prompt
|
||||
format (instruction line, interpolation arc, dialogue and audio rules).
|
||||
- `references/t2i-character-sheet.md` — the four-panel photorealistic turnaround
|
||||
sheet rules and the two-key JSON output contract.
|
||||
- `references/worked-example.md` — full end-to-end example, including an audio variant.
|
||||
- `references/frame-image-prompts.md` — per-segment first/last frame image prompt
|
||||
rules (`Segment-img-N.json`, `Segment-img-N-last.json`).
|
||||
- `references/worked-example.md` — full end-to-end example, including audio and
|
||||
FL2VA variants.
|
||||
|
||||
141
references/fl2va-segment-format.md
Normal file
141
references/fl2va-segment-format.md
Normal file
@ -0,0 +1,141 @@
|
||||
# FL2VA Segment Format (first–last frame mode, optional)
|
||||
|
||||
This reference defines the per-segment video prompt when the run uses FL2VA mode —
|
||||
the user-opt-in alternative to the default ref2va format. Select FL2VA mode only when
|
||||
the user asks for it (phrases like "FL2V", "FL2VA", "first-last", "first and last
|
||||
frame", "start and end frame"). The default is always ref2va
|
||||
(`references/ref2va-segment-format.md`).
|
||||
|
||||
## 1. What changes in FL2VA mode
|
||||
|
||||
| | ref2va (default) | FL2VA (opt-in) |
|
||||
| --- | --- | --- |
|
||||
| Images the video side receives | the character-sheet images cited as `<Picture N>` | ONLY the segment's first-frame image and last-frame image |
|
||||
| Video prompt format | six sections, full-reference labels | instruction line + three fields, no reference labels |
|
||||
| Frame image prompts | `Segment-img-N.json` (first frames) | `Segment-img-N.json` AND `Segment-img-N-last.json` |
|
||||
| Character sheets + manifest | yes | YES — still generated; they feed the image side so it can render the frames with consistent characters, but they are NOT supplied to the video side |
|
||||
| `segments.txt` | n segments joined by bare `|` | identical (n segments, n−1 separators) |
|
||||
|
||||
## 2. Prompt anatomy (exact)
|
||||
|
||||
Each segment is one plain-text document. Line 1 is the instruction line, followed by
|
||||
ONE blank line, then three fields, each inline (`label: content`) with a blank line
|
||||
between fields:
|
||||
|
||||
```text
|
||||
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot N) aligns with the S.SS-second mark of the target video.
|
||||
|
||||
integrated_multimodal_description: [Shot 1] <the full description>
|
||||
|
||||
overall_soundscape: <1–4 sentences of ambience>
|
||||
|
||||
non_diegetic_music: <1–3 sentences of score, or N/A>
|
||||
```
|
||||
|
||||
Instruction-line substitutions, and nothing else may change:
|
||||
|
||||
- `Picture 1` is the segment's first-frame image; `Picture 2` is the segment's
|
||||
last-frame image.
|
||||
- The first `Shot N` is literally `Shot 1` (the video starts in the first shot).
|
||||
- The second `Shot N` is the index of the segment's FINAL shot. With the preferred
|
||||
single-shot layout this is `Shot 1`.
|
||||
- `S.SS` is the segment's duration — total_seconds / n — formatted with EXACTLY two
|
||||
decimals: 21s/2 segments → `10.50`; 34s/3 → `11.33`; 10s/1 → `10.00`.
|
||||
- The em dash and the semicolon are part of the fixed template.
|
||||
- No full-reference labels anywhere in an FL2VA prompt: never `<Subject N>`,
|
||||
`<Picture N>`, `<Video N>`, `<Audio N>`.
|
||||
|
||||
## 3. `integrated_multimodal_description`
|
||||
|
||||
One flowing description starting directly with `[Shot 1]` (the style is woven into
|
||||
the opening sentences — there is no separate style paragraph). Characters are named
|
||||
by natural identity phrases that carry the locked story-bible traits at first mention
|
||||
(the trait wording must match the sheets and the story bible verbatim). Speaker IDs
|
||||
`(S1)`, `(S2)` identify voices; see section 5.
|
||||
|
||||
**Structure — the interpolation arc.** FL2VA generates one segment between two still
|
||||
frames, so the description is a continuous path from the first frame to the last
|
||||
frame:
|
||||
|
||||
1. Open exactly on the FIRST-FRAME state (what `Segment-img-N.json` depicts).
|
||||
2. Observable intermediate changes — actions, light shifts, approach, reactions.
|
||||
3. Progressively narrowing differences — the motion and framing converge on the
|
||||
target state.
|
||||
4. Land exactly on the LAST-FRAME state (what `Segment-img-N-last.json` depicts) in
|
||||
the final sentence; the last frame is reached at the segment's end.
|
||||
|
||||
Target roughly 350–500 words distributed across the whole segment.
|
||||
|
||||
**Shots.** Prefer ONE continuous `[Shot 1]` per segment. Split into
|
||||
`[Shot N] At MM:SS.mmm, ...` (N ≥ 2, timestamps strictly increasing, relative to the
|
||||
segment start) only when the segment genuinely needs a cut. `[Shot 1]` never carries
|
||||
a timestamp. When Picture 2 is cited in the instruction with `from Shot N`, the
|
||||
last-frame state must be reached inside that final shot.
|
||||
|
||||
**Cuts.** Use only these phrases for a cut: `the camera cuts to`, `the shot cuts
|
||||
to`, `the shot transitions to`, `the shot changes to`, `the shot switches to`.
|
||||
Cross-dissolves, fades, wipes appear only when the user explicitly requests them.
|
||||
Small distance or micro-angle changes are camera moves, never cuts.
|
||||
|
||||
**Camera movement vocabulary** (write as natural English sentences, never stacked
|
||||
tags; add amplitude and speed where relevant — `with small amplitude`,
|
||||
`at slow speed`): Zoom In, Zoom Out, Push In, Pull Out, Pan Left, Pan Right, Truck
|
||||
Left, Truck Right, Tilt Up, Tilt Down, Pedestal Up, Pedestal Down, Arc Shot,
|
||||
Tracking Shot, Static Shot, Shake Slightly, Shake Strongly, POV, Roll Clockwise,
|
||||
Roll Counterclockwise.
|
||||
|
||||
**Screen text** visible in the scene is preserved exactly, inside English double
|
||||
quotes.
|
||||
|
||||
## 4. `overall_soundscape` and `non_diegetic_music`
|
||||
|
||||
- `overall_soundscape` — 1–4 sentences of diegetic ambience and physical sounds.
|
||||
Write `N/A` only when the user explicitly demands total silence.
|
||||
- `non_diegetic_music` — 1–3 sentences describing audience-only score:
|
||||
instrumentation, tempo, dynamics. No emotion words. `N/A` when there is no score.
|
||||
Never place a reused song here.
|
||||
|
||||
**Song reuse in FL2VA mode** (the user supplies a song file): the audio file is
|
||||
supplied to the video engine alongside the two frames and is the segment's complete
|
||||
final audio track. The prompt expresses it with these fixed pieces, substituting
|
||||
only the speaker number:
|
||||
|
||||
```text
|
||||
overall_soundscape: The supplied song plays as the segment's complete final audio track, with (S1) performing its vocals live.
|
||||
|
||||
non_diegetic_music: N/A
|
||||
```
|
||||
|
||||
Inside `integrated_multimodal_description`, at each visible singing point, include
|
||||
the exact phrase `(S1) lips stay synced to the song's vocals` (substitute the real
|
||||
speaker number; no possessive apostrophe). Never quote lyrics — lyric words shape
|
||||
pacing only and never appear in any output.
|
||||
|
||||
## 5. Dialogue, voices, and speakers
|
||||
|
||||
- Give every vocal source a stable ID in order of first speech: `(S1)`, `(S2)`, ...
|
||||
Two speakers at once: `(S1,S2)`. Reuse the same ID for the same voice everywhere.
|
||||
- Identity phrases, speaker IDs, and actions stay OUTSIDE the `<d>` tags; inside
|
||||
`<d>` there is ONLY the language tag and the user's exact words:
|
||||
`<d>[English] Last summer, I went to my grandfather's house.</d>`
|
||||
- Off-screen speech uses the exact phrase `says in an off-screen voiceover`,
|
||||
followed by a clause stating the speaker's lips stay closed.
|
||||
- Dialogue that continues across a cut is wrapped in `<scenetrans>` ... `</scenetrans>`
|
||||
around the whole exchange; speech truncated by the video ending is wrapped in
|
||||
`<cutoff>` ... `</cutoff>`.
|
||||
- Ordinary spoken dialogue only; song lyrics never use `<d>` (see section 4).
|
||||
|
||||
## 6. Worked micro-example
|
||||
|
||||
One 10.50-second segment (segment 1 of the 21s desert project; full example in
|
||||
`references/worked-example.md`):
|
||||
|
||||
```text
|
||||
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot 1) aligns with the 10.50-second mark of the target video.
|
||||
|
||||
integrated_multimodal_description: [Shot 1] In a cinematic sci-fi realism style with low dusk light, long violet shadows, and a muted red-and-violet palette, the video opens exactly on the first frame: a wide establishing view of a red desert at dusk, the sky a deep gradient of burnt orange fading into indigo, where the astronaut — the woman in her thirties with short black hair under a clear dome helmet, wearing the matte-white EVA suit with orange stripe accents, grey reinforced gloves, and the wrist-mounted scanner on her left forearm — crests the rippled dune ridge on the left of frame mid-step, and the scout robot — the boxy machine with the pale-grey shell, single round blue lens eye, and short antenna — wheels up the slope to her right, treads leaving a neat triple track, while the three-story monolith of black glass stands small ahead on the basin floor, its surface drinking the last light. The camera tracks laterally at walking pace as the pair descends the dune's near face, boots sinking into cooling sand and wheels humming softly, the monolith growing taller in frame with each stride until a faint sub-bass hum pushes fine grit outward from its base in slow pulses. Reaching the flat basin, the camera eases into a slow push-in with small amplitude: the astronaut stops two meters short of the glass, raises the wrist scanner, and its amber readout reflects in her dome while the robot's blue lens dilates with a soft mechanical whir and its head tilts toward the monolith. The differences to the final frame narrow steadily — the walk ends, the scanner rises, the lens settles — and the shot lands exactly on the last frame: the two figures framed side by side before the monolith's mirrored face at dusk, scanner up, lens dilated, still and close, as the hum deepens.
|
||||
|
||||
overall_soundscape: Dry desert wind with fine grit against the helmet shell, the crunch of boots in cooling sand, the soft whir of the robot's wheels, and the monolith's low pulsing hum.
|
||||
|
||||
non_diegetic_music: N/A
|
||||
```
|
||||
89
references/frame-image-prompts.md
Normal file
89
references/frame-image-prompts.md
Normal file
@ -0,0 +1,89 @@
|
||||
# Segment Frame Image Prompts (Segment-img-N.json)
|
||||
|
||||
This reference defines how the skill writes the per-segment frame image prompts.
|
||||
The image generation engine accepts image input, so it renders every segment frame
|
||||
while using the character sheets (`Image-N.json`) as reference images — that is what
|
||||
keeps the people consistent from sheet to frame to video.
|
||||
|
||||
Two kinds of frame files exist:
|
||||
|
||||
| File | Frame | Modes |
|
||||
| --- | --- | --- |
|
||||
| `Segment-img-N.json` | the FIRST frame of segment N (what the segment's video opens on) | always (ref2va default and FL2VA) |
|
||||
| `Segment-img-N-last.json` | the LAST frame of segment N (what the segment's video ends on) | FL2VA mode only — never written in ref2va mode |
|
||||
|
||||
N is the segment number (1..n), not a subject number. One file per frame, written into
|
||||
the project folder next to `segments.txt`.
|
||||
|
||||
## 1. Output contract (exact)
|
||||
|
||||
Identical to the character-sheet contract — one compact single-line JSON object with
|
||||
exactly two keys, no markdown fence, no commentary:
|
||||
|
||||
```json
|
||||
{"expanded_caption": "ONE_DETAILED_ENGLISH_CAPTION", "aspect_ratio": "16:9"}
|
||||
```
|
||||
|
||||
- `aspect_ratio` is ALWAYS the string `"16:9"`.
|
||||
- Every ASCII double quote inside the caption is serialized as `\"`; prefer wording
|
||||
without quotes at all. The file must parse with a standard JSON parser.
|
||||
|
||||
## 2. What each frame depicts
|
||||
|
||||
A frame caption describes ONE story state, frozen. Derive it from the segment text
|
||||
that is already drafted — never invent new story content:
|
||||
|
||||
- `Segment-img-1.json` — the scene's opening composition: the state at the very start
|
||||
of segment 1's `[Shot 1]`.
|
||||
- `Segment-img-N.json` (N > 1) — segment N's opening state, which continues segment
|
||||
N−1's ending story state: same characters, positions, wardrobe, props, and lighting
|
||||
(the camera may re-frame). This is the handoff that keeps segments flowing as one
|
||||
longer scene.
|
||||
- `Segment-img-N-last.json` (FL2VA mode) — the state at the very END of segment N's
|
||||
final `[Shot N]`, i.e. the state the FL2VA instruction pins to the `S.SS`-second
|
||||
mark. In FL2VA mode, segment N's last frame and segment N+1's first frame describe
|
||||
the same story state (word them independently; they must not contradict).
|
||||
|
||||
Compose from the matching shot description: keep its composition, character placement,
|
||||
wardrobe, environment, lighting, and palette. Convert process verbs into pose states
|
||||
(where she "is walking", write "mid-step"; where a light "is flickering", write its
|
||||
visible glow state).
|
||||
|
||||
## 3. Caption design rules
|
||||
|
||||
ONE fluent English descriptive paragraph, present tense, covering:
|
||||
|
||||
1. **Medium declaration**: a photorealistic, cinematic film still in 16:9 widescreen.
|
||||
This is a single full-bleed frame — no panels, no grids, no borders, no text, no
|
||||
labels, no watermark, no character-sheet layout of any kind.
|
||||
2. **Reference anchor**: for each on-screen character, state that it is the same
|
||||
character as in the supplied character reference images (e.g. "the same woman as
|
||||
in the supplied character reference images", "the same scout robot as in the
|
||||
supplied reference images").
|
||||
3. **Locked traits**: each on-screen character's full story-bible look in the exact
|
||||
locked wording — hair, eyes, skin, build, every wardrobe piece with color and
|
||||
material, worn props. Off-screen characters are simply absent; never mention them.
|
||||
4. **Composition**: shot size and camera angle, each character's position and pose in
|
||||
the frame, the environment, the light, and the palette — mirroring the opening
|
||||
(or closing) state written in the segment's shot description.
|
||||
5. **Photorealism cues**: natural skin texture, realistic fabric/material texture,
|
||||
sharp focus, believable proportions — the same photographic language the sheets
|
||||
use, so the frame reads as the same world as the reference sheets.
|
||||
|
||||
Banned in frame captions (same list as the sheets — these words pull the generation
|
||||
toward cartoons): cartoon, anime, manga, illustration, stylized, CGI, 3D render,
|
||||
mascot, avatar, character design board, name tag, swatch, palette strip, drafting
|
||||
paper. Also never write: "four panels" or any sheet vocabulary, any meta wording
|
||||
("first frame", "last frame", "segment", timestamps, shot labels), any
|
||||
`<Subject N>` / `<Picture N>` markup, and never any lyric text.
|
||||
|
||||
## 4. Example
|
||||
|
||||
First frame of segment 1 for the desert project (story bible: astronaut — woman in
|
||||
her thirties, short black hair, brown eyes, matte-white EVA suit with orange stripe
|
||||
accents, grey reinforced gloves, clear dome helmet, wrist scanner on the left
|
||||
forearm; robot — boxy, pale-grey shell, single round blue lens eye, short antenna):
|
||||
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic cinematic film still in 16:9 widescreen showing the same characters as in the supplied character reference images: the astronaut, a woman in her thirties with short black hair and brown eyes, and her scout robot. Wide establishing shot at dusk on a red desert dune: the woman crests the rippled dune ridge on the left of frame, mid-step with boots pressing into the sand, wearing the exact matte-white EVA suit with orange stripe accents, grey reinforced gloves, clear dome helmet, and the wrist-mounted scanner on her left forearm, her face visible through the dome with natural skin texture. The boxy scout robot with the pale-grey shell, single round blue lens eye, and short antenna wheels up the slope to her right, treads leaving a neat triple track. Ahead of them the three-story monolith of black glass stands small on the basin floor under a sky grading from burnt orange into indigo, low dusk light casting long violet shadows across the sand, true-to-life fabric and rubber texture, sharp focus, single full-bleed frame with no text, no borders, no panels.", "aspect_ratio": "16:9"}
|
||||
```
|
||||
@ -1,10 +1,12 @@
|
||||
# Ref2VA Segment Format (generation from vague prompt)
|
||||
|
||||
This reference defines the exact format of ONE segment prompt produced by this skill.
|
||||
It adapts the full-reference mode output format (`segment-guide.md` in this repo) to
|
||||
the skill's use case: generating a ~10-second segment from scratch, where the only
|
||||
reference assets are character images (user-provided or generated `Image-N.json`
|
||||
sheets) and, optionally, one reused song audio file.
|
||||
This reference defines the exact format of ONE segment prompt produced by this skill
|
||||
in the DEFAULT mode. It adapts the full-reference mode output format
|
||||
(`segment-guide.md` in this repo) to the skill's use case: generating a ~10-second
|
||||
segment from scratch, where the only reference assets are character images
|
||||
(user-provided or generated `Image-N.json` sheets) and, optionally, one reused song
|
||||
audio file. The opt-in FL2VA first–last frame format is defined separately in
|
||||
`fl2va-segment-format.md`; never mix the two formats in one run.
|
||||
|
||||
Write everything in English. Timestamps are relative to the segment start
|
||||
(`00:00.000` = first frame of this segment), because the video creator generates each
|
||||
|
||||
@ -1,7 +1,7 @@
|
||||
# Worked Example (end-to-end)
|
||||
|
||||
This example shows the complete deliverable for one run of the skill, followed by the
|
||||
audio variant. Copy its structure, not its content.
|
||||
audio variant and the FL2VA variant. Copy its structure, not its content.
|
||||
|
||||
## Input
|
||||
|
||||
@ -150,6 +150,22 @@ If Subject 1 had a user-provided reference instead, the line would be
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic full-color character reference sheet photographed on a clean white background, divided into FOUR panels arranged side by side with equal white spacing between the panels, and no text, no labels, and no borders. Panel 1, the leftmost and narrower panel, is a close-up of the machine's front face panel only \u2014 filling most of the panel: a boxy scout robot with a pale-grey shell and a single large round blue lens eye. Panels 2 through 4 show the same machine full body with no cropping: Panel 2 a front view facing directly forward, Panel 3 a side view facing 90 degrees to the right, and Panel 4 a back view facing completely away with the rear panel and short antenna visible. All four panels show the identical machine: a boxy pale-grey shell with soft rounded corners, a short antenna on the roof panel, a triple-wheel base with dark rubber treads, and a folded scan projector arm at the right shoulder, captured with real metal and rubber texture, faint scuffs, and even soft studio lighting.", "aspect_ratio": "16:9"}```
|
||||
|
||||
## First-frame prompts (`Segment-img-N.json`)
|
||||
|
||||
Every run also writes one first-frame image prompt per segment (here: 2 segments, so
|
||||
`Segment-img-1.json` and `Segment-img-2.json`). The image engine renders them using
|
||||
the character sheets above as reference images. Each caption depicts exactly the
|
||||
state the segment opens on — segment 2's first frame continues segment 1's ending
|
||||
story state.
|
||||
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic cinematic film still in 16:9 widescreen showing the same characters as in the supplied character reference images: the astronaut, a woman in her thirties with short black hair and brown eyes, and her scout robot. Wide establishing shot at dusk on a red desert dune: the woman crests the rippled dune ridge on the left of frame, mid-step with boots pressing into the sand, wearing the exact matte-white EVA suit with orange stripe accents, grey reinforced gloves, clear dome helmet, and the wrist-mounted scanner on her left forearm, her face visible through the dome with natural skin texture. The boxy scout robot with the pale-grey shell, single round blue lens eye, and short antenna wheels up the slope to her right, treads leaving a neat triple track. Ahead of them the three-story monolith of black glass stands small on the basin floor under a sky grading from burnt orange into indigo, low dusk light casting long violet shadows across the sand, true-to-life fabric and rubber texture, sharp focus, single full-bleed frame with no text, no borders, no panels.", "aspect_ratio": "16:9"}
|
||||
```
|
||||
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic cinematic film still in 16:9 widescreen showing the same characters as in the supplied character reference images: the astronaut, a woman in her thirties with short black hair and brown eyes, and her scout robot. Medium shot at shoulder height at dusk in the red desert basin: the two figures stand side by side before the three-story monolith of black glass, the woman on the left wearing the exact matte-white EVA suit with orange stripe accents, grey reinforced gloves, and clear dome helmet, her left forearm raised with the wrist-mounted scanner held up and its small amber readout glowing against the glass, natural skin texture visible through the dome. The boxy scout robot with the pale-grey shell, single round blue lens eye, and short antenna hovers its head between her and the monolith, antenna upright, lens turned toward the glass. Long violet shadows stretch across the rippled sand, the sky grades from burnt orange into indigo, true-to-life fabric and rubber texture, sharp focus, single full-bleed frame with no text, no borders, no panels.", "aspect_ratio": "16:9"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# Audio variant (one singing segment)
|
||||
@ -185,3 +201,90 @@ overall_soundscape:
|
||||
non_diegetic_music:
|
||||
N/A
|
||||
```
|
||||
|
||||
Its single segment also gets a first-frame prompt:
|
||||
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic cinematic film still in 16:9 widescreen showing the same woman as in the supplied character reference images: the cabaret singer in her late forties with voluminous auburn curls. Wide shot of a smoky cabaret stage at night: the woman stands centered in a single tungsten spotlight cone cutting through drifting cigarette smoke, wearing the exact emerald satin slip dress with a soft sheen and black opera gloves to the elbow, one gloved hand resting on the vintage chrome microphone stand, her face turned in profile with natural skin texture. Dark stage curtains and small round tables with shadowed patrons fade into the haze behind her, deep amber-and-emerald palette, warm tungsten glow on the satin, true-to-life fabric texture, sharp focus, single full-bleed frame with no text, no borders, no panels.", "aspect_ratio": "16:9"}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
# FL2VA variant (first–last frames, same desert input)
|
||||
|
||||
Same input as the main example, but the user opted into FL2VA mode ("give me
|
||||
first-last frames"). The video side now receives ONLY each segment's first and last
|
||||
frame images plus the FL2VA-format prompt below — never the character sheets. The
|
||||
sheets (`Image-1.json`, `Image-2.json`) and `image-map.txt` are still generated
|
||||
exactly as in the main example, because the image side needs them to render the
|
||||
frames with consistent characters. Two segments of 21/2 = 10.50 s each, so every
|
||||
instruction line pins Picture 2 to the `10.50`-second mark; both segments use the
|
||||
preferred single-shot interpolation arc. Note there are no `<Subject N>` /
|
||||
`<Picture N>` labels in these prompts.
|
||||
|
||||
## Segment 1 (FL2VA, ~0–10.5 s)
|
||||
|
||||
```text
|
||||
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot 1) aligns with the 10.50-second mark of the target video.
|
||||
|
||||
integrated_multimodal_description: [Shot 1] In a cinematic sci-fi realism style with low dusk light, long violet shadows, and a muted red-and-violet palette, the video opens exactly on the first frame: a wide establishing view of a red desert at dusk, the sky a deep gradient of burnt orange fading into indigo, where the astronaut — the woman in her thirties with short black hair under a clear dome helmet, wearing the matte-white EVA suit with orange stripe accents, grey reinforced gloves, and the wrist-mounted scanner on her left forearm — crests the rippled dune ridge on the left of frame mid-step, and the scout robot — the boxy machine with the pale-grey shell, single round blue lens eye, and short antenna — wheels up the slope to her right, treads leaving a neat triple track, while the three-story monolith of black glass stands small ahead on the basin floor, its surface drinking the last light. The camera tracks laterally at walking pace as the pair descends the dune's near face, boots sinking into cooling sand with each stride, wheels humming softly over the ripples, and the monolith grows taller in frame step by step until a faint sub-bass hum pushes fine grit outward from its base in slow visible pulses. Reaching the flat basin, the camera eases into a slow push-in with small amplitude: the astronaut stops two meters short of the glass, raises the wrist scanner, and its amber readout reflects in her dome while the robot's blue lens dilates with a soft mechanical whir and its head tilts toward the monolith. From there the differences to the final frame narrow steadily — the walk ends, the scanner holds steady, the lens settles — as the camera drifts down to shoulder height and the dusk wind keeps dragging thin veils of red dust between the lens and the pair. The shot lands exactly on the last frame: the two figures framed side by side before the monolith's mirrored black glass, her scanner up with its amber glow steady on the glass, the robot's lens fully dilated toward the monolith, both reflections floating in the towering surface as the hum deepens.
|
||||
|
||||
overall_soundscape: Dry desert wind with fine grit against the helmet shell, the crunch of boots in cooling sand, the soft whir of the robot's wheels, and the monolith's low pulsing hum.
|
||||
|
||||
non_diegetic_music: N/A
|
||||
```
|
||||
|
||||
## Segment 2 (FL2VA, ~10.5–21 s)
|
||||
|
||||
```text
|
||||
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot 1) aligns with the 10.50-second mark of the target video.
|
||||
|
||||
integrated_multimodal_description: [Shot 1] In the same cinematic sci-fi realism style with low dusk light, long violet shadows, and a muted red-and-violet palette, the video opens exactly on its first frame: a medium shot at shoulder height on the pair standing before the monolith of black glass in the red desert basin at dusk — the astronaut, the woman in her thirties with short black hair under a clear dome helmet, matte-white EVA suit with orange stripe accents, grey reinforced gloves, and the wrist-mounted scanner held up in her left hand with its amber readout glowing steadily, and the scout robot with the boxy pale-grey shell, single round blue lens eye, and short antenna, its head hovered between her and the glass with the lens fully dilated toward the monolith. For the first moments the scene breathes almost still: dry wind drags thin veils of red dust across the basin, her readout's glow quickens, and the camera holds nearly static with only a slight tremor, the robot's antenna trembling faintly in the gusts while her gloved fingers tighten one notch on the scanner housing. Then the monolith ignites: slow rivers of amber circuitry light climb its black surface from the base to the top edge, throwing hard orange reflections across her dome and the robot's shell, and fine red dust lifts from the basin floor in a widening, slowly rising ring. The camera begins a smooth Tilt Up at slow speed, following the climbing light past their reflected faces toward the monolith's crown as the amber brightness reaches its full, even glow, the climbing light strobing softly across the rippled sand at its base. The tilt eases and reverses gently into a slow Pedestal Down, the framing converging back toward the two small figures as the dust ring thins and the light steadies. The shot lands exactly on the last frame: the astronaut and the robot standing small and still against the towering glass now fully lit with rivers of amber circuitry light at full brightness, hard orange reflections lying across her dome and its pale-grey shell, the dusk sky grading from burnt orange into indigo above the crown.
|
||||
|
||||
overall_soundscape: Dry desert wind rising in gusts, the deepening pulse of the monolith's hum, the whir of the robot's scan projector, and fine grit ticking against the helmet shell.
|
||||
|
||||
non_diegetic_music: A low sustained analog-synth drone enters softly with the ignition and swells in volume and harmonic richness through the final frames, with no percussion.
|
||||
```
|
||||
|
||||
## `segments.txt` (FL2VA)
|
||||
|
||||
The two FL2VA segment texts above, stripped and joined by a single `|`, exactly as in
|
||||
the default mode:
|
||||
|
||||
```text
|
||||
How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot 1) aligns with the 10.50-second mark of the target video.
|
||||
|
||||
integrated_multimodal_description: [Shot 1] In a cinematic sci-fi realism style with low dusk light, long violet shadows, and a muted red-and-violet palette, the video opens exactly on the first frame: a wide establishing view of a red desert at dusk, the sky a deep gradient of burnt orange fading into indigo, where the astronaut — the woman in her thirties with short black hair under a clear dome helmet, wearing the matte-white EVA suit with orange stripe accents, grey reinforced gloves, and the wrist-mounted scanner on her left forearm — crests the rippled dune ridge on the left of frame mid-step, and the scout robot — the boxy machine with the pale-grey shell, single round blue lens eye, and short antenna — wheels up the slope to her right, treads leaving a neat triple track, while the three-story monolith of black glass stands small ahead on the basin floor, its surface drinking the last light. The camera tracks laterally at walking pace as the pair descends the dune's near face, boots sinking into cooling sand with each stride, wheels humming softly over the ripples, and the monolith grows taller in frame step by step until a faint sub-bass hum pushes fine grit outward from its base in slow visible pulses. Reaching the flat basin, the camera eases into a slow push-in with small amplitude: the astronaut stops two meters short of the glass, raises the wrist scanner, and its amber readout reflects in her dome while the robot's blue lens dilates with a soft mechanical whir and its head tilts toward the monolith. From there the differences to the final frame narrow steadily — the walk ends, the scanner holds steady, the lens settles — as the camera drifts down to shoulder height and the dusk wind keeps dragging thin veils of red dust between the lens and the pair. The shot lands exactly on the last frame: the two figures framed side by side before the monolith's mirrored black glass, her scanner up with its amber glow steady on the glass, the robot's lens fully dilated toward the monolith, both reflections floating in the towering surface as the hum deepens.
|
||||
|
||||
overall_soundscape: Dry desert wind with fine grit against the helmet shell, the crunch of boots in cooling sand, the soft whir of the robot's wheels, and the monolith's low pulsing hum.
|
||||
|
||||
non_diegetic_music: N/A
|
||||
|How the reference pictures align with the target video — Picture 1 (from Shot 1) aligns with the 0.00-second mark of the target video; Picture 2 (from Shot 1) aligns with the 10.50-second mark of the target video.
|
||||
|
||||
integrated_multimodal_description: [Shot 1] In the same cinematic sci-fi realism style with low dusk light, long violet shadows, and a muted red-and-violet palette, the video opens exactly on its first frame: a medium shot at shoulder height on the pair standing before the monolith of black glass in the red desert basin at dusk — the astronaut, the woman in her thirties with short black hair under a clear dome helmet, matte-white EVA suit with orange stripe accents, grey reinforced gloves, and the wrist-mounted scanner held up in her left hand with its amber readout glowing steadily, and the scout robot with the boxy pale-grey shell, single round blue lens eye, and short antenna, its head hovered between her and the glass with the lens fully dilated toward the monolith. For the first moments the scene breathes almost still: dry wind drags thin veils of red dust across the basin, her readout's glow quickens, and the camera holds nearly static with only a slight tremor, the robot's antenna trembling faintly in the gusts while her gloved fingers tighten one notch on the scanner housing. Then the monolith ignites: slow rivers of amber circuitry light climb its black surface from the base to the top edge, throwing hard orange reflections across her dome and the robot's shell, and fine red dust lifts from the basin floor in a widening, slowly rising ring. The camera begins a smooth Tilt Up at slow speed, following the climbing light past their reflected faces toward the monolith's crown as the amber brightness reaches its full, even glow, the climbing light strobing softly across the rippled sand at its base. The tilt eases and reverses gently into a slow Pedestal Down, the framing converging back toward the two small figures as the dust ring thins and the light steadies. The shot lands exactly on the last frame: the astronaut and the robot standing small and still against the towering glass now fully lit with rivers of amber circuitry light at full brightness, hard orange reflections lying across her dome and its pale-grey shell, the dusk sky grading from burnt orange into indigo above the crown.
|
||||
|
||||
overall_soundscape: Dry desert wind rising in gusts, the deepening pulse of the monolith's hum, the whir of the robot's scan projector, and fine grit ticking against the helmet shell.
|
||||
|
||||
non_diegetic_music: A low sustained analog-synth drone enters softly with the ignition and swells in volume and harmonic richness through the final frames, with no percussion.
|
||||
```
|
||||
|
||||
## Frame image prompts (FL2VA)
|
||||
|
||||
FL2VA mode writes BOTH frames per segment. Segment 2's first frame describes the
|
||||
same story state as segment 1's last frame (the handoff), worded independently.
|
||||
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic cinematic film still in 16:9 widescreen showing the same characters as in the supplied character reference images: the astronaut, a woman in her thirties with short black hair and brown eyes, and her scout robot. Wide establishing shot at dusk on a red desert dune: the woman in the exact matte-white EVA suit with orange stripe accents, grey reinforced gloves, clear dome helmet, and wrist-mounted scanner on her left forearm crests the rippled dune ridge on the left of frame, mid-step, her face visible through the dome with natural skin texture, while the boxy scout robot with the pale-grey shell, single round blue lens eye, and short antenna wheels up the slope to her right leaving a triple track. The three-story monolith of black glass stands small ahead on the basin floor under a sky grading from burnt orange into indigo, low dusk light casting long violet shadows, true-to-life fabric and rubber texture, sharp focus, single full-bleed frame with no text, no borders, no panels.", "aspect_ratio": "16:9"}
|
||||
```
|
||||
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic cinematic film still in 16:9 widescreen showing the same characters as in the supplied character reference images: the astronaut, a woman in her thirties with short black hair and brown eyes, and her scout robot. Medium shot at shoulder height at dusk: the two figures stand side by side before the monolith's mirrored black glass face, the woman in the exact matte-white EVA suit with orange stripe accents and grey reinforced gloves holding her wrist-mounted scanner up with its small amber readout glowing steady against the glass, her features clear through the dome with natural skin texture, the boxy pale-grey robot's single round blue lens eye fully dilated toward the monolith and its short antenna upright. Both figures reflect faintly in the polished black surface, long violet shadows run across the empty basin sand behind them, the dusk sky grades from burnt orange into indigo, true-to-life fabric and rubber texture, sharp focus, single full-bleed frame with no text, no borders, no panels.", "aspect_ratio": "16:9"}
|
||||
```
|
||||
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic cinematic film still in 16:9 widescreen showing the same characters as in the supplied character reference images: the astronaut, a woman in her thirties with short black hair and brown eyes, and her scout robot. Shoulder-height medium shot at dusk in the red desert basin: the woman in the exact matte-white EVA suit with orange stripe accents, grey reinforced gloves, and clear dome helmet stands on the left with her left forearm raised, the wrist-mounted scanner held up with its small amber readout glowing steadily, natural skin texture visible through the dome glass, while the boxy scout robot with the pale-grey shell, single round blue lens eye, and short antenna hovers its head between her and the three-story monolith of black glass, lens dilated toward it. Rippled sand fills the foreground with long violet shadows, the dusk sky grades from burnt orange into indigo, true-to-life fabric and rubber texture, sharp focus, single full-bleed frame with no text, no borders, no panels.", "aspect_ratio": "16:9"}
|
||||
```
|
||||
|
||||
```json
|
||||
{"expanded_caption": "A photorealistic cinematic film still in 16:9 widescreen showing the same characters as in the supplied character reference images: the astronaut, a woman in her thirties with short black hair and brown eyes, and her scout robot. Wide settled shot at dusk: the two small figures stand still against the towering three-story monolith of black glass now fully lit by slow rivers of amber circuitry light at full brightness climbing its surface, hard orange reflections lying across the woman's clear dome helmet and the robot's pale-grey shell, the woman in the exact matte-white EVA suit with orange stripe accents and grey reinforced gloves, the boxy robot with its single round blue lens eye and short antenna beside her. Fine red dust hangs in a thin widening ring above the basin floor, long violet shadows stretch across the rippled sand, the dusk sky grades from burnt orange into indigo above the monolith's crown, natural skin texture through the dome, true-to-life fabric and rubber texture, sharp focus, single full-bleed frame with no text, no borders, no panels.", "aspect_ratio": "16:9"}
|
||||
```
|
||||
|
||||
@ -2,17 +2,25 @@
|
||||
"""Validator for mm-prompter skill outputs.
|
||||
|
||||
Checks the output contract described in SKILL.md:
|
||||
- segments.txt: n segments, n-1 "|" separators, six sections per segment in order
|
||||
- global gapless Subject numbering, <Picture N> paired with <Subject N>
|
||||
- image-map.txt manifest covers every subject; Image-N.json valid (two keys, 16:9)
|
||||
- segments.txt: n segments, n-1 "|" separators
|
||||
- ref2va mode (default): six sections per segment in order, global gapless
|
||||
Subject numbering, <Picture N> paired with <Subject N>, summary prefixes,
|
||||
audio template / lip-sync phrase / fully_copy retention line,
|
||||
[Shot 1] timestamp rules
|
||||
- fl2va mode (opt-in): exact instruction line (final shot index + two-decimal
|
||||
duration), the three fields in order, no reference labels, no ref2va
|
||||
section labels, shot rules, fixed song sentences when a song is reused
|
||||
- image-map.txt manifest covers every subject; Image-N.json valid (two keys,
|
||||
16:9, four-panel photorealistic sheet language)
|
||||
- Segment-img-N.json per segment (both modes) and Segment-img-N-last.json
|
||||
per segment (fl2va only): two keys, 16:9, single-frame photorealistic
|
||||
language, no banned vocabulary
|
||||
- no <Video> labels; lyrics (optional file) never appear in the prompts
|
||||
- audio segments: exact summary prefix, exact audio template, lip-sync phrase,
|
||||
fully_copy retention line
|
||||
- [Shot 1] has no timestamp; shot numbers contiguous; timestamps only on N>=2
|
||||
|
||||
Usage:
|
||||
python3 scripts/validate_outputs.py --segments segments.txt [--expected-n 3] \
|
||||
[--dir .] [--manifest image-map.txt] [--lyrics-file lyrics.txt] [--expect-audio]
|
||||
[--mode ref2va|fl2va] [--total-seconds 21] [--dir .] [--manifest image-map.txt] \
|
||||
[--lyrics-file lyrics.txt] [--expect-audio]
|
||||
|
||||
Exit code 0 = all checks passed, 1 = failures listed.
|
||||
"""
|
||||
@ -46,11 +54,37 @@ SHOT_RE = re.compile(r"\[Shot (\d+)\]( At (\d{2}):(\d{2})\.(\d{3}))?")
|
||||
SUBJECT_RE = re.compile(r"<Subject (\d+)>")
|
||||
PICTURE_RE = re.compile(r"<Picture (\d+)>")
|
||||
|
||||
FL2VA_INSTRUCTION_RE = re.compile(
|
||||
r"How the reference pictures align with the target video "
|
||||
r"— Picture 1 \(from Shot 1\) aligns with the 0\.00-second mark of the "
|
||||
r"target video; Picture 2 \(from Shot (\d+)\) aligns with the "
|
||||
r"(\d+)\.(\d{2})-second mark of the target video\."
|
||||
)
|
||||
FL2VA_FIELDS = [
|
||||
"integrated_multimodal_description",
|
||||
"overall_soundscape",
|
||||
"non_diegetic_music",
|
||||
]
|
||||
FL2VA_SONG_SENTENCE_RE = re.compile(
|
||||
r"overall_soundscape:\s*"
|
||||
r"(?:The supplied song plays as the segment's complete final audio track, "
|
||||
r"with \(S\d+\) performing its vocals live\.\s*)+$"
|
||||
)
|
||||
FL2VA_LIPSYNC_RE = re.compile(r"\(S(\d+)\) lips stay synced to the song's vocals")
|
||||
|
||||
BANNED_WORDS = ["cartoon", "anime", "manga", "illustration", "stylized",
|
||||
"cgi", "3d render", "mascot", "avatar", "name tag",
|
||||
"swatch", "palette strip", "design board", "drafting paper"]
|
||||
|
||||
|
||||
def fail(errors, seg_no, msg):
|
||||
errors.append(f"[segment {seg_no}] {msg}" if seg_no else msg)
|
||||
|
||||
|
||||
def normalize(text):
|
||||
return re.sub(r"\s+", " ", text).strip().lower()
|
||||
|
||||
|
||||
def split_sections(segment_text, seg_no, errors):
|
||||
"""Return {name: content} or None if headings are missing/out of order."""
|
||||
positions = []
|
||||
@ -72,16 +106,45 @@ def split_sections(segment_text, seg_no, errors):
|
||||
return sections
|
||||
|
||||
|
||||
def normalize(text):
|
||||
return re.sub(r"\s+", " ", text).strip().lower()
|
||||
def check_shots(body, seg_no, errors):
|
||||
"""Shared shot checks: [Shot 1] no timestamp, contiguous numbers, increasing."""
|
||||
shot_nums = [int(m.group(1)) for m in SHOT_RE.finditer(body)]
|
||||
shot_lines = re.findall(r"^\[Shot (\d+)\]", body, re.MULTILINE)
|
||||
if shot_lines:
|
||||
nums = [int(x) for x in shot_lines]
|
||||
if nums != list(range(1, len(nums) + 1)):
|
||||
fail(errors, seg_no, f"shot numbers must start at [Shot 1] and be "
|
||||
f"contiguous (found {nums})")
|
||||
elif shot_nums and shot_nums != list(range(1, max(shot_nums) + 1)):
|
||||
fail(errors, seg_no, f"shot numbers must start at [Shot 1] and be "
|
||||
f"contiguous (found {shot_nums})")
|
||||
prev_ts = 0.0
|
||||
for m in SHOT_RE.finditer(body):
|
||||
n = int(m.group(1))
|
||||
if m.group(2):
|
||||
if n == 1:
|
||||
fail(errors, seg_no, "[Shot 1] must not carry a timestamp")
|
||||
ts = int(m.group(3)) * 60 + int(m.group(4)) + int(m.group(5)) / 1000.0
|
||||
if ts < prev_ts:
|
||||
fail(errors, seg_no, f"timestamp on [Shot {n}] goes backwards")
|
||||
prev_ts = ts
|
||||
elif n > 1:
|
||||
fail(errors, seg_no, f"[Shot {n}] must carry an 'At MM:SS.mmm' timestamp")
|
||||
if not shot_nums:
|
||||
fail(errors, seg_no, "description contains no [Shot N] block")
|
||||
return shot_nums
|
||||
|
||||
|
||||
def check_segment(seg_text, seg_no, errors):
|
||||
"""Per-segment checks. Returns (defined_subjects, has_audio)."""
|
||||
def check_segment_ref2va(seg_text, seg_no, errors):
|
||||
"""Per-segment ref2va checks. Returns (defined_subjects, has_audio)."""
|
||||
sections = split_sections(seg_text, seg_no, errors)
|
||||
if sections is None:
|
||||
return set(), False
|
||||
|
||||
if "integrated_multimodal_description" in seg_text:
|
||||
fail(errors, seg_no, "ref2va prompts must not use the FL2VA field "
|
||||
"'integrated_multimodal_description'")
|
||||
|
||||
defs = sections["subject_definitions"]
|
||||
body_after_defs = seg_text.split("detailed_description:", 1)[1] \
|
||||
if "detailed_description:" in seg_text else ""
|
||||
@ -134,42 +197,159 @@ def check_segment(seg_text, seg_no, errors):
|
||||
fail(errors, seg_no, "segment mixes <Audio> usage with non-audio prefix")
|
||||
|
||||
# Shots
|
||||
shots = SHOT_RE.findall(sections["detailed_description"])
|
||||
shot_lines = re.findall(r"^\[Shot (\d+)\].*$", sections["detailed_description"],
|
||||
re.MULTILINE)
|
||||
if shot_lines:
|
||||
nums = [int(n) for n in shot_lines]
|
||||
if nums != list(range(1, len(nums) + 1)):
|
||||
fail(errors, seg_no, f"shot numbers must start at [Shot 1] and be "
|
||||
f"contiguous (found {nums})")
|
||||
prev_ts = 0.0
|
||||
for m in SHOT_RE.finditer(sections["detailed_description"]):
|
||||
n = int(m.group(1))
|
||||
if m.group(2):
|
||||
if n == 1:
|
||||
fail(errors, seg_no, "[Shot 1] must not carry a timestamp")
|
||||
ts = int(m.group(3)) * 60 + int(m.group(4)) + int(m.group(5)) / 1000.0
|
||||
if ts < prev_ts:
|
||||
fail(errors, seg_no, f"timestamp on [Shot {n}] goes backwards")
|
||||
prev_ts = ts
|
||||
elif n > 1:
|
||||
fail(errors, seg_no, f"[Shot {n}] must carry an 'At MM:SS.mmm' timestamp")
|
||||
if not shots:
|
||||
fail(errors, seg_no, "detailed_description contains no [Shot N] block")
|
||||
check_shots(sections["detailed_description"], seg_no, errors)
|
||||
|
||||
return defined, has_audio
|
||||
|
||||
|
||||
def check_segment_fl2va(seg_text, seg_no, errors, expected_duration=None):
|
||||
"""Per-segment FL2VA checks. Returns has_audio."""
|
||||
lines = seg_text.split("\n")
|
||||
first = lines[0].strip() if lines else ""
|
||||
m = FL2VA_INSTRUCTION_RE.fullmatch(first)
|
||||
cited_shot = None
|
||||
if not m:
|
||||
fail(errors, seg_no, "first line must be the exact FL2VA instruction: 'How "
|
||||
"the reference pictures align with the target video — Picture 1 (from "
|
||||
"Shot 1) aligns with the 0.00-second mark of the target video; Picture "
|
||||
"2 (from Shot N) aligns with the S.SS-second mark of the target video.'")
|
||||
else:
|
||||
cited_shot = int(m.group(1))
|
||||
secs = int(m.group(2)) + int(m.group(3)) / 100.0
|
||||
if expected_duration is not None and abs(secs - expected_duration) > 0.0051:
|
||||
fail(errors, seg_no, f"instruction pins Picture 2 to the {secs:.2f}-second "
|
||||
f"mark, but the segment duration is "
|
||||
f"{expected_duration:.2f} seconds (total/n, two decimals)")
|
||||
if len(lines) < 2 or lines[1].strip() != "":
|
||||
fail(errors, seg_no, "the instruction line must be followed by exactly "
|
||||
"one blank line")
|
||||
|
||||
# The three fields, in order, inline
|
||||
spans = []
|
||||
for name in FL2VA_FIELDS:
|
||||
matches = list(re.finditer(rf"^{name}: ", seg_text, re.MULTILINE))
|
||||
if len(matches) != 1:
|
||||
fail(errors, seg_no, f"field '{name}:' must appear exactly once as an "
|
||||
f"inline field (found {len(matches)})")
|
||||
spans.append(None)
|
||||
else:
|
||||
spans.append(matches[0])
|
||||
integ = ""
|
||||
if all(s is not None for s in spans):
|
||||
starts = [s.start() for s in spans]
|
||||
if starts != sorted(starts):
|
||||
fail(errors, seg_no, "fields out of order; expected "
|
||||
"integrated_multimodal_description, overall_soundscape, "
|
||||
"non_diegetic_music")
|
||||
integ = seg_text[spans[0].end():spans[1].start()].strip()
|
||||
soundscape = seg_text[spans[1].end():spans[2].start()].strip()
|
||||
music = seg_text[spans[2].end():].strip()
|
||||
if not integ:
|
||||
fail(errors, seg_no, "integrated_multimodal_description is empty")
|
||||
elif not integ.startswith("[Shot 1]"):
|
||||
fail(errors, seg_no, "integrated_multimodal_description must start "
|
||||
"with '[Shot 1]'")
|
||||
if not soundscape:
|
||||
fail(errors, seg_no, "overall_soundscape is empty")
|
||||
if not music:
|
||||
fail(errors, seg_no, "non_diegetic_music is empty")
|
||||
|
||||
# ref2va machinery must not leak into FL2VA prompts
|
||||
for name in SECTION_ORDER:
|
||||
if re.search(rf"^{name}:\s*$", seg_text, re.MULTILINE):
|
||||
fail(errors, seg_no, f"FL2VA prompts must not contain the ref2va section "
|
||||
f"label '{name}:'")
|
||||
for label in ("<Subject", "<Picture", "<Video", "<Audio"):
|
||||
if label in seg_text:
|
||||
fail(errors, seg_no, f"FL2VA prompts must not contain full-reference "
|
||||
f"labels ({label} N)")
|
||||
|
||||
# Shots
|
||||
shot_nums = check_shots(integ or seg_text, seg_no, errors)
|
||||
if cited_shot is not None and shot_nums:
|
||||
if max(shot_nums) != cited_shot:
|
||||
fail(errors, seg_no, f"instruction cites Picture 2 from Shot {cited_shot} "
|
||||
f"but the highest shot in the description is "
|
||||
f"Shot {max(shot_nums)}")
|
||||
|
||||
# Song reuse
|
||||
has_song = bool(FL2VA_SONG_SENTENCE_RE.search(seg_text))
|
||||
lipsync = FL2VA_LIPSYNC_RE.search(seg_text)
|
||||
if has_song:
|
||||
if not lipsync:
|
||||
fail(errors, seg_no, "FL2VA singing segment must contain "
|
||||
"'(S1) lips stay synced to the song's vocals' inside "
|
||||
"integrated_multimodal_description")
|
||||
if not re.search(r"non_diegetic_music:\s*N/A", seg_text):
|
||||
fail(errors, seg_no, "FL2VA singing segment must set non_diegetic_music "
|
||||
"to N/A (the song is the complete final audio track)")
|
||||
if "<d>" in seg_text:
|
||||
print(f"WARN: [segment {seg_no}] <d> block inside a singing FL2VA "
|
||||
f"segment; song lyrics must never be written out", file=sys.stderr)
|
||||
elif lipsync:
|
||||
fail(errors, seg_no, "'(S1) lips stay synced to the song's vocals' present but "
|
||||
"the fixed song sentence is missing from overall_soundscape: "
|
||||
"'The supplied song plays as the segment's complete final "
|
||||
"audio track, with (S1) performing its vocals live.'")
|
||||
|
||||
return has_song
|
||||
|
||||
|
||||
def load_two_key_json(fpath, errors):
|
||||
"""Return the parsed object or None; logs contract errors."""
|
||||
try:
|
||||
obj = json.loads(fpath.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError as e:
|
||||
errors.append(f"{fpath.name} is not valid JSON: {e}")
|
||||
return None
|
||||
if set(obj.keys()) != {"expanded_caption", "aspect_ratio"}:
|
||||
errors.append(f"{fpath.name} must have exactly the two keys "
|
||||
f"expanded_caption and aspect_ratio (found {sorted(obj.keys())})")
|
||||
return None
|
||||
if obj["aspect_ratio"] != "16:9":
|
||||
errors.append(f"{fpath.name} aspect_ratio must be \"16:9\" "
|
||||
f"(found {obj['aspect_ratio']!r})")
|
||||
if not isinstance(obj["expanded_caption"], str) or \
|
||||
len(obj["expanded_caption"].strip()) < 40:
|
||||
errors.append(f"{fpath.name} expanded_caption is missing or too short")
|
||||
return None
|
||||
return obj
|
||||
|
||||
|
||||
def check_frame_caption(fname, obj, errors):
|
||||
"""Frame-file caption checks (single full-bleed film still)."""
|
||||
cap = obj["expanded_caption"].lower()
|
||||
hits = [w for w in BANNED_WORDS if w in cap]
|
||||
if hits:
|
||||
errors.append(f"{fname} caption contains illustration/design-board vocabulary "
|
||||
f"that produces cartoons: {hits}")
|
||||
if "photorealistic" not in cap and "photograph" not in cap:
|
||||
errors.append(f"{fname} caption must pin the photorealistic photographic "
|
||||
f"style for the frame")
|
||||
if "four panels" in cap:
|
||||
errors.append(f"{fname} caption describes a multi-panel sheet; a segment "
|
||||
f"frame must be one single full-bleed film still")
|
||||
if "supplied character reference images" not in cap and \
|
||||
"supplied reference images" not in cap:
|
||||
errors.append(f"{fname} caption must anchor its characters to the supplied "
|
||||
f"character reference images")
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__,
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter)
|
||||
ap.add_argument("--segments", required=True)
|
||||
ap.add_argument("--expected-n", type=int, default=None)
|
||||
ap.add_argument("--dir", default=".", help="directory containing Image-N.json files")
|
||||
ap.add_argument("--mode", choices=["ref2va", "fl2va"], default="ref2va",
|
||||
help="ref2va (default) or fl2va first-last mode")
|
||||
ap.add_argument("--total-seconds", type=float, default=None,
|
||||
help="total video length; enables the S.SS duration check in "
|
||||
"fl2va mode")
|
||||
ap.add_argument("--dir", default=".", help="directory containing the image files")
|
||||
ap.add_argument("--manifest", default=None, help="default: <dir>/image-map.txt")
|
||||
ap.add_argument("--lyrics-file", default=None)
|
||||
ap.add_argument("--expect-audio", action="store_true",
|
||||
help="fail unless at least one segment references <Audio>")
|
||||
help="fail unless at least one segment reuses audio")
|
||||
args = ap.parse_args()
|
||||
|
||||
errors = []
|
||||
@ -189,10 +369,17 @@ def main():
|
||||
if not part.strip():
|
||||
errors.append(f"segment {i} is empty")
|
||||
|
||||
expected_duration = None
|
||||
if args.mode == "fl2va" and args.total_seconds:
|
||||
expected_duration = round(args.total_seconds / n, 2)
|
||||
|
||||
subjects, audio_flags = set(), []
|
||||
for i, part in enumerate(parts, 1):
|
||||
defined, has_audio = check_segment(part.strip(), i, errors)
|
||||
subjects |= defined
|
||||
if args.mode == "fl2va":
|
||||
has_audio = check_segment_fl2va(part.strip(), i, errors, expected_duration)
|
||||
else:
|
||||
defined, has_audio = check_segment_ref2va(part.strip(), i, errors)
|
||||
subjects |= defined
|
||||
audio_flags.append(has_audio)
|
||||
|
||||
if "<Video" in raw:
|
||||
@ -200,7 +387,8 @@ def main():
|
||||
"video references")
|
||||
|
||||
all_nums = set(int(x) for x in SUBJECT_RE.findall(raw))
|
||||
if all_nums and all_nums != set(range(1, max(all_nums) + 1)):
|
||||
if args.mode == "ref2va" and all_nums and \
|
||||
all_nums != set(range(1, max(all_nums) + 1)):
|
||||
errors.append(f"global Subject numbering must be gapless 1..{max(all_nums)} "
|
||||
f"(found {sorted(all_nums)})")
|
||||
|
||||
@ -224,6 +412,9 @@ def main():
|
||||
if num in mapping:
|
||||
errors.append(f"manifest lists Subject {num} twice")
|
||||
mapping[num] = fname
|
||||
if mapping and set(mapping) != set(range(1, max(mapping) + 1)):
|
||||
errors.append(f"manifest subject numbering must be gapless 1.."
|
||||
f"{max(mapping)} (found {sorted(mapping)})")
|
||||
|
||||
for num in sorted(all_nums):
|
||||
if num not in mapping:
|
||||
@ -248,31 +439,14 @@ def main():
|
||||
if not fpath.exists():
|
||||
errors.append(f"generated sheet missing: {fpath}")
|
||||
continue
|
||||
try:
|
||||
obj = json.loads(fpath.read_text(encoding="utf-8"))
|
||||
except json.JSONDecodeError as e:
|
||||
errors.append(f"{fname} is not valid JSON: {e}")
|
||||
continue
|
||||
if set(obj.keys()) != {"expanded_caption", "aspect_ratio"}:
|
||||
errors.append(f"{fname} must have exactly the two keys "
|
||||
f"expanded_caption and aspect_ratio (found "
|
||||
f"{sorted(obj.keys())})")
|
||||
continue
|
||||
if obj["aspect_ratio"] != "16:9":
|
||||
errors.append(f"{fname} aspect_ratio must be \"16:9\" "
|
||||
f"(found {obj['aspect_ratio']!r})")
|
||||
if not isinstance(obj["expanded_caption"], str) or \
|
||||
len(obj["expanded_caption"].strip()) < 40:
|
||||
errors.append(f"{fname} expanded_caption is missing or too short")
|
||||
obj = load_two_key_json(fpath, errors)
|
||||
if obj is None:
|
||||
continue
|
||||
cap = obj["expanded_caption"].lower()
|
||||
if "four panels" not in cap:
|
||||
errors.append(f"{fname} caption must describe the four-panel "
|
||||
f"turnaround sheet (face close-up, front, side, back)")
|
||||
banned = ["cartoon", "anime", "manga", "illustration", "stylized",
|
||||
"cgi", "3d render", "mascot", "avatar", "name tag",
|
||||
"swatch", "palette strip", "design board", "drafting paper"]
|
||||
hits = [w for w in banned if w in cap]
|
||||
hits = [w for w in BANNED_WORDS if w in cap]
|
||||
if hits:
|
||||
errors.append(f"{fname} caption contains illustration/design-board "
|
||||
f"vocabulary that produces cartoons: {hits}")
|
||||
@ -280,6 +454,28 @@ def main():
|
||||
errors.append(f"{fname} caption must pin the photorealistic "
|
||||
f"photographic style for the sheet")
|
||||
|
||||
# Frame image prompts
|
||||
if out_dir.exists():
|
||||
expected_frames = {f"Segment-img-{i}.json" for i in range(1, n + 1)}
|
||||
if args.mode == "fl2va":
|
||||
expected_frames |= {f"Segment-img-{i}-last.json" for i in range(1, n + 1)}
|
||||
for fname in sorted(expected_frames):
|
||||
fpath = out_dir / fname
|
||||
if not fpath.exists():
|
||||
kind = "last-frame" if "-last." in fname else "first-frame"
|
||||
errors.append(f"{kind} prompt missing: {fpath}")
|
||||
continue
|
||||
obj = load_two_key_json(fpath, errors)
|
||||
if obj is None:
|
||||
continue
|
||||
check_frame_caption(fname, obj, errors)
|
||||
stray = {p.name for p in out_dir.glob("Segment-img-*.json")} - expected_frames
|
||||
if stray:
|
||||
hint = (" (Segment-img-N-last.json files exist only in fl2va mode)"
|
||||
if args.mode == "ref2va" else "")
|
||||
errors.append(f"unexpected Segment-img files not matching {n} segment(s): "
|
||||
f"{sorted(stray)}{hint}")
|
||||
|
||||
# Lyrics must never appear
|
||||
if args.lyrics_file:
|
||||
lpath = Path(args.lyrics_file)
|
||||
@ -294,14 +490,15 @@ def main():
|
||||
f"{words[:60]}...")
|
||||
|
||||
if args.expect_audio and not any(audio_flags):
|
||||
errors.append("no segment references <Audio>; --expect-audio was set")
|
||||
errors.append("no segment reuses audio; --expect-audio was set")
|
||||
|
||||
if errors:
|
||||
print(f"FAIL: {len(errors)} problem(s)")
|
||||
for e in errors:
|
||||
print(f" - {e}")
|
||||
return 1
|
||||
print(f"PASS: {n} segment(s), subjects {sorted(subjects)}, "
|
||||
print(f"PASS: mode={args.mode}, {n} segment(s), "
|
||||
f"subjects {sorted(subjects) if subjects else 'per manifest'}, "
|
||||
f"audio={'yes' if any(audio_flags) else 'no'}")
|
||||
return 0
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user