- references/t2i-character-sheet.md: rewrite around the fixed four-panel layout (face close-up, front, side 90deg right, back) on a white background with no text/labels/borders; add photorealism rules and banned illustration vocabulary - worked-example.md: regenerate Image-1.json / Image-2.json in the new style - validate_outputs.py: guard sheet captions (four-panel structure, photorealistic language, banned cartoon/design-board vocabulary) - SKILL.md: step 6 + checklist item 9 updated accordingly
153 lines
8.2 KiB
Markdown
153 lines
8.2 KiB
Markdown
---
|
||
name: mm-prompter
|
||
description: Writes multi-segment ref2va video prompts plus T2I character-sheet prompts from a vague scene idea and a total video length in seconds. Use when the user gives a rough video/scene concept (optionally with song lyrics or an audio file name) and wants numbered video-generation segments joined by "|", character sheets as Image-N.json for characters that lack an image reference, and an image-map.txt manifest.
|
||
---
|
||
|
||
# MM Prompt Creator
|
||
|
||
Turn one vague prompt and one number (total seconds) into a complete, ready-to-run
|
||
generation package:
|
||
|
||
1. `segments.txt` — N segment prompts in full-reference (ref2va) format, joined by `|`.
|
||
The downstream video creator node splits the total length evenly across segments and
|
||
generates them one by one.
|
||
2. `Image-N.json` — one 16:9 character-sheet text-to-image prompt per character that has
|
||
no user-provided image reference. Consumed one at a time by a ComfyUI workflow.
|
||
3. `image-map.txt` — manifest that says which `<Subject N>` maps to which image file.
|
||
|
||
Read `references/ref2va-segment-format.md` before drafting the first segment, and
|
||
`references/t2i-character-sheet.md` before writing the first JSON. A complete end-to-end
|
||
example (including the audio variant) is in `references/worked-example.md`.
|
||
|
||
## Inputs
|
||
|
||
| Input | Required | Notes |
|
||
| --- | --- | --- |
|
||
| Vague prompt | yes | Rough scene/idea text from the user |
|
||
| Total video length in seconds | yes | Drives segment count |
|
||
| Song lyrics | no | Guide pacing only; NEVER appear in any output |
|
||
| Audio file mention | no | e.g. "use midnight.mp3" → enables `<Audio 1>` |
|
||
| Per-character image references | no | User-supplied files (e.g. `mary.png`) |
|
||
| Style hints | no | Honor if given; otherwise invent and lock |
|
||
|
||
Make reasonable creative choices and proceed. Do not block on questions; this skill
|
||
exists to expand vagueness into concrete, consistent detail.
|
||
|
||
## Scope rules
|
||
|
||
- Subjects are **recurring on-screen characters only**: people, animals, creatures,
|
||
robots that act as characters. Settings, props, and crowds stay in prose; they never
|
||
get `<Subject N>` numbers or sheets.
|
||
- Numbering is **global across the whole video**, assigned in order of first on-screen
|
||
appearance: `<Subject 1>`, `<Subject 2>`, ... For every subject, the image number
|
||
follows the subject number: Subject 2's reference is cited as `<Picture 2>` and, if
|
||
generated, saved as `Image-2.json`.
|
||
- A subject that does not appear in a segment must not be listed in that segment's
|
||
`subject_definitions`.
|
||
- No `<Video N>` labels: this skill never takes video input.
|
||
- Output language: English.
|
||
|
||
## Pipeline (follow in order)
|
||
|
||
**Step 1 — Intake.** Parse the vague prompt. List every character that will be on
|
||
screen. For each, decide: user-provided image reference (record filename) or needs a
|
||
generated sheet. Detect audio file mentions and lyrics. Detect style hints.
|
||
|
||
**Step 2 — Segment count.** `n = max(1, round(total_seconds / 10))` with round-half-up:
|
||
8s→1, 15s→2, 21s→2, 34s→3, 55s→6. Plan each segment as one beat of roughly
|
||
`total_seconds / n`. Assume segments flow into each other as one longer scene unless
|
||
the user says otherwise, so segment k's end state matches segment k+1's start state.
|
||
|
||
**Step 3 — Lock the story bible (before drafting).** Decide once, in a short working
|
||
note, and never change it afterward: each subject's hair, eyes, build, wardrobe, props,
|
||
and palette; the setting; time of day; lighting; color palette; camera character; style.
|
||
Where the vague prompt is silent, invent specifics and lock them. Every segment and
|
||
every sheet must use the identical locked values.
|
||
|
||
**Step 4 — Assign numbering.** Subject 1..M by first on-screen appearance across the
|
||
whole video. Map references: user file (e.g. `mary.png`) or `Image-N.json`.
|
||
|
||
**Step 5 — Draft each segment** exactly per `references/ref2va-segment-format.md`:
|
||
six sections in order, global labels, default one continuous `[Shot 1]` per segment,
|
||
extra shots only for time-synced changes, timestamps relative to the segment start.
|
||
When a subject sings to the provided audio, use the exact fixed audio template and the
|
||
exact lip-sync phrase defined in that reference.
|
||
|
||
**Step 6 — Write the character sheets** per `references/t2i-character-sheet.md`, one
|
||
`Image-N.json` per subject lacking a user reference. Every caption describes the
|
||
fixed four-panel photorealistic turnaround (face close-up, front, side facing 90°
|
||
right, back) on a white background with no text, no labels, and no borders — studio
|
||
photography of a real person, never a cartoon or designed card — and must reuse the
|
||
locked story-bible values verbatim.
|
||
|
||
**Step 7 — Write the manifest** `image-map.txt`, one line per subject:
|
||
|
||
```
|
||
Subject 1 - mary.png
|
||
Subject 2 - Image-2.json
|
||
```
|
||
|
||
User-supplied files keep their filenames; generated sheets are always `Image-N.json`.
|
||
|
||
**Step 8 — Assemble `segments.txt`.** Strip each segment text, join with a single `|`
|
||
(no spaces added around it). Exactly n segments, exactly n−1 `|` characters, no
|
||
trailing separator, no extra headers or numbering. The `|` character must never appear
|
||
inside a segment's own text.
|
||
|
||
**Step 9 — Validate** against the checklist below. When the environment can run Python,
|
||
also run `python scripts/validate_outputs.py --segments segments.txt --expected-n <n>`.
|
||
|
||
## Output location
|
||
|
||
Write all outputs (`segments.txt`, `Image-N.json`, `image-map.txt`) into the current
|
||
project folder of the project the skill is running on. If no project folder is
|
||
obvious, use the current working directory. Never write outputs into the skill's own
|
||
folder.
|
||
|
||
## Hard rules
|
||
|
||
1. Lyrics NEVER appear in any output text — not in `detailed_description`, not in
|
||
`<d>` tags, not in the audio sections. They only shape pacing and per-segment
|
||
content. Singing is expressed solely by the lip-sync phrase and the fixed audio
|
||
template.
|
||
2. Song vocals come from the reused audio: do not invent `(Sx)` speaker IDs or `<d>`
|
||
blocks for song lyrics. Ordinary spoken dialogue (non-song) may still use `(Sx)` +
|
||
`<d>[Language] ...</d>` per the format reference.
|
||
3. Global numbering is immutable across segments, sheets, and manifest.
|
||
4. The audio template and lip-sync phrase are used verbatim with only `N` substituted.
|
||
5. `[Shot 1]` never carries a timestamp; later shots use `At MM:SS.mmm` relative to the
|
||
segment start.
|
||
6. Every subject appearing anywhere in the video must have a manifest line.
|
||
|
||
## Validation checklist (all must pass before delivery)
|
||
|
||
1. `segments.txt` has exactly n segments and exactly n−1 `|` separators.
|
||
2. Each segment contains all six sections in order: `subject_definitions`, `summary`,
|
||
`retention_analysis`, `detailed_description`, `overall_soundscape`,
|
||
`non_diegetic_music`.
|
||
3. Subject numbering is global and gapless (1..M); `<Picture N>` cited inside
|
||
`<Subject N>` always uses the same N.
|
||
4. Every subject in the prompts has a manifest line; every `Image-N.json` in the
|
||
manifest exists, parses, has exactly the two keys `expanded_caption` and
|
||
`aspect_ratio`, and `aspect_ratio` is `"16:9"`. No orphan `Image-*.json` files.
|
||
5. No `<Video` anywhere; no lyric text anywhere.
|
||
6. Singing segments: summary prefix `[reference generation + audio reuse]`, the exact
|
||
fixed audio template in the sound sections, and the exact lip-sync phrase inside
|
||
`detailed_description`. Non-singing segments use `[reference generation]`.
|
||
7. `[Shot 1]` has no timestamp; any `At MM:SS.mmm` appears only on `[Shot N]`, N≥2.
|
||
8. Sheet captions and segment descriptions agree with the locked story bible.
|
||
9. Each sheet caption describes the four-panel photorealistic turnaround (face
|
||
close-up panel, full-body front, side facing 90° right, back) on a clean white
|
||
background with equal spacing and no text, no labels, no borders; it uses
|
||
photorealistic/photographic language and contains no cartoon, illustration, or
|
||
design-board vocabulary (cartoon, anime, mascot, name tag, swatch, palette strip,
|
||
design board, CGI, 3D render...).
|
||
|
||
## References
|
||
|
||
- `references/ref2va-segment-format.md` — authoritative per-segment format (sections,
|
||
labels, shots, audio template, retention markers).
|
||
- `references/t2i-character-sheet.md` — the four-panel photorealistic turnaround
|
||
sheet rules and the two-key JSON output contract.
|
||
- `references/worked-example.md` — full end-to-end example, including an audio variant.
|