14 KiB
| name | description |
|---|---|
| mm-prompter | Writes multi-segment ref2va video prompts plus T2I character-sheet prompts and per-segment first-frame image prompts from a vague scene idea and a total video length in seconds. Use when the user gives a rough video/scene concept (optionally with song lyrics or an audio file name) and wants numbered video-generation segments joined by "|", character sheets as Image-N.json for characters that lack an image reference, Segment-img-N.json frame prompts, and an image-map.txt manifest. Optional opt-in FL2VA mode adds last-frame prompts and first-last video prompts. Optional opt-in segmented mode writes one segment-N/segment.txt file per segment using @PictureN/@Audio1-style @-references instead of the joined file. |
MM Prompt Creator
Turn one vague prompt and one number (total seconds) into a complete, ready-to-run generation package:
segments.txt— N segment prompts, joined by|. The downstream video creator node splits the total length evenly across segments and generates them one by one. Default format is full-reference (ref2va); FL2VA (first–last frame) and segmented (onesegment-N/segment.txtfile per segment with@PictureN/@AudioNtokens) are opt-in alternatives (see Mode selection).Image-N.json— one 16:9 character-sheet text-to-image prompt per character that has no user-provided image reference. Consumed one at a time by a ComfyUI workflow.Segment-img-N.json— one 16:9 first-frame image prompt per segment (both modes); the image engine renders it using the character sheets as reference images. In FL2VA mode alsoSegment-img-N-last.jsonper segment for the last frame.image-map.txt— manifest that says which<Subject N>maps to which image file.
Read references/ref2va-segment-format.md before drafting the first segment and
references/t2i-character-sheet.md before writing the first JSON. When the run uses
FL2VA mode, also read references/fl2va-segment-format.md; always read
references/frame-image-prompts.md before writing frame files. A complete end-to-end
example (including the audio and FL2VA variants) is in references/worked-example.md.
Mode selection
- ref2va (default). Every run uses the six-section ref2va segment format unless the user explicitly opts into first–last frames.
- FL2VA (opt-in). Switch when the user asks for "FL2V", "FL2VA", "first-last",
"first and last frame", or "start and end frame" per segment. In FL2VA mode the
video side receives ONLY the segment's first-frame image, last-frame image, and the
FL2VA-format prompt defined in
references/fl2va-segment-format.md— never the character sheets. The sheets are still generated: they feed the image side, which renders the first/last frames with consistent characters. If the user's wording is ambiguous, stay in ref2va mode; never mix the two formats in one run. - Segmented (opt-in). Switch when the user asks for each segment as its own
segment.txtfile or for@Picture1/@Audio1-style references. This is an adaptation of ref2va defined inreferences/segmented-output-format.md: the segment content followsreferences/ref2va-segment-format.mdunchanged, but each segment is written to its ownsegment-N/segment.txtfile (nosegments.txt, no|separators) and reference tokens are written as@PictureN/@AudioN;<Subject N>stays unchanged. Never mix formats in one run.
Inputs
| Input | Required | Notes |
|---|---|---|
| Vague prompt | yes | Rough scene/idea text from the user |
| Total video length in seconds | yes | Drives segment count |
| Song lyrics | no | Guide pacing only; NEVER appear in any output |
| Audio file mention | no | e.g. "use midnight.mp3" → enables <Audio 1> |
| Per-character image references | no | User-supplied files (e.g. mary.png) |
| Style hints | no | Honor if given; otherwise invent and lock |
| Frame mode | no | Default ref2va; FL2VA (first–last) only when explicitly requested |
Make reasonable creative choices and proceed. Do not block on questions; this skill exists to expand vagueness into concrete, consistent detail.
Scope rules
- Subjects are recurring on-screen characters only: people, animals, creatures,
robots that act as characters. Settings, props, and crowds stay in prose; they never
get
<Subject N>numbers or sheets. - Numbering is global across the whole video, assigned in order of first on-screen
appearance:
<Subject 1>,<Subject 2>, ... For every subject, the image number follows the subject number: Subject 2's reference is cited as<Picture 2>and, if generated, saved asImage-2.json. - A subject that does not appear in a segment must not be listed in that segment's
subject_definitions. - No
<Video N>labels: this skill never takes video input. - Output language: English.
Pipeline (follow in order)
Step 1 — Intake. Parse the vague prompt. List every character that will be on screen. For each, decide: user-provided image reference (record filename) or needs a generated sheet. Detect audio file mentions and lyrics. Detect style hints.
Step 2 — Segment count. n = max(1, round(total_seconds / 10)) with round-half-up:
8s→1, 15s→2, 21s→2, 34s→3, 55s→6. Plan each segment as one beat of roughly
total_seconds / n. Assume segments flow into each other as one longer scene unless
the user says otherwise, so segment k's end state matches segment k+1's start state.
Step 3 — Lock the story bible (before drafting). Decide once, in a short working note, and never change it afterward: each subject's hair, eyes, build, wardrobe, props, and palette; the setting; time of day; lighting; color palette; camera character; style. Where the vague prompt is silent, invent specifics and lock them. Every segment and every sheet must use the identical locked values.
Step 4 — Assign numbering. Subject 1..M by first on-screen appearance across the
whole video. Map references: user file (e.g. mary.png) or Image-N.json.
Step 5 — Draft each segment in the selected mode. ref2va: exactly per
references/ref2va-segment-format.md — six sections in order, global labels, default
one continuous [Shot 1] per segment, extra shots only for time-synced changes,
timestamps relative to the segment start; when a subject sings to the provided audio,
use the exact fixed audio template and the exact lip-sync phrase defined in that
reference. FL2VA: exactly per references/fl2va-segment-format.md — instruction line
- three fields, single-shot interpolation arc from first-frame state to last-frame state, no reference labels.
Step 6 — Write the frame image prompts per references/frame-image-prompts.md:
one Segment-img-N.json per segment (N = 1..n) describing that segment's opening
state, and — FL2VA mode only — one Segment-img-N-last.json per segment describing
its final state. Captions reuse the locked story-bible values verbatim and anchor
each character to the supplied reference images.
Step 7 — Write the character sheets per references/t2i-character-sheet.md, one
Image-N.json per subject lacking a user reference. Every caption describes the
fixed four-panel photorealistic turnaround (face close-up, front, side facing 90°
right, back) on a white background with no text, no labels, and no borders — studio
photography of a real person, never a cartoon or designed card — and must reuse the
locked story-bible values verbatim.
Step 8 — Write the manifest image-map.txt, one line per subject:
Subject 1 - mary.png
Subject 2 - Image-2.json
User-supplied files keep their filenames; generated sheets are always Image-N.json.
The manifest never lists frame files — Segment-img-N numbering follows segments,
not subjects.
Step 9 — Assemble the segment outputs. ref2va/FL2VA: strip each segment text and
join with a single | (no spaces added around it). Exactly n segments, exactly n−1
| characters, no trailing separator, no extra headers or numbering; the |
character must never appear inside a segment's own text. Segmented: instead write
each segment to its own segment-N/segment.txt file per
references/segmented-output-format.md — no joined file, no | anywhere, and
<Picture N> / <Audio N> written as @PictureN / @AudioN (<Subject N>
unchanged).
Step 10 — Validate against the checklist below. When the environment can run Python,
also run python scripts/validate_outputs.py --segments segments.txt --expected-n <n>
(add --mode fl2va --total-seconds <total> in FL2VA mode; in segmented mode run
python scripts/validate_outputs.py --mode segmented --segments-dir <folder> --expected-n <n>).
Output location
Write all outputs (segments.txt, Image-N.json, Segment-img-N.json frames,
Segment-img-N-last.json in FL2VA mode, image-map.txt) into the current project
folder of the project the skill is running on. If no project folder is
obvious, use the current working directory. Never write outputs into the skill's own
folder.
Hard rules
- Lyrics NEVER appear in any output text — not in
detailed_description, not in<d>tags, not in the audio sections. They only shape pacing and per-segment content. Singing is expressed solely by the lip-sync phrase and the fixed audio template. - Song vocals come from the reused audio: do not invent
(Sx)speaker IDs or<d>blocks for song lyrics. Ordinary spoken dialogue (non-song) may still use(Sx)+<d>[Language] ...</d>per the format reference. - Global numbering is immutable across segments, sheets, and manifest.
- The audio template and lip-sync phrase are used verbatim with only
Nsubstituted. [Shot 1]never carries a timestamp; later shots useAt MM:SS.mmmrelative to the segment start.- Every subject appearing anywhere in the video must have a manifest line.
- Frame captions describe single full-bleed 16:9 film stills in the locked style — photorealistic photographic language, no sheet/panel vocabulary, no text or borders, no lyrics, no timestamps or labels, and every on-screen character anchored to the supplied reference images with verbatim story-bible traits.
- FL2VA mode: the instruction line of every segment is the exact fixed template with
only the final shot index and the two-decimal duration substituted; the prompt
contains no
<Subject N>/<Picture N>/<Video N>/<Audio N>labels.
Validation checklist (all must pass before delivery)
segments.txthas exactly n segments and exactly n−1|separators.- (ref2va mode) Each segment contains all six sections in order:
subject_definitions,summary,retention_analysis,detailed_description,overall_soundscape,non_diegetic_music. - (ref2va mode) Subject numbering is global and gapless (1..M);
<Picture N>cited inside<Subject N>always uses the same N. In FL2VA mode numbering lives on the image side (manifest + sheets) only. - Every subject in the prompts has a manifest line; every
Image-N.jsonin the manifest exists, parses, has exactly the two keysexpanded_captionandaspect_ratio, andaspect_ratiois"16:9". No orphanImage-*.jsonfiles. - No
<Videoanywhere; no lyric text anywhere. - (ref2va mode) Singing segments: summary prefix
[reference generation + audio reuse], the exact fixed audio template in the sound sections, and the exact lip-sync phrase insidedetailed_description. Non-singing segments use[reference generation]. [Shot 1]has no timestamp; anyAt MM:SS.mmmappears only on[Shot N], N≥2.- Sheet captions and segment descriptions agree with the locked story bible.
- Each sheet caption describes the four-panel photorealistic turnaround (face close-up panel, full-body front, side facing 90° right, back) on a clean white background with equal spacing and no text, no labels, no borders; it uses photorealistic/photographic language and contains no cartoon, illustration, or design-board vocabulary (cartoon, anime, mascot, name tag, swatch, palette strip, design board, CGI, 3D render...).
- One
Segment-img-N.jsonexists per segment (plusSegment-img-N-last.jsonper segment in FL2VA mode, and none in ref2va mode); each is two-key JSON with"aspect_ratio": "16:9", photorealistic language, no banned vocabulary. - FL2VA mode: each segment's first line is the exact FL2VA instruction with the
correct final shot index and two-decimal duration; then
integrated_multimodal_description:,overall_soundscape:,non_diegetic_music:in order, and none of the six ref2va section labels appear. - (Segmented mode) Exactly n
segment-N/segment.txtfiles, folders numbered 1..n with no gaps; each file is the ref2va single-segment format with<Picture N>/<Audio N>written as@PictureN/@AudioN,<Subject N>unchanged, and no|character anywhere.
References
references/ref2va-segment-format.md— authoritative per-segment format for the default mode (sections, labels, shots, audio template, retention markers).references/fl2va-segment-format.md— the opt-in FL2VA first–last frame prompt format (instruction line, interpolation arc, dialogue and audio rules).references/segmented-output-format.md— the opt-in segmented layout: onesegment-N/segment.txtfile per segment with@PictureN/@AudioNreference tokens; otherwise identical to the ref2va format.references/t2i-character-sheet.md— the four-panel photorealistic turnaround sheet rules and the two-key JSON output contract.references/frame-image-prompts.md— per-segment first/last frame image prompt rules (Segment-img-N.json,Segment-img-N-last.json).references/worked-example.md— full end-to-end example, including audio and FL2VA variants.