Rename segmented mode to studio; tokens are @imageN/@audioN (lowercase)

This commit is contained in:
mm-prompter 2026-11-13 19:35:53 +00:00
parent 738b2e30e8
commit 0e4ab2db7c
5 changed files with 84 additions and 76 deletions

View File

@ -6,8 +6,8 @@ ready-to-run generation package:
- `segments.txt` — N segment prompts, joined by `|`. The video creator node splits the
total length evenly across segments (`N = max(1, round(total_seconds / 10))`) and
generates them one by one. Default format is full-reference (ref2va); FL2VA
(first–last frame) and segmented (one `segment-N/segment.txt` file per segment with
`@PictureN`/`@AudioN` tokens) are opt-in alternatives.
(first–last frame) and studio (one `segment-N/segment.txt` file per segment with
`@imageN`/`@audioN` tokens) are opt-in alternatives.
- `Image-N.json` — one 16:9 character-sheet text-to-image prompt per character that
has no user-provided image reference, consumed one at a time by a ComfyUI workflow.
- `Segment-img-N.json` — one 16:9 first-frame image prompt per segment, rendered by
@ -23,7 +23,7 @@ ready-to-run generation package:
| `SKILL.md` | Skill entry point: triggers, mode selection, pipeline, output contract, validation checklist |
| `references/ref2va-segment-format.md` | Authoritative per-segment format for the default mode (sections, labels, shots, audio template) |
| `references/fl2va-segment-format.md` | The opt-in FL2VA first–last frame video prompt format |
| `references/segmented-output-format.md` | The opt-in segmented layout: one `segment-N/segment.txt` per segment with `@PictureN`/`@AudioN` tokens, otherwise identical to ref2va |
| `references/studio-output-format.md` | The opt-in studio layout: one `segment-N/segment.txt` per segment with `@imageN`/`@audioN` tokens, otherwise identical to ref2va |
| `references/t2i-character-sheet.md` | Character-sheet caption rules (four-panel photorealistic turnaround) and the two-key JSON contract |
| `references/frame-image-prompts.md` | Per-segment first/last frame image prompt rules (`Segment-img-N.json`) |
| `references/worked-example.md` | Complete end-to-end example, including the audio (singing) and FL2VA variants |
@ -36,12 +36,12 @@ ready-to-run generation package:
2. Give the agent a vague scene prompt and the total video length in seconds, plus
optionally: song lyrics, an audio file mention (e.g. "use midnight.mp3"),
per-character image references, style hints, and — only if wanted — an explicit
request for FL2VA / first-last frame mode or segmented one-file-per-segment
output with `@PictureN`/`@AudioN` references.
request for FL2VA / first-last frame mode or studio one-file-per-segment
output with `@imageN`/`@audioN` references.
3. The agent writes `segments.txt`, `Image-N.json` for characters lacking references,
`Segment-img-N.json` (and `Segment-img-N-last.json` in FL2VA mode), and
`image-map.txt` into the project folder.
4. Verify before use: `python3 scripts/validate_outputs.py --segments segments.txt
--expected-n <N> [--dir .] [--lyrics-file lyrics.txt]` — add
`--mode fl2va --total-seconds <seconds>` for FL2VA runs, or
`--mode segmented --segments-dir <folder>` for segmented runs.
`--mode studio --segments-dir <folder>` for studio runs.

View File

@ -1,6 +1,6 @@
---
name: mm-prompter
description: Writes multi-segment ref2va video prompts plus T2I character-sheet prompts and per-segment first-frame image prompts from a vague scene idea and a total video length in seconds. Use when the user gives a rough video/scene concept (optionally with song lyrics or an audio file name) and wants numbered video-generation segments joined by "|", character sheets as Image-N.json for characters that lack an image reference, Segment-img-N.json frame prompts, and an image-map.txt manifest. Optional opt-in FL2VA mode adds last-frame prompts and first-last video prompts. Optional opt-in segmented mode writes one segment-N/segment.txt file per segment using @PictureN/@Audio1-style @-references instead of the joined file.
description: Writes multi-segment ref2va video prompts plus T2I character-sheet prompts and per-segment first-frame image prompts from a vague scene idea and a total video length in seconds. Use when the user gives a rough video/scene concept (optionally with song lyrics or an audio file name) and wants numbered video-generation segments joined by "|", character sheets as Image-N.json for characters that lack an image reference, Segment-img-N.json frame prompts, and an image-map.txt manifest. Optional opt-in FL2VA mode adds last-frame prompts and first-last video prompts. Optional opt-in studio mode writes one segment-N/segment.txt file per segment using @imageN/@audioN references instead of the joined file.
---
# MM Prompt Creator
@ -10,8 +10,8 @@ generation package:
1. `segments.txt` — N segment prompts, joined by `|`. The downstream video creator
node splits the total length evenly across segments and generates them one by one.
Default format is full-reference (ref2va); FL2VA (first–last frame) and segmented
(one `segment-N/segment.txt` file per segment with `@PictureN`/`@AudioN` tokens)
Default format is full-reference (ref2va); FL2VA (first–last frame) and studio
(one `segment-N/segment.txt` file per segment with `@imageN`/`@audioN` tokens)
are opt-in alternatives (see Mode selection).
2. `Image-N.json` — one 16:9 character-sheet text-to-image prompt per character that has
no user-provided image reference. Consumed one at a time by a ComfyUI workflow.
@ -37,12 +37,12 @@ example (including the audio and FL2VA variants) is in `references/worked-exampl
character sheets. The sheets are still generated: they feed the image side, which
renders the first/last frames with consistent characters. If the user's wording is
ambiguous, stay in ref2va mode; never mix the two formats in one run.
- **Segmented (opt-in).** Switch when the user asks for each segment as its own
`segment.txt` file or for `@Picture1` / `@Audio1`-style references. This is an
adaptation of ref2va defined in `references/segmented-output-format.md`: the
- **Studio (opt-in).** Switch when the user asks for studio mode, for each segment
as its own `segment.txt` file, or for `@image1` / `@audio1`-style references. This
is an adaptation of ref2va defined in `references/studio-output-format.md`: the
segment content follows `references/ref2va-segment-format.md` unchanged, but each
segment is written to its own `segment-N/segment.txt` file (no `segments.txt`, no
`|` separators) and reference tokens are written as `@PictureN` / `@AudioN`;
`|` separators) and reference tokens are written as `@imageN` / `@audioN`;
`<Subject N>` stays unchanged. Never mix formats in one run.
## Inputs
@ -130,16 +130,16 @@ not subjects.
**Step 9 — Assemble the segment outputs.** ref2va/FL2VA: strip each segment text and
join with a single `|` (no spaces added around it). Exactly n segments, exactly n−1
`|` characters, no trailing separator, no extra headers or numbering; the `|`
character must never appear inside a segment's own text. Segmented: instead write
character must never appear inside a segment's own text. Studio: instead write
each segment to its own `segment-N/segment.txt` file per
`references/segmented-output-format.md` — no joined file, no `|` anywhere, and
`<Picture N>` / `<Audio N>` written as `@PictureN` / `@AudioN` (`<Subject N>`
`references/studio-output-format.md` — no joined file, no `|` anywhere, and
`<Picture N>` / `<Audio N>` written as `@imageN` / `@audioN` (`<Subject N>`
unchanged).
**Step 10 — Validate** against the checklist below. When the environment can run Python,
also run `python scripts/validate_outputs.py --segments segments.txt --expected-n <n>`
(add `--mode fl2va --total-seconds <total>` in FL2VA mode; in segmented mode run
`python scripts/validate_outputs.py --mode segmented --segments-dir <folder> --expected-n <n>`).
(add `--mode fl2va --total-seconds <total>` in FL2VA mode; in studio mode run
`python scripts/validate_outputs.py --mode studio --segments-dir <folder> --expected-n <n>`).
## Output location
@ -203,10 +203,10 @@ segment in FL2VA mode, and none in ref2va mode); each is two-key JSON with
correct final shot index and two-decimal duration; then `integrated_multimodal_description:`,
`overall_soundscape:`, `non_diegetic_music:` in order, and none of the six ref2va
section labels appear.
12. (Segmented mode) Exactly n `segment-N/segment.txt` files, folders numbered 1..n
12. (Studio mode) Exactly n `segment-N/segment.txt` files, folders numbered 1..n
with no gaps; each file is the ref2va single-segment format with `<Picture N>` /
`<Audio N>` written as `@PictureN` / `@AudioN`, `<Subject N>` unchanged, and no `|`
character anywhere.
`<Audio N>` written as `@imageN` / `@audioN` (lowercase), `<Subject N>` unchanged,
and no `|` character anywhere.
## References
@ -214,8 +214,8 @@ character anywhere.
default mode (sections, labels, shots, audio template, retention markers).
- `references/fl2va-segment-format.md` — the opt-in FL2VA first–last frame prompt
format (instruction line, interpolation arc, dialogue and audio rules).
- `references/segmented-output-format.md` — the opt-in segmented layout: one
`segment-N/segment.txt` file per segment with `@PictureN` / `@AudioN` reference
- `references/studio-output-format.md` — the opt-in studio layout: one
`segment-N/segment.txt` file per segment with `@imageN` / `@audioN` reference
tokens; otherwise identical to the ref2va format.
- `references/t2i-character-sheet.md` — the four-panel photorealistic turnaround
sheet rules and the two-key JSON output contract.

View File

@ -1,20 +1,20 @@
# Segmented Output Format (one segment.txt per segment, optional)
# Studio Output Format (one segment.txt per segment, optional)
An adaptation of the ref2va format (`references/ref2va-segment-format.md`) for tools
that consume one prompt file per segment instead of one joined file. Opt in only when
the user asks for it (phrases like "each segment gets its own segment.txt", "one file
per segment", "@Picture1", "@Audio1"). Everything not changed below is exactly the
ref2va format — same six sections, same summary prefixes, same audio template and
lip-sync phrase, same shot rules, same sheets, frame prompts, and manifest. Never mix
formats in one run.
the user asks for studio mode (phrases like "studio", "each segment gets its own
segment.txt", "one file per segment", "@image1", "@audio1"). Everything not changed
below is exactly the ref2va format — same six sections, same summary prefixes, same
audio template and lip-sync phrase, same shot rules, same sheets, frame prompts, and
manifest. Never mix formats in one run.
## 1. What changes vs ref2va
| | ref2va (default) | segmented (opt-in) |
| | ref2va (default) | studio (opt-in) |
| --- | --- | --- |
| Output files | one `segments.txt`, n segments joined by bare `\|` | n files: `segment-1/segment.txt`, `segment-2/segment.txt`, ... |
| `\|` separators | exactly n−1, bare | none — the `\|` character never appears in any segment file |
| Reference tokens | `<Picture 1>`, `<Audio 1>` | `@Picture1`, `@Audio1` |
| Reference tokens | `<Picture 1>`, `<Audio 1>` | `@image1`, `@audio1` |
| Subject tokens | `<Subject N>` | `<Subject N>` — unchanged |
| Segment content | ref2va single-segment format | identical, one document per file |
@ -36,32 +36,34 @@ trailing whitespace. Folder numbers start at 1 with no gaps and no extra
## 3. Reference token syntax
Convert every reference token to its @-form — same number, no space, no angle
brackets:
brackets, all lowercase:
- `<Picture N>` → `@PictureN` (example: `<Picture 1>` → `@Picture1`)
- `<Audio N>` → `@AudioN` (example: `<Audio 1>` → `@Audio1`)
- `<Picture N>` → `@imageN` (example: `<Picture 1>` → `@image1`)
- `<Audio N>` → `@audioN` (example: `<Audio 1>` → `@audio1`)
The conversion applies everywhere the token appears: the `subject_definitions`
citations and audio definition line, `summary`, `retention_analysis` (including the
`fully_copy` line), and the lip-sync phrase inside `detailed_description`. Examples:
- `<Audio 1> is the user-supplied song file "midnight.mp3", ...` →
`@Audio1 is the user-supplied song file "midnight.mp3", ...`
`@audio1 is the user-supplied song file "midnight.mp3", ...`
- `<Subject 1> lips are synced to the audio in <Audio 1>` →
`<Subject 1> lips are synced to the audio in @Audio1`
`<Subject 1> lips are synced to the audio in @audio1`
- `<Audio 1>: fully_copy - <Audio 1> is reused 1:1 as the target video's complete
final audio track.` → `@Audio1: fully_copy - @Audio1 is reused 1:1 as the target
final audio track.` → `@audio1: fully_copy - @audio1 is reused 1:1 as the target
video's complete final audio track.`
`<Subject N>` is NOT converted: never write `@SubjectN`. The angle-bracket forms
`<Picture ...>` and `<Audio ...>` must not appear anywhere in a segmented file.
`<Subject N>` is NOT converted: never write `@subjectN`. The angle-bracket forms
`<Picture ...>` and `<Audio ...>` must not appear anywhere in a studio file, and
capitalized variants like `@Image1` or `@Audio1` are wrong — the tokens are exactly
`@imageN` and `@audioN`, lowercase.
## 4. Numbering
Subject and Picture numbering stays global across the whole video (1..M, gapless when
all segment files are taken together), exactly as in ref2va; each file defines only
the subjects that appear in it, using their global numbers. `Image-N.json` sheets and
`image-map.txt` are unchanged, and `@PictureN` still corresponds to the sheet mapped
`image-map.txt` are unchanged, and `@imageN` still corresponds to the sheet mapped
to Subject N.
## 5. Worked micro-example
@ -72,11 +74,11 @@ tokens converted:
```text
subject_definitions:
<Subject 1> is the astronaut in @Picture1, a woman in her thirties with short black hair, wearing a matte-white EVA suit with orange stripe accents and a wrist-mounted scanner.
<Subject 2> is the scout robot in @Picture2, a boxy machine on three wheels with a pale-grey shell and a single round blue lens eye.
<Subject 1> is the astronaut in @image1, a woman in her thirties with short black hair, wearing a matte-white EVA suit with orange stripe accents and a wrist-mounted scanner.
<Subject 2> is the scout robot in @image2, a boxy machine on three wheels with a pale-grey shell and a single round blue lens eye.
summary:
[reference generation] The target video shows <Subject 1> and <Subject 2> cresting a red dune at dusk and stopping before a monolith of black glass. <Subject 1>'s appearance and suit come from @Picture1 and <Subject 2>'s shell and lens come from @Picture2.
[reference generation] The target video shows <Subject 1> and <Subject 2> cresting a red dune at dusk and stopping before a monolith of black glass. <Subject 1>'s appearance and suit come from @image1 and <Subject 2>'s shell and lens come from @image2.
retention_analysis:
<Subject 1> (appears in [Shot 1]): fully_preserved - the matte-white EVA suit with orange stripe accents, short black hair, and the wrist-mounted scanner are retained.

View File

@ -291,11 +291,11 @@ same story state as segment 1's last frame (the handoff), worded independently.
---
# Segmented variant (one segment.txt per segment, @-references)
# Studio variant (one segment.txt per segment, @-references)
The same desert run in the opt-in segmented layout
(`references/segmented-output-format.md`): an adaptation of ref2va where each segment
is its own file and reference tokens are written `@PictureN` / `@AudioN`. The segment
The same desert run in the opt-in studio layout
(`references/studio-output-format.md`): an adaptation of ref2va where each segment
is its own file and reference tokens are written `@imageN` / `@audioN`. The segment
texts are the Segment 1 / Segment 2 texts from the default run above with only the
token syntax converted — content, sections, prefixes, and numbering are unchanged.
Sheets, frame prompts, and `image-map.txt` are identical to the default run.
@ -315,11 +315,11 @@ above):
```text
subject_definitions:
<Subject 1> is the astronaut in @Picture1, a woman in her thirties with short black hair and brown eyes, wearing a matte-white EVA suit with orange stripe accents, grey reinforced gloves, a clear dome helmet, and a wrist-mounted scanner on her left forearm.
<Subject 2> is the scout robot in @Picture2, a boxy machine on three wheels with a pale-grey shell, a single round blue lens eye, and a short antenna.
<Subject 1> is the astronaut in @image1, a woman in her thirties with short black hair and brown eyes, wearing a matte-white EVA suit with orange stripe accents, grey reinforced gloves, a clear dome helmet, and a wrist-mounted scanner on her left forearm.
<Subject 2> is the scout robot in @image2, a boxy machine on three wheels with a pale-grey shell, a single round blue lens eye, and a short antenna.
summary:
[reference generation] The target video shows <Subject 1> and <Subject 2> cresting a red dune at dusk and stopping before a monolith of black glass. <Subject 1>'s appearance and suit come from @Picture1 and <Subject 2>'s shell and lens come from @Picture2.
[reference generation] The target video shows <Subject 1> and <Subject 2> cresting a red dune at dusk and stopping before a monolith of black glass. <Subject 1>'s appearance and suit come from @image1 and <Subject 2>'s shell and lens come from @image2.
retention_analysis:
<Subject 1> (appears in [Shot 1]): fully_preserved - the matte-white EVA suit with orange stripe accents, grey reinforced gloves, short black hair, clear dome helmet, and the wrist-mounted scanner on her left forearm are retained.
@ -337,8 +337,8 @@ N/A
```
Note what did NOT change: `<Subject 1>` keeps its angle brackets (only `<Picture N>`
and `<Audio N>` become `@PictureN` / `@AudioN`), there is no `|` character anywhere,
and no joined `segments.txt` is written. Validate with
`python3 scripts/validate_outputs.py --mode segmented --segments-dir <folder>
and `<Audio N>` become `@imageN` / `@audioN`, lowercase), there is no `|` character
anywhere, and no joined `segments.txt` is written. Validate with
`python3 scripts/validate_outputs.py --mode studio --segments-dir <folder>
--expected-n 2`.

View File

@ -10,9 +10,10 @@ Checks the output contract described in SKILL.md:
- fl2va mode (opt-in): exact instruction line (final shot index + two-decimal
duration), the three fields in order, no reference labels, no ref2va
section labels, shot rules, fixed song sentences when a song is reused
- segmented mode (opt-in, ref2va adaptation): n files segment-N/segment.txt, no
'|' anywhere, @PictureN / @AudioN tokens (angle-bracket <Picture>/<Audio> and
@Subject are failures), otherwise the full ref2va per-segment checks
- studio mode (opt-in, ref2va adaptation): n files segment-N/segment.txt, no
'|' anywhere, @imageN / @audioN tokens (angle-bracket <Picture>/<Audio>,
@subject, and capitalized @Image/@Audio variants are failures), otherwise
the full ref2va per-segment checks
- image-map.txt manifest covers every subject; Image-N.json valid (two keys,
16:9, four-panel photorealistic sheet language)
- Segment-img-N.json per segment (both modes) and Segment-img-N-last.json
@ -24,7 +25,7 @@ Usage:
python3 scripts/validate_outputs.py --segments segments.txt [--expected-n 3] \
[--mode ref2va|fl2va] [--total-seconds 21] [--dir .] [--manifest image-map.txt] \
[--lyrics-file lyrics.txt] [--expect-audio]
python3 scripts/validate_outputs.py --mode segmented --segments-dir <folder> \
python3 scripts/validate_outputs.py --mode studio --segments-dir <folder> \
[--expected-n 3] [shared options] # folder holds segment-1/segment.txt ...
Exit code 0 = all checks passed, 1 = failures listed.
@ -77,8 +78,9 @@ FL2VA_SONG_SENTENCE_RE = re.compile(
)
FL2VA_LIPSYNC_RE = re.compile(r"\(S(\d+)\) lips stay synced to the song's vocals")
SEGMENTED_PICTURE_RE = re.compile(r"@Picture(\d+)")
SEGMENTED_AUDIO_RE = re.compile(r"@Audio(\d+)")
STUDIO_PICTURE_RE = re.compile(r"@image(\d+)")
STUDIO_AUDIO_RE = re.compile(r"@audio(\d+)")
STUDIO_BAD_CASE_RE = re.compile(r"@(Image\d|Audio\d|IMAGE\d|AUDIO\d)")
BANNED_WORDS = ["cartoon", "anime", "manga", "illustration", "stylized",
"cgi", "3d render", "mascot", "avatar", "name tag",
@ -303,20 +305,24 @@ def check_segment_fl2va(seg_text, seg_no, errors, expected_duration=None):
return has_song
def check_segment_segmented(seg_text, seg_no, errors):
"""Segmented mode: ref2va content with @-references, one file per segment."""
def check_segment_studio(seg_text, seg_no, errors):
"""Studio mode: ref2va content with @imageN/@audioN references, one file per segment."""
if "|" in seg_text:
fail(errors, seg_no, "segmented segment.txt must not contain the '|' "
fail(errors, seg_no, "studio segment.txt must not contain the '|' "
"separator (each segment is its own file)")
for bad, good in (("<Picture", "@PictureN"), ("<Audio", "@AudioN")):
for bad, good in (("<Picture", "@imageN"), ("<Audio", "@audioN")):
if bad in seg_text:
fail(errors, seg_no, f"segmented files use @-references: '{bad} N' must "
fail(errors, seg_no, f"studio files use @-references: '{bad} N' must "
f"be written as {good} (no space, no angle brackets)")
if "@Subject" in seg_text:
if "@Subject" in seg_text or "@subject" in seg_text:
fail(errors, seg_no, "subject labels are not converted: keep '<Subject N>', "
"never '@SubjectN'")
normalized = SEGMENTED_PICTURE_RE.sub(r"<Picture \1>", seg_text)
normalized = SEGMENTED_AUDIO_RE.sub(r"<Audio \1>", normalized)
"never '@subjectN'")
bad_case = STUDIO_BAD_CASE_RE.findall(seg_text)
if bad_case:
fail(errors, seg_no, f"reference tokens are lowercase: {bad_case} must be "
f"written as @imageN / @audioN")
normalized = STUDIO_PICTURE_RE.sub(r"<Picture \1>", seg_text)
normalized = STUDIO_AUDIO_RE.sub(r"<Audio \1>", normalized)
return check_segment_ref2va(normalized, seg_no, errors)
@ -366,11 +372,11 @@ def main():
ap.add_argument("--segments", required=False, default=None,
help="joined segments.txt (not used with --segments-dir)")
ap.add_argument("--segments-dir", default=None,
help="folder holding segment-1/segment.txt ... (segmented mode)")
help="folder holding segment-1/segment.txt ... (studio mode)")
ap.add_argument("--expected-n", type=int, default=None)
ap.add_argument("--mode", choices=["ref2va", "fl2va", "segmented"],
ap.add_argument("--mode", choices=["ref2va", "fl2va", "studio"],
default="ref2va",
help="ref2va (default), fl2va first-last, or segmented "
help="ref2va (default), fl2va first-last, or studio "
"per-file layout")
ap.add_argument("--total-seconds", type=float, default=None,
help="total video length; enables the S.SS duration check in "
@ -392,9 +398,9 @@ def main():
if args.segments_dir:
if args.mode == "fl2va":
print("FAIL: --segments-dir is the segmented layout; use --mode segmented")
print("FAIL: --segments-dir is the studio layout; use --mode studio")
return 1
args.mode = "segmented"
args.mode = "studio"
seg_dir = Path(args.segments_dir)
if not seg_dir.exists():
print(f"FAIL: {seg_dir} not found")
@ -448,8 +454,8 @@ def main():
for i, part in enumerate(parts, 1):
if args.mode == "fl2va":
has_audio = check_segment_fl2va(part.strip(), i, errors, expected_duration)
elif args.mode == "segmented":
defined, has_audio = check_segment_segmented(part.strip(), i, errors)
elif args.mode == "studio":
defined, has_audio = check_segment_studio(part.strip(), i, errors)
subjects |= defined
else:
defined, has_audio = check_segment_ref2va(part.strip(), i, errors)
@ -461,7 +467,7 @@ def main():
"video references")
all_nums = set(int(x) for x in SUBJECT_RE.findall(raw))
if args.mode in ("ref2va", "segmented") and all_nums and \
if args.mode in ("ref2va", "studio") and all_nums and \
all_nums != set(range(1, max(all_nums) + 1)):
errors.append(f"global Subject numbering must be gapless 1..{max(all_nums)} "
f"(found {sorted(all_nums)})")