Switch plan.json output to the native H3 ref2va prompt format
Each scene prompt is now the six native reference sections — subject_definitions / summary [reference generation] / retention_analysis / detailed_description (1-2 style sentences then [Shot 1]) / overall_soundscape / non_diegetic_music — one line per element, headers on their own line. The full head-to-toe look (nude-not-bare) lives in subject_definitions and is restated compressed in the [Shot 1] body; both cast members are defined per scene against the wired subject references. verify_plan_rules.py auto-detects the format per shot and validates the ref2va contract (canonical order, directive, definitions, retention, style prefix, thin floor), so legacy t2va plans keep passing. Both system prompts, the assembler prompt and SKILL.md carry the new skeleton, worked examples and self-checks; example_output/README.md marks the old run as the t2va craft reference.
This commit is contained in:
parent
e5778f8536
commit
30cbaeb8f6
8
example_output/README.md
Normal file → Executable file
8
example_output/README.md
Normal file → Executable file
@ -4,6 +4,14 @@ A complete reference run of the `h3-plan-gate` agent workflow (Step 1 through St
|
||||
on a 12-shot story — also a worked demonstration that the workflow scales past the
|
||||
3-scene example in `h3_agent_workflow_examples/` to 10+ scenes with no code changes.
|
||||
|
||||
> **Format note:** this run was written in the legacy **t2va** three-line prompt format
|
||||
> (`integrated_multimodal_description` / `overall_soundscape` / `non_diegetic_music`).
|
||||
> The skill has since switched to the native H3 **ref2va** full-reference format
|
||||
> (`subject_definitions` / `summary` / `retention_analysis` / `detailed_description` /
|
||||
> `overall_soundscape` / `non_diegetic_music`, `summary: [reference generation]`). The
|
||||
> validator auto-detects the format per shot, so this run still passes unchanged; use
|
||||
> it as a craft reference, not a format reference.
|
||||
|
||||
## Input
|
||||
|
||||
> story: in a functional universe there is a new way of handling criminals when found
|
||||
|
||||
@ -107,9 +107,19 @@ Each fragment is exactly this shape:
|
||||
"id": "from the outline row",
|
||||
"duration_seconds": 15,
|
||||
"prompt": [
|
||||
"integrated_multimodal_description: [Shot 1] ...",
|
||||
"overall_soundscape: ...",
|
||||
"non_diegetic_music: N/A"
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is ...full head-to-toe look, nude for uncovered skin...",
|
||||
"summary:",
|
||||
"[reference generation]",
|
||||
"retention_analysis:",
|
||||
"<Subject 1>: fully_preserved",
|
||||
"detailed_description:",
|
||||
"...1-2 style sentences from the guide's style...",
|
||||
"[Shot 1] ...the full scene composition...",
|
||||
"overall_soundscape:",
|
||||
"...the sound the scene itself makes...",
|
||||
"non_diegetic_music:",
|
||||
"N/A"
|
||||
],
|
||||
"_handoff": { "end_state": "...", "soundscape": "...", "dialogue_used": [] }
|
||||
}
|
||||
@ -150,15 +160,18 @@ and sampler latent or the video can come out square.
|
||||
|
||||
- Raw JSON only: first char `{`, last char `}`, no markdown fences, no prose, double
|
||||
quotes, no trailing commas, no raw line break inside a string.
|
||||
- The `prompt` array has EXACTLY three lines in this order; every scene is its own single
|
||||
H3 generation, so the bracket number is always `[Shot 1]`:
|
||||
`integrated_multimodal_description:` / `overall_soundscape:` / `non_diegetic_music:`.
|
||||
- The description line is self-sufficient: re-state `<Subject 1>` and the full look in
|
||||
EVERY scene — identity, hair, build, every garment head to toe, footwear, and the
|
||||
visual style — nothing is shared between scenes or from a prefix. Name every body
|
||||
part the clothing leaves uncovered and call uncovered skin "nude", never "bare"
|
||||
- The `prompt` array carries the six native H3 ref2va sections in canonical order —
|
||||
`subject_definitions:` / `summary:` / `retention_analysis:` / `detailed_description:` /
|
||||
`overall_soundscape:` / `non_diegetic_music:` — one line per element, headers on their
|
||||
own line; `summary` is exactly `[reference generation]`; every scene is its own single
|
||||
H3 generation, so the bracket number is always `[Shot 1]`.
|
||||
- The full look is re-stated in EVERY scene — subject_definitions carries it complete
|
||||
(identity, hair, build, every garment head to toe, footwear), and the [Shot 1] body
|
||||
opens by re-stating `<Subject 1>` with a wardrobe+skin restatement — nothing is
|
||||
shared between scenes or from a prefix. Name every body part the clothing leaves
|
||||
uncovered and call uncovered skin "nude", never "bare"
|
||||
("a man wearing a white tank-top with nude arms").
|
||||
- Scene composition, not word count: the description line must carry every element —
|
||||
- Scene composition, not word count: the [Shot 1] body must carry every element —
|
||||
subject anchor (`<Subject 1>` and wardrobe restated), style anchor, continuity anchor
|
||||
(opens inside the previous end_state), blocking, one camera idea, performance beats,
|
||||
physical texture (wardrobe behavior, weather on skin and fabric, lighting), and a
|
||||
@ -202,12 +215,12 @@ and sampler latent or the video can come out square.
|
||||
are dialogue and never live in the soundscape line.
|
||||
- Dialogue: silence is the default. Spoken words come only from the outline row or the
|
||||
story, reproduced word for word inside case-sensitive `<d>[English] words</d>` (one space
|
||||
after the marker; other real languages allowed), inside the description line only, with
|
||||
after the marker; other real languages allowed), inside the [Shot 1] body only, with
|
||||
a "silently with mouth {position}" cue before and after every span. Mirror each span in
|
||||
`_handoff.dialogue_used` (words only, no marker); empty array when silent.
|
||||
- `_handoff` must be truthful: `end_state` describes the real final frame (2–3 sentences:
|
||||
pose, motion in progress, camera state); `soundscape` copies the overall_soundscape line
|
||||
character for character.
|
||||
pose, motion in progress, camera state); `soundscape` copies the soundscape content
|
||||
line (the element after the `overall_soundscape:` header) character for character.
|
||||
- Keys are exactly `id`, `duration_seconds`, `prompt`, `_handoff`; add `steps` or `seed`
|
||||
only when the user explicitly asked. Every duration 2–15 s.
|
||||
|
||||
|
||||
@ -43,10 +43,10 @@ Emit exactly this shape and no other keys:
|
||||
"defaults": { "duration_seconds": 10, "steps": 20 },
|
||||
"shots": [
|
||||
{ "id": "scene_one_name",
|
||||
"prompt": ["...the fragment's three prompt lines..."],
|
||||
"prompt": ["...the fragment's prompt lines, copied exactly..."],
|
||||
"duration_seconds": 10 },
|
||||
{ "id": "scene_two_name",
|
||||
"prompt": ["...the fragment's three prompt lines..."],
|
||||
"prompt": ["...the fragment's prompt lines, copied exactly..."],
|
||||
"duration_seconds": 10 }
|
||||
]
|
||||
}
|
||||
@ -64,22 +64,23 @@ Emit exactly this shape and no other keys:
|
||||
- Copy any extra "steps" or "seed" keys a fragment carries, unchanged.
|
||||
- If any fragment is broken JSON, stop and reply exactly: BROKEN FRAGMENT <n>
|
||||
|
||||
# WORKED EXAMPLE (shape only — real fragments carry full 200-word description lines)
|
||||
# WORKED EXAMPLE (shape only — real fragments carry full ref2va sections and
|
||||
# 200-word description bodies)
|
||||
Input:
|
||||
|
||||
DEFAULTS: {"duration_seconds": 8, "steps": 20}
|
||||
|
||||
SCENE 1
|
||||
{"id": "corridor_walk", "duration_seconds": 8, "prompt": ["integrated_multimodal_description: [Shot 1] <Subject 1>, the night nurse in pale green scrubs, pushes the trolley down the corridor.", "overall_soundscape: low fluorescent hum, a distant trolley wheel squeak.", "non_diegetic_music: N/A"], "_handoff": {"end_state": "She is mid-stride at the far end.", "soundscape": "overall_soundscape: low fluorescent hum, a distant trolley wheel squeak.", "dialogue_used": []}}
|
||||
{"id": "corridor_walk", "duration_seconds": 8, "prompt": ["subject_definitions:", "<Subject 1> is a night nurse in pale green scrubs and white sneakers, with nude hands below the scrub cuffs.", "summary:", "[reference generation]", "retention_analysis:", "<Subject 1>: fully_preserved", "detailed_description:", "Photorealistic cool fluorescent lighting, soft clinical palette.", "[Shot 1] <Subject 1>, the night nurse in pale green scrubs, pushes the trolley down the corridor.", "overall_soundscape:", "low fluorescent hum, a distant trolley wheel squeak.", "non_diegetic_music:", "N/A"], "_handoff": {"end_state": "She is mid-stride at the far end.", "soundscape": "low fluorescent hum, a distant trolley wheel squeak.", "dialogue_used": []}}
|
||||
|
||||
SCENE 2
|
||||
{"id": "door_stop", "duration_seconds": 8, "prompt": ["integrated_multimodal_description: [Shot 1] Continue the same walk as <Subject 1>, the same night nurse in pale green scrubs, stops the trolley at the blue door.", "overall_soundscape: the same fluorescent hum continues, the trolley wheel falls silent.", "non_diegetic_music: N/A"], "_handoff": {"end_state": "Her hand rests on the push bar.", "soundscape": "overall_soundscape: the same fluorescent hum continues, the trolley wheel falls silent.", "dialogue_used": []}}
|
||||
{"id": "door_stop", "duration_seconds": 8, "prompt": ["subject_definitions:", "<Subject 1> is a night nurse in pale green scrubs and white sneakers, with nude hands below the scrub cuffs.", "summary:", "[reference generation]", "retention_analysis:", "<Subject 1>: fully_preserved", "detailed_description:", "Photorealistic cool fluorescent lighting, soft clinical palette.", "[Shot 1] Continue the same walk as <Subject 1>, the same night nurse in pale green scrubs, stops the trolley at the blue door.", "overall_soundscape:", "the same fluorescent hum continues, the trolley wheel falls silent.", "non_diegetic_music:", "N/A"], "_handoff": {"end_state": "Her hand rests on the push bar.", "soundscape": "the same fluorescent hum continues, the trolley wheel falls silent.", "dialogue_used": []}}
|
||||
|
||||
Correct reply:
|
||||
|
||||
{"prompt_prefix": [""], "defaults": {"duration_seconds": 8, "steps": 20}, "shots": [
|
||||
{"id": "corridor_walk", "prompt": ["integrated_multimodal_description: [Shot 1] <Subject 1>, the night nurse in pale green scrubs, pushes the trolley down the corridor.", "overall_soundscape: low fluorescent hum, a distant trolley wheel squeak.", "non_diegetic_music: N/A"], "duration_seconds": 8},
|
||||
{"id": "door_stop", "prompt": ["integrated_multimodal_description: [Shot 1] Continue the same walk as <Subject 1>, the same night nurse in pale green scrubs, stops the trolley at the blue door.", "overall_soundscape: the same fluorescent hum continues, the trolley wheel falls silent.", "non_diegetic_music: N/A"], "duration_seconds": 8}
|
||||
{"id": "corridor_walk", "prompt": ["subject_definitions:", "<Subject 1> is a night nurse in pale green scrubs and white sneakers, with nude hands below the scrub cuffs.", "summary:", "[reference generation]", "retention_analysis:", "<Subject 1>: fully_preserved", "detailed_description:", "Photorealistic cool fluorescent lighting, soft clinical palette.", "[Shot 1] <Subject 1>, the night nurse in pale green scrubs, pushes the trolley down the corridor.", "overall_soundscape:", "low fluorescent hum, a distant trolley wheel squeak.", "non_diegetic_music:", "N/A"], "duration_seconds": 8},
|
||||
{"id": "door_stop", "prompt": ["subject_definitions:", "<Subject 1> is a night nurse in pale green scrubs and white sneakers, with nude hands below the scrub cuffs.", "summary:", "[reference generation]", "retention_analysis:", "<Subject 1>: fully_preserved", "detailed_description:", "Photorealistic cool fluorescent lighting, soft clinical palette.", "[Shot 1] Continue the same walk as <Subject 1>, the same night nurse in pale green scrubs, stops the trolley at the blue door.", "overall_soundscape:", "the same fluorescent hum continues, the trolley wheel falls silent.", "non_diegetic_music:", "N/A"], "duration_seconds": 8}
|
||||
]}
|
||||
|
||||
Note what the example shows: the "_handoff" objects vanished, every prompt string is
|
||||
@ -105,8 +106,9 @@ DEFAULTS block.
|
||||
`plan_json_input` (a non-empty connected value overrides the visual editor's plan for
|
||||
that execution and receives the identical validation).
|
||||
- **QA after assembly:** `python scripts/verify_plan_rules.py plan plan.json` re-runs the
|
||||
node-rule replica plus the H3 three-line / word-budget / handoff-free checks on the
|
||||
final file. The deterministic merger runs all of this automatically.
|
||||
node-rule replica plus the H3 ref2va six-section (or legacy t2va three-line, detected
|
||||
per shot) / word-budget / handoff-free checks on the final file. The deterministic
|
||||
merger runs all of this automatically.
|
||||
- **Token footprint (measured, o200k proxy):** the prompt is ~1.2k tokens; input is
|
||||
roughly the sum of all fragments (~0.5k per scene); output is nearly the same size
|
||||
again. A 12-scene assembly turn is ~12k tokens — fine at 128k ctx; if you ever run
|
||||
|
||||
@ -50,13 +50,23 @@ Emit exactly this shape and no other keys:
|
||||
"shots": [
|
||||
{ "id": "scene_one",
|
||||
"prompt": [
|
||||
"integrated_multimodal_description: [Shot 1] ...",
|
||||
"overall_soundscape: ...",
|
||||
"non_diegetic_music: N/A"
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is ...full head-to-toe look, nude for uncovered skin...",
|
||||
"summary:",
|
||||
"[reference generation]",
|
||||
"retention_analysis:",
|
||||
"<Subject 1>: fully_preserved",
|
||||
"detailed_description:",
|
||||
"<1-2 style sentences from the story's look>",
|
||||
"[Shot 1] ...the full scene composition...",
|
||||
"overall_soundscape:",
|
||||
"...the sound the scene itself makes...",
|
||||
"non_diegetic_music:",
|
||||
"N/A"
|
||||
],
|
||||
"duration_seconds": 10 },
|
||||
{ "id": "scene_two",
|
||||
"prompt": [ "...the same three AUDIO lines with this scene's content..." ],
|
||||
"prompt": [ "...the same ref2va sections with this scene's content..." ],
|
||||
"duration_seconds": 10 }
|
||||
]
|
||||
}
|
||||
@ -64,7 +74,8 @@ Emit exactly this shape and no other keys:
|
||||
# TOP-LEVEL FIELDS
|
||||
- "prompt_prefix": always exactly [""] — one empty string, nothing else. Subject identity,
|
||||
wardrobe, art style, camera style, and continuity rules belong inside every scene's
|
||||
integrated_multimodal_description line instead, so each scene is self-sufficient.
|
||||
subject_definitions and detailed_description sections instead, so each scene is
|
||||
self-sufficient.
|
||||
- "defaults": object with exactly two keys: "duration_seconds" (positive number) and "steps"
|
||||
(integer). Use 10 and 20 unless the user says otherwise.
|
||||
- "shots": array of shot objects in story order. Usually 3 to 12 shots. If the user names a
|
||||
@ -74,7 +85,8 @@ Emit exactly this shape and no other keys:
|
||||
Each shot object has exactly these keys:
|
||||
- "id": unique short snake_case name. Lowercase letters, digits, underscores only.
|
||||
Every id must be different. Example: "kitchen_argument".
|
||||
- "prompt": array of exactly three strings in the AUDIO order given below. Never empty.
|
||||
- "prompt": array of strings in the REF2VA PROMPT order given below — one line per
|
||||
element, section headers on their own line. Never empty.
|
||||
- "duration_seconds": positive number between 2 and 15. Omit it to inherit the default.
|
||||
Only when the user explicitly asks, a shot may also include:
|
||||
- "steps": integer from 1 to 10000.
|
||||
@ -112,21 +124,24 @@ Shots are flat. A shot never contains a nested object.
|
||||
|
||||
# PROMPT CRAFT
|
||||
- "prompt_prefix" stays empty, exactly [""]. Never put identity, wardrobe, style, or
|
||||
continuity text in it. Every scene's integrated_multimodal_description line must be
|
||||
self-sufficient: introduce or re-state the <Subject 1> tag, wardrobe, look, and visual
|
||||
style in every scene, because nothing is shared from a prefix.
|
||||
- The integrated_multimodal_description line of every scene must:
|
||||
continuity text in it. Every scene's prompt must be self-sufficient:
|
||||
subject_definitions carries the full look and the [Shot 1] body re-states the
|
||||
<Subject 1> tag, wardrobe, and look in every scene, because nothing is shared from
|
||||
a prefix or between scenes.
|
||||
- The [Shot 1] body of every scene's detailed_description must:
|
||||
1. Continue the exact action, pose, camera direction, and lighting from the previous
|
||||
scene's description line. Scene 1 instead sets the opening.
|
||||
scene's [Shot 1] body. Scene 1 instead sets the opening.
|
||||
2. Describe this scene's new action concretely: blocking, camera move, emotion, physics.
|
||||
3. End mid-action or on a movement still in progress so the next scene can continue it.
|
||||
Only the final scene may resolve and come to rest.
|
||||
- Compose every scene from the same eight elements, and never chase a word count —
|
||||
chase the elements. A description that genuinely carries them comes out long enough
|
||||
on its own; padding to a number is as much a failure as a thin one-liner:
|
||||
1. SUBJECT ANCHOR — <Subject 1> restated: identity, every aspect of how they look,
|
||||
wardrobe head to toe, hair, build — naming every body part the clothing leaves
|
||||
uncovered, using "nude" for uncovered skin (PROMPT CRAFT).
|
||||
1. SUBJECT ANCHOR — the full look lives in subject_definitions (REF2VA PROMPT
|
||||
below): identity, every aspect of how they look, wardrobe head to toe, hair,
|
||||
build — naming every body part the clothing leaves uncovered with "nude"
|
||||
(PROMPT CRAFT). The [Shot 1] body then opens by re-stating <Subject 1> with a
|
||||
compressed wardrobe+skin restatement.
|
||||
2. STYLE ANCHOR — the one consistent look: medium, lighting quality, palette.
|
||||
3. CONTINUITY ANCHOR — open inside the previous scene's end state (scene 1 sets the
|
||||
opening) and name what carries over: the stride, the camera direction, the light,
|
||||
@ -190,40 +205,60 @@ Shots are flat. A shot never contains a nested object.
|
||||
fades until only the doorway lamp lights the room". A summarized change or movement
|
||||
is a gap H3 fills by guessing, and guessed motion drifts off intent.
|
||||
- Describe the complete look in every scene: every aspect of how the person looks —
|
||||
face, hair, build, each garment head to toe, footwear — because nothing carries
|
||||
over between scenes. Any body part the clothing leaves uncovered must be named and
|
||||
called "nude", never "bare": not "a man wearing a white tank-top" but "a man
|
||||
wearing a white tank-top with nude arms"; likewise nude legs under shorts, nude
|
||||
shoulders under a sleeveless top, nude feet in sandals. "Nude" is the reliable
|
||||
face, hair, build, each garment head to toe, footwear — goes in the
|
||||
subject_definitions section, because nothing carries over between scenes. Any body
|
||||
part the clothing leaves uncovered must be named and called "nude", never "bare":
|
||||
not "a man wearing a white tank-top" but "a man wearing a white tank-top with nude
|
||||
arms"; likewise nude legs under shorts, nude shoulders under a sleeveless top, nude
|
||||
feet in sandals. The [Shot 1] body opens by re-stating <Subject 1> with the same
|
||||
wardrobe+skin wording compressed into its first sentence. "Nude" is the reliable
|
||||
skin-state word for uncovered body parts; "bare" is not.
|
||||
- One string per line in the prompt array. A shot's array holds exactly the three AUDIO
|
||||
lines below; never add blank entries, extra prose lines, or a second section label.
|
||||
- One line per element in the prompt array. A shot's array holds exactly the six ref2va
|
||||
sections below; never add blank entries, extra prose lines, or a second section header.
|
||||
- Useful continuity phrases: "Continue the same stride", "without resetting the motion",
|
||||
"keep the incoming camera direction", "same lighting", "no cut".
|
||||
- Only mention reference tags like <Picture 1>, <Video 1>, <Audio 1> if the user's request
|
||||
describes reference images, videos, or audio. Otherwise write self-sufficient prose and
|
||||
never invent tags.
|
||||
- <Subject 1> (and <Subject 2> when the story casts a second recurring person) are
|
||||
defined in subject_definitions and used in the [Shot 1] body — they resolve against
|
||||
the subject reference images the user wires in ComfyUI. Never invent <Picture N>,
|
||||
<Video N>, or <Audio N> tags unless the user's request actually provides those
|
||||
references.
|
||||
- Keep one consistent style across the whole plan. Never switch style mid-plan.
|
||||
|
||||
# AUDIO (every shot prompt is a three-line H3 prompt)
|
||||
- Each scene is rendered as ONE H3 generation, and the H3 model reserves special tokens for
|
||||
three exact section labels. Give every shot a prompt array with exactly these three lines,
|
||||
in this order:
|
||||
1. "integrated_multimodal_description: [Shot 1] ..." — one line holding subject identity,
|
||||
wardrobe, visual style, all action, camera work, continuity cues, and any <d> dialogue.
|
||||
Every scene is its own single-shot call, so the bracketed number is always [Shot 1].
|
||||
2. "overall_soundscape: ..." — the sound the scene itself makes: weather, room tone,
|
||||
footsteps, surfaces, machines, animals, crowd, and diegetic music always named with its
|
||||
source inside the scene, e.g. "a battery radio on the shelf plays soft instrumental
|
||||
jazz".
|
||||
3. "non_diegetic_music: N/A" — write N/A unless the user asks for background music that
|
||||
nobody in the scene can hear; only then give one short clause describing it.
|
||||
# REF2VA PROMPT (every shot prompt is the six native H3 reference sections)
|
||||
- Each scene is rendered as ONE H3 generation built from the subject reference images
|
||||
the user has wired, and the H3 model reserves special tokens for six exact section
|
||||
labels. Give every shot a prompt array with exactly these sections, in this order,
|
||||
one line per element — the header on its own line, the content on the following
|
||||
element:
|
||||
1. "subject_definitions:" then one line per subject: "<Subject 1> is ..." carrying
|
||||
the full head-to-toe look in PROMPT CRAFT wording ("nude" for every uncovered
|
||||
body part). Define <Subject 2> the same way when the story casts a second
|
||||
recurring person. Never define a label twice; never define a label you do not
|
||||
use.
|
||||
2. "summary:" then "[reference generation]" — exactly this directive on every scene,
|
||||
nothing else in the section.
|
||||
3. "retention_analysis:" then one line per defined label: "<Subject 1>:
|
||||
fully_preserved" (alternatives only when the scene really calls for them:
|
||||
partially_preserved, attribute_transfer, weak_reference; default
|
||||
fully_preserved).
|
||||
4. "detailed_description:" then 1-2 style sentences (medium, lighting quality,
|
||||
palette — the look the whole plan shares), then the "[Shot 1] ..." body — one
|
||||
line holding the subject restatement, visual style cues, all action, camera
|
||||
work, continuity cues, and any <d> dialogue. Every scene is its own single-shot
|
||||
call, so the bracketed number is always [Shot 1].
|
||||
5. "overall_soundscape:" then the sound the scene itself makes: weather, room tone,
|
||||
footsteps, surfaces, machines, animals, crowd, and diegetic music always named
|
||||
with its source inside the scene, e.g. "a battery radio on the shelf plays soft
|
||||
instrumental jazz".
|
||||
6. "non_diegetic_music:" then "N/A" — write N/A unless the user asks for background
|
||||
music that nobody in the scene can hear; only then give one short clause
|
||||
describing it.
|
||||
- Scene 1 establishes the soundscape. Later scenes keep the same audio bed and change only
|
||||
what the picture changes; say "the same rain continues" instead of inventing new weather.
|
||||
- The soundscape must match what the picture shows. No visible or implied source, no sound.
|
||||
- Diegetic music stays instrumental unless the user gave lyrics. Sung or spoken words are
|
||||
speech: they follow DIALOGUE, live in the description line, and never appear in the
|
||||
soundscape line.
|
||||
speech: they follow DIALOGUE, live in the [Shot 1] body, and never appear in the
|
||||
soundscape section.
|
||||
- Keep each soundscape to one or two dense clauses so audio prose never crowds out action.
|
||||
|
||||
# DIALOGUE (silence is the default)
|
||||
@ -242,9 +277,9 @@ Shots are flat. A shot never contains a nested object.
|
||||
- Approved markers: [English] [French] [Spanish] [German] [Italian] [Portuguese]
|
||||
[Chinese] [Japanese] [Korean] [Arabic]. Another real language name in brackets also
|
||||
works, like [Hindi]. Never use [unclear].
|
||||
- Keep each <d>...</d> span inside ONE prompt array entry, always the
|
||||
integrated_multimodal_description line. Never split a span across two entries, never put
|
||||
dialogue in a soundscape line, and never put a raw line break inside a span.
|
||||
- Keep each <d>...</d> span inside ONE prompt array entry, always the [Shot 1] element
|
||||
of detailed_description. Never split a span across two entries, never put dialogue in
|
||||
the soundscape section, and never put a raw line break inside a span.
|
||||
- Bracket every dialogue span with mouth-state cues, or H3 chatters gibberish around
|
||||
the line. Use the phrase "silently with mouth" plus a concrete position:
|
||||
1. Before the span: the speaker is silent with a stated mouth position, then speaks.
|
||||
@ -269,57 +304,85 @@ Correct reply:
|
||||
{
|
||||
"id": "greenhouse_arrival",
|
||||
"prompt": [
|
||||
"integrated_multimodal_description: [Shot 1] Opening scene. <Subject 1>, the
|
||||
adult bicycle courier, steps off his silver bicycle at the wet gravel path
|
||||
outside the glass greenhouse and walks it slowly through the entrance, right
|
||||
hand on the handlebar, left thumb hooked under the strap of the olive
|
||||
messenger bag as the kraft paper parcel presses against his hip. The camera
|
||||
is a stable landscape tracking shot at eye level, gliding sideways at his
|
||||
exact walking pace, keeping his face, the parcel, and the fogged glass wall
|
||||
behind him in frame; the camera height and the direction of travel never
|
||||
change. Photorealistic overcast rainy daylight filters through the fogged panes and lays
|
||||
soft grey light across his mustard-yellow waterproof jacket; rain beads and
|
||||
long streaks run down the glass, and the wet paving glistens under the
|
||||
bicycle wheels. Droplets dot his charcoal helmet, his dark teal trousers
|
||||
darken at the shins, and each footstep squeezes a small splash between the
|
||||
gravel stones. He breathes steadily with a relaxed jaw, eyes flicking up to
|
||||
the aisle numbers as he passes benches of potted ferns and clay pots, and
|
||||
the kraft paper shifts faintly against the bag strap with every step. End
|
||||
while he is crossing the interior threshold, front wheel still rolling over
|
||||
the concrete sill, coat sleeve mid-swing, motion clearly still in progress.",
|
||||
"overall_soundscape: steady light rain drumming on glass roofing, wet tires
|
||||
crunching over wet paving, low distant traffic hum under a grey sky.",
|
||||
"non_diegetic_music: N/A"
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is an adult bicycle courier with a lean build and short dark
|
||||
brown hair, wearing a mustard-yellow waterproof jacket zipped to the collar,
|
||||
dark teal trousers, and a charcoal helmet, silver-grey sneakers, and an olive
|
||||
messenger bag across his chest, with nude hands below the jacket cuffs and
|
||||
nude ears under the helmet straps.",
|
||||
"summary:",
|
||||
"[reference generation]",
|
||||
"retention_analysis:",
|
||||
"<Subject 1>: fully_preserved",
|
||||
"detailed_description:",
|
||||
"Photorealistic overcast rainy daylight, soft grey palette, stable landscape
|
||||
tracking shots at eye level.",
|
||||
"[Shot 1] Opening scene. <Subject 1>, the adult bicycle courier, steps off his
|
||||
silver bicycle at the wet gravel path outside the glass greenhouse and walks
|
||||
it slowly through the entrance, right hand on the handlebar, left thumb
|
||||
hooked under the strap of the olive messenger bag as the kraft paper parcel
|
||||
presses against his hip. The camera is a stable landscape tracking shot at
|
||||
eye level, gliding sideways at his exact walking pace, keeping his face, the
|
||||
parcel, and the fogged glass wall behind him in frame; the camera height and
|
||||
the direction of travel never change. Photorealistic overcast rainy daylight
|
||||
filters through the fogged panes and lays soft grey light across his
|
||||
mustard-yellow waterproof jacket; rain beads and long streaks run down the
|
||||
glass, and the wet paving glistens under the bicycle wheels. Droplets dot his
|
||||
charcoal helmet, his dark teal trousers darken at the shins, and each
|
||||
footstep squeezes a small splash between the gravel stones. He breathes
|
||||
steadily with a relaxed jaw, eyes flicking up to the aisle numbers as he
|
||||
passes benches of potted ferns and clay pots, and the kraft paper shifts
|
||||
faintly against the bag strap with every step. End while he is crossing the
|
||||
interior threshold, front wheel still rolling over the concrete sill, coat
|
||||
sleeve mid-swing, motion clearly still in progress.",
|
||||
"overall_soundscape:",
|
||||
"steady light rain drumming on glass roofing, wet tires crunching over wet
|
||||
paving, low distant traffic hum under a grey sky.",
|
||||
"non_diegetic_music:",
|
||||
"N/A"
|
||||
],
|
||||
"duration_seconds": 15
|
||||
},
|
||||
{
|
||||
"id": "greenhouse_delivery",
|
||||
"prompt": [
|
||||
"integrated_multimodal_description: [Shot 1] Continue the same forward stride
|
||||
and the same sideways camera move through the already-crossed threshold
|
||||
without resetting the motion, wet gravel giving way to polished concrete
|
||||
underfoot. <Subject 1>, the same adult bicycle courier in his mustard-yellow
|
||||
waterproof jacket, dark teal trousers, and charcoal helmet, rolls the silver
|
||||
bicycle to the wooden potting table at the end of the aisle, wedges the front
|
||||
wheel against the table leg with his left hand, unbuckles the olive messenger
|
||||
flap, lifts the kraft paper parcel with both
|
||||
hands, and sets it down on the table with a soft, deliberate press. The
|
||||
camera glides to a stop at a calm medium-wide angle as he straightens to his
|
||||
full height and turns his shoulders toward the lens. He stands silently with
|
||||
mouth closed, then speaks. <d>[English] Package for the greenhouse, right on
|
||||
time.</d> He falls silently with mouth closed and lets a brief smile lift the
|
||||
corners of his mouth before it fades. Overhead, the photorealistic overcast rain
|
||||
continues to drum on the glass roof and the grey light has not shifted;
|
||||
condensation trails keep
|
||||
sliding down the panes behind him, his jacket shoulders stay dotted with
|
||||
droplets, and the bicycle stands dripping quietly at his side, front wheel
|
||||
slowly ticking to a stop. End on the calm wide composition as the delivery
|
||||
completes, his two fingers still resting on the parcel's edge.",
|
||||
"overall_soundscape: the same rain on glass and wet paving continues, tires
|
||||
creak to a stop, the kraft paper parcel thuds softly on the wooden potting
|
||||
table.",
|
||||
"non_diegetic_music: N/A"
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is an adult bicycle courier with a lean build and short dark
|
||||
brown hair, wearing a mustard-yellow waterproof jacket zipped to the collar,
|
||||
dark teal trousers, and a charcoal helmet, silver-grey sneakers, and an olive
|
||||
messenger bag across his chest, with nude hands below the jacket cuffs and
|
||||
nude ears under the helmet straps.",
|
||||
"summary:",
|
||||
"[reference generation]",
|
||||
"retention_analysis:",
|
||||
"<Subject 1>: fully_preserved",
|
||||
"detailed_description:",
|
||||
"Photorealistic overcast rainy daylight, soft grey palette, stable landscape
|
||||
tracking shots at eye level.",
|
||||
"[Shot 1] Continue the same forward stride and the same sideways camera move
|
||||
through the already-crossed threshold without resetting the motion, wet
|
||||
gravel giving way to polished concrete underfoot. <Subject 1>, the same adult
|
||||
bicycle courier in his mustard-yellow waterproof jacket, dark teal trousers,
|
||||
and charcoal helmet, rolls the silver bicycle to the wooden potting table at
|
||||
the end of the aisle, wedges the front wheel against the table leg with his
|
||||
left hand, unbuckles the olive messenger flap, lifts the kraft paper parcel
|
||||
with both hands, and sets it down on the table with a soft, deliberate press.
|
||||
The camera glides to a stop at a calm medium-wide angle as he straightens to
|
||||
his full height and turns his shoulders toward the lens. He stands silently
|
||||
with mouth closed, then speaks. <d>[English] Package for the greenhouse,
|
||||
right on time.</d> He falls silently with mouth closed and lets a brief smile
|
||||
lift the corners of his mouth before it fades. Overhead, the photorealistic
|
||||
overcast rain continues to drum on the glass roof and the grey light has not
|
||||
shifted; condensation trails keep sliding down the panes behind him, his
|
||||
jacket shoulders stay dotted with droplets, and the bicycle stands dripping
|
||||
quietly at his side, front wheel slowly ticking to a stop. End on the calm
|
||||
wide composition as the delivery completes, his two fingers still resting on
|
||||
the parcel's edge.",
|
||||
"overall_soundscape:",
|
||||
"the same rain on glass and wet paving continues, tires creak to a stop, the
|
||||
kraft paper parcel thuds softly on the wooden potting table.",
|
||||
"non_diegetic_music:",
|
||||
"N/A"
|
||||
],
|
||||
"duration_seconds": 15
|
||||
}
|
||||
@ -336,10 +399,11 @@ Correct reply:
|
||||
5. shots is non-empty; ids are unique snake_case; every prompt array has a non-empty line.
|
||||
6. Every duration_seconds is between 2 and 15.
|
||||
7. Scene 1 sets an opening; every later scene continues the previous final action; every
|
||||
scene except the last ends mid-action. Every description line is self-sufficient: it
|
||||
re-states who <Subject 1> is, the full look, and visual style, because the prefix is
|
||||
empty, naming every uncovered body part as "nude" — never "bare".
|
||||
8. Every description line carries every element of a scene composition — subject,
|
||||
scene except the last ends mid-action. subject_definitions re-defines <Subject 1>
|
||||
with the full look in every scene, and the [Shot 1] body re-states who <Subject 1>
|
||||
is with a wardrobe+skin restatement, because the prefix is empty — naming every
|
||||
uncovered body part as "nude", never "bare".
|
||||
8. Every [Shot 1] body carries every element of a scene composition — subject,
|
||||
style, continuity, blocking, camera, performance, texture, exit state — and no
|
||||
change anywhere is stated as a bare outcome like "is now"; every transformation is
|
||||
shown as a continuous on-screen progression. Length is a side effect of the
|
||||
@ -349,17 +413,17 @@ Correct reply:
|
||||
wardrobe — is spelled out step by step the same way, never summarized. No
|
||||
common action is only mentioned — unbuttoning, diaper changes, lamp switches,
|
||||
doors, straps, and hand-offs all appear as their on-screen steps.
|
||||
9. Every shot's prompt array has exactly three lines in this order:
|
||||
integrated_multimodal_description: [Shot 1] ... / overall_soundscape: ... /
|
||||
non_diegetic_music: ...
|
||||
9. Every shot's prompt array carries the six ref2va sections in canonical order —
|
||||
subject_definitions / summary / retention_analysis / detailed_description /
|
||||
overall_soundscape / non_diegetic_music — each header on its own line, and
|
||||
summary is exactly "[reference generation]".
|
||||
10. overall_soundscape matches what the picture shows; the same audio bed continues across
|
||||
scenes; non_diegetic_music: is N/A unless the user asked for a score; no lyrics anywhere
|
||||
unless the user wrote them.
|
||||
11. There are no <d> spans at all unless the user's story provided spoken words; if it
|
||||
did, the text inside every <d>...</d> matches the user's words exactly.
|
||||
12. Every spoken line is wrapped as <d>[Language] words</d> inside the
|
||||
integrated_multimodal_description entry, with a "silently with mouth" cue before and
|
||||
after the span.
|
||||
12. Every spoken line is wrapped as <d>[Language] words</d> inside the [Shot 1] body,
|
||||
with a "silently with mouth" cue before and after the span.
|
||||
```
|
||||
|
||||
---
|
||||
@ -382,7 +446,7 @@ A retired boxer teaches his granddaughter to dance in their cluttered kitchen.
|
||||
| LLM settings | Temperature 0.3–0.7. If your server supports `response_format: {"type": "json_object"}` or a grammar/JSON-mode constraint, enable it — this prompt is fully compatible with it. |
|
||||
| LLM context window | The prompt itself is ~3.8k tokens (measured); with the 200-word-per-scene description minimum, generated plans add ~0.5k tokens per scene — a 12-shot plan turn is ~9.5k tokens. Set ctx ≥ 16384 for 12-shot plans; 8192 still covers ≤8 shots. (User runs at 128k — far more than enough for any plan size, including 128-shot maximums.) |
|
||||
| Dialogue format | Spoken lines go in `<d>[Language] words</d>` (case-sensitive tags, marker + one space; language list verified against the node's own `h3_prompt_schema_core.mjs` and `h3_prompt_completion_core.mjs`). Each span is bracketed by "silently with mouth {position}" cues before and after — the repo's `AUDIO_AND_CONTINUITY.md` and MiniMax's official H3 guide both require describing mouth state in the prompt, and without it H3 chirps/gibberishes around dialogue. |
|
||||
| Soundscape / H3 sections | Every shot prompt is written in the **native H3 template**: `integrated_multimodal_description: [Shot 1] …` → `overall_soundscape: …` → `non_diegetic_music: …` — order fixed, verified against `web/h3_prompt_schema_core.mjs` (`H3_BASE_SECTIONS`, kept aligned with H3 tokenizer special tokens per ComfyUI PR #15808). The chain node does **not** add these labels — it compiles each shot as `prompt_prefix` + blank line + scene prompt verbatim (`chain_nodes.py`), so the labels must be inside the JSON prompt lines. Diegetic music (in-scene source) goes in the soundscape line; score goes in `non_diegetic_music` only when the user asks. Audio is heard only if the node's Generation Profile actually generates it (e.g. "Generate audio"). |
|
||||
| Soundscape / H3 sections | Every shot prompt is written in the **native H3 ref2va (full-reference) template**: `subject_definitions:` → `summary:` (`[reference generation]`) → `retention_analysis:` → `detailed_description:` (1–2 style sentences, then `[Shot 1] …`) → `overall_soundscape:` → `non_diegetic_music:` — order fixed, verified against `web/h3_prompt_schema_core.mjs` (`H3_REFERENCE_SECTIONS`, kept aligned with H3 tokenizer special tokens per ComfyUI PR #15808). The chain node does **not** add these labels — it compiles each shot as `prompt_prefix` + blank line + scene prompt verbatim (`chain_nodes.py`), so the labels must be inside the JSON prompt lines, headers on their own line. `<Subject N>` definitions resolve against the subject reference images the user wires (Scheduled Ref2VA / `native_ref2va`). Diegetic music (in-scene source) goes in the soundscape section; score goes in `non_diegetic_music` only when the user asks. Audio is heard only if the node's Generation Profile actually generates it (e.g. "Generate audio"). |
|
||||
| Resolution / aspect | **Not a plan-JSON field.** `_normalize_plan` takes width/height only from the Plan node's `width`/`height` widgets (defaults 960×544, step 32, max 4096); a JSON `compatibility` block is ignored and per-shot `resolution` applies only via chapters (which this prompt forbids). To change aspect: set the Plan widgets (e.g. 960×544 ≈16:9, 896×672 4:3, 544×960 portrait) **and** connect the Plan/Current Shot width & height outputs to the H3 conditioning node and sampler latent — README: "Connect Current Shot width/height to the H3 conditioning node." Disconnected widgets leave whatever value (often square) is baked into the saved workflow. |
|
||||
| Caching behavior | The plan omits `prompt_seed_mode` and `chapters`, so the node's `IS_CHANGED` stays cache-stable between identical queue runs. |
|
||||
|
||||
@ -396,21 +460,23 @@ A retired boxer teaches his granddaughter to dance in their cluttered kitchen.
|
||||
- **Exact frame control:** if you ever need frame-exact scenes, allow `"length"` with values
|
||||
on the 17k+5 grid (5, 22, 39, 56 … 3592) — but for a local model, `duration_seconds` is far
|
||||
safer; the node rounds up for you.
|
||||
- **Reference-tag prompts:** the tag rule (`<Picture N>` / `<Video N>` / `<Audio N>` only when
|
||||
the user describes references) matches how Scheduled Ref2VA aliases compile; tags that don't
|
||||
correspond to real wiring will hurt generation, hence the strict gate.
|
||||
- **Prose-only shot prompts:** the three-line AUDIO template is the native H3 format and the
|
||||
strongest audio steering. If you ever want plain prose shots instead, delete the AUDIO
|
||||
section and its self-check items — the node passes prose through unchanged (that is how the
|
||||
repo's own chain templates are written), but H3 then loses its dedicated soundscape and
|
||||
music slots and audio quality is down to luck.
|
||||
- **Reference-tag prompts:** `<Subject N>` definitions are the ref2va reference format
|
||||
and resolve against the subject reference images wired in ComfyUI (Scheduled
|
||||
Ref2VA / `native_ref2va`). The `<Picture N>` / `<Video N>` / `<Audio N>` gate stays
|
||||
strict: tags that don't correspond to real wiring will hurt generation.
|
||||
- **Prose-only shot prompts:** the six-section REF2VA template is the native H3
|
||||
full-reference format and the strongest steering. If you ever want plain prose shots
|
||||
instead, delete the REF2VA section and its self-check items — the node passes prose
|
||||
through unchanged, but H3 then loses its dedicated subject, summary, retention,
|
||||
soundscape, and music slots and both identity and audio quality are down to luck.
|
||||
- **Empty prompt_prefix (empirical):** manual A/B testing showed much better H3 results when
|
||||
the prefix is exactly [""] — the compiled shot prompt (`prefix + blank line + scene prompt`,
|
||||
`chain_nodes.py`) then starts directly at the trained `integrated_multimodal_description:`
|
||||
header instead of a prose preamble. Identity, wardrobe, and style therefore live inside
|
||||
every description line, restated per scene — which also suits each scene being its own
|
||||
generation. The validator accepts an empty prefix as long as every scene prompt is
|
||||
non-empty. If you reintroduce prose, expect some format dilution.
|
||||
`chain_nodes.py`) then starts directly at the trained `subject_definitions:` header
|
||||
instead of a prose preamble. Identity, wardrobe, and style therefore live inside
|
||||
every scene's subject_definitions and [Shot 1] body, restated per scene — which also
|
||||
suits each scene being its own generation. The validator accepts an empty prefix as
|
||||
long as every scene prompt is non-empty. If you reintroduce prose, expect some
|
||||
format dilution.
|
||||
|
||||
## Verified constraint appendix (from source, for maintenance)
|
||||
|
||||
|
||||
@ -164,9 +164,19 @@ Emit exactly this shape and no other keys:
|
||||
"id": "scene_two_name",
|
||||
"duration_seconds": 15,
|
||||
"prompt": [
|
||||
"integrated_multimodal_description: [Shot 1] ...",
|
||||
"overall_soundscape: ...",
|
||||
"non_diegetic_music: N/A"
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is ...full head-to-toe look, nude for uncovered skin...",
|
||||
"summary:",
|
||||
"[reference generation]",
|
||||
"retention_analysis:",
|
||||
"<Subject 1>: fully_preserved",
|
||||
"detailed_description:",
|
||||
"<1-2 style sentences from the outline's style>",
|
||||
"[Shot 1] ...the full scene composition...",
|
||||
"overall_soundscape:",
|
||||
"...the sound the scene itself makes...",
|
||||
"non_diegetic_music:",
|
||||
"N/A"
|
||||
],
|
||||
"_handoff": {
|
||||
"end_state": "...",
|
||||
@ -177,14 +187,15 @@ Emit exactly this shape and no other keys:
|
||||
|
||||
- "id" and "duration_seconds": copied from your scene's outline row. Never invent them
|
||||
when an outline is given.
|
||||
- "prompt": the three H3 lines defined in AUDIO below.
|
||||
- "prompt": the ref2va section lines defined in REF2VA PROMPT below — one line per
|
||||
element, section headers on their own line.
|
||||
- "_handoff": planning metadata for the NEXT scene's turn. It is stripped before
|
||||
rendering and never reaches H3. Fill it truthfully from what you actually wrote:
|
||||
"end_state" — 2-3 sentences describing the exact visible state of your final frame:
|
||||
pose, the motion still in progress, camera state, where everything stands. The next
|
||||
scene opens from this.
|
||||
"soundscape" — your overall_soundscape line copied character for character, label
|
||||
included.
|
||||
"soundscape" — your soundscape content line (the element after the
|
||||
"overall_soundscape:" header) copied character for character, without the header.
|
||||
"dialogue_used" — the spoken words you used, in order, without the [Language] marker;
|
||||
empty array when the scene is silent.
|
||||
- Plus "steps" or "seed" only when the user explicitly asked; never otherwise.
|
||||
@ -204,8 +215,9 @@ lowercase snake_case id and a duration of 5 to 15 seconds; without STYLE AND CAS
|
||||
identity and style from the story alone; without a handoff you are scene 1.
|
||||
|
||||
# IDENTITY AND CONTINUITY
|
||||
- Every scene is generated alone: nothing is shared between scenes. Your description
|
||||
line must be self-sufficient — re-state <Subject 1>, the full wardrobe, and the visual
|
||||
- Every scene is generated alone: nothing is shared between scenes. Your prompt must
|
||||
be self-sufficient — subject_definitions carries the full head-to-toe look, and the
|
||||
[Shot 1] body re-states <Subject 1> with a wardrobe+skin restatement and the visual
|
||||
style in every scene, even though the previous scene already did.
|
||||
- Copy the subject descriptions and "style" from STYLE AND CAST faithfully. Never change
|
||||
wardrobe, hair, age, build, or look mid-plan.
|
||||
@ -220,16 +232,18 @@ identity and style from the story alone; without a handoff you are scene 1.
|
||||
can continue it. Only the final outline scene may resolve and come to rest.
|
||||
|
||||
# SCENE COMPOSITION (why you write one scene per turn)
|
||||
Your whole turn exists so this ONE scene can be composed with full attention. Build the
|
||||
integrated_multimodal_description line from every element below; a scene missing an
|
||||
Your whole turn exists so this ONE scene can be composed with full attention. Build
|
||||
the [Shot 1] body of detailed_description from every element below; a scene missing an
|
||||
element is a failed scene. Do not chase a word count — chase the elements. A
|
||||
description that genuinely carries them comes out long enough on its own; padding to
|
||||
body that genuinely carries them comes out long enough on its own; padding to
|
||||
a number is as much a failure as a thin one-liner.
|
||||
|
||||
1. SUBJECT ANCHOR — <Subject 1> restated: identity, every aspect of how they look,
|
||||
wardrobe head to toe, hair, build — and name every body part the clothing leaves
|
||||
uncovered, using "nude" for uncovered skin (PROMPT CRAFT). Nothing is shared
|
||||
between scenes or from a prefix.
|
||||
1. SUBJECT ANCHOR — the full look lives in subject_definitions (REF2VA PROMPT
|
||||
below): identity, every aspect of how they look, wardrobe head to toe, hair,
|
||||
build — naming every body part the clothing leaves uncovered with "nude"
|
||||
(PROMPT CRAFT). The [Shot 1] body then opens by re-stating <Subject 1> with a
|
||||
compressed wardrobe+skin restatement. Nothing is shared between scenes or from
|
||||
a prefix.
|
||||
2. STYLE ANCHOR — the one consistent look: medium, lighting quality, palette.
|
||||
3. CONTINUITY ANCHOR — open inside the previous end_state (scene 1 instead
|
||||
establishes place, light, and the first action). Name what carries over: the
|
||||
@ -247,8 +261,9 @@ a number is as much a failure as a thin one-liner.
|
||||
outline scene may come to rest), concrete enough that the next scene opens
|
||||
from it.
|
||||
|
||||
The sound picture is its own element and lives in the overall_soundscape line (AUDIO
|
||||
below); dialogue lives inside the description line (DIALOGUE below).
|
||||
The sound picture is its own element and lives in the overall_soundscape section
|
||||
(REF2VA PROMPT below); dialogue lives inside the [Shot 1] body of detailed_description
|
||||
(DIALOGUE below).
|
||||
|
||||
# PROMPT CRAFT
|
||||
- Never compress a change into a stated outcome. A line like "<Subject 1> is now an 18
|
||||
@ -299,41 +314,58 @@ below); dialogue lives inside the description line (DIALOGUE below).
|
||||
fades until only the doorway lamp lights the room". A summarized change or movement
|
||||
is a gap H3 fills by guessing, and guessed motion drifts off intent.
|
||||
- Describe the complete look in every scene: every aspect of how the person looks —
|
||||
face, hair, build, each garment head to toe, footwear — because each scene is a
|
||||
separate generation and nothing carries over. Any body part the clothing leaves
|
||||
uncovered must be named and called "nude", never "bare": not "a man wearing a
|
||||
white tank-top" but "a man wearing a white tank-top with nude arms"; likewise nude
|
||||
legs under shorts, nude shoulders under a sleeveless top, nude feet in sandals.
|
||||
face, hair, build, each garment head to toe, footwear — goes in the
|
||||
subject_definitions section, because each scene is a separate generation and nothing
|
||||
carries over. Any body part the clothing leaves uncovered must be named and called
|
||||
"nude", never "bare": not "a man wearing a white tank-top" but "a man wearing a
|
||||
white tank-top with nude arms"; likewise nude legs under shorts, nude shoulders
|
||||
under a sleeveless top, nude feet in sandals. The [Shot 1] body opens by re-stating
|
||||
<Subject 1> with the same wardrobe+skin wording compressed into its first sentence.
|
||||
"Nude" is the reliable skin-state word for uncovered body parts; "bare" is not.
|
||||
- One string per line in the prompt array: exactly the three AUDIO lines; never add
|
||||
blank entries, extra prose lines, or a second section label.
|
||||
- One line per element in the prompt array: the six ref2va sections exactly as
|
||||
specified in REF2VA PROMPT below; never add blank entries, extra prose lines, or a
|
||||
second section header.
|
||||
- Useful continuity phrases: "Continue the same stride", "without resetting the motion",
|
||||
"keep the incoming camera direction", "same lighting", "no cut".
|
||||
- Only mention reference tags like <Picture 1>, <Video 1>, <Audio 1> if the user's
|
||||
request describes reference images, videos, or audio. Otherwise write self-sufficient
|
||||
prose and never invent tags.
|
||||
- <Subject 1> (and <Subject 2> when the outline casts a second person) are defined in
|
||||
subject_definitions and used in the [Shot 1] body — they resolve against the subject
|
||||
reference images the user wires in ComfyUI. Never invent <Picture N>, <Video N>, or
|
||||
<Audio N> tags unless the user's request actually provides those references.
|
||||
|
||||
# AUDIO (every scene prompt is a three-line H3 prompt)
|
||||
- Each scene is rendered as ONE H3 generation, and the H3 model reserves special tokens
|
||||
for three exact section labels. Give your prompt array exactly these three lines, in
|
||||
this order:
|
||||
1. "integrated_multimodal_description: [Shot 1] ..." — one line holding subject
|
||||
identity, wardrobe, visual style, all action, camera work, continuity cues, and any
|
||||
<d> dialogue. Every scene is its own single-shot call, so the bracketed number is
|
||||
always [Shot 1].
|
||||
2. "overall_soundscape: ..." — the sound the scene itself makes: weather, room tone,
|
||||
footsteps, surfaces, machines, animals, crowd, and diegetic music always named with
|
||||
its source inside the scene, e.g. "a battery radio on the shelf plays soft
|
||||
# REF2VA PROMPT (every scene prompt is the six native H3 reference sections)
|
||||
- Each scene is rendered as ONE H3 generation built from the subject reference images
|
||||
the user has wired, and the H3 model reserves special tokens for six exact section
|
||||
labels. Give your prompt array exactly these sections, in this order, one line per
|
||||
element — the header on its own line, the content on the following element:
|
||||
1. "subject_definitions:" then one line per subject: "<Subject 1> is ..." carrying
|
||||
the full head-to-toe look in PROMPT CRAFT wording ("nude" for every uncovered
|
||||
body part). Define <Subject 2> the same way when the outline casts a second
|
||||
person. Never define a label twice; never define a label you do not use.
|
||||
2. "summary:" then "[reference generation]" — exactly this directive on every scene
|
||||
of this workflow, nothing else in the section.
|
||||
3. "retention_analysis:" then one line per defined label: "<Subject 1>:
|
||||
fully_preserved" (alternatives only when the scene really calls for them:
|
||||
partially_preserved, attribute_transfer, weak_reference; default
|
||||
fully_preserved).
|
||||
4. "detailed_description:" then 1-2 style sentences taken from the STYLE AND CAST
|
||||
"style" (medium, lighting quality, palette), then the "[Shot 1] ..." body — one
|
||||
line holding the subject restatement, visual style cues, all action, camera
|
||||
work, continuity cues, and any <d> dialogue. Every scene is its own single-shot
|
||||
call, so the bracketed number is always [Shot 1].
|
||||
5. "overall_soundscape:" then the sound the scene itself makes: weather, room tone,
|
||||
footsteps, surfaces, machines, animals, crowd, and diegetic music always named
|
||||
with its source inside the scene, e.g. "a battery radio on the shelf plays soft
|
||||
instrumental jazz".
|
||||
3. "non_diegetic_music: N/A" — write N/A unless the user asks for background music
|
||||
that nobody in the scene can hear; only then give one short clause describing it.
|
||||
6. "non_diegetic_music:" then "N/A" — write N/A unless the user asks for background
|
||||
music that nobody in the scene can hear; only then give one short clause
|
||||
describing it.
|
||||
- Scene 1 establishes the soundscape from the STYLE AND CAST "soundscape_seed". Later
|
||||
scenes keep the same audio bed and change only what the picture changes; say "the same
|
||||
rain continues" instead of inventing new weather.
|
||||
- The soundscape must match what the picture shows. No visible or implied source, no
|
||||
sound.
|
||||
- Diegetic music stays instrumental unless the user gave lyrics. Sung or spoken words
|
||||
are speech: they follow DIALOGUE, live in the description line, and never appear in
|
||||
are speech: they follow DIALOGUE, live in the [Shot 1] body, and never appear in
|
||||
the soundscape line.
|
||||
- Keep each soundscape to one or two dense clauses so audio prose never crowds out
|
||||
action.
|
||||
@ -355,9 +387,9 @@ below); dialogue lives inside the description line (DIALOGUE below).
|
||||
- Approved markers: [English] [French] [Spanish] [German] [Italian] [Portuguese]
|
||||
[Chinese] [Japanese] [Korean] [Arabic]. Another real language name in brackets also
|
||||
works, like [Hindi]. Never use [unclear].
|
||||
- Keep each <d>...</d> span inside ONE prompt array entry, always the
|
||||
integrated_multimodal_description line. Never split a span across two entries, never
|
||||
put dialogue in a soundscape line, and never put a raw line break inside a span.
|
||||
- Keep each <d>...</d> span inside ONE prompt array entry, always the [Shot 1] element
|
||||
of detailed_description. Never split a span across two entries, never put dialogue in
|
||||
the soundscape section, and never put a raw line break inside a span.
|
||||
- Bracket every dialogue span with mouth-state cues, or H3 chatters gibberish around
|
||||
the line. Use the phrase "silently with mouth" plus a concrete position:
|
||||
1. Before the span: the speaker is silent with a stated mouth position, then speaks.
|
||||
@ -396,9 +428,9 @@ User message:
|
||||
PREVIOUS HANDOFF: {"end_state": "The courier is crossing the interior threshold onto
|
||||
polished concrete, the silver bicycle's front wheel still rolling over the concrete
|
||||
sill, his coat sleeve mid-swing and the motion clearly still in progress, the camera
|
||||
gliding sideways at his exact walking pace.", "soundscape": "overall_soundscape:
|
||||
steady light rain drumming on glass roofing, wet tires crunching over wet paving, low
|
||||
distant traffic hum under a grey sky.", "dialogue_used": []}
|
||||
gliding sideways at his exact walking pace.", "soundscape": "steady light rain
|
||||
drumming on glass roofing, wet tires crunching over wet paving, low distant traffic
|
||||
hum under a grey sky.", "dialogue_used": []}
|
||||
|
||||
Correct reply:
|
||||
|
||||
@ -406,7 +438,20 @@ Correct reply:
|
||||
"id": "greenhouse_aisle",
|
||||
"duration_seconds": 15,
|
||||
"prompt": [
|
||||
"integrated_multimodal_description: [Shot 1] Continue the exact mid-threshold
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is an adult bicycle courier with a lean build and short dark brown
|
||||
hair, wearing a mustard-yellow waterproof jacket zipped to the collar, dark teal
|
||||
trousers, and a charcoal helmet, silver-grey sneakers, and an olive messenger
|
||||
bag across his chest, with nude hands below the jacket cuffs and nude ears under
|
||||
the helmet straps.",
|
||||
"summary:",
|
||||
"[reference generation]",
|
||||
"retention_analysis:",
|
||||
"<Subject 1>: fully_preserved",
|
||||
"detailed_description:",
|
||||
"Photorealistic overcast rainy daylight, soft grey palette, stable landscape
|
||||
tracking shots at eye level.",
|
||||
"[Shot 1] Continue the exact mid-threshold
|
||||
motion from the previous scene without resetting it: the front wheel rolls off the
|
||||
concrete sill onto polished interior concrete as <Subject 1>, the same adult
|
||||
bicycle courier in his mustard-yellow waterproof jacket, dark teal trousers, and
|
||||
@ -423,19 +468,21 @@ Correct reply:
|
||||
on the floor; the wet hems of his trousers slap lightly with each step. Approaching
|
||||
the wooden potting table at the aisle's end, he slows the bicycle to a crawl and
|
||||
his right hand leaves the handlebar, closing on the olive messenger flap.",
|
||||
"overall_soundscape: the same steady rain drumming on the glass roof continues
|
||||
"overall_soundscape:",
|
||||
"the same steady rain drumming on the glass roof continues
|
||||
overhead, tires humming on polished concrete, a single fender drop bursting on the
|
||||
floor, the distant traffic hum muffled behind the glass.",
|
||||
"non_diegetic_music: N/A"
|
||||
"non_diegetic_music:",
|
||||
"N/A"
|
||||
],
|
||||
"_handoff": {
|
||||
"end_state": "The courier has slowed the silver bicycle to a crawl at the wooden
|
||||
potting table at the end of the aisle, his right hand leaving the handlebar and
|
||||
closing on the olive messenger flap at his hip, the kraft parcel still pressed
|
||||
against him, the camera still gliding at his pace.",
|
||||
"soundscape": "overall_soundscape: the same steady rain drumming on the glass roof
|
||||
continues overhead, tires humming on polished concrete, a single fender drop
|
||||
bursting on the floor, the distant traffic hum muffled behind the glass.",
|
||||
"soundscape": "the same steady rain drumming on the glass roof continues
|
||||
overhead, tires humming on polished concrete, a single fender drop bursting on the
|
||||
floor, the distant traffic hum muffled behind the glass.",
|
||||
"dialogue_used": []
|
||||
}
|
||||
}
|
||||
@ -444,14 +491,18 @@ Correct reply:
|
||||
1. One raw JSON object: first character { and last character }. No fences, no prose.
|
||||
2. Keys are exactly id, duration_seconds, prompt, _handoff (plus steps or seed only if
|
||||
the user asked). id and duration_seconds match your outline row.
|
||||
3. The prompt array has exactly three lines in AUDIO order, and line 1 starts with
|
||||
"integrated_multimodal_description: [Shot 1]".
|
||||
4. The description line is self-sufficient: it re-states <Subject 1>, the full look
|
||||
(identity, hair, build, every garment head to toe, footwear), and the visual
|
||||
style, naming every uncovered body part as "nude" — never "bare".
|
||||
5. The description opens by continuing the PREVIOUS HANDOFF's end_state (scene 1 sets
|
||||
3. The prompt array carries the six ref2va sections in canonical order —
|
||||
subject_definitions / summary / retention_analysis / detailed_description /
|
||||
overall_soundscape / non_diegetic_music — each header on its own line, and
|
||||
summary is exactly "[reference generation]".
|
||||
4. subject_definitions defines <Subject 1> (and <Subject 2> when the outline casts a
|
||||
second person) exactly once each with the full look (identity, hair, build, every
|
||||
garment head to toe, footwear), naming every uncovered body part as "nude" —
|
||||
never "bare"; retention_analysis carries one valid entry per defined label; the
|
||||
[Shot 1] body re-states <Subject 1> with a wardrobe+skin restatement.
|
||||
5. The [Shot 1] body opens by continuing the PREVIOUS HANDOFF's end_state (scene 1 sets
|
||||
the opening instead) and ends mid-action — except the final outline scene.
|
||||
6. The description line carries every element of a scene composition — subject anchor,
|
||||
6. The [Shot 1] body carries every element of a scene composition — subject anchor,
|
||||
style anchor, continuity anchor, blocking, camera, performance, physical texture,
|
||||
exit state — and no change anywhere is stated as a bare outcome like "is now";
|
||||
every transformation is shown as a continuous on-screen progression. Length is a
|
||||
@ -466,10 +517,10 @@ Correct reply:
|
||||
8. There is no <d> span anywhere unless this scene's outline row or the story gave
|
||||
spoken words; if it did, the text inside every <d>...</d> matches those words
|
||||
exactly, in order.
|
||||
9. Every spoken line is wrapped as <d>[Language] words</d> inside the description line,
|
||||
9. Every spoken line is wrapped as <d>[Language] words</d> inside the [Shot 1] body,
|
||||
with a "silently with mouth" cue before and after the span.
|
||||
10. _handoff is truthful: end_state describes your real final frame; soundscape is your
|
||||
soundscape line character for character; dialogue_used lists the spoken words in
|
||||
soundscape content line character for character; dialogue_used lists the spoken words in
|
||||
order, or is empty.
|
||||
11. All strings use double quotes; no raw line break inside a string; no trailing
|
||||
commas.
|
||||
@ -492,8 +543,8 @@ Correct reply:
|
||||
distributed by the outline.
|
||||
- **Prefix rule is unchanged:** fragments never carry `prompt_prefix`; the final plan
|
||||
always gets `"prompt_prefix": [""]` (the empirically better empty prefix), and each
|
||||
description line restates identity/wardrobe/style because nothing is shared from a
|
||||
prefix.
|
||||
scene's subject_definitions plus its [Shot 1] body restate identity/wardrobe/style
|
||||
because nothing is shared from a prefix.
|
||||
- **Fragment file convention:** save each reply as `scene_NN.json` (zero-padded). The
|
||||
assembler and `scripts/build_plan.py` accept the raw object; fenced output also
|
||||
parses.
|
||||
|
||||
@ -22,6 +22,15 @@ MAX_H3_FRAMES = 3592
|
||||
H3_SECTIONS = ["integrated_multimodal_description", "overall_soundscape",
|
||||
"non_diegetic_music"]
|
||||
|
||||
# Ref2VA (full-reference) mode: the six native H3 reference sections in canonical
|
||||
# order, mirroring web/h3_prompt_schema_core.mjs H3_REFERENCE_SECTIONS. The skill
|
||||
# emits this format; t2va (3-line) stays as an auto-detected legacy path.
|
||||
REF2VA_SECTIONS = ["subject_definitions", "summary", "retention_analysis",
|
||||
"detailed_description", "overall_soundscape", "non_diegetic_music"]
|
||||
REF2VA_TASK_DIRECTIVE = "[reference generation]"
|
||||
REF2VA_RETENTION_MARKERS = {"fully_preserved", "partially_preserved",
|
||||
"attribute_transfer", "weak_reference"}
|
||||
|
||||
FENCE_RE = re.compile(r"^\s*```[a-zA-Z0-9_-]*\s*\n(.*?)\n?\s*```\s*$", re.S)
|
||||
DIALOGUE_SPAN_RE = re.compile(r"<d>(.*?)</d>")
|
||||
MARKER_RE = re.compile(r"^\[[^\]]+\]\s*")
|
||||
@ -69,6 +78,145 @@ def _h3_frame_length(seconds):
|
||||
return length
|
||||
|
||||
|
||||
def detect_prompt_mode(prompt_text):
|
||||
"""t2va (3-line) vs ref2va (six-section) - mirrors detectH3Mode()."""
|
||||
if re.search(r"^(?:subject_definitions|summary|retention_analysis|"
|
||||
r"detailed_description):", prompt_text, re.M):
|
||||
return "ref2va"
|
||||
return "t2va"
|
||||
|
||||
|
||||
def _split_ref2va_sections(lines, where):
|
||||
"""Group ref2va prompt lines into [(section, [content lines])] in file order.
|
||||
A line whose stripped form starts with '<section>:' opens that section (its
|
||||
content is the rest of that line plus following lines until the next header) -
|
||||
the same rule parseH3Sections() applies."""
|
||||
records = []
|
||||
for line in lines:
|
||||
stripped = line.strip()
|
||||
hit = None
|
||||
for sec in REF2VA_SECTIONS:
|
||||
if stripped.startswith(sec + ":"):
|
||||
hit = sec
|
||||
break
|
||||
if hit is not None:
|
||||
rest = stripped[len(hit) + 1:].strip()
|
||||
records.append((hit, [rest] if rest else []))
|
||||
elif records:
|
||||
records[-1][1].append(stripped)
|
||||
else:
|
||||
raise ValueError(f"{where}: ref2va prompt must open with "
|
||||
f"'subject_definitions:', got {stripped[:60]!r}")
|
||||
return records
|
||||
|
||||
|
||||
def check_ref2va_prompt(lines, dur, where, prev_handoff=None):
|
||||
"""Structural + craft checks for one ref2va prompt array.
|
||||
|
||||
Returns (shot_body, soundscape_text, dialogue_spans, defined_labels, wc) so
|
||||
the fragment chain (handoff mirror, outline match) can reuse them; wc is the
|
||||
detailed_description word count the thin-scene floor was applied to.
|
||||
"""
|
||||
if (not isinstance(lines, list) or len(lines) < 6
|
||||
or not all(isinstance(line, str) and line.strip() for line in lines)):
|
||||
raise ValueError(f"{where}: ref2va prompt must be an array of non-empty "
|
||||
f"strings (headers on their own line, content on the "
|
||||
f"following lines)")
|
||||
records = _split_ref2va_sections(lines, where)
|
||||
names = [name for name, _ in records]
|
||||
if sorted(names) != sorted(REF2VA_SECTIONS):
|
||||
missing = [s for s in REF2VA_SECTIONS if s not in names]
|
||||
raise ValueError(f"{where}: ref2va sections missing {missing} - all six of "
|
||||
f"{REF2VA_SECTIONS} are required, each exactly once")
|
||||
if names != REF2VA_SECTIONS:
|
||||
raise ValueError(f"{where}: ref2va sections must be in canonical order "
|
||||
f"{REF2VA_SECTIONS}, got {names}")
|
||||
by_name = dict(records)
|
||||
|
||||
summary = " ".join(by_name["summary"]).strip()
|
||||
if summary != REF2VA_TASK_DIRECTIVE:
|
||||
raise ValueError(f"{where}: summary must be exactly {REF2VA_TASK_DIRECTIVE!r} "
|
||||
f"(every scene is built from the wired subject references), "
|
||||
f"got {summary[:60]!r}")
|
||||
|
||||
defs = by_name["subject_definitions"]
|
||||
if not defs:
|
||||
raise ValueError(f"{where}: subject_definitions is empty")
|
||||
defined = []
|
||||
for line in defs:
|
||||
m = re.match(r"^(<(?:Subject|Picture|Video|Audio)\s+\d+>)\s+\S", line)
|
||||
if not m:
|
||||
raise ValueError(f"{where}: subject_definitions lines must start with a "
|
||||
f"<Subject N> label and describe the person: {line[:60]!r}")
|
||||
if m.group(1) in defined:
|
||||
raise ValueError(f"{where}: {m.group(1)} is defined more than once")
|
||||
defined.append(m.group(1))
|
||||
if "<Subject 1>" not in defined:
|
||||
raise ValueError(f"{where}: <Subject 1> must be defined in subject_definitions")
|
||||
|
||||
retained = {}
|
||||
for line in by_name["retention_analysis"]:
|
||||
m = re.match(r"^(<(?:Subject|Picture|Video|Audio)\s+\d+>)\s*:\s*(\S+)\s*$", line)
|
||||
if not m or m.group(2) not in REF2VA_RETENTION_MARKERS:
|
||||
raise ValueError(f"{where}: retention_analysis lines must be "
|
||||
f"'<Subject N>: fully_preserved' (or partially_preserved / "
|
||||
f"attribute_transfer / weak_reference): {line[:60]!r}")
|
||||
retained[m.group(1)] = m.group(2)
|
||||
for label in defined:
|
||||
if label not in retained:
|
||||
raise ValueError(f"{where}: {label} has no retention_analysis entry")
|
||||
|
||||
dd = "\n".join(by_name["detailed_description"]).strip()
|
||||
if "[Shot 1]" not in dd:
|
||||
raise ValueError(f"{where}: detailed_description must contain '[Shot 1]'")
|
||||
style_part, _, body = dd.partition("[Shot 1]")
|
||||
body = body.strip()
|
||||
if not style_part.strip():
|
||||
raise ValueError(f"{where}: detailed_description needs 1-2 style sentences "
|
||||
f"before [Shot 1] (medium, lighting quality, palette)")
|
||||
wc = len((style_part + " " + body).split())
|
||||
floor = THIN_FLOOR_LONG if dur >= 10 else THIN_FLOOR_SHORT
|
||||
if wc < floor:
|
||||
raise ValueError(f"{where}: detailed_description is only {wc} words - too thin "
|
||||
f"to carry the elements of a scene composition (subject, "
|
||||
f"wardrobe, style, continuity, blocking, camera, performance, "
|
||||
f"texture, sound, exit state)")
|
||||
if "<Subject 1>" not in body:
|
||||
raise ValueError(f"{where}: the [Shot 1] body must re-state <Subject 1> with a "
|
||||
f"wardrobe+skin restatement (definitions alone do not render)")
|
||||
if " is now " in dd.lower():
|
||||
raise ValueError(f"{where}: change stated as bare outcome instead of shown progression")
|
||||
if re.search(r"\bbare\b", dd, re.I):
|
||||
print(f" note: {where} description says 'bare' - call uncovered body parts "
|
||||
f"'nude' (e.g. 'nude arms')")
|
||||
|
||||
ssc_text = "\n".join(by_name["overall_soundscape"]).strip()
|
||||
if not ssc_text:
|
||||
raise ValueError(f"{where}: overall_soundscape is empty")
|
||||
ndm_text = "\n".join(by_name["non_diegetic_music"]).strip()
|
||||
if ndm_text.upper() != "N/A":
|
||||
print(f" note: {where} non_diegetic_music is {ndm_text[:40]!r} - N/A unless "
|
||||
f"the user asked for a score")
|
||||
|
||||
spans = [MARKER_RE.sub("", span, count=1).strip()
|
||||
for span in DIALOGUE_SPAN_RE.findall(dd)]
|
||||
if spans and not re.search(r"silently with mouth", dd):
|
||||
raise ValueError(f"{where}: dialogue span lacks a 'silently with mouth' cue")
|
||||
|
||||
if prev_handoff is not None:
|
||||
prev_ssc = ""
|
||||
if isinstance(prev_handoff, dict):
|
||||
prev_ssc = str(prev_handoff.get("soundscape", ""))
|
||||
if prev_ssc and not CONTINUITY_RE.search(ssc_text):
|
||||
raise ValueError(f"{where}: soundscape does not continue the previous audio "
|
||||
f"bed (say 'the same ... continues')")
|
||||
if prev_ssc and not CONTINUITY_RE.search(body):
|
||||
raise ValueError(f"{where}: description does not open inside the previous "
|
||||
f"end_state (use a continuity cue: 'Continue the same ...', "
|
||||
f"'without resetting the motion')")
|
||||
return body, ssc_text, spans, defined, wc
|
||||
|
||||
|
||||
def validate_plan(raw, default_duration=15.0, default_steps=20):
|
||||
if isinstance(raw, list):
|
||||
raw = {"shots": raw}
|
||||
@ -166,47 +314,57 @@ WORKED_EXAMPLE = r"""
|
||||
|
||||
|
||||
def check_h3_sections(plan):
|
||||
"""Structural check: every shot prompt is exactly the three native H3
|
||||
sections in canonical order; dialogue lives only in the description line.
|
||||
Mirrors web/h3_prompt_schema_core.mjs H3_BASE_SECTIONS ordering."""
|
||||
"""Structural check per shot, mode-detected: t2va (the 3 native sections) or
|
||||
ref2va (the six native reference sections). Mirrors
|
||||
web/h3_prompt_schema_core.mjs H3_BASE_SECTIONS / H3_REFERENCE_SECTIONS."""
|
||||
assert plan["prompt_prefix"] == [""], \
|
||||
"prompt_prefix must be exactly [''] (empirical: better H3 results, no prose preamble)"
|
||||
counts = []
|
||||
modes = []
|
||||
for idx, shot in enumerate(plan["shots"], 1):
|
||||
lines = shot["prompt"]
|
||||
assert len(lines) == 3, f"shot {idx}: expected 3 prompt lines, got {len(lines)}"
|
||||
positions = []
|
||||
for sec in H3_SECTIONS:
|
||||
m = re.search(rf"^{sec}:", lines[len(positions)], re.M)
|
||||
assert m, f"shot {idx}: line {len(positions)+1} does not start with {sec}:"
|
||||
positions.append(sec)
|
||||
assert positions == H3_SECTIONS, f"shot {idx}: sections out of order"
|
||||
assert lines[0].lstrip().startswith(
|
||||
"integrated_multimodal_description: [Shot 1]"), f"shot {idx}: missing [Shot 1]"
|
||||
wc = len(lines[0].split())
|
||||
assert "<Subject 1>" in lines[0], \
|
||||
f"shot {idx}: description line must re-state <Subject 1> (prefix is empty)"
|
||||
if re.search(r"\bbare\b", lines[0], re.I):
|
||||
print(f" note: shot {idx} description says 'bare' - call uncovered body "
|
||||
f"parts 'nude' (e.g. 'nude arms')")
|
||||
raw_prompt = shot["prompt"]
|
||||
dur = float(shot.get("duration_seconds",
|
||||
plan.get("defaults", {}).get("duration_seconds", 15)))
|
||||
floor = THIN_FLOOR_LONG if dur >= 10 else THIN_FLOOR_SHORT
|
||||
assert wc >= floor, \
|
||||
(f"shot {idx}: description is only {wc} words - too thin to carry the "
|
||||
f"elements of a scene composition (subject, wardrobe, style, continuity, "
|
||||
f"blocking, camera, performance, texture, sound, exit state)")
|
||||
assert " is now " not in lines[0].lower(), \
|
||||
f"shot {idx}: change stated as bare outcome instead of shown progression"
|
||||
assert "<d>" not in lines[1] and "</d>" not in lines[1], \
|
||||
f"shot {idx}: dialogue must not appear in the soundscape line"
|
||||
if "<d>" in lines[0]:
|
||||
assert re.search(r"silently with mouth", lines[0]), \
|
||||
f"shot {idx}: dialogue span lacks a mouth-state cue"
|
||||
print(" H3 section structure: all shots have the 3 sections in canonical order")
|
||||
print(" description lengths:",
|
||||
[(f"{len(s['prompt'][0].split())}w @ "
|
||||
f"{s.get('duration_seconds', plan.get('defaults', {}).get('duration_seconds', 15))}s")
|
||||
for s in plan["shots"]],
|
||||
if not isinstance(raw_prompt, list):
|
||||
continue # string prompt: validate_plan already ensured it is non-empty
|
||||
mode = detect_prompt_mode("\n".join(raw_prompt))
|
||||
modes.append(mode)
|
||||
if mode == "ref2va":
|
||||
body, _ssc, _spans, _defs, wc = check_ref2va_prompt(
|
||||
raw_prompt, dur, f"shot {idx}")
|
||||
else:
|
||||
lines = raw_prompt
|
||||
assert len(lines) == 3, f"shot {idx}: expected 3 prompt lines, got {len(lines)}"
|
||||
positions = []
|
||||
for sec in H3_SECTIONS:
|
||||
m = re.search(rf"^{sec}:", lines[len(positions)], re.M)
|
||||
assert m, f"shot {idx}: line {len(positions)+1} does not start with {sec}:"
|
||||
positions.append(sec)
|
||||
assert positions == H3_SECTIONS, f"shot {idx}: sections out of order"
|
||||
assert lines[0].lstrip().startswith(
|
||||
"integrated_multimodal_description: [Shot 1]"), f"shot {idx}: missing [Shot 1]"
|
||||
wc = len(lines[0].split())
|
||||
assert "<Subject 1>" in lines[0], \
|
||||
f"shot {idx}: description line must re-state <Subject 1> (prefix is empty)"
|
||||
if re.search(r"\bbare\b", lines[0], re.I):
|
||||
print(f" note: shot {idx} description says 'bare' - call uncovered body "
|
||||
f"parts 'nude' (e.g. 'nude arms')")
|
||||
assert " is now " not in lines[0].lower(), \
|
||||
f"shot {idx}: change stated as bare outcome instead of shown progression"
|
||||
assert "<d>" not in lines[1] and "</d>" not in lines[1], \
|
||||
f"shot {idx}: dialogue must not appear in the soundscape line"
|
||||
if "<d>" in lines[0]:
|
||||
assert re.search(r"silently with mouth", lines[0]), \
|
||||
f"shot {idx}: dialogue span lacks a mouth-state cue"
|
||||
counts.append(f"{wc}w @ {dur:g}s")
|
||||
if modes and len(set(modes)) == 1 and modes[0] == "ref2va":
|
||||
print(" H3 section structure: all shots are ref2va - the six reference "
|
||||
"sections in canonical order, summary [reference generation]")
|
||||
elif modes and len(set(modes)) == 1:
|
||||
print(" H3 section structure: all shots have the 3 sections in canonical order")
|
||||
else:
|
||||
print(" H3 section structure: mixed t2va/ref2va shots (each shot auto-detected)")
|
||||
print(" description lengths:", counts,
|
||||
"| composition elements are the gate - no word band")
|
||||
return True
|
||||
|
||||
@ -253,38 +411,47 @@ def validate_fragment(obj, index=1, total=None, prev_handoff=None, expect=None):
|
||||
raw_frames = _h3_frame_length(dur)
|
||||
|
||||
prompt = obj["prompt"]
|
||||
if (not isinstance(prompt, list) or len(prompt) != 3
|
||||
if (not isinstance(prompt, list) or not prompt
|
||||
or not all(isinstance(line, str) and line.strip() for line in prompt)):
|
||||
raise ValueError(f"{where}: prompt must be an array of exactly 3 non-empty strings")
|
||||
desc, ssc, ndm = prompt
|
||||
if not desc.lstrip().startswith("integrated_multimodal_description: [Shot 1]"):
|
||||
raise ValueError(f"{where}: line 1 must start with "
|
||||
f"'integrated_multimodal_description: [Shot 1]'")
|
||||
if not ssc.lstrip().startswith("overall_soundscape:"):
|
||||
raise ValueError(f"{where}: line 2 must start with 'overall_soundscape:'")
|
||||
if not ndm.lstrip().startswith("non_diegetic_music:"):
|
||||
raise ValueError(f"{where}: line 3 must start with 'non_diegetic_music:'")
|
||||
raise ValueError(f"{where}: prompt must be an array of non-empty strings")
|
||||
mode = detect_prompt_mode("\n".join(prompt))
|
||||
spans = []
|
||||
if mode == "ref2va":
|
||||
desc, ssc, spans, _defs, wc = check_ref2va_prompt(
|
||||
prompt, dur, where, prev_handoff=prev_handoff)
|
||||
else:
|
||||
if len(prompt) != 3:
|
||||
raise ValueError(f"{where}: t2va prompt must be an array of exactly 3 "
|
||||
f"non-empty strings")
|
||||
desc, ssc, ndm = prompt
|
||||
if not desc.lstrip().startswith("integrated_multimodal_description: [Shot 1]"):
|
||||
raise ValueError(f"{where}: line 1 must start with "
|
||||
f"'integrated_multimodal_description: [Shot 1]'")
|
||||
if not ssc.lstrip().startswith("overall_soundscape:"):
|
||||
raise ValueError(f"{where}: line 2 must start with 'overall_soundscape:'")
|
||||
if not ndm.lstrip().startswith("non_diegetic_music:"):
|
||||
raise ValueError(f"{where}: line 3 must start with 'non_diegetic_music:'")
|
||||
|
||||
wc = len(desc.split())
|
||||
floor = THIN_FLOOR_LONG if dur >= 10 else THIN_FLOOR_SHORT
|
||||
if wc < floor:
|
||||
raise ValueError(f"{where}: description is only {wc} words - too thin to carry "
|
||||
f"the elements of a scene composition (subject, wardrobe, "
|
||||
f"style, continuity, blocking, camera, performance, texture, "
|
||||
f"sound, exit state)")
|
||||
if "<Subject 1>" not in desc:
|
||||
raise ValueError(f"{where}: description must re-state <Subject 1> (prefix is empty)")
|
||||
if re.search(r"\bbare\b", desc, re.I):
|
||||
print(f" note: {where} description says 'bare' - call uncovered body parts "
|
||||
f"'nude' (e.g. 'nude arms')")
|
||||
if " is now " in desc.lower():
|
||||
raise ValueError(f"{where}: change stated as bare outcome instead of shown progression")
|
||||
if "<d>" in ssc or "</d>" in ssc:
|
||||
raise ValueError(f"{where}: dialogue must not appear in the soundscape line")
|
||||
spans = [MARKER_RE.sub("", span, count=1).strip()
|
||||
for span in DIALOGUE_SPAN_RE.findall(desc)]
|
||||
if spans and not re.search(r"silently with mouth", desc):
|
||||
raise ValueError(f"{where}: dialogue span lacks a 'silently with mouth' cue")
|
||||
wc = len(desc.split())
|
||||
floor = THIN_FLOOR_LONG if dur >= 10 else THIN_FLOOR_SHORT
|
||||
if wc < floor:
|
||||
raise ValueError(f"{where}: description is only {wc} words - too thin to carry "
|
||||
f"the elements of a scene composition (subject, wardrobe, "
|
||||
f"style, continuity, blocking, camera, performance, texture, "
|
||||
f"sound, exit state)")
|
||||
if "<Subject 1>" not in desc:
|
||||
raise ValueError(f"{where}: description must re-state <Subject 1> (prefix is empty)")
|
||||
if re.search(r"\bbare\b", desc, re.I):
|
||||
print(f" note: {where} description says 'bare' - call uncovered body parts "
|
||||
f"'nude' (e.g. 'nude arms')")
|
||||
if " is now " in desc.lower():
|
||||
raise ValueError(f"{where}: change stated as bare outcome instead of shown progression")
|
||||
if "<d>" in ssc or "</d>" in ssc:
|
||||
raise ValueError(f"{where}: dialogue must not appear in the soundscape line")
|
||||
spans = [MARKER_RE.sub("", span, count=1).strip()
|
||||
for span in DIALOGUE_SPAN_RE.findall(desc)]
|
||||
if spans and not re.search(r"silently with mouth", desc):
|
||||
raise ValueError(f"{where}: dialogue span lacks a 'silently with mouth' cue")
|
||||
|
||||
handoff = obj["_handoff"]
|
||||
if not isinstance(handoff, dict):
|
||||
@ -306,7 +473,7 @@ def validate_fragment(obj, index=1, total=None, prev_handoff=None, expect=None):
|
||||
raise ValueError(f"{where}: _handoff.dialogue_used {used} does not mirror the "
|
||||
f"<d> spans {spans}")
|
||||
|
||||
if prev_handoff is not None:
|
||||
if mode == "t2va" and prev_handoff is not None:
|
||||
prev_line = ""
|
||||
if isinstance(prev_handoff, dict):
|
||||
prev_line = str(prev_handoff.get("soundscape", ""))
|
||||
@ -375,12 +542,106 @@ NURSE_OK = {
|
||||
}
|
||||
|
||||
|
||||
# Ref2VA fragment: the six native H3 reference sections, one line per element,
|
||||
# headers on their own line. handoff.soundscape copies the soundscape CONTENT
|
||||
# line (the element after the overall_soundscape: header), not the header.
|
||||
REF2VA_NURSE = {
|
||||
"id": "corridor_walk",
|
||||
"duration_seconds": 8,
|
||||
"prompt": [
|
||||
"subject_definitions:",
|
||||
"<Subject 1> is a night nurse in her thirties with a slim build, dark brown "
|
||||
"hair pulled back in a low bun, and calm grey eyes, wearing a pale green "
|
||||
"scrub top and matching scrub trousers, white sneakers, with nude hands below "
|
||||
"the scrub cuffs and nude lower legs above the sneaker collars.",
|
||||
"summary:",
|
||||
"[reference generation]",
|
||||
"retention_analysis:",
|
||||
"<Subject 1>: fully_preserved",
|
||||
"detailed_description:",
|
||||
"Photorealistic cool fluorescent lighting, soft clinical palette, stable "
|
||||
"over-the-shoulder tracking shots.",
|
||||
"[Shot 1] Continue the same walk along the corridor without resetting the "
|
||||
"motion. <Subject 1>, the same night nurse in her pale green scrubs and white "
|
||||
"sneakers, pushes the steel trolley forward with both hands, its wheels "
|
||||
"squeaking softly on the linoleum, a paper cup and a clipped chart riding on "
|
||||
"top. The camera keeps a steady over-the-shoulder framing at her shoulder "
|
||||
"height, gliding at her pace, fluorescent panels passing overhead in even "
|
||||
"rhythm, the corridor walls sliding past in soft focus. Her sneakers land "
|
||||
"heel first and roll to the toe, the cup's surface trembling with each push, "
|
||||
"and the hem of her scrub top sways in counterpoint to the trolley's roll. "
|
||||
"Her jaw stays set, eyes forward, one loose strand of hair lifting with each "
|
||||
"step; the trolley's shadow stretches and compresses under each ceiling "
|
||||
"panel. Reaching the blue door at the corridor's end, she slows the trolley "
|
||||
"to a stop and her right hand leaves the handle, closing on the door's cold "
|
||||
"push bar.",
|
||||
"overall_soundscape:",
|
||||
"the same low fluorescent hum and distant trolley squeak continue, a soft "
|
||||
"draught breathing under the far door.",
|
||||
"non_diegetic_music:",
|
||||
"N/A",
|
||||
],
|
||||
"_handoff": {
|
||||
"end_state": "She is stopped at the blue door at the corridor's end, right "
|
||||
"hand closing on the cold push bar, trolley at her hip, camera "
|
||||
"still at her shoulder.",
|
||||
"soundscape": "the same low fluorescent hum and distant trolley squeak "
|
||||
"continue, a soft draught breathing under the far door.",
|
||||
"dialogue_used": [],
|
||||
},
|
||||
}
|
||||
|
||||
REF2VA_DOOR = {
|
||||
"id": "door_stop",
|
||||
"duration_seconds": 8,
|
||||
"prompt": [
|
||||
"subject_definitions:",
|
||||
REF2VA_NURSE["prompt"][1],
|
||||
"summary:",
|
||||
"[reference generation]",
|
||||
"retention_analysis:",
|
||||
"<Subject 1>: fully_preserved",
|
||||
"detailed_description:",
|
||||
REF2VA_NURSE["prompt"][7],
|
||||
"[Shot 1] Continue the same corridor walk without resetting the motion, the "
|
||||
"trolley still rolling at her pace. <Subject 1>, the same night nurse in her "
|
||||
"pale green scrubs and white sneakers, eases the steel trolley to a stop "
|
||||
"beside the blue door, both hands relaxing on the handle one after the other, "
|
||||
"its last wheel squeak dying on the linoleum. The camera holds its "
|
||||
"over-the-shoulder framing while her shoulders settle and the paper cup's "
|
||||
"surface stills on the trolley top; the fluorescent panels keep their even "
|
||||
"rhythm overhead, the corridor walls stay in soft focus, and her scrub-top "
|
||||
"hem sways once and hangs. Her right hand leaves the handle, opens, and "
|
||||
"closes on the door's cold push bar while the clipped chart keeps swaying on "
|
||||
"its clip.",
|
||||
"overall_soundscape:",
|
||||
"the same low fluorescent hum continues, the trolley wheels fall silent, a "
|
||||
"soft draught breathing under the far door.",
|
||||
"non_diegetic_music:",
|
||||
"N/A",
|
||||
],
|
||||
"_handoff": {
|
||||
"end_state": "Her right hand is closing on the cold push bar, trolley at her "
|
||||
"hip, cup stilled, camera at her shoulder.",
|
||||
"soundscape": "the same low fluorescent hum continues, the trolley wheels "
|
||||
"fall silent, a soft draught breathing under the far door.",
|
||||
"dialogue_used": [],
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _short_fragment(**over):
|
||||
base = json.loads(json.dumps(NURSE_OK)) # deep copy
|
||||
base.update(over)
|
||||
return base
|
||||
|
||||
|
||||
def _ref2va_short(**over):
|
||||
base = json.loads(json.dumps(REF2VA_NURSE)) # deep copy
|
||||
base.update(over)
|
||||
return base
|
||||
|
||||
|
||||
def run_fragment_suite():
|
||||
print("Fragment edge cases that must PASS:")
|
||||
handoff = validate_fragment(dict(NURSE_OK), index=1, total=2)
|
||||
@ -415,6 +676,40 @@ def run_fragment_suite():
|
||||
validate_fragment(spoken, index=1, total=1)
|
||||
print(" all fragment PASS cases ok")
|
||||
|
||||
print("\nRef2VA fragments (six-section format) that must PASS:")
|
||||
rh = validate_fragment(json.loads(json.dumps(REF2VA_NURSE)), index=1, total=2)
|
||||
validate_fragment(json.loads(json.dumps(REF2VA_DOOR)), index=2, total=2,
|
||||
prev_handoff=rh)
|
||||
print(" all ref2va PASS cases ok")
|
||||
|
||||
print("\nRef2VA edge cases that must FAIL:")
|
||||
no_ret = _ref2va_short()
|
||||
no_ret["prompt"] = [l for l in no_ret["prompt"]
|
||||
if not l.startswith("<Subject 1>: ")]
|
||||
wrong_summary = _ref2va_short()
|
||||
idx_sum = wrong_summary["prompt"].index("[reference generation]")
|
||||
wrong_summary["prompt"][idx_sum] = "[video continuation]"
|
||||
no_style = _ref2va_short()
|
||||
no_style["prompt"].remove(
|
||||
"Photorealistic cool fluorescent lighting, soft clinical palette, stable "
|
||||
"over-the-shoulder tracking shots.")
|
||||
out_of_order = _ref2va_short()
|
||||
p = out_of_order["prompt"]
|
||||
i_sum, i_def = p.index("summary:"), p.index("subject_definitions:")
|
||||
p[i_sum], p[i_def] = p[i_def], p[i_sum]
|
||||
r2_cases = [
|
||||
("missing retention_analysis entry", no_ret, {}),
|
||||
("summary directive not [reference generation]", wrong_summary, {}),
|
||||
("no style sentences before [Shot 1]", no_style, {}),
|
||||
("sections out of canonical order", out_of_order, {}),
|
||||
]
|
||||
for i, (label, obj, kwargs) in enumerate(r2_cases, 1):
|
||||
try:
|
||||
validate_fragment(obj, index=1, total=1, **kwargs)
|
||||
print(f" case {i} ({label}) UNEXPECTEDLY PASSED <-- BUG")
|
||||
except (ValueError, AssertionError) as exc:
|
||||
print(f" case {i} ({label}) rejected: {exc}")
|
||||
|
||||
print("\nFragment edge cases that must FAIL:")
|
||||
prev = dict(NURSE_OK["_handoff"])
|
||||
no_continuity = _short_fragment(
|
||||
@ -491,6 +786,20 @@ def run_embedded_suite():
|
||||
validate_plan(plan)
|
||||
check_h3_sections(plan)
|
||||
|
||||
print("\nRef2VA plan (six-section reference format):")
|
||||
ref2va_plan = {
|
||||
"prompt_prefix": [""],
|
||||
"defaults": {"duration_seconds": 8, "steps": 20},
|
||||
"shots": [
|
||||
{"id": "corridor_walk",
|
||||
"prompt": list(REF2VA_NURSE["prompt"]), "duration_seconds": 8},
|
||||
{"id": "door_stop",
|
||||
"prompt": list(REF2VA_DOOR["prompt"]), "duration_seconds": 8},
|
||||
],
|
||||
}
|
||||
validate_plan(ref2va_plan)
|
||||
check_h3_sections(ref2va_plan)
|
||||
|
||||
print("\nRepo I2V Studio example (with length/frames + string seeds):")
|
||||
repo = {
|
||||
"defaults": {"steps": 20},
|
||||
|
||||
Loading…
Reference in New Issue
Block a user