338 lines
20 KiB
Markdown
Executable File
338 lines
20 KiB
Markdown
Executable File
---
|
||
name: h3-plan-gate
|
||
description: Turn a story idea into a validated plan_json_input JSON for the MiniMax H3
|
||
Production Plan node (ComfyUI-MiniMaxH3-Context-Loop). Use when the user wants to plan
|
||
scenes/shots for H3 video generation, write or extend a scene plan, regenerate one scene,
|
||
or validate an existing H3 plan JSON. The skill lives in the h3-plan-gate folder of the
|
||
project; all commands run from the project root with the project's Python.
|
||
---
|
||
|
||
# H3 Plan Gate — agent procedure
|
||
|
||
Goal: produce `plan.json`, the JSON pasted into the Production Plan node's `plan_json_input`,
|
||
in five moves: the user gives a story and a scene count → you build a scene guide
|
||
(`outline.json`) and show it for **approval** → with the approved guide you compose the
|
||
scenes one at a time (about 10 seconds each), each turn existing so that ONE scene can
|
||
be composed from every element of a good scene composition, never to fill a word count
|
||
→ the scripts verify the continuity across the whole chain → `build_plan.py` puts all
|
||
the scenes together at the end. Never deliver a plan that did not pass `--check`, and
|
||
never compose a scene from a guide the user has not approved.
|
||
|
||
## Paths and Python (read first — these exact forms prevent most errors)
|
||
|
||
- Work from the **project root**: the folder that contains `h3-plan-gate/`. Every path below
|
||
is relative to it. Do not cd into `h3-plan-gate`, and do not cd out of the project root.
|
||
- Write generated files into the project root: `outline.json`, `scene_01.json`,
|
||
`scene_02.json`, … , `plan.json`.
|
||
- Python command: on Windows run `.venv\Scripts\python.exe`; if the project has no `.venv`,
|
||
use `python` instead. (macOS/Linux: `.venv/bin/python` or `python`.) The scripts need
|
||
only the Python standard library — nothing to install.
|
||
- Start every task by running the smoke test under "Before you start". If it fails because
|
||
`h3-plan-gate` is missing files, stop and tell the user the folder is incomplete.
|
||
|
||
## Before you start (smoke test)
|
||
|
||
```sh
|
||
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py
|
||
.venv\Scripts\python.exe h3-plan-gate/build_plan.py --check h3-plan-gate/h3_agent_workflow_examples/plan.json
|
||
```
|
||
|
||
Expect the suite to pass and the second command to print
|
||
`PLAN OK` with `total delivered: 685 frames = 28.54s`.
|
||
|
||
## Step 1 — Build the scene guide (outline)
|
||
|
||
The user gives you a story and how many scenes they want. Read
|
||
`h3-plan-gate/h3_scene_agent_system_prompt.md` (PROMPT A and its WORKED EXAMPLE are the
|
||
quality bar) and write `outline.json` — the guide for every scene — in exactly this
|
||
shape:
|
||
|
||
```json
|
||
{
|
||
"title": "short_snake_case_name",
|
||
"defaults": { "duration_seconds": 10, "steps": 20 },
|
||
"style_and_cast": {
|
||
"subjects": ["<Subject 1> — who they are; wardrobe head to toe; build; hair"],
|
||
"style": "one consistent look: photoreal or otherwise, lighting, palette, camera feel",
|
||
"soundscape_seed": "the base ambient sound carried across all scenes"
|
||
},
|
||
"scenes": [
|
||
{ "index": 1, "id": "scene_one_name", "duration_seconds": 10,
|
||
"beat": "one sentence: what happens AND where the scene ends",
|
||
"dialogue": [] }
|
||
]
|
||
}
|
||
```
|
||
|
||
3–12 scenes unless the user names a count (hard max 128). Ids unique lowercase snake_case,
|
||
never renamed later. Durations whole seconds 2–15 — about 10 seconds unless the user says
|
||
otherwise. Copy the user's quoted speech EXACTLY into the right scene's `dialogue` array;
|
||
if the story has no quoted speech, every `dialogue` array stays `[]`. Only the last
|
||
scene's beat may come to rest. The `subjects` array is the cast inventory: collect the
|
||
full head-to-toe look there once — when writing scenes it is spent by weaving it into
|
||
each scene's [Shot 1] body, never parked in subject_definitions (a refmod subject is
|
||
the one exception: its verbatim identity/voice text is spent exactly once, in
|
||
subject_definitions, and the body references only `<Subject N>`). If the story's script
|
||
carries a `characters:` block with refmod entries — `sam:&identity: <identity text>;
|
||
voice: <voice text>&` — copy the identity and voice text verbatim into that subject's
|
||
inventory string (the & marks are delimiters, never content); the scene turns must carry
|
||
them into subject_definitions untouched.
|
||
|
||
### Subject pinning (three forms — decide here, use in every scene)
|
||
|
||
Each `subjects` string decides how that subject is pinned in the final prompt, and every
|
||
scene's `subject_definitions` line must use exactly that form:
|
||
|
||
- **No reference image** — `<Subject N> is a <age> ... with <hair>, <eyes> and <build>`:
|
||
physical traits only (age, hair, eyes, build, skin tone). Never clothing, never a
|
||
nude/bare skin state — the scene body carries the wardrobe.
|
||
- **Reference image** — `<Subject N> binds to <Image N> : <what the image actually
|
||
shows>`: a faithful description of the image; the writer is multimodal.
|
||
- **User refmod** — the user's story carries their own tested character block, like
|
||
`&identity: ...; voice: ...&`: strip the `&` wrapper and carry the text verbatim into
|
||
`<Subject N> identity: ...; voice: ...`. Exempt from the no-image content bans.
|
||
|
||
No form carries a length cap — describe fully what the model needs.
|
||
|
||
## Step 2 — Present the guide for approval (do not skip)
|
||
|
||
Show the user the finished guide in plain language — one line per scene: number, id,
|
||
duration, what happens and where it ends, any dialogue — plus the cast and style block.
|
||
Then STOP and wait for their reply. The user approves it or asks for changes; apply
|
||
every change to `outline.json` (scene count, ids, durations, beats, dialogue, cast,
|
||
style) and, if anything changed, show the changed rows again. Start Step 3 only after
|
||
an explicit approval. The approved guide is the contract: scenes are composed from it,
|
||
ids and durations are never silently altered, and new story ideas from the user go
|
||
through the guide first — never straight into a scene file.
|
||
|
||
## Step 3 — Compose the scenes one at a time
|
||
|
||
Work only from the approved guide. Before writing scene 1, read the PROMPT B section of
|
||
`h3_scene_agent_system_prompt.md` — its SCENE COMPOSITION checklist (subject anchor,
|
||
style anchor, continuity anchor, blocking, camera, performance, physical texture, exit
|
||
state) is what each turn is for. Then, in story order: compose ONE scene file, validate
|
||
it, fix it until it passes, and only then move to the next scene. Since you wrote the
|
||
previous file yourself, open it and reuse its `_handoff` as the opening state of the
|
||
next scene. Rewrite a scene as many times as it takes for every element to be present
|
||
and concrete — the validator has no word band, only a thin-scene floor, so never pad
|
||
and never summarize.
|
||
|
||
```sh
|
||
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py fragment scene_01.json --index 1 --total 5
|
||
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py fragment scene_02.json --index 2 --total 5 --prev scene_01.json
|
||
```
|
||
|
||
(`--total` is the scene count from the outline; `--prev` chains the previous file.)
|
||
|
||
Each fragment is exactly this shape (the six-section Ref2VA prompt; the subject lines
|
||
use the pinning form chosen in the guide):
|
||
|
||
```json
|
||
{
|
||
"id": "from the outline row",
|
||
"duration_seconds": 10,
|
||
"prompt": [
|
||
"subject_definitions:",
|
||
"<Subject 1> identity: ...; voice: ...",
|
||
"<Subject 2> is a ...",
|
||
"summary:",
|
||
"[reference generation]",
|
||
"retention_analysis:",
|
||
"<Subject 1>: fully_preserved",
|
||
"<Subject 2>: fully_preserved",
|
||
"detailed_description:",
|
||
"<1-2 style sentences>",
|
||
"[Shot 1] ...",
|
||
"overall_soundscape:",
|
||
"<the scene's own sound>",
|
||
"non_diegetic_music:",
|
||
"N/A"
|
||
],
|
||
"_handoff": { "end_state": "...", "soundscape": "...", "dialogue_used": [] }
|
||
}
|
||
```
|
||
|
||
## Step 4 — Verify the continuity, put it all together at the end
|
||
|
||
Once every scene file passes, the merge is the continuity check: `build_plan.py`
|
||
re-walks the whole `_handoff` chain — every scene after the first must open inside the
|
||
previous scene's real end_state, every soundscape must continue the previous audio bed,
|
||
every id, duration, and spoken line must match the approved guide — and only then does
|
||
it write the ONE plan JSON. Never write the plan JSON by hand and never edit fragment
|
||
prose during the merge. Run:
|
||
|
||
```sh
|
||
.venv\Scripts\python.exe h3-plan-gate/build_plan.py outline.json scene_01.json scene_02.json scene_03.json -o plan.json
|
||
.venv\Scripts\python.exe h3-plan-gate/build_plan.py --check plan.json
|
||
```
|
||
|
||
The merger strips `_handoff`, injects `"prompt_prefix": [""]` and `defaults`, validates
|
||
everything, and writes the single deliverable: `plan.json` — `"prompt_prefix": [""]`,
|
||
`defaults`, and one self-sufficient shot per scene, exactly the shape the single-pass
|
||
system prompt emits. Report the delivered frames/seconds it prints.
|
||
|
||
## Step 5 — Deliver (the node makes the multi-clip video)
|
||
|
||
Verified against the node pack source (`chain_nodes.py`, `MiniMaxH3ChainPlan` /
|
||
`MiniMaxH3ChainPlanModern`): the parser is a plain `json.loads` over the effective
|
||
plan string — a non-empty STRING connected to the forced `plan_json_input` socket
|
||
wins, otherwise the node uses its internal `plan_json` widget — and any decode
|
||
failure is reported as `H3 Chain Plan JSON is invalid: <detail>`. If the effective
|
||
string were exactly the delivered `plan.json` bytes, this parse cannot fail; a
|
||
reported error always means the field content differs from the file.
|
||
|
||
**Paste into the node's internal `plan_json` widget — the only safe hand-paste
|
||
surface.** Both Plan nodes declare it `"dynamicPrompts": False` in the pack source
|
||
(`chain_nodes.py`), on purpose: ComfyUI's frontend resolves `{option|option}`
|
||
dynamic-prompt groups in any widget declared `dynamicPrompts: true` at queue time,
|
||
and a plan JSON is dense with braces (the 3-shot example plan carries 5, the
|
||
12-shot plan 14). One pass of the frontend serializer strips them, and the node
|
||
then reports for a byte-perfect copy:
|
||
|
||
H3 Chain Plan JSON is invalid: Extra data: line 1 column 16 (char 15)
|
||
|
||
The parsed head is exactly the plan's quoted first key — `"prompt_prefix"` is 15
|
||
characters, so column = length of the quoted first key + 1. That fingerprint means
|
||
a dynamic-prompt surface ate the braces; it reproduces on perfect pastes, pipes or
|
||
no pipes, and is NOT a copy mistake. The pack's own prompt-typed widgets (e.g.
|
||
`MiniMaxH3TaggedReferenceToVideo`'s `prompt`, `dynamicPrompts: True`) and any
|
||
third-party string or prompt node with dynamic prompts enabled are all such
|
||
surfaces. Never paste a plan into one, and never wire one into `plan_json_input`.
|
||
|
||
Internal-widget procedure: disconnect `plan_json_input` first (a non-empty
|
||
connected socket silently overrides the widget at queue time), select all, delete,
|
||
confirm the widget is empty, paste the full plan, and queue ONCE — before the
|
||
visual editor re-serializes anything over it. Prefer the single-line variant of the
|
||
plan file (`json.dumps(..., separators=(',', ':'), ensure_ascii=True)` — one line,
|
||
pure ASCII) so no newline handling can touch the paste. The widget ships with a
|
||
two-shot demo plan as its default: the pasted plan must still contain EXACTLY the
|
||
one JSON object, first character `{`, last character `}`, nothing before or after.
|
||
|
||
The `plan_json_input` socket is for programmatic delivery — an LLM story director
|
||
or other node producing the plan at execution time; socket values flow node-to-node
|
||
and skip frontend widget serialization. A hand paste may use the socket only when
|
||
the source is a string primitive that provably leaves braces alone. Verify any
|
||
surface with one 100-byte probe pasted where the plan would go, then queue — it is
|
||
shipped as `h3_agent_workflow_examples/brace_probe_oneline.json`:
|
||
|
||
{"shots":[{"id":"probe","prompt":["static test card, one red cube on white"]}]}
|
||
|
||
A rendered one-shot test card means the surface is clean; `Extra data: line 1
|
||
column 8 (char 7)` (`"shots"` is 7 characters) means it eats braces — deliver
|
||
through the internal widget instead.
|
||
|
||
A leftover demo or earlier plan AHEAD of the paste also reports `Extra data`, but
|
||
only after the whole leftover object parses — its `char N` is far larger than the
|
||
plan's first key length. Read the reported `char N` against the first key to tell
|
||
brace-eating and leftover-text apart; ask the user for the exact error text.
|
||
|
||
The node renders each shot as its own clip and chains the clips through
|
||
the H3 context loop — scene 2 continues from scene 1's final frames, scene 3 from scene
|
||
2's — producing one continuous multi-clip video. Every scene after the first loses about
|
||
1 second to the fixed overlap trim that stitches the clips together.
|
||
|
||
Resolution, context length, audio mode, and `base_seed` stay on the node's widgets — the
|
||
JSON cannot set them; the width/height outputs must be wired into the conditioning node
|
||
and sampler latent or the video can come out square.
|
||
|
||
## Craft rules for every fragment (violations fail the validator)
|
||
|
||
- Raw JSON only: first char `{`, last char `}`, no markdown fences, no prose, double
|
||
quotes, no trailing commas, no raw line break inside a string.
|
||
- The `prompt` array compiles to the six-section Ref2VA prompt in canonical order —
|
||
`subject_definitions:` / `summary:` / `retention_analysis:` / `detailed_description:` /
|
||
`overall_soundscape:` / `non_diegetic_music:` — with `summary:` exactly
|
||
`[reference generation]`, one `<Subject N>: fully_preserved` retention line per defined
|
||
subject, 1–2 style sentences before the body, and `[Shot 1]` starting the body (every
|
||
scene is its own single H3 generation, so the bracket number is always 1).
|
||
- `subject_definitions` lines come in exactly one of three forms: `<Subject N> is a ...`
|
||
(no image — physical traits only, never clothing, never a nude/bare skin state),
|
||
`<Subject N> binds to <Image N> : ...` (reference image — faithful to the image), or
|
||
`<Subject N> identity: ...; voice: ...` (user refmod — the story's tested wording
|
||
verbatim, the `&...&` wrapper stripped; exempt from the no-image bans). No form carries
|
||
a length cap.
|
||
- **Refmod isolation:** a refmod subject is referenced by the bare `<Subject N>` token
|
||
everywhere outside `subject_definitions` — the body never re-states the subject's name
|
||
and never re-describes the look or voice the refmod already pins (the workflow injects
|
||
the reference blocks itself; repeating them fights the injection). No-image and binds
|
||
subjects keep the opposite treatment: their full look is still woven into every scene
|
||
body.
|
||
- No-image and binds subjects are self-sufficient: re-state `<Subject 1>` and their full
|
||
look in EVERY scene — identity, hair, build, every garment head to toe, footwear, and
|
||
the visual style — nothing is shared between scenes or from a prefix. Name every body
|
||
part the clothing leaves uncovered and call uncovered skin "nude", never "bare"
|
||
("a man wearing a white tank-top with nude arms").
|
||
- Scene composition, not word count: the body must carry every element —
|
||
subject anchor (`<Subject 1>` restated; for no-image/binds subjects with wardrobe),
|
||
style anchor, continuity anchor
|
||
(opens inside the previous end_state), blocking, one camera idea, performance beats,
|
||
physical texture (wardrobe behavior, weather on skin and fabric, lighting), and a
|
||
concrete mid-action exit state. Never pad to a length and never hand in a thin
|
||
one-liner — a scene with every element is long enough on its own. The validator only
|
||
trips a thin-scene floor (about 100 body words); there is no target band.
|
||
- Decompose every action into body mechanics: never a single outcome verb ("picks up
|
||
the ball", "crosses her legs") — write the ordered joint-level sequence ("bends his
|
||
knees, grabs the ball with his left hand, then stands back up"; "raises her left
|
||
leg and places it across her right leg"), naming the hand or limb for each step.
|
||
Every movement and motion of every character is this detailed, large or small. H3
|
||
plays back only what is stated; a compressed verb leaves gaps the model fills by
|
||
guessing, and guessed motion drifts off intent.
|
||
- Show common actions, never just mention them: an action named without its steps is
|
||
a report, not a render — H3 displays only what the description displays. The common
|
||
everyday actions are the worst offenders, and each is written as its ordered
|
||
on-screen steps: unbuttoning a shirt (pinch the button, push it through the
|
||
buttonhole, pull the plackets apart, work down one at a time), changing a diaper
|
||
(undo the waist tabs, fold the panel down, lift the ankles, slide the old one out,
|
||
wipe, lay the fresh one under, press each tab closed), turning on a lamp (flip the
|
||
switch with one finger, hold still while the bulb flickers and settles), opening a
|
||
door (press the lever down, pull the door toward you, step through), buckling a
|
||
strap (pull it across, click the tongue home, tug the free end to tighten), handing
|
||
over an object (lift it, hold it out, wait for the other hand to close, release).
|
||
- Spell out every change and movement step by step — not only the subject's actions.
|
||
Camera, lighting, weather, background, wardrobe, and any transformation each get
|
||
their own ordered sequence of visible stages ("the camera pushes in one slow step,
|
||
then another, until his hands fill the lower frame"); never a summarized change
|
||
("the lights dim"). H3 plays back only what is stated; a summarized change or
|
||
movement is a gap the model fills by guessing.
|
||
- Open by continuing the previous `_handoff` end_state ("Continue the same stride",
|
||
"without resetting the motion", "keep the incoming camera direction"); scene 1 sets the
|
||
opening. End mid-action except the final scene. No camera cuts inside a scene.
|
||
- Never compress a change into a stated outcome: a line like "<Subject 1> is now an 18
|
||
year old man" is FORBIDDEN. Show every transformation (age, wardrobe, weather, lighting,
|
||
location, size) as a continuous on-screen progression while motion and camera keep
|
||
flowing.
|
||
- Soundscape: one or two dense clauses matching the picture, continuing the previous audio
|
||
bed ("the same rain on glass continues"); diegetic music always named with its in-scene
|
||
source; `non_diegetic_music: N/A` unless the user asked for a score; sung/spoken words
|
||
are dialogue and never live in the soundscape line.
|
||
- Dialogue: silence is the default. Spoken words come only from the outline row or the
|
||
story, reproduced word for word inside case-sensitive `<d>[English] words</d>` (one space
|
||
after the marker; other real languages allowed), inside the [Shot 1] body only, with
|
||
a "silently with mouth {position}" cue before and after every span. Mirror each span in
|
||
`_handoff.dialogue_used` (words only, no marker); empty array when silent.
|
||
- `_handoff` must be truthful: `end_state` describes the real final frame (2–3 sentences:
|
||
pose, motion in progress, camera state); `soundscape` copies the overall_soundscape text
|
||
(no section label) character for character.
|
||
- Keys are exactly `id`, `duration_seconds`, `prompt`, `_handoff`; add `steps` or `seed`
|
||
only when the user explicitly asked. Every duration 2–15 s.
|
||
|
||
## Revising after approval
|
||
|
||
- Guide-level change (durations, beats, dialogue, scene count): update `outline.json`
|
||
first, show the changed rows, then rewrite only the affected scenes against it.
|
||
- One scene reads badly: rewrite that `scene_NN.json` (keep its id and duration from
|
||
the guide), re-validate it with the fragment command, then re-run the Step 4 merge so
|
||
`plan.json` refreshes. Never patch `plan.json` directly.
|
||
|
||
## Short stories (2–4 scenes)
|
||
|
||
You may use the single-pass prompt instead — read
|
||
`h3-plan-gate/h3_plan_json_system_prompt.md`, have its rules govern one turn that writes
|
||
the whole plan, then run `--check` on the file. Use the full workflow for long stories,
|
||
per-scene iteration, or when the user wants to revise single scenes or the guide.
|
||
|
||
## If you cannot run Python
|
||
|
||
Fall back to `h3-plan-gate/h3_plan_assembler_system_prompt.md` for the merge, copy fragment
|
||
text verbatim, and say plainly that the result skipped script validation.
|