mmprompt/README.md
Z User 54a8271072 Multi-clip videos as the primary deliverable: per-scene clip export
build_plan.py --clips [DIR] (default clips/) now exports one validated
single-shot plan JSON per scene (clip_01.json, ...) alongside the merged
plan. Each clip renders through plan_json_input as its own video at FULL
duration (no chain overlap trim), then the clips are stitched in order;
the mid-motion open/close craft rules make the cuts read as one take.
plan.json stays as the optional single continuous-take render.

SKILL.md Step 3/4 rewritten (merge exports clips + plan; deliver =
render-per-clip then stitch), README intro/example/wiring/validating
updated to multi-clip-first with real clip output shown. Verified from a
simulated project root: export green, every clip re-checks PLAN OK.
2026-10-29 18:40:18 +00:00

232 lines
12 KiB
Markdown
Executable File
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# mmprompt
## h3-plan-gate — MiniMax H3 plan-JSON agent workflow + validator
Prompt set and tooling that turn a story idea into the `plan_json_input` JSON for the
**Production Plan** node (`MiniMax H3 Context Loop Plan` / `MiniMax H3 Plan (Modern)`) in
[ComfyUI-MiniMaxH3-Context-Loop](https://github.com/ethanfel/ComfyUI-MiniMaxH3-Context-Loop),
verified against commit `43afeff`. The workflow is agent-shaped: one outline turn, one turn
per scene, then a validated merge — so long stories are written scene by scene and single
scenes can be regenerated without touching the rest. The merge exports **one render-ready
JSON per scene** (`clips/clip_NN.json`) for making **multi-clip videos** — each scene is
rendered as its own clip and stitched in an editor — plus an optional merged `plan.json`
for a single continuous chain take.
### Files
| File | Role |
|---|---|
| `SKILL.md` | Agent Skills entry point — this whole folder is a skill. Cline: copy it to `.cline/skills/h3-plan-gate/` (workspace) or `~/.cline/skills/` (global); any Agent-Skills-compatible agent (Claude Code, Cursor, OpenCode, Codex) can load it too |
| `h3_scene_agent_system_prompt.md` | Stage 1 + 2 system prompts: **PROMPT A** (outline planner) and **PROMPT B** (one-scene-per-turn writer with a `_handoff` chain) |
| `h3_plan_assembler_system_prompt.md` | Stage 3 LLM merge prompt — only when you cannot run scripts |
| `h3_plan_json_system_prompt.md` | Single-pass variant (whole plan in one turn) — fastest for 2–4 scene stories |
| `build_plan.py` | Deterministic Stage 3 merger: strips `_handoff`, injects `"prompt_prefix": [""]` + defaults, validates everything before writing; `--clips` exports one single-scene plan JSON per scene for multi-clip rendering |
| `verify_plan_rules.py` | Node-replica validator (final plans *and* per-turn fragments) with an embedded test suite |
| `count_prompt_tokens.py` | o200k token counter for the prompt fences (needs `pip install tiktoken`) |
| `h3_agent_workflow_examples/` | Worked 3-scene greenhouse-courier run: `outline.json` → `scene_01..03.json` → `plan.json` |
### Agent workflow (3 stages)
**Stage 1 — Outline.** Put PROMPT A (from `h3_scene_agent_system_prompt.md`) in the system
field, the story idea in the user field. Save the reply as `outline.json`. It freezes scene
count, ids, durations, beats, verbatim dialogue distribution, and the `style_and_cast` block
so later turns cannot drift. You may hand-edit it before Stage 2 — the scene agents follow
the edited skeleton.
**Stage 2 — One scene per turn.** Put PROMPT B in the system field and paste this turn
template as the user message:
```text
STORY: <the full story idea>
STYLE AND CAST: <paste the "style_and_cast" object from the outline>
OUTLINE: <paste the "scenes" array from the outline>
WRITE SCENE: 2
PREVIOUS HANDOFF: <paste the previous scene's "_handoff" object — or: This is scene 1.>
```
Save each reply as `scene_NN.json` (zero-padded). Each fragment carries the native H3
three-line prompt (`integrated_multimodal_description: [Shot 1] …` / `overall_soundscape: …`
/ `non_diegetic_music: …`), the duration-tiered word budget, and a truthful `_handoff`
object for the next turn. Optional gate after each turn:
```sh
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py fragment scene_02.json --index 2 --prev scene_01.json
```
(All command blocks below use the Windows venv form; on macOS/Linux replace
`.venv\Scripts\python.exe` with `python3`.)
**Stage 3 — Assembly (deterministic, preferred):**
```sh
.venv\Scripts\python.exe h3-plan-gate/build_plan.py outline.json scene_01.json scene_02.json scene_03.json -o plan.json
```
The merger validates every fragment in the `_handoff` chain (against the outline when
present), then validates the merged plan — only clean JSON is written. Do **not** edit
fragment prose by hand during assembly; if you want a scene changed, re-run its Stage 2
turn and re-merge. Use the LLM assembler prompt only in a pure-chat setup with no script
execution, and diff its output against the fragments if you do (small models transcribe
long strings unreliably; `build_plan.py` is byte-exact by construction).
## Example: story idea → final output
The complete run of the shipped example (`h3_agent_workflow_examples/`) — the same files
your agent should produce for any story.
**1. Give Cline the story idea.** With the skill installed, the whole request is:
```text
Use the h3-plan-gate skill. Story: a courier delivers a package to a greenhouse in the
rain, three shots. When he reaches the table he says exactly: Package for the greenhouse,
right on time.
```
**2. The agent runs the smoke test** — proving the toolchain before writing anything:
```sh
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py
.venv\Scripts\python.exe h3-plan-gate/build_plan.py --check h3-plan-gate/h3_agent_workflow_examples/plan.json
```
Expect the suite to pass and `PLAN OK — 1042 delivered frames = 43.42s`.
**3. The agent writes `outline.json`** — the skeleton that freezes cast, style, ids,
durations, and dialogue so later scenes cannot drift:
```json
{
"title": "greenhouse_courier",
"defaults": { "duration_seconds": 15, "steps": 20 },
"style_and_cast": {
"subjects": ["<Subject 1> — an adult bicycle courier; mustard-yellow waterproof jacket, dark teal trousers, charcoal helmet, olive messenger bag"],
"style": "Photorealistic overcast rainy daylight, soft grey palette, stable landscape tracking shots at eye level",
"soundscape_seed": "steady light rain drumming on glass roofing, low distant traffic hum"
},
"scenes": [
{ "index": 1, "id": "greenhouse_arrival", "duration_seconds": 15,
"beat": "He wheels his silver bicycle through the greenhouse entrance out of the rain and crosses the interior threshold mid-stride.",
"dialogue": [] },
{ "index": 2, "id": "greenhouse_aisle", "duration_seconds": 15,
"beat": "He pushes the bike down the central aisle toward the potting table and slows it to a crawl, one hand reaching for the bag flap.",
"dialogue": [] },
{ "index": 3, "id": "greenhouse_delivery", "duration_seconds": 15,
"beat": "He sets the kraft parcel on the wooden potting table and speaks his line as the delivery completes.",
"dialogue": ["Package for the greenhouse, right on time."] }
]
}
```
**4. The agent writes the scenes one file at a time**, validating each before moving on:
```sh
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py fragment scene_01.json --index 1 --total 3
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py fragment scene_02.json --index 2 --total 3 --prev scene_01.json
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py fragment scene_03.json --index 3 --total 3 --prev scene_02.json
```
Each fragment carries the three-line H3 prompt and a truthful `_handoff`. Scene 3
(description elided — the real file carries ~237 words of action, camera, wardrobe, and
lighting detail):
```json
{
"id": "greenhouse_delivery",
"duration_seconds": 15,
"prompt": [
"integrated_multimodal_description: [Shot 1] Continue the same forward stride … He stands silently with mouth closed, then speaks. <d>[English] Package for the greenhouse, right on time.</d> He falls silently with mouth closed … End on the calm wide composition as the delivery completes.",
"overall_soundscape: the same rain on glass and wet paving continues, tires creak to a stop, the kraft paper parcel thuds softly on the wooden potting table.",
"non_diegetic_music: N/A"
],
"_handoff": { "end_state": "…", "soundscape": "…", "dialogue_used": ["Package for the greenhouse, right on time."] }
}
```
**5. The agent merges, exports the clips, and checks** — never writing plan JSON by hand:
```sh
.venv\Scripts\python.exe h3-plan-gate/build_plan.py outline.json scene_01.json scene_02.json scene_03.json -o plan.json --clips
.venv\Scripts\python.exe h3-plan-gate/build_plan.py --check plan.json
```
Output:
```text
outline: outline.json (3 scenes)
fragment 1/3 … fragment 2/3 … fragment 3/3 …
Merged plan:
shot 1 id='greenhouse_arrival' raw=362 delivered=362 steps=20
shot 2 id='greenhouse_aisle' raw=362 delivered=340 steps=20
shot 3 id='greenhouse_delivery' raw=362 delivered=340 steps=20
total delivered: 1042 frames = 43.42s
wrote plan.json - paste its full contents into plan_json_input
clip 1/3 -> clips/clip_01.json (id=greenhouse_arrival)
shot 1 id='greenhouse_arrival' raw=362 delivered=362 steps=20
total delivered: 362 frames = 15.08s
…
clip 3/3 -> clips/clip_03.json (id=greenhouse_delivery)
shot 1 id='greenhouse_delivery' raw=362 delivered=362 steps=20
total delivered: 362 frames = 15.08s
…
wrote 3 single-scene clip plans -> clips/ - render each clip_NN.json through
plan_json_input as its own video, then stitch the clips in order
```
Two deliverables: **`clips/clip_01.json`, `clip_02.json`, `clip_03.json`** — one
single-scene plan per clip, the input for a multi-clip video — and the merged
`plan.json` for an optional single continuous take (full file:
`h3_agent_workflow_examples/plan.json`).
**6. Render the multi-clip video (the main use).** For each clip in order: paste the full
contents of `clips/clip_NN.json` into the Production Plan node's `plan_json_input`, queue
one render, save the output — then stitch the clips in order in any editor. Every clip
renders at its FULL duration (three 15-second scenes = 45 seconds of footage; the chain
overlap trim does not apply to independent clips), and because every description opens
mid-motion and ends mid-action, the cuts read as one continuous take.
*Alternative — one continuous take:* paste `plan.json` instead and the node renders the
whole story as a single chain video (1042 delivered frames ≈ 43.4 s at 24 fps — every
scene after the first loses ~1 second to overlap trimming).
Want scene 2 different? Just tell the agent "rewrite scene 2, keep everything else": it
rewrites only `scene_02.json`, re-validates, and re-merges — refreshing both the clip and
the merged plan.
### Validating
```sh
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py # embedded pass/fail suite
.venv\Scripts\python.exe h3-plan-gate/verify_plan_rules.py plan plan.json # node-rule replica on a final plan
.venv\Scripts\python.exe h3-plan-gate/build_plan.py --check plan.json # same checks + frame/duration report
.venv\Scripts\python.exe h3-plan-gate/build_plan.py --check clips/clip_01.json # clips validate the same way
```
Smoke test with the shipped examples:
```sh
.venv\Scripts\python.exe h3-plan-gate/build_plan.py --check h3-plan-gate/h3_agent_workflow_examples/plan.json
# expect: PLAN OK — 1042 delivered frames = 43.42s
```
### Wiring the result
For a **multi-clip video**, render one clip at a time: paste the full contents of
`clips/clip_NN.json` into the Plan node's forced STRING input **`plan_json_input`** (a
non-empty connected value overrides the visual editor's plan for that run), queue, save,
next clip — then stitch in order. For a **single continuous take**, paste `plan.json`
instead and render once. Resolution, context length, audio mode, and `base_seed` stay on
the node's own widgets (the JSON cannot set them; default canvas is 960×544 — set
width/height widgets and wire the width/height outputs into the conditioning node and
sampler latent to avoid square output).
### Requirements
- `build_plan.py` and `verify_plan_rules.py`: Python 3 stdlib only — no venv packages
needed; the `.venv\Scripts\python.exe` form just matches a typical Windows project
layout, and plain `python` / `python3` works anywhere.
- `count_prompt_tokens.py`: needs `tiktoken`.
- LLM settings: temperature 0.3–0.7; enable `response_format: {"type": "json_object"}` or
grammar-constrained JSON if the server supports it. Each scene turn is ≈5k tokens, so
ctx 16384 is comfortable; 128k runs the whole workflow many times over.