easy-media-track-gate: emit the minimax_prompt_override paste payload

The multitrack editor node consumes its prompt_override input as a
minimax_prompt_override JSON payload (type/version/prompts/duration/
generation_type/continuity_mode/system_prompt/track locks). Any other
JSON pasted there falls back to the '|'-separated text parser and
renders ONE clip over the whole timeline - the reported single-clip
failure - so the paste payload is now the primary deliverable.

- verify_track_rules.py: build_prompt_override() assembles the payload
  from validated fragments (one prompt per row, <Picture N>/<Audio N>/
  <Video N> tokens by inventory position, duration 2-15s, per-segment
  generation/continuity, tail reuse like the runtime); validate_prompt_
  override() checks type, field set, prompt count, duration/mode
  agreement with the outline, exact reference-token sets, pipe-free
  prompts, and track locks; new 'override' sub-command; suite 25->34.
- build_track_plan.py: merge now writes prompt_override.json (one line,
  ASCII) next to plan.json and validates it before reporting.
- references/track-schema.md: full prompt_override section with the
  single-clip fallback warning and slot-position convention.
- SKILL.md: delivery restructured (5A paste default, 5B patch script
  alternative), smoke test extended, read-cap sizes refreshed.
- examples/prompt_override.json: shipped example payload (runtime-checked
  against the cloned node pack parser: 3 task segments, 720 frames).
This commit is contained in:
Z User 2026-11-01 02:17:23 +00:00
parent 56b9d3b9e2
commit 8434bb4645
5 changed files with 456 additions and 45 deletions

View File

@ -1,25 +1,32 @@
--- ---
name: easy-media-track-gate name: easy-media-track-gate
description: Turn a video idea into a validated MultiTrack patch plan for ComfyUI-Easy-Media description: Turn a video idea into a validated MultiTrack plan for ComfyUI-Easy-Media
(the `easy multiTrackEditor` / `easy multitrackProject` nodes). Use when the user wants to (the `easy multiTrackEditor` node). Use when the user wants to plan task segments/shots
plan task segments/shots on the multitrack timeline, write or extend a MultiTrack timeline, on the multitrack timeline, write or extend a MultiTrack timeline, arrange prompts,
arrange prompts, reference images, master audio or reference-video locks, regenerate one reference images, master audio or reference-video locks, regenerate one task segment,
task segment, or produce the patch plan JSON that `patch_workflow.py apply` consumes. The or produce the `prompt_override` paste payload for the multitrack editor node (plus the
scripts and references live next to this SKILL.md (the skill's install folder); run them optional patch plan JSON that `patch_workflow.py apply` consumes). The scripts and
from the user's working folder, where all generated files are written. references live next to this SKILL.md (the skill's install folder); run them from the
user's working folder, where all generated files are written.
--- ---
# Easy-Media Track Gate — agent procedure # Easy-Media Track Gate — agent procedure
Goal: produce `plan.json`, the patch plan applied to a ComfyUI workflow with the Easy-Media Goal: produce a validated plan for the `easy multiTrackEditor` node in five moves: the user
node pack's own `patch_workflow.py`, in five moves: the user gives a video idea and a gives a video idea and a segment count (or an existing workflow to edit) → you build a
segment count (or an existing workflow to edit) → you build a timeline guide (`outline.json`) timeline guide (`outline.json`) and show it for **approval** → with the approved guide you
and show it for **approval** → with the approved guide you compose the task segments one at compose the task segments one at a time (about 10 seconds each), each turn existing so that
a time (about 10 seconds each), each turn existing so that ONE segment gets a complete ONE segment gets a complete prompt, the right references and the right continuity mode →
prompt, the right references and the right continuity mode → the scripts verify schema, the scripts verify schema, timing and continuity across the whole chain →
timing and continuity across the whole chain → `build_track_plan.py` assembles the TRACK_DATA `build_track_plan.py` assembles the deliverables at the end. The PRIMARY deliverable is
and the patch plan at the end. Never deliver a plan that did not pass validation, and never `prompt_override.json`, a `minimax_prompt_override` payload the user pastes into the
compose a segment from a guide the user has not approved. multitrack editor node's `prompt_override` input (format `MiniMax`) — the editor then
renders one clip per prompt, chained back-to-back. The patch plan (`plan.json` for the node
pack's `patch_workflow.py`) is the optional whole-workflow alternative. Never deliver a plan
that did not pass validation, and never compose a segment from a guide the user has not
approved. Never hand the user any other JSON as the paste: a non-`minimax_prompt_override`
value in `prompt_override` falls back to the editor's `|`-separated text parser and renders
the whole story as ONE clip over the entire timeline.
## Paths and Python (read first — these exact forms prevent most errors) ## Paths and Python (read first — these exact forms prevent most errors)
@ -38,14 +45,15 @@ compose a segment from a guide the user has not approved.
SKILL_DIR is missing files, stop and tell the user the skill folder is incomplete. SKILL_DIR is missing files, stop and tell the user the skill folder is incomplete.
- **Never read `verify_track_rules.py`** — run it and work from its printed output. Cline's - **Never read `verify_track_rules.py`** — run it and work from its printed output. Cline's
read_file is capped by hard-coded budgets that do NOT grow with the model's context read_file is capped by hard-coded budgets that do NOT grow with the model's context
window: about 48,000 characters per read, 2,000 lines per read. The 66 KB validator window: about 48,000 characters per read, 2,000 lines per read. The 81 KB validator
arrives cut no matter how large the context window is. A truncation notice is Cline's arrives cut no matter how large the context window is. A truncation notice is Cline's
read cap, not repo corruption — do not "fix" truncated files. `build_track_plan.py` read cap, not repo corruption — do not "fix" truncated files. `build_track_plan.py`
(~9 KB) and `references/track-schema.md` (~22 KB, 465 lines) are safe to read whole. (~10 KB) and `references/track-schema.md` (~26 KB) are safe to read whole.
- **Never read back `plan.json` or `track_data.json` once real segments exist** — a - **Never read back `plan.json`, `track_data.json` or `prompt_override.json` once real
10-segment plan passes the read caps today and exceeds them tomorrow. Validate with segments exist** — a 10-segment plan passes the read caps today and exceeds them
`--check` and trust the one-line OK summary; the shipped `examples/plan.json` (6.8 KB) tomorrow. Validate with `--check` / the `override` command and trust the one-line OK
is safe to read if you need to see the shape. summary; the shipped `examples/prompt_override.json` (2.5 KB) and `examples/plan.json`
(6.8 KB) are safe to read if you need to see the shape.
- Cline's editor rejects writes over 6,000 characters ("Editor input too large"). A segment - Cline's editor rejects writes over 6,000 characters ("Editor input too large"). A segment
fragment with a rich prompt often does. NEVER write part 2 with write_to_file — it fragment with a rich prompt often does. NEVER write part 2 with write_to_file — it
replaces the whole file and part 1 is lost. Split instead: write the first part ending replaces the whole file and part 1 is lost. Split instead: write the first part ending
@ -61,10 +69,12 @@ compose a segment from a guide the user has not approved.
```sh ```sh
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\plan.json" py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\plan.json"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\prompt_override.json" --outline "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\outline.json"
``` ```
Expect the suite to print `SUITE PASS (25/25 cases)` and the second command to print Expect the suite to print `SUITE PASS (34/34 cases)` and the other two commands to print
`PLAN OK` with `total_length 720 frames = 30.00s`. `PLAN OK` with `total_length 720 frames = 30.00s` and `OVERRIDE OK: 3 prompts, duration 10,
total 30.00s`.
## Step 0 — Media paths come first (never invent one) ## Step 0 — Media paths come first (never invent one)
@ -128,7 +138,10 @@ Field rules the validator enforces:
- `media_inventory` keys are UPPERCASE; every `file_path` is copied from Step 0 sources — - `media_inventory` keys are UPPERCASE; every `file_path` is copied from Step 0 sources —
this inventory is the **machine-checked verbatim source**: every segment's image items and this inventory is the **machine-checked verbatim source**: every segment's image items and
every locked media track are diffed against it, so a retyped path fails on the spot. A every locked media track are diffed against it, so a retyped path fails on the spot. A
paraphrase in the inventory cannot be repaired later. paraphrase in the inventory cannot be repaired later. Order matters too: the inventory's
image rows map to the editor's `image` input positions (`<Picture 1>`, `<Picture 2>`, …),
audio rows to `audio` input positions, video rows to `video` input positions — the user
wires the media into those node inputs in inventory order.
- `tracks.master_audio` / `tracks.reference_video` name the ONE locked audio and ONE locked - `tracks.master_audio` / `tracks.reference_video` name the ONE locked audio and ONE locked
video track (the schema allows at most one of each). Lock the master song for MV/lyric/ video track (the schema allows at most one of each). Lock the master song for MV/lyric/
dance/beat projects, the narration track when narration drives the cut, and the reference dance/beat projects, the narration track when narration drives the cut, and the reference
@ -264,23 +277,56 @@ Once every segment file passes, the merge is the continuity check: `build_track_
re-walks every fragment against the guide (ids, frames, modes, verbatim media paths, prompt re-walks every fragment against the guide (ids, frames, modes, verbatim media paths, prompt
rules, handoff chain), assembles the TRACK_DATA (task track from the fragments; the locked rules, handoff chain), assembles the TRACK_DATA (task track from the fragments; the locked
master-audio / reference-video tracks built straight from the outline), computes master-audio / reference-video tracks built straight from the outline), computes
`total_length`, and only then writes the deliverable. Never write the plan by hand and `total_length`, builds the `minimax_prompt_override` paste payload from the same validated
never edit fragment prose during the merge. fragments, and only then writes the deliverables. Never write the plan by hand and never
edit fragment prose during the merge.
```sh ```sh
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" outline.json segment_01.json segment_02.json segment_03.json -o plan.json py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" outline.json segment_01.json segment_02.json segment_03.json -o plan.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check plan.json --outline outline.json py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check plan.json --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override prompt_override.json --outline outline.json
``` ```
The merge writes `plan.json` (the patch plan) and `track_data.json` (pretty, for The merge writes `prompt_override.json` (the paste payload, one line), `plan.json` (the
inspection) and prints the delivered frames/seconds, the track list and the locked tracks. patch plan) and `track_data.json` (pretty, for inspection), and prints the delivered
Segment, track and image ids are stable UUIDv5 values derived from the project title, so a durations/seconds, the track list and the locked tracks. Segment, track and image ids are
re-merge keeps ids consistent. stable UUIDv5 values derived from the project title, so a re-merge keeps ids consistent.
The `override` command re-checks the paste payload against the guide: one prompt per
segment, durations inside the runtime's 2–15s window and equal to the approved rows,
per-segment generation/continuity modes, the exact `<Picture N>` / `<Audio N>` / `<Video N>`
reference set per prompt (a missing token renders with no reference media; a stray one
attaches the wrong image), and the track locks.
## Step 5 — Deliver (the node pack's own script applies it) ## Step 5 — Deliver (paste payload first, patch script as the alternative)
The deliverable is applied to the user's workflow by the Easy-Media node pack's script — ### 5A — Default: the `prompt_override` paste
find it under the node pack folder (typically
`prompt_override.json` is what the user pastes into the **multitrack editor node's
`prompt_override` input** (`easy multiTrackEditor`). It is self-contained: one prompt per
clip, per-clip durations and continuity modes, media reference tokens inside the prompts.
Deliver it with these instructions:
1. In the editor node set `format` to `MiniMax` (the payload is parsed as MiniMax only in
that format).
2. Wire the media: the inventory's image rows (in inventory order) into the editor's
`image` input, the master song into `audio`, the reference video into `video`. The
`<Picture N>` / `<Audio N>` / `<Video N>` tokens resolve against those input lists
positionally.
3. Select the `prompt_override` widget, clear it, paste the whole one-line JSON, queue.
The editor rebuilds the task track: one clip per prompt, back-to-back, durations as
specified — 3 prompts at 10s render as 3 chained ~10-second clips.
**The paste target is the `prompt_override` input and only that payload type.** Never give
the user a `plan.json`, an outline, an H3 plan or any other JSON to paste there: without
`"type": "minimax_prompt_override"` the editor falls back to its `|`-separated text parser
and renders the whole thing as ONE clip spanning the timeline (the single-clip failure).
Never add `|` inside prompts for the same reason.
### 5B — Alternative: the node pack's patch script (whole-workflow patch)
When the user wants the timeline written into a workflow file directly (or needs `edit`
task modes / extra tracks the paste payload cannot express), the deliverable is applied by
the Easy-Media node pack's script — find it under the node pack folder (typically
`<ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py`; `<ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py`;
ask the user for the path and the workflow JSON if they are not at hand): ask the user for the path and the workflow JSON if they are not at hand):
@ -306,10 +352,11 @@ the node pack's own flow (see its `references/execution-api.md`). "Queued" never
- Guide-level change (timings, beats, modes, segment count, media): update `outline.json` - Guide-level change (timings, beats, modes, segment count, media): update `outline.json`
first, show the changed rows, then rewrite only the affected segments against it and first, show the changed rows, then rewrite only the affected segments against it and
re-run the Step 4 merge so `plan.json` refreshes. re-run the Step 4 merge so `plan.json` and `prompt_override.json` refresh.
- One segment reads badly: rewrite that `segment_NN.json` (keep its id and frames from the - One segment reads badly: rewrite that `segment_NN.json` (keep its id and frames from the
guide), re-validate it with the `segment` command, then re-merge. Never patch `plan.json` guide), re-validate it with the `segment` command, then re-merge. Never patch `plan.json`
directly — the merge rebuilds it from the fragments every time. or `prompt_override.json` directly — the merge rebuilds both from the fragments every
time.
## If you cannot run Python ## If you cannot run Python

View File

@ -1,19 +1,22 @@
#!/usr/bin/env python3 #!/usr/bin/env python3
"""Merge approved outline + task-segment fragments into the Easy-Media MultiTrack patch plan. """Merge approved outline + task-segment fragments into the Easy-Media MultiTrack deliverables.
py -3 build_track_plan.py outline.json segment_01.json segment_02.json -o plan.json py -3 build_track_plan.py outline.json segment_01.json segment_02.json -o plan.json
py -3 build_track_plan.py --check plan.json [--outline outline.json] py -3 build_track_plan.py --check plan.json [--outline outline.json]
The merge is the continuity check: it re-validates every fragment against the approved The merge is the continuity check: it re-validates every fragment against the approved
outline (ids, frames, modes, verbatim media paths, context-prompt rules, handoff chain), outline (ids, frames, modes, verbatim media paths, context-prompt rules, handoff chain),
then assembles the TRACK_DATA (task track + locked master-audio / reference-video tracks then assembles TWO deliverables from it:
built straight from the outline, never retyped) and the patch plan the node pack's
`patch_workflow.py apply` consumes:
{"editor": {"resolution": ..., "track_data": {...}, "format": ...}, 1. `prompt_override.json` — the PRIMARY paste payload for the `easy multiTrackEditor`
"project": {...}} node's `prompt_override` input (format MiniMax): {"type": "minimax_prompt_override",
...} with one prompt per segment, per-segment durations and continuity modes. Paste it
into the widget and queue; wire the inventory's media into the editor's image/audio/
video inputs in inventory order so the <Picture N>/<Audio N>/<Video N> tokens resolve.
2. `plan.json` — the patch plan the node pack's `patch_workflow.py apply` consumes
({"editor": {..., "track_data": {...}}, "project": {...}}) for patching a whole
workflow file, plus `track_data.json` (pretty, for inspection).
It writes `plan.json` (the deliverable) and `track_data.json` (pretty, for inspection).
Segment/track/image ids are deterministic UUIDv5 values derived from the project title, so Segment/track/image ids are deterministic UUIDv5 values derived from the project title, so
re-running the merge keeps ids stable. Never edit fragment prose here and never write the re-running the merge keeps ids stable. Never edit fragment prose here and never write the
plan by hand — fix the fragment, re-validate, re-merge. plan by hand — fix the fragment, re-validate, re-merge.
@ -208,9 +211,22 @@ def _main(argv: list[str]) -> int:
out_path = Path(args.output) out_path = Path(args.output)
_write_json(out_path, plan) _write_json(out_path, plan)
_write_json(out_path.parent / "track_data.json", plan["editor"]["track_data"]) _write_json(out_path.parent / "track_data.json", plan["editor"]["track_data"])
override = vtr.build_prompt_override(outline, fragments)
vtr.validate_prompt_override(copy.deepcopy(override), outline)
override_path = out_path.parent / "prompt_override.json"
override_path.write_text(
json.dumps(override, ensure_ascii=True, separators=(",", ":")) + "\n",
encoding="utf-8")
prompts = len(override["prompts"])
total_s = sum(float(row["duration_seconds"]) for row in outline["rows"])
print(f"OVERRIDE OK: {prompts} prompts, duration {override['duration']}, "
f"total {total_s:.2f}s; wrote {override_path} "
f"({override_path.stat().st_size} chars, one line)")
print(f"PLAN OK: {_summary(plan)}") print(f"PLAN OK: {_summary(plan)}")
print(f"wrote {out_path} and {out_path.parent / 'track_data.json'}") print(f"wrote {out_path} and {out_path.parent / 'track_data.json'}")
print("next: dry-run the node pack's patch script before writing:") print("deliver: paste prompt_override.json into the multitrack editor's "
"prompt_override input (format MiniMax), then queue")
print("optional whole-workflow patch (dry-run first):")
print(' py -3 <ComfyUI-Easy-Media>\\skills\\easy-media-multitrack-workflow\\scripts\\patch_workflow.py apply WORKFLOW.json --plan plan.json') print(' py -3 <ComfyUI-Easy-Media>\\skills\\easy-media-multitrack-workflow\\scripts\\patch_workflow.py apply WORKFLOW.json --plan plan.json')
print(' ... then add --output RESULT.json --write and inspect RESULT.json') print(' ... then add --output RESULT.json --write and inspect RESULT.json')
return 0 return 0

View File

@ -0,0 +1 @@
{"type":"minimax_prompt_override","version":1,"prompts":["<Picture 1> <Audio 1> Dawn light spills across an empty rooftop terrace above a hushed city. A young woman with a brown ponytail, cream cardigan over a white tee and blue jeans stands at the steel rail, matching the pose reference image for her identity only. She cradles a matte-black kettle in her right hand and tips it in one slow arc; a thread of steam rises as coffee fills the white cup in her left hand, her wrist rotating smoothly, elbow dropping as the kettle empties. She sets the kettle on the rail with a soft clink, lifts the cup, and brings it halfway to her lips as the sun's first rim crests the skyline and warms her cheek. The camera holds one slow push-in toward her profile, never cutting, while pigeon wings clap once somewhere below and the kettle's metal ticks as it cools.","<Picture 1> <Audio 1> The camera holds the same framing it arrived with, and she keeps the pose, motion and light it arrived with: the cup still traveling toward her lips, steam bending in the same breeze. Her chest rises with one slow breath, weight easing onto her left hip as the cardigan's hem starts to lift. Around the two-second mark the wind strengthens into a gust that ruffles her ponytail and snaps the cardigan open a finger's width; her fingers reclose it while she turns her face slowly into the light, eyelids narrowing. Steam tears off the cup in ribbons. The camera keeps its single slow drift leftward with no cut, the city below sharpening frame by frame, traffic swelling from hum to roar, a rooftop antenna creaking softly, her cardigan snapping in steady wind as the turn continues.","<Picture 1> <Audio 1> The join keeps everything it is handed: same framing, same breeze, the tail of her turn still in motion as her face comes level with the camera. She continues the turn until her eyes find the skyline, then lowers the cup in one unhurried arc to the rail, her shoulder rolling with the descent. Below, windows begin to light up floor by floor, a slow tide of warm rectangles climbing the tower across the street while a bus door hisses open. She exhales a long breath, steam curling off the cup in the same wind, ponytail still lifting, cardigan snapping softly. The camera keeps its slow lateral drift and never cuts; the city's morning roar builds under the song as her chest keeps rising and falling mid-breath."],"duration":"10","generation_type":"r2v","continuity_mode":"shot,context,context","system_prompt":"","video_track_lock":0,"audio_track_lock":1}

View File

@ -463,3 +463,56 @@ python3 scripts/patch_workflow.py apply SOURCE.json --plan PLAN.json
python3 scripts/patch_workflow.py apply SOURCE.json --plan PLAN.json --output RESULT.json --write python3 scripts/patch_workflow.py apply SOURCE.json --plan PLAN.json --output RESULT.json --write
python3 scripts/patch_workflow.py inspect RESULT.json python3 scripts/patch_workflow.py inspect RESULT.json
``` ```
---
## prompt_override — the paste payload (format MiniMax)
The `easy multiTrackEditor` node has an optional `prompt_override` input. When the editor's
`format` is `MiniMax` and the pasted value is a `minimax_prompt_override` JSON payload, the
editor rebuilds its whole task track from that payload: one task segment per `prompts`
entry, laid back-to-back, each with its own duration. `total_length` becomes the sum of the
durations, so a 3-prompt payload at 10s each renders as THREE chained ~10-second clips.
```json
{"type":"minimax_prompt_override","version":1,
"prompts":["<Picture 1> <Audio 1> prompt for segment one ...",
"<Picture 1> <Audio 1> prompt for segment two ..."],
"duration":"10","generation_type":"r2v","continuity_mode":"shot,context,context",
"system_prompt":"","video_track_lock":0,"audio_track_lock":1}
```
Field rules (mirrors `utils/prompt_override.py` in the node pack):
- `type` must be exactly `minimax_prompt_override` and `version` `1`. **Any other JSON
pasted here is not rejected — it falls back to the generic text parser that splits on
`|`.** A JSON blob contains no `|`, so it becomes ONE task segment spanning the whole
timeline with the JSON dump as its prompt: the classic "rendered the whole story as a
single 10 second clip" failure.
- `prompts` — one entry per clip, in timeline order. The prompt text is the clip's full
generation prompt (same standalone composition rules as task segments: no cross-segment
references, context joins keep-then-change, only the last clip may come to rest).
- Media attaches ONLY through tokens in the prompt text: `<Picture N>`, `<Audio N>`,
`<Video N>` (the `@imageN`/`@audioN`/`@videoN` forms are accepted and rewritten, but this
gate standardizes on the bracket form). N is the 1-based position of the item in the
editor's matching `image` / `audio` / `video` input list — wire the media inventory into
those inputs in inventory order. A prompt with no token renders with no reference media.
The runtime renumbers tokens compactly for display, but slots are wired from the raw N.
- `duration` — seconds per clip, single value ("10" applies to every clip) or comma list
("10,5,10") with tail reuse; each value must be 2–15, enforced at parse time.
- `generation_type` — per clip, same single/comma-list rule. `r2v` = reference-image task
(task_mode ref), `l2v` = last-frame task, `t2v`/`i2v`/`v2v` land as default/edit tasks.
This gate emits `r2v` for ref rows, `l2v` for l2v rows, `t2v` for default rows.
- `continuity_mode` — per clip: `shot`, `context`, `context_drift` (legacy `context_swap`
is normalized to `context_drift`). Matches the task-segment semantics.
- `system_prompt` — written into every clip when non-empty; an empty string lets the H3
runtime auto-append its reference rules for context joins (the gate's default).
- `video_track_lock` / `audio_track_lock` — 1-based index of the video/audio track to lock
(0 = none). With a locked master-audio inventory row, every prompt carries
`<Audio 1>` and `audio_track_lock` is 1; with a locked reference video, `<Video 1>` and
`video_track_lock` are 1.
The paste replaces the editor's existing task track (and derives its audio/video tracks
from the tokens), so the payload is self-contained: prompts, durations, modes and media
references all travel inside it. Edit task modes (`edit`) and full extra tracks are beyond
this payload — use the TRACK_DATA patch plan for those.

View File

@ -24,6 +24,9 @@ Sub-commands:
segment SEGMENT_NN.json --index N --total T --outline OUTLINE.json [--prev PREV.json] segment SEGMENT_NN.json --index N --total T --outline OUTLINE.json [--prev PREV.json]
trackdata TRACK_DATA.json [--outline OUTLINE.json] trackdata TRACK_DATA.json [--outline OUTLINE.json]
plan PLAN.json [--outline OUTLINE.json] plan PLAN.json [--outline OUTLINE.json]
override PROMPT_OVERRIDE.json --outline OUTLINE.json
validate the minimax_prompt_override paste
payload for the multitrack editor node
suite embedded self-test suite suite embedded self-test suite
Exit code 0 = pass, 1 = fail. Standard library only. Exit code 0 = pass, 1 = fail. Standard library only.
@ -68,6 +71,23 @@ PROJECT_FIELDS = {
EDITOR_PLAN_FIELDS = {"node_id", "track_data", "format", "resolution", "recalculate_total_length"} EDITOR_PLAN_FIELDS = {"node_id", "track_data", "format", "resolution", "recalculate_total_length"}
SNAKE_RE = re.compile(r"^[a-z][a-z0-9_]*$") SNAKE_RE = re.compile(r"^[a-z][a-z0-9_]*$")
# The multitrack editor's prompt_override input (format MiniMax) accepts exactly one JSON
# payload type. Anything else falls back to the '|'-separated text parser and renders ONE
# clip over the whole timeline — the classic single-clip failure.
OVERRIDE_TYPE = "minimax_prompt_override"
OVERRIDE_VERSION = 1
OVERRIDE_GEN_TYPES = {"r2v", "t2v", "i2v", "v2v", "l2v"}
OVERRIDE_DURATION_MIN = 2.0
OVERRIDE_DURATION_MAX = 15.0
OVERRIDE_FIELDS = {"type", "version", "prompts", "duration", "generation_type",
"continuity_mode", "system_prompt", "video_track_lock", "audio_track_lock"}
_TASK_MODE_TO_GEN = {"ref": "r2v", "l2v": "l2v", "default": "t2v"}
_PICTURE_RE = re.compile(r"<Picture\s*(\d+)\s*>", re.IGNORECASE)
_AUDIO_RE = re.compile(r"<Audio\s*(\d+)\s*>", re.IGNORECASE)
_VIDEO_RE = re.compile(r"<Video\s*(\d+)\s*>", re.IGNORECASE)
_ATREF_RE = re.compile(r"@(?:image|img|audio|video|\u56fe\u7247|\u56fe|\u97f3\u9891|\u89c6\u9891)\s*\d+", re.IGNORECASE)
MAX_IMAGES = 9 MAX_IMAGES = 9
PROMPT_MIN_WORDS = 60 PROMPT_MIN_WORDS = 60
END_STATE_MIN_WORDS = 15 END_STATE_MIN_WORDS = 15
@ -923,6 +943,223 @@ def validate_plan(plan: dict[str, Any], *, outline: dict[str, Any] | None = None
return track_data return track_data
# ------------------------------------------------- minimax prompt_override paste
def _override_media_positions(outline: dict[str, Any]) -> dict[str, int]:
"""1-based slot position per inventory key.
The editor's prompt_override attaches media ONLY through tokens in the prompt text
(`<Picture N>` / `<Audio N>` / `<Video N>`). N is the 1-based position of the media
item in the editor's image / audio / video input list, so this gate's convention is:
wire the inventory's rows into the editor's matching input in inventory order, then
N is the row's 1-based position among the rows of its own kind.
"""
positions: dict[str, int] = {}
counters = {"image": 0, "audio": 0, "video": 0}
for key, entry in outline["inventory"].items():
counters[entry["kind"]] += 1
positions[key] = counters[entry["kind"]]
return positions
def _override_tail_values(raw: Any, count: int, *, default: str, allowed: set[str],
field: str) -> list[str]:
"""Mirror the runtime's comma-list parsing: per-segment values, tail reused."""
text = default if raw is None else str(raw).strip()
if not text:
text = default
parts = [p.strip() for p in text.split(",") if p.strip()]
if not parts:
raise Fail(f"prompt_override.{field} must contain at least one value")
values: list[str] = []
for part in parts:
norm = part.lower()
if norm not in allowed:
raise Fail(f"prompt_override.{field} value {part!r} is not one of {sorted(allowed)}")
values.append(norm)
return [values[min(i, len(values) - 1)] for i in range(count)]
def _override_durations(raw: Any, count: int) -> list[float]:
text = "10" if raw is None else str(raw).strip()
if not text:
text = "10"
parts = [p.strip() for p in text.split(",") if p.strip()]
if not parts:
raise Fail("prompt_override.duration must contain at least one value")
out: list[float] = []
for part in parts:
try:
sec = float(part)
except ValueError:
raise Fail(f"prompt_override.duration has a non-numeric value: {part!r}")
if not OVERRIDE_DURATION_MIN <= sec <= OVERRIDE_DURATION_MAX:
raise Fail(
f"prompt_override.duration value {part!r} is outside the runtime's "
f"{OVERRIDE_DURATION_MIN:g}-{OVERRIDE_DURATION_MAX:g}s window; the editor "
"rejects the paste outright at that range")
out.append(sec)
return [out[min(i, len(out) - 1)] for i in range(count)]
def build_prompt_override(outline: dict[str, Any], fragments: list[dict[str, Any]]) -> dict[str, Any]:
"""Assemble the multitrack editor's paste payload from validated fragments.
One prompt per outline row (variant b honored), media reference tokens prepended,
per-segment duration / generation_type / continuity_mode lists with tail reuse.
"""
rows = outline["rows"]
if len(fragments) != len(rows):
raise Fail(f"expected {len(rows)} fragments (one per approved outline row), got {len(fragments)}")
positions = _override_media_positions(outline)
roles = outline["_roles"]
audio_pos = positions.get(roles["master_audio"]["key"]) if roles["master_audio"] else None
video_pos = positions.get(roles["reference_video"]["key"]) if roles["reference_video"] else None
prompts: list[str] = []
durations: list[str] = []
gen_types: list[str] = []
cont_modes: list[str] = []
for row, frag in zip(rows, fragments):
content = frag.get("content", {})
variant = content.get("user_prompt_variant", "a")
text = content.get("user_prompt_b") if variant == "b" else content.get("user_prompt")
if not isinstance(text, str) or not text.strip():
raise Fail(f"fragment {frag.get('id', '?')}: no prompt text for the override payload")
refs = ""
for img in row.get("images", []):
refs += f"<Picture {positions[img['key']]}> "
if audio_pos is not None:
refs += f"<Audio {audio_pos}> "
if video_pos is not None:
refs += f"<Video {video_pos}> "
prompts.append(" ".join((refs + text).split()))
durations.append(f"{float(row['duration_seconds']):g}")
gen_types.append(_TASK_MODE_TO_GEN[row["task_mode"]])
cont_modes.append(row["continuity_mode"])
def _compact(values: list[str]) -> str:
return ",".join(values) if len(set(values)) > 1 else values[0]
return {
"type": OVERRIDE_TYPE,
"version": OVERRIDE_VERSION,
"prompts": prompts,
"duration": _compact(durations),
"generation_type": _compact(gen_types),
"continuity_mode": _compact(cont_modes),
"system_prompt": "",
"video_track_lock": 1 if video_pos is not None else 0,
"audio_track_lock": 1 if audio_pos is not None else 0,
}
def _check_override_refs(prompt: str, row: dict[str, Any], outline: dict[str, Any], where: str) -> None:
positions = _override_media_positions(outline)
inventory = outline["inventory"]
expected_pics = sorted({positions[img["key"]] for img in row.get("images", [])})
found_pics = sorted({int(m.group(1)) for m in _PICTURE_RE.finditer(prompt)})
unexpected = [n for n in found_pics if n not in expected_pics]
if unexpected:
raise Fail(
f"{where} references <Picture {unexpected[0]}> which is not one of this segment's "
f"approved images (expected {expected_pics}); a stray token attaches the wrong "
"slot image from the editor's image input")
missing = [n for n in expected_pics if n not in found_pics]
if missing:
raise Fail(
f"{where} is missing <Picture {missing[0]}>. In a prompt_override paste the images "
"are attached ONLY through these tokens — a prompt without them renders with no "
"reference images at all")
roles = outline["_roles"]
expected_audio = [positions[roles["master_audio"]["key"]]] if roles["master_audio"] else []
expected_video = [positions[roles["reference_video"]["key"]]] if roles["reference_video"] else []
found_audio = sorted({int(m.group(1)) for m in _AUDIO_RE.finditer(prompt)})
found_video = sorted({int(m.group(1)) for m in _VIDEO_RE.finditer(prompt)})
if found_audio != expected_audio:
raise Fail(
f"{where} <Audio N> refs {found_audio} != expected {expected_audio}: with a locked "
"master audio every prompt must reference the song exactly once "
"(and audio_track_lock 1); with none, no audio token may appear")
if found_video != expected_video:
raise Fail(
f"{where} <Video N> refs {found_video} != expected {expected_video}: with a locked "
"reference video every prompt must reference it exactly once "
"(and video_track_lock 1); with none, no video token may appear")
def validate_prompt_override(payload: Any, outline: dict[str, Any]) -> dict[str, Any]:
"""Validate a minimax_prompt_override payload against the approved outline."""
if not isinstance(payload, dict):
raise Fail("prompt_override must be a JSON object")
if payload.get("type") != OVERRIDE_TYPE:
raise Fail(
f"prompt_override.type must be {OVERRIDE_TYPE!r}; got {payload.get('type')!r}. "
"Any other JSON pasted into the multitrack editor's prompt_override falls back to "
"the '|'-separated text parser and renders ONE clip over the whole timeline")
if payload.get("version") != OVERRIDE_VERSION:
raise Fail(f"prompt_override.version must be {OVERRIDE_VERSION}")
unknown = sorted(set(payload) - OVERRIDE_FIELDS)
if unknown:
raise Fail(f"prompt_override has unsupported fields {unknown}")
prompts = payload.get("prompts")
rows = outline["rows"]
if not isinstance(prompts, list) or not prompts:
raise Fail("prompt_override.prompts must be a non-empty list")
if len(prompts) != len(rows):
raise Fail(
f"prompt_override.prompts has {len(prompts)} entries but the approved outline has "
f"{len(rows)} segments — one prompt per segment, in order")
durations = _override_durations(payload.get("duration"), len(prompts))
gen_types = _override_tail_values(payload.get("generation_type"), len(prompts),
default="r2v", allowed=OVERRIDE_GEN_TYPES,
field="generation_type")
cont_modes = _override_tail_values(payload.get("continuity_mode"), len(prompts),
default="shot",
allowed=CONTINUITY_MODES | {"context_swap"},
field="continuity_mode")
cont_modes = ["context_drift" if m == "context_swap" else m for m in cont_modes]
for i, (row, prompt, dur, gen, cont) in enumerate(
zip(rows, prompts, durations, gen_types, cont_modes)):
where = f"prompts[{i}]"
if not isinstance(prompt, str) or not prompt.strip():
raise Fail(f"{where} must be a non-empty string")
if "|" in prompt:
raise Fail(f"{where} contains '|'; the fallback parser splits prompts on it — keep prompts pipe-free")
if _ATREF_RE.search(prompt):
raise Fail(f"{where} uses @media references; this gate's canonical form is "
"<Picture N> / <Audio N> / <Video N>")
expected_gen = _TASK_MODE_TO_GEN[row["task_mode"]]
if gen != expected_gen:
raise Fail(f"{where} generation_type {gen!r} != {expected_gen!r} for task_mode {row['task_mode']!r}")
if cont != row["continuity_mode"]:
raise Fail(f"{where} continuity_mode {cont!r} != approved {row['continuity_mode']!r}")
if abs(dur - float(row["duration_seconds"])) > 1e-6:
raise Fail(f"{where} duration {dur:g}s != approved {float(row['duration_seconds']):g}s")
prompt_rules(prompt, row["continuity_mode"], row["followed_by_join"], where)
_check_override_refs(prompt, row, outline, where)
system_prompt = payload.get("system_prompt", "")
if not isinstance(system_prompt, str):
raise Fail("prompt_override.system_prompt must be a string")
roles = outline["_roles"]
expected_audio_lock = 1 if roles["master_audio"] else 0
expected_video_lock = 1 if roles["reference_video"] else 0
for field, expected in (("audio_track_lock", expected_audio_lock),
("video_track_lock", expected_video_lock)):
value = payload.get(field, 0)
if not is_int(value) or not 0 <= int(value) <= 3:
raise Fail(f"prompt_override.{field} must be an integer 0-3")
if int(value) != expected:
label = "master audio" if field == "audio_track_lock" else "reference video"
raise Fail(
f"prompt_override.{field} must be {expected}: the outline "
+ ("locks a " + label + " track (reference it in every prompt and lock slot 1)"
if expected else "has no " + label + ", so there is nothing to lock"))
return payload
# ------------------------------------------------------------------- suite # ------------------------------------------------------------------- suite
def _suite_outline() -> dict[str, Any]: def _suite_outline() -> dict[str, Any]:
@ -1194,6 +1431,42 @@ def run_embedded_suite() -> int:
bad["editor"]["track_data"]["total_length"] = 12 bad["editor"]["track_data"]["total_length"] = 12
check("plan total_length recalc", True, validate_plan, bad, outline=outline) check("plan total_length recalc", True, validate_plan, bad, outline=outline)
override = build_prompt_override(outline, [frag1, frag2, frag3])
check("override build+validate", True, validate_prompt_override,
copy.deepcopy(override), outline)
bad = copy.deepcopy(override)
bad["type"] = "plan"
check("override wrong type caught", False, validate_prompt_override, bad, outline)
bad = copy.deepcopy(override)
bad["duration"] = "30"
check("override duration over 15s caught", False, validate_prompt_override, bad, outline)
bad = copy.deepcopy(override)
bad["duration"] = "5"
check("override duration off outline caught", False, validate_prompt_override, bad, outline)
bad = copy.deepcopy(override)
bad["continuity_mode"] = "shot"
check("override mode off outline caught", False, validate_prompt_override, bad, outline)
bad = copy.deepcopy(override)
bad["prompts"][1] = bad["prompts"][1].replace("<Picture 1>", "", 1)
check("override missing picture ref caught", False, validate_prompt_override, bad, outline)
bad = copy.deepcopy(override)
bad["prompts"][0] = bad["prompts"][0] + " She watches <Picture 2> drift closer."
check("override stray picture ref caught", False, validate_prompt_override, bad, outline)
bad = copy.deepcopy(override)
bad["prompts"][0] = bad["prompts"][0].replace("<Audio 1>", "@audio1", 1)
check("override @ref caught", False, validate_prompt_override, bad, outline)
bad = copy.deepcopy(override)
bad["audio_track_lock"] = 0
check("override audio lock zeroed caught", False, validate_prompt_override, bad, outline)
failed = [c for c in cases if not c[1]] failed = [c for c in cases if not c[1]]
for name, ok, detail in cases: for name, ok, detail in cases:
if ok: if ok:
@ -1296,7 +1569,28 @@ def _main(argv: list[str]) -> int:
f"frames = {data['total_length'] / data['frame_rate']:.2f}s, " f"frames = {data['total_length'] / data['frame_rate']:.2f}s, "
f"format {plan['editor'].get('format')!r}") f"format {plan['editor'].get('format')!r}")
return 0 return 0
raise Fail(f"unknown sub-command {cmd!r} (use outline/segment/trackdata/plan/suite)") if cmd == "override":
args = argv[1:]
outline_path = None
positional: list[str] = []
i = 0
while i < len(args):
if args[i] == "--outline":
outline_path = args[i + 1]; i += 2
else:
positional.append(args[i]); i += 1
if not positional or not outline_path:
raise Fail("usage: override PROMPT_OVERRIDE.json --outline outline.json")
payload = load_json(positional[0], "prompt_override payload")
outline = load_json(outline_path, "outline")
validate_outline(outline)
validate_prompt_override(payload, outline)
total_s = sum(float(r["duration_seconds"]) for r in outline["rows"])
print(f"OVERRIDE OK: {len(payload['prompts'])} prompts, duration {payload['duration']}, "
f"total {total_s:.2f}s at {outline['defaults']['frame_rate']} fps — paste into "
f"the multitrack editor's prompt_override input (format {outline['editor']['format']!r})")
return 0
raise Fail(f"unknown sub-command {cmd!r} (use outline/segment/trackdata/plan/override/suite)")
except Fail as exc: except Fail as exc:
print(f"FAIL: {exc}") print(f"FAIL: {exc}")
return 1 return 1