easy-media-track-gate: emit the minimax_prompt_override paste payload
The multitrack editor node consumes its prompt_override input as a minimax_prompt_override JSON payload (type/version/prompts/duration/ generation_type/continuity_mode/system_prompt/track locks). Any other JSON pasted there falls back to the '|'-separated text parser and renders ONE clip over the whole timeline - the reported single-clip failure - so the paste payload is now the primary deliverable. - verify_track_rules.py: build_prompt_override() assembles the payload from validated fragments (one prompt per row, <Picture N>/<Audio N>/ <Video N> tokens by inventory position, duration 2-15s, per-segment generation/continuity, tail reuse like the runtime); validate_prompt_ override() checks type, field set, prompt count, duration/mode agreement with the outline, exact reference-token sets, pipe-free prompts, and track locks; new 'override' sub-command; suite 25->34. - build_track_plan.py: merge now writes prompt_override.json (one line, ASCII) next to plan.json and validates it before reporting. - references/track-schema.md: full prompt_override section with the single-clip fallback warning and slot-position convention. - SKILL.md: delivery restructured (5A paste default, 5B patch script alternative), smoke test extended, read-cap sizes refreshed. - examples/prompt_override.json: shipped example payload (runtime-checked against the cloned node pack parser: 3 task segments, 720 frames).
This commit is contained in:
parent
56b9d3b9e2
commit
8434bb4645
@ -1,25 +1,32 @@
|
||||
---
|
||||
name: easy-media-track-gate
|
||||
description: Turn a video idea into a validated MultiTrack patch plan for ComfyUI-Easy-Media
|
||||
(the `easy multiTrackEditor` / `easy multitrackProject` nodes). Use when the user wants to
|
||||
plan task segments/shots on the multitrack timeline, write or extend a MultiTrack timeline,
|
||||
arrange prompts, reference images, master audio or reference-video locks, regenerate one
|
||||
task segment, or produce the patch plan JSON that `patch_workflow.py apply` consumes. The
|
||||
scripts and references live next to this SKILL.md (the skill's install folder); run them
|
||||
from the user's working folder, where all generated files are written.
|
||||
description: Turn a video idea into a validated MultiTrack plan for ComfyUI-Easy-Media
|
||||
(the `easy multiTrackEditor` node). Use when the user wants to plan task segments/shots
|
||||
on the multitrack timeline, write or extend a MultiTrack timeline, arrange prompts,
|
||||
reference images, master audio or reference-video locks, regenerate one task segment,
|
||||
or produce the `prompt_override` paste payload for the multitrack editor node (plus the
|
||||
optional patch plan JSON that `patch_workflow.py apply` consumes). The scripts and
|
||||
references live next to this SKILL.md (the skill's install folder); run them from the
|
||||
user's working folder, where all generated files are written.
|
||||
---
|
||||
|
||||
# Easy-Media Track Gate — agent procedure
|
||||
|
||||
Goal: produce `plan.json`, the patch plan applied to a ComfyUI workflow with the Easy-Media
|
||||
node pack's own `patch_workflow.py`, in five moves: the user gives a video idea and a
|
||||
segment count (or an existing workflow to edit) → you build a timeline guide (`outline.json`)
|
||||
and show it for **approval** → with the approved guide you compose the task segments one at
|
||||
a time (about 10 seconds each), each turn existing so that ONE segment gets a complete
|
||||
prompt, the right references and the right continuity mode → the scripts verify schema,
|
||||
timing and continuity across the whole chain → `build_track_plan.py` assembles the TRACK_DATA
|
||||
and the patch plan at the end. Never deliver a plan that did not pass validation, and never
|
||||
compose a segment from a guide the user has not approved.
|
||||
Goal: produce a validated plan for the `easy multiTrackEditor` node in five moves: the user
|
||||
gives a video idea and a segment count (or an existing workflow to edit) → you build a
|
||||
timeline guide (`outline.json`) and show it for **approval** → with the approved guide you
|
||||
compose the task segments one at a time (about 10 seconds each), each turn existing so that
|
||||
ONE segment gets a complete prompt, the right references and the right continuity mode →
|
||||
the scripts verify schema, timing and continuity across the whole chain →
|
||||
`build_track_plan.py` assembles the deliverables at the end. The PRIMARY deliverable is
|
||||
`prompt_override.json`, a `minimax_prompt_override` payload the user pastes into the
|
||||
multitrack editor node's `prompt_override` input (format `MiniMax`) — the editor then
|
||||
renders one clip per prompt, chained back-to-back. The patch plan (`plan.json` for the node
|
||||
pack's `patch_workflow.py`) is the optional whole-workflow alternative. Never deliver a plan
|
||||
that did not pass validation, and never compose a segment from a guide the user has not
|
||||
approved. Never hand the user any other JSON as the paste: a non-`minimax_prompt_override`
|
||||
value in `prompt_override` falls back to the editor's `|`-separated text parser and renders
|
||||
the whole story as ONE clip over the entire timeline.
|
||||
|
||||
## Paths and Python (read first — these exact forms prevent most errors)
|
||||
|
||||
@ -38,14 +45,15 @@ compose a segment from a guide the user has not approved.
|
||||
SKILL_DIR is missing files, stop and tell the user the skill folder is incomplete.
|
||||
- **Never read `verify_track_rules.py`** — run it and work from its printed output. Cline's
|
||||
read_file is capped by hard-coded budgets that do NOT grow with the model's context
|
||||
window: about 48,000 characters per read, 2,000 lines per read. The 66 KB validator
|
||||
window: about 48,000 characters per read, 2,000 lines per read. The 81 KB validator
|
||||
arrives cut no matter how large the context window is. A truncation notice is Cline's
|
||||
read cap, not repo corruption — do not "fix" truncated files. `build_track_plan.py`
|
||||
(~9 KB) and `references/track-schema.md` (~22 KB, 465 lines) are safe to read whole.
|
||||
- **Never read back `plan.json` or `track_data.json` once real segments exist** — a
|
||||
10-segment plan passes the read caps today and exceeds them tomorrow. Validate with
|
||||
`--check` and trust the one-line OK summary; the shipped `examples/plan.json` (6.8 KB)
|
||||
is safe to read if you need to see the shape.
|
||||
(~10 KB) and `references/track-schema.md` (~26 KB) are safe to read whole.
|
||||
- **Never read back `plan.json`, `track_data.json` or `prompt_override.json` once real
|
||||
segments exist** — a 10-segment plan passes the read caps today and exceeds them
|
||||
tomorrow. Validate with `--check` / the `override` command and trust the one-line OK
|
||||
summary; the shipped `examples/prompt_override.json` (2.5 KB) and `examples/plan.json`
|
||||
(6.8 KB) are safe to read if you need to see the shape.
|
||||
- Cline's editor rejects writes over 6,000 characters ("Editor input too large"). A segment
|
||||
fragment with a rich prompt often does. NEVER write part 2 with write_to_file — it
|
||||
replaces the whole file and part 1 is lost. Split instead: write the first part ending
|
||||
@ -61,10 +69,12 @@ compose a segment from a guide the user has not approved.
|
||||
```sh
|
||||
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py"
|
||||
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\plan.json"
|
||||
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\prompt_override.json" --outline "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\outline.json"
|
||||
```
|
||||
|
||||
Expect the suite to print `SUITE PASS (25/25 cases)` and the second command to print
|
||||
`PLAN OK` with `total_length 720 frames = 30.00s`.
|
||||
Expect the suite to print `SUITE PASS (34/34 cases)` and the other two commands to print
|
||||
`PLAN OK` with `total_length 720 frames = 30.00s` and `OVERRIDE OK: 3 prompts, duration 10,
|
||||
total 30.00s`.
|
||||
|
||||
## Step 0 — Media paths come first (never invent one)
|
||||
|
||||
@ -128,7 +138,10 @@ Field rules the validator enforces:
|
||||
- `media_inventory` keys are UPPERCASE; every `file_path` is copied from Step 0 sources —
|
||||
this inventory is the **machine-checked verbatim source**: every segment's image items and
|
||||
every locked media track are diffed against it, so a retyped path fails on the spot. A
|
||||
paraphrase in the inventory cannot be repaired later.
|
||||
paraphrase in the inventory cannot be repaired later. Order matters too: the inventory's
|
||||
image rows map to the editor's `image` input positions (`<Picture 1>`, `<Picture 2>`, …),
|
||||
audio rows to `audio` input positions, video rows to `video` input positions — the user
|
||||
wires the media into those node inputs in inventory order.
|
||||
- `tracks.master_audio` / `tracks.reference_video` name the ONE locked audio and ONE locked
|
||||
video track (the schema allows at most one of each). Lock the master song for MV/lyric/
|
||||
dance/beat projects, the narration track when narration drives the cut, and the reference
|
||||
@ -264,23 +277,56 @@ Once every segment file passes, the merge is the continuity check: `build_track_
|
||||
re-walks every fragment against the guide (ids, frames, modes, verbatim media paths, prompt
|
||||
rules, handoff chain), assembles the TRACK_DATA (task track from the fragments; the locked
|
||||
master-audio / reference-video tracks built straight from the outline), computes
|
||||
`total_length`, and only then writes the deliverable. Never write the plan by hand and
|
||||
never edit fragment prose during the merge.
|
||||
`total_length`, builds the `minimax_prompt_override` paste payload from the same validated
|
||||
fragments, and only then writes the deliverables. Never write the plan by hand and never
|
||||
edit fragment prose during the merge.
|
||||
|
||||
```sh
|
||||
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" outline.json segment_01.json segment_02.json segment_03.json -o plan.json
|
||||
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check plan.json --outline outline.json
|
||||
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override prompt_override.json --outline outline.json
|
||||
```
|
||||
|
||||
The merge writes `plan.json` (the patch plan) and `track_data.json` (pretty, for
|
||||
inspection) and prints the delivered frames/seconds, the track list and the locked tracks.
|
||||
Segment, track and image ids are stable UUIDv5 values derived from the project title, so a
|
||||
re-merge keeps ids consistent.
|
||||
The merge writes `prompt_override.json` (the paste payload, one line), `plan.json` (the
|
||||
patch plan) and `track_data.json` (pretty, for inspection), and prints the delivered
|
||||
durations/seconds, the track list and the locked tracks. Segment, track and image ids are
|
||||
stable UUIDv5 values derived from the project title, so a re-merge keeps ids consistent.
|
||||
The `override` command re-checks the paste payload against the guide: one prompt per
|
||||
segment, durations inside the runtime's 2–15s window and equal to the approved rows,
|
||||
per-segment generation/continuity modes, the exact `<Picture N>` / `<Audio N>` / `<Video N>`
|
||||
reference set per prompt (a missing token renders with no reference media; a stray one
|
||||
attaches the wrong image), and the track locks.
|
||||
|
||||
## Step 5 — Deliver (the node pack's own script applies it)
|
||||
## Step 5 — Deliver (paste payload first, patch script as the alternative)
|
||||
|
||||
The deliverable is applied to the user's workflow by the Easy-Media node pack's script —
|
||||
find it under the node pack folder (typically
|
||||
### 5A — Default: the `prompt_override` paste
|
||||
|
||||
`prompt_override.json` is what the user pastes into the **multitrack editor node's
|
||||
`prompt_override` input** (`easy multiTrackEditor`). It is self-contained: one prompt per
|
||||
clip, per-clip durations and continuity modes, media reference tokens inside the prompts.
|
||||
Deliver it with these instructions:
|
||||
|
||||
1. In the editor node set `format` to `MiniMax` (the payload is parsed as MiniMax only in
|
||||
that format).
|
||||
2. Wire the media: the inventory's image rows (in inventory order) into the editor's
|
||||
`image` input, the master song into `audio`, the reference video into `video`. The
|
||||
`<Picture N>` / `<Audio N>` / `<Video N>` tokens resolve against those input lists
|
||||
positionally.
|
||||
3. Select the `prompt_override` widget, clear it, paste the whole one-line JSON, queue.
|
||||
The editor rebuilds the task track: one clip per prompt, back-to-back, durations as
|
||||
specified — 3 prompts at 10s render as 3 chained ~10-second clips.
|
||||
|
||||
**The paste target is the `prompt_override` input and only that payload type.** Never give
|
||||
the user a `plan.json`, an outline, an H3 plan or any other JSON to paste there: without
|
||||
`"type": "minimax_prompt_override"` the editor falls back to its `|`-separated text parser
|
||||
and renders the whole thing as ONE clip spanning the timeline (the single-clip failure).
|
||||
Never add `|` inside prompts for the same reason.
|
||||
|
||||
### 5B — Alternative: the node pack's patch script (whole-workflow patch)
|
||||
|
||||
When the user wants the timeline written into a workflow file directly (or needs `edit`
|
||||
task modes / extra tracks the paste payload cannot express), the deliverable is applied by
|
||||
the Easy-Media node pack's script — find it under the node pack folder (typically
|
||||
`<ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py`;
|
||||
ask the user for the path and the workflow JSON if they are not at hand):
|
||||
|
||||
@ -306,10 +352,11 @@ the node pack's own flow (see its `references/execution-api.md`). "Queued" never
|
||||
|
||||
- Guide-level change (timings, beats, modes, segment count, media): update `outline.json`
|
||||
first, show the changed rows, then rewrite only the affected segments against it and
|
||||
re-run the Step 4 merge so `plan.json` refreshes.
|
||||
re-run the Step 4 merge so `plan.json` and `prompt_override.json` refresh.
|
||||
- One segment reads badly: rewrite that `segment_NN.json` (keep its id and frames from the
|
||||
guide), re-validate it with the `segment` command, then re-merge. Never patch `plan.json`
|
||||
directly — the merge rebuilds it from the fragments every time.
|
||||
or `prompt_override.json` directly — the merge rebuilds both from the fragments every
|
||||
time.
|
||||
|
||||
## If you cannot run Python
|
||||
|
||||
|
||||
@ -1,19 +1,22 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Merge approved outline + task-segment fragments into the Easy-Media MultiTrack patch plan.
|
||||
"""Merge approved outline + task-segment fragments into the Easy-Media MultiTrack deliverables.
|
||||
|
||||
py -3 build_track_plan.py outline.json segment_01.json segment_02.json -o plan.json
|
||||
py -3 build_track_plan.py --check plan.json [--outline outline.json]
|
||||
|
||||
The merge is the continuity check: it re-validates every fragment against the approved
|
||||
outline (ids, frames, modes, verbatim media paths, context-prompt rules, handoff chain),
|
||||
then assembles the TRACK_DATA (task track + locked master-audio / reference-video tracks
|
||||
built straight from the outline, never retyped) and the patch plan the node pack's
|
||||
`patch_workflow.py apply` consumes:
|
||||
then assembles TWO deliverables from it:
|
||||
|
||||
{"editor": {"resolution": ..., "track_data": {...}, "format": ...},
|
||||
"project": {...}}
|
||||
1. `prompt_override.json` — the PRIMARY paste payload for the `easy multiTrackEditor`
|
||||
node's `prompt_override` input (format MiniMax): {"type": "minimax_prompt_override",
|
||||
...} with one prompt per segment, per-segment durations and continuity modes. Paste it
|
||||
into the widget and queue; wire the inventory's media into the editor's image/audio/
|
||||
video inputs in inventory order so the <Picture N>/<Audio N>/<Video N> tokens resolve.
|
||||
2. `plan.json` — the patch plan the node pack's `patch_workflow.py apply` consumes
|
||||
({"editor": {..., "track_data": {...}}, "project": {...}}) for patching a whole
|
||||
workflow file, plus `track_data.json` (pretty, for inspection).
|
||||
|
||||
It writes `plan.json` (the deliverable) and `track_data.json` (pretty, for inspection).
|
||||
Segment/track/image ids are deterministic UUIDv5 values derived from the project title, so
|
||||
re-running the merge keeps ids stable. Never edit fragment prose here and never write the
|
||||
plan by hand — fix the fragment, re-validate, re-merge.
|
||||
@ -208,9 +211,22 @@ def _main(argv: list[str]) -> int:
|
||||
out_path = Path(args.output)
|
||||
_write_json(out_path, plan)
|
||||
_write_json(out_path.parent / "track_data.json", plan["editor"]["track_data"])
|
||||
override = vtr.build_prompt_override(outline, fragments)
|
||||
vtr.validate_prompt_override(copy.deepcopy(override), outline)
|
||||
override_path = out_path.parent / "prompt_override.json"
|
||||
override_path.write_text(
|
||||
json.dumps(override, ensure_ascii=True, separators=(",", ":")) + "\n",
|
||||
encoding="utf-8")
|
||||
prompts = len(override["prompts"])
|
||||
total_s = sum(float(row["duration_seconds"]) for row in outline["rows"])
|
||||
print(f"OVERRIDE OK: {prompts} prompts, duration {override['duration']}, "
|
||||
f"total {total_s:.2f}s; wrote {override_path} "
|
||||
f"({override_path.stat().st_size} chars, one line)")
|
||||
print(f"PLAN OK: {_summary(plan)}")
|
||||
print(f"wrote {out_path} and {out_path.parent / 'track_data.json'}")
|
||||
print("next: dry-run the node pack's patch script before writing:")
|
||||
print("deliver: paste prompt_override.json into the multitrack editor's "
|
||||
"prompt_override input (format MiniMax), then queue")
|
||||
print("optional whole-workflow patch (dry-run first):")
|
||||
print(' py -3 <ComfyUI-Easy-Media>\\skills\\easy-media-multitrack-workflow\\scripts\\patch_workflow.py apply WORKFLOW.json --plan plan.json')
|
||||
print(' ... then add --output RESULT.json --write and inspect RESULT.json')
|
||||
return 0
|
||||
|
||||
1
easy-media-track-gate/examples/prompt_override.json
Normal file
1
easy-media-track-gate/examples/prompt_override.json
Normal file
@ -0,0 +1 @@
|
||||
{"type":"minimax_prompt_override","version":1,"prompts":["<Picture 1> <Audio 1> Dawn light spills across an empty rooftop terrace above a hushed city. A young woman with a brown ponytail, cream cardigan over a white tee and blue jeans stands at the steel rail, matching the pose reference image for her identity only. She cradles a matte-black kettle in her right hand and tips it in one slow arc; a thread of steam rises as coffee fills the white cup in her left hand, her wrist rotating smoothly, elbow dropping as the kettle empties. She sets the kettle on the rail with a soft clink, lifts the cup, and brings it halfway to her lips as the sun's first rim crests the skyline and warms her cheek. The camera holds one slow push-in toward her profile, never cutting, while pigeon wings clap once somewhere below and the kettle's metal ticks as it cools.","<Picture 1> <Audio 1> The camera holds the same framing it arrived with, and she keeps the pose, motion and light it arrived with: the cup still traveling toward her lips, steam bending in the same breeze. Her chest rises with one slow breath, weight easing onto her left hip as the cardigan's hem starts to lift. Around the two-second mark the wind strengthens into a gust that ruffles her ponytail and snaps the cardigan open a finger's width; her fingers reclose it while she turns her face slowly into the light, eyelids narrowing. Steam tears off the cup in ribbons. The camera keeps its single slow drift leftward with no cut, the city below sharpening frame by frame, traffic swelling from hum to roar, a rooftop antenna creaking softly, her cardigan snapping in steady wind as the turn continues.","<Picture 1> <Audio 1> The join keeps everything it is handed: same framing, same breeze, the tail of her turn still in motion as her face comes level with the camera. She continues the turn until her eyes find the skyline, then lowers the cup in one unhurried arc to the rail, her shoulder rolling with the descent. Below, windows begin to light up floor by floor, a slow tide of warm rectangles climbing the tower across the street while a bus door hisses open. She exhales a long breath, steam curling off the cup in the same wind, ponytail still lifting, cardigan snapping softly. The camera keeps its slow lateral drift and never cuts; the city's morning roar builds under the song as her chest keeps rising and falling mid-breath."],"duration":"10","generation_type":"r2v","continuity_mode":"shot,context,context","system_prompt":"","video_track_lock":0,"audio_track_lock":1}
|
||||
@ -463,3 +463,56 @@ python3 scripts/patch_workflow.py apply SOURCE.json --plan PLAN.json
|
||||
python3 scripts/patch_workflow.py apply SOURCE.json --plan PLAN.json --output RESULT.json --write
|
||||
python3 scripts/patch_workflow.py inspect RESULT.json
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## prompt_override — the paste payload (format MiniMax)
|
||||
|
||||
The `easy multiTrackEditor` node has an optional `prompt_override` input. When the editor's
|
||||
`format` is `MiniMax` and the pasted value is a `minimax_prompt_override` JSON payload, the
|
||||
editor rebuilds its whole task track from that payload: one task segment per `prompts`
|
||||
entry, laid back-to-back, each with its own duration. `total_length` becomes the sum of the
|
||||
durations, so a 3-prompt payload at 10s each renders as THREE chained ~10-second clips.
|
||||
|
||||
```json
|
||||
{"type":"minimax_prompt_override","version":1,
|
||||
"prompts":["<Picture 1> <Audio 1> prompt for segment one ...",
|
||||
"<Picture 1> <Audio 1> prompt for segment two ..."],
|
||||
"duration":"10","generation_type":"r2v","continuity_mode":"shot,context,context",
|
||||
"system_prompt":"","video_track_lock":0,"audio_track_lock":1}
|
||||
```
|
||||
|
||||
Field rules (mirrors `utils/prompt_override.py` in the node pack):
|
||||
|
||||
- `type` must be exactly `minimax_prompt_override` and `version` `1`. **Any other JSON
|
||||
pasted here is not rejected — it falls back to the generic text parser that splits on
|
||||
`|`.** A JSON blob contains no `|`, so it becomes ONE task segment spanning the whole
|
||||
timeline with the JSON dump as its prompt: the classic "rendered the whole story as a
|
||||
single 10 second clip" failure.
|
||||
- `prompts` — one entry per clip, in timeline order. The prompt text is the clip's full
|
||||
generation prompt (same standalone composition rules as task segments: no cross-segment
|
||||
references, context joins keep-then-change, only the last clip may come to rest).
|
||||
- Media attaches ONLY through tokens in the prompt text: `<Picture N>`, `<Audio N>`,
|
||||
`<Video N>` (the `@imageN`/`@audioN`/`@videoN` forms are accepted and rewritten, but this
|
||||
gate standardizes on the bracket form). N is the 1-based position of the item in the
|
||||
editor's matching `image` / `audio` / `video` input list — wire the media inventory into
|
||||
those inputs in inventory order. A prompt with no token renders with no reference media.
|
||||
The runtime renumbers tokens compactly for display, but slots are wired from the raw N.
|
||||
- `duration` — seconds per clip, single value ("10" applies to every clip) or comma list
|
||||
("10,5,10") with tail reuse; each value must be 2–15, enforced at parse time.
|
||||
- `generation_type` — per clip, same single/comma-list rule. `r2v` = reference-image task
|
||||
(task_mode ref), `l2v` = last-frame task, `t2v`/`i2v`/`v2v` land as default/edit tasks.
|
||||
This gate emits `r2v` for ref rows, `l2v` for l2v rows, `t2v` for default rows.
|
||||
- `continuity_mode` — per clip: `shot`, `context`, `context_drift` (legacy `context_swap`
|
||||
is normalized to `context_drift`). Matches the task-segment semantics.
|
||||
- `system_prompt` — written into every clip when non-empty; an empty string lets the H3
|
||||
runtime auto-append its reference rules for context joins (the gate's default).
|
||||
- `video_track_lock` / `audio_track_lock` — 1-based index of the video/audio track to lock
|
||||
(0 = none). With a locked master-audio inventory row, every prompt carries
|
||||
`<Audio 1>` and `audio_track_lock` is 1; with a locked reference video, `<Video 1>` and
|
||||
`video_track_lock` are 1.
|
||||
|
||||
The paste replaces the editor's existing task track (and derives its audio/video tracks
|
||||
from the tokens), so the payload is self-contained: prompts, durations, modes and media
|
||||
references all travel inside it. Edit task modes (`edit`) and full extra tracks are beyond
|
||||
this payload — use the TRACK_DATA patch plan for those.
|
||||
|
||||
@ -24,6 +24,9 @@ Sub-commands:
|
||||
segment SEGMENT_NN.json --index N --total T --outline OUTLINE.json [--prev PREV.json]
|
||||
trackdata TRACK_DATA.json [--outline OUTLINE.json]
|
||||
plan PLAN.json [--outline OUTLINE.json]
|
||||
override PROMPT_OVERRIDE.json --outline OUTLINE.json
|
||||
validate the minimax_prompt_override paste
|
||||
payload for the multitrack editor node
|
||||
suite embedded self-test suite
|
||||
|
||||
Exit code 0 = pass, 1 = fail. Standard library only.
|
||||
@ -68,6 +71,23 @@ PROJECT_FIELDS = {
|
||||
EDITOR_PLAN_FIELDS = {"node_id", "track_data", "format", "resolution", "recalculate_total_length"}
|
||||
|
||||
SNAKE_RE = re.compile(r"^[a-z][a-z0-9_]*$")
|
||||
|
||||
# The multitrack editor's prompt_override input (format MiniMax) accepts exactly one JSON
|
||||
# payload type. Anything else falls back to the '|'-separated text parser and renders ONE
|
||||
# clip over the whole timeline — the classic single-clip failure.
|
||||
OVERRIDE_TYPE = "minimax_prompt_override"
|
||||
OVERRIDE_VERSION = 1
|
||||
OVERRIDE_GEN_TYPES = {"r2v", "t2v", "i2v", "v2v", "l2v"}
|
||||
OVERRIDE_DURATION_MIN = 2.0
|
||||
OVERRIDE_DURATION_MAX = 15.0
|
||||
OVERRIDE_FIELDS = {"type", "version", "prompts", "duration", "generation_type",
|
||||
"continuity_mode", "system_prompt", "video_track_lock", "audio_track_lock"}
|
||||
_TASK_MODE_TO_GEN = {"ref": "r2v", "l2v": "l2v", "default": "t2v"}
|
||||
_PICTURE_RE = re.compile(r"<Picture\s*(\d+)\s*>", re.IGNORECASE)
|
||||
_AUDIO_RE = re.compile(r"<Audio\s*(\d+)\s*>", re.IGNORECASE)
|
||||
_VIDEO_RE = re.compile(r"<Video\s*(\d+)\s*>", re.IGNORECASE)
|
||||
_ATREF_RE = re.compile(r"@(?:image|img|audio|video|\u56fe\u7247|\u56fe|\u97f3\u9891|\u89c6\u9891)\s*\d+", re.IGNORECASE)
|
||||
|
||||
MAX_IMAGES = 9
|
||||
PROMPT_MIN_WORDS = 60
|
||||
END_STATE_MIN_WORDS = 15
|
||||
@ -923,6 +943,223 @@ def validate_plan(plan: dict[str, Any], *, outline: dict[str, Any] | None = None
|
||||
return track_data
|
||||
|
||||
|
||||
# ------------------------------------------------- minimax prompt_override paste
|
||||
|
||||
def _override_media_positions(outline: dict[str, Any]) -> dict[str, int]:
|
||||
"""1-based slot position per inventory key.
|
||||
|
||||
The editor's prompt_override attaches media ONLY through tokens in the prompt text
|
||||
(`<Picture N>` / `<Audio N>` / `<Video N>`). N is the 1-based position of the media
|
||||
item in the editor's image / audio / video input list, so this gate's convention is:
|
||||
wire the inventory's rows into the editor's matching input in inventory order, then
|
||||
N is the row's 1-based position among the rows of its own kind.
|
||||
"""
|
||||
positions: dict[str, int] = {}
|
||||
counters = {"image": 0, "audio": 0, "video": 0}
|
||||
for key, entry in outline["inventory"].items():
|
||||
counters[entry["kind"]] += 1
|
||||
positions[key] = counters[entry["kind"]]
|
||||
return positions
|
||||
|
||||
|
||||
def _override_tail_values(raw: Any, count: int, *, default: str, allowed: set[str],
|
||||
field: str) -> list[str]:
|
||||
"""Mirror the runtime's comma-list parsing: per-segment values, tail reused."""
|
||||
text = default if raw is None else str(raw).strip()
|
||||
if not text:
|
||||
text = default
|
||||
parts = [p.strip() for p in text.split(",") if p.strip()]
|
||||
if not parts:
|
||||
raise Fail(f"prompt_override.{field} must contain at least one value")
|
||||
values: list[str] = []
|
||||
for part in parts:
|
||||
norm = part.lower()
|
||||
if norm not in allowed:
|
||||
raise Fail(f"prompt_override.{field} value {part!r} is not one of {sorted(allowed)}")
|
||||
values.append(norm)
|
||||
return [values[min(i, len(values) - 1)] for i in range(count)]
|
||||
|
||||
|
||||
def _override_durations(raw: Any, count: int) -> list[float]:
|
||||
text = "10" if raw is None else str(raw).strip()
|
||||
if not text:
|
||||
text = "10"
|
||||
parts = [p.strip() for p in text.split(",") if p.strip()]
|
||||
if not parts:
|
||||
raise Fail("prompt_override.duration must contain at least one value")
|
||||
out: list[float] = []
|
||||
for part in parts:
|
||||
try:
|
||||
sec = float(part)
|
||||
except ValueError:
|
||||
raise Fail(f"prompt_override.duration has a non-numeric value: {part!r}")
|
||||
if not OVERRIDE_DURATION_MIN <= sec <= OVERRIDE_DURATION_MAX:
|
||||
raise Fail(
|
||||
f"prompt_override.duration value {part!r} is outside the runtime's "
|
||||
f"{OVERRIDE_DURATION_MIN:g}-{OVERRIDE_DURATION_MAX:g}s window; the editor "
|
||||
"rejects the paste outright at that range")
|
||||
out.append(sec)
|
||||
return [out[min(i, len(out) - 1)] for i in range(count)]
|
||||
|
||||
|
||||
def build_prompt_override(outline: dict[str, Any], fragments: list[dict[str, Any]]) -> dict[str, Any]:
|
||||
"""Assemble the multitrack editor's paste payload from validated fragments.
|
||||
|
||||
One prompt per outline row (variant b honored), media reference tokens prepended,
|
||||
per-segment duration / generation_type / continuity_mode lists with tail reuse.
|
||||
"""
|
||||
rows = outline["rows"]
|
||||
if len(fragments) != len(rows):
|
||||
raise Fail(f"expected {len(rows)} fragments (one per approved outline row), got {len(fragments)}")
|
||||
positions = _override_media_positions(outline)
|
||||
roles = outline["_roles"]
|
||||
audio_pos = positions.get(roles["master_audio"]["key"]) if roles["master_audio"] else None
|
||||
video_pos = positions.get(roles["reference_video"]["key"]) if roles["reference_video"] else None
|
||||
|
||||
prompts: list[str] = []
|
||||
durations: list[str] = []
|
||||
gen_types: list[str] = []
|
||||
cont_modes: list[str] = []
|
||||
for row, frag in zip(rows, fragments):
|
||||
content = frag.get("content", {})
|
||||
variant = content.get("user_prompt_variant", "a")
|
||||
text = content.get("user_prompt_b") if variant == "b" else content.get("user_prompt")
|
||||
if not isinstance(text, str) or not text.strip():
|
||||
raise Fail(f"fragment {frag.get('id', '?')}: no prompt text for the override payload")
|
||||
refs = ""
|
||||
for img in row.get("images", []):
|
||||
refs += f"<Picture {positions[img['key']]}> "
|
||||
if audio_pos is not None:
|
||||
refs += f"<Audio {audio_pos}> "
|
||||
if video_pos is not None:
|
||||
refs += f"<Video {video_pos}> "
|
||||
prompts.append(" ".join((refs + text).split()))
|
||||
durations.append(f"{float(row['duration_seconds']):g}")
|
||||
gen_types.append(_TASK_MODE_TO_GEN[row["task_mode"]])
|
||||
cont_modes.append(row["continuity_mode"])
|
||||
|
||||
def _compact(values: list[str]) -> str:
|
||||
return ",".join(values) if len(set(values)) > 1 else values[0]
|
||||
|
||||
return {
|
||||
"type": OVERRIDE_TYPE,
|
||||
"version": OVERRIDE_VERSION,
|
||||
"prompts": prompts,
|
||||
"duration": _compact(durations),
|
||||
"generation_type": _compact(gen_types),
|
||||
"continuity_mode": _compact(cont_modes),
|
||||
"system_prompt": "",
|
||||
"video_track_lock": 1 if video_pos is not None else 0,
|
||||
"audio_track_lock": 1 if audio_pos is not None else 0,
|
||||
}
|
||||
|
||||
|
||||
def _check_override_refs(prompt: str, row: dict[str, Any], outline: dict[str, Any], where: str) -> None:
|
||||
positions = _override_media_positions(outline)
|
||||
inventory = outline["inventory"]
|
||||
expected_pics = sorted({positions[img["key"]] for img in row.get("images", [])})
|
||||
found_pics = sorted({int(m.group(1)) for m in _PICTURE_RE.finditer(prompt)})
|
||||
unexpected = [n for n in found_pics if n not in expected_pics]
|
||||
if unexpected:
|
||||
raise Fail(
|
||||
f"{where} references <Picture {unexpected[0]}> which is not one of this segment's "
|
||||
f"approved images (expected {expected_pics}); a stray token attaches the wrong "
|
||||
"slot image from the editor's image input")
|
||||
missing = [n for n in expected_pics if n not in found_pics]
|
||||
if missing:
|
||||
raise Fail(
|
||||
f"{where} is missing <Picture {missing[0]}>. In a prompt_override paste the images "
|
||||
"are attached ONLY through these tokens — a prompt without them renders with no "
|
||||
"reference images at all")
|
||||
roles = outline["_roles"]
|
||||
expected_audio = [positions[roles["master_audio"]["key"]]] if roles["master_audio"] else []
|
||||
expected_video = [positions[roles["reference_video"]["key"]]] if roles["reference_video"] else []
|
||||
found_audio = sorted({int(m.group(1)) for m in _AUDIO_RE.finditer(prompt)})
|
||||
found_video = sorted({int(m.group(1)) for m in _VIDEO_RE.finditer(prompt)})
|
||||
if found_audio != expected_audio:
|
||||
raise Fail(
|
||||
f"{where} <Audio N> refs {found_audio} != expected {expected_audio}: with a locked "
|
||||
"master audio every prompt must reference the song exactly once "
|
||||
"(and audio_track_lock 1); with none, no audio token may appear")
|
||||
if found_video != expected_video:
|
||||
raise Fail(
|
||||
f"{where} <Video N> refs {found_video} != expected {expected_video}: with a locked "
|
||||
"reference video every prompt must reference it exactly once "
|
||||
"(and video_track_lock 1); with none, no video token may appear")
|
||||
|
||||
|
||||
def validate_prompt_override(payload: Any, outline: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Validate a minimax_prompt_override payload against the approved outline."""
|
||||
if not isinstance(payload, dict):
|
||||
raise Fail("prompt_override must be a JSON object")
|
||||
if payload.get("type") != OVERRIDE_TYPE:
|
||||
raise Fail(
|
||||
f"prompt_override.type must be {OVERRIDE_TYPE!r}; got {payload.get('type')!r}. "
|
||||
"Any other JSON pasted into the multitrack editor's prompt_override falls back to "
|
||||
"the '|'-separated text parser and renders ONE clip over the whole timeline")
|
||||
if payload.get("version") != OVERRIDE_VERSION:
|
||||
raise Fail(f"prompt_override.version must be {OVERRIDE_VERSION}")
|
||||
unknown = sorted(set(payload) - OVERRIDE_FIELDS)
|
||||
if unknown:
|
||||
raise Fail(f"prompt_override has unsupported fields {unknown}")
|
||||
prompts = payload.get("prompts")
|
||||
rows = outline["rows"]
|
||||
if not isinstance(prompts, list) or not prompts:
|
||||
raise Fail("prompt_override.prompts must be a non-empty list")
|
||||
if len(prompts) != len(rows):
|
||||
raise Fail(
|
||||
f"prompt_override.prompts has {len(prompts)} entries but the approved outline has "
|
||||
f"{len(rows)} segments — one prompt per segment, in order")
|
||||
durations = _override_durations(payload.get("duration"), len(prompts))
|
||||
gen_types = _override_tail_values(payload.get("generation_type"), len(prompts),
|
||||
default="r2v", allowed=OVERRIDE_GEN_TYPES,
|
||||
field="generation_type")
|
||||
cont_modes = _override_tail_values(payload.get("continuity_mode"), len(prompts),
|
||||
default="shot",
|
||||
allowed=CONTINUITY_MODES | {"context_swap"},
|
||||
field="continuity_mode")
|
||||
cont_modes = ["context_drift" if m == "context_swap" else m for m in cont_modes]
|
||||
|
||||
for i, (row, prompt, dur, gen, cont) in enumerate(
|
||||
zip(rows, prompts, durations, gen_types, cont_modes)):
|
||||
where = f"prompts[{i}]"
|
||||
if not isinstance(prompt, str) or not prompt.strip():
|
||||
raise Fail(f"{where} must be a non-empty string")
|
||||
if "|" in prompt:
|
||||
raise Fail(f"{where} contains '|'; the fallback parser splits prompts on it — keep prompts pipe-free")
|
||||
if _ATREF_RE.search(prompt):
|
||||
raise Fail(f"{where} uses @media references; this gate's canonical form is "
|
||||
"<Picture N> / <Audio N> / <Video N>")
|
||||
expected_gen = _TASK_MODE_TO_GEN[row["task_mode"]]
|
||||
if gen != expected_gen:
|
||||
raise Fail(f"{where} generation_type {gen!r} != {expected_gen!r} for task_mode {row['task_mode']!r}")
|
||||
if cont != row["continuity_mode"]:
|
||||
raise Fail(f"{where} continuity_mode {cont!r} != approved {row['continuity_mode']!r}")
|
||||
if abs(dur - float(row["duration_seconds"])) > 1e-6:
|
||||
raise Fail(f"{where} duration {dur:g}s != approved {float(row['duration_seconds']):g}s")
|
||||
prompt_rules(prompt, row["continuity_mode"], row["followed_by_join"], where)
|
||||
_check_override_refs(prompt, row, outline, where)
|
||||
|
||||
system_prompt = payload.get("system_prompt", "")
|
||||
if not isinstance(system_prompt, str):
|
||||
raise Fail("prompt_override.system_prompt must be a string")
|
||||
roles = outline["_roles"]
|
||||
expected_audio_lock = 1 if roles["master_audio"] else 0
|
||||
expected_video_lock = 1 if roles["reference_video"] else 0
|
||||
for field, expected in (("audio_track_lock", expected_audio_lock),
|
||||
("video_track_lock", expected_video_lock)):
|
||||
value = payload.get(field, 0)
|
||||
if not is_int(value) or not 0 <= int(value) <= 3:
|
||||
raise Fail(f"prompt_override.{field} must be an integer 0-3")
|
||||
if int(value) != expected:
|
||||
label = "master audio" if field == "audio_track_lock" else "reference video"
|
||||
raise Fail(
|
||||
f"prompt_override.{field} must be {expected}: the outline "
|
||||
+ ("locks a " + label + " track (reference it in every prompt and lock slot 1)"
|
||||
if expected else "has no " + label + ", so there is nothing to lock"))
|
||||
return payload
|
||||
|
||||
|
||||
# ------------------------------------------------------------------- suite
|
||||
|
||||
def _suite_outline() -> dict[str, Any]:
|
||||
@ -1194,6 +1431,42 @@ def run_embedded_suite() -> int:
|
||||
bad["editor"]["track_data"]["total_length"] = 12
|
||||
check("plan total_length recalc", True, validate_plan, bad, outline=outline)
|
||||
|
||||
override = build_prompt_override(outline, [frag1, frag2, frag3])
|
||||
check("override build+validate", True, validate_prompt_override,
|
||||
copy.deepcopy(override), outline)
|
||||
|
||||
bad = copy.deepcopy(override)
|
||||
bad["type"] = "plan"
|
||||
check("override wrong type caught", False, validate_prompt_override, bad, outline)
|
||||
|
||||
bad = copy.deepcopy(override)
|
||||
bad["duration"] = "30"
|
||||
check("override duration over 15s caught", False, validate_prompt_override, bad, outline)
|
||||
|
||||
bad = copy.deepcopy(override)
|
||||
bad["duration"] = "5"
|
||||
check("override duration off outline caught", False, validate_prompt_override, bad, outline)
|
||||
|
||||
bad = copy.deepcopy(override)
|
||||
bad["continuity_mode"] = "shot"
|
||||
check("override mode off outline caught", False, validate_prompt_override, bad, outline)
|
||||
|
||||
bad = copy.deepcopy(override)
|
||||
bad["prompts"][1] = bad["prompts"][1].replace("<Picture 1>", "", 1)
|
||||
check("override missing picture ref caught", False, validate_prompt_override, bad, outline)
|
||||
|
||||
bad = copy.deepcopy(override)
|
||||
bad["prompts"][0] = bad["prompts"][0] + " She watches <Picture 2> drift closer."
|
||||
check("override stray picture ref caught", False, validate_prompt_override, bad, outline)
|
||||
|
||||
bad = copy.deepcopy(override)
|
||||
bad["prompts"][0] = bad["prompts"][0].replace("<Audio 1>", "@audio1", 1)
|
||||
check("override @ref caught", False, validate_prompt_override, bad, outline)
|
||||
|
||||
bad = copy.deepcopy(override)
|
||||
bad["audio_track_lock"] = 0
|
||||
check("override audio lock zeroed caught", False, validate_prompt_override, bad, outline)
|
||||
|
||||
failed = [c for c in cases if not c[1]]
|
||||
for name, ok, detail in cases:
|
||||
if ok:
|
||||
@ -1296,7 +1569,28 @@ def _main(argv: list[str]) -> int:
|
||||
f"frames = {data['total_length'] / data['frame_rate']:.2f}s, "
|
||||
f"format {plan['editor'].get('format')!r}")
|
||||
return 0
|
||||
raise Fail(f"unknown sub-command {cmd!r} (use outline/segment/trackdata/plan/suite)")
|
||||
if cmd == "override":
|
||||
args = argv[1:]
|
||||
outline_path = None
|
||||
positional: list[str] = []
|
||||
i = 0
|
||||
while i < len(args):
|
||||
if args[i] == "--outline":
|
||||
outline_path = args[i + 1]; i += 2
|
||||
else:
|
||||
positional.append(args[i]); i += 1
|
||||
if not positional or not outline_path:
|
||||
raise Fail("usage: override PROMPT_OVERRIDE.json --outline outline.json")
|
||||
payload = load_json(positional[0], "prompt_override payload")
|
||||
outline = load_json(outline_path, "outline")
|
||||
validate_outline(outline)
|
||||
validate_prompt_override(payload, outline)
|
||||
total_s = sum(float(r["duration_seconds"]) for r in outline["rows"])
|
||||
print(f"OVERRIDE OK: {len(payload['prompts'])} prompts, duration {payload['duration']}, "
|
||||
f"total {total_s:.2f}s at {outline['defaults']['frame_rate']} fps — paste into "
|
||||
f"the multitrack editor's prompt_override input (format {outline['editor']['format']!r})")
|
||||
return 0
|
||||
raise Fail(f"unknown sub-command {cmd!r} (use outline/segment/trackdata/plan/override/suite)")
|
||||
except Fail as exc:
|
||||
print(f"FAIL: {exc}")
|
||||
return 1
|
||||
|
||||
Loading…
Reference in New Issue
Block a user