Compare commits

..

No commits in common. "7b6a2da4725e97f5f03836298496110bb9265716" and "f6be7ffcc5974abc952d0791670a952c2447a0cd" have entirely different histories.

15 changed files with 1 additions and 3431 deletions

View File

@ -1,81 +0,0 @@
# Easy-Media Track Gate
A portable, self-contained agent skill that turns a video idea into a **validated MultiTrack
patch plan** for the ComfyUI-Easy-Media nodes (`easy multiTrackEditor` +
`easy multitrackProject`), built with the same outline → approval → one-segment-per-turn →
merge → gate flow as `h3-plan-gate`, adapted to the Easy-Media multitrack schema.
The deliverable is a `plan.json` patch plan that the node pack's own
`skills/easy-media-multitrack-workflow/scripts/patch_workflow.py apply` consumes — it is
not pasted into a widget.
## What the gate enforces
| Layer | Check |
| --- | --- |
| Schema | Mirrors every check of the node pack's `patch_workflow.py` at apply time: track/segment ids, sorting, overlap, frame ranges, media `source_type`↔field matching, ≤ 9 images per task, continuity enum, one locked audio + one locked video max, locked-media coverage of task ranges, shared-reference rules, DynamicCombo resolution tokens, project fields — so a plan that passes here cannot be rejected there for schema reasons |
| Outline pins | Each segment's id, start/end frames (`round(seconds × fps)`), `task_mode`, `continuity_mode` and reference-image set are diffed against the approved guide at the segment turn AND again at the merge |
| Verbatim media contract | Every `file_path` in fragments and locked tracks must be a character-for-character copy of the outline's `media_inventory` entry (the same discipline `h3-plan-gate` applies to refmods; a retyped path fails with a closest-match hint) |
| Context prompt rules | Self-contained prompts in every mode (no "same as before"); context/context_drift joins must open by keeping the delivered tail and must not open with a new composition; a join followed by another join must not end at rest |
| Drift discipline | `context_drift` is all-or-nothing across the task range (first segment included), with the consecutive 10-second split unless the user pinned lengths |
## Folder
```text
easy-media-track-gate/
├── SKILL.md agent procedure (the entry point)
├── README.md this file
├── verify_track_rules.py the gate: outline / segment / trackdata / plan / suite
├── build_track_plan.py merge fragments → TRACK_DATA + patch plan.json
├── references/
│ └── track-schema.md English translation of the node pack's Chinese
│ multitrack-schema.md (all field names preserved)
└── examples/ worked 3-segment example: outline, segments, plan, track_data
```
Standard library only; Windows-first (`py -3`), runs anywhere Python 3.8+ lives.
## Install (portable skills folder)
Copy the whole folder next to your other skills:
```text
%USERPROFILE%\.agent\skills\easy-media-track-gate\
```
`SKILL.md` must sit directly in that folder (not nested one level deeper). Start a fresh
agent task after installing so the skill list refreshes. The node pack's own
`easy-media-multitrack-workflow` skill stays installed independently — this gate produces
its plan input and defers execution to it.
## Quick start
```sh
:: smoke test
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py"
:: validate a guide / one segment / the merged plan
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_01.json --index 1 --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_02.json --index 2 --prev segment_01.json --outline outline.json
:: merge everything (writes plan.json + track_data.json)
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" outline.json segment_01.json segment_02.json segment_03.json -o plan.json
:: apply with the node pack's own script (dry-run, then write)
py -3 <ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json
py -3 <ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json --output RESULT.json --write
```
## Provenance notes
- The schema reference is a faithful English translation of
`ComfyUI-Easy-Media/skills/easy-media-multitrack-workflow/references/multitrack-schema.md`
(which is written in Chinese); field names, enums and limits were additionally
cross-checked against the node pack's `patch_workflow.py` source, and the generated
example plan was dry-run-applied to the bundled v1.3.0 blank template successfully.
- Upstream leaves prompt quality, outline discipline and id/timing pins to discipline;
this gate makes them machine-checked. Everything upstream checks mechanically is
checked here with the same rules.
- Upload (`/upload/image`) and queue (`submit_workflow.py`) remain the node pack's own
flows and are only used on the user's explicit request.

View File

@ -1,389 +0,0 @@
---
name: easy-media-track-gate
description: Turn a video idea into a validated MultiTrack plan for ComfyUI-Easy-Media
(the `easy multiTrackEditor` node). Use when the user wants to plan task segments/shots
on the multitrack timeline, write or extend a MultiTrack timeline, arrange prompts,
reference images, master audio or reference-video locks, regenerate one task segment,
or produce the `prompt_override` paste payload for the multitrack editor node (plus the
optional patch plan JSON that `patch_workflow.py apply` consumes). The scripts and
references live next to this SKILL.md (the skill's install folder); run them from the
user's working folder, where all generated files are written.
---
# Easy-Media Track Gate — agent procedure
Goal: produce a validated plan for the `easy multiTrackEditor` node in five moves: the user
gives a video idea and a segment count (or an existing workflow to edit) → you build a
timeline guide (`outline.json`) and show it for **approval** → with the approved guide you
compose the task segments one at a time (about 10 seconds each), each turn existing so that
ONE segment gets a complete prompt, the right references and the right continuity mode →
the scripts verify schema, timing and continuity across the whole chain →
`build_track_plan.py` assembles the deliverables at the end. The PRIMARY deliverable is
`prompt_override.json`, a `minimax_prompt_override` payload the user pastes into the
multitrack editor node's `prompt_override` input (format `MiniMax`) — the editor then
renders one clip per prompt, chained back-to-back. The patch plan (`plan.json` for the node
pack's `patch_workflow.py`) is the optional whole-workflow alternative. Never deliver a plan
that did not pass validation, and never compose a segment from a guide the user has not
approved. Never hand the user any other JSON as the paste: a non-`minimax_prompt_override`
value in `prompt_override` falls back to the editor's `|`-separated text parser and renders
the whole story as ONE clip over the entire timeline.
## Paths and Python (read first — these exact forms prevent most errors)
- **SKILL_DIR** is the folder this SKILL.md was loaded from — the skill's install folder
(default on Windows: `%userprofile%\.agent\skills\easy-media-track-gate`; macOS/Linux:
`~/.agent/skills/easy-media-track-gate`). The scripts, the schema reference and the worked
example all live in SKILL_DIR, not in the working folder. Every command below uses the
Windows default install path; substitute your real SKILL_DIR if it differs.
- Work in the user's **working folder** (the folder open in the editor). Write generated
files there: `outline.json`, `segment_01.json`, `segment_02.json`, …, `plan.json`,
`track_data.json`. Never write generated files into SKILL_DIR.
- Python command: Windows `py -3` (or `python`); macOS/Linux `python3`. If the working
folder has a `.venv`, its interpreter works too (`.venv\Scripts\python.exe`). The scripts
need only the Python standard library — nothing to install.
- Start every task by running the smoke test under "Before you start". If it fails because
SKILL_DIR is missing files, stop and tell the user the skill folder is incomplete.
- **Never read `verify_track_rules.py`** — run it and work from its printed output. Cline's
read_file is capped by hard-coded budgets that do NOT grow with the model's context
window: about 48,000 characters per read, 2,000 lines per read. The 81 KB validator
arrives cut no matter how large the context window is. A truncation notice is Cline's
read cap, not repo corruption — do not "fix" truncated files. `build_track_plan.py`
(~10 KB) and `references/track-schema.md` (~26 KB) are safe to read whole.
- **Never read back `plan.json`, `track_data.json` or `prompt_override.json` once real
segments exist** — a 10-segment plan passes the read caps today and exceeds them
tomorrow. Validate with `--check` / the `override` command and trust the one-line OK
summary; the shipped `examples/prompt_override.json` (2.5 KB) and `examples/plan.json`
(6.8 KB) are safe to read if you need to see the shape.
- Cline's editor rejects writes over 6,000 characters ("Editor input too large"). A segment
fragment with a rich prompt often does. NEVER write part 2 with write_to_file — it
replaces the whole file and part 1 is lost. Split instead: write the first part ending
with the bare marker line `@@CONTINUE@@`, then replace_in_file with SEARCH `@@CONTINUE@@`
and REPLACE the remaining content. The finished file must not contain the marker.
- Cline also cuts command output at 8,000 characters. The embedded suite prints well under
1 KB when it passes; a failing validation prints one actionable line. If output ever
arrives truncated, re-run the single-file command (`segment`, `outline`, `plan`) — its
output is tiny.
## Before you start (smoke test)
```sh
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\plan.json"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\prompt_override.json" --outline "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\outline.json"
```
Expect the suite to print `SUITE PASS (34/34 cases)` and the other two commands to print
`PLAN OK` with `total_length 720 frames = 30.00s` and `OVERRIDE OK: 3 prompts, duration 10,
total 30.00s`.
## Step 0 — Media paths come first (never invent one)
Every `file_path` in the plan must be a real path on the user's ComfyUI instance. Before
writing the outline, establish the media inventory one of two ways:
- The user names the exact input paths (they know their `input/` subfolders). Use their
strings verbatim — they become the pinned inventory.
- The user explicitly asks you to upload → upload first and record the returned relative
path (`subfolder/name`, `source_type: "input"`) for every file. Upload uses the node
pack's own API (`POST /upload/image` with a `subfolder` like `codex/<project>`); never
write a local absolute path into the plan.
- If there is no real path and no upload request, stop and ask. A plan is still deliverable
with obviously-fake placeholder paths only if you say plainly it cannot run until real
media replaces them — never dress a placeholder up as a real path.
Upload and execution are external state changes: only upload/queue when the user explicitly
asked (Step 5).
## Step 1 — Build the timeline guide (outline)
Read SKILL_DIR's `references/track-schema.md` once (safe whole-file read; it is the English
translation of the node pack's Chinese schema doc). Then write `outline.json` — the guide
for the whole timeline — in exactly this shape (the shipped `examples/outline.json` is the
worked model):
```json
{
"title": "short_snake_case_name",
"lengths_user_pinned": false,
"editor": {
"format": "MiniMax",
"resolution": { "resolution": "width x height (megapixels)",
"aspect_ratio": "16:9 (Widescreen)", "megapixels": 0.8 }
},
"defaults": { "frame_rate": 24, "segment_seconds": 10 },
"media_inventory": [
{ "key": "SONG", "kind": "audio", "source_type": "input",
"file_path": "codex/rooftop/song.wav", "file_name": "song.wav" },
{ "key": "POSE_REF", "kind": "image", "source_type": "input",
"file_path": "codex/rooftop/pose_ref.png", "file_name": "pose_ref.png" }
],
"tracks": {
"master_audio": { "key": "SONG", "lock": true, "name": "Master Song", "role": "why it drives the timeline" },
"reference_video": null
},
"project": { "project_save": "override", "segment_start_number": 1,
"segment_count": -1, "sampling_plan": "medium", "sampling_mode": "single" },
"segments": [
{ "index": 1, "id": "seg_one_open", "start_seconds": 0, "duration_seconds": 10,
"task_mode": "ref", "continuity_mode": "shot",
"images": [ { "key": "POSE_REF", "shared_reference": false } ],
"beat": "one sentence: what happens AND where the segment ends",
"prompt_brief": "1-2 sentences: what the prompt keeps first, changes later, ends on" }
]
}
```
Field rules the validator enforces:
- `media_inventory` keys are UPPERCASE; every `file_path` is copied from Step 0 sources —
this inventory is the **machine-checked verbatim source**: every segment's image items and
every locked media track are diffed against it, so a retyped path fails on the spot. A
paraphrase in the inventory cannot be repaired later. Order matters too: the inventory's
image rows map to the editor's `image` input positions (`<Picture 1>`, `<Picture 2>`, …),
audio rows to `audio` input positions, video rows to `video` input positions — the user
wires the media into those node inputs in inventory order.
- `tracks.master_audio` / `tracks.reference_video` name the ONE locked audio and ONE locked
video track (the schema allows at most one of each). Lock the master song for MV/lyric/
dance/beat projects, the narration track when narration drives the cut, and the reference
video when the user supplies one for subject/partial replacement. Ordinary BGM/SFX/ambience
is never locked. If two candidates could be the master, stop and ask the user — never pick
by track order or duration.
- Resolution: copy the user's intent into the exact DynamicCombo tokens — `16:9` means
`16:9 (Widescreen)`, `0.9mp` means megapixels mode with the number taken verbatim; never
convert MP into width/height. Keep the current workflow resolution when the user expressed
no intent (then omit nothing — still write the block the user approved).
- Segments: `start_seconds` are cumulative (previous start + duration + gap); ids are unique
lowercase snake_case, never renamed later; `duration_seconds` about 10 unless the user
says otherwise; only the last segment may come to rest. `continuity_mode` per segment:
`shot` (standalone/new composition), `context` (continues the previous segment's real
delivered tail), `context_drift` (context + subject/partial replacement from the locked
reference video). **context_drift is all-or-nothing**: if any segment drifts, all do,
first included; without user-pinned lengths the drift split is consecutive 10-second
pieces (last takes the remainder) — set `lengths_user_pinned: true` only when the user
actually specified the lengths.
- `images` per segment: at most 9, each an inventory image key with its `shared_reference`
flag. Reference images define IDENTITY, not composition. When several context segments
reuse one image, prefer explicit private items (`shared_reference: false`, fixed ids,
same file) over shared-image metadata — the current front-end regenerates UUIDs for
sparse shared images, and unverified shared data risks unstable loads.
- `project`: `project_save: "override"` with `segment_count: -1` deletes saved segments
after the start index on execution — warn the user before choosing it. `upscale_by` is
only written when the user asked (default 1.250, three decimals); the editor resolution
is the first-pass size, and dual-sampling second pass = `round(editor_size * upscale_by /
32) * 32`.
## Step 2 — Present the guide for approval (do not skip)
Show the user the finished guide in plain language — one line per segment: number, id,
seconds, continuity mode, what happens and where it ends — plus the media inventory, the
locked tracks, the resolution block and the project parameters. Then STOP and wait for
their reply. The user approves it or asks for changes; apply every change to `outline.json`
and, if anything changed, show the changed rows again. Start Step 3 only after an explicit
approval. The approved guide is the contract: segments are composed from it, ids, frames
and modes are never silently altered, and new story ideas from the user go through the
guide first — never straight into a segment file.
## Step 3 — Compose the segments one at a time
Work only from the approved guide. In story order: compose ONE segment file, validate it,
fix it until it passes, and only then move to the next. Since you wrote the previous file
yourself, open it and reuse its `_handoff.end_state` as the opening state of the next
context join. Rewrite a segment as many times as it takes. If Cline rejects a write with
'Editor input too large', split it with the marker pattern from the rules above.
**Media copy rule (the one writers keep breaking):** every `file_path` in a segment's
`images` is a character-for-character COPY of the outline's `media_inventory` path for that
key. Select the text in `outline.json`, copy, paste — never retype it from memory, never
fix a typo in it, never reword a folder name. The commands below pass `--outline`, so the
validator diffs the paths against the guide and a rewrite fails the segment on the spot.
The same discipline covers the resolution tokens and file names: the merge copies
`editor.resolution` and the locked media tracks straight from the outline, so the outline
is the only place they are ever typed.
Each fragment is exactly this shape:
```json
{
"id": "from the outline row",
"start_frame": 0,
"end_frame": 240,
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "shot",
"ref_image_size": "match",
"images": [
{ "id": "img-seg-one-open-1", "source_type": "input",
"file_path": "copied verbatim from the inventory", "file_name": "pose_ref.png",
"shared_reference": false }
],
"user_prompt": "the full segment prompt",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
},
"_handoff": { "end_state": "the real final frame in 2-3 sentences: pose, motion in progress, camera state" }
}
```
`start_frame`/`end_frame` are computed, not guessed: `round(seconds * frame_rate)`,
start inclusive, end exclusive, and they must match the approved row exactly. If the
variant is `b`, write `user_prompt_b` and leave `user_prompt` alone — never switch the
variant on your own.
```sh
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_01.json --index 1 --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_02.json --index 2 --prev segment_01.json --outline outline.json
```
(`--total` is optional with `--outline`; `--prev` chains the previous file and enforces
consecutive frames `[0,240), [240,480), …` — never an extra frame at a boundary.)
### Prompt craft rules (violations fail the validator)
- Every task segment renders standalone — the model never sees any other segment's prompt.
Fully restate scene, light and subject identity inside each prompt; never say "the same
one", "as before", "the previous shot". This ban is checked in every mode.
- Reference images define identity only: they are not the first frame or a composition
anchor; context joins never use image anchoring or `[video continuation]`.
- **context segments open by keeping the delivered tail** — the join starts from the
previous segment's real last frames (about 0.92 s earlier than the saved boundary at
24 fps), whose framing can differ from what the previous prompt imagined: open with the
camera keeping its direction and the subject keeping pose, motion and light. Give the
hold an action (a breath, a weight shift) so it does not render as a freeze-frame.
- **Keep first, change later**: before about `00:02.000` no cut, no new key action; a
required composition change is written as an explicit cut later in the prompt. For a
genuinely new composition use a `shot` segment instead of `context`.
- **End in motion**: a context segment followed by another context segment never ends at
rest — no "comes to a stop", "settles", "holds still", "falls asleep" in the closing
lines. Only the timeline's final segment may come to rest.
- Compose, never summarize: subject anchor with its look, blocking, one camera idea,
performance beats, physical texture (wardrobe behavior, weather on skin and fabric,
light), and a concrete mid-action exit state. Decompose every action into ordered
body mechanics ("bends his knees, grips the handle with his left hand, then stands"),
naming the limb; camera, light and weather each get their own ordered visible stages.
The validator trips a thin-prompt floor (60 words) — never pad to a band, and never
hand in a one-liner.
- `system_prompt` stays `""` unless the user asked for one: an empty system prompt lets the
node auto-append the MiniMax reference rules for context joins; a custom one disables
that.
- `_handoff.end_state` must be truthful — it is the contract the next join opens inside.
## Step 4 — Verify the chain, put it all together at the end
Once every segment file passes, the merge is the continuity check: `build_track_plan.py`
re-walks every fragment against the guide (ids, frames, modes, verbatim media paths, prompt
rules, handoff chain), assembles the TRACK_DATA (task track from the fragments; the locked
master-audio / reference-video tracks built straight from the outline), computes
`total_length`, builds the `minimax_prompt_override` paste payload from the same validated
fragments, and only then writes the deliverables. Never write the plan by hand and never
edit fragment prose during the merge.
```sh
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" outline.json segment_01.json segment_02.json segment_03.json -o plan.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check plan.json --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override prompt_override.json --outline outline.json
```
The merge writes `prompt_override.json` (the paste payload, one line), `plan.json` (the
patch plan) and `track_data.json` (pretty, for inspection), and prints the delivered
durations/seconds, the track list and the locked tracks. Segment, track and image ids are
stable UUIDv5 values derived from the project title, so a re-merge keeps ids consistent.
The `override` command re-checks the paste payload against the guide: one prompt per
segment, durations inside the runtime's 2–15s window and equal to the approved rows,
per-segment generation/continuity modes, the exact `<Picture N>` / `<Audio N>` / `<Video N>`
reference set per prompt (a missing token renders with no reference media; a stray one
attaches the wrong image), and the track locks.
## Step 5 — Deliver (paste payload first, patch script as the alternative)
### 5A — Default: the `prompt_override` paste
`prompt_override.json` is what the user pastes into the **multitrack editor node's
`prompt_override` input** (`easy multiTrackEditor`). It is self-contained: one prompt per
clip, per-clip durations and continuity modes, media reference tokens inside the prompts.
Deliver it with these instructions:
1. In the editor node set `format` to `MiniMax` (the payload is parsed as MiniMax only in
that format).
2. Wire the media IF the outline has any: the inventory's image rows (in inventory order)
into the editor's `image` input, the master song into `audio`, the reference video into
`video`. The `<Picture N>` / `<Audio N>` / `<Video N>` tokens resolve against those
input lists positionally. With an empty inventory (`[]`, pure t2v) nothing gets wired —
the editor demands no media inputs at all.
3. Select the `prompt_override` widget, clear it, paste the whole one-line JSON. The editor
rebuilds the task track: one clip per prompt, back-to-back, durations as specified.
4. **Rendering walks the timeline through `easy multiTrackTaskOutput` — and that node
outputs ONE task segment per pass, selected by its `task_index` int input (default 0 =
only the FIRST clip).** A workflow whose `task_index` stays 0 renders only the first
scene no matter how many prompts the override carries. Two ways to render all clips:
- **Loop workflow (proper multi-shot):** use the node pack's
`example_workflows/MinimaxH3_ForLoop_MultiShot.json`. Its `forLoopStart` takes
`total` from `easy multiTrackInfoOutput.TASK_COUNT` (via a math node that also lets a
PrimitiveInt pick one segment: -1 = all), and its index feeds
`multiTrackTaskOutput.task_index`; each iteration generates one clip, batches it, and
`mergeVideosFromPaths` concatenates everything after the loop. Paste the override
into THAT editor and set the PrimitiveInt to -1.
- **Manual fallback (no rewiring):** set `task_index` to 0, queue; then 1, queue; then
2, queue. Three clip files, join them afterwards. (`task_index = -1` outputs the whole
timeline's pre-rendered MEDIA — it is not a 3-clip generation.)
5. **Optional LLM polish slot** (only if the user asks for it): inside the loop,
`TaskOutput.SYSTEM_PROMPT/USER_PROMPT` may feed an LLM node whose output replaces the
prompt at the render node, with an ifElse bypass back to raw `USER_PROMPT`. The skill's
prompts are already final — leave the bypass OFF unless requested. Ways to fill the
slot: any ComfyUI LLM node (system+user in, string out), or the pack's
`easy multiTrackPromptEnhancer` with model `openai_compatible (local)` pointed at the
user's own endpoint (`base_url` accepts any OpenAI-compatible server: Ollama, LM
Studio, vLLM, llama.cpp server). Full wiring: `references/workflow_wiring.md`.
**The paste target is the `prompt_override` input and only that payload type.** Never give
the user a `plan.json`, an outline, an H3 plan or any other JSON to paste there: without
`"type": "minimax_prompt_override"` the editor falls back to its `|`-separated text parser
and renders the whole thing as ONE clip spanning the timeline (the single-clip failure).
Never add `|` inside prompts for the same reason.
### 5B — Alternative: the node pack's patch script (whole-workflow patch)
When the user wants the timeline written into a workflow file directly (or needs `edit`
task modes / extra tracks the paste payload cannot express), the deliverable is applied by
the Easy-Media node pack's script — find it under the node pack folder (typically
`<ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py`;
ask the user for the path and the workflow JSON if they are not at hand):
```sh
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json --output RESULT.json --write
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py inspect RESULT.json
```
Always dry-run first; only write to a NEW output file (`--output RESULT.json --write`),
never over the user's source workflow. `inspect` must show the editor's resolution, format,
frame rate, `total_length`, the task track with all segments and the locked tracks with
their media. Report: base workflow path, output file, editor/project node ids, resolution,
timeline summary, media paths, continuity modes, locked tracks and whether the node graph
stayed untouched (`graph_preserved: true`).
Actually opening the workflow in ComfyUI or queueing it (`submit_workflow.py`, UI-tab
submission) is a separate authorization — only on the user's explicit request, and it is
the node pack's own flow (see its `references/execution-api.md`). "Queued" never means
"generated"; wait for the real completion status before reporting success.
## Revising after approval
- Guide-level change (timings, beats, modes, segment count, media): update `outline.json`
first, show the changed rows, then rewrite only the affected segments against it and
re-run the Step 4 merge so `plan.json` and `prompt_override.json` refresh.
- One segment reads badly: rewrite that `segment_NN.json` (keep its id and frames from the
guide), re-validate it with the `segment` command, then re-merge. Never patch `plan.json`
or `prompt_override.json` directly — the merge rebuilds both from the fragments every
time.
## If you cannot run Python
Say plainly that the result skipped script validation. Follow the rules manually: copy the
`examples/` shapes exactly, compute frames as `round(seconds * frame_rate)`, copy every
`file_path` and resolution token by selection (never retyping), and hand-assemble the
TRACK_DATA the way Step 4 describes. Manual assembly is error-prone — prefer getting
Python running (any of `py -3`, `python`, a `.venv`) over hand-building the plan.

View File

@ -1,244 +0,0 @@
#!/usr/bin/env python3
"""Merge approved outline + task-segment fragments into the Easy-Media MultiTrack deliverables.
py -3 build_track_plan.py outline.json segment_01.json segment_02.json -o plan.json
py -3 build_track_plan.py --check plan.json [--outline outline.json]
The merge is the continuity check: it re-validates every fragment against the approved
outline (ids, frames, modes, verbatim media paths, context-prompt rules, handoff chain),
then assembles TWO deliverables from it:
1. `prompt_override.json` — the PRIMARY paste payload for the `easy multiTrackEditor`
node's `prompt_override` input (format MiniMax): {"type": "minimax_prompt_override",
...} with one prompt per segment, per-segment durations and continuity modes. Paste it
into the widget and queue; wire the inventory's media into the editor's image/audio/
video inputs in inventory order so the <Picture N>/<Audio N>/<Video N> tokens resolve.
2. `plan.json` — the patch plan the node pack's `patch_workflow.py apply` consumes
({"editor": {..., "track_data": {...}}, "project": {...}}) for patching a whole
workflow file, plus `track_data.json` (pretty, for inspection).
Segment/track/image ids are deterministic UUIDv5 values derived from the project title, so
re-running the merge keeps ids stable. Never edit fragment prose here and never write the
plan by hand — fix the fragment, re-validate, re-merge.
Standard library only.
"""
from __future__ import annotations
import argparse
import copy
import json
import sys
import uuid
from pathlib import Path
from typing import Any
sys.path.insert(0, str(Path(__file__).resolve().parent))
import verify_track_rules as vtr # noqa: E402
def stable_id(*parts: str) -> str:
seed = "easy-media-track-gate/" + "/".join(parts)
return str(uuid.uuid5(uuid.NAMESPACE_URL, seed))
def _finalize_image(img: dict[str, Any], inv_entry: dict[str, Any], seg_id: str, pos: int,
shared_default: bool) -> dict[str, Any]:
out = {
"id": img.get("id") or stable_id(seg_id, f"image-{pos}"),
"source_type": img.get("source_type", inv_entry["source_type"]),
"file_path": inv_entry["path"],
"file_name": img.get("file_name") or inv_entry.get("file_name")
or inv_entry["path"].rsplit("/", 1)[-1],
"shared_reference": img.get("shared_reference", shared_default),
}
if out["source_type"] == "url":
out["url"] = img.get("url") or inv_entry["path"]
out.pop("file_path", None)
out.pop("file_name", None)
return out
def build(outline: dict[str, Any], fragments: list[dict[str, Any]]) -> dict[str, Any]:
rows = outline["rows"]
if len(fragments) != len(rows):
raise vtr.Fail(f"expected {len(rows)} fragments (one per approved outline row), got {len(fragments)}")
prev: dict[str, Any] | None = None
for i, frag in enumerate(fragments):
vtr.validate_fragment(frag, i + 1, len(rows), prev, outline)
prev = frag
task_segments: list[dict[str, Any]] = []
for i, (frag, row) in enumerate(zip(fragments, rows)):
seg_id = stable_id(outline["title"], "segment", row["id"])
content = copy.deepcopy(frag["content"])
content.setdefault("media_type", "none")
content["media_type"] = "none"
content.setdefault("user_prompt_variant", "a")
if content.get("user_prompt_variant") == "a":
content.setdefault("user_prompt_b", "")
content.setdefault("system_prompt", "")
content.setdefault("muted", False)
content.setdefault("volume_db", 0)
if "ref_image_size" not in content:
content["ref_image_size"] = "match"
row_imgs = row.get("images", [])
final_imgs = []
for pos, (img, row_img) in enumerate(zip(content.get("images", []), row_imgs)):
inv_entry = outline["inventory"][row_img["key"]]
final_imgs.append(_finalize_image(img, inv_entry, seg_id, pos,
bool(row_img.get("shared_reference", False))))
content["images"] = final_imgs
task_segments.append({
"id": seg_id,
"start_frame": frag["start_frame"],
"end_frame": frag["end_frame"],
"color": "var(--multitrack-task-bg)",
"content": content,
})
total = max((s["end_frame"] for s in task_segments), default=0)
tracks: list[dict[str, Any]] = [{
"id": stable_id(outline["title"], "track", "tasks"),
"name": "Tasks",
"type": "task",
"color": "var(--multitrack-task-bg)",
"muted": False,
"solo": False,
"volume_db": 0,
"locked": False,
"segments": task_segments,
}]
roles = outline["_roles"]
for role, track_type in (("master_audio", "audio"), ("reference_video", "video")):
spec = roles.get(role)
if not spec:
continue
inv = outline["inventory"][spec["key"]]
media_content: dict[str, Any] = {
"media_type": track_type,
"source_type": inv["source_type"],
}
if inv["source_type"] == "url":
media_content["url"] = inv["path"]
else:
media_content["file_path"] = inv["path"]
media_content["file_name"] = inv.get("file_name") or inv["path"].rsplit("/", 1)[-1]
tracks.append({
"id": stable_id(outline["title"], "track", role),
"name": spec["name"],
"type": track_type,
"color": "var(--primary)",
"muted": False,
"solo": False,
"volume_db": 0,
"locked": False,
"audio_locked": True,
"segments": [{
"id": stable_id(outline["title"], "media", role),
"start_frame": 0,
"end_frame": max(total, 1),
"color": "var(--primary)",
"content": media_content,
}],
})
track_data: dict[str, Any] = {
"muted": False,
"volume_db": 0,
"task_markers": [],
"task_overview": False,
"tracks": tracks,
"total_length": total,
"frame_rate": outline["defaults"]["frame_rate"],
}
plan: dict[str, Any] = {
"editor": {
"resolution": copy.deepcopy(outline["editor"]["resolution"]),
"track_data": track_data,
"format": outline["editor"]["format"],
},
"project": copy.deepcopy(outline["project"]),
}
project = plan["project"]
vtr.validate_plan(plan, outline=outline)
return plan
def _summary(plan: dict[str, Any]) -> str:
data = plan["editor"]["track_data"]
task_count = sum(len(t.get("segments", [])) for t in data["tracks"] if t.get("type") == "task")
locked = [f"{t['type']}:{t.get('name', t['id'])}" for t in data["tracks"]
if t.get("audio_locked") is True]
fps = data["frame_rate"]
return (f"{task_count} task segments, total_length {data['total_length']} frames "
f"= {data['total_length'] / fps:.2f}s at {fps} fps; "
f"tracks: {', '.join(t['type'] + ' ' + t.get('name', '') for t in data['tracks'])}; "
f"audio_locked: {', '.join(locked) if locked else 'none'}")
def _write_json(path: Path, value: dict[str, Any]) -> None:
path.write_text(json.dumps(value, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
def _main(argv: list[str]) -> int:
parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
parser.add_argument("inputs", nargs="*", help="outline.json then segment_01.json ... (merge) or plan.json (--check)")
parser.add_argument("-o", "--output", default="plan.json")
parser.add_argument("--check", action="store_true", help="re-validate an existing plan.json")
parser.add_argument("--outline", help="outline to re-check the plan against")
args = parser.parse_args(argv)
try:
if args.check:
if not args.inputs:
raise vtr.Fail("--check needs the plan file path")
plan = vtr.load_json(args.inputs[0], "patch plan")
outline = None
if args.outline:
outline = vtr.load_json(args.outline, "outline")
vtr.validate_outline(outline)
vtr.validate_plan(plan, outline=outline)
print(f"PLAN OK: {_summary(plan)}")
return 0
if len(args.inputs) < 2:
raise vtr.Fail("usage: build_track_plan.py outline.json segment_01.json [segment_02.json ...] -o plan.json")
outline = vtr.load_json(args.inputs[0], "outline")
vtr.validate_outline(outline)
fragments = [vtr.load_json(p, "segment fragment") for p in args.inputs[1:]]
plan = build(outline, fragments)
out_path = Path(args.output)
_write_json(out_path, plan)
_write_json(out_path.parent / "track_data.json", plan["editor"]["track_data"])
override = vtr.build_prompt_override(outline, fragments)
vtr.validate_prompt_override(copy.deepcopy(override), outline)
override_path = out_path.parent / "prompt_override.json"
override_path.write_text(
json.dumps(override, ensure_ascii=True, separators=(",", ":")) + "\n",
encoding="utf-8")
prompts = len(override["prompts"])
total_s = sum(float(row["duration_seconds"]) for row in outline["rows"])
print(f"OVERRIDE OK: {prompts} prompts, duration {override['duration']}, "
f"total {total_s:.2f}s; wrote {override_path} "
f"({override_path.stat().st_size} chars, one line)")
print(f"PLAN OK: {_summary(plan)}")
print(f"wrote {out_path} and {out_path.parent / 'track_data.json'}")
print("deliver: paste prompt_override.json into the multitrack editor's "
"prompt_override input (format MiniMax)")
print("NOTE: easy multiTrackTaskOutput renders ONE task segment per pass "
"(task_index, default 0 = first clip only).")
print(" - all clips: use the pack's example_workflows/MinimaxH3_ForLoop_MultiShot.json "
"(loop index -> task_index, PrimitiveInt -1 = all, auto-merge)")
print(f" - or queue manually with task_index 0..{prompts - 1} and join the clips")
print("optional whole-workflow patch (dry-run first):")
print(' py -3 <ComfyUI-Easy-Media>\\skills\\easy-media-multitrack-workflow\\scripts\\patch_workflow.py apply WORKFLOW.json --plan plan.json')
print(' ... then add --output RESULT.json --write and inspect RESULT.json')
return 0
except vtr.Fail as exc:
print(f"FAIL: {exc}")
return 1
if __name__ == "__main__":
sys.exit(_main(sys.argv[1:]))

View File

@ -1,89 +0,0 @@
{
"title": "sunrise_rooftop_coffee",
"lengths_user_pinned": true,
"editor": {
"format": "MiniMax",
"resolution": {
"resolution": "width x height (megapixels)",
"aspect_ratio": "16:9 (Widescreen)",
"megapixels": 0.8
}
},
"defaults": {
"frame_rate": 24,
"segment_seconds": 10
},
"media_inventory": [
{
"key": "SONG",
"kind": "audio",
"source_type": "input",
"file_path": "codex/rooftop/song.wav",
"file_name": "song.wav"
},
{
"key": "POSE_REF",
"kind": "image",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png"
}
],
"tracks": {
"master_audio": {
"key": "SONG",
"lock": true,
"name": "Master Song",
"role": "the user's song drives the whole timeline"
},
"reference_video": null
},
"project": {
"project_save": "override",
"segment_start_number": 1,
"segment_count": -1,
"sampling_plan": "medium",
"sampling_mode": "single"
},
"segments": [
{
"index": 1,
"id": "seg_one_open",
"start_seconds": 0,
"duration_seconds": 10,
"task_mode": "ref",
"continuity_mode": "shot",
"images": [
{ "key": "POSE_REF", "shared_reference": false }
],
"beat": "She pours coffee at the rooftop rail as the sun clears the skyline; ends mid-sip, city brightening.",
"prompt_brief": "Establish rooftop, rail, skyline; pour coffee; end mid-sip with motion alive."
},
{
"index": 2,
"id": "seg_two_sip_wind",
"start_seconds": 10,
"duration_seconds": 10,
"task_mode": "ref",
"continuity_mode": "context",
"images": [
{ "key": "POSE_REF", "shared_reference": false }
],
"beat": "Wind tugs her cardigan mid-sip; she turns her face into the light; ends still turning.",
"prompt_brief": "Keep framing and pose from the delivered tail; wind arrives at 00:02; end turning."
},
{
"index": 3,
"id": "seg_three_city_wakes",
"start_seconds": 20,
"duration_seconds": 10,
"task_mode": "ref",
"continuity_mode": "context",
"images": [
{ "key": "POSE_REF", "shared_reference": false }
],
"beat": "She lowers the cup as windows light up below; ends mid-exhale, steam curling.",
"prompt_brief": "Keep the turn; lower the cup slowly; city windows light up; end mid-exhale."
}
]
}

View File

@ -1,145 +0,0 @@
{
"editor": {
"resolution": {
"resolution": "width x height (megapixels)",
"aspect_ratio": "16:9 (Widescreen)",
"megapixels": 0.8
},
"track_data": {
"muted": false,
"volume_db": 0,
"task_markers": [],
"task_overview": false,
"tracks": [
{
"id": "3b53d88d-07b5-5f0e-9fc0-aa87503aae80",
"name": "Tasks",
"type": "task",
"color": "var(--multitrack-task-bg)",
"muted": false,
"solo": false,
"volume_db": 0,
"locked": false,
"segments": [
{
"id": "d2c2d695-f725-56ae-9024-bf9126443256",
"start_frame": 0,
"end_frame": 240,
"color": "var(--multitrack-task-bg)",
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "shot",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-one-open-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "Dawn light spills across an empty rooftop terrace above a hushed city. A young woman with a brown ponytail, cream cardigan over a white tee and blue jeans stands at the steel rail, matching the pose reference image for her identity only. She cradles a matte-black kettle in her right hand and tips it in one slow arc; a thread of steam rises as coffee fills the white cup in her left hand, her wrist rotating smoothly, elbow dropping as the kettle empties. She sets the kettle on the rail with a soft clink, lifts the cup, and brings it halfway to her lips as the sun's first rim crests the skyline and warms her cheek. The camera holds one slow push-in toward her profile, never cutting, while pigeon wings clap once somewhere below and the kettle's metal ticks as it cools.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
}
},
{
"id": "69749e2e-3bd5-5e50-ae29-61bf607cba20",
"start_frame": 240,
"end_frame": 480,
"color": "var(--multitrack-task-bg)",
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "context",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-two-sip-wind-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "The camera holds the same framing it arrived with, and she keeps the pose, motion and light it arrived with: the cup still traveling toward her lips, steam bending in the same breeze. Her chest rises with one slow breath, weight easing onto her left hip as the cardigan's hem starts to lift. Around the two-second mark the wind strengthens into a gust that ruffles her ponytail and snaps the cardigan open a finger's width; her fingers reclose it while she turns her face slowly into the light, eyelids narrowing. Steam tears off the cup in ribbons. The camera keeps its single slow drift leftward with no cut, the city below sharpening frame by frame, traffic swelling from hum to roar, a rooftop antenna creaking softly, her cardigan snapping in steady wind as the turn continues.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
}
},
{
"id": "6d0f444e-c6b1-5d14-a3c6-a29cd594beb7",
"start_frame": 480,
"end_frame": 720,
"color": "var(--multitrack-task-bg)",
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "context",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-three-city-wakes-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "The join keeps everything it is handed: same framing, same breeze, the tail of her turn still in motion as her face comes level with the camera. She continues the turn until her eyes find the skyline, then lowers the cup in one unhurried arc to the rail, her shoulder rolling with the descent. Below, windows begin to light up floor by floor, a slow tide of warm rectangles climbing the tower across the street while a bus door hisses open. She exhales a long breath, steam curling off the cup in the same wind, ponytail still lifting, cardigan snapping softly. The camera keeps its slow lateral drift and never cuts; the city's morning roar builds under the song as her chest keeps rising and falling mid-breath.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
}
}
]
},
{
"id": "51945496-eaa4-5ba0-bb78-201bbfee5819",
"name": "Master Song",
"type": "audio",
"color": "var(--primary)",
"muted": false,
"solo": false,
"volume_db": 0,
"locked": false,
"audio_locked": true,
"segments": [
{
"id": "ee335ed8-75cd-5278-969f-144cba015861",
"start_frame": 0,
"end_frame": 720,
"color": "var(--primary)",
"content": {
"media_type": "audio",
"source_type": "input",
"file_path": "codex/rooftop/song.wav",
"file_name": "song.wav"
}
}
]
}
],
"total_length": 720,
"frame_rate": 24
},
"format": "MiniMax"
},
"project": {
"project_save": "override",
"segment_start_number": 1,
"segment_count": -1,
"sampling_plan": "medium",
"sampling_mode": "single"
}
}

View File

@ -1 +0,0 @@
{"type":"minimax_prompt_override","version":1,"prompts":["<Picture 1> <Audio 1> Dawn light spills across an empty rooftop terrace above a hushed city. A young woman with a brown ponytail, cream cardigan over a white tee and blue jeans stands at the steel rail, matching the pose reference image for her identity only. She cradles a matte-black kettle in her right hand and tips it in one slow arc; a thread of steam rises as coffee fills the white cup in her left hand, her wrist rotating smoothly, elbow dropping as the kettle empties. She sets the kettle on the rail with a soft clink, lifts the cup, and brings it halfway to her lips as the sun's first rim crests the skyline and warms her cheek. The camera holds one slow push-in toward her profile, never cutting, while pigeon wings clap once somewhere below and the kettle's metal ticks as it cools.","<Picture 1> <Audio 1> The camera holds the same framing it arrived with, and she keeps the pose, motion and light it arrived with: the cup still traveling toward her lips, steam bending in the same breeze. Her chest rises with one slow breath, weight easing onto her left hip as the cardigan's hem starts to lift. Around the two-second mark the wind strengthens into a gust that ruffles her ponytail and snaps the cardigan open a finger's width; her fingers reclose it while she turns her face slowly into the light, eyelids narrowing. Steam tears off the cup in ribbons. The camera keeps its single slow drift leftward with no cut, the city below sharpening frame by frame, traffic swelling from hum to roar, a rooftop antenna creaking softly, her cardigan snapping in steady wind as the turn continues.","<Picture 1> <Audio 1> The join keeps everything it is handed: same framing, same breeze, the tail of her turn still in motion as her face comes level with the camera. She continues the turn until her eyes find the skyline, then lowers the cup in one unhurried arc to the rail, her shoulder rolling with the descent. Below, windows begin to light up floor by floor, a slow tide of warm rectangles climbing the tower across the street while a bus door hisses open. She exhales a long breath, steam curling off the cup in the same wind, ponytail still lifting, cardigan snapping softly. The camera keeps its slow lateral drift and never cuts; the city's morning roar builds under the song as her chest keeps rising and falling mid-breath."],"duration":"10","generation_type":"r2v","continuity_mode":"shot,context,context","system_prompt":"","video_track_lock":0,"audio_track_lock":1}

View File

@ -1,30 +0,0 @@
{
"id": "seg_one_open",
"start_frame": 0,
"end_frame": 240,
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "shot",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-one-open-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "Dawn light spills across an empty rooftop terrace above a hushed city. A young woman with a brown ponytail, cream cardigan over a white tee and blue jeans stands at the steel rail, matching the pose reference image for her identity only. She cradles a matte-black kettle in her right hand and tips it in one slow arc; a thread of steam rises as coffee fills the white cup in her left hand, her wrist rotating smoothly, elbow dropping as the kettle empties. She sets the kettle on the rail with a soft clink, lifts the cup, and brings it halfway to her lips as the sun's first rim crests the skyline and warms her cheek. The camera holds one slow push-in toward her profile, never cutting, while pigeon wings clap once somewhere below and the kettle's metal ticks as it cools.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
},
"_handoff": {
"end_state": "She stands at the rail with the cup halfway to her lips, wrist still rotating it inward, ponytail and cardigan hem lifted by a first light breeze, the camera mid push-in on her profile as the sun rim crests the skyline.",
"beat_used": "She pours coffee at the rooftop rail as the sun clears the skyline; ends mid-sip, city brightening."
}
}

View File

@ -1,30 +0,0 @@
{
"id": "seg_two_sip_wind",
"start_frame": 240,
"end_frame": 480,
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "context",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-two-sip-wind-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "The camera holds the same framing it arrived with, and she keeps the pose, motion and light it arrived with: the cup still traveling toward her lips, steam bending in the same breeze. Her chest rises with one slow breath, weight easing onto her left hip as the cardigan's hem starts to lift. Around the two-second mark the wind strengthens into a gust that ruffles her ponytail and snaps the cardigan open a finger's width; her fingers reclose it while she turns her face slowly into the light, eyelids narrowing. Steam tears off the cup in ribbons. The camera keeps its single slow drift leftward with no cut, the city below sharpening frame by frame, traffic swelling from hum to roar, a rooftop antenna creaking softly, her cardigan snapping in steady wind as the turn continues.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
},
"_handoff": {
"end_state": "Her face is mid-turn toward the morning light, chin still traveling, cardigan snapping in the sustained gust, cup lowered to chest height with steam tearing off it, the camera still drifting left without a cut.",
"beat_used": "Wind tugs her cardigan mid-sip; she turns her face into the light; ends still turning."
}
}

View File

@ -1,30 +0,0 @@
{
"id": "seg_three_city_wakes",
"start_frame": 480,
"end_frame": 720,
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "context",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-three-city-wakes-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "The join keeps everything it is handed: same framing, same breeze, the tail of her turn still in motion as her face comes level with the camera. She continues the turn until her eyes find the skyline, then lowers the cup in one unhurried arc to the rail, her shoulder rolling with the descent. Below, windows begin to light up floor by floor, a slow tide of warm rectangles climbing the tower across the street while a bus door hisses open. She exhales a long breath, steam curling off the cup in the same wind, ponytail still lifting, cardigan snapping softly. The camera keeps its slow lateral drift and never cuts; the city's morning roar builds under the song as her chest keeps rising and falling mid-breath.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
},
"_handoff": {
"end_state": "She rests the cup on the rail mid-exhale, chest still rising, ponytail lifted by the wind, the camera drifting left as windows keep lighting up across the city below.",
"beat_used": "She lowers the cup as windows light up below; ends mid-exhale, steam curling."
}
}

View File

@ -1,128 +0,0 @@
{
"muted": false,
"volume_db": 0,
"task_markers": [],
"task_overview": false,
"tracks": [
{
"id": "3b53d88d-07b5-5f0e-9fc0-aa87503aae80",
"name": "Tasks",
"type": "task",
"color": "var(--multitrack-task-bg)",
"muted": false,
"solo": false,
"volume_db": 0,
"locked": false,
"segments": [
{
"id": "d2c2d695-f725-56ae-9024-bf9126443256",
"start_frame": 0,
"end_frame": 240,
"color": "var(--multitrack-task-bg)",
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "shot",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-one-open-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "Dawn light spills across an empty rooftop terrace above a hushed city. A young woman with a brown ponytail, cream cardigan over a white tee and blue jeans stands at the steel rail, matching the pose reference image for her identity only. She cradles a matte-black kettle in her right hand and tips it in one slow arc; a thread of steam rises as coffee fills the white cup in her left hand, her wrist rotating smoothly, elbow dropping as the kettle empties. She sets the kettle on the rail with a soft clink, lifts the cup, and brings it halfway to her lips as the sun's first rim crests the skyline and warms her cheek. The camera holds one slow push-in toward her profile, never cutting, while pigeon wings clap once somewhere below and the kettle's metal ticks as it cools.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
}
},
{
"id": "69749e2e-3bd5-5e50-ae29-61bf607cba20",
"start_frame": 240,
"end_frame": 480,
"color": "var(--multitrack-task-bg)",
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "context",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-two-sip-wind-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "The camera holds the same framing it arrived with, and she keeps the pose, motion and light it arrived with: the cup still traveling toward her lips, steam bending in the same breeze. Her chest rises with one slow breath, weight easing onto her left hip as the cardigan's hem starts to lift. Around the two-second mark the wind strengthens into a gust that ruffles her ponytail and snaps the cardigan open a finger's width; her fingers reclose it while she turns her face slowly into the light, eyelids narrowing. Steam tears off the cup in ribbons. The camera keeps its single slow drift leftward with no cut, the city below sharpening frame by frame, traffic swelling from hum to roar, a rooftop antenna creaking softly, her cardigan snapping in steady wind as the turn continues.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
}
},
{
"id": "6d0f444e-c6b1-5d14-a3c6-a29cd594beb7",
"start_frame": 480,
"end_frame": 720,
"color": "var(--multitrack-task-bg)",
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "context",
"ref_image_size": "match",
"images": [
{
"id": "img-seg-three-city-wakes-1",
"source_type": "input",
"file_path": "codex/rooftop/pose_ref.png",
"file_name": "pose_ref.png",
"shared_reference": false
}
],
"user_prompt": "The join keeps everything it is handed: same framing, same breeze, the tail of her turn still in motion as her face comes level with the camera. She continues the turn until her eyes find the skyline, then lowers the cup in one unhurried arc to the rail, her shoulder rolling with the descent. Below, windows begin to light up floor by floor, a slow tide of warm rectangles climbing the tower across the street while a bus door hisses open. She exhales a long breath, steam curling off the cup in the same wind, ponytail still lifting, cardigan snapping softly. The camera keeps its slow lateral drift and never cuts; the city's morning roar builds under the song as her chest keeps rising and falling mid-breath.",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
}
}
]
},
{
"id": "51945496-eaa4-5ba0-bb78-201bbfee5819",
"name": "Master Song",
"type": "audio",
"color": "var(--primary)",
"muted": false,
"solo": false,
"volume_db": 0,
"locked": false,
"audio_locked": true,
"segments": [
{
"id": "ee335ed8-75cd-5278-969f-144cba015861",
"start_frame": 0,
"end_frame": 720,
"color": "var(--primary)",
"content": {
"media_type": "audio",
"source_type": "input",
"file_path": "codex/rooftop/song.wav",
"file_name": "song.wav"
}
}
]
}
],
"total_length": 720,
"frame_rate": 24
}

View File

@ -1,518 +0,0 @@
# MultiTrack Data and Orchestration Rules (English reference)
English reference for the `easy multiTrackEditor` TRACK_DATA and the patch plan, translated
from the node pack's own skill doc (`skills/easy-media-multitrack-workflow/references/multitrack-schema.md`
in ComfyUI-Easy-Media, which is written in Chinese). Front-end types live in
`frontend/src/types/multitrack.ts`; execution logic lives in `nodes/basic.py` of the node pack.
Field names are preserved exactly as the schema spells them.
## Resolution (DynamicCombo)
`resolution` is a DynamicCombo widget of `easy multiTrackEditor`. It is NOT part of
TRACK_DATA. Changing it requires rebuilding the sub-widgets for the chosen mode and syncing
the workflow's existing `widgets_values_named`. Write it through `patch_workflow.py`'s
`editor.resolution` — never by guessing array positions.
| Need | `resolution` label | Required sub-fields |
| --- | --- | --- |
| Fixed preset output size | e.g. `1344 x 768 (16:9)` | optional `resize_method`, default `stretch` |
| Explicit custom size | `width x height (custom)` | `width`, `height`, optional `resize_method` |
| Follow first video's ratio | `width x height (auto)` | optional `resize_method` |
| Fixed short/long edge | `width x height (shortest)` / `width x height (longest)` | `resize_to_pixel`, optional `resize_method` |
| Ratio + total pixel budget | `width x height (megapixels)` | `aspect_ratio`, `megapixels` |
Selection order: exact width/height from the user → an existing fixed preset, else custom;
only orientation or aspect + quality/pixel budget given → megapixels; keep source video
ratio → auto; "short/long edge = N" → shortest/longest. If the user expresses no resolution
intent, keep the current setting. The model format's default size is only a reference and
never overrides an explicit user requirement.
### MP and ratio in natural language
Case-insensitive `mp`, `Mpx`, `megapixel(s)` (and Chinese "百万像素") mean the megapixels
unit. `0.9mp`, `1 MP`, `0.8 megapixels` select megapixels mode and the number is taken
verbatim. A ratio in the same sentence maps to the full DynamicCombo `aspect_ratio` value:
| User ratio | `aspect_ratio` |
| --- | --- |
| `1:1` | `1:1 (Square)` |
| `2:3` | `2:3 (Portrait Photo)` |
| `3:2` | `3:2 (Photo)` |
| `3:4` | `3:4 (Portrait Standard)` |
| `4:3` | `4:3 (Standard)` |
| `9:16` | `9:16 (Portrait Widescreen)` |
| `16:9` | `16:9 (Widescreen)` |
| `21:9` | `21:9 (Ultrawide)` |
Example: "generate a 16:9 0.9mp video" normalizes with no follow-up question to:
```json
{
"resolution": "width x height (megapixels)",
"aspect_ratio": "16:9 (Widescreen)",
"megapixels": 0.9
}
```
Never convert the MP number into width/height and never round it. MP only: if already in
megapixels mode keep its `aspect_ratio`; else pick `16:9`/`9:16` from explicit
landscape/portrait wording; still unclear → ask. Contradictory exact size, ratio and MP →
ask. Custom example:
```json
{
"resolution": "width x height (custom)",
"width": 1024,
"height": 576,
"resize_method": "crop"
}
```
Valid `resize_method`: `stretch`, `resize`, `pad`, `pad (white)`, `pad_edge`,
`pad_edge_pixel`, `crop`, `pillarbox_blur`. Prefer pad/resize to keep the whole picture,
crop to fill the canvas; never change an existing resize method the user did not mention.
## Time semantics
- `start_frame` is inclusive, `end_frame` exclusive; segment frames = `end_frame - start_frame`.
- Seconds → frames: `round(seconds * frame_rate)`; every segment has at least 1 frame.
- Segments in one track are sorted by `start_frame` and never overlap; gaps are allowed.
Different tracks of the same kind schedule independently.
- With segments present, `total_length` should be the maximum `end_frame` over all tracks.
With no segments the editor's current default length may stay.
- Changing FPS must re-map every segment boundary, `origin_start_frame` and task marker
proportionally — not just `frame_rate`.
## Top-level TRACK_DATA object
```json
{
"muted": false,
"volume_db": 0,
"task_markers": [],
"task_overview": false,
"tracks": [],
"total_length": 240,
"frame_rate": 24
}
```
Track `type` is one of `task`, `video`, `audio`, `subtitle`. Keep unknown fields for
forward compatibility.
## Common track and segment shape
```json
{
"id": "uuid",
"name": "Video 0",
"type": "video",
"color": "var(--primary)",
"muted": false,
"solo": false,
"volume_db": 0,
"locked": false,
"segments": [
{
"id": "uuid",
"start_frame": 0,
"end_frame": 120,
"color": "var(--primary)",
"content": {}
}
]
}
```
New tracks use the project's existing color tokens: task tracks `var(--multitrack-task-bg)`,
video tracks `var(--primary)`. Keep an existing track's own color.
## Task segment content
```json
{
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "shot",
"ref_image_size": "match",
"images": [],
"user_prompt": "the user prompt",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
}
```
- `task_mode`: `default`, `l2v`, `ref`, `edit`.
- `continuity_mode`: `shot` (standalone shot), `context` (normal context join) or
`context_drift` (context drift control); the legacy value `context_swap` is still read.
- `ref_image_size`: `match` or `max`.
- `images` holds at most 9 items; their order defines the picture-reference order.
- When editing an existing task keep its `system_prompt`, prompt variant, continuity and
reference-image policy unless the user asked to change them.
- If the current variant is `b`, edit `user_prompt_b`; otherwise edit `user_prompt`. Never
switch the variant unintentionally.
Task reference image item:
```json
{
"id": "uuid",
"source_type": "input",
"file_path": "codex/session-123/reference.png",
"file_name": "reference.png"
}
```
`source_type` is one of `input`, `output`, `local`, `url`, `slot`. The path field must
match the source: `input`/`output` use `file_path`, `local` uses `local_path`, `url` uses
`url`, `slot` uses `slot_name`. Never label a local absolute path as `input`.
When several context segments reuse the same reference image, an image item may set
`"shared_reference": true`; it becomes a common image that is automatically ordered before
each task's private images; common + private images still total at most 9 per task. The
current front-end `synchronizeSharedTaskImages` generates new UUIDs for missing common-image
metadata on later tasks, so repeatedly normalizing the same sparse raw data yields different
IDs; from this code alone the full React maximum-update-depth / widget-unmount trigger chain
cannot be confirmed. Until the target front-end is verified to load such data stably, store
an explicit private image item per task (`shared_reference: false`) with a fixed ID pointing
at the same uploaded file — no duplication or re-upload needed. Switching continuity mode
does not fix this.
## Context drift control mode
`context_drift` is the "context drift control" join: it carries the previous segment's scene
and motion context while letting the model replace subjects, people, objects or局部 regions
from a reference video. When the user supplies a video as a subject/person/object/clothing/face
replacement reference, orchestrate as follows:
- Set `content.continuity_mode` to `context_drift` on ALL task segments in scope — not only
the second and later ones. The first segment keeps the same mode too (it has no previous
generated context, but consistency matters for project metadata and re-generation).
- Put the reference video on a video track covering the target task segments that need motion
reference. If the project also has passages that do not depend on the dance/reference video
(e.g. a verse), keep one complete project — the full locked audio covers all tasks and the
video only covers its part. If the video is also needed across tasks as a model reference,
set `shared_reference: true` on the video segment's `content`.
- Set track-level `audio_locked: true` on that video track so the source video's timeline
drives generation, and explicitly set `audio_locked: false` on every OTHER video track. To
replace the original sound, lock a separate audio track, which then wins as the audio
source. When only reusing the video's own sound, confirm before locking that the video
actually contains a usable audio track and covers all target task segments.
- If the user pinned segment lengths or boundaries, the user wins. Otherwise split task
segments into consecutive 10-second pieces: each piece is `round(10 * frame_rate)` frames
and the last piece takes the reference video's remaining frames — do not pad, do not drop
the remainder, do not create an empty tail segment. The reference video itself can stay a
single video segment covering the whole range; it does not need to be duplicated per task
segment.
- If the user explicitly asked to drop/replace/not keep the reference video's original sound,
do not auto-lock that video track and follow the user's audio plan instead.
## Context segment prompts
Applies to tasks with `task_mode: "ref"` and `continuity_mode: "context"`;
`context_drift` (and legacy `context_swap`) do not use this section. Such a segment starts at
the previous segment's actually delivered tail (22 frames ≈ 0.92 s at 24 fps) whose framing
and camera motion may differ from what the previous prompt imagined — and those frames are
locked: text contradicting them tends to render as an overlay, not a replacement. When
writing or editing `user_prompt`:
- **Keep first, change later**: open with only "the camera keeps the direction and framing it
arrived with; the subject keeps the pose, motion and light it arrived with". Never describe
the previous segment's INTENDED ending as fact, and never open with a new composition,
character layout or scene.
- **Give the hold an action**: give the subject a breath, a weight shift or a gaze change so
the hold does not render as a freeze-frame.
- **Changes go later**: before about `00:02.000` no cut, no spoken line, no new key action;
write a required composition or scene change as an explicit cut.
- **Start actions mid-motion**: things already in progress at the seam start in motion —
never as if completing on frame one.
- **Self-contained**: the model never sees the previous prompt; fully define every `<Subject N>`,
state scene and light, and never say "the same one" or "as before" for identity or scene.
- **Images define identity only**: a reference image is not the first frame or a composition
anchor of `[Shot 1]`; context joins use neither image anchoring nor `[video continuation]`.
- **When more `context` segments follow, end in motion**: never write "ends", "arrives",
"settles", "centers", "slows down"; no cut and no new spoken line in the last second.
- **Need a clearly new composition**: use a `shot` segment with keyframes instead of `context`.
- **Timestamps**: timestamps in the prompt map onto the generated segment, whose start sits
about 0.92 s earlier than the saved segment; the node does not shift them automatically.
When `system_prompt` is empty, the task output node automatically appends the same rules to
the default MiniMax reference system prompt (entry rule: this segment is not the first and
uses `context`; closing rule: the next segment uses `context`). The rules reach the rewriting
LLM only through the `api` and `llm` prompt formats; `default` and `promptRelay` emit no
system prompt. A custom `system_prompt` disables the auto-append — write the rules yourself.
## Video and audio segment content
```json
{
"media_type": "video",
"source_type": "input",
"file_path": "codex/session-123/clip.mp4",
"file_name": "clip.mp4",
"duration": 5.0,
"muted": false,
"volume_db": 0,
"speed": 1
}
```
Audio sets `media_type` to `audio`. Get source durations with `ffprobe`; the timeline
duration follows the user's target, trimming and speed. If media must align with a task
segment, reuse the exact same `[start_frame, end_frame)` — never just similar seconds.
## Audio lock decisions
`audio_locked` is a track-level field on audio/video tracks. Locking an audio track keeps
its audio; locking a video track makes generation follow that video track's timeline and,
with no locked audio track, keeps the video's own sound. Both locks can coexist; the audio
track then wins as the `audio_lock` audio source while the video track still constrains the
picture timeline. It is different from the track's `locked` flag: `locked` only blocks
timeline editing and cannot substitute for `audio_locked`.
```json
{
"id": "music-track",
"name": "Master Song",
"type": "audio",
"audio_locked": true,
"muted": false,
"segments": [
{
"id": "song-segment",
"start_frame": 0,
"end_frame": 240,
"content": {
"media_type": "audio",
"source_type": "input",
"file_path": "codex/mv/song.wav",
"file_name": "song.wav"
}
}
]
}
```
A complete locked audio track can coexist with a partial locked video track. At runtime, when
the current task has no intersection with the video, the video is simply not passed in and
the master audio is still used — never split the project, extend the video, or stretch audio
just to satisfy validation. Validation allows a partial video lock only when the locked audio
continuously covers the selected tasks (multiple end-to-end audio segments are fine); without
a complete master audio it still checks the locked media's intersection with each task.
One TRACK_DATA may contain at most one `audio_locked: true` audio track and one
`audio_locked: true` video track. When switching the locked source of one type, explicitly set
the other tracks of that same type to `false`; never clear the other type's lock. Never write
`audio_locked` into a media segment's `content`; the legacy segment-level field only exists
for migration.
Decide in this order, from context:
1. The user supplies a video for subject/partial replacement and did not ask to remove or
replace its sound → lock that video track.
2. The user explicitly says "lock", "master audio", "keep the original sound", "don't
regenerate audio" → use the audio/video track they mean.
3. MV, lyric video, beat-sync video, dance video, or visuals generated to a full song's beat
and length → lock the song/score master track.
4. Narration, podcast or dialogue is the only basis for video length and shot rhythm → that
narration master track may be locked.
5. Ordinary background music, ambience, SFX, character reference audio or temporary scratch
audio is NOT auto-treated as locked, unless the user's wording makes it the generation
driver.
6. One locked media track already exists and the user did not ask to change it → prefer
keeping it; still check it intersects the task range being generated.
With multiple audio tracks, disambiguate first by name, file name, source duration, track
content and the user's description — e.g. `Master Song` or 完整歌曲 (full song) beats `SFX` or
ambience. If two or more plausible candidates remain, STOP and ask the user, listing each
candidate's track ID, name, file name and time range. Never decide by track order, first
audio track or longest duration.
Before locking also check:
- The locked track contains at least one media segment matching its type: audio track →
`media_type: "audio"`, video track → `media_type: "video"`; when locking a video's own
sound, confirm the source video really has an audio track.
- The locked media segments intersect every target task segment's `[start_frame, end_frame)`;
if not, adjust the timeline or ask first.
- As master audio it normally stays `muted: false`. Never touch unrelated solo, volume or
edit-lock state just "because locking".
- If the context clearly wants no original audio kept, ensure every audio/video track has
`audio_locked: false`.
## Shared media
`shared_reference` is a boolean on task image items and in audio/video segment `content`.
When several context task segments need the same image, audio clip or video as a reference,
mark that media shared; it does NOT need to overlap the task segments on the timeline.
```json
{
"id": "shared-audio-segment",
"start_frame": 0,
"end_frame": 120,
"content": {
"media_type": "audio",
"source_type": "input",
"file_path": "codex/voice/reference.wav",
"file_name": "reference.wav",
"shared_reference": true
}
}
```
Decision rules:
- Mark media shared only when the user says "make it shared/common media", "all context
segments reference this image/audio/video" or equivalent. Never auto-share just because
file names match.
- Task images are marked on their `content.images[]` items; several shared images are
allowed; they are ordered before each task's private images and common + private still
total at most 9 per task.
- Audio/video are marked on the media segment's `content`; at most one per audio/video
track; when switching, explicitly set the track's other segments to `false`.
- Shared audio is used from the start of the source file, at most 15 seconds; shared video
uses that source video as each task's reference video.
- Shared media is a cross-task reference, not timeline-audio reuse. Never set `audio_locked`
just because audio is shared; evaluate `audio_locked` separately, only when the user also
wants the audio kept as-is and driving the timeline.
- If the user wants one audio/video clip BOTH as a shared reference AND as normal timeline
output, express the two roles with a separate track or segment and say so in the delivery
summary.
- `speaker_reference` is the legacy audio-reference field, only for reading and migrating old
workflows; new or updated workflows always write `shared_reference`.
- With multiple unresolvable candidates, list the media/segment IDs, track names, file names,
durations or frame ranges and ask the user; never pick the first or the longest.
## Auto-orchestration decisions
- User-given shot/segment lengths win; otherwise derive from source durations or the
prompts' timing information, and state the derivation.
- Subject/partial video replacement mode with no user-specified lengths always splits task
segments into consecutive 10-second pieces, the last taking the remainder.
- "N seconds each" means consecutive `[0, N*fps)`, `[N*fps, 2N*fps)`, … — never add one extra
frame at the boundaries.
- Prompts go into their task segments; reference images into that task segment's `images`;
master video and audio into their own media tracks.
- For subject-replacement, MV or other master-audio-driven projects, after arranging the
timeline lock the single audio/video track and re-check it covers the target task range.
- Keep gaps between tracks by default; close gaps and shift later segments only when the
user asks for a tight arrangement.
- When adjusting existing segments keep their segment IDs; generate new UUIDs only for new
segments, tracks and image items.
## MultiTrack Project parameters
- `segment_start_number` is the 1-based first task segment index; execution converts with
`segment_start_number - 1`.
- `segment_count = -1` processes from the start index to the end; a non-negative value limits
how many task segments run this time.
- `project_save`: `new` or `override`. With `override` and count `-1`, execution deletes
saved segments after the start index before generating — say so clearly before changing it.
- `sampling_mode`: `single` or `dual`; `sampling_plan` must be a preset available on the
current node — never assume values from another workflow's environment.
- The editor resolution is the first-pass size; `dual` never scales the first pass down. The
second pass is `round(editor_size * upscale_by / 32) * 32` per dimension, Python-round to
nearest, ties to even — not a fixed ceiling. `upscale_by` is an independent Project
parameter, default `1.250`, three decimals, step `0.001`; old workflows keep their saved
value. Both upsample paths and the merged export use the upscaled, aligned size. Example:
`1344 x 768` at `1.250` gives a second pass of `1664 x 960`. If the user specifies a final
output size, check BOTH the editor size and the multiplier; the editor size alone is not
the dual-sampling final size.
- Single sampling / first-pass-only preview keeps the editor size; `upscale_by = 1.000`
upscales nothing but still allows a second pass; pure audio projects (`32 x 32`) do not
upscale. Old first-pass checkpoints and context latents that were shrunk by the old
back-calculation should be regenerated before continuing under the new rule.
- `1st_pass_only` only applies to explicit one-stage checkpoint flows; never enable it
automatically.
## The patch plan
The plan contains only actual changes. `node_id` may be omitted (topology locates the nodes);
`editor.resolution` is the complete DynamicCombo setting; `editor.track_data` is the complete
TRACK_DATA. Never put a whole node or workflow into the plan.
```json
{
"editor": {
"resolution": {
"resolution": "width x height (megapixels)",
"aspect_ratio": "16:9 (Widescreen)",
"megapixels": 0.9
},
"track_data": { "tracks": [], "total_length": 120, "frame_rate": 24 },
"format": "MiniMax"
},
"project": {
"project_save": "override",
"segment_start_number": 1,
"segment_count": -1,
"sampling_plan": "medium",
"sampling_mode": "single"
}
}
```
Applied with the node pack's own script:
```sh
python3 scripts/patch_workflow.py apply SOURCE.json --plan PLAN.json # dry-run
python3 scripts/patch_workflow.py apply SOURCE.json --plan PLAN.json --output RESULT.json --write
python3 scripts/patch_workflow.py inspect RESULT.json
```
---
## prompt_override — the paste payload (format MiniMax)
The `easy multiTrackEditor` node has an optional `prompt_override` input. When the editor's
`format` is `MiniMax` and the pasted value is a `minimax_prompt_override` JSON payload, the
editor rebuilds its whole task track from that payload: one task segment per `prompts`
entry, laid back-to-back, each with its own duration. `total_length` becomes the sum of the
durations, so a 3-prompt payload at 10s each renders as THREE chained ~10-second clips.
```json
{"type":"minimax_prompt_override","version":1,
"prompts":["<Picture 1> <Audio 1> prompt for segment one ...",
"<Picture 1> <Audio 1> prompt for segment two ..."],
"duration":"10","generation_type":"r2v","continuity_mode":"shot,context,context",
"system_prompt":"","video_track_lock":0,"audio_track_lock":1}
```
Field rules (mirrors `utils/prompt_override.py` in the node pack):
- `type` must be exactly `minimax_prompt_override` and `version` `1`. **Any other JSON
pasted here is not rejected — it falls back to the generic text parser that splits on
`|`.** A JSON blob contains no `|`, so it becomes ONE task segment spanning the whole
timeline with the JSON dump as its prompt: the classic "rendered the whole story as a
single 10 second clip" failure.
- `prompts` — one entry per clip, in timeline order. The prompt text is the clip's full
generation prompt (same standalone composition rules as task segments: no cross-segment
references, context joins keep-then-change, only the last clip may come to rest).
- Media attaches ONLY through tokens in the prompt text: `<Picture N>`, `<Audio N>`,
`<Video N>` (the `@imageN`/`@audioN`/`@videoN` forms are accepted and rewritten, but this
gate standardizes on the bracket form). N is the 1-based position of the item in the
editor's matching `image` / `audio` / `video` input list — wire the media inventory into
those inputs in inventory order. A prompt with no token renders with no reference media.
The runtime renumbers tokens compactly for display, but slots are wired from the raw N.
- `duration` — seconds per clip, single value ("10" applies to every clip) or comma list
("10,5,10") with tail reuse; each value must be 2–15, enforced at parse time.
- `generation_type` — per clip, same single/comma-list rule. `r2v` = reference-image task
(task_mode ref), `l2v` = last-frame task, `t2v`/`i2v`/`v2v` land as default/edit tasks.
This gate emits `r2v` for ref rows, `l2v` for l2v rows, `t2v` for default rows.
- `continuity_mode` — per clip: `shot`, `context`, `context_drift` (legacy `context_swap`
is normalized to `context_drift`). Matches the task-segment semantics.
- `system_prompt` — written into every clip when non-empty; an empty string lets the H3
runtime auto-append its reference rules for context joins (the gate's default).
- `video_track_lock` / `audio_track_lock` — 1-based index of the video/audio track to lock
(0 = none). With a locked master-audio inventory row, every prompt carries
`<Audio 1>` and `audio_track_lock` is 1; with a locked reference video, `<Video 1>` and
`video_track_lock` are 1.
The paste replaces the editor's existing task track (and derives its audio/video tracks
from the tokens), so the payload is self-contained: prompts, durations, modes and media
references all travel inside it. Edit task modes (`edit`) and full extra tracks are beyond
this payload — use the TRACK_DATA patch plan for those.

View File

@ -1,126 +0,0 @@
# Workflow wiring: render ALL scenes in one run + the LLM slot
Verified against ComfyUI-Easy-Media source (`nodes/basic.py`, `nodes/minimax.py`,
`nodes/project.py`, `utils/llm_api.py`) and `example_workflows/MinimaxH3_ForLoop_MultiShot.json`.
## Why only scene 1 renders
The render chain reads the timeline through `easy multiTrackTaskOutput`, which outputs
**one task segment per pass**, picked by its `task_index` int input. Default `0` →
the first clip only. The number of prompts inside the pasted `prompt_override` JSON
does not change this: the JSON is complete (all scenes are in it); the loop lives in
the workflow, not in the JSON. Nothing needs to be added to the payload for multi-shot.
Side facts (same node, verified in source):
- `task_index = -1` outputs the **whole timeline as one merged segment** (all task
contents joined, full frame length). It is for relay/audio-lock cases, NOT a way to
render 3 clips: the local H3 model is trained at ~124–362 frames (~5–15 s); a 30 s
merged pass is out of range.
- `easy multiTrackInfoOutput` (tracks_info in) exposes `TASK_COUNT` = number of task
segments — 3 for the Sam payload. Also WIDTH/HEIGHT/TOTAL_FRAMES/FPS.
## The single-run loop (from the pack's own ForLoop example)
One queue run renders every clip and concatenates them:
```
editor.TRACKS_INFO ─→ multiTrackInfoOutput ─→ TASK_COUNT ─→ forLoopStart.total
editor.TRACKS_INFO ─→ multiTrackTaskOutput (images/audio/video too)
forLoopStart.index ─→ multiTrackTaskOutput.task_index
multiTrackTaskOutput.SYSTEM_PROMPT/USER_PROMPT/TYPE/LENGTH/IMAGES/AUDIO ─→ render chain
render chain → saveVideo.file_path ─→ batchAnything ─→ forLoopEnd.initial_value1
forLoopEnd.value1 ─→ mergeVideosFromPaths ─→ SaveVideo
```
The example adds two math nodes so a single PrimitiveInt `k` (default `-1`) chooses:
- `total = (k<=-1)*TASK_COUNT + (k>-1)*1`
- `task_index = (k<=-1)*loop_index + (k>-1)*k`
`k = -1` → loop all segments (all scenes, one run). `k = 2` → render only segment 2.
Manual fallback without rewiring: queue once per index (0, 1, 2) and join the files.
Per-iteration the task output gives LENGTH ≈ frames for that clip only (240 frames =
10 s — inside the model's trained range), so each loop pass is a normal single-clip
generation.
## The LLM slot (where your own LLM connects)
In the ForLoop example the per-scene prompt may pass through an LLM before rendering:
```
TaskOutput.SYSTEM_PROMPT ─→ [LLM node].system_prompt
TaskOutput.USER_PROMPT ─→ [LLM node].custom_prompt
TaskOutput.IMAGES ─→ [LLM node].images (optional, vision only)
[LLM node].output ─→ ifElse("Using Prompt Enhance") ─→ cleanGpuUsed ─→ render.prompt
ifElse.on_false = raw TaskOutput.USER_PROMPT (bypass switch)
```
The example's LLM is `llama_cpp_instruct_adv` + `llama_cpp_model_loader` (a local GGUF).
That node is replaceable — it is just "system+user (+images) in, one string out".
Three ways to fill the slot:
### Option A — no LLM node at all (default for this skill)
The skill already wrote every scene prompt (identity text, continuity rules, LoRA
trigger) into `prompt_override.json`. Leave the enhance toggle OFF: raw
`USER_PROMPT` flows to the render node each pass. The "LLM that made all the scenes"
is this skill, outside ComfyUI.
### Option B — the pack's built-in enhancer with any OpenAI-compatible server
Add `easy multiTrackPromptEnhancer` inside the loop:
- `system_prompt` ← TaskOutput.SYSTEM_PROMPT, `user_prompt` ← TaskOutput.USER_PROMPT,
`type` ← TaskOutput.TYPE, `length` ← TaskOutput.LENGTH, images/audio/video optional.
- `model` = `openai_compatible (local)`; set `base_url` to ANY OpenAI-compatible
Chat Completions server — Ollama `http://127.0.0.1:11434/v1`, LM Studio
`http://127.0.0.1:1234/v1`, vLLM, llama.cpp server, or a hosted endpoint
(empty base_url reads `OPENAI_LOCAL_BASE_URL` from the pack's config.yaml, else
`http://127.0.0.1:8080/v1`). `apikey`/`model_name` optional depending on server.
Built-in presets also include MiniMax H3-Context-IR (`v2/h3_context_ir`),
bigmodel.cn (GLM) and RunningHub LLM keys from config.yaml.
- Output `PROMPT` → render node's prompt.
### Option C — any ComfyUI LLM node in the loop
Keep the example's wiring shape and swap node 86 for the preferred LLM node
(system_prompt + user_prompt inputs, STRING output). `prompt_format` on TaskOutput
can stay `api`; setting it to `llm` instead makes USER_PROMPT a single ready-made
chat transcript for LLM nodes that have only one prompt input.
## Local project path (alternative, no loop nodes)
`easy multitrackProject` (editor tracks_info in, `segment_count = -1`) builds and runs
every segment itself in one queue — see `MiniMaxH3_v1.3.1_Easy_Project_AllInOne.json`.
It contains no LLM node; prompts come from the timeline JSON. This is the pack's own
agent-driven setup: an external LLM (agent) writes the whole `track_data`, then one
run renders it. The pack even ships `skills/easy-media-multitrack-workflow` +
`patch_workflow.py` for exactly that flow (our SKILL.md step 5B).
## The built-in LLM system prompts (published, not locked)
The rewrite-LLM contract ships in the open in the pack:
`utils/prompt_builder.py` — `MINIMAX_BASE_PROMPT` (~15.7 KB, "Video Prompt Writing
Guide (T2VA / I2VA / FL2VA / L2VA)") and `MINIMAX_REF_PROMPT` (~23.5 KB, "Full-
Reference Mode Rewrite Output Format Guide" — six-section rewrite with
`<Subject N>` / `<Picture N>` / `<Video N>` / `<Audio N>` labels).
Selection (`build_prompt_request`, MiniMax branch):
- `task_mode` `default` / `l2v` (our t2v payloads) → `MINIMAX_BASE_PROMPT`
- `task_mode` `ref` / `edit` → `MINIMAX_REF_PROMPT`
- a non-empty per-segment `system_prompt` overrides both; with an empty one the
context-segment continuity rules ride the MiniMax rewrite API server-side
(entry rule: not-first segment using `context`; tail rule: next segment uses
`context`) — per the pack's own `multitrack-schema.md`.
Delivery: the system prompt only leaves TaskOutput via `prompt_format`
`api` or `llm` (`default` / `promptRelay` output an empty SYSTEM_PROMPT).
`build_llm_prompt` folds system+user into one transcript for single-input LLM
nodes. To use the user's own LLM with the same behavior: paste
`minimax_base_prompt.txt` (or the ref guide for ref-mode work) into the LLM
node's system prompt field and wire TaskOutput SYSTEM_PROMPT/USER_PROMPT into
it — or leave the LLM's system prompt empty and pass TaskOutput.SYSTEM_PROMPT
through, which already carries the selected guide verbatim.

File diff suppressed because it is too large Load Diff

View File

@ -224,15 +224,6 @@ system prompt emits. Report the delivered frames/seconds it prints.
## Step 5 — Deliver (the node makes the multi-clip video)
**This plan targets the standalone MiniMax H3 Context Loop pack only.** ComfyUI-Easy-Media
(yolain) ships its own H3 node family built on the MultiTrack system and has NO
`plan_json`/`shots` consumer — a chain plan pasted into an Easy-Media workflow is read by
no node, and the workflow's default single task segment (0-240 frames = exactly 10 s at
24 fps) renders the whole story as one 10-second clip. For that pack use the
`easy-media-track-gate` skill instead: one task segment per shot in the multiTrackEditor
TRACK_DATA, a for-loop rendering each segment as its own H3 generation, and
mergeVideosFromPaths stitching the clips.
Verified against the node pack source (`chain_nodes.py`, `MiniMaxH3ChainPlan` /
`MiniMaxH3ChainPlanModern`): the parser is a plain `json.loads` over the effective
plan string — a non-empty STRING connected to the forced `plan_json_input` socket

View File

@ -133,7 +133,7 @@ a an the and or but nor so yet if then else as is are was were be been being am
do does did done have has had having will would shall should can could may might
must of to in on at by for from into onto over under with without within near
past through during before after above below between against along across behind
beyond than too very just even also still already always never only own same all like
beyond than too very just even also still already always never only own same all
any both each every few more most other some such no not nor while when where how
why who whom whose which what that this these those there here now then once
twice s t ok up down out off again further