The paste payload builds the full task timeline, but easy multiTrackTaskOutput emits ONE task segment per pass, selected by its task_index int (default 0 = first clip only). A single-segment workflow therefore renders only the first scene no matter how many prompts the override carries. SKILL.md step 5A and the builder's delivery print now say so, point to the pack's MinimaxH3_ForLoop_MultiShot.json example (TASK_COUNT -> loop total, loop index -> task_index, mergeVideosFromPaths after the loop) and offer the manual task_index 0..N-1 queue fallback.
24 KiB
Executable File
| name | description |
|---|---|
| easy-media-track-gate | Turn a video idea into a validated MultiTrack plan for ComfyUI-Easy-Media (the `easy multiTrackEditor` node). Use when the user wants to plan task segments/shots on the multitrack timeline, write or extend a MultiTrack timeline, arrange prompts, reference images, master audio or reference-video locks, regenerate one task segment, or produce the `prompt_override` paste payload for the multitrack editor node (plus the optional patch plan JSON that `patch_workflow.py apply` consumes). The scripts and references live next to this SKILL.md (the skill's install folder); run them from the user's working folder, where all generated files are written. |
Easy-Media Track Gate — agent procedure
Goal: produce a validated plan for the easy multiTrackEditor node in five moves: the user
gives a video idea and a segment count (or an existing workflow to edit) → you build a
timeline guide (outline.json) and show it for approval → with the approved guide you
compose the task segments one at a time (about 10 seconds each), each turn existing so that
ONE segment gets a complete prompt, the right references and the right continuity mode →
the scripts verify schema, timing and continuity across the whole chain →
build_track_plan.py assembles the deliverables at the end. The PRIMARY deliverable is
prompt_override.json, a minimax_prompt_override payload the user pastes into the
multitrack editor node's prompt_override input (format MiniMax) — the editor then
renders one clip per prompt, chained back-to-back. The patch plan (plan.json for the node
pack's patch_workflow.py) is the optional whole-workflow alternative. Never deliver a plan
that did not pass validation, and never compose a segment from a guide the user has not
approved. Never hand the user any other JSON as the paste: a non-minimax_prompt_override
value in prompt_override falls back to the editor's |-separated text parser and renders
the whole story as ONE clip over the entire timeline.
Paths and Python (read first — these exact forms prevent most errors)
- SKILL_DIR is the folder this SKILL.md was loaded from — the skill's install folder
(default on Windows:
%userprofile%\.agent\skills\easy-media-track-gate; macOS/Linux:~/.agent/skills/easy-media-track-gate). The scripts, the schema reference and the worked example all live in SKILL_DIR, not in the working folder. Every command below uses the Windows default install path; substitute your real SKILL_DIR if it differs. - Work in the user's working folder (the folder open in the editor). Write generated
files there:
outline.json,segment_01.json,segment_02.json, …,plan.json,track_data.json. Never write generated files into SKILL_DIR. - Python command: Windows
py -3(orpython); macOS/Linuxpython3. If the working folder has a.venv, its interpreter works too (.venv\Scripts\python.exe). The scripts need only the Python standard library — nothing to install. - Start every task by running the smoke test under "Before you start". If it fails because SKILL_DIR is missing files, stop and tell the user the skill folder is incomplete.
- Never read
verify_track_rules.py— run it and work from its printed output. Cline's read_file is capped by hard-coded budgets that do NOT grow with the model's context window: about 48,000 characters per read, 2,000 lines per read. The 81 KB validator arrives cut no matter how large the context window is. A truncation notice is Cline's read cap, not repo corruption — do not "fix" truncated files.build_track_plan.py(~10 KB) andreferences/track-schema.md(~26 KB) are safe to read whole. - Never read back
plan.json,track_data.jsonorprompt_override.jsononce real segments exist — a 10-segment plan passes the read caps today and exceeds them tomorrow. Validate with--check/ theoverridecommand and trust the one-line OK summary; the shippedexamples/prompt_override.json(2.5 KB) andexamples/plan.json(6.8 KB) are safe to read if you need to see the shape. - Cline's editor rejects writes over 6,000 characters ("Editor input too large"). A segment
fragment with a rich prompt often does. NEVER write part 2 with write_to_file — it
replaces the whole file and part 1 is lost. Split instead: write the first part ending
with the bare marker line
@@CONTINUE@@, then replace_in_file with SEARCH@@CONTINUE@@and REPLACE the remaining content. The finished file must not contain the marker. - Cline also cuts command output at 8,000 characters. The embedded suite prints well under
1 KB when it passes; a failing validation prints one actionable line. If output ever
arrives truncated, re-run the single-file command (
segment,outline,plan) — its output is tiny.
Before you start (smoke test)
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\plan.json"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\prompt_override.json" --outline "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\outline.json"
Expect the suite to print SUITE PASS (34/34 cases) and the other two commands to print
PLAN OK with total_length 720 frames = 30.00s and OVERRIDE OK: 3 prompts, duration 10, total 30.00s.
Step 0 — Media paths come first (never invent one)
Every file_path in the plan must be a real path on the user's ComfyUI instance. Before
writing the outline, establish the media inventory one of two ways:
- The user names the exact input paths (they know their
input/subfolders). Use their strings verbatim — they become the pinned inventory. - The user explicitly asks you to upload → upload first and record the returned relative
path (
subfolder/name,source_type: "input") for every file. Upload uses the node pack's own API (POST /upload/imagewith asubfolderlikecodex/<project>); never write a local absolute path into the plan. - If there is no real path and no upload request, stop and ask. A plan is still deliverable with obviously-fake placeholder paths only if you say plainly it cannot run until real media replaces them — never dress a placeholder up as a real path.
Upload and execution are external state changes: only upload/queue when the user explicitly asked (Step 5).
Step 1 — Build the timeline guide (outline)
Read SKILL_DIR's references/track-schema.md once (safe whole-file read; it is the English
translation of the node pack's Chinese schema doc). Then write outline.json — the guide
for the whole timeline — in exactly this shape (the shipped examples/outline.json is the
worked model):
{
"title": "short_snake_case_name",
"lengths_user_pinned": false,
"editor": {
"format": "MiniMax",
"resolution": { "resolution": "width x height (megapixels)",
"aspect_ratio": "16:9 (Widescreen)", "megapixels": 0.8 }
},
"defaults": { "frame_rate": 24, "segment_seconds": 10 },
"media_inventory": [
{ "key": "SONG", "kind": "audio", "source_type": "input",
"file_path": "codex/rooftop/song.wav", "file_name": "song.wav" },
{ "key": "POSE_REF", "kind": "image", "source_type": "input",
"file_path": "codex/rooftop/pose_ref.png", "file_name": "pose_ref.png" }
],
"tracks": {
"master_audio": { "key": "SONG", "lock": true, "name": "Master Song", "role": "why it drives the timeline" },
"reference_video": null
},
"project": { "project_save": "override", "segment_start_number": 1,
"segment_count": -1, "sampling_plan": "medium", "sampling_mode": "single" },
"segments": [
{ "index": 1, "id": "seg_one_open", "start_seconds": 0, "duration_seconds": 10,
"task_mode": "ref", "continuity_mode": "shot",
"images": [ { "key": "POSE_REF", "shared_reference": false } ],
"beat": "one sentence: what happens AND where the segment ends",
"prompt_brief": "1-2 sentences: what the prompt keeps first, changes later, ends on" }
]
}
Field rules the validator enforces:
media_inventorykeys are UPPERCASE; everyfile_pathis copied from Step 0 sources — this inventory is the machine-checked verbatim source: every segment's image items and every locked media track are diffed against it, so a retyped path fails on the spot. A paraphrase in the inventory cannot be repaired later. Order matters too: the inventory's image rows map to the editor'simageinput positions (<Picture 1>,<Picture 2>, …), audio rows toaudioinput positions, video rows tovideoinput positions — the user wires the media into those node inputs in inventory order.tracks.master_audio/tracks.reference_videoname the ONE locked audio and ONE locked video track (the schema allows at most one of each). Lock the master song for MV/lyric/ dance/beat projects, the narration track when narration drives the cut, and the reference video when the user supplies one for subject/partial replacement. Ordinary BGM/SFX/ambience is never locked. If two candidates could be the master, stop and ask the user — never pick by track order or duration.- Resolution: copy the user's intent into the exact DynamicCombo tokens —
16:9means16:9 (Widescreen),0.9mpmeans megapixels mode with the number taken verbatim; never convert MP into width/height. Keep the current workflow resolution when the user expressed no intent (then omit nothing — still write the block the user approved). - Segments:
start_secondsare cumulative (previous start + duration + gap); ids are unique lowercase snake_case, never renamed later;duration_secondsabout 10 unless the user says otherwise; only the last segment may come to rest.continuity_modeper segment:shot(standalone/new composition),context(continues the previous segment's real delivered tail),context_drift(context + subject/partial replacement from the locked reference video). context_drift is all-or-nothing: if any segment drifts, all do, first included; without user-pinned lengths the drift split is consecutive 10-second pieces (last takes the remainder) — setlengths_user_pinned: trueonly when the user actually specified the lengths. imagesper segment: at most 9, each an inventory image key with itsshared_referenceflag. Reference images define IDENTITY, not composition. When several context segments reuse one image, prefer explicit private items (shared_reference: false, fixed ids, same file) over shared-image metadata — the current front-end regenerates UUIDs for sparse shared images, and unverified shared data risks unstable loads.project:project_save: "override"withsegment_count: -1deletes saved segments after the start index on execution — warn the user before choosing it.upscale_byis only written when the user asked (default 1.250, three decimals); the editor resolution is the first-pass size, and dual-sampling second pass =round(editor_size * upscale_by / 32) * 32.
Step 2 — Present the guide for approval (do not skip)
Show the user the finished guide in plain language — one line per segment: number, id,
seconds, continuity mode, what happens and where it ends — plus the media inventory, the
locked tracks, the resolution block and the project parameters. Then STOP and wait for
their reply. The user approves it or asks for changes; apply every change to outline.json
and, if anything changed, show the changed rows again. Start Step 3 only after an explicit
approval. The approved guide is the contract: segments are composed from it, ids, frames
and modes are never silently altered, and new story ideas from the user go through the
guide first — never straight into a segment file.
Step 3 — Compose the segments one at a time
Work only from the approved guide. In story order: compose ONE segment file, validate it,
fix it until it passes, and only then move to the next. Since you wrote the previous file
yourself, open it and reuse its _handoff.end_state as the opening state of the next
context join. Rewrite a segment as many times as it takes. If Cline rejects a write with
'Editor input too large', split it with the marker pattern from the rules above.
Media copy rule (the one writers keep breaking): every file_path in a segment's
images is a character-for-character COPY of the outline's media_inventory path for that
key. Select the text in outline.json, copy, paste — never retype it from memory, never
fix a typo in it, never reword a folder name. The commands below pass --outline, so the
validator diffs the paths against the guide and a rewrite fails the segment on the spot.
The same discipline covers the resolution tokens and file names: the merge copies
editor.resolution and the locked media tracks straight from the outline, so the outline
is the only place they are ever typed.
Each fragment is exactly this shape:
{
"id": "from the outline row",
"start_frame": 0,
"end_frame": 240,
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "shot",
"ref_image_size": "match",
"images": [
{ "id": "img-seg-one-open-1", "source_type": "input",
"file_path": "copied verbatim from the inventory", "file_name": "pose_ref.png",
"shared_reference": false }
],
"user_prompt": "the full segment prompt",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
},
"_handoff": { "end_state": "the real final frame in 2-3 sentences: pose, motion in progress, camera state" }
}
start_frame/end_frame are computed, not guessed: round(seconds * frame_rate),
start inclusive, end exclusive, and they must match the approved row exactly. If the
variant is b, write user_prompt_b and leave user_prompt alone — never switch the
variant on your own.
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_01.json --index 1 --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_02.json --index 2 --prev segment_01.json --outline outline.json
(--total is optional with --outline; --prev chains the previous file and enforces
consecutive frames [0,240), [240,480), … — never an extra frame at a boundary.)
Prompt craft rules (violations fail the validator)
- Every task segment renders standalone — the model never sees any other segment's prompt. Fully restate scene, light and subject identity inside each prompt; never say "the same one", "as before", "the previous shot". This ban is checked in every mode.
- Reference images define identity only: they are not the first frame or a composition
anchor; context joins never use image anchoring or
[video continuation]. - context segments open by keeping the delivered tail — the join starts from the previous segment's real last frames (about 0.92 s earlier than the saved boundary at 24 fps), whose framing can differ from what the previous prompt imagined: open with the camera keeping its direction and the subject keeping pose, motion and light. Give the hold an action (a breath, a weight shift) so it does not render as a freeze-frame.
- Keep first, change later: before about
00:02.000no cut, no new key action; a required composition change is written as an explicit cut later in the prompt. For a genuinely new composition use ashotsegment instead ofcontext. - End in motion: a context segment followed by another context segment never ends at rest — no "comes to a stop", "settles", "holds still", "falls asleep" in the closing lines. Only the timeline's final segment may come to rest.
- Compose, never summarize: subject anchor with its look, blocking, one camera idea, performance beats, physical texture (wardrobe behavior, weather on skin and fabric, light), and a concrete mid-action exit state. Decompose every action into ordered body mechanics ("bends his knees, grips the handle with his left hand, then stands"), naming the limb; camera, light and weather each get their own ordered visible stages. The validator trips a thin-prompt floor (60 words) — never pad to a band, and never hand in a one-liner.
system_promptstays""unless the user asked for one: an empty system prompt lets the node auto-append the MiniMax reference rules for context joins; a custom one disables that._handoff.end_statemust be truthful — it is the contract the next join opens inside.
Step 4 — Verify the chain, put it all together at the end
Once every segment file passes, the merge is the continuity check: build_track_plan.py
re-walks every fragment against the guide (ids, frames, modes, verbatim media paths, prompt
rules, handoff chain), assembles the TRACK_DATA (task track from the fragments; the locked
master-audio / reference-video tracks built straight from the outline), computes
total_length, builds the minimax_prompt_override paste payload from the same validated
fragments, and only then writes the deliverables. Never write the plan by hand and never
edit fragment prose during the merge.
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" outline.json segment_01.json segment_02.json segment_03.json -o plan.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check plan.json --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override prompt_override.json --outline outline.json
The merge writes prompt_override.json (the paste payload, one line), plan.json (the
patch plan) and track_data.json (pretty, for inspection), and prints the delivered
durations/seconds, the track list and the locked tracks. Segment, track and image ids are
stable UUIDv5 values derived from the project title, so a re-merge keeps ids consistent.
The override command re-checks the paste payload against the guide: one prompt per
segment, durations inside the runtime's 2–15s window and equal to the approved rows,
per-segment generation/continuity modes, the exact <Picture N> / <Audio N> / <Video N>
reference set per prompt (a missing token renders with no reference media; a stray one
attaches the wrong image), and the track locks.
Step 5 — Deliver (paste payload first, patch script as the alternative)
5A — Default: the prompt_override paste
prompt_override.json is what the user pastes into the multitrack editor node's
prompt_override input (easy multiTrackEditor). It is self-contained: one prompt per
clip, per-clip durations and continuity modes, media reference tokens inside the prompts.
Deliver it with these instructions:
- In the editor node set
formattoMiniMax(the payload is parsed as MiniMax only in that format). - Wire the media IF the outline has any: the inventory's image rows (in inventory order)
into the editor's
imageinput, the master song intoaudio, the reference video intovideo. The<Picture N>/<Audio N>/<Video N>tokens resolve against those input lists positionally. With an empty inventory ([], pure t2v) nothing gets wired — the editor demands no media inputs at all. - Select the
prompt_overridewidget, clear it, paste the whole one-line JSON. The editor rebuilds the task track: one clip per prompt, back-to-back, durations as specified. - Rendering walks the timeline through
easy multiTrackTaskOutput— and that node outputs ONE task segment per pass, selected by itstask_indexint input (default 0 = only the FIRST clip). A workflow whosetask_indexstays 0 renders only the first scene no matter how many prompts the override carries. Two ways to render all clips:- Loop workflow (proper multi-shot): use the node pack's
example_workflows/MinimaxH3_ForLoop_MultiShot.json. ItsforLoopStarttakestotalfromeasy multiTrackInfoOutput.TASK_COUNT(via a math node that also lets a PrimitiveInt pick one segment: -1 = all), and its index feedsmultiTrackTaskOutput.task_index; each iteration generates one clip, batches it, andmergeVideosFromPathsconcatenates everything after the loop. Paste the override into THAT editor and set the PrimitiveInt to -1. - Manual fallback (no rewiring): set
task_indexto 0, queue; then 1, queue; then 2, queue. Three clip files, join them afterwards. (task_index = -1outputs the whole timeline's pre-rendered MEDIA — it is not a 3-clip generation.)
- Loop workflow (proper multi-shot): use the node pack's
The paste target is the prompt_override input and only that payload type. Never give
the user a plan.json, an outline, an H3 plan or any other JSON to paste there: without
"type": "minimax_prompt_override" the editor falls back to its |-separated text parser
and renders the whole thing as ONE clip spanning the timeline (the single-clip failure).
Never add | inside prompts for the same reason.
5B — Alternative: the node pack's patch script (whole-workflow patch)
When the user wants the timeline written into a workflow file directly (or needs edit
task modes / extra tracks the paste payload cannot express), the deliverable is applied by
the Easy-Media node pack's script — find it under the node pack folder (typically
<ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py;
ask the user for the path and the workflow JSON if they are not at hand):
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json --output RESULT.json --write
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py inspect RESULT.json
Always dry-run first; only write to a NEW output file (--output RESULT.json --write),
never over the user's source workflow. inspect must show the editor's resolution, format,
frame rate, total_length, the task track with all segments and the locked tracks with
their media. Report: base workflow path, output file, editor/project node ids, resolution,
timeline summary, media paths, continuity modes, locked tracks and whether the node graph
stayed untouched (graph_preserved: true).
Actually opening the workflow in ComfyUI or queueing it (submit_workflow.py, UI-tab
submission) is a separate authorization — only on the user's explicit request, and it is
the node pack's own flow (see its references/execution-api.md). "Queued" never means
"generated"; wait for the real completion status before reporting success.
Revising after approval
- Guide-level change (timings, beats, modes, segment count, media): update
outline.jsonfirst, show the changed rows, then rewrite only the affected segments against it and re-run the Step 4 merge soplan.jsonandprompt_override.jsonrefresh. - One segment reads badly: rewrite that
segment_NN.json(keep its id and frames from the guide), re-validate it with thesegmentcommand, then re-merge. Never patchplan.jsonorprompt_override.jsondirectly — the merge rebuilds both from the fragments every time.
If you cannot run Python
Say plainly that the result skipped script validation. Follow the rules manually: copy the
examples/ shapes exactly, compute frames as round(seconds * frame_rate), copy every
file_path and resolution token by selection (never retyping), and hand-assemble the
TRACK_DATA the way Step 4 describes. Manual assembly is error-prone — prefer getting
Python running (any of py -3, python, a .venv) over hand-building the plan.