24 KiB
Executable File
| name | description |
|---|---|
| easy-media-track-gate | Turn a video idea into a validated MultiTrack plan for ComfyUI-Easy-Media (the `easy multiTrackEditor` node). Use when the user wants to plan task segments/shots on the multitrack timeline, write or extend a MultiTrack timeline, arrange prompts, reference images, master audio or reference-video locks, regenerate one task segment, or produce the `prompt_override` paste payload for the multitrack editor node (plus the optional patch plan JSON that `patch_workflow.py apply` consumes). The scripts and references live next to this SKILL.md (the skill's install folder); run them from the user's working folder, where all generated files are written. |
Easy-Media Track Gate — agent procedure
Goal: produce a validated plan for the easy multiTrackEditor node in five moves: the user
gives a video idea and a segment count (or an existing workflow to edit) → you build a
timeline guide (outline.json) and show it for approval → with the approved guide you
compose the task segments one at a time (about 10 seconds each), each turn existing so that
ONE segment gets a complete prompt, the right references and the right continuity mode →
the scripts verify schema, timing and continuity across the whole chain →
build_track_plan.py assembles the deliverables at the end. The PRIMARY deliverable is
prompt_override.json, a minimax_prompt_override payload the user pastes into the
multitrack editor node's prompt_override input (format MiniMax) — the editor then
renders one clip per prompt, chained back-to-back. The patch plan (plan.json for the node
pack's patch_workflow.py) is the optional whole-workflow alternative. Never deliver a plan
that did not pass validation, and never compose a segment from a guide the user has not
approved. Never hand the user any other JSON as the paste: a non-minimax_prompt_override
value in prompt_override falls back to the editor's |-separated text parser and renders
the whole story as ONE clip over the entire timeline.
Paths and Python (read first — these exact forms prevent most errors)
- SKILL_DIR is the folder this SKILL.md was loaded from — the skill's install folder
(default on Windows:
%userprofile%\.agent\skills\easy-media-track-gate; macOS/Linux:~/.agent/skills/easy-media-track-gate). The scripts, the schema reference and the worked example all live in SKILL_DIR, not in the working folder. Every command below uses the Windows default install path; substitute your real SKILL_DIR if it differs. - Work in the user's working folder (the folder open in the editor). Write generated
files there:
outline.json,segment_01.json,segment_02.json, …,plan.json,track_data.json. Never write generated files into SKILL_DIR. - Python command: Windows
py -3(orpython); macOS/Linuxpython3. If the working folder has a.venv, its interpreter works too (.venv\Scripts\python.exe). The scripts need only the Python standard library — nothing to install. - Start every task by running the smoke test under "Before you start". If it fails because SKILL_DIR is missing files, stop and tell the user the skill folder is incomplete.
- Never read
verify_track_rules.py— run it and work from its printed output. Cline's read_file is capped by hard-coded budgets that do NOT grow with the model's context window: about 48,000 characters per read, 2,000 lines per read. The 81 KB validator arrives cut no matter how large the context window is. A truncation notice is Cline's read cap, not repo corruption — do not "fix" truncated files.build_track_plan.py(~10 KB) andreferences/track-schema.md(~26 KB) are safe to read whole. - Never read back
plan.json,track_data.jsonorprompt_override.jsononce real segments exist — a 10-segment plan passes the read caps today and exceeds them tomorrow. Validate with--check/ theoverridecommand and trust the one-line OK summary; the shippedexamples/prompt_override.json(2.5 KB) andexamples/plan.json(6.8 KB) are safe to read if you need to see the shape. - Cline's editor rejects writes over 6,000 characters ("Editor input too large"). A segment
fragment with a rich prompt often does. NEVER write part 2 with write_to_file — it
replaces the whole file and part 1 is lost. Split instead: write the first part ending
with the bare marker line
@@CONTINUE@@, then replace_in_file with SEARCH@@CONTINUE@@and REPLACE the remaining content. The finished file must not contain the marker. - Cline also cuts command output at 8,000 characters. The embedded suite prints well under
1 KB when it passes; a failing validation prints one actionable line. If output ever
arrives truncated, re-run the single-file command (
segment,outline,plan) — its output is tiny.
Before you start (smoke test)
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\plan.json"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\prompt_override.json" --outline "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\outline.json"
Expect the suite to print SUITE PASS (34/34 cases) and the other two commands to print
PLAN OK with total_length 720 frames = 30.00s and OVERRIDE OK: 3 prompts, duration 10, total 30.00s.
Step 0 — Media paths come first (never invent one)
Every file_path in the plan must be a real path on the user's ComfyUI instance. Before
writing the outline, establish the media inventory one of two ways:
- The user names the exact input paths (they know their
input/subfolders). Use their strings verbatim — they become the pinned inventory. - The user explicitly asks you to upload → upload first and record the returned relative
path (
subfolder/name,source_type: "input") for every file. Upload uses the node pack's own API (POST /upload/imagewith asubfolderlikecodex/<project>); never write a local absolute path into the plan. - If there is no real path and no upload request, stop and ask. A plan is still deliverable with obviously-fake placeholder paths only if you say plainly it cannot run until real media replaces them — never dress a placeholder up as a real path.
Upload and execution are external state changes: only upload/queue when the user explicitly asked (Step 5).
Step 1 — Build the timeline guide (outline)
Read SKILL_DIR's references/track-schema.md once (safe whole-file read; it is the English
translation of the node pack's Chinese schema doc). Then write outline.json — the guide
for the whole timeline — in exactly this shape (the shipped examples/outline.json is the
worked model):
{
"title": "short_snake_case_name",
"lengths_user_pinned": false,
"editor": {
"format": "MiniMax",
"resolution": { "resolution": "width x height (megapixels)",
"aspect_ratio": "16:9 (Widescreen)", "megapixels": 0.8 }
},
"defaults": { "frame_rate": 24, "segment_seconds": 10 },
"media_inventory": [
{ "key": "SONG", "kind": "audio", "source_type": "input",
"file_path": "codex/rooftop/song.wav", "file_name": "song.wav" },
{ "key": "POSE_REF", "kind": "image", "source_type": "input",
"file_path": "codex/rooftop/pose_ref.png", "file_name": "pose_ref.png" }
],
"tracks": {
"master_audio": { "key": "SONG", "lock": true, "name": "Master Song", "role": "why it drives the timeline" },
"reference_video": null
},
"project": { "project_save": "override", "segment_start_number": 1,
"segment_count": -1, "sampling_plan": "medium", "sampling_mode": "single" },
"segments": [
{ "index": 1, "id": "seg_one_open", "start_seconds": 0, "duration_seconds": 10,
"task_mode": "ref", "continuity_mode": "shot",
"images": [ { "key": "POSE_REF", "shared_reference": false } ],
"beat": "one sentence: what happens AND where the segment ends",
"prompt_brief": "1-2 sentences: what the prompt keeps first, changes later, ends on" }
]
}
Field rules the validator enforces:
media_inventorykeys are UPPERCASE; everyfile_pathis copied from Step 0 sources — this inventory is the machine-checked verbatim source: every segment's image items and every locked media track are diffed against it, so a retyped path fails on the spot. A paraphrase in the inventory cannot be repaired later. Order matters too: the inventory's image rows map to the editor'simageinput positions (<Picture 1>,<Picture 2>, …), audio rows toaudioinput positions, video rows tovideoinput positions — the user wires the media into those node inputs in inventory order.tracks.master_audio/tracks.reference_videoname the ONE locked audio and ONE locked video track (the schema allows at most one of each). Lock the master song for MV/lyric/ dance/beat projects, the narration track when narration drives the cut, and the reference video when the user supplies one for subject/partial replacement. Ordinary BGM/SFX/ambience is never locked. If two candidates could be the master, stop and ask the user — never pick by track order or duration.- Resolution: copy the user's intent into the exact DynamicCombo tokens —
16:9means16:9 (Widescreen),0.9mpmeans megapixels mode with the number taken verbatim; never convert MP into width/height. Keep the current workflow resolution when the user expressed no intent (then omit nothing — still write the block the user approved). - Segments:
start_secondsare cumulative (previous start + duration + gap); ids are unique lowercase snake_case, never renamed later;duration_secondsabout 10 unless the user says otherwise; only the last segment may come to rest.continuity_modeper segment:shot(standalone/new composition),context(continues the previous segment's real delivered tail),context_drift(context + subject/partial replacement from the locked reference video). context_drift is all-or-nothing: if any segment drifts, all do, first included; without user-pinned lengths the drift split is consecutive 10-second pieces (last takes the remainder) — setlengths_user_pinned: trueonly when the user actually specified the lengths. imagesper segment: at most 9, each an inventory image key with itsshared_referenceflag. Reference images define IDENTITY, not composition. When several context segments reuse one image, prefer explicit private items (shared_reference: false, fixed ids, same file) over shared-image metadata — the current front-end regenerates UUIDs for sparse shared images, and unverified shared data risks unstable loads.project:project_save: "override"withsegment_count: -1deletes saved segments after the start index on execution — warn the user before choosing it.upscale_byis only written when the user asked (default 1.250, three decimals); the editor resolution is the first-pass size, and dual-sampling second pass =round(editor_size * upscale_by / 32) * 32.
Step 2 — Present the guide for approval (do not skip)
Show the user the finished guide in plain language — one line per segment: number, id,
seconds, continuity mode, what happens and where it ends — plus the media inventory, the
locked tracks, the resolution block and the project parameters. Then STOP and wait for
their reply. The user approves it or asks for changes; apply every change to outline.json
and, if anything changed, show the changed rows again. Start Step 3 only after an explicit
approval. The approved guide is the contract: segments are composed from it, ids, frames
and modes are never silently altered, and new story ideas from the user go through the
guide first — never straight into a segment file.
Step 3 — Compose the segments one at a time
Work only from the approved guide. In story order: compose ONE segment file, validate it,
fix it until it passes, and only then move to the next. Since you wrote the previous file
yourself, open it and reuse its _handoff.end_state as the opening state of the next
context join. Rewrite a segment as many times as it takes. If Cline rejects a write with
'Editor input too large', split it with the marker pattern from the rules above.
Media copy rule (the one writers keep breaking): every file_path in a segment's
images is a character-for-character COPY of the outline's media_inventory path for that
key. Select the text in outline.json, copy, paste — never retype it from memory, never
fix a typo in it, never reword a folder name. The commands below pass --outline, so the
validator diffs the paths against the guide and a rewrite fails the segment on the spot.
The same discipline covers the resolution tokens and file names: the merge copies
editor.resolution and the locked media tracks straight from the outline, so the outline
is the only place they are ever typed.
Each fragment is exactly this shape:
{
"id": "from the outline row",
"start_frame": 0,
"end_frame": 240,
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "shot",
"ref_image_size": "match",
"images": [
{ "id": "img-seg-one-open-1", "source_type": "input",
"file_path": "copied verbatim from the inventory", "file_name": "pose_ref.png",
"shared_reference": false }
],
"user_prompt": "the full segment prompt",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
},
"_handoff": { "end_state": "the real final frame in 2-3 sentences: pose, motion in progress, camera state" }
}
start_frame/end_frame are computed, not guessed: round(seconds * frame_rate),
start inclusive, end exclusive, and they must match the approved row exactly. If the
variant is b, write user_prompt_b and leave user_prompt alone — never switch the
variant on your own.
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_01.json --index 1 --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_02.json --index 2 --prev segment_01.json --outline outline.json
(--total is optional with --outline; --prev chains the previous file and enforces
consecutive frames [0,240), [240,480), … — never an extra frame at a boundary.)
Prompt craft rules (violations fail the validator)
- Every task segment renders standalone — the model never sees any other segment's prompt. Fully restate scene, light and subject identity inside each prompt; never say "the same one", "as before", "the previous shot". This ban is checked in every mode.
- Reference images define identity only: they are not the first frame or a composition
anchor; context joins never use image anchoring or
[video continuation]. - context segments open by keeping the delivered tail — the join starts from the previous segment's real last frames (about 0.92 s earlier than the saved boundary at 24 fps), whose framing can differ from what the previous prompt imagined: open with the camera keeping its direction and the subject keeping pose, motion and light. Give the hold an action (a breath, a weight shift) so it does not render as a freeze-frame.
- Keep first, change later: before about
00:02.000no cut, no new key action; a required composition change is written as an explicit cut later in the prompt. For a genuinely new composition use ashotsegment instead ofcontext. - End in motion: a context segment followed by another context segment never ends at rest — no "comes to a stop", "settles", "holds still", "falls asleep" in the closing lines. Only the timeline's final segment may come to rest.
- Compose, never summarize: subject anchor with its look, blocking, one camera idea, performance beats, physical texture (wardrobe behavior, weather on skin and fabric, light), and a concrete mid-action exit state. Decompose every action into ordered body mechanics ("bends his knees, grips the handle with his left hand, then stands"), naming the limb; camera, light and weather each get their own ordered visible stages. The validator trips a thin-prompt floor (60 words) — never pad to a band, and never hand in a one-liner.
system_promptstays""unless the user asked for one: an empty system prompt lets the node auto-append the MiniMax reference rules for context joins; a custom one disables that._handoff.end_statemust be truthful — it is the contract the next join opens inside.
Step 4 — Verify the chain, put it all together at the end
Once every segment file passes, the merge is the continuity check: build_track_plan.py
re-walks every fragment against the guide (ids, frames, modes, verbatim media paths, prompt
rules, handoff chain), assembles the TRACK_DATA (task track from the fragments; the locked
master-audio / reference-video tracks built straight from the outline), computes
total_length, builds the minimax_prompt_override paste payload from the same validated
fragments, and only then writes the deliverables. Never write the plan by hand and never
edit fragment prose during the merge.
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" outline.json segment_01.json segment_02.json segment_03.json -o plan.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check plan.json --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" override prompt_override.json --outline outline.json
The merge writes prompt_override.json (the paste payload, one line), plan.json (the
patch plan) and track_data.json (pretty, for inspection), and prints the delivered
durations/seconds, the track list and the locked tracks. Segment, track and image ids are
stable UUIDv5 values derived from the project title, so a re-merge keeps ids consistent.
The override command re-checks the paste payload against the guide: one prompt per
segment, durations inside the runtime's 2–15s window and equal to the approved rows,
per-segment generation/continuity modes, the exact <Picture N> / <Audio N> / <Video N>
reference set per prompt (a missing token renders with no reference media; a stray one
attaches the wrong image), and the track locks.
Step 5 — Deliver (paste payload first, patch script as the alternative)
5A — Default: the prompt_override paste
prompt_override.json is what the user pastes into the multitrack editor node's
prompt_override input (easy multiTrackEditor). It is minimal by design — the parser
requires only type and prompts; optional keys appear only when non-default (duration is
SECONDS, generation_type defaults to r2v in the pack's code). It is self-contained: one
prompt per clip, media reference tokens inside the prompts, and no frame fields anywhere.
Deliver it with these instructions:
- In the editor node set
formattoMiniMax(the payload is parsed as MiniMax only in that format). - Wire the media IF the outline has any: the inventory's image rows (in inventory order)
into the editor's
imageinput, the master song intoaudio, the reference video intovideo. The<Picture N>/<Audio N>/<Video N>tokens resolve against those input lists positionally. With an empty inventory ([], pure t2v) nothing gets wired — the editor demands no media inputs at all. - Select the
prompt_overridewidget, clear it, paste the whole one-line JSON. The editor rebuilds the task track: one clip per prompt, back-to-back, durations as specified. - Rendering walks the timeline through
easy multiTrackTaskOutput— and that node outputs ONE task segment per pass, selected by itstask_indexint input (default 0 = only the FIRST clip). A workflow whosetask_indexstays 0 renders only the first scene no matter how many prompts the override carries. Two ways to render all clips:- Loop workflow (proper multi-shot): use the node pack's
example_workflows/MinimaxH3_ForLoop_MultiShot.json. ItsforLoopStarttakestotalfromeasy multiTrackInfoOutput.TASK_COUNT(via a math node that also lets a PrimitiveInt pick one segment: -1 = all), and its index feedsmultiTrackTaskOutput.task_index; each iteration generates one clip, batches it, andmergeVideosFromPathsconcatenates everything after the loop. Paste the override into THAT editor and set the PrimitiveInt to -1. - Manual fallback (no rewiring): set
task_indexto 0, queue; then 1, queue; then 2, queue. Three clip files, join them afterwards. (task_index = -1outputs the whole timeline's pre-rendered MEDIA — it is not a 3-clip generation.)
- Loop workflow (proper multi-shot): use the node pack's
- Optional LLM polish slot (only if the user asks for it): inside the loop,
TaskOutput.SYSTEM_PROMPT/USER_PROMPTmay feed an LLM node whose output replaces the prompt at the render node, with an ifElse bypass back to rawUSER_PROMPT. The skill's prompts are already final — leave the bypass OFF unless requested. Ways to fill the slot: any ComfyUI LLM node (system+user in, string out), or the pack'seasy multiTrackPromptEnhancerwith modelopenai_compatible (local)pointed at the user's own endpoint (base_urlaccepts any OpenAI-compatible server: Ollama, LM Studio, vLLM, llama.cpp server). Full wiring:references/workflow_wiring.md.
The paste target is the prompt_override input and only that payload type. Never give
the user a plan.json, an outline, an H3 plan or any other JSON to paste there: without
"type": "minimax_prompt_override" the editor falls back to its |-separated text parser
and renders the whole thing as ONE clip spanning the timeline (the single-clip failure).
Never add | inside prompts for the same reason.
5B — Alternative: the node pack's patch script (whole-workflow patch)
When the user wants the timeline written into a workflow file directly (or needs edit
task modes / extra tracks the paste payload cannot express), the deliverable is applied by
the Easy-Media node pack's script — find it under the node pack folder (typically
<ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py;
ask the user for the path and the workflow JSON if they are not at hand):
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json --output RESULT.json --write
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py inspect RESULT.json
Always dry-run first; only write to a NEW output file (--output RESULT.json --write),
never over the user's source workflow. inspect must show the editor's resolution, format,
frame rate, total_length, the task track with all segments and the locked tracks with
their media. Report: base workflow path, output file, editor/project node ids, resolution,
timeline summary, media paths, continuity modes, locked tracks and whether the node graph
stayed untouched (graph_preserved: true).
Actually opening the workflow in ComfyUI or queueing it (submit_workflow.py, UI-tab
submission) is a separate authorization — only on the user's explicit request, and it is
the node pack's own flow (see its references/execution-api.md). "Queued" never means
"generated"; wait for the real completion status before reporting success.
Revising after approval
- Guide-level change (timings, beats, modes, segment count, media): update
outline.jsonfirst, show the changed rows, then rewrite only the affected segments against it and re-run the Step 4 merge soplan.jsonandprompt_override.jsonrefresh. - One segment reads badly: rewrite that
segment_NN.json(keep its id and frames from the guide), re-validate it with thesegmentcommand, then re-merge. Never patchplan.jsonorprompt_override.jsondirectly — the merge rebuilds both from the fragments every time.
If you cannot run Python
Say plainly that the result skipped script validation. Follow the rules manually: copy the
examples/ shapes exactly, compute frames as round(seconds * frame_rate), copy every
file_path and resolution token by selection (never retyping), and hand-assemble the
TRACK_DATA the way Step 4 describes. Manual assembly is error-prone — prefer getting
Python running (any of py -3, python, a .venv) over hand-building the plan.