mmprompt/easy-media-track-gate/SKILL.md
Z User ea5c5a71b4 easy-media-track-gate: the h3-plan-gate methodology adapted to ComfyUI-Easy-Media multitrack nodes
Portable self-contained skill (%USERPROFILE%\.agent\skills\easy-media-track-gate\):
- references/track-schema.md: full English translation of the node pack's Chinese
  multitrack-schema.md (resolution DynamicCombo rules, time semantics, task segment
  content, context/context_drift joins, audio locks, shared media, project params,
  patch plan shape); field names cross-checked against patch_workflow.py source.
- verify_track_rules.py (66 KB, stdlib only): mirrors every check the node pack's
  patch_workflow.py performs at apply time (ids, sorting, overlap, frames, source_type
  field matching, <=9 images, continuity enums, one locked audio + one locked video,
  locked-media task coverage, shared_reference rules, resolution tokens, project
  fields) and adds the gates upstream leaves to discipline: outline pins (id/frames/
  modes/image set diffed per fragment AND at the merge), verbatim media contract
  (file_path character-for-character vs media_inventory, closest-match hint on
  rewrite), context prompt rules (self-contained in all modes, keep-then-change
  opener for joins, end-in-motion tail when another join follows), drift all-or-
  nothing + consecutive 10s split unless lengths_user_pinned. Subcommands: outline/
  segment/trackdata/plan/suite; no-outline runs stay schema-only. Embedded suite
  25 cases, prints ~0.7 KB (far under Cline's 8k cap).
- build_track_plan.py: merge re-walks every fragment against the guide, assembles
  TRACK_DATA (task track from fragments; locked master-audio/reference-video tracks
  built from the outline, never retyped), writes plan.json + track_data.json with
  stable UUIDv5 ids; --check mode re-validates.
- examples/: worked 3-segment rooftop example (outline, 3 fragments, plan.json,
  track_data.json) - merge is byte-deterministic and the generated plan dry-run-
  applies to the bundled v1.3.0 blank template with graph_preserved: true.
- SKILL.md: five-move procedure with Cline cap workarounds (never read the 66 KB
  validator, @@CONTINUE@@ split, never read back generated plans), Step 0 media
  paths (upload only on explicit request; never invent a path), approval pause,
  prompt craft rules, delivery through the node pack's own patch_workflow.py
  (dry-run -> --write to a NEW file -> inspect); queueing stays the node pack's
  own flow on explicit user request.
2026-11-01 01:09:26 +00:00

321 lines
19 KiB
Markdown

---
name: easy-media-track-gate
description: Turn a video idea into a validated MultiTrack patch plan for ComfyUI-Easy-Media
(the `easy multiTrackEditor` / `easy multitrackProject` nodes). Use when the user wants to
plan task segments/shots on the multitrack timeline, write or extend a MultiTrack timeline,
arrange prompts, reference images, master audio or reference-video locks, regenerate one
task segment, or produce the patch plan JSON that `patch_workflow.py apply` consumes. The
scripts and references live next to this SKILL.md (the skill's install folder); run them
from the user's working folder, where all generated files are written.
---
# Easy-Media Track Gate — agent procedure
Goal: produce `plan.json`, the patch plan applied to a ComfyUI workflow with the Easy-Media
node pack's own `patch_workflow.py`, in five moves: the user gives a video idea and a
segment count (or an existing workflow to edit) → you build a timeline guide (`outline.json`)
and show it for **approval** → with the approved guide you compose the task segments one at
a time (about 10 seconds each), each turn existing so that ONE segment gets a complete
prompt, the right references and the right continuity mode → the scripts verify schema,
timing and continuity across the whole chain → `build_track_plan.py` assembles the TRACK_DATA
and the patch plan at the end. Never deliver a plan that did not pass validation, and never
compose a segment from a guide the user has not approved.
## Paths and Python (read first — these exact forms prevent most errors)
- **SKILL_DIR** is the folder this SKILL.md was loaded from — the skill's install folder
(default on Windows: `%userprofile%\.agent\skills\easy-media-track-gate`; macOS/Linux:
`~/.agent/skills/easy-media-track-gate`). The scripts, the schema reference and the worked
example all live in SKILL_DIR, not in the working folder. Every command below uses the
Windows default install path; substitute your real SKILL_DIR if it differs.
- Work in the user's **working folder** (the folder open in the editor). Write generated
files there: `outline.json`, `segment_01.json`, `segment_02.json`, …, `plan.json`,
`track_data.json`. Never write generated files into SKILL_DIR.
- Python command: Windows `py -3` (or `python`); macOS/Linux `python3`. If the working
folder has a `.venv`, its interpreter works too (`.venv\Scripts\python.exe`). The scripts
need only the Python standard library — nothing to install.
- Start every task by running the smoke test under "Before you start". If it fails because
SKILL_DIR is missing files, stop and tell the user the skill folder is incomplete.
- **Never read `verify_track_rules.py`** — run it and work from its printed output. Cline's
read_file is capped by hard-coded budgets that do NOT grow with the model's context
window: about 48,000 characters per read, 2,000 lines per read. The 66 KB validator
arrives cut no matter how large the context window is. A truncation notice is Cline's
read cap, not repo corruption — do not "fix" truncated files. `build_track_plan.py`
(~9 KB) and `references/track-schema.md` (~22 KB, 465 lines) are safe to read whole.
- **Never read back `plan.json` or `track_data.json` once real segments exist** — a
10-segment plan passes the read caps today and exceeds them tomorrow. Validate with
`--check` and trust the one-line OK summary; the shipped `examples/plan.json` (6.8 KB)
is safe to read if you need to see the shape.
- Cline's editor rejects writes over 6,000 characters ("Editor input too large"). A segment
fragment with a rich prompt often does. NEVER write part 2 with write_to_file — it
replaces the whole file and part 1 is lost. Split instead: write the first part ending
with the bare marker line `@@CONTINUE@@`, then replace_in_file with SEARCH `@@CONTINUE@@`
and REPLACE the remaining content. The finished file must not contain the marker.
- Cline also cuts command output at 8,000 characters. The embedded suite prints well under
1 KB when it passes; a failing validation prints one actionable line. If output ever
arrives truncated, re-run the single-file command (`segment`, `outline`, `plan`) — its
output is tiny.
## Before you start (smoke test)
```sh
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py"
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check "%USERPROFILE%\.agent\skills\easy-media-track-gate\examples\plan.json"
```
Expect the suite to print `SUITE PASS (25/25 cases)` and the second command to print
`PLAN OK` with `total_length 720 frames = 30.00s`.
## Step 0 — Media paths come first (never invent one)
Every `file_path` in the plan must be a real path on the user's ComfyUI instance. Before
writing the outline, establish the media inventory one of two ways:
- The user names the exact input paths (they know their `input/` subfolders). Use their
strings verbatim — they become the pinned inventory.
- The user explicitly asks you to upload → upload first and record the returned relative
path (`subfolder/name`, `source_type: "input"`) for every file. Upload uses the node
pack's own API (`POST /upload/image` with a `subfolder` like `codex/<project>`); never
write a local absolute path into the plan.
- If there is no real path and no upload request, stop and ask. A plan is still deliverable
with obviously-fake placeholder paths only if you say plainly it cannot run until real
media replaces them — never dress a placeholder up as a real path.
Upload and execution are external state changes: only upload/queue when the user explicitly
asked (Step 5).
## Step 1 — Build the timeline guide (outline)
Read SKILL_DIR's `references/track-schema.md` once (safe whole-file read; it is the English
translation of the node pack's Chinese schema doc). Then write `outline.json` — the guide
for the whole timeline — in exactly this shape (the shipped `examples/outline.json` is the
worked model):
```json
{
"title": "short_snake_case_name",
"lengths_user_pinned": false,
"editor": {
"format": "MiniMax",
"resolution": { "resolution": "width x height (megapixels)",
"aspect_ratio": "16:9 (Widescreen)", "megapixels": 0.8 }
},
"defaults": { "frame_rate": 24, "segment_seconds": 10 },
"media_inventory": [
{ "key": "SONG", "kind": "audio", "source_type": "input",
"file_path": "codex/rooftop/song.wav", "file_name": "song.wav" },
{ "key": "POSE_REF", "kind": "image", "source_type": "input",
"file_path": "codex/rooftop/pose_ref.png", "file_name": "pose_ref.png" }
],
"tracks": {
"master_audio": { "key": "SONG", "lock": true, "name": "Master Song", "role": "why it drives the timeline" },
"reference_video": null
},
"project": { "project_save": "override", "segment_start_number": 1,
"segment_count": -1, "sampling_plan": "medium", "sampling_mode": "single" },
"segments": [
{ "index": 1, "id": "seg_one_open", "start_seconds": 0, "duration_seconds": 10,
"task_mode": "ref", "continuity_mode": "shot",
"images": [ { "key": "POSE_REF", "shared_reference": false } ],
"beat": "one sentence: what happens AND where the segment ends",
"prompt_brief": "1-2 sentences: what the prompt keeps first, changes later, ends on" }
]
}
```
Field rules the validator enforces:
- `media_inventory` keys are UPPERCASE; every `file_path` is copied from Step 0 sources —
this inventory is the **machine-checked verbatim source**: every segment's image items and
every locked media track are diffed against it, so a retyped path fails on the spot. A
paraphrase in the inventory cannot be repaired later.
- `tracks.master_audio` / `tracks.reference_video` name the ONE locked audio and ONE locked
video track (the schema allows at most one of each). Lock the master song for MV/lyric/
dance/beat projects, the narration track when narration drives the cut, and the reference
video when the user supplies one for subject/partial replacement. Ordinary BGM/SFX/ambience
is never locked. If two candidates could be the master, stop and ask the user — never pick
by track order or duration.
- Resolution: copy the user's intent into the exact DynamicCombo tokens — `16:9` means
`16:9 (Widescreen)`, `0.9mp` means megapixels mode with the number taken verbatim; never
convert MP into width/height. Keep the current workflow resolution when the user expressed
no intent (then omit nothing — still write the block the user approved).
- Segments: `start_seconds` are cumulative (previous start + duration + gap); ids are unique
lowercase snake_case, never renamed later; `duration_seconds` about 10 unless the user
says otherwise; only the last segment may come to rest. `continuity_mode` per segment:
`shot` (standalone/new composition), `context` (continues the previous segment's real
delivered tail), `context_drift` (context + subject/partial replacement from the locked
reference video). **context_drift is all-or-nothing**: if any segment drifts, all do,
first included; without user-pinned lengths the drift split is consecutive 10-second
pieces (last takes the remainder) — set `lengths_user_pinned: true` only when the user
actually specified the lengths.
- `images` per segment: at most 9, each an inventory image key with its `shared_reference`
flag. Reference images define IDENTITY, not composition. When several context segments
reuse one image, prefer explicit private items (`shared_reference: false`, fixed ids,
same file) over shared-image metadata — the current front-end regenerates UUIDs for
sparse shared images, and unverified shared data risks unstable loads.
- `project`: `project_save: "override"` with `segment_count: -1` deletes saved segments
after the start index on execution — warn the user before choosing it. `upscale_by` is
only written when the user asked (default 1.250, three decimals); the editor resolution
is the first-pass size, and dual-sampling second pass = `round(editor_size * upscale_by /
32) * 32`.
## Step 2 — Present the guide for approval (do not skip)
Show the user the finished guide in plain language — one line per segment: number, id,
seconds, continuity mode, what happens and where it ends — plus the media inventory, the
locked tracks, the resolution block and the project parameters. Then STOP and wait for
their reply. The user approves it or asks for changes; apply every change to `outline.json`
and, if anything changed, show the changed rows again. Start Step 3 only after an explicit
approval. The approved guide is the contract: segments are composed from it, ids, frames
and modes are never silently altered, and new story ideas from the user go through the
guide first — never straight into a segment file.
## Step 3 — Compose the segments one at a time
Work only from the approved guide. In story order: compose ONE segment file, validate it,
fix it until it passes, and only then move to the next. Since you wrote the previous file
yourself, open it and reuse its `_handoff.end_state` as the opening state of the next
context join. Rewrite a segment as many times as it takes. If Cline rejects a write with
'Editor input too large', split it with the marker pattern from the rules above.
**Media copy rule (the one writers keep breaking):** every `file_path` in a segment's
`images` is a character-for-character COPY of the outline's `media_inventory` path for that
key. Select the text in `outline.json`, copy, paste — never retype it from memory, never
fix a typo in it, never reword a folder name. The commands below pass `--outline`, so the
validator diffs the paths against the guide and a rewrite fails the segment on the spot.
The same discipline covers the resolution tokens and file names: the merge copies
`editor.resolution` and the locked media tracks straight from the outline, so the outline
is the only place they are ever typed.
Each fragment is exactly this shape:
```json
{
"id": "from the outline row",
"start_frame": 0,
"end_frame": 240,
"content": {
"media_type": "none",
"task_mode": "ref",
"continuity_mode": "shot",
"ref_image_size": "match",
"images": [
{ "id": "img-seg-one-open-1", "source_type": "input",
"file_path": "copied verbatim from the inventory", "file_name": "pose_ref.png",
"shared_reference": false }
],
"user_prompt": "the full segment prompt",
"system_prompt": "",
"user_prompt_variant": "a",
"user_prompt_b": "",
"muted": false,
"volume_db": 0
},
"_handoff": { "end_state": "the real final frame in 2-3 sentences: pose, motion in progress, camera state" }
}
```
`start_frame`/`end_frame` are computed, not guessed: `round(seconds * frame_rate)`,
start inclusive, end exclusive, and they must match the approved row exactly. If the
variant is `b`, write `user_prompt_b` and leave `user_prompt` alone — never switch the
variant on your own.
```sh
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_01.json --index 1 --outline outline.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\verify_track_rules.py" segment segment_02.json --index 2 --prev segment_01.json --outline outline.json
```
(`--total` is optional with `--outline`; `--prev` chains the previous file and enforces
consecutive frames `[0,240), [240,480), …` — never an extra frame at a boundary.)
### Prompt craft rules (violations fail the validator)
- Every task segment renders standalone — the model never sees any other segment's prompt.
Fully restate scene, light and subject identity inside each prompt; never say "the same
one", "as before", "the previous shot". This ban is checked in every mode.
- Reference images define identity only: they are not the first frame or a composition
anchor; context joins never use image anchoring or `[video continuation]`.
- **context segments open by keeping the delivered tail** — the join starts from the
previous segment's real last frames (about 0.92 s earlier than the saved boundary at
24 fps), whose framing can differ from what the previous prompt imagined: open with the
camera keeping its direction and the subject keeping pose, motion and light. Give the
hold an action (a breath, a weight shift) so it does not render as a freeze-frame.
- **Keep first, change later**: before about `00:02.000` no cut, no new key action; a
required composition change is written as an explicit cut later in the prompt. For a
genuinely new composition use a `shot` segment instead of `context`.
- **End in motion**: a context segment followed by another context segment never ends at
rest — no "comes to a stop", "settles", "holds still", "falls asleep" in the closing
lines. Only the timeline's final segment may come to rest.
- Compose, never summarize: subject anchor with its look, blocking, one camera idea,
performance beats, physical texture (wardrobe behavior, weather on skin and fabric,
light), and a concrete mid-action exit state. Decompose every action into ordered
body mechanics ("bends his knees, grips the handle with his left hand, then stands"),
naming the limb; camera, light and weather each get their own ordered visible stages.
The validator trips a thin-prompt floor (60 words) — never pad to a band, and never
hand in a one-liner.
- `system_prompt` stays `""` unless the user asked for one: an empty system prompt lets the
node auto-append the MiniMax reference rules for context joins; a custom one disables
that.
- `_handoff.end_state` must be truthful — it is the contract the next join opens inside.
## Step 4 — Verify the chain, put it all together at the end
Once every segment file passes, the merge is the continuity check: `build_track_plan.py`
re-walks every fragment against the guide (ids, frames, modes, verbatim media paths, prompt
rules, handoff chain), assembles the TRACK_DATA (task track from the fragments; the locked
master-audio / reference-video tracks built straight from the outline), computes
`total_length`, and only then writes the deliverable. Never write the plan by hand and
never edit fragment prose during the merge.
```sh
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" outline.json segment_01.json segment_02.json segment_03.json -o plan.json
py -3 "%USERPROFILE%\.agent\skills\easy-media-track-gate\build_track_plan.py" --check plan.json --outline outline.json
```
The merge writes `plan.json` (the patch plan) and `track_data.json` (pretty, for
inspection) and prints the delivered frames/seconds, the track list and the locked tracks.
Segment, track and image ids are stable UUIDv5 values derived from the project title, so a
re-merge keeps ids consistent.
## Step 5 — Deliver (the node pack's own script applies it)
The deliverable is applied to the user's workflow by the Easy-Media node pack's script —
find it under the node pack folder (typically
`<ComfyUI>\custom_nodes\ComfyUI-Easy-Media\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py`;
ask the user for the path and the workflow JSON if they are not at hand):
```sh
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py apply WORKFLOW.json --plan plan.json --output RESULT.json --write
py -3 <node-pack>\skills\easy-media-multitrack-workflow\scripts\patch_workflow.py inspect RESULT.json
```
Always dry-run first; only write to a NEW output file (`--output RESULT.json --write`),
never over the user's source workflow. `inspect` must show the editor's resolution, format,
frame rate, `total_length`, the task track with all segments and the locked tracks with
their media. Report: base workflow path, output file, editor/project node ids, resolution,
timeline summary, media paths, continuity modes, locked tracks and whether the node graph
stayed untouched (`graph_preserved: true`).
Actually opening the workflow in ComfyUI or queueing it (`submit_workflow.py`, UI-tab
submission) is a separate authorization — only on the user's explicit request, and it is
the node pack's own flow (see its `references/execution-api.md`). "Queued" never means
"generated"; wait for the real completion status before reporting success.
## Revising after approval
- Guide-level change (timings, beats, modes, segment count, media): update `outline.json`
first, show the changed rows, then rewrite only the affected segments against it and
re-run the Step 4 merge so `plan.json` refreshes.
- One segment reads badly: rewrite that `segment_NN.json` (keep its id and frames from the
guide), re-validate it with the `segment` command, then re-merge. Never patch `plan.json`
directly — the merge rebuilds it from the fragments every time.
## If you cannot run Python
Say plainly that the result skipped script validation. Follow the rules manually: copy the
`examples/` shapes exactly, compute frames as `round(seconds * frame_rate)`, copy every
`file_path` and resolution token by selection (never retyping), and hand-assemble the
TRACK_DATA the way Step 4 describes. Manual assembly is error-prone — prefer getting
Python running (any of `py -3`, `python`, a `.venv`) over hand-building the plan.