9.0 KiB
Workflow wiring: render ALL scenes in one run + the LLM slot
Verified against ComfyUI-Easy-Media source (nodes/basic.py, nodes/minimax.py,
nodes/project.py, utils/llm_api.py) and example_workflows/MinimaxH3_ForLoop_MultiShot.json.
Why only scene 1 renders
The render chain reads the timeline through easy multiTrackTaskOutput, which outputs
one task segment per pass, picked by its task_index int input. Default 0 →
the first clip only. The number of prompts inside the pasted prompt_override JSON
does not change this: the JSON is complete (all scenes are in it); the loop lives in
the workflow, not in the JSON. Nothing needs to be added to the payload for multi-shot.
Side facts (same node, verified in source):
task_index = -1switches the node to full-timeline mode: window 0..720, LENGTH = all frames snapped to the H3 17k+5 grid (720 → 719), media from all segments. What the prompt becomes depends onprompt_format— see the single-pass section below (promptRelaymakes it render all scenes in one pass).easy multiTrackInfoOutput(tracks_info in) exposesTASK_COUNT= number of task segments — 3 for the Sam payload. Also WIDTH/HEIGHT/TOTAL_FRAMES/FPS.
The prompt_override input requires the type field (proven)
MultiTrackEditor.execute (nodes/basic.py L1949-1957): with format = "MiniMax"
it calls is_minimax_prompt_override(prompt_override) FIRST. Only a payload with
"type": "minimax_prompt_override" and a prompts list passes
(utils/prompt_override.py parse_minimax_prompt_override). Any other non-empty
value — including a full TRACKS_INFO JSON — falls to the |-text parser, which
treats the ENTIRE string as one prompt: one segment 0..479 whose text is the
literal JSON. Proven by running the repo's parser on the captured workflow
(scripts/prove_relay_chain.py STEP A). So: paste the compact payload (with the
type field) into prompt_override; full TRACKS_INFO belongs only in track_data.
Single-pass timeline render: task_index=-1 + prompt_format=promptRelay
This is the pattern the user's own graph (my timeline ref.json) is wired for —
no ForLoop, one sampler pass, all scenes in one generation:
- Node 186 (String →
prompt_override): the compact payload WITHtypefield. easy multiTrackTaskOutput:task_index = -1ANDprompt_format = "promptRelay".- Everything else stays as-is: TYPE output stays
t2v(first task's explicit task_type → matchLine →multi_frames), LENGTH output becomes 719, one SamplerCustomAdvanced pass renders the whole timeline.
With promptRelay + full-timeline mode, TaskOutput builds the prompt from ALL
task segments (_format_marker_task_prompt_relay), each scene suffixed with its
native H3 frame range, joined by |:
<scene 1 text> [0-240] | <scene 2 text> [240-480] | <scene 3 text> [480-720]
SYSTEM_PROMPT output is empty in this mode; the ranges are the H3 timeline
syntax (_FRAME_RANGE_RE parses the same [s-e] / [s-e,s-type] tokens in the
override parser). Proven by executing the repo's own functions on the Sam
payload (STEP C: 3 ranges, length 719, type t2v).
Both settings are required — task_index = -1 with prompt_format = "default"
keeps only the FIRST task's prompt (STEP D): a 719-frame latent conditioned on
scene 1 alone.
Caveat from the node's own tooltip: the trained range is ~124–362 frames and
"longer is untested". 719 frames (~30 s) is the intended use of this graph shape
(multi_frames mode + BlockSparseAttention min_tokens 12288 exist for long
sequences), but if quality degrades, fall back to the ForLoop pattern above,
which renders each 240-frame clip inside the trained range.
The single-run loop (from the pack's own ForLoop example)
One queue run renders every clip and concatenates them:
editor.TRACKS_INFO ─→ multiTrackInfoOutput ─→ TASK_COUNT ─→ forLoopStart.total
editor.TRACKS_INFO ─→ multiTrackTaskOutput (images/audio/video too)
forLoopStart.index ─→ multiTrackTaskOutput.task_index
multiTrackTaskOutput.SYSTEM_PROMPT/USER_PROMPT/TYPE/LENGTH/IMAGES/AUDIO ─→ render chain
render chain → saveVideo.file_path ─→ batchAnything ─→ forLoopEnd.initial_value1
forLoopEnd.value1 ─→ mergeVideosFromPaths ─→ SaveVideo
The example adds two math nodes so a single PrimitiveInt k (default -1) chooses:
total = (k<=-1)*TASK_COUNT + (k>-1)*1task_index = (k<=-1)*loop_index + (k>-1)*k
k = -1 → loop all segments (all scenes, one run). k = 2 → render only segment 2.
Manual fallback without rewiring: queue once per index (0, 1, 2) and join the files.
Per-iteration the task output gives LENGTH ≈ frames for that clip only (240 frames = 10 s — inside the model's trained range), so each loop pass is a normal single-clip generation.
The LLM slot (where your own LLM connects)
In the ForLoop example the per-scene prompt may pass through an LLM before rendering:
TaskOutput.SYSTEM_PROMPT ─→ [LLM node].system_prompt
TaskOutput.USER_PROMPT ─→ [LLM node].custom_prompt
TaskOutput.IMAGES ─→ [LLM node].images (optional, vision only)
[LLM node].output ─→ ifElse("Using Prompt Enhance") ─→ cleanGpuUsed ─→ render.prompt
ifElse.on_false = raw TaskOutput.USER_PROMPT (bypass switch)
The example's LLM is llama_cpp_instruct_adv + llama_cpp_model_loader (a local GGUF).
That node is replaceable — it is just "system+user (+images) in, one string out".
Three ways to fill the slot:
Option A — no LLM node at all (default for this skill)
The skill already wrote every scene prompt (identity text, continuity rules, LoRA
trigger) into prompt_override.json. Leave the enhance toggle OFF: raw
USER_PROMPT flows to the render node each pass. The "LLM that made all the scenes"
is this skill, outside ComfyUI.
Option B — the pack's built-in enhancer with any OpenAI-compatible server
Add easy multiTrackPromptEnhancer inside the loop:
system_prompt← TaskOutput.SYSTEM_PROMPT,user_prompt← TaskOutput.USER_PROMPT,type← TaskOutput.TYPE,length← TaskOutput.LENGTH, images/audio/video optional.model=openai_compatible (local); setbase_urlto ANY OpenAI-compatible Chat Completions server — Ollamahttp://127.0.0.1:11434/v1, LM Studiohttp://127.0.0.1:1234/v1, vLLM, llama.cpp server, or a hosted endpoint (empty base_url readsOPENAI_LOCAL_BASE_URLfrom the pack's config.yaml, elsehttp://127.0.0.1:8080/v1).apikey/model_nameoptional depending on server. Built-in presets also include MiniMax H3-Context-IR (v2/h3_context_ir), bigmodel.cn (GLM) and RunningHub LLM keys from config.yaml.- Output
PROMPT→ render node's prompt.
Option C — any ComfyUI LLM node in the loop
Keep the example's wiring shape and swap node 86 for the preferred LLM node
(system_prompt + user_prompt inputs, STRING output). prompt_format on TaskOutput
can stay api; setting it to llm instead makes USER_PROMPT a single ready-made
chat transcript for LLM nodes that have only one prompt input.
Local project path (alternative, no loop nodes)
easy multitrackProject (editor tracks_info in, segment_count = -1) builds and runs
every segment itself in one queue — see MiniMaxH3_v1.3.1_Easy_Project_AllInOne.json.
It contains no LLM node; prompts come from the timeline JSON. This is the pack's own
agent-driven setup: an external LLM (agent) writes the whole track_data, then one
run renders it. The pack even ships skills/easy-media-multitrack-workflow +
patch_workflow.py for exactly that flow (our SKILL.md step 5B).
The built-in LLM system prompts (published, not locked)
The rewrite-LLM contract ships in the open in the pack:
utils/prompt_builder.py — MINIMAX_BASE_PROMPT (~15.7 KB, "Video Prompt Writing
Guide (T2VA / I2VA / FL2VA / L2VA)") and MINIMAX_REF_PROMPT (~23.5 KB, "Full-
Reference Mode Rewrite Output Format Guide" — six-section rewrite with
<Subject N> / <Picture N> / <Video N> / <Audio N> labels).
Selection (build_prompt_request, MiniMax branch):
task_modedefault/l2v(our t2v payloads) →MINIMAX_BASE_PROMPTtask_moderef/edit→MINIMAX_REF_PROMPT- a non-empty per-segment
system_promptoverrides both; with an empty one the context-segment continuity rules ride the MiniMax rewrite API server-side (entry rule: not-first segment usingcontext; tail rule: next segment usescontext) — per the pack's ownmultitrack-schema.md.
Delivery: the system prompt only leaves TaskOutput via prompt_format
api or llm (default / promptRelay output an empty SYSTEM_PROMPT).
build_llm_prompt folds system+user into one transcript for single-input LLM
nodes. To use the user's own LLM with the same behavior: paste
minimax_base_prompt.txt (or the ref guide for ref-mode work) into the LLM
node's system prompt field and wire TaskOutput SYSTEM_PROMPT/USER_PROMPT into
it — or leave the LLM's system prompt empty and pass TaskOutput.SYSTEM_PROMPT
through, which already carries the selected guide verbatim.