11 KiB
Workflow wiring: render ALL scenes in one run + the LLM slot
Verified against ComfyUI-Easy-Media source (nodes/basic.py, nodes/minimax.py,
nodes/project.py, utils/llm_api.py) and example_workflows/MinimaxH3_ForLoop_MultiShot.json.
Why only scene 1 renders
The render chain reads the timeline through easy multiTrackTaskOutput, which outputs
one task segment per pass, picked by its task_index int input. Default 0 →
the first clip only. The number of scenes inside the pasted prompt_override (plain
pipe text or the JSON payload) does not change this: the paste is complete (all scenes
are in it); the loop lives in the workflow, not in the paste. Nothing needs to be added
to it for multi-shot.
Side facts (same node, verified in source):
task_index = -1switches the node to full-timeline mode: window 0..720, LENGTH = all frames snapped to the H3 17k+5 grid (720 → 719), media from all segments. What the prompt becomes depends onprompt_format— see the single-pass section below (promptRelaymakes it render all scenes in one pass — which also puts every frame into ONE sampler call; OOM ceiling below).easy multiTrackInfoOutput(tracks_info in) exposesTASK_COUNT= number of task segments — 3 for the Sam payload. Also WIDTH/HEIGHT/TOTAL_FRAMES/FPS.
The prompt_override input takes TWO forms (proven)
MultiTrackEditor.execute (nodes/basic.py L1954-1957): with format = "MiniMax" it
calls is_minimax_prompt_override(prompt_override) FIRST. Only a payload with
"type": "minimax_prompt_override" and a prompts list takes the JSON path
(utils/prompt_override.py parse_minimax_prompt_override). Anything else with a value
falls to the generic |-text parser — which is a legitimate form in its own right, not
just a failure mode (track-schema.md "plain text form"):
- Plain pipe text
scene1|scene2|scene3→ N task segments, evenly sliced over the editor'stotal_length(proven by executing the pack's code,scripts/prove_plain_text_override.py). This is the DEFAULT form — this gate'sprompt_override.txt— with optional[start-end(,type)]markers per scene for explicit frame ranges / non-t2v types (examples/prompt_override_r2v_refmod.txt). - A full TRACKS_INFO JSON contains no
|→ ONE segment whose prompt is the literal JSON dump (STEP A ofprove_relay_chain.py): the garbage single-clip failure. So: paste plain text (default) or the minimalminimax_prompt_overrideJSON payload (media tokens / continuity modes / locks) intoprompt_override; full TRACKS_INFO belongs only intrack_data.
Single-pass timeline render: task_index=-1 + prompt_format=promptRelay (short timelines only)
This is the pattern the user's own graph (my timeline ref.json) is wired for —
no ForLoop, one sampler pass, all scenes in one generation. It renders, but only
when the WHOLE timeline fits in one pass — the OOM ceiling at the end of this
section is a hard limit:
- Node 186 (String →
prompt_override): the minimal payload —type+promptsare the only required keys (track-schema.md); optional keys only when non-default. easy multiTrackTaskOutput:task_index = -1ANDprompt_format = "promptRelay".- Everything else stays as-is: TYPE output is the payload's
generation_typestring verbatim (task_typeis set from it in the override path, and the parser's code default isr2v— so even the minimal 2-key payload renders TYPEr2v→ matchLine →multi_frames), LENGTH output becomes 719, one SamplerCustomAdvanced pass renders the whole timeline.
With promptRelay + full-timeline mode, TaskOutput builds the prompt from ALL
task segments (_format_marker_task_prompt_relay), each scene suffixed with its
native H3 frame range, joined by |:
<scene 1 text> [0-240] | <scene 2 text> [240-480] | <scene 3 text> [480-720]
SYSTEM_PROMPT output is empty in this mode; the ranges are the H3 timeline
syntax (_FRAME_RANGE_RE parses the same [s-e] / [s-e,s-type] tokens in the
override parser). Proven by executing the repo's own functions on the Sam
payload (STEP C: 3 ranges, length 719, type t2v).
Both settings are required — task_index = -1 with prompt_format = "default"
keeps only the FIRST task's prompt (STEP D): a 719-frame latent conditioned on
scene 1 alone.
OOM ceiling (user-verified 2026-11): one pass = the ENTIRE timeline's frames
in a single sampler call. At the Sam story's 3 × 240 = 720 total frames the
render attempted one ~30 s clip and died with an out-of-memory error — and 720
is also ~2× the trained range (~124–362 frames, node tooltip: "longer is
untested"), so quality was never validated that far either. The pack's own
long-sequence plumbing (multi_frames, BlockSparseAttention min_tokens 12288)
does not remove the VRAM cost of one 720-frame pass on a consumer card.
Rule: use task_index = -1 + promptRelay only when the whole timeline stays
inside the trained range — total ≤ ~360 frames, i.e. two short scenes. Anything
longer: render per clip (next section) and join the files.
The single-run loop (from the pack's own ForLoop example)
One queue run renders every clip and concatenates them:
editor.TRACKS_INFO ─→ multiTrackInfoOutput ─→ TASK_COUNT ─→ forLoopStart.total
editor.TRACKS_INFO ─→ multiTrackTaskOutput (images/audio/video too)
forLoopStart.index ─→ multiTrackTaskOutput.task_index
multiTrackTaskOutput.SYSTEM_PROMPT/USER_PROMPT/TYPE/LENGTH/IMAGES/AUDIO ─→ render chain
render chain → saveVideo.file_path ─→ batchAnything ─→ forLoopEnd.initial_value1
forLoopEnd.value1 ─→ mergeVideosFromPaths ─→ SaveVideo
The example adds two math nodes so a single PrimitiveInt k (default -1) chooses:
total = (k<=-1)*TASK_COUNT + (k>-1)*1task_index = (k<=-1)*loop_index + (k>-1)*k
k = -1 → loop all segments (all scenes, one run). k = 2 → render only segment 2.
Manual fallback without any rewiring (verified in source, nodes/basic.py
MultiTrackTaskOutput): keep prompt_format = "promptRelay" and just set
task_index to 0, 1, 2 — one queue run each. The all-tasks relay fires only when
output_full_timeline is set (-1) or the segment's own prompt contains [s-e]
markers (marker_mode); for a marker-free payload, promptRelay at
task_index = k formats the ONE selected clip as <scene k text> [start-end]
with LENGTH = that clip's frame count (240 — inside the trained range). default
gives the bare scene text if the range suffix is not wanted. Join the files.
Per-iteration the task output gives LENGTH ≈ frames for that clip only (240 frames = 10 s — inside the model's trained range), so each loop pass is a normal single-clip generation.
The LLM slot (where your own LLM connects)
In the ForLoop example the per-scene prompt may pass through an LLM before rendering:
TaskOutput.SYSTEM_PROMPT ─→ [LLM node].system_prompt
TaskOutput.USER_PROMPT ─→ [LLM node].custom_prompt
TaskOutput.IMAGES ─→ [LLM node].images (optional, vision only)
[LLM node].output ─→ ifElse("Using Prompt Enhance") ─→ cleanGpuUsed ─→ render.prompt
ifElse.on_false = raw TaskOutput.USER_PROMPT (bypass switch)
The example's LLM is llama_cpp_instruct_adv + llama_cpp_model_loader (a local GGUF).
That node is replaceable — it is just "system+user (+images) in, one string out".
Three ways to fill the slot:
Option A — no LLM node at all (default for this skill)
The skill already wrote every scene prompt (identity text, continuity rules, LoRA
trigger) into the paste (prompt_override.txt, or prompt_override.json for the JSON
form). Leave the enhance toggle OFF: raw
USER_PROMPT flows to the render node each pass. The "LLM that made all the scenes"
is this skill, outside ComfyUI.
Option B — the pack's built-in enhancer with any OpenAI-compatible server
Add easy multiTrackPromptEnhancer inside the loop:
system_prompt← TaskOutput.SYSTEM_PROMPT,user_prompt← TaskOutput.USER_PROMPT,type← TaskOutput.TYPE,length← TaskOutput.LENGTH, images/audio/video optional.model=openai_compatible (local); setbase_urlto ANY OpenAI-compatible Chat Completions server — Ollamahttp://127.0.0.1:11434/v1, LM Studiohttp://127.0.0.1:1234/v1, vLLM, llama.cpp server, or a hosted endpoint (empty base_url readsOPENAI_LOCAL_BASE_URLfrom the pack's config.yaml, elsehttp://127.0.0.1:8080/v1).apikey/model_nameoptional depending on server. Built-in presets also include MiniMax H3-Context-IR (v2/h3_context_ir), bigmodel.cn (GLM) and RunningHub LLM keys from config.yaml.- Output
PROMPT→ render node's prompt.
Option C — any ComfyUI LLM node in the loop
Keep the example's wiring shape and swap node 86 for the preferred LLM node
(system_prompt + user_prompt inputs, STRING output). prompt_format on TaskOutput
can stay api; setting it to llm instead makes USER_PROMPT a single ready-made
chat transcript for LLM nodes that have only one prompt input.
Local project path (alternative, no loop nodes)
easy multitrackProject (editor tracks_info in, segment_count = -1) builds and runs
every segment itself in one queue — see MiniMaxH3_v1.3.1_Easy_Project_AllInOne.json.
It contains no LLM node; prompts come from the timeline JSON. This is the pack's own
agent-driven setup: an external LLM (agent) writes the whole track_data, then one
run renders it. The pack even ships skills/easy-media-multitrack-workflow +
patch_workflow.py for exactly that flow (our SKILL.md step 5B).
The built-in LLM system prompts (published, not locked)
The rewrite-LLM contract ships in the open in the pack:
utils/prompt_builder.py — MINIMAX_BASE_PROMPT (~15.7 KB, "Video Prompt Writing
Guide (T2VA / I2VA / FL2VA / L2VA)") and MINIMAX_REF_PROMPT (~23.5 KB, "Full-
Reference Mode Rewrite Output Format Guide" — six-section rewrite with
<Subject N> / <Picture N> / <Video N> / <Audio N> labels).
Selection (build_prompt_request, MiniMax branch):
task_modedefault/l2v(our t2v payloads) →MINIMAX_BASE_PROMPTtask_moderef/edit→MINIMAX_REF_PROMPT- a non-empty per-segment
system_promptoverrides both; with an empty one the context-segment continuity rules ride the MiniMax rewrite API server-side (entry rule: not-first segment usingcontext; tail rule: next segment usescontext) — per the pack's ownmultitrack-schema.md.
Delivery: the system prompt only leaves TaskOutput via prompt_format
api or llm (default / promptRelay output an empty SYSTEM_PROMPT).
build_llm_prompt folds system+user into one transcript for single-input LLM
nodes. To use the user's own LLM with the same behavior: paste
minimax_base_prompt.txt (or the ref guide for ref-mode work) into the LLM
node's system prompt field and wire TaskOutput SYSTEM_PROMPT/USER_PROMPT into
it — or leave the LLM's system prompt empty and pass TaskOutput.SYSTEM_PROMPT
through, which already carries the selected guide verbatim.