diff --git a/easy-media-track-gate/references/workflow_wiring.md b/easy-media-track-gate/references/workflow_wiring.md index 9fb61f0..3c4e752 100644 --- a/easy-media-track-gate/references/workflow_wiring.md +++ b/easy-media-track-gate/references/workflow_wiring.md @@ -16,7 +16,8 @@ Side facts (same node, verified in source): - `task_index = -1` switches the node to **full-timeline mode**: window 0..720, LENGTH = all frames snapped to the H3 17k+5 grid (720 → 719), media from all segments. What the prompt becomes depends on `prompt_format` — see the - single-pass section below (`promptRelay` makes it render all scenes in one pass). + single-pass section below (`promptRelay` makes it render all scenes in one pass — + which also puts every frame into ONE sampler call; OOM ceiling below). - `easy multiTrackInfoOutput` (tracks_info in) exposes `TASK_COUNT` = number of task segments — 3 for the Sam payload. Also WIDTH/HEIGHT/TOTAL_FRAMES/FPS. @@ -32,16 +33,19 @@ literal JSON. Proven by running the repo's parser on the captured workflow (`scripts/prove_relay_chain.py` STEP A). So: paste the compact payload (with the type field) into `prompt_override`; full TRACKS_INFO belongs only in `track_data`. -## Single-pass timeline render: task_index=-1 + prompt_format=promptRelay +## Single-pass timeline render: task_index=-1 + prompt_format=promptRelay (short timelines only) This is the pattern the user's own graph (`my timeline ref.json`) is wired for — -no ForLoop, one sampler pass, all scenes in one generation: +no ForLoop, one sampler pass, all scenes in one generation. It renders, but only +when the WHOLE timeline fits in one pass — the OOM ceiling at the end of this +section is a hard limit: 1. Node 186 (String → `prompt_override`): the compact payload WITH `type` field. 2. `easy multiTrackTaskOutput`: `task_index = -1` AND `prompt_format = "promptRelay"`. -3. Everything else stays as-is: TYPE output stays `t2v` (first task's explicit - task_type → matchLine → `multi_frames`), LENGTH output becomes 719, one - SamplerCustomAdvanced pass renders the whole timeline. +3. Everything else stays as-is: TYPE output stays the first task's task_type + (`t2v` for the pure-t2v payload; `r2v` since the ref flip → matchLine → + `multi_frames`), LENGTH output becomes 719, one SamplerCustomAdvanced pass + renders the whole timeline. With `promptRelay` + full-timeline mode, TaskOutput builds the prompt from ALL task segments (`_format_marker_task_prompt_relay`), each scene suffixed with its @@ -60,11 +64,17 @@ Both settings are required — `task_index = -1` with `prompt_format = "default" keeps only the FIRST task's prompt (STEP D): a 719-frame latent conditioned on scene 1 alone. -Caveat from the node's own tooltip: the trained range is ~124–362 frames and -"longer is untested". 719 frames (~30 s) is the intended use of this graph shape -(`multi_frames` mode + BlockSparseAttention `min_tokens 12288` exist for long -sequences), but if quality degrades, fall back to the ForLoop pattern above, -which renders each 240-frame clip inside the trained range. +**OOM ceiling (user-verified 2026-11):** one pass = the ENTIRE timeline's frames +in a single sampler call. At the Sam story's 3 × 240 = 720 total frames the +render attempted one ~30 s clip and died with an out-of-memory error — and 720 +is also ~2× the trained range (~124–362 frames, node tooltip: "longer is +untested"), so quality was never validated that far either. The pack's own +long-sequence plumbing (`multi_frames`, BlockSparseAttention `min_tokens 12288`) +does not remove the VRAM cost of one 720-frame pass on a consumer card. + +Rule: use `task_index = -1` + `promptRelay` only when the whole timeline stays +inside the trained range — total ≤ ~360 frames, i.e. two short scenes. Anything +longer: render per clip (next section) and join the files. ## The single-run loop (from the pack's own ForLoop example) @@ -85,7 +95,15 @@ The example adds two math nodes so a single PrimitiveInt `k` (default `-1`) choo - `task_index = (k<=-1)*loop_index + (k>-1)*k` `k = -1` → loop all segments (all scenes, one run). `k = 2` → render only segment 2. -Manual fallback without rewiring: queue once per index (0, 1, 2) and join the files. + +Manual fallback without any rewiring (verified in source, `nodes/basic.py` +`MultiTrackTaskOutput`): keep `prompt_format = "promptRelay"` and just set +`task_index` to 0, 1, 2 — one queue run each. The all-tasks relay fires only when +`output_full_timeline` is set (-1) or the segment's own prompt contains `[s-e]` +markers (`marker_mode`); for a marker-free payload, promptRelay at +`task_index = k` formats the ONE selected clip as ` [start-end]` +with LENGTH = that clip's frame count (240 — inside the trained range). `default` +gives the bare scene text if the range suffix is not wanted. Join the files. Per-iteration the task output gives LENGTH ≈ frames for that clip only (240 frames = 10 s — inside the model's trained range), so each loop pass is a normal single-clip