track-gate: relay single-pass OOMs at 720 total frames (user-verified) - demote task_index=-1+promptRelay to <=~360-frame timelines; per-clip task_index=0/1/2 with promptRelay kept is the production path (relay-at-index formats the single clip's [s-e] range, verified in basic.py); TYPE line updated for the r2v flip

This commit is contained in:
Z User 2026-11-01 20:32:58 +00:00
parent 9b197ae024
commit 20143da60e

View File

@ -16,7 +16,8 @@ Side facts (same node, verified in source):
- `task_index = -1` switches the node to **full-timeline mode**: window 0..720, - `task_index = -1` switches the node to **full-timeline mode**: window 0..720,
LENGTH = all frames snapped to the H3 17k+5 grid (720 → 719), media from all LENGTH = all frames snapped to the H3 17k+5 grid (720 → 719), media from all
segments. What the prompt becomes depends on `prompt_format` — see the segments. What the prompt becomes depends on `prompt_format` — see the
single-pass section below (`promptRelay` makes it render all scenes in one pass). single-pass section below (`promptRelay` makes it render all scenes in one pass —
which also puts every frame into ONE sampler call; OOM ceiling below).
- `easy multiTrackInfoOutput` (tracks_info in) exposes `TASK_COUNT` = number of task - `easy multiTrackInfoOutput` (tracks_info in) exposes `TASK_COUNT` = number of task
segments — 3 for the Sam payload. Also WIDTH/HEIGHT/TOTAL_FRAMES/FPS. segments — 3 for the Sam payload. Also WIDTH/HEIGHT/TOTAL_FRAMES/FPS.
@ -32,16 +33,19 @@ literal JSON. Proven by running the repo's parser on the captured workflow
(`scripts/prove_relay_chain.py` STEP A). So: paste the compact payload (with the (`scripts/prove_relay_chain.py` STEP A). So: paste the compact payload (with the
type field) into `prompt_override`; full TRACKS_INFO belongs only in `track_data`. type field) into `prompt_override`; full TRACKS_INFO belongs only in `track_data`.
## Single-pass timeline render: task_index=-1 + prompt_format=promptRelay ## Single-pass timeline render: task_index=-1 + prompt_format=promptRelay (short timelines only)
This is the pattern the user's own graph (`my timeline ref.json`) is wired for — This is the pattern the user's own graph (`my timeline ref.json`) is wired for —
no ForLoop, one sampler pass, all scenes in one generation: no ForLoop, one sampler pass, all scenes in one generation. It renders, but only
when the WHOLE timeline fits in one pass — the OOM ceiling at the end of this
section is a hard limit:
1. Node 186 (String → `prompt_override`): the compact payload WITH `type` field. 1. Node 186 (String → `prompt_override`): the compact payload WITH `type` field.
2. `easy multiTrackTaskOutput`: `task_index = -1` AND `prompt_format = "promptRelay"`. 2. `easy multiTrackTaskOutput`: `task_index = -1` AND `prompt_format = "promptRelay"`.
3. Everything else stays as-is: TYPE output stays `t2v` (first task's explicit 3. Everything else stays as-is: TYPE output stays the first task's task_type
task_type → matchLine → `multi_frames`), LENGTH output becomes 719, one (`t2v` for the pure-t2v payload; `r2v` since the ref flip → matchLine →
SamplerCustomAdvanced pass renders the whole timeline. `multi_frames`), LENGTH output becomes 719, one SamplerCustomAdvanced pass
renders the whole timeline.
With `promptRelay` + full-timeline mode, TaskOutput builds the prompt from ALL With `promptRelay` + full-timeline mode, TaskOutput builds the prompt from ALL
task segments (`_format_marker_task_prompt_relay`), each scene suffixed with its task segments (`_format_marker_task_prompt_relay`), each scene suffixed with its
@ -60,11 +64,17 @@ Both settings are required — `task_index = -1` with `prompt_format = "default"
keeps only the FIRST task's prompt (STEP D): a 719-frame latent conditioned on keeps only the FIRST task's prompt (STEP D): a 719-frame latent conditioned on
scene 1 alone. scene 1 alone.
Caveat from the node's own tooltip: the trained range is ~124–362 frames and **OOM ceiling (user-verified 2026-11):** one pass = the ENTIRE timeline's frames
"longer is untested". 719 frames (~30 s) is the intended use of this graph shape in a single sampler call. At the Sam story's 3 × 240 = 720 total frames the
(`multi_frames` mode + BlockSparseAttention `min_tokens 12288` exist for long render attempted one ~30 s clip and died with an out-of-memory error — and 720
sequences), but if quality degrades, fall back to the ForLoop pattern above, is also ~2× the trained range (~124–362 frames, node tooltip: "longer is
which renders each 240-frame clip inside the trained range. untested"), so quality was never validated that far either. The pack's own
long-sequence plumbing (`multi_frames`, BlockSparseAttention `min_tokens 12288`)
does not remove the VRAM cost of one 720-frame pass on a consumer card.
Rule: use `task_index = -1` + `promptRelay` only when the whole timeline stays
inside the trained range — total ≤ ~360 frames, i.e. two short scenes. Anything
longer: render per clip (next section) and join the files.
## The single-run loop (from the pack's own ForLoop example) ## The single-run loop (from the pack's own ForLoop example)
@ -85,7 +95,15 @@ The example adds two math nodes so a single PrimitiveInt `k` (default `-1`) choo
- `task_index = (k<=-1)*loop_index + (k>-1)*k` - `task_index = (k<=-1)*loop_index + (k>-1)*k`
`k = -1` → loop all segments (all scenes, one run). `k = 2` → render only segment 2. `k = -1` → loop all segments (all scenes, one run). `k = 2` → render only segment 2.
Manual fallback without rewiring: queue once per index (0, 1, 2) and join the files.
Manual fallback without any rewiring (verified in source, `nodes/basic.py`
`MultiTrackTaskOutput`): keep `prompt_format = "promptRelay"` and just set
`task_index` to 0, 1, 2 — one queue run each. The all-tasks relay fires only when
`output_full_timeline` is set (-1) or the segment's own prompt contains `[s-e]`
markers (`marker_mode`); for a marker-free payload, promptRelay at
`task_index = k` formats the ONE selected clip as `<scene k text> [start-end]`
with LENGTH = that clip's frame count (240 — inside the trained range). `default`
gives the bare scene text if the range suffix is not wanted. Join the files.
Per-iteration the task output gives LENGTH ≈ frames for that clip only (240 frames = Per-iteration the task output gives LENGTH ≈ frames for that clip only (240 frames =
10 s — inside the model's trained range), so each loop pass is a normal single-clip 10 s — inside the model's trained range), so each loop pass is a normal single-clip