feat: simple hidden half-res sampling, restored at the video combine

Single-pass projects condition and sample at exactly half the
editor-selected resolution (plain integer halving, no dimension
policies). Moxie Project Video Combine restores the exact editor
resolution on the combined frames right before its images output,
driven by the fresh project manifest so a stale frontend widget can
never misgate it.

- utils/rtx_vsr.py: RTX VSR core (Apache 2.0, from
  Comfy-Org/Nvidia_RTX_Nodes_ComfyUI) + Lanczos fallback; the gate is
  simply 'frames differ from the manifest resolution'
- nodes/project.py: halve first-pass dims for single mode only;
  dual-pass, SelfLift, passthrough and audio-only keep the editor size
- moxie_combine.py: restore before the images output
- scripts/test_rtx_upscale_integration.py: 17 standalone checks
This commit is contained in:
Z User 2026-09-27 01:18:33 +00:00
parent f94ee3e40b
commit 563322a3e2
5 changed files with 408 additions and 6 deletions

View File

@ -6,6 +6,7 @@
### ⚡ Improvements ### ⚡ Improvements
- **MultiTrack Project / Moxie Project Video Combine**: Single-pass projects now condition and sample at half the editor-selected resolution internally, and Moxie Project Video Combine restores the exact editor-selected resolution on the combined frames right before its `images` output (RTX Video Super Resolution via the optional `nvidia-vfx` package when available, with an automatic Lanczos fallback otherwise). The editor resolution therefore stays the final output resolution while the sampling chain runs smaller and faster. Dual-pass, SelfLift, and passthrough modes are unchanged.
- **Moxie Multimedia Loader**: The `lora_name` combo now defaults to the bundled turbo LoRA (`MM3step.safetensors`) so a fresh clone of the pack works out of the box, with a `none` entry to disable LoRA. Previously the default was a `"<no files found>"` placeholder, which silently skipped LoRA application even when a LoRA was bundled. - **Moxie Multimedia Loader**: The `lora_name` combo now defaults to the bundled turbo LoRA (`MM3step.safetensors`) so a fresh clone of the pack works out of the box, with a `none` entry to disable LoRA. Previously the default was a `"<no files found>"` placeholder, which silently skipped LoRA application even when a LoRA was bundled.
- **Moxie Preview Override**: The `tiny_vae` combo now lists files from the node pack's own `models/vae_approx` folder first (shipping `MM-preview.safetensors` for MiniMax H3 previews as the default), falling back to ComfyUI's standard `models/vae_approx` directory. The folder is rescanned on every schema refresh, so newly dropped files appear without restarting ComfyUI. - **Moxie Preview Override**: The `tiny_vae` combo now lists files from the node pack's own `models/vae_approx` folder first (shipping `MM-preview.safetensors` for MiniMax H3 previews as the default), falling back to ComfyUI's standard `models/vae_approx` directory. The folder is rescanned on every schema refresh, so newly dropped files appear without restarting ComfyUI.
- **Moxie Multimedia Loader**: Locked the loader to the MiniMax H3 model set. The `clip_type` dropdown (which previously exposed every CLIP recipe in ComfyUI) was removed; the H3 Qwen3-VL text encoder is now always loaded with the `minimax` CLIP recipe. Any older ComfyUI build without `CLIPType.MINIMAX` raises a clear startup error instead of silently loading the wrong encoder. - **Moxie Multimedia Loader**: Locked the loader to the MiniMax H3 model set. The `clip_type` dropdown (which previously exposed every CLIP recipe in ComfyUI) was removed; the H3 Qwen3-VL text encoder is now always loaded with the `minimax` CLIP recipe. Any older ComfyUI build without `CLIPType.MINIMAX` raises a clear startup error instead of silently loading the wrong encoder.

View File

@ -4,6 +4,8 @@ from comfy_api.latest import io
from comfy_execution.graph_utils import ExecutionBlocker from comfy_execution.graph_utils import ExecutionBlocker
from .nodes.project import EasyMultiTrackProjectVideoCombine, TYPE_PROJECT_DATA from .nodes.project import EasyMultiTrackProjectVideoCombine, TYPE_PROJECT_DATA
from .utils.h3_project import safe_h3_project_name
from .utils.rtx_vsr import upscale_images_to_project_resolution
class MoxieCombine(EasyMultiTrackProjectVideoCombine): class MoxieCombine(EasyMultiTrackProjectVideoCombine):
@ -49,10 +51,16 @@ class MoxieCombine(EasyMultiTrackProjectVideoCombine):
return io.NodeOutput(video, video, video, video, video) return io.NodeOutput(video, video, video, video, video)
components = video.get_components() components = video.get_components()
# Hidden half-res sampling contract: restore the editor-selected
# resolution right before the frames leave this node. Frames already
# at the project resolution pass through untouched.
images = upscale_images_to_project_resolution(
components.images, safe_h3_project_name(project_name)
)
return io.NodeOutput( return io.NodeOutput(
video, video,
prefix, prefix,
components.images, images,
components.audio, components.audio,
float(components.frame_rate), float(components.frame_rate),
) )

View File

@ -1169,13 +1169,24 @@ class EasyMultiTrackProject(io.ComfyNode):
"upscale_by must be a finite value greater than or equal to 1" "upscale_by must be a finite value greater than or equal to 1"
) )
first_pass_width = int(info["width"]) editor_width = int(info["width"])
first_pass_height = int(info["height"]) editor_height = int(info["height"])
audio_only = (first_pass_width, first_pass_height) == (32, 32) audio_only = (editor_width, editor_height) == (32, 32)
fps = float(info["frame_rate"]) fps = float(info["frame_rate"])
# Hidden half-res sampling (single-pass projects only): condition and
# sample at half the editor-selected resolution; Moxie Project Video
# Combine restores the exact editor resolution in pixel space before
# its images output (utils/rtx_vsr.py). Every other sampling mode
# keeps the editor size exactly as configured.
first_pass_width, first_pass_height = editor_width, editor_height
if sampling_mode == "single" and not audio_only:
half_width = max(32, editor_width // 2)
half_height = max(32, editor_height // 2)
if (half_width, half_height) != (32, 32):
first_pass_width, first_pass_height = half_width, half_height
target_width, target_height = h3_second_pass_dimensions( target_width, target_height = h3_second_pass_dimensions(
first_pass_width, editor_width,
first_pass_height, editor_height,
run_second_pass, run_second_pass,
upscale_by, upscale_by,
) )

View File

@ -0,0 +1,184 @@
"""Standalone sanity checks for the hidden half-res output restore.
Runs without ComfyUI or torch: minimal stubs exercise the gating logic of
utils/rtx_vsr.py and verify the plain-halving gate in nodes/project.py plus
the MoxieCombine wiring.
"""
from __future__ import annotations
import ast
import importlib
import py_compile
import sys
import types
from pathlib import Path
PACK = Path(__file__).resolve().parents[1]
failures: list[str] = []
def check(name: str, condition: bool) -> None:
status = "PASS" if condition else "FAIL"
print(f"[{status}] {name}")
if not condition:
failures.append(name)
# --- stub module surface ----------------------------------------------------
class FakeTensor:
def __init__(self, shape):
self.shape = tuple(shape)
self.ndim = len(self.shape)
self.device = "cpu"
self.dtype = "float32"
def __getitem__(self, item):
return self
def movedim(self, *args, **kwargs):
return self
torch_stub = types.ModuleType("torch")
torch_stub.Tensor = FakeTensor
torch_stub.empty = lambda *a, **k: FakeTensor(a[0])
sys.modules["torch"] = torch_stub
upscale_calls: list[tuple] = []
comfy_stub = types.ModuleType("comfy")
comfy_utils_stub = types.ModuleType("comfy.utils")
def _fake_common_upscale(samples, width, height, method, crop):
upscale_calls.append((width, height, method, crop))
return samples
comfy_utils_stub.common_upscale = _fake_common_upscale
comfy_stub.utils = comfy_utils_stub
sys.modules["comfy"] = comfy_stub
sys.modules["comfy.utils"] = comfy_utils_stub
MANIFEST: dict = {"width": 0, "height": 0}
h3_stub = types.ModuleType("utils.h3_project")
h3_stub.safe_h3_project_name = lambda value: str(value or "default")
def _fake_load(project_name):
if MANIFEST.get("missing"):
raise FileNotFoundError("no project")
return dict(MANIFEST)
h3_stub.load_h3_project_data = _fake_load
utils_pkg = types.ModuleType("utils")
utils_pkg.__path__ = [str(PACK / "utils")]
sys.modules["utils"] = utils_pkg
sys.modules["utils.h3_project"] = h3_stub
rtx_vsr = importlib.import_module("utils.rtx_vsr")
# 1. half-res frames are restored to the exact manifest size (lanczos here,
# since nvidia-vfx is absent in the sandbox)
MANIFEST.update({"width": 1920, "height": 1080})
upscale_calls.clear()
rtx_vsr.upscale_images_to_project_resolution(FakeTensor((4, 528, 960, 3)), "proj")
check(
"half-res frames restored to manifest size",
upscale_calls == [(1920, 1080, "lanczos", "disabled")],
)
# 2. frames already at the project resolution pass through untouched
upscale_calls.clear()
rtx_vsr.upscale_images_to_project_resolution(FakeTensor((4, 1080, 1920, 3)), "proj")
check("full-res frames pass through", upscale_calls == [])
# 3. missing manifest passes through
MANIFEST.update({"missing": True})
upscale_calls.clear()
rtx_vsr.upscale_images_to_project_resolution(FakeTensor((4, 528, 960, 3)), "proj")
check("missing manifest passes through", upscale_calls == [])
MANIFEST.pop("missing")
# 4. zero and audio-only sentinel manifests pass through
for width, height in ((0, 0), (32, 32)):
MANIFEST.update({"width": width, "height": height})
upscale_calls.clear()
rtx_vsr.upscale_images_to_project_resolution(FakeTensor((4, 528, 960, 3)), "proj")
check(f"manifest {width}x{height} passes through", upscale_calls == [])
MANIFEST.update({"width": 1920, "height": 1080})
# 5. non-tensor and empty-batch inputs pass through
check(
"non-tensor passthrough",
rtx_vsr.upscale_images_to_project_resolution("nope", "proj") == "nope",
)
check(
"empty batch passthrough",
rtx_vsr.upscale_images_to_project_resolution(
FakeTensor((0, 528, 960, 3)), "proj"
).shape[0] == 0,
)
# 6. rtx_vsr_upscale identity early-returns (no upscale machinery touched)
same = FakeTensor((2, 1080, 1920, 3))
check("vsr same-size identity", rtx_vsr.rtx_vsr_upscale(same, 1920, 1080) is same)
zero = FakeTensor((0, 528, 960, 3))
check("vsr empty identity", rtx_vsr.rtx_vsr_upscale(zero, 1920, 1080) is zero)
# 7. nodes/project.py carries the plain-halving gate for single-pass only
project_source = (PACK / "nodes" / "project.py").read_text(encoding="utf-8")
check(
"single-pass half-res gate present",
"editor_width // 2" in project_source
and 'sampling_mode == "single"' in project_source,
)
check(
"no 16px snap policy left behind",
"half_up" not in project_source
and "h3_halfres_sampling_dimensions" not in project_source,
)
# 8. MoxieCombine wires the restore right before the images output
combine_source = (PACK / "moxie_combine.py").read_text(encoding="utf-8")
check(
"combine restores before images output",
"upscale_images_to_project_resolution" in combine_source,
)
check(
"combine passes the safe project name",
"safe_h3_project_name(project_name)" in combine_source,
)
# 9. heavy imports stay lazy in rtx_vsr (pack import stays light)
tree = ast.parse((PACK / "utils" / "rtx_vsr.py").read_text(encoding="utf-8"))
module_level: list[str] = []
for node in tree.body:
if isinstance(node, ast.Import):
module_level.extend(alias.name for alias in node.names)
elif isinstance(node, ast.ImportFrom) and node.level == 0:
module_level.append(node.module or "")
check(
"no torch/nvvfx/comfy imports at module level",
not any(
name.split(".")[0] in {"torch", "nvvfx", "comfy"} for name in module_level
),
)
# 10. everything touched compiles
for rel in ("nodes/project.py", "moxie_combine.py", "utils/rtx_vsr.py"):
try:
py_compile.compile(str(PACK / rel), doraise=True)
check(f"py_compile {rel}", True)
except py_compile.PyCompileError as error:
check(f"py_compile {rel}", False)
print(f" {error}")
print()
if failures:
print(f"{len(failures)} CHECK(S) FAILED")
sys.exit(1)
print("ALL CHECKS PASSED")

198
utils/rtx_vsr.py Normal file
View File

@ -0,0 +1,198 @@
"""Hidden half-res output restore for Moxie MultiTrack projects.
Single-pass project sampling conditions and samples at half the
editor-selected resolution (see nodes/project.py). This module owns the
other half of that contract: Moxie Project Video Combine restores the exact
editor-selected resolution right before its images output, with no
user-visible wiring anywhere.
Hardware path adapted from NVIDIA's RTX Video Super Resolution ComfyUI node
(Apache License 2.0):
https://github.com/Comfy-Org/Nvidia_RTX_Nodes_ComfyUI
Copyright 2025 NVIDIA Corporation
It drives the ``nvidia-vfx`` pip package (RTX Video Super Resolution) per
frame via DLPack. When the package or a supported GPU is unavailable, the
module falls back to ComfyUI's Lanczos resize so the delivered frame size is
always exact and rendering never fails because of the upscaler.
"""
from __future__ import annotations
from typing import Any
__all__ = [
"rtx_vsr_upscale",
"upscale_images_to_project_resolution",
]
def _log(message: str) -> None:
try:
from .log import log_node_info
except Exception: # pragma: no cover - standalone tooling without the pack
print(f"[Moxie] {message}", flush=True) # noqa: T201
return
log_node_info("MultiTrack Output", message)
_FALLBACK_NOTICE_SHOWN = False
def _notice_fallback(message: str) -> None:
global _FALLBACK_NOTICE_SHOWN
if _FALLBACK_NOTICE_SHOWN:
return
_FALLBACK_NOTICE_SHOWN = True
_log(message)
def _lanczos_upscale(images: Any, output_width: int, output_height: int) -> Any:
import comfy.utils
samples = images[..., :3].movedim(-1, 1)
upscaled = comfy.utils.common_upscale(
samples, output_width, output_height, "lanczos", "disabled"
)
return upscaled.movedim(1, -1)
def rtx_vsr_upscale(
images: Any,
output_width: int,
output_height: int,
quality: str = "ULTRA",
) -> Any:
"""Upscale an IMAGE batch to the exact target size, hardware first.
Frames are processed per frame through RTX Video Super Resolution
(adapted from Comfy-Org/Nvidia_RTX_Nodes_ComfyUI, Apache 2.0). Any
failure - missing package, unsupported GPU, VRAM pressure - degrades to
the Lanczos fallback instead of breaking the render, and the delivered
size is corrected to the exact target whenever the hardware snap misses.
"""
import torch
frame_count, in_height, in_width, channels = images.shape
if frame_count == 0 or (in_height, in_width) == (output_height, output_width):
return images
hardware_ready = channels == 3
if hardware_ready:
try:
import nvvfx
except Exception:
hardware_ready = False
_notice_fallback(
"Hardware upscale unavailable (install nvidia-vfx for RTX "
"acceleration); using Lanczos fallback."
)
if hardware_ready:
# /8 snap from the NVIDIA node; editor selections are 8-aligned so
# this is an identity operation for real project resolutions.
snapped_width = max(8, round(output_width / 8) * 8)
snapped_height = max(8, round(output_height / 8) * 8)
max_batch_pixels = 1024 * 1024 * 16
batch_size = max(
1, max_batch_pixels // max(1, snapped_width * snapped_height)
)
quality_levels = {
"LOW": nvvfx.effects.QualityLevel.LOW,
"MEDIUM": nvvfx.effects.QualityLevel.MEDIUM,
"HIGH": nvvfx.effects.QualityLevel.HIGH,
"ULTRA": nvvfx.effects.QualityLevel.ULTRA,
}
selected_quality = quality_levels.get(
str(quality).upper(), nvvfx.effects.QualityLevel.HIGH
)
out_tensor = torch.empty(
(frame_count, snapped_height, snapped_width, channels),
device=images.device,
dtype=images.dtype,
)
try:
with nvvfx.VideoSuperRes(selected_quality) as super_res:
super_res.output_width = snapped_width
super_res.output_height = snapped_height
super_res.load()
for start in range(0, frame_count, batch_size):
batch = images[start : start + batch_size]
batch_cuda = (
batch.cuda().permute(0, 3, 1, 2).float().contiguous()
)
for offset in range(batch_cuda.shape[0]):
result = super_res.run(batch_cuda[offset]).image
out_tensor[start + offset : start + offset + 1] = (
torch.from_dlpack(result).movedim(0, -1).unsqueeze(0)
)
except Exception as error:
hardware_ready = False
_notice_fallback(
f"Hardware upscale failed ({error}); using Lanczos fallback."
)
else:
if (snapped_height, snapped_width) != (output_height, output_width):
return _lanczos_upscale(out_tensor, output_width, output_height)
return out_tensor
return _lanczos_upscale(images, output_width, output_height)
def _project_output_resolution(project_name: Any) -> tuple[int, int] | None:
"""Editor-selected resolution stored in the project manifest, or ``None``.
The manifest is written from the editor selection at the start of every
run (nodes/project.py), so reading it fresh here can never see a stale
frontend widget value.
"""
try:
from .h3_project import load_h3_project_data, safe_h3_project_name
data = load_h3_project_data(safe_h3_project_name(project_name))
width = int(data.get("width") or 0)
height = int(data.get("height") or 0)
except Exception:
return None
if width <= 0 or height <= 0 or (width, height) == (32, 32):
return None
return width, height
def upscale_images_to_project_resolution(
images: Any,
project_name: Any,
) -> Any:
"""Restore the project's editor resolution on combined frames.
Used by Moxie Project Video Combine right before the ``images`` output.
Frames already at the project resolution - full-res renders, dual-pass,
SelfLift and passthrough outputs - pass through untouched; only frames
whose size differs from the manifest (the hidden half-res sampling
signature) are restored to the exact editor-selected resolution.
"""
try:
import torch
except ImportError: # pragma: no cover - torch is required in ComfyUI
return images
if (
not isinstance(images, torch.Tensor)
or images.ndim != 4
or images.shape[0] == 0
):
return images
target = _project_output_resolution(project_name)
if target is None:
return images
output_width, output_height = target
if (int(images.shape[2]), int(images.shape[1])) == (
output_width,
output_height,
):
return images
_log(
f"Output upscale: {int(images.shape[2])}x{int(images.shape[1])} -> "
f"{output_width}x{output_height}"
)
return rtx_vsr_upscale(images, output_width, output_height)