refactor: remove the LoRA pipeline; distill weights ship baked into the model

The 3-step distill (speedup) LoRA is now baked into the diffusion model
itself, so the Moxie Multimedia Loader no longer applies any LoRA:

- moxie_loader.py: drop the models/loras scan, _auto_lora(), the module
  level LoRA cache and _apply_lora(); the load chain is now
  diffusion model -> attention backend -> CLIP -> VAEs. A leftover file
  in models/loras is silently ignored and the folder can be deleted.
- moxie_project.py / moxie_preview.py: tooltips reference the baked-in
  distill instead of the LoRA.
- tests + standalone loader script: assert no load_lora_for_models call
  happens (the comfy.sd stub now raises if the loader tries), and fix
  the pre-existing MMnode.tiny_vae stub gap (missing pack_vae_approx_files)
  that broke the whole test module at fixture setup.
- README: models table drops the Turbo LoRA row; CHANGELOG entry added.

Node schema, outputs and sampling behavior are unchanged; workflows
saved before this change keep working.
This commit is contained in:
Z User 2026-10-13 03:34:47 +00:00
parent 08229541c3
commit 88849ced65
7 changed files with 34 additions and 131 deletions

View File

@ -6,6 +6,7 @@
### ⚡ Improvements ### ⚡ Improvements
- **Moxie Multimedia Loader**: Removed the automatic LoRA application entirely. The 3-step distill (speedup) weights are now baked into the diffusion model itself, so the loader no longer reads `models/loras` — a leftover file there is silently ignored, and the `models/loras` folder can be deleted. The node schema, outputs, and load order (diffusion model → attention backend → CLIP → VAEs) are unchanged; workflows saved earlier keep working. The `Moxie Project` steps tooltip now references the baked-in distill instead of the LoRA.
- **FFmpeg resolution & diagnostics**: The pack now finds FFmpeg on far more Windows setups and tells you exactly what is wrong when it cannot. When `ffmpeg` is not on the ComfyUI process's PATH (PATH edits only reach processes started afterwards), it is additionally looked up in the classic `C:\ffmpeg\bin`, the ComfyUI/portable folders (including `ffmpeg\bin` next to the run bat), winget/scoop/chocolatey shim locations, and finally the ffmpeg binary bundled with ComfyUI's own `imageio-ffmpeg` dependency. When composing an H3 project still fails, the error dialog now names the concrete cause instead of a generic message: the missing executable, the exact missing segment/locked-audio file paths, odd canvas sizes, or the tail of FFmpeg's own stderr output (e.g. `Unknown encoder 'libx264'`). - **FFmpeg resolution & diagnostics**: The pack now finds FFmpeg on far more Windows setups and tells you exactly what is wrong when it cannot. When `ffmpeg` is not on the ComfyUI process's PATH (PATH edits only reach processes started afterwards), it is additionally looked up in the classic `C:\ffmpeg\bin`, the ComfyUI/portable folders (including `ffmpeg\bin` next to the run bat), winget/scoop/chocolatey shim locations, and finally the ffmpeg binary bundled with ComfyUI's own `imageio-ffmpeg` dependency. When composing an H3 project still fails, the error dialog now names the concrete cause instead of a generic message: the missing executable, the exact missing segment/locked-audio file paths, odd canvas sizes, or the tail of FFmpeg's own stderr output (e.g. `Unknown encoder 'libx264'`).
- **Installation**: The pack now ships a **`requirements.txt`**, so ComfyUI Manager installs the dependencies automatically and manual/portable installs have a one-line command (`python_embeded\python.exe -m pip install -r ...` for ComfyUI Portable — see the new "Installing dependencies" section in the README). Because this pack targets **NVIDIA RTX 3060+ GPUs only**, every feature dependency is mandatory: `soundfile`, `jieba`, `huggingface_hub`, `ffmpeg-python`, `opencv-python`, `imageio` (smart split), `nvidia-vfx` (RTX VSR upscale), `openai-whisper` and `qwen-asr` (subtitle recognition), and `voxcpm` (speech). No optional extras remain — previously these were installed on demand with error messages naming the missing package. The file also routes `nvidia-vfx` to NVIDIA's own index (`--extra-index-url https://pypi.nvidia.com/` + `--only-binary=nvidia-vfx`): the PyPI copy is a stub sdist whose `wheel_stub` build backend can abort the whole install (`Cannot import 'wheel_stub.buildapi'`), while NVIDIA's index serves real prebuilt wheels. - **Installation**: The pack now ships a **`requirements.txt`**, so ComfyUI Manager installs the dependencies automatically and manual/portable installs have a one-line command (`python_embeded\python.exe -m pip install -r ...` for ComfyUI Portable — see the new "Installing dependencies" section in the README). Because this pack targets **NVIDIA RTX 3060+ GPUs only**, every feature dependency is mandatory: `soundfile`, `jieba`, `huggingface_hub`, `ffmpeg-python`, `opencv-python`, `imageio` (smart split), `nvidia-vfx` (RTX VSR upscale), `openai-whisper` and `qwen-asr` (subtitle recognition), and `voxcpm` (speech). No optional extras remain — previously these were installed on demand with error messages naming the missing package. The file also routes `nvidia-vfx` to NVIDIA's own index (`--extra-index-url https://pypi.nvidia.com/` + `--only-binary=nvidia-vfx`): the PyPI copy is a stub sdist whose `wheel_stub` build backend can abort the whole install (`Cannot import 'wheel_stub.buildapi'`), while NVIDIA's index serves real prebuilt wheels.
- **Installation**: Removed the bundled `download_models.sh` downloader. All model files live in the Hugging Face version of the repository — cloning it downloads the complete model set, so there is no separate downloader step (the README's Models section now explains this). - **Installation**: Removed the bundled `download_models.sh` downloader. All model files live in the Hugging Face version of the repository — cloning it downloads the complete model set, so there is no separate downloader step (the README's Models section now explains this).

View File

@ -61,11 +61,10 @@ All model files ship **inside the repo** — cloning it into `custom_nodes` is a
| File | Location | | File | Location |
|---|---| |---|---|
| Diffusion model (`Moxie-Multimedia.safetensors`) | `models/diffusion/` | | Diffusion model (`Moxie-Multimedia.safetensors`, distill weights baked in) | `models/diffusion/` |
| Text encoder (`MM-VL.safetensors`) | `models/clip/` | | Text encoder (`MM-VL.safetensors`) | `models/clip/` |
| Video VAE (bundled file) | `models/vae/` | | Video VAE (bundled file) | `models/vae/` |
| Audio VAE (bundled file) | `models/audio_vae/` | | Audio VAE (bundled file) | `models/audio_vae/` |
| Turbo LoRA (`MM3step.safetensors`, applied by default) | `models/loras/` |
| Preview tiny VAE (`MM-preview.safetensors`) | `models/vae_approx/` | | Preview tiny VAE (`MM-preview.safetensors`) | `models/vae_approx/` |
All model files live in the Hugging Face version of this repository — cloning it downloads the complete set, and there is no separate downloader step. If you obtained the pack from somewhere else and the `models/` folders are missing or incomplete, re-clone from Hugging Face to get the bundled files. All model files live in the Hugging Face version of this repository — cloning it downloads the complete set, and there is no separate downloader step. If you obtained the pack from somewhere else and the `models/` folders are missing or incomplete, re-clone from Hugging Face to get the bundled files.

View File

@ -19,7 +19,6 @@ SUBDIR_HINTS = {
"vae": "models/vae", "vae": "models/vae",
"audio_vae": "models/audio_vae", "audio_vae": "models/audio_vae",
"clip": "models/clip", "clip": "models/clip",
"loras": "models/loras",
} }
@ -53,18 +52,11 @@ def _resolve(subdir, name):
return path return path
# (path, sd, metadata) cache for the last parsed LoRA, mirroring the native
# LoraLoader. Kept at module level on purpose: newer ComfyUI builds lock node
# classes after registration, so writing class attributes on the node class
# itself raises "Cannot modify class attribute ... on locked class".
_loaded_lora = None
def _auto_pick(subdir): def _auto_pick(subdir):
"""Return the model file to use for a hidden picker. """Return the model file to use for a hidden picker.
End users only see the diffusion model widget; every auxiliary model End users only see the diffusion model widget; every auxiliary model
(video VAE, audio VAE, CLIP, LoRA) is auto-selected from the pack's (video VAE, audio VAE, CLIP) is auto-selected from the pack's
models/<subdir> folder - the first file in sorted order, which for the models/<subdir> folder - the first file in sorted order, which for the
published pack is exactly the bundled file. A missing file keeps the published pack is exactly the bundled file. A missing file keeps the
loud, actionable error from _resolve. loud, actionable error from _resolve.
@ -85,14 +77,6 @@ def _auto_pick(subdir):
return selected return selected
def _auto_lora():
"""Bundled LoRA file, or ``None`` when models/loras is empty."""
files = _list_files("loras")
if files[0].startswith("<no files found"):
return None
return files[0]
def _h3_clip_type(): def _h3_clip_type():
"""MiniMax H3 CLIP recipe. The Moxie model set is H3-only, so the text """MiniMax H3 CLIP recipe. The Moxie model set is H3-only, so the text
encoder type is intentionally NOT a user-facing choice.""" encoder type is intentionally NOT a user-facing choice."""
@ -110,12 +94,12 @@ class MoxieMultimediaLoader(io.ComfyNode):
"""All-in-one loader that looks like it loads a single model. """All-in-one loader that looks like it loads a single model.
The only visible picker is the diffusion model; the attention backend is The only visible picker is the diffusion model; the attention backend is
applied to it, the bundled LoRA (if any), the video VAE, the audio VAE applied to it, and the video VAE, the audio VAE and the CLIP are
and the CLIP are auto-selected from the node pack's models/ folder. auto-selected from the node pack's models/ folder.
The model set is MiniMax H3-only: the CLIP recipe is locked to The model set is MiniMax H3-only: the CLIP recipe is locked to
CLIPType.MINIMAX and outputs are exactly (model, clip, video_vae, audio_vae) CLIPType.MINIMAX and outputs are exactly (model, clip, video_vae, audio_vae)
with the attention backend and LoRA already applied to the model.""" with the attention backend already applied to the model."""
@classmethod @classmethod
def define_schema(cls) -> io.Schema: def define_schema(cls) -> io.Schema:
@ -123,7 +107,7 @@ class MoxieMultimediaLoader(io.ComfyNode):
node_id="MoxieMultimediaLoader", node_id="MoxieMultimediaLoader",
display_name="Moxie Multimedia Loader", display_name="Moxie Multimedia Loader",
category="loaders/moxie", category="loaders/moxie",
description="Loads the Moxie diffusion model (with the attention backend and the bundled LoRA " description="Loads the Moxie diffusion model (with the attention backend "
"already applied) plus the video VAE, audio VAE and text encoder that ship with " "already applied) plus the video VAE, audio VAE and text encoder that ship with "
"this custom node - one model picker, everything else is handled automatically. " "this custom node - one model picker, everything else is handled automatically. "
"Outputs model, clip, video_vae and audio_vae.", "Outputs model, clip, video_vae and audio_vae.",
@ -148,8 +132,8 @@ class MoxieMultimediaLoader(io.ComfyNode):
), ),
], ],
outputs=[ outputs=[
io.Model.Output(display_name="model", tooltip="Diffusion model with attention backend and LoRA applied."), io.Model.Output(display_name="model", tooltip="Diffusion model with attention backend applied."),
io.Clip.Output(display_name="clip", tooltip="CLIP text encoder (with LoRA applied if present)."), io.Clip.Output(display_name="clip", tooltip="CLIP text encoder."),
io.Vae.Output(display_name="video_vae", tooltip="Loaded video/image VAE."), io.Vae.Output(display_name="video_vae", tooltip="Loaded video/image VAE."),
io.Vae.Output(display_name="audio_vae", tooltip="Loaded audio VAE."), io.Vae.Output(display_name="audio_vae", tooltip="Loaded audio VAE."),
], ],
@ -189,36 +173,6 @@ class MoxieMultimediaLoader(io.ComfyNode):
m.set_model_optimized_attention(attention_function) m.set_model_optimized_attention(attention_function)
return m return m
@classmethod
def _apply_lora(cls, model, clip, lora_name, strength_model, strength_clip):
global _loaded_lora
if (
lora_name is None
or lora_name == "none"
or lora_name.startswith("<no files found")
or (strength_model == 0 and strength_clip == 0)
):
return model, clip
lora_path = _resolve("loras", lora_name)
lora = None
lora_metadata = None
if _loaded_lora is not None:
if _loaded_lora[0] == lora_path:
lora = _loaded_lora[1]
lora_metadata = _loaded_lora[2] if len(_loaded_lora) > 2 else None
else:
_loaded_lora = None
if lora is None:
lora, lora_metadata = comfy.utils.load_torch_file(lora_path, safe_load=True, return_metadata=True)
_loaded_lora = (lora_path, lora, lora_metadata)
model_lora, clip_lora = comfy.sd.load_lora_for_models(
model, clip, lora, strength_model, strength_clip, lora_metadata=lora_metadata
)
return model_lora, clip_lora
@classmethod @classmethod
def _load_vae(cls, vae_name, subdir="vae"): def _load_vae(cls, vae_name, subdir="vae"):
path = _resolve(subdir, vae_name) path = _resolve(subdir, vae_name)
@ -256,15 +210,13 @@ class MoxieMultimediaLoader(io.ComfyNode):
weight_dtype, weight_dtype,
attention_backend, attention_backend,
) -> io.NodeOutput: ) -> io.NodeOutput:
# Chain: Load Diffusion Model -> Model Attention Backend -> Lora Loader (needs both model and clip) # Chain: Load Diffusion Model -> Model Attention Backend
model = cls._load_diffusion_model(diffusion_model, weight_dtype) model = cls._load_diffusion_model(diffusion_model, weight_dtype)
model = cls._apply_attention_backend(model, attention_backend) model = cls._apply_attention_backend(model, attention_backend)
# Hidden pickers: auxiliary models are auto-selected from the pack's # Hidden pickers: auxiliary models are auto-selected from the pack's
# models/ folder so end users only ever see the diffusion model. # models/ folder so end users only ever see the diffusion model.
clip_obj = cls._load_clip(_auto_pick("clip"), "default") clip_obj = cls._load_clip(_auto_pick("clip"), "default")
lora_name = _auto_lora()
model, clip_obj = cls._apply_lora(model, clip_obj, lora_name, 1.0, 1.0)
video_vae_obj = cls._load_vae(_auto_pick("vae"), "vae") video_vae_obj = cls._load_vae(_auto_pick("vae"), "vae")
audio_vae_obj = cls._load_vae(_auto_pick("audio_vae"), "audio_vae") audio_vae_obj = cls._load_vae(_auto_pick("audio_vae"), "audio_vae")

View File

@ -1006,7 +1006,7 @@ class ModelPreviewOverrideKJ(io.ComfyNode):
inputs=[ inputs=[
io.Model.Input( io.Model.Input(
"model", "model",
tooltip="Diffusion model (with attention backend and LoRA already applied).", tooltip="Diffusion model (with attention backend already applied).",
), ),
io.Int.Input( io.Int.Input(
"max_resolution", "max_resolution",

View File

@ -52,7 +52,7 @@ class MoxieProject(EasyMultiTrackProject):
not_idempotent=True, not_idempotent=True,
inputs=[ inputs=[
TYPE_TRACKS_INFO.Input("tracks_info"), TYPE_TRACKS_INFO.Input("tracks_info"),
io.Model.Input("model", tooltip="Diffusion model from Moxie Multimedia Loader (attention backend and LoRA already applied)."), io.Model.Input("model", tooltip="Diffusion model from Moxie Multimedia Loader (attention backend already applied)."),
io.Clip.Input("clip", tooltip="CLIP text encoder from Moxie Multimedia Loader."), io.Clip.Input("clip", tooltip="CLIP text encoder from Moxie Multimedia Loader."),
io.Vae.Input("video_vae", tooltip="Video VAE from Moxie Multimedia Loader."), io.Vae.Input("video_vae", tooltip="Video VAE from Moxie Multimedia Loader."),
io.Vae.Input("audio_vae", tooltip="Audio VAE from Moxie Multimedia Loader (required to decode Moxie audio)."), io.Vae.Input("audio_vae", tooltip="Audio VAE from Moxie Multimedia Loader (required to decode Moxie audio)."),
@ -63,8 +63,8 @@ class MoxieProject(EasyMultiTrackProject):
default=3, default=3,
min=1, min=1,
max=10000, max=10000,
tooltip="Sampling steps for the beta scheduler (3 for the 3-step turbo LoRA; " tooltip="Sampling steps for the beta scheduler (3 for the baked-in 3-step distill; "
"8 for busy motion with regular LoRAs, 6 low motion, 4 runs/no dialogue).", "8 for busy motion, 6 low motion, 4 runs/no dialogue).",
), ),
io.Int.Input( io.Int.Input(
"seed", "seed",

View File

@ -294,8 +294,7 @@ def _install_loader_stubs(state):
return (sd, {"meta": True}) if return_metadata else sd return (sd, {"meta": True}) if return_metadata else sd
def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None): def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None):
state.calls.append(("load_lora_for_models", strength_model, strength_clip)) raise AssertionError("MoxieMultimediaLoader must not load LoRAs anymore")
return state.lora_model, clip
def get_attention_function(name, *args): def get_attention_function(name, *args):
state.calls.append(("get_attention_function", name)) state.calls.append(("get_attention_function", name))
@ -352,7 +351,7 @@ def _load_loader_module():
def verify_loader(): def verify_loader():
print("\n== 4. moxie_loader.py: single-model UI + auto-picked aux models ==") print("\n== 4. moxie_loader.py: single-model UI + auto-picked aux models ==")
state = types.SimpleNamespace( state = types.SimpleNamespace(
calls=[], clip=object(), attention_fn=object(), lora_model=object(), calls=[], clip=object(), attention_fn=object(),
) )
_install_loader_stubs(state) _install_loader_stubs(state)
loader = _load_loader_module() loader = _load_loader_module()
@ -377,13 +376,14 @@ def verify_loader():
("vae", "video_vae.safetensors"), ("vae", "video_vae.safetensors"),
("audio_vae", "audio_vae.safetensors"), ("audio_vae", "audio_vae.safetensors"),
("clip", "MM-VL.safetensors"), ("clip", "MM-VL.safetensors"),
# A leftover loras file must be ignored entirely: the distill
# weights ship baked into the diffusion model now.
("loras", "MM3step.safetensors"), ("loras", "MM3step.safetensors"),
): ):
os.makedirs(os.path.join(models, subdir), exist_ok=True) os.makedirs(os.path.join(models, subdir), exist_ok=True)
with open(os.path.join(models, subdir, name), "wb") as handle: with open(os.path.join(models, subdir, name), "wb") as handle:
handle.write(b"stub") handle.write(b"stub")
loader.MODELS_DIR = models loader.MODELS_DIR = models
loader._loaded_lora = None
result = loader.MoxieMultimediaLoader.execute( result = loader.MoxieMultimediaLoader.execute(
diffusion_model="moxie.safetensors", diffusion_model="moxie.safetensors",
@ -392,37 +392,21 @@ def verify_loader():
) )
names = [call[0] for call in state.calls] names = [call[0] for call in state.calls]
check("full chain runs: model -> attention -> clip -> lora -> vaes", check("full chain runs: model -> attention -> clip -> vaes (no LoRA)",
names == [ names == [
"load_diffusion_model", "get_attention_function", "load_diffusion_model", "get_attention_function",
"set_model_optimized_attention", "load_clip", "set_model_optimized_attention", "load_clip",
"load_torch_file", "load_lora_for_models",
"load_torch_file", "load_torch_file", "load_torch_file", "load_torch_file",
], repr(names)) ], repr(names))
lora_calls = [call for call in state.calls if call[0] == "load_lora_for_models"]
check("LoRA auto-applied at strength 1.0/1.0",
lora_calls == [("load_lora_for_models", 1.0, 1.0)], repr(lora_calls))
clip_calls = [call for call in state.calls if call[0] == "load_clip"] clip_calls = [call for call in state.calls if call[0] == "load_clip"]
check("CLIP auto-picked and loaded with the locked recipe", check("CLIP auto-picked and loaded with the locked recipe",
len(clip_calls) == 1 and clip_calls[0][1] is not None, repr(clip_calls)) len(clip_calls) == 1 and clip_calls[0][1] is not None, repr(clip_calls))
check("four outputs returned", check("four outputs returned",
isinstance(result, loader.io.NodeOutput) and len(result.values) == 4) isinstance(result, loader.io.NodeOutput) and len(result.values) == 4)
check("model output carries LoRA", check("model output is the attention clone (no LoRA on top)",
result.values[0] is state.lora_model) result.values[0] is not state.model)
check("clip output is the loaded clip", result.values[1] is state.clip) check("clip output is the loaded clip", result.values[1] is state.clip)
# Empty loras folder -> LoRA silently skipped, everything else loads.
os.remove(os.path.join(models, "loras", "MM3step.safetensors"))
state.calls.clear()
loader._loaded_lora = None
loader.MoxieMultimediaLoader.execute(
diffusion_model="moxie.safetensors",
weight_dtype="default",
attention_backend="pytorch_attention",
)
check("empty loras folder skips LoRA without breaking the load",
"load_lora_for_models" not in [call[0] for call in state.calls])
# Missing clip folder -> loud setup error (same message as before). # Missing clip folder -> loud setup error (same message as before).
state.calls.clear() state.calls.clear()
os.rename(os.path.join(models, "clip"), os.path.join(models, "clip_backup")) os.rename(os.path.join(models, "clip"), os.path.join(models, "clip_backup"))

View File

@ -7,7 +7,8 @@ Verified contracts:
- MoxieMultimediaLoader: outputs exactly (model, clip, video_vae, audio_vae); - MoxieMultimediaLoader: outputs exactly (model, clip, video_vae, audio_vae);
no legacy FAST_MODEL_LOADER bundle output; no clip_type choice (the CLIP no legacy FAST_MODEL_LOADER bundle output; no clip_type choice (the CLIP
recipe is locked to CLIPType.MINIMAX); the model output has the attention recipe is locked to CLIPType.MINIMAX); the model output has the attention
backend and LoRA applied. backend applied, and any leftover file in models/loras is ignored (the
distill weights ship baked into the diffusion model).
- MoxieProject: takes model/clip/video_vae/audio_vae as required direct - MoxieProject: takes model/clip/video_vae/audio_vae as required direct
inputs instead of a bundled model_loader, and packs them into the loader inputs instead of a bundled model_loader, and packs them into the loader
dict consumed internally by EasyMultiTrackProject. dict consumed internally by EasyMultiTrackProject.
@ -122,8 +123,6 @@ class _StubState:
self.calls = [] self.calls = []
self.attention_fn = lambda pixels, *a, **k: pixels self.attention_fn = lambda pixels, *a, **k: pixels
self.clip = object() self.clip = object()
self.lora_model = None
self.lora_clip = None
def _register(state, io, monkeypatch=None): def _register(state, io, monkeypatch=None):
@ -158,10 +157,7 @@ def _register(state, io, monkeypatch=None):
return state.clip return state.clip
def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None): def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None):
state.calls.append(("load_lora_for_models", strength_model, strength_clip)) raise AssertionError("MoxieMultimediaLoader must not load LoRAs anymore")
state.lora_model = model
state.lora_clip = clip
return model, clip
comfy_sd.CLIPType = CLIPType comfy_sd.CLIPType = CLIPType
comfy_sd.load_diffusion_model = load_diffusion_model comfy_sd.load_diffusion_model = load_diffusion_model
@ -303,6 +299,7 @@ def _build_moxie():
tiny_vae = types.ModuleType("MMnode.tiny_vae") tiny_vae = types.ModuleType("MMnode.tiny_vae")
tiny_vae.load_tiny_vae_decoder = lambda *a, **k: None tiny_vae.load_tiny_vae_decoder = lambda *a, **k: None
tiny_vae.pack_vae_approx_files = lambda *a, **k: []
added["MMnode.tiny_vae"] = sys.modules.get("MMnode.tiny_vae") added["MMnode.tiny_vae"] = sys.modules.get("MMnode.tiny_vae")
sys.modules["MMnode.tiny_vae"] = tiny_vae sys.modules["MMnode.tiny_vae"] = tiny_vae
@ -360,7 +357,7 @@ def test_loader_schema_shows_a_single_model(moxie):
inputs = _input_names(schema) inputs = _input_names(schema)
# End users see one model picker plus the attention backend; every # End users see one model picker plus the attention backend; every
# auxiliary model (LoRA, video/audio VAE, CLIP) is auto-selected. # auxiliary model (video/audio VAE, CLIP) is auto-selected.
assert inputs == ["diffusion_model", "weight_dtype", "attention_backend"] assert inputs == ["diffusion_model", "weight_dtype", "attention_backend"]
for hidden in ( for hidden in (
"lora_name", "lora_name",
@ -374,7 +371,7 @@ def test_loader_schema_shows_a_single_model(moxie):
assert hidden not in inputs assert hidden not in inputs
def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path): def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch, tmp_path):
state = moxie.state state = moxie.state
models = tmp_path / "models" models = tmp_path / "models"
for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"): for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"):
@ -389,9 +386,6 @@ def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path
}[name] }[name]
(models / subdir / name).write_bytes(b"stub") (models / subdir / name).write_bytes(b"stub")
monkeypatch.setattr(moxie.loader, "MODELS_DIR", models) monkeypatch.setattr(moxie.loader, "MODELS_DIR", models)
# LoRA cache lives at module level: node classes are locked by newer
# ComfyUI builds and class attributes can no longer be written at runtime.
monkeypatch.setattr(moxie.loader, "_loaded_lora", None)
state.calls.clear() state.calls.clear()
result = moxie.loader.MoxieMultimediaLoader.execute( result = moxie.loader.MoxieMultimediaLoader.execute(
@ -400,14 +394,15 @@ def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path
attention_backend="comfy_kitchen_attention", attention_backend="comfy_kitchen_attention",
) )
# The distill weights ship baked into the diffusion model: a leftover
# file in models/loras must be ignored entirely. The comfy.sd stub keeps
# a raising load_lora_for_models, so an accidental LoRA call fails loudly.
sequence = [call[0] for call in state.calls] sequence = [call[0] for call in state.calls]
assert sequence == [ assert sequence == [
"load_diffusion_model", "load_diffusion_model",
"get_attention_function", "get_attention_function",
"load_clip", "load_clip",
"load_torch_file", "load_torch_file",
"load_lora_for_models",
"load_torch_file",
"load_torch_file", "load_torch_file",
] ]
@ -415,50 +410,22 @@ def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path
clip_calls = [call for call in state.calls if call[0] == "load_clip"] clip_calls = [call for call in state.calls if call[0] == "load_clip"]
assert clip_calls == [("load_clip", state.CLIPType.MINIMAX)] assert clip_calls == [("load_clip", state.CLIPType.MINIMAX)]
# LoRA applied on top of the attention-patched model, after the CLIP load. # Attention backend applied before the CLIP load.
assert state.calls.index(("get_attention_function", "comfy_kitchen_int8")) < state.calls.index( assert state.calls.index(("get_attention_function", "comfy_kitchen_int8")) < state.calls.index(
("load_clip", state.CLIPType.MINIMAX) ("load_clip", state.CLIPType.MINIMAX)
) < state.calls.index(("load_lora_for_models", 1.0, 1.0)) )
# Exactly four direct outputs, model carrying attention + LoRA. # Exactly four direct outputs, model carrying the attention backend.
assert isinstance(result, _NodeOutput) assert isinstance(result, _NodeOutput)
assert len(result.values) == 4 assert len(result.values) == 4
model_out, clip_out, video_vae_out, audio_vae_out = result.values model_out, clip_out, video_vae_out, audio_vae_out = result.values
assert model_out is state.lora_model assert model_out.label == "diffusion.clone"
assert model_out.attention_fn is state.attention_fn assert model_out.attention_fn is state.attention_fn
assert clip_out is state.clip assert clip_out is state.clip
assert isinstance(video_vae_out, _FakeVAE) assert isinstance(video_vae_out, _FakeVAE)
assert isinstance(audio_vae_out, _FakeVAE) assert isinstance(audio_vae_out, _FakeVAE)
def test_loader_execute_without_lora_skips_lora_loading(moxie, monkeypatch, tmp_path):
state = moxie.state
models = tmp_path / "models"
for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"):
(models / subdir).mkdir(parents=True, exist_ok=True)
for name in ("h3.safetensors", "video.safetensors", "audio.safetensors", "qwen.safetensors"):
subdir = {
"h3.safetensors": "diffusion",
"video.safetensors": "vae",
"audio.safetensors": "audio_vae",
"qwen.safetensors": "clip",
}[name]
(models / subdir / name).write_bytes(b"stub")
monkeypatch.setattr(moxie.loader, "MODELS_DIR", models)
state.calls.clear()
result = moxie.loader.MoxieMultimediaLoader.execute(
diffusion_model="h3.safetensors",
weight_dtype="default",
attention_backend="pytorch_attention",
)
assert "load_lora_for_models" not in [call[0] for call in state.calls]
assert result.values[0] is not state.clip # model output remains a model
assert result.values[0].attention_fn is state.attention_fn
assert result.values[1] is state.clip
def test_loader_requires_minimax_cliptype(moxie, monkeypatch, tmp_path): def test_loader_requires_minimax_cliptype(moxie, monkeypatch, tmp_path):
state = moxie.state state = moxie.state
models = tmp_path / "models" models = tmp_path / "models"