Compare commits
23 Commits
phase2-ben
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d5ede8964f | ||
|
|
01946fa657 | ||
|
|
e83beb18bd | ||
|
|
078a7749b3 | ||
|
|
5347719cdd | ||
|
|
46617b71b8 | ||
|
|
68d30bd69a | ||
|
|
ce03875c69 | ||
|
|
43d5bf9a1e | ||
|
|
5a3b03ef61 | ||
|
|
0a9416b91a | ||
|
|
724da63f7f | ||
|
|
88849ced65 | ||
|
|
08229541c3 | ||
|
|
d26e3297a6 | ||
|
|
4dccb4fb4c | ||
|
|
bc534da34f | ||
|
|
bac3101e4d | ||
|
|
0a6a5d6aa9 | ||
|
|
6f8be4c6d9 | ||
|
|
e7a288de1a | ||
|
|
84b02cf8ac | ||
|
|
43c7051f09 |
15
CHANGELOG.md
15
CHANGELOG.md
@ -6,8 +6,19 @@
|
||||
|
||||
### ⚡ Improvements
|
||||
|
||||
- **Updater**: The pack now ships a **`update.bat`** one-click updater for Windows. Place it in the pack folder (next to `__init__.py`) and double-click: it points the folder's git remote at the HuggingFace repository (fixing installs whose `origin` still points at a stale URL), fetches and force-syncs `main`, clears leftover files from older versions, and downloads the model weights through git-lfs — with a clear warning and install pointer when git-lfs is missing. It then self-verifies: it prints the current build id (`git log -1 --oneline`), counts the odd-latent fix marker in `nodes/project.py` (3 hits = fixed build), and flags any model file that is still a git-lfs placeholder. ComfyUI must still be restarted after updating.
|
||||
|
||||
- **Moxie MultiTrack Project Slow**: Removed the experimental node again, together with its `upscale_model` / `tiling` widgets and the even-latent first-pass snap — the shared project-node internals are restored byte-for-byte to their certified behavior. Workflows that referenced the Slow node need rewiring to **Moxie MultiTrack Project**.
|
||||
|
||||
- **MultiTrack Project**: The inter-segment context overlap is now configurable. The new `context_frames` widget (5 / 22 / 39 / 56 frames, default 22) chooses how many frames of the previous segment are copied in as continuation context for segments using Context or Context Swap continuity — previously this was fixed at 22 frames in code. Longer values hand off motion more smoothly but generate that many extra frames per continued segment (each continuation also generates 12 grid-alignment tail frames that are trimmed after decode). Values are the H3 temporal-grid lengths already offered by the Moxie context sampler nodes; existing workflows keep 22 and behave exactly as before. The same widget (between `steps` and `seed`) is also exposed on **Moxie MultiTrack Project** — its baked euler/beta sampling stack is unchanged, so end users still only need `steps` and `seed` unless they touch the overlap.
|
||||
|
||||
- **Moxie Multimedia Loader**: Removed the automatic LoRA application entirely. The 3-step distill (speedup) weights are now baked into the diffusion model itself, so the loader no longer reads `models/loras` — a leftover file there is silently ignored, and the `models/loras` folder can be deleted. The node schema, outputs, and load order (diffusion model → attention backend → CLIP → VAEs) are unchanged; workflows saved earlier keep working. The `Moxie Project` steps tooltip now references the baked-in distill instead of the LoRA.
|
||||
- **FFmpeg resolution & diagnostics**: The pack now finds FFmpeg on far more Windows setups and tells you exactly what is wrong when it cannot. When `ffmpeg` is not on the ComfyUI process's PATH (PATH edits only reach processes started afterwards), it is additionally looked up in the classic `C:\ffmpeg\bin`, the ComfyUI/portable folders (including `ffmpeg\bin` next to the run bat), winget/scoop/chocolatey shim locations, and finally the ffmpeg binary bundled with ComfyUI's own `imageio-ffmpeg` dependency. When composing an H3 project still fails, the error dialog now names the concrete cause instead of a generic message: the missing executable, the exact missing segment/locked-audio file paths, odd canvas sizes, or the tail of FFmpeg's own stderr output (e.g. `Unknown encoder 'libx264'`).
|
||||
- **Installation**: The pack now ships a **`requirements.txt`**, so ComfyUI Manager installs the dependencies automatically and manual/portable installs have a one-line command (`python_embeded\python.exe -m pip install -r ...` for ComfyUI Portable — see the new "Installing dependencies" section in the README). Because this pack targets **NVIDIA RTX 3060+ GPUs only**, every feature dependency is mandatory: `soundfile`, `jieba`, `huggingface_hub`, `ffmpeg-python`, `opencv-python`, `imageio` (smart split), `nvidia-vfx` (RTX VSR upscale), `openai-whisper` and `qwen-asr` (subtitle recognition), and `voxcpm` (speech). No optional extras remain — previously these were installed on demand with error messages naming the missing package. The file also routes `nvidia-vfx` to NVIDIA's own index (`--extra-index-url https://pypi.nvidia.com/` + `--only-binary=nvidia-vfx`): the PyPI copy is a stub sdist whose `wheel_stub` build backend can abort the whole install (`Cannot import 'wheel_stub.buildapi'`), while NVIDIA's index serves real prebuilt wheels.
|
||||
- **Installation**: Removed the bundled `download_models.sh` downloader. All model files live in the Hugging Face version of the repository — cloning it downloads the complete model set, so there is no separate downloader step (the README's Models section now explains this).
|
||||
- **MultiTrack Editor**: The editor frontend was **reverted to the pristine upstream v1.3.3 source** — the MiniMax/Moxie format display-label mapping and every workaround built around it were removed. The shipped editor bundle is now built from that pristine source, so its behavior is exactly the battle-tested upstream editor (Shot/Context continuity, reference-size, audio lock, and MiniMax prompt placeholders all key on the native `MiniMax` format value again).
|
||||
- **Editor delivery fix**: A fresh browser could previously end up without the editor bundle entirely. ComfyUI core lets a pack's `WEB_DIRECTORY` overwrite its extension web-dir registration, so the browser's `/extensions` listing only ever contained this pack's helper scripts — never the editor entry — and the editor only appeared from stale browser cache. A small static loader (`web/js/editor-loader.js`, served with the helpers) now imports the editor entry from the pack's own route, which serves `index.js` with `no-cache` so the entry is always revalidated while its content-hashed chunks stay cacheable. After updating: restart ComfyUI and reload the page once; no cache-clearing gymnastics required.
|
||||
- **Editor loader hotfix**: The loader's base-prefix computation produced `//` on standard installs (no reverse-proxy sub-path), which browsers parse as a protocol-relative URL — they then tried to resolve the pack folder name as a DNS hostname and the editor entry never loaded (`net::ERR_NAME_NOT_RESOLVED` on `moxie-multimedia/index.js` in the browser console). The base is now exactly `/` when there is no sub-path prefix. If the MultiTrack Editor node shows a plain placeholder instead of the timeline UI, update the pack and hard-refresh the browser.
|
||||
- **MultiTrack Editor / MultiTrack Project**: **T2V resolves natively again** (`t2v` stays `t2v`, matching upstream) — the model now handles text-to-video with the updated prompting, so the earlier T2V→R2V normalization was removed from the editor type computation, `h3_task_type`, `_multitrack_task_type`, and the generation-mode mapping. The reference-mode-with-zero-media cloud fallback was removed accordingly (reference mode raises for missing media again, as upstream).
|
||||
- **Moxie Multimedia Loader**: The loader now **looks like it loads a single model**. Only the diffusion-model picker (plus the attention backend) is visible; the LoRA, video VAE, audio VAE, and CLIP widgets were removed and their files are auto-selected from the pack's `models/loras`, `models/vae`, `models/audio_vae`, and `models/clip` folders (first file in sorted order, matching the bundled model set). The LoRA is applied automatically with strength 1.0; remove the file from `models/loras` to disable it. Missing auxiliary models raise the same clear setup error as before. Workflows saved before this change keep working — extra widget values are ignored.
|
||||
- **MultiTrack Project / Moxie Project Video Combine**: Single-pass projects now condition and sample at half the editor-selected resolution internally. The project timeline is composed at the segments' own size, and Moxie Project Video Combine restores the exact editor-selected resolution in pixel space before any output leaves the node (RTX Video Super Resolution via the optional `nvidia-vfx` package when available, with an automatic Lanczos fallback otherwise) — so every output carries the full editor resolution regardless of template wiring. The editor resolution therefore stays the final output resolution while the sampling chain runs smaller and faster. Dual-pass, SelfLift, and passthrough modes are unchanged.
|
||||
@ -18,6 +29,10 @@
|
||||
- **Moxie Multimedia Loader**: Locked the loader to the MiniMax H3 model set. The `clip_type` dropdown (which previously exposed every CLIP recipe in ComfyUI) was removed; the H3 Qwen3-VL text encoder is now always loaded with the `minimax` CLIP recipe. Any older ComfyUI build without `CLIPType.MINIMAX` raises a clear startup error instead of silently loading the wrong encoder.
|
||||
- **Moxie Multimedia Loader / Moxie MultiTrack Project**: Removed the legacy `FAST_MODEL_LOADER` (`model_loader`) bundle. The loader now outputs exactly `model`, `clip`, `video_vae`, and `audio_vae` (with the attention backend and LoRA already applied to the model), and the Moxie MultiTrack Project takes those four outputs as required direct inputs instead of a bundled loader. `Moxie Preview Override` likewise dropped its `model_loader` pass-through slots and now patches the `model` input directly. The generic `easy multitrackProject` node keeps its `model_loader` input for non-Moxie workflows.
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **MultiTrack Project / MiniMax H3 nodes**: Fixed the first-step `patchify_video` crash (`RuntimeError: shape '[1, 24, 1, 1, ...]' is invalid for input of size N`) that hit 11 of the 17 editor resolution dropdown entries in single-pass mode — including the app's default 960x544 and the auto fallback 544x960. The hidden half-resolution first pass floored off the H3 32px canvas grid (544x960 halved to 272x480 encodes to a 17x30 latent whose odd side comfy core cannot split into 2x2 patches); the first pass now floors onto the grid (544x960 samples at 256x480, and Moxie Project Video Combine still restores the exact editor resolution, so output size and quality are unchanged). The MiniMax H3 conditioning entry points (`MiniMaxH3ImageToVideoFallback`, `MiniMaxH3ReferenceToVideoFallback`, `EasyMiniMaxH3ToVideo`) align their canvas the same way, which also fixes dual-pass and SelfLift at any editor size that is not a multiple of 32. The misleading "No H3 task segments are available from segment_start_number." error now distinguishes an empty timeline from an out-of-range `segment_start_number` and says what to do about each. **After updating, restart ComfyUI once** — a stale off-grid latent cached in the temp directory otherwise survives the pack update and the crash appears to persist.
|
||||
|
||||
## [1.3.3] - 2026-09-25
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
45
README.md
45
README.md
@ -1,3 +1,17 @@
|
||||
---
|
||||
license: gpl-3.0
|
||||
tags:
|
||||
- comfyui
|
||||
- text-to-video
|
||||
- video-generation
|
||||
- multi-track
|
||||
- audio
|
||||
- text-to-speech
|
||||
- automatic-speech-recognition
|
||||
- subtitles
|
||||
- nvidia-rtx
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
<img src="https://github.com/user-attachments/assets/fb602a3c-4a2a-48da-8c44-d36417f4633b" height="120">
|
||||
@ -19,24 +33,41 @@ cd Your_ComfyUI_Path/custom_nodes
|
||||
git clone https://huggingface.co/turtle89431/Moxie-Multimedia
|
||||
```
|
||||
|
||||
### Installing dependencies
|
||||
|
||||
**ComfyUI Manager** installs `requirements.txt` automatically. If you cloned the repository manually, install the dependencies yourself:
|
||||
|
||||
**ComfyUI Portable (Windows)** — run from the portable root folder (the one containing `run_nvidia_gpu.bat`):
|
||||
|
||||
```bat
|
||||
cd Your_ComfyUI_windows_portable
|
||||
.\python_embeded\python.exe -m pip install -r .\ComfyUI\custom_nodes\Moxie-Multimedia\requirements.txt
|
||||
```
|
||||
|
||||
**Regular ComfyUI (venv / system Python):**
|
||||
|
||||
```bash
|
||||
pip install -r Your_ComfyUI_Path/custom_nodes/Moxie-Multimedia/requirements.txt
|
||||
```
|
||||
|
||||
Restart ComfyUI afterwards.
|
||||
|
||||
> [!NOTE]
|
||||
> The pack targets **NVIDIA RTX 3060+ GPUs only**, so every feature dependency is mandatory and installed by the command above — including speech generation (`voxcpm`), subtitle recognition (`openai-whisper`, `qwen-asr`), and the RTX Video Super Resolution upscale (`nvidia-vfx`). Expect a sizable first install: `qwen-asr` pins `transformers==4.57.6` (pip may upgrade it inside your ComfyUI environment) and `voxcpm` needs `torch>=2.5.0`, so keep ComfyUI reasonably current. `nvidia-vfx` is pulled from NVIDIA's own package index (`pypi.nvidia.com`, wired up inside `requirements.txt`) because the PyPI copy is a stub that fails to build on some setups — if the command is interrupted, re-run it and it resumes where it stopped.
|
||||
|
||||
### Models
|
||||
|
||||
All model files ship **inside the repo** — cloning it into `custom_nodes` is all that's needed. The loader finds everything automatically in the pack's `models/` folders:
|
||||
|
||||
| File | Location |
|
||||
|---|---|
|
||||
| Diffusion model (`Moxie-Multimedia.safetensors`) | `models/diffusion/` |
|
||||
| Diffusion model (`Moxie-Multimedia.safetensors`, distill weights baked in) | `models/diffusion/` |
|
||||
| Text encoder (`MM-VL.safetensors`) | `models/clip/` |
|
||||
| Video VAE (bundled file) | `models/vae/` |
|
||||
| Audio VAE (bundled file) | `models/audio_vae/` |
|
||||
| Turbo LoRA (`MM3step.safetensors`, applied by default) | `models/loras/` |
|
||||
| Preview tiny VAE (`MM-preview.safetensors`) | `models/vae_approx/` |
|
||||
|
||||
If you obtained the pack without the bundled files, the included downloader fetches and places them (idempotent and resumable — re-run if interrupted):
|
||||
|
||||
```bash
|
||||
bash custom_nodes/MMnode/download_models.sh
|
||||
```
|
||||
All model files live in the Hugging Face version of this repository — cloning it downloads the complete set, and there is no separate downloader step. If you obtained the pack from somewhere else and the `models/` folders are missing or incomplete, re-clone from Hugging Face to get the bundled files.
|
||||
|
||||
Restart ComfyUI afterwards.
|
||||
|
||||
|
||||
@ -1,116 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# download_models.sh - fetch the MiniMax H3 model set for the Moxie Multimedia
|
||||
# Loader and place it (renamed) into this node pack's models/ folders.
|
||||
#
|
||||
# Usage (from anywhere - the script targets its own pack directory):
|
||||
# bash custom_nodes/MMnode/download_models.sh
|
||||
#
|
||||
# Files are downloaded to "<name>.part" first and moved into place when
|
||||
# complete, so an interrupted run can be re-run safely (it resumes).
|
||||
# Already-complete files are skipped, so the script is idempotent.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODELS_DIR="$SCRIPT_DIR/models"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Model manifest: subdir/filename <- URL
|
||||
# Add the turbo LoRA here once a URL is available, e.g.:
|
||||
# "loras/Moxie-Turbo.safetensors|https://.../Moxie-Turbo.safetensors"
|
||||
# ---------------------------------------------------------------------------
|
||||
MANIFEST=(
|
||||
"diffusion/Moxie-Multimedia.safetensors|https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity/resolve/main/Minimax-h3_Singularity_ref2va_v1.3_Pruned_w4a8.safetensors"
|
||||
"clip/MM-VL.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
|
||||
"vae/video_vae.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_int8_convrot.safetensors"
|
||||
"audio_vae/audio_vae.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors"
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Downloader: curl first (resume via -C -), wget as fallback.
|
||||
# ---------------------------------------------------------------------------
|
||||
curl_supports_retry_all() {
|
||||
# --retry-all-errors exists since curl 7.71.0; compare versions safely.
|
||||
local v
|
||||
v="$(curl --version 2>/dev/null | head -n1 | awk '{print $2}')"
|
||||
[[ "$(printf '%s\n7.71.0\n' "$v" | sort -V | head -n1)" == "7.71.0" ]]
|
||||
}
|
||||
|
||||
fetch() {
|
||||
local url="$1" output="$2"
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
local retry_all=""
|
||||
if curl_supports_retry_all; then
|
||||
retry_all="--retry-all-errors" # HuggingFace answers bursts with 429/503
|
||||
fi
|
||||
curl -L --fail --retry 5 --retry-delay 5 $retry_all \
|
||||
--connect-timeout 30 -C - -o "$output" "$url"
|
||||
elif command -v wget >/dev/null 2>&1; then
|
||||
wget -c -q --show-progress -O "$output" "$url"
|
||||
else
|
||||
echo "ERROR: neither curl nor wget is installed." >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
download_entry() {
|
||||
local entry="$1"
|
||||
local rel="${entry%%|*}"
|
||||
local url="${entry#*|}"
|
||||
local subdir="${rel%%/*}"
|
||||
local filename="${rel#*/}"
|
||||
local target_dir="$MODELS_DIR/$subdir"
|
||||
local target="$target_dir/$filename"
|
||||
local partial="$target.part"
|
||||
|
||||
if [[ -f "$target" ]]; then
|
||||
echo "[skip] $subdir/$filename already present"
|
||||
return 0
|
||||
fi
|
||||
|
||||
mkdir -p "$target_dir"
|
||||
echo "[get ] $subdir/$filename"
|
||||
echo " from $url"
|
||||
|
||||
if ! fetch "$url" "$partial"; then
|
||||
# A stale/incomplete .part can make resume fail (e.g. HTTP 416 when the
|
||||
# file was already fully fetched but the move never happened).
|
||||
if [[ -f "$partial" ]]; then
|
||||
echo " resume failed - deleting partial and restarting download"
|
||||
rm -f "$partial"
|
||||
fetch "$url" "$partial"
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
|
||||
mv -f "$partial" "$target"
|
||||
echo "[done] $subdir/$filename ($(du -h "$target" | cut -f1))"
|
||||
}
|
||||
|
||||
echo "Moxie Multimedia - model downloader"
|
||||
echo " pack models dir: $MODELS_DIR"
|
||||
echo
|
||||
|
||||
failed=0
|
||||
for entry in "${MANIFEST[@]}"; do
|
||||
if ! download_entry "$entry"; then
|
||||
echo "ERROR: failed to download $entry" >&2
|
||||
failed=1
|
||||
fi
|
||||
done
|
||||
|
||||
echo
|
||||
if [[ "$failed" -eq 0 ]]; then
|
||||
echo "All models ready:"
|
||||
for entry in "${MANIFEST[@]}"; do
|
||||
local_rel="${entry%%|*}"
|
||||
printf ' %-40s %s\n' "$local_rel" "$(du -h "$MODELS_DIR/$local_rel" 2>/dev/null | cut -f1 || echo missing)"
|
||||
done
|
||||
echo
|
||||
echo "Restart ComfyUI so the loader picks the files up."
|
||||
else
|
||||
echo "Some downloads failed - re-run this script to resume." >&2
|
||||
exit 1
|
||||
fi
|
||||
@ -19,7 +19,6 @@ SUBDIR_HINTS = {
|
||||
"vae": "models/vae",
|
||||
"audio_vae": "models/audio_vae",
|
||||
"clip": "models/clip",
|
||||
"loras": "models/loras",
|
||||
}
|
||||
|
||||
|
||||
@ -53,18 +52,11 @@ def _resolve(subdir, name):
|
||||
return path
|
||||
|
||||
|
||||
# (path, sd, metadata) cache for the last parsed LoRA, mirroring the native
|
||||
# LoraLoader. Kept at module level on purpose: newer ComfyUI builds lock node
|
||||
# classes after registration, so writing class attributes on the node class
|
||||
# itself raises "Cannot modify class attribute ... on locked class".
|
||||
_loaded_lora = None
|
||||
|
||||
|
||||
def _auto_pick(subdir):
|
||||
"""Return the model file to use for a hidden picker.
|
||||
|
||||
End users only see the diffusion model widget; every auxiliary model
|
||||
(video VAE, audio VAE, CLIP, LoRA) is auto-selected from the pack's
|
||||
(video VAE, audio VAE, CLIP) is auto-selected from the pack's
|
||||
models/<subdir> folder - the first file in sorted order, which for the
|
||||
published pack is exactly the bundled file. A missing file keeps the
|
||||
loud, actionable error from _resolve.
|
||||
@ -85,14 +77,6 @@ def _auto_pick(subdir):
|
||||
return selected
|
||||
|
||||
|
||||
def _auto_lora():
|
||||
"""Bundled LoRA file, or ``None`` when models/loras is empty."""
|
||||
files = _list_files("loras")
|
||||
if files[0].startswith("<no files found"):
|
||||
return None
|
||||
return files[0]
|
||||
|
||||
|
||||
def _h3_clip_type():
|
||||
"""MiniMax H3 CLIP recipe. The Moxie model set is H3-only, so the text
|
||||
encoder type is intentionally NOT a user-facing choice."""
|
||||
@ -110,12 +94,12 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
"""All-in-one loader that looks like it loads a single model.
|
||||
|
||||
The only visible picker is the diffusion model; the attention backend is
|
||||
applied to it, the bundled LoRA (if any), the video VAE, the audio VAE
|
||||
and the CLIP are auto-selected from the node pack's models/ folder.
|
||||
applied to it, and the video VAE, the audio VAE and the CLIP are
|
||||
auto-selected from the node pack's models/ folder.
|
||||
|
||||
The model set is MiniMax H3-only: the CLIP recipe is locked to
|
||||
CLIPType.MINIMAX and outputs are exactly (model, clip, video_vae, audio_vae)
|
||||
with the attention backend and LoRA already applied to the model."""
|
||||
with the attention backend already applied to the model."""
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> io.Schema:
|
||||
@ -123,7 +107,7 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
node_id="MoxieMultimediaLoader",
|
||||
display_name="Moxie Multimedia Loader",
|
||||
category="loaders/moxie",
|
||||
description="Loads the Moxie diffusion model (with the attention backend and the bundled LoRA "
|
||||
description="Loads the Moxie diffusion model (with the attention backend "
|
||||
"already applied) plus the video VAE, audio VAE and text encoder that ship with "
|
||||
"this custom node - one model picker, everything else is handled automatically. "
|
||||
"Outputs model, clip, video_vae and audio_vae.",
|
||||
@ -148,8 +132,8 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
io.Model.Output(display_name="model", tooltip="Diffusion model with attention backend and LoRA applied."),
|
||||
io.Clip.Output(display_name="clip", tooltip="CLIP text encoder (with LoRA applied if present)."),
|
||||
io.Model.Output(display_name="model", tooltip="Diffusion model with attention backend applied."),
|
||||
io.Clip.Output(display_name="clip", tooltip="CLIP text encoder."),
|
||||
io.Vae.Output(display_name="video_vae", tooltip="Loaded video/image VAE."),
|
||||
io.Vae.Output(display_name="audio_vae", tooltip="Loaded audio VAE."),
|
||||
],
|
||||
@ -189,36 +173,6 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
m.set_model_optimized_attention(attention_function)
|
||||
return m
|
||||
|
||||
@classmethod
|
||||
def _apply_lora(cls, model, clip, lora_name, strength_model, strength_clip):
|
||||
global _loaded_lora
|
||||
if (
|
||||
lora_name is None
|
||||
or lora_name == "none"
|
||||
or lora_name.startswith("<no files found")
|
||||
or (strength_model == 0 and strength_clip == 0)
|
||||
):
|
||||
return model, clip
|
||||
|
||||
lora_path = _resolve("loras", lora_name)
|
||||
lora = None
|
||||
lora_metadata = None
|
||||
if _loaded_lora is not None:
|
||||
if _loaded_lora[0] == lora_path:
|
||||
lora = _loaded_lora[1]
|
||||
lora_metadata = _loaded_lora[2] if len(_loaded_lora) > 2 else None
|
||||
else:
|
||||
_loaded_lora = None
|
||||
|
||||
if lora is None:
|
||||
lora, lora_metadata = comfy.utils.load_torch_file(lora_path, safe_load=True, return_metadata=True)
|
||||
_loaded_lora = (lora_path, lora, lora_metadata)
|
||||
|
||||
model_lora, clip_lora = comfy.sd.load_lora_for_models(
|
||||
model, clip, lora, strength_model, strength_clip, lora_metadata=lora_metadata
|
||||
)
|
||||
return model_lora, clip_lora
|
||||
|
||||
@classmethod
|
||||
def _load_vae(cls, vae_name, subdir="vae"):
|
||||
path = _resolve(subdir, vae_name)
|
||||
@ -256,15 +210,13 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
weight_dtype,
|
||||
attention_backend,
|
||||
) -> io.NodeOutput:
|
||||
# Chain: Load Diffusion Model -> Model Attention Backend -> Lora Loader (needs both model and clip)
|
||||
# Chain: Load Diffusion Model -> Model Attention Backend
|
||||
model = cls._load_diffusion_model(diffusion_model, weight_dtype)
|
||||
model = cls._apply_attention_backend(model, attention_backend)
|
||||
|
||||
# Hidden pickers: auxiliary models are auto-selected from the pack's
|
||||
# models/ folder so end users only ever see the diffusion model.
|
||||
clip_obj = cls._load_clip(_auto_pick("clip"), "default")
|
||||
lora_name = _auto_lora()
|
||||
model, clip_obj = cls._apply_lora(model, clip_obj, lora_name, 1.0, 1.0)
|
||||
|
||||
video_vae_obj = cls._load_vae(_auto_pick("vae"), "vae")
|
||||
audio_vae_obj = cls._load_vae(_auto_pick("audio_vae"), "audio_vae")
|
||||
|
||||
@ -1006,7 +1006,7 @@ class ModelPreviewOverrideKJ(io.ComfyNode):
|
||||
inputs=[
|
||||
io.Model.Input(
|
||||
"model",
|
||||
tooltip="Diffusion model (with attention backend and LoRA already applied).",
|
||||
tooltip="Diffusion model (with attention backend already applied).",
|
||||
),
|
||||
io.Int.Input(
|
||||
"max_resolution",
|
||||
|
||||
@ -5,6 +5,8 @@ from comfy_api.latest import io
|
||||
|
||||
from .nodes.project import (
|
||||
EasyMultiTrackProject,
|
||||
H3_CONTEXT_LENGTH_OPTIONS,
|
||||
H3_CONTEXT_SOURCE_FRAMES,
|
||||
TYPE_TRACKS_INFO,
|
||||
_first_input,
|
||||
)
|
||||
@ -52,7 +54,7 @@ class MoxieProject(EasyMultiTrackProject):
|
||||
not_idempotent=True,
|
||||
inputs=[
|
||||
TYPE_TRACKS_INFO.Input("tracks_info"),
|
||||
io.Model.Input("model", tooltip="Diffusion model from Moxie Multimedia Loader (attention backend and LoRA already applied)."),
|
||||
io.Model.Input("model", tooltip="Diffusion model from Moxie Multimedia Loader (attention backend already applied)."),
|
||||
io.Clip.Input("clip", tooltip="CLIP text encoder from Moxie Multimedia Loader."),
|
||||
io.Vae.Input("video_vae", tooltip="Video VAE from Moxie Multimedia Loader."),
|
||||
io.Vae.Input("audio_vae", tooltip="Audio VAE from Moxie Multimedia Loader (required to decode Moxie audio)."),
|
||||
@ -63,8 +65,21 @@ class MoxieProject(EasyMultiTrackProject):
|
||||
default=3,
|
||||
min=1,
|
||||
max=10000,
|
||||
tooltip="Sampling steps for the beta scheduler (3 for the 3-step turbo LoRA; "
|
||||
"8 for busy motion with regular LoRAs, 6 low motion, 4 runs/no dialogue).",
|
||||
tooltip="Sampling steps for the beta scheduler (3 for the baked-in 3-step distill; "
|
||||
"8 for busy motion, 6 low motion, 4 runs/no dialogue).",
|
||||
),
|
||||
io.Combo.Input(
|
||||
"context_frames",
|
||||
options=[str(option) for option in H3_CONTEXT_LENGTH_OPTIONS],
|
||||
default=str(H3_CONTEXT_SOURCE_FRAMES),
|
||||
tooltip=(
|
||||
"Video frames copied from the previous segment as "
|
||||
"continuation context for segments whose continuity "
|
||||
"mode is Context or Context Swap. Longer values hand "
|
||||
"off motion more smoothly but generate that many extra "
|
||||
"frames per segment. Values are H3 temporal-grid "
|
||||
"lengths shared with the Moxie context sampler nodes."
|
||||
),
|
||||
),
|
||||
io.Int.Input(
|
||||
"seed",
|
||||
|
||||
@ -57,6 +57,25 @@ from ..utils.video import passthrough_video_media
|
||||
|
||||
CATEGORY_MINIMAX = "EasyUse/MiniMax"
|
||||
CANVAS_MULTIPLE = 32
|
||||
|
||||
|
||||
def _align_canvas_dimension(value: Any) -> int:
|
||||
"""Floor a canvas side onto the H3 32px grid (CANVAS_MULTIPLE).
|
||||
|
||||
The video VAE latent grid is pixels // 16 and the diffusion model
|
||||
patchifies that latent with a 2x2 patch, so every canvas side must be a
|
||||
multiple of 32. Anything else produces an odd latent dimension and crashes
|
||||
comfy's patchify_video reshape ("shape '[1, 24, ...]' is invalid for input
|
||||
of size N"). Floors instead of rounds so alignment never upsamples past
|
||||
the requested size.
|
||||
"""
|
||||
try:
|
||||
number = int(value)
|
||||
except (TypeError, ValueError):
|
||||
return CANVAS_MULTIPLE
|
||||
return max(CANVAS_MULTIPLE, number // CANVAS_MULTIPLE * CANVAS_MULTIPLE)
|
||||
|
||||
|
||||
BASE_SHORT_EDGE = 768
|
||||
MAX_PIXELS = 768 * 1344
|
||||
REF_IMAGE_SHORT_EDGE = 2048
|
||||
@ -489,6 +508,8 @@ class MiniMaxH3ImageToVideoFallback(io.ComfyNode):
|
||||
first_frame: torch.Tensor | None = None,
|
||||
last_frame: torch.Tensor | None = None,
|
||||
) -> io.NodeOutput:
|
||||
width = _align_canvas_dimension(width)
|
||||
height = _align_canvas_dimension(height)
|
||||
latent, frame_count = _empty_av_latent(width, height, length)
|
||||
token_images: list[torch.Tensor] = []
|
||||
keyframes: list[dict[str, Any]] = []
|
||||
@ -617,6 +638,8 @@ class MiniMaxH3ReferenceToVideoFallback(io.ComfyNode):
|
||||
ref_video_audios: dict[str, dict[str, Any] | None] | None = None,
|
||||
ref_audios: dict[str, dict[str, Any]] | None = None,
|
||||
) -> io.NodeOutput:
|
||||
width = _align_canvas_dimension(width)
|
||||
height = _align_canvas_dimension(height)
|
||||
latent, frame_count = _empty_av_latent(width, height, length)
|
||||
reference_items: list[dict[str, Any]] = []
|
||||
reference_blocks: list[dict[str, Any]] = []
|
||||
@ -662,6 +685,20 @@ class MiniMaxH3ReferenceToVideoFallback(io.ComfyNode):
|
||||
continue
|
||||
soundtrack = video_audios.get("ref_video_audio_" + name.rsplit("_", 1)[-1])
|
||||
frames = _prepare_reference_video_frames(frames, frame_count)
|
||||
# Grid-align the reference video canvas: comfy core's
|
||||
# patchify_video floors each latent side into 2x2 patches, so an
|
||||
# off-grid frame size (API callers can inject arbitrary media)
|
||||
# would produce an odd latent dimension and crash sampling with
|
||||
# "shape '[1, 24, ...]' is invalid for input of size N". ComfyUI
|
||||
# builds with native H3 nodes do this alignment inside core's
|
||||
# MiniMaxH3ReferenceToVideo; this fallback copy must match.
|
||||
ref_height, ref_width = int(frames.shape[1]), int(frames.shape[2])
|
||||
aligned_ref_width = _align_canvas_dimension(ref_width)
|
||||
aligned_ref_height = _align_canvas_dimension(ref_height)
|
||||
if (aligned_ref_width, aligned_ref_height) != (ref_width, ref_height):
|
||||
frames = _resize(
|
||||
frames, aligned_ref_width, aligned_ref_height, "disabled"
|
||||
)
|
||||
aligned_count = frames.shape[0]
|
||||
canvas_height, canvas_width = frames.shape[1], frames.shape[2]
|
||||
if aligned_count < 5:
|
||||
@ -2742,8 +2779,8 @@ class EasyMiniMaxH3ToVideo(io.ComfyNode):
|
||||
selected_vae = _first_input(vae)
|
||||
selected_audio_vae = _first_input(audio_vae)
|
||||
prompt_text = str(_first_input(prompt, ""))
|
||||
target_width = int(_first_input(width, 1344))
|
||||
target_height = int(_first_input(height, 768))
|
||||
target_width = _align_canvas_dimension(_first_input(width, 1344))
|
||||
target_height = _align_canvas_dimension(_first_input(height, 768))
|
||||
target_length = int(_first_input(length, 124))
|
||||
locked_timing_frames = int(_first_input(locked_video_timing_frames, 0))
|
||||
expanded_images = expand_image_inputs(images)
|
||||
|
||||
@ -46,6 +46,7 @@ from ..utils.multitrack import (
|
||||
MULTITRACK_RUNTIME_CACHE_KEY,
|
||||
multitrack_runtime_cache,
|
||||
)
|
||||
from .minimax import CANVAS_MULTIPLE, _align_canvas_dimension
|
||||
|
||||
|
||||
TYPE_FAST_MODEL_LOADER = io.Custom(io_type="FAST_MODEL_LOADER")
|
||||
@ -54,6 +55,12 @@ TYPE_PROJECT_DATA = io.Custom(io_type="PROJECT_DATA")
|
||||
TYPE_H3_PROJECT_STATIC_DATA = io.Custom(io_type="H3_PROJECT_STATIC_DATA")
|
||||
H3_CONTEXT_CONTINUITY_MODES = {"context", "context_swap"}
|
||||
H3_CONTEXT_SOURCE_FRAMES = 22
|
||||
# Context lengths supported by the H3 continuation sampler nodes; every
|
||||
# value lands on the H3 temporal latent grid (17k + 5 frames).
|
||||
H3_CONTEXT_LENGTH_OPTIONS = (5, 22, 39, 56)
|
||||
# Extra frames generated past the context prefix so the decoded clip ends
|
||||
# on a phase-aligned H3 grid boundary before the tail is trimmed.
|
||||
H3_CONTEXT_GRID_TAIL_FRAMES = 12
|
||||
|
||||
|
||||
def _first_input(value: Any, default: Any = None) -> Any:
|
||||
@ -62,6 +69,18 @@ def _first_input(value: Any, default: Any = None) -> Any:
|
||||
return value if value is not None else default
|
||||
|
||||
|
||||
def _h3_context_frame_count(value: Any) -> int:
|
||||
"""Resolve the context_frames widget to a supported H3 context length."""
|
||||
frames = int(_first_input(value, str(H3_CONTEXT_SOURCE_FRAMES)))
|
||||
if frames not in H3_CONTEXT_LENGTH_OPTIONS:
|
||||
options = ", ".join(str(option) for option in H3_CONTEXT_LENGTH_OPTIONS)
|
||||
raise ValueError(
|
||||
"context_frames must be one of the supported H3 context lengths: "
|
||||
f"{options} (got {frames})"
|
||||
)
|
||||
return frames
|
||||
|
||||
|
||||
def _raw_project_input(value: Any) -> Any:
|
||||
"""Unwrap list inputs without splitting a raw graph link."""
|
||||
if is_link(value):
|
||||
@ -930,6 +949,19 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
"mode preserves existing video and latent files."
|
||||
),
|
||||
),
|
||||
io.Combo.Input(
|
||||
"context_frames",
|
||||
options=[str(option) for option in H3_CONTEXT_LENGTH_OPTIONS],
|
||||
default=str(H3_CONTEXT_SOURCE_FRAMES),
|
||||
tooltip=(
|
||||
"Video frames copied from the previous segment as "
|
||||
"continuation context for segments whose continuity "
|
||||
"mode is Context or Context Swap. Longer values hand "
|
||||
"off motion more smoothly but generate that many extra "
|
||||
"frames per segment. Values are H3 temporal-grid "
|
||||
"lengths shared with the Moxie context sampler nodes."
|
||||
),
|
||||
),
|
||||
io.Int.Input(
|
||||
"seed",
|
||||
default=42,
|
||||
@ -1079,6 +1111,19 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
if not uses_linked_prepare:
|
||||
_require_minimax_h3_model(model)
|
||||
info = parse_tracks_info(kwargs.get("tracks_info"))
|
||||
# External app clients (the Moxie app drives this node through the
|
||||
# ComfyUI API) can inject project dimensions that are not on the H3
|
||||
# 32px canvas grid; the MultiTrack Editor UI only offers grid sizes.
|
||||
# Every latent side is pixels // 16 and the diffusion model patchifies
|
||||
# that latent 2x2, so any off-grid side eventually reaches comfy's
|
||||
# patchify_video as an odd latent dimension and crashes first sampling
|
||||
# with "shape '[1, 24, ...]' is invalid for input of size N". Floor
|
||||
# both sides onto the grid once, here, so media loading, reference
|
||||
# encoding, both sampling passes and the exported project all agree
|
||||
# on the same aligned canvas. Editor-selected sizes are already
|
||||
# multiples of 32, so this is a no-op for UI workflows.
|
||||
info["width"] = _align_canvas_dimension(info["width"])
|
||||
info["height"] = _align_canvas_dimension(info["height"])
|
||||
hidden_inputs = getattr(cls, "hidden", None)
|
||||
validate_h3_project_outputs(
|
||||
info,
|
||||
@ -1090,6 +1135,9 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
sampling_mode, sampling_config = _h3_sampling_mode_config(
|
||||
kwargs.get("sampling_mode")
|
||||
)
|
||||
context_source_frames = _h3_context_frame_count(
|
||||
kwargs.get("context_frames")
|
||||
)
|
||||
is_passthrough = sampling_mode == "passthrough"
|
||||
is_selflift = sampling_mode == "selflift"
|
||||
has_second_pass = sampling_mode == "dual"
|
||||
@ -1181,8 +1229,23 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
# keeps the editor size exactly as configured.
|
||||
first_pass_width, first_pass_height = editor_width, editor_height
|
||||
if sampling_mode == "single" and not audio_only:
|
||||
half_width = max(32, editor_width // 2)
|
||||
half_height = max(32, editor_height // 2)
|
||||
# The half-resolution canvas must stay on the H3 32px grid
|
||||
# (CANVAS_MULTIPLE): the video VAE downsamples by 16 and the
|
||||
# diffusion model patchifies the latent with a 2x2 patch, so a
|
||||
# first-pass side that is not a multiple of 32 yields an odd
|
||||
# latent dimension and crashes comfy's patchify_video with
|
||||
# "shape '[1, 24, 1, 1, ...]' is invalid for input of size N"
|
||||
# (e.g. a 544x960 editor project halved to 272x480 produces a
|
||||
# 17x30 latent). Floor to the grid instead of rounding so the
|
||||
# first pass never upsamples past half the editor resolution.
|
||||
half_width = max(
|
||||
32,
|
||||
(editor_width // 2) // CANVAS_MULTIPLE * CANVAS_MULTIPLE,
|
||||
)
|
||||
half_height = max(
|
||||
32,
|
||||
(editor_height // 2) // CANVAS_MULTIPLE * CANVAS_MULTIPLE,
|
||||
)
|
||||
if (half_width, half_height) != (32, 32):
|
||||
first_pass_width, first_pass_height = half_width, half_height
|
||||
target_width, target_height = h3_second_pass_dimensions(
|
||||
@ -1210,8 +1273,19 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
segment_count,
|
||||
)
|
||||
if not selected_entries:
|
||||
if not all_entries:
|
||||
raise ValueError(
|
||||
"No H3 task segments are available: the MultiTrack timeline "
|
||||
"does not contain any task-track segments. Add at least one "
|
||||
"task segment in the MultiTrack Editor and run the project "
|
||||
"again."
|
||||
)
|
||||
raise ValueError(
|
||||
"No H3 task segments are available from segment_start_number."
|
||||
"No H3 task segments are available from segment_start_number="
|
||||
f"{segment_start_number} (segment_count={segment_count}): the "
|
||||
f"timeline only contains {len(all_entries)} task segment(s), "
|
||||
"numbered from 1. Lower segment_start_number or add more task "
|
||||
"segments in the MultiTrack Editor."
|
||||
)
|
||||
|
||||
first_selected_index, first_selected_entry = selected_entries[0]
|
||||
@ -1502,12 +1576,13 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
context_latent = _h3_encode_context_media(
|
||||
graph, output_images, output_audio, vae, audio_vae,
|
||||
f"passthrough_context_{task_index}",
|
||||
context_frames=context_source_frames,
|
||||
)
|
||||
runtime_context_latent = graph.node(
|
||||
"easy h3MotionContextLatentTrim",
|
||||
id=f"trim_hires_context_latent_{task_index}",
|
||||
latent=context_latent,
|
||||
context_length=str(H3_CONTEXT_SOURCE_FRAMES),
|
||||
context_length=str(context_source_frames),
|
||||
).out(0)
|
||||
saved_video = graph.node(
|
||||
"easy saveVideo",
|
||||
@ -1568,8 +1643,9 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
uses_context
|
||||
and (previous_hires_context_latent is not None or task_index > 0)
|
||||
)
|
||||
context_source_frames = H3_CONTEXT_SOURCE_FRAMES
|
||||
context_generation_frames = 34
|
||||
context_generation_frames = (
|
||||
context_source_frames + H3_CONTEXT_GRID_TAIL_FRAMES
|
||||
)
|
||||
if will_have_context_continuity:
|
||||
task_length = graph.node(
|
||||
"ComfyMathExpression",
|
||||
@ -1926,7 +2002,7 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
id=f"hires_continuity_{task_index}",
|
||||
current_hires_latent=upscaled_latent,
|
||||
previous_hires_latent=previous_hires_context_latent,
|
||||
context_length=str(H3_CONTEXT_SOURCE_FRAMES),
|
||||
context_length=str(context_source_frames),
|
||||
video_transition_steps=4,
|
||||
video_anchor_only=not audio_only,
|
||||
)
|
||||
@ -2116,6 +2192,7 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
vae,
|
||||
audio_vae,
|
||||
f"hires_context_{task_index}",
|
||||
context_frames=context_source_frames,
|
||||
)
|
||||
hires_context_reencoded = True
|
||||
if (has_second_pass and run_second_pass) or (
|
||||
@ -2159,6 +2236,7 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
vae,
|
||||
audio_vae,
|
||||
f"low_context_{task_index}",
|
||||
context_frames=context_source_frames,
|
||||
)
|
||||
low_context_reencoded = True
|
||||
else:
|
||||
|
||||
59
requirements.txt
Normal file
59
requirements.txt
Normal file
@ -0,0 +1,59 @@
|
||||
# Moxie Multimedia Suite - Python dependencies
|
||||
#
|
||||
# ComfyUI-Manager installs this file automatically.
|
||||
# If you cloned the pack manually, install it yourself - see README.md
|
||||
# ("Installing dependencies") for the ComfyUI Portable command.
|
||||
#
|
||||
# This pack targets NVIDIA RTX 3060+ GPUs only, so every feature
|
||||
# dependency ships as mandatory - speech, subtitle recognition, and the
|
||||
# RTX Video Super Resolution upscale are all installed up front. The
|
||||
# list is ordered light -> heavy: if a large package ever fails to
|
||||
# install, everything above it is already in place and re-running the
|
||||
# command resumes.
|
||||
#
|
||||
# Notes:
|
||||
# - Core stack (torch, numpy, Pillow, PyYAML, aiohttp) comes with
|
||||
# ComfyUI itself.
|
||||
# - The two pip options below route nvidia-vfx to NVIDIA's own package
|
||||
# index: PyPI only hosts a stub sdist for nvidia-vfx whose build
|
||||
# backend (wheel_stub) fails on some setups, while pypi.nvidia.com
|
||||
# hosts the real prebuilt wheels (Windows + Linux, python 3.10-3.13).
|
||||
# --only-binary=nvidia-vfx guarantees the stub sdist is never built.
|
||||
# - qwen-asr pins transformers==4.57.6 / accelerate==1.12.0, so pip may
|
||||
# upgrade those two packages inside your ComfyUI environment.
|
||||
# - voxcpm needs torch>=2.5.0 and torchcodec (wheels exist for Windows
|
||||
# and Linux, python 3.10-3.14); keep ComfyUI reasonably current.
|
||||
# - ffmpeg-python drives the FFmpeg binary - keep FFmpeg in PATH.
|
||||
|
||||
--extra-index-url https://pypi.nvidia.com/
|
||||
--only-binary=nvidia-vfx
|
||||
|
||||
# Audio: reading/writing merged and generated audio
|
||||
soundfile
|
||||
|
||||
# Subtitles: CJK-aware smart line splitting
|
||||
jieba
|
||||
|
||||
# Model downloads from the media browser / helper routes
|
||||
huggingface_hub
|
||||
|
||||
# Smart split (shot detection) - ffmpeg CLI wrapper
|
||||
ffmpeg-python
|
||||
|
||||
# Smart split (shot detection helpers)
|
||||
opencv-python
|
||||
|
||||
# Smart split (dataset loaders)
|
||||
imageio
|
||||
|
||||
# RTX Video Super Resolution upscale (NVIDIA RTX GPUs only)
|
||||
nvidia-vfx
|
||||
|
||||
# Subtitle recognition (local Whisper)
|
||||
openai-whisper
|
||||
|
||||
# Subtitle recognition (Qwen ASR) - pins transformers==4.57.6
|
||||
qwen-asr
|
||||
|
||||
# Speech / voiceover generation (VoxCPM)
|
||||
voxcpm
|
||||
348
scripts/test_merge_failure_reason.py
Normal file
348
scripts/test_merge_failure_reason.py
Normal file
@ -0,0 +1,348 @@
|
||||
"""Standalone verification for merge_video_track_with_ffmpeg failure_reason.
|
||||
|
||||
Runs without ComfyUI/torch: stubs the heavy imports, then exercises every
|
||||
``return None`` path of merge_video_track_with_ffmpeg and the
|
||||
compose_h3_project_video error propagation, asserting each produces a
|
||||
specific, human-readable reason.
|
||||
|
||||
Usage: python scripts/test_merge_failure_reason.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
PACKAGE_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def _install_stubs() -> None:
|
||||
# Minimal torch stub: only attributes evaluated at IMPORT time are needed
|
||||
# (annotations like `torch.Tensor`, `torch.device`, `torch.dtype` in
|
||||
# image.py, and `torch.nn.functional` imported by image/audio/multitrack).
|
||||
# No torch function actually runs on the paths under test.
|
||||
if "torch" not in sys.modules:
|
||||
torch = types.ModuleType("torch")
|
||||
torch.__path__ = [] # mark as package so torch.nn.functional imports resolve
|
||||
nn = types.ModuleType("torch.nn")
|
||||
functional = types.ModuleType("torch.nn.functional")
|
||||
functional.pad = lambda *a, **k: None
|
||||
nn.functional = functional
|
||||
torch.nn = nn
|
||||
torch.Tensor = object
|
||||
torch.device = object
|
||||
torch.dtype = object
|
||||
torch.float32 = object()
|
||||
torch.float64 = object()
|
||||
sys.modules["torch"] = torch
|
||||
sys.modules["torch.nn"] = nn
|
||||
sys.modules["torch.nn.functional"] = functional
|
||||
|
||||
if "folder_paths" not in sys.modules:
|
||||
fp = types.ModuleType("folder_paths")
|
||||
temp_dir = PACKAGE_ROOT / ".tmp-temp"
|
||||
temp_dir.mkdir(exist_ok=True)
|
||||
fp.get_output_directory = lambda: str(PACKAGE_ROOT / ".tmp-output")
|
||||
fp.get_temp_directory = lambda: str(PACKAGE_ROOT / ".tmp-temp")
|
||||
fp.get_input_directory = lambda: ""
|
||||
fp.get_filename_list = lambda *a, **k: []
|
||||
fp.get_full_path = lambda *a, **k: None
|
||||
sys.modules["folder_paths"] = fp
|
||||
|
||||
if "server" not in sys.modules:
|
||||
sys.modules["server"] = types.ModuleType("server")
|
||||
|
||||
|
||||
_install_stubs()
|
||||
sys.path.insert(0, str(PACKAGE_ROOT))
|
||||
|
||||
# Import the REAL utils package: with the stubs above, the eager `from .image
|
||||
# import *` in utils/__init__.py resolves, so multitrack's `from . import
|
||||
# FFMPEG_RESIZE_METHODS, ...` also works and h3_project loads untouched.
|
||||
from utils.video import merge_video_track_with_ffmpeg # noqa: E402
|
||||
import utils.video as video_mod # noqa: E402
|
||||
import utils.h3_project as h3_project # noqa: E402
|
||||
|
||||
CHECKS: list[tuple[str, bool, str]] = []
|
||||
|
||||
|
||||
def check(name: str, condition: bool, detail: str = "") -> None:
|
||||
CHECKS.append((name, bool(condition), detail))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. FFmpeg not found: point PATH lookup at an empty search space.
|
||||
# ---------------------------------------------------------------------------
|
||||
import shutil # noqa: E402
|
||||
|
||||
_real_which = shutil.which
|
||||
shutil.which = lambda name, **k: None
|
||||
try:
|
||||
reasons: list[str] = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": "whatever.mp4", "start_frame": 0, "end_frame": 10}],
|
||||
10,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("ffmpeg-missing returns None", out is None)
|
||||
check(
|
||||
"ffmpeg-missing reason mentions FFmpeg executable",
|
||||
len(reasons) == 1 and "FFmpeg executable not found" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
finally:
|
||||
shutil.which = _real_which
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Without failure_reason the function still returns None silently and no
|
||||
# caller-visible behavior changes (backwards compatibility).
|
||||
# ---------------------------------------------------------------------------
|
||||
shutil.which = lambda name, **k: None
|
||||
try:
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": "whatever.mp4", "start_frame": 0, "end_frame": 10}],
|
||||
10,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
)
|
||||
check("no-collector path still returns None", out is None)
|
||||
finally:
|
||||
shutil.which = _real_which
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. ffmpeg "found": redirect get_ffmpeg_path via monkeypatched shutil.which to
|
||||
# a real executable so we reach the precondition checks.
|
||||
# ---------------------------------------------------------------------------
|
||||
import os # noqa: E402
|
||||
|
||||
fake_ffmpeg = Path(PACKAGE_ROOT / ".tmp-fake-ffmpeg")
|
||||
fake_ffmpeg.write_text("")
|
||||
os.chmod(fake_ffmpeg, 0o755)
|
||||
shutil.which = lambda name, **k: str(fake_ffmpeg) if name == "ffmpeg" else None
|
||||
try:
|
||||
# Empty timeline
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg([], 0, 24.0, 64, 64, failure_reason=reasons)
|
||||
check("empty timeline returns None", out is None)
|
||||
check(
|
||||
"empty timeline reason",
|
||||
len(reasons) == 1 and "timeline is empty" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
|
||||
# Invalid frame rate
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": "x.mp4", "start_frame": 0, "end_frame": 1}],
|
||||
1,
|
||||
0.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("invalid fps returns None", out is None)
|
||||
check("invalid fps reason", len(reasons) == 1 and "frame rate" in reasons[0], str(reasons))
|
||||
|
||||
# Odd dimensions
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": "x.mp4", "start_frame": 0, "end_frame": 1}],
|
||||
1,
|
||||
24.0,
|
||||
63,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("odd width returns None", out is None)
|
||||
check(
|
||||
"odd width reason names size",
|
||||
len(reasons) == 1 and "63x64" in reasons[0] and "odd" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
|
||||
# Missing segment sources
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[
|
||||
{"source": "/definitely/not/here/a.mp4", "start_frame": 0, "end_frame": 4},
|
||||
{"source": "/definitely/not/here/b.mp4", "start_frame": 4, "end_frame": 8},
|
||||
{"source": "/definitely/not/here/c.mp4", "start_frame": 8, "end_frame": 12},
|
||||
],
|
||||
12,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("missing sources returns None", out is None)
|
||||
check(
|
||||
"missing sources reason lists each file",
|
||||
len(reasons) == 1
|
||||
and "a.mp4" in reasons[0]
|
||||
and "b.mp4" in reasons[0]
|
||||
and "c.mp4" in reasons[0]
|
||||
and "missing" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
|
||||
# Missing locked audio
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[
|
||||
{
|
||||
"source": str(fake_ffmpeg),
|
||||
"audio_source": "/definitely/not/here/locked.wav",
|
||||
"start_frame": 0,
|
||||
"end_frame": 4,
|
||||
}
|
||||
],
|
||||
4,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("missing locked audio returns None", out is None)
|
||||
check(
|
||||
"locked audio reason names file",
|
||||
len(reasons) == 1 and "locked.wav" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
|
||||
# FFmpeg command failure: run the real (invalid) fake binary -> OSError or
|
||||
# non-zero return. The fake file is not executable code; on POSIX exec
|
||||
# raises OSError (ENOEXEC / format error), on the sandbox it surfaces as
|
||||
# either the OSError path or a non-zero returncode path. Both must record.
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": str(fake_ffmpeg), "start_frame": 0, "end_frame": 1}],
|
||||
1,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("command failure returns None", out is None)
|
||||
check(
|
||||
"command failure records a reason",
|
||||
len(reasons) == 1
|
||||
and ("failed to start" in reasons[0] or "exited with code" in reasons[0]),
|
||||
str(reasons),
|
||||
)
|
||||
finally:
|
||||
shutil.which = _real_which
|
||||
fake_ffmpeg.unlink(missing_ok=True)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3b. get_ffmpeg_path expanded Windows fallbacks (portable folder, winget/
|
||||
# scoop/choco shims, imageio-ffmpeg bundled binary). On this POSIX sandbox
|
||||
# os.name must be forced to "nt" so the fallback branches run.
|
||||
# ---------------------------------------------------------------------------
|
||||
import utils.video as _video_mod # noqa: E402
|
||||
|
||||
_real_os_name = os.name
|
||||
_real_base = getattr(_video_mod.folder_paths, "base_path", "")
|
||||
shutil.which = lambda name, **k: None
|
||||
os.name = "nt"
|
||||
try:
|
||||
# Portable layout: <base>/ffmpeg/bin/ffmpeg.exe found via the portable
|
||||
# root (dirname of folder_paths.base_path).
|
||||
fake_base = PACKAGE_ROOT / ".tmp-fake-base" / "ComfyUI"
|
||||
portable_exe = fake_base.parent / "ffmpeg" / "bin" / "ffmpeg.exe"
|
||||
portable_exe.parent.mkdir(parents=True, exist_ok=True)
|
||||
portable_exe.write_bytes(b"")
|
||||
_video_mod.folder_paths.base_path = str(fake_base)
|
||||
found = _video_mod.get_ffmpeg_path("ffmpeg")
|
||||
check(
|
||||
"portable ffmpeg/bin layout resolved",
|
||||
found == str(portable_exe),
|
||||
repr(found),
|
||||
)
|
||||
portable_exe.unlink()
|
||||
shutil.rmtree(PACKAGE_ROOT / ".tmp-fake-base", ignore_errors=True)
|
||||
_video_mod.folder_paths.base_path = _real_base
|
||||
|
||||
# ffmpeg (not ffprobe) falls back to the imageio-ffmpeg bundled binary.
|
||||
fake_imageio = types.ModuleType("imageio_ffmpeg")
|
||||
bundled = str(PACKAGE_ROOT / ".tmp-bundled-ffmpeg.exe")
|
||||
fake_imageio.get_ffmpeg_exe = lambda: bundled
|
||||
sys.modules["imageio_ffmpeg"] = fake_imageio
|
||||
found = _video_mod.get_ffmpeg_path("ffmpeg")
|
||||
check(
|
||||
"imageio-ffmpeg bundled binary used as last resort",
|
||||
found == bundled,
|
||||
repr(found),
|
||||
)
|
||||
# ffprobe is NOT bundled by imageio-ffmpeg: must NOT resolve to it.
|
||||
found = _video_mod.get_ffmpeg_path("ffprobe")
|
||||
check(
|
||||
"ffprobe never resolved from imageio-ffmpeg",
|
||||
found is None,
|
||||
repr(found),
|
||||
)
|
||||
del sys.modules["imageio_ffmpeg"]
|
||||
finally:
|
||||
os.name = _real_os_name
|
||||
_video_mod.folder_paths.base_path = _real_base
|
||||
shutil.which = _real_which
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. compose_h3_project_video propagates the reason into the RuntimeError.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _fake_merge(segments, total_length, frame_rate, w, h, **kwargs):
|
||||
reasons = kwargs.get("failure_reason")
|
||||
if reasons is not None:
|
||||
reasons.append("FFmpeg exited with code 1: Unknown encoder 'libx264'")
|
||||
return None
|
||||
|
||||
|
||||
# compose_h3_project_video imports merge_video_track_with_ffmpeg from the
|
||||
# `utils.video` module at call time (function-local import), so patch the
|
||||
# attribute on the already-loaded video module, not on h3_project.
|
||||
h3_project.load_h3_project_data = lambda *a, **k: {
|
||||
"width": 848,
|
||||
"height": 480,
|
||||
"frame_rate": 24.0,
|
||||
"clips": [
|
||||
{
|
||||
"index": 0,
|
||||
"enabled": True,
|
||||
"file_path": "easy_media/projects/p/out/seg_000.mp4",
|
||||
"updated_at": 1,
|
||||
"source_frame_count": 16,
|
||||
"source_start_frame": 0,
|
||||
"source_end_frame": 16,
|
||||
"video_files": [],
|
||||
}
|
||||
],
|
||||
}
|
||||
video_mod.merge_video_track_with_ffmpeg = _fake_merge
|
||||
|
||||
try:
|
||||
h3_project.compose_h3_project_video("proj", None)
|
||||
check("compose raises on None", False, "no exception raised")
|
||||
except RuntimeError as error:
|
||||
message = str(error)
|
||||
check(
|
||||
"compose error carries the real reason",
|
||||
"could not compose the H3 project video" in message
|
||||
and "Unknown encoder 'libx264'" in message,
|
||||
message,
|
||||
)
|
||||
except Exception as error: # noqa: BLE001
|
||||
check("compose raises RuntimeError only", False, repr(error))
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
passed = sum(1 for _, ok, _ in CHECKS if ok)
|
||||
failed = len(CHECKS) - passed
|
||||
for name, ok, detail in CHECKS:
|
||||
print(f"{'PASS' if ok else 'FAIL'}: {name}" + (f" [{detail}]" if detail and not ok else ""))
|
||||
print(f"\n{passed}/{len(CHECKS)} checks passed")
|
||||
sys.exit(1 if failed else 0)
|
||||
@ -294,8 +294,7 @@ def _install_loader_stubs(state):
|
||||
return (sd, {"meta": True}) if return_metadata else sd
|
||||
|
||||
def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None):
|
||||
state.calls.append(("load_lora_for_models", strength_model, strength_clip))
|
||||
return state.lora_model, clip
|
||||
raise AssertionError("MoxieMultimediaLoader must not load LoRAs anymore")
|
||||
|
||||
def get_attention_function(name, *args):
|
||||
state.calls.append(("get_attention_function", name))
|
||||
@ -352,7 +351,7 @@ def _load_loader_module():
|
||||
def verify_loader():
|
||||
print("\n== 4. moxie_loader.py: single-model UI + auto-picked aux models ==")
|
||||
state = types.SimpleNamespace(
|
||||
calls=[], clip=object(), attention_fn=object(), lora_model=object(),
|
||||
calls=[], clip=object(), attention_fn=object(),
|
||||
)
|
||||
_install_loader_stubs(state)
|
||||
loader = _load_loader_module()
|
||||
@ -377,13 +376,14 @@ def verify_loader():
|
||||
("vae", "video_vae.safetensors"),
|
||||
("audio_vae", "audio_vae.safetensors"),
|
||||
("clip", "MM-VL.safetensors"),
|
||||
# A leftover loras file must be ignored entirely: the distill
|
||||
# weights ship baked into the diffusion model now.
|
||||
("loras", "MM3step.safetensors"),
|
||||
):
|
||||
os.makedirs(os.path.join(models, subdir), exist_ok=True)
|
||||
with open(os.path.join(models, subdir, name), "wb") as handle:
|
||||
handle.write(b"stub")
|
||||
loader.MODELS_DIR = models
|
||||
loader._loaded_lora = None
|
||||
|
||||
result = loader.MoxieMultimediaLoader.execute(
|
||||
diffusion_model="moxie.safetensors",
|
||||
@ -392,37 +392,21 @@ def verify_loader():
|
||||
)
|
||||
|
||||
names = [call[0] for call in state.calls]
|
||||
check("full chain runs: model -> attention -> clip -> lora -> vaes",
|
||||
check("full chain runs: model -> attention -> clip -> vaes (no LoRA)",
|
||||
names == [
|
||||
"load_diffusion_model", "get_attention_function",
|
||||
"set_model_optimized_attention", "load_clip",
|
||||
"load_torch_file", "load_lora_for_models",
|
||||
"load_torch_file", "load_torch_file",
|
||||
], repr(names))
|
||||
lora_calls = [call for call in state.calls if call[0] == "load_lora_for_models"]
|
||||
check("LoRA auto-applied at strength 1.0/1.0",
|
||||
lora_calls == [("load_lora_for_models", 1.0, 1.0)], repr(lora_calls))
|
||||
clip_calls = [call for call in state.calls if call[0] == "load_clip"]
|
||||
check("CLIP auto-picked and loaded with the locked recipe",
|
||||
len(clip_calls) == 1 and clip_calls[0][1] is not None, repr(clip_calls))
|
||||
check("four outputs returned",
|
||||
isinstance(result, loader.io.NodeOutput) and len(result.values) == 4)
|
||||
check("model output carries LoRA",
|
||||
result.values[0] is state.lora_model)
|
||||
check("model output is the attention clone (no LoRA on top)",
|
||||
result.values[0] is not state.model)
|
||||
check("clip output is the loaded clip", result.values[1] is state.clip)
|
||||
|
||||
# Empty loras folder -> LoRA silently skipped, everything else loads.
|
||||
os.remove(os.path.join(models, "loras", "MM3step.safetensors"))
|
||||
state.calls.clear()
|
||||
loader._loaded_lora = None
|
||||
loader.MoxieMultimediaLoader.execute(
|
||||
diffusion_model="moxie.safetensors",
|
||||
weight_dtype="default",
|
||||
attention_backend="pytorch_attention",
|
||||
)
|
||||
check("empty loras folder skips LoRA without breaking the load",
|
||||
"load_lora_for_models" not in [call[0] for call in state.calls])
|
||||
|
||||
# Missing clip folder -> loud setup error (same message as before).
|
||||
state.calls.clear()
|
||||
os.rename(os.path.join(models, "clip"), os.path.join(models, "clip_backup"))
|
||||
|
||||
@ -266,6 +266,8 @@ custom 示例:
|
||||
- `segment_count = -1` 表示从起始索引处理到末尾;非负数限制本次任务片段数量。
|
||||
- `project_save`: `new` 或 `override`。在 `override` 且 count 为 `-1` 时,执行逻辑会删除起始索引之后的已保存片段再生成;修改前向用户清楚说明。
|
||||
- `sampling_mode`: `single` 或 `dual`;`sampling_plan` 必须是当前节点可用预设。不要仅凭另一份流程中的值假设当前环境支持。
|
||||
- `context_frames`(默认 `"22"`,可选 `"5"` / `"22"` / `"39"` / `"56"`)控制 Context / Context Swap 续接片段从上一段拷贝为延续上下文的视频帧数;取值必须落在 H3 时间潜空间网格(17k+5 帧)上,与 Moxie 上下文采样节点的选项一致。数值越大运动衔接越平滑,但每个续接片段会多生成等量帧(实际生成长度为 `任务长度 + context_frames + 12`,其中 12 帧为网格尾帧,解码后裁掉)。
|
||||
- `Moxie MultiTrack Project`(`MoxieProject`)同样暴露同一个 `context_frames` 下拉,位于 `steps` 与 `seed` 之间;该节点的其余采样设置仍保持隐藏(euler + beta sigmas 由 `steps` 推导)。
|
||||
- 编辑器分辨率是一采尺寸;`dual` 不会反算缩小一采。二采宽高分别为 `round(编辑器尺寸 * upscale_by / 32) * 32`,采用 Python `round` 就近对齐,恰好居中时取偶数,不是固定向上对齐。
|
||||
- `upscale_by` 是 Project 的独立参数,默认 `1.250`,三位小数,步长 `0.001`;旧工作流保留已保存的值。两条放大路径及合并导出均使用放大并对齐后的尺寸。例如 `1344 × 768`、倍率 `1.250` 的二采为 `1664 × 960`。若用户指定最终输出尺寸,应同时核对编辑器尺寸与倍率,不能将编辑器尺寸直接当作双采样的最终尺寸。
|
||||
- 单采样、仅一采预览保持编辑器尺寸;`upscale_by = 1.000` 不放大但可二采;纯音频项目(`32 × 32`)不放大。旧版反算缩小尺寸的一采检查点和上下文潜空间应重新生成后再按新规则续跑。
|
||||
@ -290,6 +292,7 @@ Plan 只包含实际变更。`node_id` 可省略并由拓扑定位;`editor.res
|
||||
"project_save": "override",
|
||||
"segment_start_number": 1,
|
||||
"segment_count": -1,
|
||||
"context_frames": "22",
|
||||
"sampling_plan": "medium",
|
||||
"sampling_mode": "single"
|
||||
}
|
||||
|
||||
@ -185,7 +185,7 @@ def _load_minimax_node(monkeypatch):
|
||||
Combo=_PortType,
|
||||
ComfyNode=object,
|
||||
Conditioning=_PortType,
|
||||
ControlAfterGenerate=types.SimpleNamespace(fixed="fixed"),
|
||||
ControlAfterGenerate=types.SimpleNamespace(fixed="fixed", randomize="randomize"),
|
||||
DynamicCombo=_DynamicCombo,
|
||||
Hidden=types.SimpleNamespace(prompt="PROMPT", unique_id="UNIQUE_ID"),
|
||||
Custom=lambda **kwargs: _PortType,
|
||||
@ -260,6 +260,12 @@ def _load_minimax_node(monkeypatch):
|
||||
comfy.model_management = model_management
|
||||
comfy.nested_tensor = nested_tensor
|
||||
comfy.utils = comfy_utils
|
||||
comfy_samplers = types.ModuleType("comfy.samplers")
|
||||
comfy_samplers.calculate_sigmas = (
|
||||
lambda *args, **kwargs: torch.arange(8.0)
|
||||
)
|
||||
comfy_samplers.sampler_object = lambda name: {"sampler": name}
|
||||
comfy.samplers = comfy_samplers
|
||||
|
||||
node_helpers = types.ModuleType("node_helpers")
|
||||
|
||||
@ -448,6 +454,7 @@ def _load_minimax_node(monkeypatch):
|
||||
"comfy.model_management": model_management,
|
||||
"comfy.nested_tensor": nested_tensor,
|
||||
"comfy.utils": comfy_utils,
|
||||
"comfy.samplers": comfy_samplers,
|
||||
"node_helpers": node_helpers,
|
||||
"torchaudio": torchaudio,
|
||||
"folder_paths": folder_paths,
|
||||
@ -516,6 +523,21 @@ def _load_moxie_combine(monkeypatch):
|
||||
return module
|
||||
|
||||
|
||||
def _load_moxie_project(monkeypatch):
|
||||
"""Load moxie_project.py into the stub environment set up above."""
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
root = Path(__file__).parents[1]
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"easy_media.moxie_project", root / "moxie_project.py"
|
||||
)
|
||||
assert spec is not None and spec.loader is not None
|
||||
moxie_module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = moxie_module
|
||||
spec.loader.exec_module(moxie_module)
|
||||
return moxie_module
|
||||
|
||||
|
||||
def _image_values(*values):
|
||||
return torch.tensor(values, dtype=torch.float32).reshape(len(values), 1, 1, 1)
|
||||
|
||||
@ -1238,6 +1260,7 @@ def test_multitrack_h3_project_schema_exposes_pipeline_configuration(monkeypatch
|
||||
"project_save",
|
||||
"segment_start_number",
|
||||
"segment_count",
|
||||
"context_frames",
|
||||
"seed",
|
||||
"sampling_plan",
|
||||
"sampling_mode",
|
||||
@ -1259,6 +1282,8 @@ def test_multitrack_h3_project_schema_exposes_pipeline_configuration(monkeypatch
|
||||
assert inputs["project_name"].kwargs["default"] == ""
|
||||
assert inputs["project_save"].kwargs["options"] == ["new", "override"]
|
||||
assert inputs["project_save"].kwargs["default"] == "override"
|
||||
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
|
||||
assert inputs["context_frames"].kwargs["default"] == "22"
|
||||
sampling_plan_options = inputs["sampling_plan"].kwargs["options"]
|
||||
assert sampling_plan_options == [
|
||||
"custom",
|
||||
@ -2447,6 +2472,53 @@ def test_multitrack_h3_project_reads_ref_image_size_from_each_segment(monkeypatc
|
||||
assert conditioning["ref_image_size"] == "max"
|
||||
|
||||
|
||||
def test_multitrack_h3_project_keeps_544x960_first_pass_on_grid(monkeypatch):
|
||||
"""Regression for the 2026-10-16 user crash report.
|
||||
|
||||
A 544x960 single-pass project used to sample its hidden first pass at the
|
||||
raw half resolution 272x480. The keyframe conditioned on that canvas is a
|
||||
17-wide latent, and comfy's patchify_video floors it into 2x2 patches,
|
||||
crashing with "shape '[1, 24, 1, 1, 15, 2, 8, 2]' is invalid for input of
|
||||
size 12240". The first pass must land on the 32px canvas grid instead.
|
||||
"""
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
module.comfy_nodes.NODE_CLASS_MAPPINGS["ImageResizeKJv2"] = _ImageResizeKJWithNvidia
|
||||
inputs = _h3_project_inputs(sampling_mode=_h3_sampling_mode("single"))
|
||||
inputs["tracks_info"][0]["width"] = 544
|
||||
inputs["tracks_info"][0]["height"] = 960
|
||||
|
||||
result = module.EasyMultiTrackProject.execute(**inputs)
|
||||
conditioning = _graph_node(result, "easy minimaxH3ToVideo")["inputs"]
|
||||
|
||||
# half of 544x960 stays on the 32px grid: 256x480 -> latent 16x30, even.
|
||||
assert conditioning["width"] == 256
|
||||
assert conditioning["height"] == 480
|
||||
|
||||
|
||||
def test_multitrack_h3_project_aligns_off_grid_api_dimensions(monkeypatch):
|
||||
"""External app clients inject project dimensions through the ComfyUI API
|
||||
that are not multiples of 32; the editor UI only offers grid sizes. The
|
||||
project must run both sampling passes on the same grid-aligned canvas
|
||||
instead of the raw injected size (odd latent -> patchify_video crash)."""
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
module.comfy_nodes.NODE_CLASS_MAPPINGS["ImageResizeKJv2"] = _ImageResizeKJWithNvidia
|
||||
inputs = _h3_project_inputs(sampling_mode=_h3_sampling_mode("dual"))
|
||||
inputs["tracks_info"][0]["width"] = 530
|
||||
inputs["tracks_info"][0]["height"] = 964
|
||||
|
||||
result = module.EasyMultiTrackProject.execute(**inputs)
|
||||
conditionings = [
|
||||
node["inputs"]
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy minimaxH3ToVideo"
|
||||
]
|
||||
|
||||
# 530x964 floors onto the 32px grid as 512x960; the default 1.25x second
|
||||
# pass rounds to 640x1216 (960*1.25=1200 rounds up to the 32px grid).
|
||||
assert sorted(item["width"] for item in conditionings) == [512, 640]
|
||||
assert sorted(item["height"] for item in conditionings) == [960, 1216]
|
||||
|
||||
|
||||
def test_multitrack_h3_project_logs_steps_and_reports_progress(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
@ -3749,6 +3821,89 @@ def test_multitrack_h3_context_chain_uses_previous_segment_latent(monkeypatch):
|
||||
assert artifacts[1][1]["inputs"]["previous"] == [artifacts[0][0], 0]
|
||||
|
||||
|
||||
def test_multitrack_context_frames_widget_propagates(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
module.comfy_nodes.NODE_CLASS_MAPPINGS["MiniMaxH3MotionContextTrim"] = (
|
||||
_MiniMaxMotionContextTrim
|
||||
)
|
||||
info = _h3_project_inputs()["tracks_info"][0]
|
||||
info["tracks"][0]["segments"].append(
|
||||
{
|
||||
"start_frame": 120,
|
||||
"end_frame": 240,
|
||||
"content": {
|
||||
"task_mode": "l2v",
|
||||
"continuity_mode": "context",
|
||||
"images": [{"media_index": 0}],
|
||||
"user_prompt": "continue",
|
||||
},
|
||||
}
|
||||
)
|
||||
info["tracks"].append(
|
||||
{
|
||||
"id": "locked-audio-track",
|
||||
"type": "audio",
|
||||
"audio_locked": True,
|
||||
"segments": [
|
||||
{
|
||||
"id": "locked-audio",
|
||||
"start_frame": 0,
|
||||
"end_frame": 240,
|
||||
"content": {"media_type": "audio"},
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
result = module.EasyMultiTrackProject.execute(
|
||||
**_h3_project_inputs(tracks_info=[info], context_frames=["5"])
|
||||
)
|
||||
|
||||
motion = next(
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy MiniMaxH3MotionContextHard"
|
||||
)
|
||||
assert motion["inputs"]["context_length"] == "5"
|
||||
context_trims = [
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy h3MotionContextLatentTrim"
|
||||
]
|
||||
assert context_trims
|
||||
assert {trim["inputs"]["context_length"] for trim in context_trims} == {"5"}
|
||||
audio_locks = [
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy minimaxH3AudioLock"
|
||||
]
|
||||
assert sorted(node["inputs"]["prepend_frames"] for node in audio_locks) == [0, 5]
|
||||
math_node = next(
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "ComfyMathExpression"
|
||||
)
|
||||
# The context continuation generates context_frames + 12 grid-tail frames.
|
||||
assert math_node["inputs"]["expression"] == "a + 17"
|
||||
context_encodes = [
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy h3ContextMediaTrim"
|
||||
and node["inputs"]["output_frames"] == 5
|
||||
]
|
||||
assert context_encodes
|
||||
|
||||
|
||||
def test_multitrack_context_frames_widget_rejects_off_grid_values(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
with pytest.raises(ValueError, match="supported H3 context lengths"):
|
||||
module.EasyMultiTrackProject.execute(
|
||||
**_h3_project_inputs(context_frames=["18"])
|
||||
)
|
||||
|
||||
|
||||
def test_multitrack_h3_context_uses_embedded_five_frame_anchor(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
info = _h3_project_inputs()["tracks_info"][0]
|
||||
@ -5214,6 +5369,32 @@ def test_reference_encodes_images_video_audio_and_standalone_audio(monkeypatch):
|
||||
assert conditioning["inputs"]["ref_video_0"][1] == 0
|
||||
|
||||
|
||||
def test_reference_fallback_grid_aligns_off_grid_ref_videos(monkeypatch):
|
||||
"""ComfyUI builds with native H3 nodes grid-align reference videos inside
|
||||
core's MiniMaxH3ReferenceToVideo; the fallback copy must do the same, or
|
||||
an off-grid media size (the API can inject arbitrary media) yields an odd
|
||||
latent dimension and crashes patchify_video during sampling."""
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
vae = _Vae()
|
||||
|
||||
module.MiniMaxH3ReferenceToVideoFallback.execute(
|
||||
clip=_Clip(),
|
||||
vae=vae,
|
||||
audio_vae=None,
|
||||
prompt="prompt",
|
||||
width=32,
|
||||
height=32,
|
||||
length=5,
|
||||
ref_videos={"ref_video_0": torch.zeros(6, 270, 480, 3)},
|
||||
)
|
||||
|
||||
encoded_frames = vae.encoded[-1]
|
||||
# Frames are [T, H, W, C]: 270 floors onto the 32px grid as 256; 480 is
|
||||
# already aligned.
|
||||
assert tuple(encoded_frames.shape[1:3]) == (256, 480)
|
||||
|
||||
|
||||
def test_reference_bridge_groups_fixed_inputs_before_direct_execute(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
@ -6094,3 +6275,68 @@ def test_minimax_prompt_override_node_has_complete_chinese_localization():
|
||||
assert translation["inputs"]["video_track_lock"]["name"] == "视频轨锁定"
|
||||
assert translation["inputs"]["audio_track_lock"]["name"] == "音频轨锁定"
|
||||
assert translation["outputs"] == {"0": {"name": "提示词覆盖"}}
|
||||
|
||||
|
||||
def test_moxie_project_schema_exposes_context_frames(monkeypatch):
|
||||
module = _load_moxie_project(monkeypatch)
|
||||
schema = module.MoxieProject.define_schema()
|
||||
inputs = {port.name: port for port in schema.inputs}
|
||||
|
||||
assert schema.node_id == "MoxieProject"
|
||||
assert schema.display_name == "Moxie MultiTrack Project"
|
||||
assert schema.is_input_list is True
|
||||
assert list(inputs) == [
|
||||
"tracks_info",
|
||||
"model",
|
||||
"clip",
|
||||
"video_vae",
|
||||
"audio_vae",
|
||||
"project_name",
|
||||
"steps",
|
||||
"context_frames",
|
||||
"seed",
|
||||
]
|
||||
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
|
||||
assert inputs["context_frames"].kwargs["default"] == "22"
|
||||
assert inputs["steps"].kwargs["default"] == 3
|
||||
assert inputs["seed"].kwargs["default"] == 42
|
||||
|
||||
|
||||
def test_moxie_project_forwards_context_frames_to_engine(monkeypatch):
|
||||
module = _load_moxie_project(monkeypatch)
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_execute(cls, **kwargs):
|
||||
captured.update(kwargs)
|
||||
return "sentinel-output"
|
||||
|
||||
monkeypatch.setattr(
|
||||
module.EasyMultiTrackProject, "execute", classmethod(fake_execute)
|
||||
)
|
||||
|
||||
model = types.SimpleNamespace(
|
||||
get_model_object=lambda name: types.SimpleNamespace()
|
||||
)
|
||||
result = module.MoxieProject.execute(
|
||||
tracks_info=[{"tracks": []}],
|
||||
model=[model],
|
||||
clip=[types.SimpleNamespace()],
|
||||
video_vae=[types.SimpleNamespace()],
|
||||
audio_vae=[types.SimpleNamespace()],
|
||||
project_name=["demo"],
|
||||
steps=["4"],
|
||||
context_frames=["5"],
|
||||
seed=["7"],
|
||||
)
|
||||
|
||||
assert result == "sentinel-output"
|
||||
assert captured["context_frames"] == ["5"]
|
||||
assert captured["steps"] == ["4"]
|
||||
assert captured["project_name"] == ["demo"]
|
||||
assert captured["seed"] == ["7"]
|
||||
# Baked settings survive the handoff and never clobber the widget.
|
||||
assert captured["sampling_plan"] == "custom"
|
||||
assert captured["sampling_mode"] == "single"
|
||||
assert captured["sampler"] == {"sampler": "euler"}
|
||||
assert captured["sigmas"].shape == (5,)
|
||||
|
||||
@ -7,7 +7,8 @@ Verified contracts:
|
||||
- MoxieMultimediaLoader: outputs exactly (model, clip, video_vae, audio_vae);
|
||||
no legacy FAST_MODEL_LOADER bundle output; no clip_type choice (the CLIP
|
||||
recipe is locked to CLIPType.MINIMAX); the model output has the attention
|
||||
backend and LoRA applied.
|
||||
backend applied, and any leftover file in models/loras is ignored (the
|
||||
distill weights ship baked into the diffusion model).
|
||||
- MoxieProject: takes model/clip/video_vae/audio_vae as required direct
|
||||
inputs instead of a bundled model_loader, and packs them into the loader
|
||||
dict consumed internally by EasyMultiTrackProject.
|
||||
@ -122,8 +123,6 @@ class _StubState:
|
||||
self.calls = []
|
||||
self.attention_fn = lambda pixels, *a, **k: pixels
|
||||
self.clip = object()
|
||||
self.lora_model = None
|
||||
self.lora_clip = None
|
||||
|
||||
|
||||
def _register(state, io, monkeypatch=None):
|
||||
@ -158,10 +157,7 @@ def _register(state, io, monkeypatch=None):
|
||||
return state.clip
|
||||
|
||||
def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None):
|
||||
state.calls.append(("load_lora_for_models", strength_model, strength_clip))
|
||||
state.lora_model = model
|
||||
state.lora_clip = clip
|
||||
return model, clip
|
||||
raise AssertionError("MoxieMultimediaLoader must not load LoRAs anymore")
|
||||
|
||||
comfy_sd.CLIPType = CLIPType
|
||||
comfy_sd.load_diffusion_model = load_diffusion_model
|
||||
@ -303,6 +299,7 @@ def _build_moxie():
|
||||
|
||||
tiny_vae = types.ModuleType("MMnode.tiny_vae")
|
||||
tiny_vae.load_tiny_vae_decoder = lambda *a, **k: None
|
||||
tiny_vae.pack_vae_approx_files = lambda *a, **k: []
|
||||
added["MMnode.tiny_vae"] = sys.modules.get("MMnode.tiny_vae")
|
||||
sys.modules["MMnode.tiny_vae"] = tiny_vae
|
||||
|
||||
@ -360,7 +357,7 @@ def test_loader_schema_shows_a_single_model(moxie):
|
||||
inputs = _input_names(schema)
|
||||
|
||||
# End users see one model picker plus the attention backend; every
|
||||
# auxiliary model (LoRA, video/audio VAE, CLIP) is auto-selected.
|
||||
# auxiliary model (video/audio VAE, CLIP) is auto-selected.
|
||||
assert inputs == ["diffusion_model", "weight_dtype", "attention_backend"]
|
||||
for hidden in (
|
||||
"lora_name",
|
||||
@ -374,7 +371,7 @@ def test_loader_schema_shows_a_single_model(moxie):
|
||||
assert hidden not in inputs
|
||||
|
||||
|
||||
def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path):
|
||||
def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch, tmp_path):
|
||||
state = moxie.state
|
||||
models = tmp_path / "models"
|
||||
for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"):
|
||||
@ -389,9 +386,6 @@ def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path
|
||||
}[name]
|
||||
(models / subdir / name).write_bytes(b"stub")
|
||||
monkeypatch.setattr(moxie.loader, "MODELS_DIR", models)
|
||||
# LoRA cache lives at module level: node classes are locked by newer
|
||||
# ComfyUI builds and class attributes can no longer be written at runtime.
|
||||
monkeypatch.setattr(moxie.loader, "_loaded_lora", None)
|
||||
state.calls.clear()
|
||||
|
||||
result = moxie.loader.MoxieMultimediaLoader.execute(
|
||||
@ -400,14 +394,15 @@ def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path
|
||||
attention_backend="comfy_kitchen_attention",
|
||||
)
|
||||
|
||||
# The distill weights ship baked into the diffusion model: a leftover
|
||||
# file in models/loras must be ignored entirely. The comfy.sd stub keeps
|
||||
# a raising load_lora_for_models, so an accidental LoRA call fails loudly.
|
||||
sequence = [call[0] for call in state.calls]
|
||||
assert sequence == [
|
||||
"load_diffusion_model",
|
||||
"get_attention_function",
|
||||
"load_clip",
|
||||
"load_torch_file",
|
||||
"load_lora_for_models",
|
||||
"load_torch_file",
|
||||
"load_torch_file",
|
||||
]
|
||||
|
||||
@ -415,50 +410,22 @@ def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path
|
||||
clip_calls = [call for call in state.calls if call[0] == "load_clip"]
|
||||
assert clip_calls == [("load_clip", state.CLIPType.MINIMAX)]
|
||||
|
||||
# LoRA applied on top of the attention-patched model, after the CLIP load.
|
||||
# Attention backend applied before the CLIP load.
|
||||
assert state.calls.index(("get_attention_function", "comfy_kitchen_int8")) < state.calls.index(
|
||||
("load_clip", state.CLIPType.MINIMAX)
|
||||
) < state.calls.index(("load_lora_for_models", 1.0, 1.0))
|
||||
)
|
||||
|
||||
# Exactly four direct outputs, model carrying attention + LoRA.
|
||||
# Exactly four direct outputs, model carrying the attention backend.
|
||||
assert isinstance(result, _NodeOutput)
|
||||
assert len(result.values) == 4
|
||||
model_out, clip_out, video_vae_out, audio_vae_out = result.values
|
||||
assert model_out is state.lora_model
|
||||
assert model_out.label == "diffusion.clone"
|
||||
assert model_out.attention_fn is state.attention_fn
|
||||
assert clip_out is state.clip
|
||||
assert isinstance(video_vae_out, _FakeVAE)
|
||||
assert isinstance(audio_vae_out, _FakeVAE)
|
||||
|
||||
|
||||
def test_loader_execute_without_lora_skips_lora_loading(moxie, monkeypatch, tmp_path):
|
||||
state = moxie.state
|
||||
models = tmp_path / "models"
|
||||
for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"):
|
||||
(models / subdir).mkdir(parents=True, exist_ok=True)
|
||||
for name in ("h3.safetensors", "video.safetensors", "audio.safetensors", "qwen.safetensors"):
|
||||
subdir = {
|
||||
"h3.safetensors": "diffusion",
|
||||
"video.safetensors": "vae",
|
||||
"audio.safetensors": "audio_vae",
|
||||
"qwen.safetensors": "clip",
|
||||
}[name]
|
||||
(models / subdir / name).write_bytes(b"stub")
|
||||
monkeypatch.setattr(moxie.loader, "MODELS_DIR", models)
|
||||
state.calls.clear()
|
||||
|
||||
result = moxie.loader.MoxieMultimediaLoader.execute(
|
||||
diffusion_model="h3.safetensors",
|
||||
weight_dtype="default",
|
||||
attention_backend="pytorch_attention",
|
||||
)
|
||||
|
||||
assert "load_lora_for_models" not in [call[0] for call in state.calls]
|
||||
assert result.values[0] is not state.clip # model output remains a model
|
||||
assert result.values[0].attention_fn is state.attention_fn
|
||||
assert result.values[1] is state.clip
|
||||
|
||||
|
||||
def test_loader_requires_minimax_cliptype(moxie, monkeypatch, tmp_path):
|
||||
state = moxie.state
|
||||
models = tmp_path / "models"
|
||||
@ -495,9 +462,12 @@ def test_project_schema_takes_direct_model_inputs(moxie):
|
||||
"audio_vae",
|
||||
"project_name",
|
||||
"steps",
|
||||
"context_frames",
|
||||
"seed",
|
||||
]
|
||||
assert "model_loader" not in inputs
|
||||
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
|
||||
assert inputs["context_frames"].kwargs["default"] == "22"
|
||||
|
||||
# The four model components are required direct connections.
|
||||
for name in ("model", "clip", "video_vae", "audio_vae"):
|
||||
|
||||
138
update.bat
Normal file
138
update.bat
Normal file
@ -0,0 +1,138 @@
|
||||
@echo off
|
||||
setlocal
|
||||
title MMnode Updater
|
||||
|
||||
REM ============================================================
|
||||
REM MMnode (Moxie-Multimedia) one-click updater
|
||||
REM
|
||||
REM Put this file INSIDE the pack folder:
|
||||
REM ComfyUI\custom_nodes\<pack-folder>\
|
||||
REM (next to __init__.py) and double-click it.
|
||||
REM
|
||||
REM It syncs code from HuggingFace, removes stale files from
|
||||
REM old versions, downloads model weights via git-lfs, verifies
|
||||
REM the build, and reminds you to restart ComfyUI.
|
||||
REM ============================================================
|
||||
|
||||
cd /d "%~dp0"
|
||||
|
||||
echo.
|
||||
echo ==================================================
|
||||
echo MMnode updater - syncing from HuggingFace
|
||||
echo ==================================================
|
||||
echo.
|
||||
|
||||
if not exist "__init__.py" (
|
||||
echo [ERROR] This is not the MMnode pack folder.
|
||||
echo Move update.bat into ComfyUI\custom_nodes\^<pack-folder^>
|
||||
echo next to __init__.py, then run it again.
|
||||
goto end_fail
|
||||
)
|
||||
|
||||
where git >nul 2>&1
|
||||
if errorlevel 1 (
|
||||
echo [ERROR] Git is not installed or not in PATH.
|
||||
echo Get it from https://git-scm.com/download/win
|
||||
goto end_fail
|
||||
)
|
||||
|
||||
netstat -ano 2>nul | findstr /c:":8188" >nul 2>&1
|
||||
if not errorlevel 1 (
|
||||
echo [WARN] ComfyUI appears to be running on port 8188.
|
||||
echo It must be fully closed for the update to take effect.
|
||||
echo Continuing in 5 seconds - close it now...
|
||||
timeout /t 5 /nobreak >nul
|
||||
)
|
||||
|
||||
if not exist ".git" (
|
||||
echo [*] No git repo here yet - initializing...
|
||||
git init
|
||||
if errorlevel 1 goto gitfail
|
||||
)
|
||||
|
||||
git remote remove origin >nul 2>&1
|
||||
git remote add origin https://huggingface.co/Moxiegen/Moxie-Multimedia
|
||||
if errorlevel 1 goto gitfail
|
||||
|
||||
echo.
|
||||
echo [*] Fetching latest version from HuggingFace...
|
||||
git fetch origin --prune
|
||||
if errorlevel 1 (
|
||||
echo [ERROR] Could not reach HuggingFace - check your internet.
|
||||
goto end_fail
|
||||
)
|
||||
|
||||
echo.
|
||||
echo [*] Syncing files...
|
||||
git reset --hard origin/main
|
||||
if errorlevel 1 goto gitfail
|
||||
|
||||
echo.
|
||||
echo [*] Cleaning stale files from older versions...
|
||||
git clean -fd -e update.bat >nul 2>&1
|
||||
|
||||
echo.
|
||||
echo [*] Syncing model weights...
|
||||
git lfs version >nul 2>&1
|
||||
if errorlevel 1 goto nolfs
|
||||
git lfs install >nul 2>&1
|
||||
echo Downloading weights - first run can take several minutes...
|
||||
git lfs fetch origin main
|
||||
if errorlevel 1 echo [WARN] Weight download failed - check connection, re-run later.
|
||||
git lfs checkout
|
||||
goto after_lfs
|
||||
|
||||
:nolfs
|
||||
echo [WARN] git-lfs is NOT installed on this PC.
|
||||
echo Code was updated, but model weights may be broken placeholders.
|
||||
echo Install git-lfs from https://git-lfs.com
|
||||
echo or run: winget install GitHub.git-lfs
|
||||
echo ...then run this updater again.
|
||||
|
||||
:after_lfs
|
||||
|
||||
REM ---- verify build ----
|
||||
set FIXCOUNT=0
|
||||
for /f %%A in ('findstr /c:"_align_canvas_dimension" "nodes\project.py" ^| find /c /v ""') do set FIXCOUNT=%%A
|
||||
|
||||
set POINTER=0
|
||||
for /r models %%F in (*.safetensors) do (
|
||||
if %%~zF LSS 1000 set /a POINTER+=1
|
||||
)
|
||||
|
||||
echo.
|
||||
echo ==================================================
|
||||
for /f "delims=" %%L in ('git log -1 --oneline') do echo Build: %%L
|
||||
echo Fix marker: %FIXCOUNT% found in nodes\project.py
|
||||
if %FIXCOUNT% GEQ 3 (
|
||||
echo [OK] This is the fixed build.
|
||||
) else (
|
||||
echo [ERROR] Fix marker missing - update did not complete!
|
||||
)
|
||||
if %POINTER% GTR 0 (
|
||||
echo [WARN] %POINTER% model file^(s^) are broken placeholders.
|
||||
echo Install git-lfs from https://git-lfs.com and re-run.
|
||||
) else (
|
||||
echo [OK] Model weights look good.
|
||||
)
|
||||
echo ==================================================
|
||||
|
||||
echo.
|
||||
echo All done. RESTART COMFYUI COMPLETELY before testing.
|
||||
echo Close this window and any ComfyUI console, then start ComfyUI.
|
||||
goto end_ok
|
||||
|
||||
:gitfail
|
||||
echo.
|
||||
echo [ERROR] A git command failed - scroll up for the message.
|
||||
goto end_fail
|
||||
|
||||
:end_fail
|
||||
echo.
|
||||
pause
|
||||
exit /b 1
|
||||
|
||||
:end_ok
|
||||
echo.
|
||||
pause
|
||||
exit /b 0
|
||||
@ -1327,15 +1327,18 @@ def compose_h3_project_video(project_name: Any, project_data: Any = None) -> Pat
|
||||
ffprobe_info,
|
||||
)
|
||||
|
||||
failure_reasons: list[str] = []
|
||||
temporary = merge_video_track_with_ffmpeg(
|
||||
timeline_segments,
|
||||
cursor,
|
||||
frame_rate,
|
||||
compose_size[0] if compose_size else width,
|
||||
compose_size[1] if compose_size else height,
|
||||
failure_reason=failure_reasons,
|
||||
)
|
||||
if temporary is None:
|
||||
raise RuntimeError("FFmpeg could not compose the H3 project video")
|
||||
detail = "; ".join(failure_reasons) if failure_reasons else "no reason was recorded"
|
||||
raise RuntimeError(f"FFmpeg could not compose the H3 project video: {detail}")
|
||||
return Path(temporary)
|
||||
|
||||
|
||||
|
||||
133
utils/video.py
133
utils/video.py
@ -7,6 +7,7 @@ import math
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import urllib.request
|
||||
from .media import AUDIO_EXTENSIONS
|
||||
@ -307,21 +308,72 @@ def merge_video_track_with_ffmpeg(
|
||||
width: int,
|
||||
height: int,
|
||||
resize_method: str | None = None,
|
||||
*,
|
||||
failure_reason: list[str] | None = None,
|
||||
) -> str | None:
|
||||
"""Compose file-backed segments on a black full-length timeline."""
|
||||
"""Compose file-backed segments on a black full-length timeline.
|
||||
|
||||
Returns ``None`` when composition is impossible so callers can fall back
|
||||
to another strategy. Callers that need to surface the real cause instead
|
||||
pass a ``failure_reason`` list, which receives one human-readable entry
|
||||
per failed precondition (missing FFmpeg, missing source files, ...) or
|
||||
the FFmpeg stderr tail when the command itself fails.
|
||||
"""
|
||||
|
||||
def record(reason: str) -> None:
|
||||
if failure_reason is not None:
|
||||
failure_reason.append(reason)
|
||||
|
||||
ffmpeg = get_ffmpeg_path("ffmpeg")
|
||||
resize_filter = _ffmpeg_resize_filter(width, height, resize_method) if resize_method else None
|
||||
if ffmpeg is None or total_length <= 0 or frame_rate <= 0 or width % 2 or height % 2:
|
||||
if ffmpeg is None:
|
||||
record(
|
||||
"FFmpeg executable not found (checked PATH, common install folders, "
|
||||
"package-manager shims and the imageio-ffmpeg bundled binary). "
|
||||
"Install FFmpeg with ffmpeg.exe on PATH - or simply copy ffmpeg.exe "
|
||||
"into C:\\ffmpeg\\bin - and restart ComfyUI from a NEW terminal so "
|
||||
"the updated PATH is picked up"
|
||||
)
|
||||
return None
|
||||
if total_length <= 0:
|
||||
record(f"timeline is empty (total_length={total_length} frames)")
|
||||
return None
|
||||
if frame_rate <= 0:
|
||||
record(f"invalid frame rate {frame_rate}")
|
||||
return None
|
||||
if width % 2 or height % 2:
|
||||
record(
|
||||
f"canvas size {width}x{height} has odd dimensions "
|
||||
"(video encoders require even width and height)"
|
||||
)
|
||||
return None
|
||||
if resize_method is not None and resize_filter is None:
|
||||
record(f"resize method '{resize_method}' is not supported by this FFmpeg build")
|
||||
return None
|
||||
if any(not os.path.isfile(str(segment.get("source", ""))) for segment in segments):
|
||||
return None
|
||||
if any(
|
||||
segment.get("audio_source") is not None
|
||||
and not os.path.isfile(str(segment["audio_source"]))
|
||||
missing_sources = [
|
||||
str(segment.get("source", ""))
|
||||
for segment in segments
|
||||
):
|
||||
if not os.path.isfile(str(segment.get("source", "")))
|
||||
]
|
||||
if missing_sources:
|
||||
record(
|
||||
"segment source file(s) missing: "
|
||||
+ " | ".join(missing_sources[:5])
|
||||
+ (f" (+{len(missing_sources) - 5} more)" if len(missing_sources) > 5 else "")
|
||||
)
|
||||
return None
|
||||
missing_audio = [
|
||||
str(segment["audio_source"])
|
||||
for segment in segments
|
||||
if segment.get("audio_source") is not None
|
||||
and not os.path.isfile(str(segment["audio_source"]))
|
||||
]
|
||||
if missing_audio:
|
||||
record(
|
||||
"locked audio file(s) missing: "
|
||||
+ " | ".join(missing_audio[:5])
|
||||
+ (f" (+{len(missing_audio) - 5} more)" if len(missing_audio) > 5 else "")
|
||||
)
|
||||
return None
|
||||
|
||||
total_seconds = total_length / frame_rate
|
||||
@ -495,6 +547,7 @@ def merge_video_track_with_ffmpeg(
|
||||
) from exc
|
||||
except OSError as exc:
|
||||
logger.warning("FFmpeg video track merge failed to start: %s", exc)
|
||||
record(f"FFmpeg failed to start ({exc})")
|
||||
try:
|
||||
os.unlink(output_path)
|
||||
except OSError:
|
||||
@ -503,9 +556,10 @@ def merge_video_track_with_ffmpeg(
|
||||
if result.returncode == 0:
|
||||
return output_path
|
||||
|
||||
logger.warning(
|
||||
"FFmpeg video track merge failed: %s",
|
||||
result.stderr.decode(errors="replace")[-600:],
|
||||
stderr_tail = result.stderr.decode(errors="replace")[-600:]
|
||||
logger.warning("FFmpeg video track merge failed: %s", stderr_tail)
|
||||
record(
|
||||
f"FFmpeg exited with code {result.returncode}: {stderr_tail.strip() or 'no stderr output'}"
|
||||
)
|
||||
try:
|
||||
os.unlink(output_path)
|
||||
@ -515,19 +569,58 @@ def merge_video_track_with_ffmpeg(
|
||||
|
||||
|
||||
def get_ffmpeg_path(name: str = "ffmpeg") -> str | None:
|
||||
"""Find FFmpeg/ffprobe executable, with Windows-specific fallbacks."""
|
||||
"""Find FFmpeg/ffprobe executable, with Windows-specific fallbacks.
|
||||
|
||||
PATH edits on Windows only reach processes started afterwards, and
|
||||
Portable users routinely have ffmpeg.exe sitting next to the run bat or
|
||||
installed via winget/scoop/choco without the ComfyUI process ever seeing
|
||||
it. After ``which`` fails, probe the locations such installs actually
|
||||
use, and finally fall back to the binary bundled with imageio-ffmpeg
|
||||
(a ComfyUI core dependency) so a zero-setup machine can still encode.
|
||||
"""
|
||||
# Try standard which first (works on all platforms)
|
||||
ffmpeg = shutil.which(name) or shutil.which(f"{name}.exe")
|
||||
if ffmpeg:
|
||||
return ffmpeg
|
||||
# Windows fallback: common installation paths
|
||||
if os.name == "nt":
|
||||
for path in (
|
||||
r"C:\ffmpeg\bin\{}.exe".format(name),
|
||||
os.path.expandvars(r"%LOCALAPPDATA%\Programs\{}\bin\{}.exe".format(name, name)),
|
||||
):
|
||||
if os.path.isfile(path):
|
||||
return path
|
||||
if os.name != "nt":
|
||||
return None
|
||||
exe = f"{name}.exe"
|
||||
candidates: list[str] = [
|
||||
r"C:\ffmpeg\bin\{}".format(name),
|
||||
os.path.expandvars(r"%LOCALAPPDATA%\Programs\{}\bin\{}".format(name, name)),
|
||||
# Package-manager shims/links.
|
||||
os.path.expandvars(r"%LOCALAPPDATA%\Microsoft\WinGet\Links\{}".format(exe)),
|
||||
os.path.expandvars(r"%USERPROFILE%\scoop\shims\{}".format(exe)),
|
||||
r"C:\ProgramData\chocolatey\bin\{}".format(exe),
|
||||
]
|
||||
# Relative to the ComfyUI install: the portable root (parent of the
|
||||
# ComfyUI folder) and the folder itself. ffmpeg.exe next to the run bat
|
||||
# or in an ffmpeg/bin subfolder is a very common portable layout.
|
||||
base = str(getattr(folder_paths, "base_path", "") or "")
|
||||
roots: list[str] = []
|
||||
if base:
|
||||
roots.append(base)
|
||||
roots.append(os.path.dirname(os.path.abspath(base)))
|
||||
roots.append(os.path.dirname(os.path.abspath(sys.executable)))
|
||||
roots.append(os.getcwd())
|
||||
for root in roots:
|
||||
candidates.extend([
|
||||
os.path.join(root, exe),
|
||||
os.path.join(root, "bin", exe),
|
||||
os.path.join(root, "ffmpeg", "bin", exe),
|
||||
])
|
||||
for candidate in candidates:
|
||||
if os.path.isfile(candidate):
|
||||
return candidate
|
||||
# Last resort: imageio-ffmpeg ships a full ffmpeg build (libx264
|
||||
# included). It bundles no ffprobe, so only use it for ffmpeg itself.
|
||||
if name == "ffmpeg":
|
||||
try:
|
||||
import imageio_ffmpeg
|
||||
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
except Exception:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
|
||||
@ -39,8 +39,12 @@ if (!packName) {
|
||||
} else {
|
||||
// Base prefix covers deployments behind a sub-path reverse proxy
|
||||
// ("/comfyui/extensions/<pack>/..." -> "/comfyui/"); the pack's own
|
||||
// static mount sits at "<base><pack>/".
|
||||
const base = '/' + segments.slice(0, extensionsIndex).join('/') + '/';
|
||||
// static mount sits at "<base><pack>/". For the standard deployment
|
||||
// there is no prefix and base is exactly "/" - it must NEVER be "//",
|
||||
// which the browser would parse as a protocol-relative URL and try to
|
||||
// resolve "<pack>" as a DNS hostname (net::ERR_NAME_NOT_RESOLVED).
|
||||
const prefix = segments.slice(0, extensionsIndex);
|
||||
const base = prefix.length ? '/' + prefix.join('/') + '/' : '/';
|
||||
// Top-level await: extension scripts are awaited before ComfyUI registers
|
||||
// node definitions, so the editor's registerExtension()/custom-widget
|
||||
// hooks stay in the normal loading order.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user