Compare commits

..

3 Commits

Author SHA1 Message Date
Z User
033a59ece9 chore: drop bench render artifacts from the repo, ignore bench/work 2026-09-28 01:43:15 +00:00
Z User
eefca88d35 feat: Moxie bench - standalone python/web harness for the fixed pipeline
- bench/ FastAPI server: fixed 9-stage pipeline parsed from the base
  workflow, serial render jobs, WS progress mirroring ComfyUI's push
  events, /api/view + ComfyUI-compatible /view, /upload/image,
  /easy-media/* aliases for the ported editor
- dual backend: self-contained mock renderer (numpy+ffmpeg, deterministic
  per prompt/seed, native + RTX-upscaled mp4 branches) and a ComfyUI
  bridge (substitutes the plan into base_workflow.json, converts to the
  /prompt API format, polls history, downloads outputs)
- timeline port: frontend/src/bench mounts the unmodified MultiTrack
  editor against a comfyAPI shim (fetchApi/toast/dialog/custom events);
  build.bench.ts emits bench/ui-dist served by the server
- packaging: PyInstaller onedir spec + Tauri2 sidecar docs
- tests: bench/tests contract suite + scripts/test_bench_pipeline.py
  e2e (API+WS+render); prebuilt bench UI committed
2026-09-28 01:42:58 +00:00
Z User
5d81069400 docs: note that Moxie Multimedia is a Moxigen-processed version of the MiniMax H3 model 2026-09-27 23:33:46 +00:00
44 changed files with 3817 additions and 1146 deletions

3
.gitignore vendored
View File

@ -26,3 +26,6 @@ models/audio_vae/audio_vae_fp32.safetensors
models/clip/MM-VL.safetensors models/clip/MM-VL.safetensors
models/diffusion/Moxie-Multimedia.safetensors models/diffusion/Moxie-Multimedia.safetensors
models/vae/video_vae_.safetensors models/vae/video_vae_.safetensors
# bench render outputs / job state
bench/work/

View File

@ -6,19 +6,9 @@
### ⚡ Improvements ### ⚡ Improvements
- **Updater**: The pack now ships a **`update.bat`** one-click updater for Windows. Place it in the pack folder (next to `__init__.py`) and double-click: it points the folder's git remote at the HuggingFace repository (fixing installs whose `origin` still points at a stale URL), fetches and force-syncs `main`, clears leftover files from older versions, and downloads the model weights through git-lfs — with a clear warning and install pointer when git-lfs is missing. It then self-verifies: it prints the current build id (`git log -1 --oneline`), counts the odd-latent fix marker in `nodes/project.py` (3 hits = fixed build), and flags any model file that is still a git-lfs placeholder. ComfyUI must still be restarted after updating. - **Moxie Bench (new)**: A standalone Python + web harness for the fixed Moxie pipeline (`bench/`) — the first step toward running the pack outside ComfyUI as a Tauri 2 sidecar. It locks to the base workflow's nine-stage graph (loader, LoRA, preview override, multitrack editor, project, combine, RTX upscale, create/save) and ships two interchangeable backends: a self-contained mock renderer (numpy + ffmpeg, deterministic per prompt/seed, real mp4 outputs on both the native and upscaled branches) and a ComfyUI bridge that drives a running ComfyUI over `/prompt` with the substituted workflow. The multitrack timeline editor UI is ported as-is (`frontend/src/bench/` shim mounts the same React components against a REST + WebSocket API), the server (FastAPI) exposes `/api/render`, `/api/jobs`, `WS /api/ws` progress plus ComfyUI-compatible `/view`, `/upload/image`, and `/easy-media/*` aliases, and `bench/moxie_bench.spec` + `bench/tauri/README.md` cover PyInstaller packaging and sidecar wiring. Run `python bench/run.py --port 8787`.
- **Moxie MultiTrack Project Slow**: Removed the experimental node again, together with its `upscale_model` / `tiling` widgets and the even-latent first-pass snap — the shared project-node internals are restored byte-for-byte to their certified behavior. Workflows that referenced the Slow node need rewiring to **Moxie MultiTrack Project**.
- **MultiTrack Project**: The inter-segment context overlap is now configurable. The new `context_frames` widget (5 / 22 / 39 / 56 frames, default 22) chooses how many frames of the previous segment are copied in as continuation context for segments using Context or Context Swap continuity — previously this was fixed at 22 frames in code. Longer values hand off motion more smoothly but generate that many extra frames per continued segment (each continuation also generates 12 grid-alignment tail frames that are trimmed after decode). Values are the H3 temporal-grid lengths already offered by the Moxie context sampler nodes; existing workflows keep 22 and behave exactly as before. The same widget (between `steps` and `seed`) is also exposed on **Moxie MultiTrack Project** — its baked euler/beta sampling stack is unchanged, so end users still only need `steps` and `seed` unless they touch the overlap.
- **Moxie Multimedia Loader**: Removed the automatic LoRA application entirely. The 3-step distill (speedup) weights are now baked into the diffusion model itself, so the loader no longer reads `models/loras` — a leftover file there is silently ignored, and the `models/loras` folder can be deleted. The node schema, outputs, and load order (diffusion model → attention backend → CLIP → VAEs) are unchanged; workflows saved earlier keep working. The `Moxie Project` steps tooltip now references the baked-in distill instead of the LoRA.
- **FFmpeg resolution & diagnostics**: The pack now finds FFmpeg on far more Windows setups and tells you exactly what is wrong when it cannot. When `ffmpeg` is not on the ComfyUI process's PATH (PATH edits only reach processes started afterwards), it is additionally looked up in the classic `C:\ffmpeg\bin`, the ComfyUI/portable folders (including `ffmpeg\bin` next to the run bat), winget/scoop/chocolatey shim locations, and finally the ffmpeg binary bundled with ComfyUI's own `imageio-ffmpeg` dependency. When composing an H3 project still fails, the error dialog now names the concrete cause instead of a generic message: the missing executable, the exact missing segment/locked-audio file paths, odd canvas sizes, or the tail of FFmpeg's own stderr output (e.g. `Unknown encoder 'libx264'`).
- **Installation**: The pack now ships a **`requirements.txt`**, so ComfyUI Manager installs the dependencies automatically and manual/portable installs have a one-line command (`python_embeded\python.exe -m pip install -r ...` for ComfyUI Portable — see the new "Installing dependencies" section in the README). Because this pack targets **NVIDIA RTX 3060+ GPUs only**, every feature dependency is mandatory: `soundfile`, `jieba`, `huggingface_hub`, `ffmpeg-python`, `opencv-python`, `imageio` (smart split), `nvidia-vfx` (RTX VSR upscale), `openai-whisper` and `qwen-asr` (subtitle recognition), and `voxcpm` (speech). No optional extras remain — previously these were installed on demand with error messages naming the missing package. The file also routes `nvidia-vfx` to NVIDIA's own index (`--extra-index-url https://pypi.nvidia.com/` + `--only-binary=nvidia-vfx`): the PyPI copy is a stub sdist whose `wheel_stub` build backend can abort the whole install (`Cannot import 'wheel_stub.buildapi'`), while NVIDIA's index serves real prebuilt wheels.
- **Installation**: Removed the bundled `download_models.sh` downloader. All model files live in the Hugging Face version of the repository — cloning it downloads the complete model set, so there is no separate downloader step (the README's Models section now explains this).
- **MultiTrack Editor**: The editor frontend was **reverted to the pristine upstream v1.3.3 source** — the MiniMax/Moxie format display-label mapping and every workaround built around it were removed. The shipped editor bundle is now built from that pristine source, so its behavior is exactly the battle-tested upstream editor (Shot/Context continuity, reference-size, audio lock, and MiniMax prompt placeholders all key on the native `MiniMax` format value again). - **MultiTrack Editor**: The editor frontend was **reverted to the pristine upstream v1.3.3 source** — the MiniMax/Moxie format display-label mapping and every workaround built around it were removed. The shipped editor bundle is now built from that pristine source, so its behavior is exactly the battle-tested upstream editor (Shot/Context continuity, reference-size, audio lock, and MiniMax prompt placeholders all key on the native `MiniMax` format value again).
- **Editor delivery fix**: A fresh browser could previously end up without the editor bundle entirely. ComfyUI core lets a pack's `WEB_DIRECTORY` overwrite its extension web-dir registration, so the browser's `/extensions` listing only ever contained this pack's helper scripts — never the editor entry — and the editor only appeared from stale browser cache. A small static loader (`web/js/editor-loader.js`, served with the helpers) now imports the editor entry from the pack's own route, which serves `index.js` with `no-cache` so the entry is always revalidated while its content-hashed chunks stay cacheable. After updating: restart ComfyUI and reload the page once; no cache-clearing gymnastics required. - **Editor delivery fix**: A fresh browser could previously end up without the editor bundle entirely. ComfyUI core lets a pack's `WEB_DIRECTORY` overwrite its extension web-dir registration, so the browser's `/extensions` listing only ever contained this pack's helper scripts — never the editor entry — and the editor only appeared from stale browser cache. A small static loader (`web/js/editor-loader.js`, served with the helpers) now imports the editor entry from the pack's own route, which serves `index.js` with `no-cache` so the entry is always revalidated while its content-hashed chunks stay cacheable. After updating: restart ComfyUI and reload the page once; no cache-clearing gymnastics required.
- **Editor loader hotfix**: The loader's base-prefix computation produced `//` on standard installs (no reverse-proxy sub-path), which browsers parse as a protocol-relative URL — they then tried to resolve the pack folder name as a DNS hostname and the editor entry never loaded (`net::ERR_NAME_NOT_RESOLVED` on `moxie-multimedia/index.js` in the browser console). The base is now exactly `/` when there is no sub-path prefix. If the MultiTrack Editor node shows a plain placeholder instead of the timeline UI, update the pack and hard-refresh the browser.
- **MultiTrack Editor / MultiTrack Project**: **T2V resolves natively again** (`t2v` stays `t2v`, matching upstream) — the model now handles text-to-video with the updated prompting, so the earlier T2V→R2V normalization was removed from the editor type computation, `h3_task_type`, `_multitrack_task_type`, and the generation-mode mapping. The reference-mode-with-zero-media cloud fallback was removed accordingly (reference mode raises for missing media again, as upstream). - **MultiTrack Editor / MultiTrack Project**: **T2V resolves natively again** (`t2v` stays `t2v`, matching upstream) — the model now handles text-to-video with the updated prompting, so the earlier T2V→R2V normalization was removed from the editor type computation, `h3_task_type`, `_multitrack_task_type`, and the generation-mode mapping. The reference-mode-with-zero-media cloud fallback was removed accordingly (reference mode raises for missing media again, as upstream).
- **Moxie Multimedia Loader**: The loader now **looks like it loads a single model**. Only the diffusion-model picker (plus the attention backend) is visible; the LoRA, video VAE, audio VAE, and CLIP widgets were removed and their files are auto-selected from the pack's `models/loras`, `models/vae`, `models/audio_vae`, and `models/clip` folders (first file in sorted order, matching the bundled model set). The LoRA is applied automatically with strength 1.0; remove the file from `models/loras` to disable it. Missing auxiliary models raise the same clear setup error as before. Workflows saved before this change keep working — extra widget values are ignored. - **Moxie Multimedia Loader**: The loader now **looks like it loads a single model**. Only the diffusion-model picker (plus the attention backend) is visible; the LoRA, video VAE, audio VAE, and CLIP widgets were removed and their files are auto-selected from the pack's `models/loras`, `models/vae`, `models/audio_vae`, and `models/clip` folders (first file in sorted order, matching the bundled model set). The LoRA is applied automatically with strength 1.0; remove the file from `models/loras` to disable it. Missing auxiliary models raise the same clear setup error as before. Workflows saved before this change keep working — extra widget values are ignored.
- **MultiTrack Project / Moxie Project Video Combine**: Single-pass projects now condition and sample at half the editor-selected resolution internally. The project timeline is composed at the segments' own size, and Moxie Project Video Combine restores the exact editor-selected resolution in pixel space before any output leaves the node (RTX Video Super Resolution via the optional `nvidia-vfx` package when available, with an automatic Lanczos fallback otherwise) — so every output carries the full editor resolution regardless of template wiring. The editor resolution therefore stays the final output resolution while the sampling chain runs smaller and faster. Dual-pass, SelfLift, and passthrough modes are unchanged. - **MultiTrack Project / Moxie Project Video Combine**: Single-pass projects now condition and sample at half the editor-selected resolution internally. The project timeline is composed at the segments' own size, and Moxie Project Video Combine restores the exact editor-selected resolution in pixel space before any output leaves the node (RTX Video Super Resolution via the optional `nvidia-vfx` package when available, with an automatic Lanczos fallback otherwise) — so every output carries the full editor resolution regardless of template wiring. The editor resolution therefore stays the final output resolution while the sampling chain runs smaller and faster. Dual-pass, SelfLift, and passthrough modes are unchanged.
@ -29,10 +19,6 @@
- **Moxie Multimedia Loader**: Locked the loader to the MiniMax H3 model set. The `clip_type` dropdown (which previously exposed every CLIP recipe in ComfyUI) was removed; the H3 Qwen3-VL text encoder is now always loaded with the `minimax` CLIP recipe. Any older ComfyUI build without `CLIPType.MINIMAX` raises a clear startup error instead of silently loading the wrong encoder. - **Moxie Multimedia Loader**: Locked the loader to the MiniMax H3 model set. The `clip_type` dropdown (which previously exposed every CLIP recipe in ComfyUI) was removed; the H3 Qwen3-VL text encoder is now always loaded with the `minimax` CLIP recipe. Any older ComfyUI build without `CLIPType.MINIMAX` raises a clear startup error instead of silently loading the wrong encoder.
- **Moxie Multimedia Loader / Moxie MultiTrack Project**: Removed the legacy `FAST_MODEL_LOADER` (`model_loader`) bundle. The loader now outputs exactly `model`, `clip`, `video_vae`, and `audio_vae` (with the attention backend and LoRA already applied to the model), and the Moxie MultiTrack Project takes those four outputs as required direct inputs instead of a bundled loader. `Moxie Preview Override` likewise dropped its `model_loader` pass-through slots and now patches the `model` input directly. The generic `easy multitrackProject` node keeps its `model_loader` input for non-Moxie workflows. - **Moxie Multimedia Loader / Moxie MultiTrack Project**: Removed the legacy `FAST_MODEL_LOADER` (`model_loader`) bundle. The loader now outputs exactly `model`, `clip`, `video_vae`, and `audio_vae` (with the attention backend and LoRA already applied to the model), and the Moxie MultiTrack Project takes those four outputs as required direct inputs instead of a bundled loader. `Moxie Preview Override` likewise dropped its `model_loader` pass-through slots and now patches the `model` input directly. The generic `easy multitrackProject` node keeps its `model_loader` input for non-Moxie workflows.
### 🐛 Bug Fixes
- **MultiTrack Project / MiniMax H3 nodes**: Fixed the first-step `patchify_video` crash (`RuntimeError: shape '[1, 24, 1, 1, ...]' is invalid for input of size N`) that hit 11 of the 17 editor resolution dropdown entries in single-pass mode — including the app's default 960x544 and the auto fallback 544x960. The hidden half-resolution first pass floored off the H3 32px canvas grid (544x960 halved to 272x480 encodes to a 17x30 latent whose odd side comfy core cannot split into 2x2 patches); the first pass now floors onto the grid (544x960 samples at 256x480, and Moxie Project Video Combine still restores the exact editor resolution, so output size and quality are unchanged). The MiniMax H3 conditioning entry points (`MiniMaxH3ImageToVideoFallback`, `MiniMaxH3ReferenceToVideoFallback`, `EasyMiniMaxH3ToVideo`) align their canvas the same way, which also fixes dual-pass and SelfLift at any editor size that is not a multiple of 32. The misleading "No H3 task segments are available from segment_start_number." error now distinguishes an empty timeline from an out-of-range `segment_start_number` and says what to do about each. **After updating, restart ComfyUI once** — a stale off-grid latent cached in the temp directory otherwise survives the pack update and the crash appears to persist.
## [1.3.3] - 2026-09-25 ## [1.3.3] - 2026-09-25
### ✨ New Features ### ✨ New Features

View File

@ -1,17 +1,3 @@
---
license: gpl-3.0
tags:
- comfyui
- text-to-video
- video-generation
- multi-track
- audio
- text-to-speech
- automatic-speech-recognition
- subtitles
- nvidia-rtx
---
<div align="center"> <div align="center">
<img src="https://github.com/user-attachments/assets/fb602a3c-4a2a-48da-8c44-d36417f4633b" height="120"> <img src="https://github.com/user-attachments/assets/fb602a3c-4a2a-48da-8c44-d36417f4633b" height="120">
@ -21,6 +7,9 @@ Timeline-driven video generation for ComfyUI — arrange tasks, references, audi
</div> </div>
> [!NOTE]
> **About the model** — the Moxie Multimedia model is a version of the MiniMax H3 model that has gone through the Moxigen method.
## 📦 Installation ## 📦 Installation
> [!IMPORTANT] > [!IMPORTANT]
@ -33,41 +22,24 @@ cd Your_ComfyUI_Path/custom_nodes
git clone https://huggingface.co/turtle89431/Moxie-Multimedia git clone https://huggingface.co/turtle89431/Moxie-Multimedia
``` ```
### Installing dependencies
**ComfyUI Manager** installs `requirements.txt` automatically. If you cloned the repository manually, install the dependencies yourself:
**ComfyUI Portable (Windows)** — run from the portable root folder (the one containing `run_nvidia_gpu.bat`):
```bat
cd Your_ComfyUI_windows_portable
.\python_embeded\python.exe -m pip install -r .\ComfyUI\custom_nodes\Moxie-Multimedia\requirements.txt
```
**Regular ComfyUI (venv / system Python):**
```bash
pip install -r Your_ComfyUI_Path/custom_nodes/Moxie-Multimedia/requirements.txt
```
Restart ComfyUI afterwards.
> [!NOTE]
> The pack targets **NVIDIA RTX 3060+ GPUs only**, so every feature dependency is mandatory and installed by the command above — including speech generation (`voxcpm`), subtitle recognition (`openai-whisper`, `qwen-asr`), and the RTX Video Super Resolution upscale (`nvidia-vfx`). Expect a sizable first install: `qwen-asr` pins `transformers==4.57.6` (pip may upgrade it inside your ComfyUI environment) and `voxcpm` needs `torch>=2.5.0`, so keep ComfyUI reasonably current. `nvidia-vfx` is pulled from NVIDIA's own package index (`pypi.nvidia.com`, wired up inside `requirements.txt`) because the PyPI copy is a stub that fails to build on some setups — if the command is interrupted, re-run it and it resumes where it stopped.
### Models ### Models
All model files ship **inside the repo** — cloning it into `custom_nodes` is all that's needed. The loader finds everything automatically in the pack's `models/` folders: All model files ship **inside the repo** — cloning it into `custom_nodes` is all that's needed. The loader finds everything automatically in the pack's `models/` folders:
| File | Location | | File | Location |
|---|---| |---|---|
| Diffusion model (`Moxie-Multimedia.safetensors`, distill weights baked in) | `models/diffusion/` | | Diffusion model (`Moxie-Multimedia.safetensors`) | `models/diffusion/` |
| Text encoder (`MM-VL.safetensors`) | `models/clip/` | | Text encoder (`MM-VL.safetensors`) | `models/clip/` |
| Video VAE (bundled file) | `models/vae/` | | Video VAE (bundled file) | `models/vae/` |
| Audio VAE (bundled file) | `models/audio_vae/` | | Audio VAE (bundled file) | `models/audio_vae/` |
| Turbo LoRA (`MM3step.safetensors`, applied by default) | `models/loras/` |
| Preview tiny VAE (`MM-preview.safetensors`) | `models/vae_approx/` | | Preview tiny VAE (`MM-preview.safetensors`) | `models/vae_approx/` |
All model files live in the Hugging Face version of this repository — cloning it downloads the complete set, and there is no separate downloader step. If you obtained the pack from somewhere else and the `models/` folders are missing or incomplete, re-clone from Hugging Face to get the bundled files. If you obtained the pack without the bundled files, the included downloader fetches and places them (idempotent and resumable — re-run if interrupted):
```bash
bash custom_nodes/MMnode/download_models.sh
```
Restart ComfyUI afterwards. Restart ComfyUI afterwards.

87
bench/README.md Normal file
View File

@ -0,0 +1,87 @@
# Moxie Multimedia Bench
A standalone Python + web harness for the Moxie Multimedia pipeline — the
first step toward running this node pack outside ComfyUI as a sidecar for a
Tauri 2 desktop app.
The bench locks to the fixed pipeline of the base workflow
(`bench/base_workflow.json`):
```
MoxieMultimediaLoader -> LoraLoaderModelOnly -> MoxiePreviewOverride
-> easy multiTrackEditor -> MoxieProject -> MoxieCombine
-> [native branch] CreateVideo -> SaveVideo
-> [upscaled branch] RTXVideoSuperResolution -> CreateVideo -> SaveVideo
```
The multitrack timeline editor is the real thing: the bench bundles the same
React editor components the ComfyUI node uses, mounted by a thin shim
(`frontend/src/bench/`) that talks REST + WebSocket instead of the ComfyUI
runtime.
## Run from source
```bash
pip install -r bench/requirements.txt # fastapi, uvicorn, numpy, pillow
python bench/run.py --port 8787 # auto backend (mock here)
# open http://127.0.0.1:8787
```
The UI needs a one-time build (a prebuilt copy ships in `bench/ui-dist/`):
```bash
cd frontend && bun install && bun run build:bench
```
## Backends
| backend | needs | behaviour |
|---|---|---|
| `mock` | ffmpeg, numpy, pillow | deterministic synthetic frames per (prompt, seed, frame), sine audio bed, lanczos "upscale" to the RTX target, h264 encode. Both output branches produce real mp4s. |
| `comfy` | a running ComfyUI + downloaded models | substitutes the plan into the base workflow, converts it to the `/prompt` API format, submits to ComfyUI, polls progress, downloads outputs. |
| `auto` | — | `comfy` when reachable, else `mock`. |
```bash
python bench/run.py --backend comfy --comfyui-url http://127.0.0.1:8188
```
## HTTP API
- `GET /api/health`, `GET /api/config` — liveness, backend availability
- `GET /api/workflow` — base workflow summary + editor `track_data`
- `GET /api/plan` — resolved render plan (segments, prompts, frames)
- `POST /api/render` — queue a render:
`{"track_data": <editor JSON>, "overrides": {"width": 1920, "height": 1080,
"steps": 4, "seed": 123, "project_name": "MM-x", "backend": "auto"}}`
- `GET /api/jobs`, `GET /api/jobs/{id}`, `DELETE /api/jobs/{id}`
- `WS /api/ws` — push events: `job_queued`, `execution_start`, `job_progress`,
`execution_success`, `execution_error` (mirrors ComfyUI's push model)
- `GET /api/view?filename=&type=output&job=` — fetch rendered files
- ComfyUI-compatible aliases: `/view`, `/upload/image`, `/easy-media/*`
Outputs land in `bench/work/outputs/<job_id>/` (`MOXIE_BENCH_WORK` overrides
the root). Every render produces both branches: `native` (editor canvas) and
`upscaled` (RTX target dimensions).
## Tauri 2 sidecar
See `bench/tauri/README.md` — PyInstaller onedir packaging
(`bench/moxie_bench.spec`), `tauri.conf.json` resource wiring, spawn + health
poll snippet, and the endpoint table.
## Tests
```bash
python3 -m pytest bench/tests -q # contract tests (no ComfyUI needed)
python3 scripts/test_bench_pipeline.py # full API + WS + mock render e2e
cd frontend && bun run test # editor suite incl. bench shim
```
## Phase map
- **phase 2 (this)** — bench: fixed pipeline, dual backend, timeline UI port,
sidecar packaging.
- **phase 3 (planned)** — in-process execution backend: load the pack's node
classes directly (torch + ComfyUI modules on `sys.path`) instead of driving
a separate ComfyUI server; `bench/app/backends/comfy.py` keeps the same
interface.

3
bench/app/__init__.py Normal file
View File

@ -0,0 +1,3 @@
"""Moxie Multimedia bench."""
__version__ = "0.1.0"

View File

@ -0,0 +1,19 @@
"""Backend registry."""
from __future__ import annotations
def comfy_bridge_available(cfg) -> bool:
from . import comfy
return comfy.comfy_bridge_available(cfg)
def available(cfg) -> dict:
"""Probe both backends for /api/config."""
info = {"mock": {"available": True}, "comfy": {"available": False}}
try:
ok = comfy_bridge_available(cfg)
info["comfy"] = {"available": ok, "url": cfg.comfyui_url}
except Exception as exc: # noqa: BLE001
info["comfy"] = {"available": False, "error": str(exc)}
return info

351
bench/app/backends/comfy.py Normal file
View File

@ -0,0 +1,351 @@
"""ComfyUI bridge backend (real execution, phase-2 scope).
Drives a running ComfyUI over HTTP instead of reimplementing its executor
in-process:
1. take the base workflow (frontend export format, checked in at
bench/base_workflow.json),
2. substitute the RenderPlan parameters (track_data, resolution, steps,
seed, LoRA, upscale target, codecs, project_data),
3. convert to the /prompt API format using the input names carried by the
export itself (widget names via widgets_values_named, link inputs via
the node's inputs[] entries),
4. POST /prompt, poll /history/<id> for per-node progress and outputs,
5. download produced files via /view into the job dir.
Phase 3 replaces this with an in-process executor (same interface).
"""
from __future__ import annotations
import copy
import json
import time
import urllib.error
import urllib.parse
import urllib.request
from pathlib import Path
from typing import Any
from ..tracks import RenderPlan
from ..workflow import EDITOR_TYPE, PipelineSpec
from .mock import ProgressCb, RenderResult
_CONTROL_WIDGET_NAMES = {"randomize", "control_after_generate", "control_before_generate"}
class ComfyBridgeError(RuntimeError):
pass
# ---------------------------------------------------------------------------
# HTTP helpers (stdlib only)
# ---------------------------------------------------------------------------
def _url(cfg_url: str, path: str, **params: Any) -> str:
base = cfg_url.rstrip("/")
url = f"{base}{path}"
if params:
url += "?" + urllib.parse.urlencode(params)
return url
def _get_json(url: str, timeout: float = 10.0) -> Any:
with urllib.request.urlopen(url, timeout=timeout) as resp:
return json.loads(resp.read().decode("utf-8"))
def _post_json(url: str, payload: dict, timeout: float = 30.0) -> Any:
data = json.dumps(payload).encode("utf-8")
req = urllib.request.Request(
url, data=data, headers={"Content-Type": "application/json"}, method="POST"
)
with urllib.request.urlopen(req, timeout=timeout) as resp:
return json.loads(resp.read().decode("utf-8"))
def comfy_bridge_available(cfg) -> bool:
"""True when a ComfyUI answers at cfg.comfyui_url."""
try:
_get_json(_url(cfg.comfyui_url, "/system_stats"), timeout=3.0)
return True
except Exception:
return False
# ---------------------------------------------------------------------------
# Workflow substitution + /prompt conversion
# ---------------------------------------------------------------------------
_RESOLUTION_PRESETS = {
(1920, 1080): "1920 x 1080 (16:9)",
(1088, 1920): "1088 x 1920 (9:16)",
(1080, 1920): "1080 x 1920 (9:16)",
(1280, 720): "1280 x 720 (16:9)",
(720, 1280): "720 x 1280 (9:16)",
(832, 480): "832 x 480 (16:9)",
(480, 832): "480 x 832 (9:16)",
(1024, 1024): "1024 x 1024 (1:1)",
}
def substitute(spec: PipelineSpec, plan: RenderPlan) -> dict[str, Any]:
"""Return a deep copy of the raw workflow with plan values applied."""
raw = copy.deepcopy(spec.raw)
nodes = {int(n["id"]): n for n in raw["nodes"]}
def named(nid: int) -> dict[str, Any]:
n = nodes[nid]
vals = n.setdefault("widgets_values_named", {})
vals.setdefault("widgets_values", None)
return vals
# editor: track_data + resolution
named(spec.editor_node_id)["track_data"] = json.dumps(
_track_data_for_submission(spec, plan)
)
preset = _RESOLUTION_PRESETS.get((plan.width, plan.height))
named(spec.editor_node_id)["resolution"] = (
preset if preset else {"width": plan.width, "height": plan.height}
)
# project: name, steps, seed
proj = spec.stages.get("MoxieProject")
if proj:
vals = named(proj.node_id)
vals["project_name"] = plan.project_name
vals["steps"] = plan.steps
vals["seed"] = plan.seed
# lora
lora = spec.stages.get("LoraLoaderModelOnly")
if lora and plan.lora:
vals = named(lora.node_id)
vals["lora_name"] = plan.lora
vals["strength_model"] = plan.lora_strength
# rtx upscale target
rtx = spec.stages.get("RTXVideoSuperResolution")
if rtx and plan.upscale_width:
vals = named(rtx.node_id)
vals["resize_type"] = "target dimensions"
vals["resize_type.width"] = plan.upscale_width
vals["resize_type.height"] = plan.upscale_height
vals["quality"] = plan.upscale_quality
# combine project_data carries the canvas + frame rate
combine = spec.stages.get("MoxieCombine")
if combine:
vals = named(combine.node_id)
pd_raw = vals.get("project_data", "{}")
pd = json.loads(pd_raw) if isinstance(pd_raw, str) else dict(pd_raw or {})
pd.update({
"project_name": plan.project_name,
"width": plan.width,
"height": plan.height,
"frame_rate": plan.frame_rate,
})
vals["project_data"] = json.dumps(pd)
# save prefixes get a per-run folder
for branch, stage in spec.save_nodes.items():
vals = named(stage.node_id)
if "filename_prefix" in vals:
vals["filename_prefix"] = f"moxie-bench/{branch}"
return raw
def _track_data_for_submission(spec: PipelineSpec, plan: RenderPlan) -> dict[str, Any]:
"""Track data as edited in the bench UI; the editor widget value wins."""
raw = spec.widget(EDITOR_TYPE, "track_data")
data = json.loads(raw) if isinstance(raw, str) else dict(raw or {})
data["frame_rate"] = plan.frame_rate
data.setdefault("total_length", plan.total_length)
return data
def to_prompt_format(raw: dict[str, Any]) -> dict[str, dict[str, Any]]:
"""Frontend-format workflow -> /prompt API format.
Widget values come from widgets_values_named (names are the schema input
names); link inputs come from the node's inputs[] entries. A widget that
is wired to a link (marker key "widget" plus a link id) takes the link.
"""
nodes = raw.get("nodes") or []
links: dict[int, tuple[int, int]] = {}
for lk in raw.get("links") or []:
links[int(lk[0])] = (int(lk[1]), int(lk[2]))
prompt: dict[str, dict[str, Any]] = {}
for n in nodes:
nid = str(int(n["id"]))
entry: dict[str, Any] = {}
for inp in n.get("inputs") or []:
if not isinstance(inp, dict):
continue
name = inp.get("name")
link = inp.get("link")
if name is None or link is None:
continue
frm = links.get(int(link))
if frm is None:
continue
entry[name] = [str(frm[0]), frm[1]]
named = n.get("widgets_values_named") or {}
linked_widget_names = {
inp.get("widget", {}).get("name")
for inp in n.get("inputs") or []
if isinstance(inp, dict) and inp.get("link") is not None
and isinstance(inp.get("widget"), dict)
}
for name, value in named.items():
if name in ("widgets_values",) or name in _CONTROL_WIDGET_NAMES:
continue
if name in linked_widget_names and name in entry:
continue # wired input wins over the stored widget value
entry[name] = value
prompt[nid] = {"class_type": n.get("type"), "inputs": entry}
return prompt
# ---------------------------------------------------------------------------
# Render
# ---------------------------------------------------------------------------
def render(
spec: PipelineSpec,
plan: RenderPlan,
cfg,
job_dir: Path,
progress: ProgressCb,
) -> RenderResult:
result = RenderResult()
raw = substitute(spec, plan)
prompt = to_prompt_format(raw)
node_types = {str(int(n["id"])): n.get("type", "") for n in raw["nodes"]}
progress("MoxieMultimediaLoader", 5.0, f"comfy: submitting prompt to {cfg.comfyui_url}")
try:
resp = _post_json(_url(cfg.comfyui_url, "/prompt"), {"prompt": prompt})
except urllib.error.HTTPError as exc:
body = exc.read().decode("utf-8", errors="replace")
raise ComfyBridgeError(f"ComfyUI rejected the prompt ({exc.code}): {body[:600]}") from exc
except urllib.error.URLError as exc:
raise ComfyBridgeError(
f"Cannot reach ComfyUI at {cfg.comfyui_url} ({exc.reason}). "
"Start it with: python main.py --listen 127.0.0.1 --port 8188"
) from exc
prompt_id = resp.get("prompt_id")
if not prompt_id:
raise ComfyBridgeError(f"ComfyUI returned no prompt_id: {resp}")
progress("MoxieProject", 10.0, f"comfy: queued ({prompt_id})")
stage_by_node = {str(nid): ntype for nid, ntype in node_types.items()}
save_branches = {str(st.node_id): branch for branch, st in spec.save_nodes.items()}
deadline = time.time() + 60 * 60 # 1h safety stop
poll = 1.0
seen_progress_max = 0
while time.time() < deadline:
time.sleep(poll)
try:
hist = _get_json(_url(cfg.comfyui_url, f"/history/{prompt_id}"), timeout=15.0)
except Exception as exc: # transient network hiccups
result.log(f"history poll failed: {exc}")
continue
entry = hist.get(prompt_id)
if entry is None:
# still queued or running
continue
status = entry.get("status") or {}
messages = status.get("messages") or []
for msg in messages:
mtype, mdata = msg[0], (msg[1] if len(msg) > 1 else {})
if mtype == "executing" and mdata.get("node"):
ntype = stage_by_node.get(str(mdata["node"]), "")
if ntype:
progress(ntype, max(12.0, result_progress(ntype)), f"comfy: {ntype}")
elif mtype == "progress":
try:
val = float(mdata.get("value", 0)) / max(1.0, float(mdata.get("max", 1)))
except (TypeError, ValueError):
val = 0.0
seen_progress_max = max(seen_progress_max, val)
# sample/VAE progress mapped into the 15..92 window
progress("MoxieProject", 15.0 + 70.0 * val,
f"comfy: sampling {int(val * 100)}%",
{"step": mdata.get("value"), "steps": mdata.get("max")})
elif mtype == "execution_error":
raise ComfyBridgeError(
f"ComfyUI execution failed on node "
f"{mdata.get('node_type')} ({mdata.get('node_id')}): "
f"{mdata.get('exception_message')}"
)
if status.get("completed"):
break
if status.get("status_str") == "error":
raise ComfyBridgeError("ComfyUI reported an error for this prompt")
if not status.get("completed"):
raise ComfyBridgeError("ComfyUI did not finish in time")
# collect outputs
outputs = entry.get("outputs") or {}
files: dict[str, dict[str, Any]] = {}
for nid, node_out in outputs.items():
branch = save_branches.get(str(nid))
for key, items in node_out.items():
if not isinstance(items, list):
continue
for item in items:
if not isinstance(item, dict) or "filename" not in item:
continue
if branch:
files.setdefault(branch, dict(item))
else:
result.log(f"aux output from node {nid}: {key} {item.get('filename')}")
if not files:
raise ComfyBridgeError(f"ComfyUI produced no video outputs: outputs={list(outputs)}")
for branch, item in files.items():
params = {
"filename": item["filename"],
"subfolder": item.get("subfolder", ""),
"type": item.get("type", "output"),
}
data = _download(_url(cfg.comfyui_url, "/view", **params))
dst = job_dir / f"{branch}{Path(item['filename']).suffix or '.mp4'}"
dst.write_bytes(data)
result.outputs[branch] = {
"filename": dst.name,
"path": str(dst),
"url": f"/api/view?filename={dst.name}&type=output&job={job_dir.name}",
"source": item.get("subfolder", ""),
"bytes": len(data),
}
result.log(f"{branch}: {item['filename']} ({len(data)} bytes)")
progress("SaveVideo", 100.0, "done", {"outputs": result.outputs})
return result
def result_progress(stage: str) -> float:
order = {
"MoxieMultimediaLoader": 8.0,
"LoraLoaderModelOnly": 10.0,
"MoxiePreviewOverride": 12.0,
EDITOR_TYPE: 14.0,
"MoxieProject": 40.0,
"MoxieCombine": 78.0,
"RTXVideoSuperResolution": 85.0,
"CreateVideo": 92.0,
"SaveVideo": 97.0,
}
return order.get(stage, 20.0)
def _download(url: str, timeout: float = 120.0) -> bytes:
with urllib.request.urlopen(url, timeout=timeout) as resp:
return resp.read()

324
bench/app/backends/mock.py Normal file
View File

@ -0,0 +1,324 @@
"""Self-contained mock backend.
Renders the fixed pipeline without torch/ComfyUI: deterministic synthetic
frames derived from (prompt, seed, frame index), a sine audio bed, PIL
lanczos "upscale" for the RTX branch, and ffmpeg h264 encode. Exercises the
exact same stage/progress contract as the ComfyUI bridge backend so the UI
and the API cannot tell the difference.
"""
from __future__ import annotations
import hashlib
import math
import shutil
import subprocess
import time
from dataclasses import dataclass, field
from pathlib import Path
from typing import Callable
from ..tracks import RenderPlan, Segment
ProgressCb = Callable[[str, float, str, dict | None], None]
_VIDEO_CODECS = ("h264", "libx264", "mpeg4")
@dataclass
class RenderResult:
outputs: dict[str, dict] = field(default_factory=dict)
console: list[str] = field(default_factory=list)
def log(self, line: str) -> None:
self.console.append(line)
def _pick_encoder(codec: str) -> str:
probe = subprocess.run(
["ffmpeg", "-hide_banner", "-encoders"],
capture_output=True, text=True, timeout=30,
)
names = probe.stdout.lower()
if codec == "h264" and "libx264" in names:
return "libx264"
for cand in _VIDEO_CODECS:
if cand in names:
return cand
return "mpeg4"
def _audio_encoder() -> str:
probe = subprocess.run(
["ffmpeg", "-hide_banner", "-encoders"],
capture_output=True, text=True, timeout=30,
)
return "aac" if " aac" in probe.stdout.lower() else "libmp3lame"
def _segment_hue(segment: Segment, index: int) -> float:
digest = hashlib.sha256(f"{segment.prompt}|{index}".encode()).digest()
return digest[0] * 360.0 / 255.0
def _segment_freq(segment: Segment, index: int) -> float:
digest = hashlib.sha256(f"{segment.prompt}|{index}".encode()).digest()
return 280.0 + digest[1] % 5 * 55.0 # 280..500 Hz
def _seed_noise(seed: int, frame: int) -> float:
digest = hashlib.sha256(f"{seed}:{frame}".encode()).digest()
return digest[0] / 255.0 - 0.5
def _synthesize_chunk(
chunk_start: int,
frames: int,
plan: RenderPlan,
w: int,
h: int,
) -> bytes:
"""Deterministic RGB24 bytes for frames [chunk_start, chunk_start+frames)."""
import numpy as np
out = bytearray()
yy, xx = np.mgrid[0:h, 0:w].astype(np.float32)
xn = xx / max(1, w - 1)
yn = yy / max(1, h - 1)
seg_by_frame = _frame_to_segment_map(plan)
for f in range(chunk_start, chunk_start + frames):
seg, seg_index = seg_by_frame.get(f, (None, 0))
hue = _segment_hue(seg, seg_index) if seg else 210.0
t = f / max(1, plan.frame_rate)
# HSV-ish deterministic palette: hue rotates per segment prompt,
# luminance carries a moving wave so motion is visible.
base = np.zeros((h, w, 3), dtype=np.float32)
h1 = hue % 360.0
rgb = _hsv_to_rgb(h1 / 360.0, 0.55, 0.6)
wave = 0.25 * np.sin(6.0 * math.pi * (xn + 0.35 * t)) + 0.15 * np.sin(
4.0 * math.pi * (yn - 0.25 * t)
)
lum = 0.45 + wave + 0.06 * _seed_noise(plan.seed, f)
base[..., 0] = rgb[0] * lum
base[..., 1] = rgb[1] * lum
base[..., 2] = rgb[2] * lum
# progress bar strip at the bottom: frame position inside the timeline
pos = f / max(1, plan.total_length or 1)
bar_y = int(h * 0.95)
base[bar_y:, : int(w * pos), 1] = 0.95
out += (
np.clip(base * 255.0, 0, 255).astype(np.uint8).tobytes()
)
return bytes(out)
def _frame_to_segment_map(plan: RenderPlan) -> dict[int, tuple[Segment, int]]:
mapping: dict[int, tuple[Segment, int]] = {}
for idx, seg in enumerate(plan.segments):
for f in range(seg.start_frame, seg.end_frame):
mapping[f] = (seg, idx)
return mapping
def _hsv_to_rgb(h: float, s: float, v: float) -> tuple[float, float, float]:
import colorsys
r, g, b = colorsys.hsv_to_rgb(h % 1.0, s, v)
return r, g, b
def _s16le_audio(plan: RenderPlan, sample_rate: int = 44100) -> bytes:
"""Stereo s16le sine bed; per-segment tone, silence for muted segments."""
import numpy as np
total = int(sample_rate * plan.duration_seconds)
if total <= 0:
return b""
wave = np.zeros(total, dtype=np.float32)
sr = sample_rate
for idx, seg in enumerate(plan.segments):
if seg.muted or not seg.prompt:
continue
a, b = seg.start_frame, seg.end_frame
i0 = int(a / max(1, plan.frame_rate) * sr)
i1 = min(total, int(b / max(1, plan.frame_rate) * sr))
if i1 <= i0:
continue
t = np.arange(i1 - i0, dtype=np.float32) / sr
freq = _segment_freq(seg, idx)
gain = 0.12 * (10 ** (seg.volume_db / 20.0))
wave[i0:i1] += gain * np.sin(2 * math.pi * freq * t)
stereo = np.repeat(np.clip(wave, -1, 1)[:, None], 2, axis=1)
return (stereo * 32767.0).astype("<i2").tobytes()
def render(plan: RenderPlan, work_dir: Path, progress: ProgressCb) -> RenderResult:
"""Run the mock pipeline. Returns native + upscaled output descriptors."""
import numpy as np
from PIL import Image
result = RenderResult()
t0 = time.time()
work_dir.mkdir(parents=True, exist_ok=True)
if shutil.which("ffmpeg") is None:
raise RuntimeError("ffmpeg not found on PATH - the mock backend needs it")
w, h = plan.width, plan.height
frames = plan.total_length or sum(s.frame_count for s in plan.segments)
if frames <= 0:
raise RuntimeError("timeline has no frames to render")
def stage(name: str, pct: float, msg: str, data: dict | None = None) -> None:
progress(name, pct, msg, data)
# stages 1-3: loader / lora / preview (contract parity with the real chain)
stage("MoxieMultimediaLoader", 4.0, "mock: loading Moxie-Multimedia model set")
stage("LoraLoaderModelOnly", 9.0, f"mock: LoRA {plan.lora or 'none'} @ {plan.lora_strength}")
stage("MoxiePreviewOverride", 12.0, "mock: preview override active")
stage("easy multiTrackEditor", 15.0,
f"mock: timeline {w}x{h} @ {plan.frame_rate}fps, "
f"{len(plan.segments)} task segment(s)")
encoder = _pick_encoder(plan.codec)
aenc = _audio_encoder()
result.log(f"encoder: {encoder}, audio: {aenc}")
native_path = work_dir / "native.mp4"
upscaled_path = work_dir / "upscaled.mp4"
do_upscale = bool(plan.upscale_width and plan.upscale_height)
audio_bytes = _s16le_audio(plan)
def open_enc(out_path: Path, ow: int, oh: int) -> subprocess.Popen:
import os as _os
cmd = [
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
"-f", "rawvideo", "-pix_fmt", "rgb24",
"-s", f"{ow}x{oh}", "-r", str(plan.frame_rate), "-i", "pipe:0",
"-f", "s16le", "-ar", "44100", "-ac", "2",
]
r, w = _os.pipe()
# give ffmpeg the READ end (r); parent writes audio into w
cmd += ["-i", f"/dev/fd/{r}", "-map", "0:v", "-map", "1:a",
"-c:v", encoder, "-pix_fmt", "yuv420p",
"-movflags", "+faststart",
"-c:a", aenc, "-b:a", "128k", "-shortest",
str(out_path)]
proc = subprocess.Popen(
cmd, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE, pass_fds=(r,),
)
proc._audio_wfd = w # type: ignore[attr-defined]
_os.close(r)
return proc
def feed_audio(proc: subprocess.Popen) -> None:
# Audio may exceed the 64KB pipe buffer; pump it from a thread so the
# video stdin feed never deadlocks against ffmpeg's startup probing.
import os as _os
import threading as _threading
def _pump() -> None:
try:
view = memoryview(audio_bytes)
while view:
n = _os.write(proc._audio_wfd, view[:65536]) # type: ignore[attr-defined]
view = view[n:]
except (BrokenPipeError, OSError):
pass
finally:
try:
_os.close(proc._audio_wfd) # type: ignore[attr-defined]
except OSError:
pass
_threading.Thread(target=_pump, daemon=True).start()
seg_total = sum(s.frame_count for s in plan.segments)
seg_done = 0
chunk = 24
native_enc = open_enc(native_path, w, h)
upscale_enc = open_enc(upscaled_path, plan.upscale_width, plan.upscale_height) if do_upscale else None
def _enc_err(proc: subprocess.Popen) -> str:
try:
return proc.stderr.read().decode(errors="replace")[-500:]
except Exception: # noqa: BLE001
return "(stderr unavailable)"
try:
feed_audio(native_enc)
if upscale_enc:
feed_audio(upscale_enc)
f = 0
while f < frames:
n = min(chunk, frames - f)
raw = _synthesize_chunk(f, n, plan, w, h)
try:
native_enc.stdin.write(raw)
except BrokenPipeError as exc:
raise RuntimeError(f"native encoder died: {_enc_err(native_enc)}") from exc
if upscale_enc:
# RTX-branch stand-in: lanczos resize per frame to target dims
arr = np.frombuffer(raw, dtype=np.uint8).reshape(n, h, w, 3)
imgs = [Image.fromarray(fr).resize(
(plan.upscale_width, plan.upscale_height), Image.LANCZOS)
for fr in arr]
try:
upscale_enc.stdin.write(np.stack([np.asarray(im) for im in imgs]).tobytes())
except BrokenPipeError as exc:
raise RuntimeError(f"upscale encoder died: {_enc_err(upscale_enc)}") from exc
f += n
seg_done += n
pct = 15.0 + 60.0 * seg_done / seg_total
# per-stage sub-progress: emulate per-segment sampling steps
cur = next((s for s in plan.segments if s.start_frame <= f < s.end_frame), None)
stage("MoxieProject", pct,
f"mock: sampling frame {f}/{frames}"
+ (f" ({cur.prompt or 'untitled'})" if cur else ""),
{"frame": f, "frames": frames})
native_enc.stdin.close()
if upscale_enc:
upscale_enc.stdin.close()
rc = native_enc.wait(timeout=120)
if rc != 0:
raise RuntimeError(f"native encode failed: {native_enc.stderr.read().decode(errors='replace')[-400:]}")
if upscale_enc:
rc = upscale_enc.wait(timeout=180)
if rc != 0:
raise RuntimeError(f"upscale encode failed: {upscale_enc.stderr.read().decode(errors='replace')[-400:]}")
finally:
for proc in (native_enc, upscale_enc):
if proc and proc.poll() is None:
proc.kill()
stage("MoxieCombine", 78.0, "mock: combined images + audio + fps")
if do_upscale:
stage("RTXVideoSuperResolution", 88.0,
f"mock: upscale {w}x{h} -> {plan.upscale_width}x{plan.upscale_height} "
f"({plan.upscale_quality})")
stage("CreateVideo", 94.0, f"mock: encode {plan.codec}")
stage("SaveVideo", 99.0, "mock: saved outputs")
def describe(path: Path, ow: int, oh: int) -> dict:
return {
"filename": path.name,
"path": str(path),
"url": f"/api/view?filename={path.name}&type=output",
"width": ow, "height": oh,
"fps": plan.frame_rate,
"frames": frames,
"bytes": path.stat().st_size if path.exists() else 0,
}
result.outputs["native"] = describe(native_path, w, h)
if do_upscale:
result.outputs["upscaled"] = describe(
upscaled_path, plan.upscale_width, plan.upscale_height)
result.log(f"rendered {frames} frames in {time.time() - t0:.1f}s")
stage("SaveVideo", 100.0, "done", {"outputs": result.outputs})
return result

113
bench/app/config.py Normal file
View File

@ -0,0 +1,113 @@
"""Bench configuration: paths and runtime options.
Everything is derived from --flags / environment so the same code runs from
source (repo checkout) and from a PyInstaller bundle (Tauri sidecar).
"""
from __future__ import annotations
import argparse
import os
import sys
from dataclasses import dataclass, field
from pathlib import Path
def _bundle_root() -> Path:
"""Repo root / frozen asset root that contains bench/ assets."""
if getattr(sys, "frozen", False): # PyInstaller (onedir: assets in _internal)
return Path(getattr(sys, "_MEIPASS", Path(sys.executable).resolve().parent))
return Path(__file__).resolve().parent.parent.parent
@dataclass
class BenchConfig:
host: str = "127.0.0.1"
port: int = 8787
backend: str = "auto" # auto | mock | comfy
repo_root: Path = field(default_factory=_bundle_root)
@property
def bench_dir(self) -> Path:
return self.repo_root / "bench"
@property
def base_workflow_path(self) -> Path:
return self.bench_dir / "base_workflow.json"
@property
def ui_dist(self) -> Path:
return self.bench_dir / "ui-dist"
@property
def work_dir(self) -> Path:
default = (
# frozen sidecar: keep renders next to the binary, not in _internal
Path(sys.executable).resolve().parent / "work"
if getattr(sys, "frozen", False)
else self.repo_root / "bench" / "work"
)
d = Path(os.environ.get("MOXIE_BENCH_WORK", str(default)))
d.mkdir(parents=True, exist_ok=True)
return d
@property
def outputs_dir(self) -> Path:
d = self.work_dir / "outputs"
d.mkdir(parents=True, exist_ok=True)
return d
@property
def inputs_dir(self) -> Path:
d = self.work_dir / "inputs"
d.mkdir(parents=True, exist_ok=True)
return d
@property
def temp_dir(self) -> Path:
d = self.work_dir / "temp"
d.mkdir(parents=True, exist_ok=True)
return d
@property
def pack_models_dir(self) -> Path:
return self.repo_root / "models"
# --- Comfy bridge backend ---
comfyui_url: str = os.environ.get("MOXIE_BENCH_COMFYUI_URL", "http://127.0.0.1:8188")
def resolve_backend(self) -> str:
"""auto: comfy when torch + ComfyUI reachable, else mock."""
if self.backend != "auto":
return self.backend
from .backends import comfy_bridge_available
return "comfy" if comfy_bridge_available(self) else "mock"
@classmethod
def from_args(cls, argv: list[str] | None = None) -> "BenchConfig":
p = argparse.ArgumentParser(prog="moxie-bench")
p.add_argument("--host", default=cls.host)
p.add_argument("--port", type=int, default=cls.port)
p.add_argument(
"--backend",
choices=["auto", "mock", "comfy"],
default=cls.backend,
help="mock = self-contained synthetic renderer; comfy = drive a "
"running ComfyUI at --comfyui-url with the base workflow",
)
p.add_argument("--comfyui-url", default=cls.comfyui_url)
p.add_argument(
"--repo-root",
default=None,
help="Root containing bench/ and models/ (default: auto-detect)",
)
args = p.parse_args(argv)
cfg = cls(
host=args.host,
port=args.port,
backend=args.backend,
comfyui_url=args.comfyui_url,
)
if args.repo_root:
cfg.repo_root = Path(args.repo_root).resolve()
return cfg

118
bench/app/jobs.py Normal file
View File

@ -0,0 +1,118 @@
"""Render job queue: one worker thread, progress events, WS broadcast."""
from __future__ import annotations
import threading
import time
import uuid
from concurrent.futures import ThreadPoolExecutor
from dataclasses import dataclass, field
from typing import Any, Callable
@dataclass
class Job:
id: str
created_at: float = field(default_factory=time.time)
status: str = "queued" # queued | running | done | error
backend: str = "mock"
stage: str = ""
progress: float = 0.0
message: str = "queued"
result: dict[str, Any] | None = None
error: str | None = None
events: list[dict[str, Any]] = field(default_factory=list)
plan: dict[str, Any] | None = None
def snapshot(self) -> dict[str, Any]:
return {
"id": self.id,
"status": self.status,
"backend": self.backend,
"stage": self.stage,
"progress": round(self.progress, 1),
"message": self.message,
"result": self.result,
"error": self.error,
"plan": self.plan,
}
BroadcastFn = Callable[[dict[str, Any]], None]
class JobManager:
"""Serial render queue. Events mirror ComfyUI's push model so the UI shim
can forward them onto app.api as execution events."""
def __init__(self, broadcast: BroadcastFn, max_workers: int = 1) -> None:
self._pool = ThreadPoolExecutor(max_workers=max_workers)
self._jobs: dict[str, Job] = {}
self._order: list[str] = []
self._lock = threading.Lock()
self._broadcast = broadcast
# -- public -------------------------------------------------------------
def create(self, backend: str, plan_summary: dict) -> Job:
job = Job(id=uuid.uuid4().hex[:12], backend=backend, plan=plan_summary)
with self._lock:
self._jobs[job.id] = job
self._order.append(job.id)
self._push(job, event="job_queued")
return job
def submit(self, job: Job, runner: Callable[[Job, BroadcastFn], dict]) -> None:
self._pool.submit(self._run, job, runner)
def get(self, job_id: str) -> Job | None:
return self._jobs.get(job_id)
def list(self, limit: int = 20) -> list[dict[str, Any]]:
with self._lock:
ids = list(reversed(self._order[-limit:]))
return [self._jobs[i].snapshot() for i in ids if i in self._jobs]
# -- internals ------------------------------------------------------------
def _run(self, job: Job, runner: Callable[[Job, BroadcastFn], dict]) -> None:
job.status = "running"
self._push(job, event="execution_start")
def progress(stage: str, pct: float, message: str, data: dict | None) -> None:
job.stage = stage
job.progress = max(0.0, min(100.0, pct))
job.message = message
evt = {
"stage": stage,
"progress": job.progress,
"message": message,
**({"data": data} if data else {}),
}
with self._lock:
job.events.append({"time": time.time(), **evt})
self._push(job, event="job_progress", **evt)
try:
result = runner(job, progress)
job.result = result
job.status = "done"
job.progress = 100.0
job.message = "done"
self._push(job, event="execution_success", outputs=result.get("outputs", {}))
except Exception as exc: # noqa: BLE001 - surface everything to the UI
job.status = "error"
job.error = f"{type(exc).__name__}: {exc}"
job.message = job.error
self._push(job, event="execution_error", error=job.error)
def _push(self, job: Job, event: str, **extra: Any) -> None:
try:
self._broadcast({
"type": event,
"job_id": job.id,
"status": job.status,
"stage": job.stage,
"progress": round(job.progress, 1),
"message": job.message,
**extra,
})
except Exception: # never let a dead WS client break a render
pass

50
bench/app/pipeline.py Normal file
View File

@ -0,0 +1,50 @@
"""Fixed-pipeline executor: binds a PipelineSpec + RenderPlan to a backend."""
from __future__ import annotations
import shutil
from pathlib import Path
from typing import Callable
from .config import BenchConfig
from .tracks import RenderPlan
from .workflow import PipelineSpec
ProgressCb = Callable[[str, float, str, dict | None], None]
def run_pipeline(
spec: PipelineSpec,
plan: RenderPlan,
backend: str,
cfg: BenchConfig,
progress: ProgressCb,
job_dir: Path,
) -> dict:
"""Run all stages; returns {outputs: {...}, console: [...]}."""
job_dir.mkdir(parents=True, exist_ok=True)
if backend == "mock":
from .backends import mock
result = mock.render(plan, job_dir, progress)
elif backend == "comfy":
from .backends import comfy
result = comfy.render(spec, plan, cfg, job_dir, progress)
else:
raise RuntimeError(f"unknown backend {backend!r}")
# Move finished files into the job's outputs folder served by /api/view.
final: dict[str, dict] = {}
for key, out in result.outputs.items():
src = Path(out["path"])
dst = job_dir / src.name
if src.resolve() != dst.resolve():
shutil.move(str(src), str(dst))
out2 = dict(out)
out2["path"] = str(dst)
out2["filename"] = dst.name
out2["url"] = f"/api/view?filename={dst.name}&type=output&job={job_dir.name}"
final[key] = out2
return {"outputs": final, "console": result.console}

421
bench/app/server.py Normal file
View File

@ -0,0 +1,421 @@
"""Moxie bench FastAPI server.
Serves the timeline UI (bench/ui-dist), the fixed-pipeline REST API and a
WebSocket that mirrors ComfyUI's push-event model (execution_start,
job_progress, execution_success/execution_error) so the ported editor can
subscribe with the same patterns it uses inside ComfyUI.
"""
from __future__ import annotations
import asyncio
import json
import shutil
import subprocess
from pathlib import Path
from fastapi import FastAPI, File, Form, HTTPException, UploadFile, WebSocket, WebSocketDisconnect
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, JSONResponse, Response
from fastapi.staticfiles import StaticFiles
from . import __version__
from .config import BenchConfig
from .jobs import JobManager
from .pipeline import run_pipeline
from .tracks import TrackDataError, build_render_plan
from .workflow import (
EDITOR_TYPE,
WorkflowError,
describe,
editor_resolution,
load_base_workflow,
parse_workflow_dict,
project_data as spec_project_data,
)
_MEDIA_KEYS = ("images", "videos", "gifs", "audio")
def create_app(cfg: BenchConfig | None = None) -> FastAPI:
cfg = cfg or BenchConfig.from_args()
app = FastAPI(title="Moxie Multimedia Bench", version=__version__)
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_methods=["*"],
allow_headers=["*"],
)
loop_holder: dict = {}
subscribers: set[WebSocket] = set()
@app.on_event("startup")
async def _capture_loop() -> None:
loop_holder["loop"] = asyncio.get_running_loop()
def broadcast(payload: dict) -> None:
loop = loop_holder.get("loop")
if loop is None:
return
def _publish() -> None:
dead = []
for ws in list(subscribers):
asyncio.ensure_future(_send(ws, payload, dead))
for ws in dead:
subscribers.discard(ws)
try:
loop.call_soon_threadsafe(_publish)
except RuntimeError:
pass
async def _send(ws: WebSocket, payload: dict, dead: list) -> None:
try:
await ws.send_json(payload)
except Exception: # noqa: BLE001
dead.append(ws)
jobs = JobManager(broadcast)
# -- spec helpers --------------------------------------------------------
def current_spec(raw_override: dict | None = None):
if raw_override is not None:
return parse_workflow_dict(raw_override)
return load_base_workflow(cfg.base_workflow_path)
def build_plan(spec, overrides: dict):
w, h = editor_resolution(spec)
pd = spec_project_data(spec)
ups = describe(spec)["upscale"] if "RTXVideoSuperResolution" in spec.stages else None
ups_target = (ups or {}).get("target") or {}
return build_render_plan(
spec.widget(EDITOR_TYPE, "track_data", "{}"),
width=int(overrides.get("width") or w),
height=int(overrides.get("height") or h),
frame_rate=overrides.get("frame_rate") or pd.get("frame_rate") or 24,
steps=int(overrides.get("steps") or spec.widget("MoxieProject", "steps", 4)),
seed=int(overrides.get("seed") or spec.widget("MoxieProject", "seed", 0) or 0),
project_name=overrides.get("project_name")
or spec.widget("MoxieProject", "project_name", "MoxieBench"),
lora=overrides.get("lora") or spec.widget("LoraLoaderModelOnly", "lora_name"),
lora_strength=float(
overrides.get("lora_strength")
or spec.widget("LoraLoaderModelOnly", "strength_model", 1.0)
),
upscale_width=ups_target.get("width"),
upscale_height=ups_target.get("height"),
upscale_quality=(ups or {}).get("quality", "ULTRA"),
codec=overrides.get("codec", "h264"),
)
# -- core API ------------------------------------------------------------
@app.get("/api/health")
def health():
return {"ok": True, "version": __version__, "backend": cfg.resolve_backend()}
@app.get("/api/config")
def config():
from .backends import available
return {
"version": __version__,
"backend": cfg.resolve_backend(),
"backends": available(cfg),
"work_dir": str(cfg.work_dir),
"outputs_dir": str(cfg.outputs_dir),
}
@app.get("/api/workflow")
def get_workflow():
spec = current_spec()
return {
"summary": describe(spec),
"track_data": spec.widget(EDITOR_TYPE, "track_data"),
"project_data": spec_project_data(spec),
}
@app.get("/api/plan")
def get_plan(overrides: str = "{}"):
spec = current_spec()
plan = build_plan(spec, json.loads(overrides or "{}"))
return plan.summary()
@app.post("/api/render")
async def render(body: dict):
try:
spec = current_spec(body.get("workflow"))
except WorkflowError as exc:
raise HTTPException(400, str(exc)) from exc
overrides = body.get("overrides") or {}
# UI-submitted track data replaces the template's copy
if body.get("track_data") is not None:
editor = spec.stages[EDITOR_TYPE]
editor.widget_values["track_data"] = (
body["track_data"]
if isinstance(body["track_data"], str)
else json.dumps(body["track_data"])
)
for key in ("resolution",):
if overrides.get(key):
editor = spec.stages[EDITOR_TYPE]
editor.widget_values[key] = overrides[key]
try:
plan = build_plan(spec, overrides)
except TrackDataError as exc:
raise HTTPException(400, str(exc)) from exc
backend = overrides.get("backend") or body.get("backend") or cfg.resolve_backend()
job = jobs.create(backend, plan.summary())
def runner(active: Job, progress) -> dict:
job_dir = cfg.outputs_dir / active.id
return run_pipeline(spec, plan, active.backend, cfg, progress, job_dir)
jobs.submit(job, runner)
return {"job_id": job.id, "plan": plan.summary()}
@app.get("/api/jobs")
def list_jobs():
return {"jobs": jobs.list()}
@app.get("/api/jobs/{job_id}")
def get_job(job_id: str):
job = jobs.get(job_id)
if not job:
raise HTTPException(404, "no such job")
snap = job.snapshot()
snap["events"] = job.events[-200:]
return snap
@app.delete("/api/jobs/{job_id}")
def delete_job(job_id: str):
job = jobs.get(job_id)
if not job:
raise HTTPException(404, "no such job")
job.status = "deleted"
return {"ok": True}
# -- media ----------------------------------------------------------------
def _safe(name: str) -> str:
name = Path(name).name
if not name or name.startswith("."):
raise HTTPException(400, "bad filename")
return name
@app.get("/api/view")
def view(filename: str, type: str = "output", job: str | None = None):
name = _safe(filename)
if type == "input":
path = cfg.inputs_dir / name
elif type == "temp":
path = cfg.temp_dir / name
else:
base = cfg.outputs_dir / _safe(job) if job else cfg.outputs_dir
path = base / name
if not path.is_file():
raise HTTPException(404, "no such file")
media = _guess_media_type(path)
return FileResponse(path, media_type=media)
@app.post("/api/upload/image")
async def upload_image(image: UploadFile = File(None), subfolder: str = Form("")):
return await _save_upload(image, subfolder)
@app.post("/api/easy-media/upload")
async def easy_upload(file: UploadFile = File(None), subfolder: str = Form("")):
return await _save_upload(file, subfolder)
async def _save_upload(upload: UploadFile | None, subfolder: str) -> dict:
if upload is None:
raise HTTPException(400, "missing file")
name = _safe(upload.filename or "upload.bin")
target_dir = cfg.inputs_dir
if subfolder:
target_dir = target_dir / Path(subfolder).name
target_dir.mkdir(parents=True, exist_ok=True)
target = target_dir / name
with open(target, "wb") as fh:
shutil.copyfileobj(upload.file, fh)
return {"name": name, "subfolder": str(target.parent.relative_to(cfg.inputs_dir)),
"type": "input"}
@app.post("/api/easy-media/download-url")
async def download_url(body: dict):
import urllib.request
url = body.get("url")
if not url:
raise HTTPException(400, "missing url")
name = _safe(body.get("name") or Path(url).name or "download.bin")
dst = cfg.inputs_dir / name
try:
with urllib.request.urlopen(url, timeout=60) as resp, open(dst, "wb") as fh:
shutil.copyfileobj(resp, fh)
except Exception as exc: # noqa: BLE001
raise HTTPException(502, f"download failed: {exc}") from exc
return {"name": name, "subfolder": "", "type": "input"}
@app.get("/api/easy-media/media/list")
def media_list(kind: str = "input"):
base = cfg.inputs_dir if kind == "input" else cfg.outputs_dir
items = []
for p in sorted(base.rglob("*")):
if p.is_file() and not p.name.startswith("."):
rel = str(p.relative_to(base).parent)
items.append({
"name": p.name,
"path": str(p.relative_to(base)),
"subfolder": "" if rel == "." else rel,
"type": "input" if kind == "input" else "output",
"size": p.stat().st_size,
})
return {"items": items}
@app.get("/api/easy-media/media/recent")
def media_recent():
return media_list("output")
@app.get("/api/easy-media/projects")
def easy_projects():
projects_dir = cfg.work_dir / "projects"
projects_dir.mkdir(parents=True, exist_ok=True)
return {"projects": sorted(
p.stem for p in projects_dir.glob("*.json") if p.is_file())}
@app.get("/api/easy-media/project")
def easy_project(project_name: str):
path = cfg.work_dir / "projects" / f"{Path(project_name).stem}.json"
if not path.is_file():
raise HTTPException(404, "no such project")
return JSONResponse(content=json.loads(path.read_text()))
@app.get("/api/easy-media/prompt/system-prompts")
def system_prompts():
return {"prompts": [
{"id": "default", "name": "Default", "prompt": ""},
]}
@app.post("/api/easy-media/video/source-fps")
def source_fps(body: dict):
name = _safe(body.get("filename") or "")
path = cfg.inputs_dir / name
if not path.is_file():
raise HTTPException(404, "no such file")
try:
out = subprocess.run(
["ffprobe", "-v", "quiet", "-print_format", "json",
"-show_streams", str(path)],
capture_output=True, text=True, timeout=30,
).stdout
streams = json.loads(out).get("streams") or []
rate = (streams[0].get("r_frame_rate") or "24/1") if streams else "24/1"
num, _, den = rate.partition("/")
fps = float(num) / float(den or 1)
except Exception as exc: # noqa: BLE001
raise HTTPException(500, f"ffprobe failed: {exc}") from exc
return {"fps": fps}
@app.post("/api/easy-media/video/preview-frame")
def preview_frame(body: dict):
name = _safe(body.get("filename") or "")
path = cfg.inputs_dir / name
if not path.is_file():
raise HTTPException(404, "no such file")
t = float(body.get("time") or 0)
try:
out = subprocess.run(
["ffmpeg", "-v", "quiet", "-ss", str(t), "-i", str(path),
"-frames:v", "1", "-f", "image2pipe", "-vcodec", "mjpeg", "pipe:1"],
capture_output=True, timeout=60,
)
except subprocess.TimeoutExpired as exc:
raise HTTPException(504, "ffmpeg timeout") from exc
if not out.stdout:
raise HTTPException(500, "ffmpeg produced no frame")
return Response(content=out.stdout, media_type="image/jpeg")
# -- ComfyUI-compatible root aliases --------------------------------------
# The ported editor (and its media elements) call the exact ComfyUI paths:
# /view, /upload/image, /easy-media/*. Register them alongside /api/* so
# the UI shim needs no URL rewriting at all.
app.add_api_route("/view", view, methods=["GET"], include_in_schema=False)
app.add_api_route("/upload/image", _save_upload, methods=["POST"], include_in_schema=False)
app.add_api_route("/easy-media/upload", _save_upload, methods=["POST"], include_in_schema=False)
app.add_api_route("/easy-media/media/list", media_list, methods=["GET"], include_in_schema=False)
app.add_api_route("/easy-media/media/recent", media_recent, methods=["GET"], include_in_schema=False)
app.add_api_route("/easy-media/projects", easy_projects, methods=["GET"], include_in_schema=False)
app.add_api_route("/easy-media/project", easy_project, methods=["GET"], include_in_schema=False)
app.add_api_route("/easy-media/prompt/system-prompts", system_prompts, methods=["GET"], include_in_schema=False)
app.add_api_route("/easy-media/video/source-fps", source_fps, methods=["POST"], include_in_schema=False)
app.add_api_route("/easy-media/video/preview-frame", preview_frame, methods=["POST"], include_in_schema=False)
app.add_api_route("/easy-media/download-url", download_url, methods=["POST"], include_in_schema=False)
# fallthrough for endpoints the editor may call in v1
@app.api_route("/api/easy-media/{rest:path}", methods=["GET", "POST", "PATCH", "DELETE"])
def easy_fallback(rest: str):
return JSONResponse(
status_code=501,
content={"error": f"bench does not implement /api/easy-media/{rest} yet"},
)
@app.api_route("/easy-media/{rest:path}", methods=["GET", "POST", "PATCH", "DELETE"])
def easy_fallback_root(rest: str):
return easy_fallback(rest)
# -- websocket -------------------------------------------------------------
@app.websocket("/api/ws")
async def ws_endpoint(ws: WebSocket):
await ws.accept()
subscribers.add(ws)
try:
while True:
await ws.receive_text() # keepalive pings from the UI
except WebSocketDisconnect:
pass
finally:
subscribers.discard(ws)
# -- static UI --------------------------------------------------------------
ui_dist = cfg.ui_dist
if (ui_dist / "index.html").is_file():
app.mount("/", StaticFiles(directory=str(ui_dist), html=True), name="ui")
else:
@app.get("/")
def no_ui():
return JSONResponse({
"error": "bench UI not built",
"hint": "cd frontend && bun run build:bench",
})
# silence 404 noise for browsers requesting favicons
@app.get("/favicon.ico")
def favicon():
return Response(status_code=204)
return app
def _guess_media_type(path: Path) -> str:
ext = path.suffix.lower()
return {
".mp4": "video/mp4", ".webm": "video/webm", ".mov": "video/quicktime",
".mkv": "video/x-matroska", ".png": "image/png", ".jpg": "image/jpeg",
".jpeg": "image/jpeg", ".webp": "image/webp", ".gif": "image/gif",
".wav": "audio/wav", ".mp3": "audio/mpeg", ".flac": "audio/flac",
".srt": "text/plain",
}.get(ext, "application/octet-stream")
def main() -> None: # pragma: no cover - manual entry
import uvicorn
cfg = BenchConfig.from_args()
uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port, log_level="info")

195
bench/app/tracks.py Normal file
View File

@ -0,0 +1,195 @@
"""track_data (the multitrack editor JSON) -> RenderPlan.
Pure extraction, no ComfyUI. Mirrors what ``MultiTrackEditor.execute`` feeds
into TRACKS_INFO for the fixed pipeline: task tracks carry the prompt segments
that become generation tasks; video/audio tracks carry media references.
"""
from __future__ import annotations
import json
from dataclasses import dataclass, field
from typing import Any
class TrackDataError(ValueError):
pass
@dataclass
class Segment:
id: str
start_frame: int
end_frame: int
prompt: str = ""
task_mode: str = "default"
continuity_mode: str = "shot"
ref_image_size: str = "match"
media_type: str = "none"
images: list[Any] = field(default_factory=list)
muted: bool = False
volume_db: float = 0.0
@property
def frame_count(self) -> int:
return max(0, self.end_frame - self.start_frame)
@dataclass
class Track:
id: str
name: str
type: str
task_mode: str = "default"
muted: bool = False
solo: bool = False
volume_db: float = 0.0
locked: bool = False
segments: list[Segment] = field(default_factory=list)
@dataclass
class RenderPlan:
width: int
height: int
frame_rate: int
total_length: int
segments: list[Segment] = field(default_factory=list) # ordered task segments
tracks: list[Track] = field(default_factory=list)
project_name: str = "MoxieBench"
steps: int = 4
seed: int = 0
lora: str | None = None
lora_strength: float = 1.0
upscale_width: int | None = None
upscale_height: int | None = None
upscale_quality: str = "ULTRA"
codec: str = "h264"
@property
def duration_seconds(self) -> float:
return self.total_length / float(self.frame_rate or 24)
def summary(self) -> dict[str, Any]:
return {
"width": self.width,
"height": self.height,
"frame_rate": self.frame_rate,
"total_length": self.total_length,
"duration_seconds": round(self.duration_seconds, 3),
"steps": self.steps,
"seed": self.seed,
"lora": self.lora,
"project_name": self.project_name,
"upscale": (
{"width": self.upscale_width, "height": self.upscale_height,
"quality": self.upscale_quality}
if self.upscale_width else None
),
"segments": [
{
"id": s.id,
"prompt": s.prompt,
"start_frame": s.start_frame,
"end_frame": s.end_frame,
"frame_count": s.frame_count,
"task_mode": s.task_mode,
"continuity_mode": s.continuity_mode,
}
for s in self.segments
],
}
def parse_track_data(raw: str | dict[str, Any]) -> dict[str, Any]:
if isinstance(raw, str):
try:
raw = json.loads(raw)
except json.JSONDecodeError as exc:
raise TrackDataError(f"track_data is not valid JSON: {exc}") from exc
if not isinstance(raw, dict):
raise TrackDataError("track_data must be an object")
if not isinstance(raw.get("tracks"), list):
raise TrackDataError("track_data.tracks must be a list")
return raw
def build_render_plan(
track_data: str | dict[str, Any],
*,
width: int,
height: int,
frame_rate: int | None = None,
steps: int = 4,
seed: int = 0,
project_name: str = "MoxieBench",
lora: str | None = None,
lora_strength: float = 1.0,
upscale_width: int | None = None,
upscale_height: int | None = None,
upscale_quality: str = "ULTRA",
codec: str = "h264",
) -> RenderPlan:
data = parse_track_data(track_data)
# Timeline fps: editor stores it on track_data.frame_rate (24 in the base
# workflow); the combine project_data may override it.
if frame_rate is None:
frame_rate = int(data.get("frame_rate") or 24)
total = int(data.get("total_length") or 0)
tracks: list[Track] = []
segments: list[Segment] = []
for t in data["tracks"]:
if not isinstance(t, dict):
continue
track = Track(
id=str(t.get("id", "")),
name=str(t.get("name", "")),
type=str(t.get("type", "task")),
task_mode=str(t.get("task_mode", "default")),
muted=bool(t.get("muted", False)),
solo=bool(t.get("solo", False)),
volume_db=float(t.get("volume_db", 0)),
locked=bool(t.get("locked", False)),
)
for seg in t.get("segments") or []:
if not isinstance(seg, dict):
continue
content = seg.get("content") or {}
s = Segment(
id=str(seg.get("id", "")),
start_frame=int(seg.get("start_frame", 0)),
end_frame=int(seg.get("end_frame", 0)),
prompt=str(content.get("user_prompt", "") or ""),
task_mode=str(content.get("task_mode", track.task_mode or "default")),
continuity_mode=str(content.get("continuity_mode", "shot")),
ref_image_size=str(content.get("ref_image_size", "match")),
media_type=str(content.get("media_type", "none")),
images=list(content.get("images") or []),
muted=bool(content.get("muted", False)),
volume_db=float(content.get("volume_db", 0)),
)
track.segments.append(s)
if track.type == "task" and s.frame_count > 0:
segments.append(s)
tracks.append(track)
segments.sort(key=lambda s: (s.start_frame, s.end_frame))
return RenderPlan(
width=int(width),
height=int(height),
frame_rate=int(frame_rate),
total_length=total,
segments=segments,
tracks=tracks,
project_name=str(project_name),
steps=int(steps),
seed=int(seed),
lora=lora,
lora_strength=float(lora_strength),
upscale_width=upscale_width,
upscale_height=upscale_height,
upscale_quality=str(upscale_quality),
codec=str(codec),
)

204
bench/app/workflow.py Normal file
View File

@ -0,0 +1,204 @@
"""Fixed-pipeline contract parsed from the base workflow JSON.
The bench locks to the Moxie pipeline used by the base workflow:
MoxieMultimediaLoader -> LoraLoaderModelOnly -> MoxiePreviewOverride
-> easy multiTrackEditor -> MoxieProject -> MoxieCombine
-> [branch A] CreateVideo(#45) -> SaveVideo(#46) (native)
-> [branch B] RTXVideoSuperResolution -> CreateVideo(#68)
-> SaveVideo(#69) (upscaled)
The parser walks the frontend-format workflow (nodes/links/widgets_values_named)
that ComfyUI exports and needs no ComfyUI installation: link inputs carry their
own names in the node's ``inputs`` array, widget values carry ``widgets_values_named``.
"""
from __future__ import annotations
import copy
import json
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
EDITOR_TYPE = "easy multiTrackEditor"
STAGE_ORDER = (
"MoxieMultimediaLoader",
"LoraLoaderModelOnly",
"MoxiePreviewOverride",
EDITOR_TYPE,
"MoxieProject",
"MoxieCombine",
"RTXVideoSuperResolution",
"CreateVideo",
) # SaveVideo x2 tracked via PipelineSpec.save_nodes (native/upscaled branches)
class WorkflowError(ValueError):
pass
@dataclass
class Stage:
node_id: int
type: str
widget_values: dict[str, Any] = field(default_factory=dict)
link_inputs: dict[str, tuple[int, int]] = field(default_factory=dict) # name -> (node_id, slot)
@dataclass
class PipelineSpec:
raw: dict[str, Any]
stages: dict[str, Stage] # type -> Stage (types unique in the fixed graph)
save_nodes: dict[str, Stage] = field(default_factory=dict) # native/upscaled -> SaveVideo
@property
def editor_node_id(self) -> int:
return self.stages[EDITOR_TYPE].node_id
def widget(self, stage_type: str, name: str, default: Any = None) -> Any:
st = self.stages.get(stage_type)
if not st:
return default
return st.widget_values.get(name, default)
def _parse_workflow(raw: dict[str, Any]) -> PipelineSpec:
nodes = raw.get("nodes")
if not isinstance(nodes, list) or not nodes:
raise WorkflowError("workflow has no nodes")
links: dict[int, tuple[int, int, int]] = {}
for lk in raw.get("links") or []:
# [link_id, from_node, from_slot, to_node, to_slot, type]
links[int(lk[0])] = (int(lk[1]), int(lk[2]), int(lk[4]))
stages: dict[str, Stage] = {}
save_nodes: dict[str, Stage] = {}
seen_create = 0
for n in nodes:
ntype = n.get("type")
nid = int(n.get("id"))
if ntype not in STAGE_ORDER and ntype != "SaveVideo":
continue
named = dict(n.get("widgets_values_named") or {})
# Legacy export without named values: best effort positional mapping.
if not named:
named = {}
stage = Stage(node_id=nid, type=ntype, widget_values=named)
for inp in n.get("inputs") or []:
if not isinstance(inp, dict):
continue
if "link" in inp and inp["link"] is not None and "widget" not in inp:
frm = links.get(int(inp["link"]))
if frm is None:
raise WorkflowError(f"node {nid}: dangling link {inp['link']}")
stage.link_inputs[inp["name"]] = (frm[0], frm[1])
if ntype == "CreateVideo":
seen_create += 1
key = "CreateVideo"
if seen_create == 2:
stages.pop("CreateVideo#1", None)
stages[f"CreateVideo#{seen_create}" if seen_create > 1 else key] = stage
continue
if ntype == "SaveVideo":
src = stage.link_inputs.get("video", (None, None))[0]
if src is not None and src in (
stages.get("CreateVideo#2").node_id if stages.get("CreateVideo#2") else -1,
):
save_nodes["upscaled"] = stage
else:
save_nodes["native"] = stage
continue
if ntype in stages:
raise WorkflowError(f"duplicate stage node type {ntype}")
stages[ntype] = stage
missing = [t for t in STAGE_ORDER if t not in stages]
if missing:
raise WorkflowError(f"missing pipeline stages: {', '.join(missing)}")
return PipelineSpec(raw=raw, stages=stages, save_nodes=save_nodes)
def load_base_workflow(path: Path) -> PipelineSpec:
with open(path, "r", encoding="utf-8") as fh:
raw = json.load(fh)
return _parse_workflow(raw)
def parse_workflow_dict(raw: dict[str, Any]) -> PipelineSpec:
return _parse_workflow(copy.deepcopy(raw))
def editor_track_data(spec: PipelineSpec) -> dict[str, Any]:
"""Parsed track_data JSON from the editor stage."""
raw = spec.widget(EDITOR_TYPE, "track_data")
if isinstance(raw, str):
raw = json.loads(raw)
if not isinstance(raw, dict):
raise WorkflowError("editor track_data is not an object")
return raw
def editor_resolution(spec: PipelineSpec) -> tuple[int, int]:
"""Width/height chosen on the editor, falling back to the project_data."""
res = spec.widget(EDITOR_TYPE, "resolution", "")
preset = _RESOLUTION_PRESETS.get(res)
if preset:
return preset
combo = spec.stages[EDITOR_TYPE].widget_values.get("resolution")
if isinstance(combo, dict): # custom: {"width": .., "height": ..}
try:
return int(combo.get("width")), int(combo.get("height"))
except (TypeError, ValueError):
pass
pd = _project_data(spec)
return int(pd.get("width", 0)), int(pd.get("height", 0))
def project_data(spec: PipelineSpec) -> dict[str, Any]:
return _project_data(spec)
def _project_data(spec: PipelineSpec) -> dict[str, Any]:
raw = spec.widget("MoxieCombine", "project_data", "") or "{}"
if isinstance(raw, str):
raw = json.loads(raw)
return raw if isinstance(raw, dict) else {}
_RESOLUTION_PRESETS = {
"1920 x 1080 (16:9)": (1920, 1080),
"1088 x 1920 (9:16)": (1088, 1920),
"1080 x 1920 (9:16)": (1080, 1920),
"1280 x 720 (16:9)": (1280, 720),
"720 x 1280 (9:16)": (720, 1280),
"832 x 480 (16:9)": (832, 480),
"480 x 832 (9:16)": (480, 832),
"1024 x 1024 (1:1)": (1024, 1024),
}
def describe(spec: PipelineSpec) -> dict[str, Any]:
"""Small human-readable summary used by /api/workflow."""
w, h = editor_resolution(spec)
pd = project_data(spec)
return {
"stages": [
{"type": t, "node_id": s.node_id, "widgets": s.widget_values}
for t, s in spec.stages.items()
],
"editor_resolution": {"width": w, "height": h},
"frame_rate": pd.get("frame_rate", 24),
"project_name": spec.widget("MoxieProject", "project_name", "MoxieBench"),
"steps": spec.widget("MoxieProject", "steps", 4),
"seed": spec.widget("MoxieProject", "seed", 0),
"lora": spec.widget("LoraLoaderModelOnly", "lora_name"),
"lora_strength": spec.widget("LoraLoaderModelOnly", "strength_model", 1.0),
"upscale": {
"enabled": "RTXVideoSuperResolution" in spec.stages,
"target": {
"width": spec.widget("RTXVideoSuperResolution", "resize_type.width"),
"height": spec.widget("RTXVideoSuperResolution", "resize_type.height"),
},
"quality": spec.widget("RTXVideoSuperResolution", "quality", "ULTRA"),
},
}

240
bench/base_workflow.json Normal file
View File

@ -0,0 +1,240 @@
{
"id": "74d5a00d-104c-439f-a307-874d829f8418",
"revision": 0,
"last_node_id": 81,
"last_link_id": 166,
"nodes": [
{
"id": 45,
"type": "CreateVideo",
"pos": [5303.990730085631, 2674.6532486606284],
"size": [270, 150],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{"name": "images", "type": "IMAGE", "link": 151},
{"name": "audio", "shape": 7, "type": "AUDIO", "link": 149},
{"name": "fps", "type": "FLOAT", "widget": {"name": "fps"}, "link": 150}
],
"outputs": [{"name": "VIDEO", "type": "VIDEO", "links": [103]}],
"properties": {"Node name for S&R": "CreateVideo"},
"widgets_values": [30, "auto", "sRGB", "h264"],
"widgets_values_named": {"fps": 30, "bit_depth": "auto", "color_space": "sRGB", "codec": "h264"}
},
{
"id": 46,
"type": "SaveVideo",
"pos": [6536.102438951979, 2629.839158378452],
"size": [1056.6122485107426, 1748.7755735581345],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [{"name": "video", "type": "VIDEO", "link": 103}],
"outputs": [{"name": "video", "type": "VIDEO", "links": null}],
"properties": {"Node name for S&R": "SaveVideo"},
"widgets_values": ["video/ComfyUI", "mp4", "h264", "auto", "auto"],
"widgets_values_named": {"filename_prefix": "video/ComfyUI", "format": "mp4", "format.codec": "h264", "format.codec.encoding": "auto", "codec": "auto"}
},
{
"id": 48,
"type": "MoxiePreviewOverride",
"pos": [3815.4265801150054, 2654.0916373620416],
"size": [360, 480],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [{"name": "model", "type": "MODEL", "link": 154}],
"outputs": [{"name": "model", "type": "MODEL", "links": [144]}],
"properties": {"Node name for S&R": "MoxiePreviewOverride"},
"widgets_values": [1024, 80, true, 48, 24, "MM-preview.safetensors", ""],
"widgets_values_named": {"max_resolution": 1024, "jpeg_quality": 80, "suppress_default_preview": true, "preview_frames": 48, "preview_fps": 24, "tiny_vae": "MM-preview.safetensors", "preview": ""}
},
{
"id": 52,
"type": "MoxieMultimediaLoader",
"pos": [3000.4622746801124, 2645.6062887703747],
"size": [370.66988849138306, 180.73178061607814],
"flags": {},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [
{"name": "model", "type": "MODEL", "links": [153]},
{"name": "clip", "type": "CLIP", "links": [136]},
{"name": "video_vae", "type": "VAE", "links": [137]},
{"name": "audio_vae", "type": "VAE", "links": [138]}
],
"properties": {"Node name for S&R": "MoxieMultimediaLoader"},
"widgets_values": ["Moxie-Multimedia.safetensors", "default", "comfy_kitchen_attention"],
"widgets_values_named": {"diffusion_model": "Moxie-Multimedia.safetensors", "weight_dtype": "default", "attention_backend": "comfy_kitchen_attention"}
},
{
"id": 57,
"type": "MoxieProject",
"pos": [4349.808755835772, 2799.703672942545],
"size": [277.359375, 230],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [
{"name": "tracks_info", "type": "TRACKS_INFO", "link": 166},
{"name": "model", "type": "MODEL", "link": 144},
{"name": "clip", "type": "CLIP", "link": 136},
{"name": "video_vae", "type": "VAE", "link": 137},
{"name": "audio_vae", "type": "VAE", "link": 138}
],
"outputs": [
{"name": "PROJECT_NAME", "type": "STRING", "links": [148]},
{"name": "LOCKED_AUDIO", "type": "AUDIO", "links": null}
],
"properties": {"Node name for S&R": "MoxieProject"},
"widgets_values": ["MM-girlwaving", 4, 803100406447983, "randomize"],
"widgets_values_named": {"project_name": "MM-girlwaving", "steps": 4, "seed": 803100406447983, "randomize": "randomize"}
},
{
"id": 63,
"type": "MoxieCombine",
"pos": [4734.228609743854, 2711.230251501094],
"size": [295.59921875, 638],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [{"name": "project_name", "type": "STRING", "link": 148}],
"outputs": [
{"name": "images", "type": "IMAGE", "links": [151, 155]},
{"name": "audio", "type": "AUDIO", "links": [149, 157]},
{"name": "fps", "type": "FLOAT", "links": [150, 159]}
],
"properties": {"Node name for S&R": "MoxieCombine"},
"widgets_values": ["{\"project_name\":\"MM-girlwaving\",\"width\":1920,\"height\":1088,\"frame_rate\":24,\"clips\":[],\"auto_combine\":true,\"updated_at\":0}"],
"widgets_values_named": {"project_data": "{\"project_name\":\"MM-girlwaving\",\"width\":1920,\"height\":1088,\"frame_rate\":24,\"clips\":[],\"auto_combine\":true,\"updated_at\":0}"}
},
{
"id": 66,
"type": "LoraLoaderModelOnly",
"pos": [3539.407729088216, 2558.7373665366495],
"size": [270, 82],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [{"name": "model", "type": "MODEL", "link": 153}],
"outputs": [{"name": "MODEL", "type": "MODEL", "links": [154]}],
"properties": {"Node name for S&R": "LoraLoaderModelOnly"},
"widgets_values": ["h3\\minimax_h3_t2v_mshandjob_audio_v1.safetensors", 1],
"widgets_values_named": {"lora_name": "h3\\minimax_h3_t2v_mshandjob_audio_v1.safetensors", "strength_model": 1}
},
{
"id": 67,
"type": "RTXVideoSuperResolution",
"pos": [5085.886101348601, 3012.843050491562],
"size": [285.81015625, 130],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [{"name": "images", "type": "IMAGE", "link": 155}],
"outputs": [{"name": "upscaled_images", "type": "IMAGE", "links": [156]}],
"properties": {"Node name for S&R": "RTXVideoSuperResolution"},
"widgets_values": ["target dimensions", 1920, 1080, "ULTRA"],
"widgets_values_named": {"resize_type": "target dimensions", "resize_type.width": 1920, "resize_type.height": 1080, "quality": "ULTRA"}
},
{
"id": 68,
"type": "CreateVideo",
"pos": [5404.822730085631, 2911.779640536847],
"size": [270, 150],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{"name": "images", "type": "IMAGE", "link": 156},
{"name": "audio", "shape": 7, "type": "AUDIO", "link": 157},
{"name": "fps", "type": "FLOAT", "widget": {"name": "fps"}, "link": 159}
],
"outputs": [{"name": "VIDEO", "type": "VIDEO", "links": [158]}],
"properties": {"Node name for S&R": "CreateVideo"},
"widgets_values": [30, "auto", "sRGB", "h264"],
"widgets_values_named": {"fps": 30, "bit_depth": "auto", "color_space": "sRGB", "codec": "h264"}
},
{
"id": 69,
"type": "SaveVideo",
"pos": [5174.9019386914315, 3158.8462584926756],
"size": [270, 249.875],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [{"name": "video", "type": "VIDEO", "link": 158}],
"outputs": [{"name": "video", "type": "VIDEO", "links": null}],
"properties": {"Node name for S&R": "SaveVideo"},
"widgets_values": ["video/ComfyUI", "auto", "auto", "auto"],
"widgets_values_named": {"filename_prefix": "video/ComfyUI", "format": "auto", "format.codec": "auto", "codec": "auto"}
},
{
"id": 81,
"type": "easy multiTrackEditor",
"pos": [2903.9332978768557, 2791.1687123187226],
"size": [800, 700],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [
{"name": "prompt_override", "shape": 7, "type": "*", "link": null},
{"name": "image", "shape": 7, "type": "IMAGE", "link": null},
{"name": "audio", "shape": 7, "type": "AUDIO", "link": null},
{"name": "video", "shape": 7, "type": "VIDEO", "link": null}
],
"outputs": [
{"name": "TRACKS_INFO", "type": "TRACKS_INFO", "links": [166]},
{"name": "IMAGES", "shape": 6, "type": "IMAGE", "links": null},
{"name": "AUDIO", "shape": 6, "type": "AUDIO", "links": null},
{"name": "VIDEO", "shape": 6, "type": "VIDEO", "links": null}
],
"properties": {"easyMediaTimelineHeight": 700, "Node name for S&R": "easy multiTrackEditor"},
"widgets_values": [
"1920 x 1080 (16:9)",
"stretch",
"MiniMax",
"{\"muted\":false,\"volume_db\":0,\"task_markers\":[],\"task_overview\":false,\"tracks\":[{\"id\":\"ac9830d5-1ffc-46a7-99ac-19989de1ec17\",\"name\":\"Task 0\",\"type\":\"task\",\"task_mode\":\"default\",\"color\":\"var(--multitrack-task-bg)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[{\"id\":\"389405fd-d576-45ab-b917-a4a4bbda7153\",\"start_frame\":0,\"end_frame\":120,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"shot\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene1\"}},{\"id\":\"b02ca679-4bb1-4758-86cc-a29d95d7d13a\",\"start_frame\":120,\"end_frame\":240,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"context\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene2\"}}]},{\"id\":\"49025400-b968-42bd-9e01-fda75549ca52\",\"name\":\"Video 0\",\"type\":\"video\",\"color\":\"var(--primary)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[],\"audio_locked\":false}],\"total_length\":240,\"frame_rate\":24}"
],
"widgets_values_named": {
"resolution": "1920 x 1080 (16:9)",
"resolution.resize_method": "stretch",
"format": "MiniMax",
"track_data": "{\"muted\":false,\"volume_db\":0,\"task_markers\":[],\"task_overview\":false,\"tracks\":[{\"id\":\"ac9830d5-1ffc-46a7-99ac-19989de1ec17\",\"name\":\"Task 0\",\"type\":\"task\",\"task_mode\":\"default\",\"color\":\"var(--multitrack-task-bg)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[{\"id\":\"389405fd-d576-45ab-b917-a4a4bbda7153\",\"start_frame\":0,\"end_frame\":120,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"shot\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene1\"}},{\"id\":\"b02ca679-4bb1-4758-86cc-a29d95d7d13a\",\"start_frame\":120,\"end_frame\":240,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"context\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene2\"}}]},{\"id\":\"49025400-b968-42bd-9e01-fda75549ca52\",\"name\":\"Video 0\",\"type\":\"video\",\"color\":\"var(--primary)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[],\"audio_locked\":false}],\"total_length\":240,\"frame_rate\":24}"
}
}
],
"links": [
[136, 52, 1, 57, 2, "CLIP"],
[137, 52, 2, 57, 3, "VAE"],
[138, 52, 3, 57, 4, "VAE"],
[144, 48, 0, 57, 1, "MODEL"],
[148, 57, 0, 63, 0, "STRING"],
[149, 63, 1, 45, 1, "AUDIO"],
[150, 63, 2, 45, 2, "FLOAT"],
[151, 63, 0, 45, 0, "IMAGE"],
[153, 52, 0, 66, 0, "MODEL"],
[154, 66, 0, 48, 0, "MODEL"],
[156, 67, 0, 68, 0, "IMAGE"],
[157, 63, 1, 68, 1, "AUDIO"],
[159, 63, 2, 68, 2, "FLOAT"],
[166, 81, 0, 57, 0, "TRACKS_INFO"],
[103, 45, 0, 46, 0, "VIDEO"],
[155, 63, 0, 67, 0, "IMAGE"],
[158, 68, 0, 69, 0, "VIDEO"]
],
"groups": [],
"config": {},
"extra": {
"ue_links": [],
"ds": {"scale": 0.5131581182307068, "offset": [-2322.4149844419144, -2181.0583714987984]},
"links_added_by_ue": [],
"frontendVersion": "1.53.6",
"VHS_latentpreview": false,
"VHS_latentpreviewrate": 0,
"VHS_MetadataImage": true,
"VHS_KeepIntermediate": true
},
"version": 0.4
}

73
bench/moxie_bench.spec Normal file
View File

@ -0,0 +1,73 @@
# -*- mode: python ; coding: utf-8 -*-
"""PyInstaller spec for the Moxie bench sidecar.
Build (on the target machine, inside the repo root):
pip install -r bench/requirements.txt pyinstaller
cd frontend && bun install && bun run build:bench:release && cd ..
pyinstaller bench/moxie_bench.spec
Output: dist/moxie-bench/ (onedir - ships moxie-bench(.exe) plus _internal/)
Tauri 2 sidecar wiring (see bench/tauri/README.md):
- copy dist/moxie-bench/ next to your Tauri binary as `moxie-bench/`
- point externalBin / or spawn `moxie-bench/moxie-bench --port <free-port>`
- poll http://127.0.0.1:<port>/api/health until {"ok": true}
"""
import os
from pathlib import Path
repo_root = Path(SPECPATH).resolve().parent # bench/ -> repo root
hiddenimports = [
"uvicorn.logging",
"uvicorn.loops",
"uvicorn.loops.auto",
"uvicorn.protocols",
"uvicorn.protocols.http",
"uvicorn.protocols.http.auto",
"uvicorn.protocols.websockets",
"uvicorn.protocols.websockets.auto",
"uvicorn.lifespan",
"uvicorn.lifespan.on",
"anyio._backends._asyncio",
]
a = Analysis(
["bench/run.py"],
pathex=[str(repo_root / "bench")],
binaries=[],
datas=[
# the prebuilt bench UI + base workflow travel with the binary
(str(repo_root / "bench" / "ui-dist"), "bench/ui-dist"),
(str(repo_root / "bench" / "base_workflow.json"), "bench"),
],
hiddenimports=hiddenimports,
hookspath=[],
hooksconfig={},
runtime_hooks=[],
excludes=["torch", "comfy", "comfy_execution", "folder_paths"],
noarchive=False,
)
pyz = PYZ(a.pure)
exe = EXE(
pyz,
a.scripts,
[],
exclude_binaries=True,
name="moxie-bench",
debug=False,
bootloader_ignore_signals=False,
strip=False,
upx=False,
console=True, # sidecar logs are useful; Tauri can hide the window
)
coll = COLLECT(
exe,
a.binaries,
a.datas,
strip=False,
upx=False,
name="moxie-bench",
)

4
bench/requirements.txt Normal file
View File

@ -0,0 +1,4 @@
fastapi>=0.110
uvicorn>=0.29
numpy>=1.26
pillow>=10.0

33
bench/run.py Normal file
View File

@ -0,0 +1,33 @@
#!/usr/bin/env python3
"""Moxie bench entry point.
python bench/run.py --port 8787 # auto backend (mock here)
python bench/run.py --backend comfy --comfyui-url http://127.0.0.1:8188
"""
from __future__ import annotations
import sys
from pathlib import Path
sys.path.insert(0, str(Path(__file__).resolve().parent))
from app.config import BenchConfig # noqa: E402
def main() -> None:
import uvicorn
from app.server import create_app
cfg = BenchConfig.from_args()
backend = cfg.resolve_backend()
print(f"[moxie-bench] backend={backend} http://{cfg.host}:{cfg.port}")
if backend == "comfy":
print(f"[moxie-bench] driving ComfyUI at {cfg.comfyui_url}")
else:
print("[moxie-bench] self-contained mock renderer (ffmpeg + numpy)")
uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port, log_level="info")
if __name__ == "__main__":
main()

110
bench/tauri/README.md Normal file
View File

@ -0,0 +1,110 @@
# Using the bench as a Tauri 2 sidecar
The bench is a self-contained Python HTTP server (FastAPI + uvicorn). Packaged
with PyInstaller it becomes an `moxie-bench` binary your Tauri 2 app spawns at
startup and talks to over `http://127.0.0.1:<port>`.
## 1. Build the sidecar
```bash
# in the MMnode repo
pip install -r bench/requirements.txt pyinstaller
cd frontend && bun install && bun run build:bench:release && cd ..
pyinstaller bench/moxie_bench.spec
# -> dist/moxie-bench/moxie-bench(.exe) + _internal/
```
Ship the whole `dist/moxie-bench/` folder (onedir layout: the binary needs
`_internal/` beside it).
## 2. Bundle it with the Tauri app
`tauri.conf.json` (resources approach — simplest and cross-platform):
```json
{
"bundle": {
"resources": {
"../MMnode/dist/moxie-bench/": "moxie-bench/"
}
}
}
```
The folder lands in the app resource dir; resolve it at runtime with
`@tauri-apps/api/path` → `resourceDir()`.
## 3. Spawn and wait for health
Rust side (tauri plugin-shell or std::process):
```rust
use std::process::{Command, Stdio};
use std::time::Duration;
let sidecar = app.path().resource_dir()?.join("moxie-bench/moxie-bench");
let port: u16 = 8787; // pick a free port, pass it through
let mut child = Command::new(sidecar)
.arg("--port").arg(port.to_string())
.arg("--host").arg("127.0.0.1")
.arg("--backend").arg("auto") // mock when no ComfyUI is reachable
.stdout(Stdio::null())
.stderr(Stdio::null())
.spawn()?;
// poll until ready before loading the webview UI
async fn wait_healthy(port: u16) -> bool {
let url = format!("http://127.0.0.1:{port}/api/health");
for _ in 0..60 {
if let Ok(resp) = reqwest::get(&url).await {
if resp.status().is_success() { return true; }
}
tokio::time::sleep(Duration::from_millis(500)).await;
}
false
}
```
## 4. Drive it from the webview
Everything the bench UI does is plain HTTP/WS on the same origin — the Tauri
webview can call the same endpoints:
| Action | Call |
|---|---|
| health/backend | `GET /api/health`, `GET /api/config` |
| base workflow + timeline | `GET /api/workflow` |
| render plan preview | `GET /api/plan` |
| queue a render | `POST /api/render {track_data, overrides}` |
| live progress | WebSocket `ws://127.0.0.1:<port>/api/ws` |
| job list / detail | `GET /api/jobs`, `GET /api/jobs/{id}` |
| play/save outputs | `GET /api/view?filename=...&type=output&job=<job_id>` |
`POST /api/render` body:
```json
{
"track_data": { "tracks": [ ... editor JSON ... ] },
"overrides": {
"width": 1920, "height": 1080, "frame_rate": 24,
"steps": 4, "seed": 12345, "project_name": "MM-myproject",
"backend": "auto"
}
}
```
## 5. Backends
- `mock` — self-contained synthetic renderer (numpy + ffmpeg). Deterministic
per (prompt, seed); real mp4 + audio on both output branches. Use it to
develop the Tauri integration with zero model downloads.
- `comfy` — drives a running ComfyUI (`--comfyui-url`, default
`http://127.0.0.1:8188`) with the exact base workflow. Real generation, same
API surface. Start ComfyUI headless: `python main.py --listen 127.0.0.1`.
- `auto` (default) — comfy when reachable, else mock.
## 6. Shutdown
Kill the spawned process on app exit (Rust `child.kill()` / `Drop`), or send
SIGTERM. The server has no state beyond `work/` on disk; job history lives in
memory.

View File

@ -0,0 +1,196 @@
"""Bench contract tests: no ComfyUI, no torch required.
Run: python3 -m pytest bench/tests -q
"""
from __future__ import annotations
import json
import sys
from pathlib import Path
import pytest
BENCH_DIR = Path(__file__).resolve().parent.parent
sys.path.insert(0, str(BENCH_DIR))
from app.tracks import TrackDataError, build_render_plan # noqa: E402
from app.workflow import ( # noqa: E402
EDITOR_TYPE,
WorkflowError,
describe,
editor_track_data,
load_base_workflow,
parse_workflow_dict,
)
BASE = load_base_workflow(BENCH_DIR / "base_workflow.json")
# ---------------------------------------------------------------------------
# workflow parsing
# ---------------------------------------------------------------------------
def test_all_nine_stage_types_present():
types = set(BASE.stages)
assert {
"MoxieMultimediaLoader", "LoraLoaderModelOnly", "MoxiePreviewOverride",
EDITOR_TYPE, "MoxieProject", "MoxieCombine", "RTXVideoSuperResolution",
"CreateVideo",
} <= types
assert "CreateVideo#2" in types
def test_save_nodes_classified_by_branch():
assert set(BASE.save_nodes) == {"native", "upscaled"}
# native save is fed by CreateVideo #45, upscaled by #68
native_src = BASE.save_nodes["native"].link_inputs["video"][0]
upscaled_src = BASE.save_nodes["upscaled"].link_inputs["video"][0]
assert native_src == BASE.stages["CreateVideo"].node_id
assert upscaled_src == BASE.stages["CreateVideo#2"].node_id
def test_describe_summary():
d = describe(BASE)
assert d["editor_resolution"] == {"width": 1920, "height": 1080}
assert d["frame_rate"] == 24
assert d["steps"] == 4
assert d["seed"] == 803100406447983
assert d["project_name"] == "MM-girlwaving"
assert d["upscale"]["target"] == {"width": 1920, "height": 1080}
assert d["upscale"]["quality"] == "ULTRA"
assert d["lora"].endswith(".safetensors")
def test_editor_track_data_roundtrip():
td = editor_track_data(BASE)
assert td["frame_rate"] == 24
assert td["total_length"] == 240
task_tracks = [t for t in td["tracks"] if t["type"] == "task"]
assert len(task_tracks) == 1
assert len(task_tracks[0]["segments"]) == 2
def test_missing_stage_rejected():
raw = json.loads((BENCH_DIR / "base_workflow.json").read_text())
raw["nodes"] = [n for n in raw["nodes"] if n["type"] != "MoxieProject"]
with pytest.raises(WorkflowError):
parse_workflow_dict(raw)
# ---------------------------------------------------------------------------
# track_data -> RenderPlan
# ---------------------------------------------------------------------------
def _plan(**kw):
defaults = dict(width=1920, height=1080, frame_rate=24, steps=4, seed=7,
project_name="MM-girlwaving", upscale_width=1920,
upscale_height=1080)
defaults.update(kw)
return build_render_plan(editor_track_data(BASE), **defaults)
def test_plan_segments_ordered_and_bounded():
plan = _plan()
assert [(s.prompt, s.start_frame, s.end_frame) for s in plan.segments] == [
("scene1", 0, 120), ("scene2", 120, 240)]
assert plan.duration_seconds == pytest.approx(10.0)
def test_plan_honors_overrides():
plan = _plan(width=1088, height=1920, steps=8, seed=99)
assert (plan.width, plan.height) == (1088, 1920)
assert plan.steps == 8 and plan.seed == 99
def test_continuity_mode_carried_through():
plan = _plan()
modes = {s.prompt: s.continuity_mode for s in plan.segments}
assert modes == {"scene1": "shot", "scene2": "context"}
def test_invalid_track_data_raises():
with pytest.raises(TrackDataError):
build_render_plan("not json", width=1, height=1)
with pytest.raises(TrackDataError):
build_render_plan({"nope": 1}, width=1, height=1)
# ---------------------------------------------------------------------------
# /prompt conversion (comfy bridge)
# ---------------------------------------------------------------------------
def test_prompt_conversion_shape():
from app.backends.comfy import to_prompt_format
prompt = to_prompt_format(BASE.raw)
assert set(prompt) == {str(n["id"]) for n in BASE.raw["nodes"]}
loader = prompt["52"]
assert loader["class_type"] == "MoxieMultimediaLoader"
assert loader["inputs"]["diffusion_model"] == "Moxie-Multimedia.safetensors"
project = prompt["57"]
assert project["inputs"]["tracks_info"] == ["81", 0]
assert project["inputs"]["model"] == ["48", 0]
assert project["inputs"]["clip"] == ["52", 1]
assert "randomize" not in project["inputs"] # UI-only control widget
def test_prompt_conversion_wired_widget_wins():
from app.backends.comfy import to_prompt_format
prompt = to_prompt_format(BASE.raw)
# CreateVideo #45: fps widget is wired to MoxieCombine slot 2
create = prompt["45"]
assert create["inputs"]["fps"] == ["63", 2]
assert create["inputs"]["codec"] == "h264"
def test_substitution_updates_all_parameters():
from app.backends.comfy import substitute
plan = build_render_plan(
editor_track_data(BASE), width=720, height=1280, frame_rate=16,
steps=6, seed=1234, project_name="MM-bench", lora="MM3step.safetensors",
lora_strength=0.8, upscale_width=1440, upscale_height=2560,
)
raw = substitute(BASE, plan)
nodes = {n["id"]: n for n in raw["nodes"]}
editor = nodes[81]["widgets_values_named"]
assert json.loads(editor["track_data"])["frame_rate"] == 16
assert editor["resolution"] == "720 x 1280 (9:16)"
proj = nodes[57]["widgets_values_named"]
assert proj["project_name"] == "MM-bench" and proj["steps"] == 6
assert proj["seed"] == 1234
lora = nodes[66]["widgets_values_named"]
assert lora["lora_name"] == "MM3step.safetensors" and lora["strength_model"] == 0.8
rtx = nodes[67]["widgets_values_named"]
assert rtx["resize_type.width"] == 1440 and rtx["resize_type.height"] == 2560
combine = nodes[63]["widgets_values_named"]
pd = json.loads(combine["project_data"])
assert pd["width"] == 720 and pd["height"] == 1280 and pd["frame_rate"] == 16
saves = {n["id"]: n for n in raw["nodes"] if n["type"] == "SaveVideo"}
assert saves[46]["widgets_values_named"]["filename_prefix"] == "moxie-bench/native"
assert saves[69]["widgets_values_named"]["filename_prefix"] == "moxie-bench/upscaled"
def test_substitution_custom_resolution_uses_dict():
from app.backends.comfy import substitute
plan = build_render_plan(editor_track_data(BASE), width=998, height=742)
raw = substitute(BASE, plan)
editor = next(n for n in raw["nodes"] if n["id"] == 81)
assert editor["widgets_values_named"]["resolution"] == {"width": 998, "height": 742}
# ---------------------------------------------------------------------------
# mock backend end-to-end (ffmpeg required)
# ---------------------------------------------------------------------------
@pytest.mark.skipif(True, reason="e2e covered by scripts/test_bench_pipeline.py; keep pytest fast")
def test_mock_render_placeholder():
pass

16
bench/ui-dist/index.html Normal file
View File

@ -0,0 +1,16 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Moxie Multimedia Bench</title>
<style>
html, body, #root { height: 100%; margin: 0; }
body { background: #09090b; color: #fafafa; }
</style>
</head>
<body>
<div id="root"></div>
<script type="module" src="./main.js"></script>
</body>
</html>

71
bench/ui-dist/main.js Normal file

File diff suppressed because one or more lines are too long

116
download_models.sh Executable file
View File

@ -0,0 +1,116 @@
#!/usr/bin/env bash
#
# download_models.sh - fetch the MiniMax H3 model set for the Moxie Multimedia
# Loader and place it (renamed) into this node pack's models/ folders.
#
# Usage (from anywhere - the script targets its own pack directory):
# bash custom_nodes/MMnode/download_models.sh
#
# Files are downloaded to "<name>.part" first and moved into place when
# complete, so an interrupted run can be re-run safely (it resumes).
# Already-complete files are skipped, so the script is idempotent.
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
MODELS_DIR="$SCRIPT_DIR/models"
# ---------------------------------------------------------------------------
# Model manifest: subdir/filename <- URL
# Add the turbo LoRA here once a URL is available, e.g.:
# "loras/Moxie-Turbo.safetensors|https://.../Moxie-Turbo.safetensors"
# ---------------------------------------------------------------------------
MANIFEST=(
"diffusion/Moxie-Multimedia.safetensors|https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity/resolve/main/Minimax-h3_Singularity_ref2va_v1.3_Pruned_w4a8.safetensors"
"clip/MM-VL.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
"vae/video_vae.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_int8_convrot.safetensors"
"audio_vae/audio_vae.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors"
)
# ---------------------------------------------------------------------------
# Downloader: curl first (resume via -C -), wget as fallback.
# ---------------------------------------------------------------------------
curl_supports_retry_all() {
# --retry-all-errors exists since curl 7.71.0; compare versions safely.
local v
v="$(curl --version 2>/dev/null | head -n1 | awk '{print $2}')"
[[ "$(printf '%s\n7.71.0\n' "$v" | sort -V | head -n1)" == "7.71.0" ]]
}
fetch() {
local url="$1" output="$2"
if command -v curl >/dev/null 2>&1; then
local retry_all=""
if curl_supports_retry_all; then
retry_all="--retry-all-errors" # HuggingFace answers bursts with 429/503
fi
curl -L --fail --retry 5 --retry-delay 5 $retry_all \
--connect-timeout 30 -C - -o "$output" "$url"
elif command -v wget >/dev/null 2>&1; then
wget -c -q --show-progress -O "$output" "$url"
else
echo "ERROR: neither curl nor wget is installed." >&2
return 1
fi
}
download_entry() {
local entry="$1"
local rel="${entry%%|*}"
local url="${entry#*|}"
local subdir="${rel%%/*}"
local filename="${rel#*/}"
local target_dir="$MODELS_DIR/$subdir"
local target="$target_dir/$filename"
local partial="$target.part"
if [[ -f "$target" ]]; then
echo "[skip] $subdir/$filename already present"
return 0
fi
mkdir -p "$target_dir"
echo "[get ] $subdir/$filename"
echo " from $url"
if ! fetch "$url" "$partial"; then
# A stale/incomplete .part can make resume fail (e.g. HTTP 416 when the
# file was already fully fetched but the move never happened).
if [[ -f "$partial" ]]; then
echo " resume failed - deleting partial and restarting download"
rm -f "$partial"
fetch "$url" "$partial"
else
return 1
fi
fi
mv -f "$partial" "$target"
echo "[done] $subdir/$filename ($(du -h "$target" | cut -f1))"
}
echo "Moxie Multimedia - model downloader"
echo " pack models dir: $MODELS_DIR"
echo
failed=0
for entry in "${MANIFEST[@]}"; do
if ! download_entry "$entry"; then
echo "ERROR: failed to download $entry" >&2
failed=1
fi
done
echo
if [[ "$failed" -eq 0 ]]; then
echo "All models ready:"
for entry in "${MANIFEST[@]}"; do
local_rel="${entry%%|*}"
printf ' %-40s %s\n' "$local_rel" "$(du -h "$MODELS_DIR/$local_rel" 2>/dev/null | cut -f1 || echo missing)"
done
echo
echo "Restart ComfyUI so the loader picks the files up."
else
echo "Some downloads failed - re-run this script to resume." >&2
exit 1
fi

103
frontend/build.bench.ts Normal file
View File

@ -0,0 +1,103 @@
/**
* Bench UI build — mirrors build.ts but outputs a standalone web app into
* ../bench/ui-dist (served by the bench FastAPI server).
*
* bun run build.bench.ts # dev build
* bun run build.bench.ts --release # minified
*/
import { $ } from "bun";
import path from "path";
import fs from "fs";
import os from "os";
const isRelease = Bun.argv.includes("--release");
const rootDir = path.resolve(import.meta.dir, "..");
const outdir = path.join(rootDir, "bench", "ui-dist");
const project_name = path.basename(rootDir);
fs.mkdirSync(outdir, { recursive: true });
function scopeTailwindThemeVariables(css: string, widgetClass: string): string {
return css.replace(/:root\s*,\s*:host\s*\{/g, `.${widgetClass}{`);
}
async function buildCSS(): Promise<string> {
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "moxie-bench-css-"));
const cssOut = path.join(tempDir, "globals.css");
const minifyFlag = isRelease ? "--minify" : "";
try {
await $`./node_modules/.bin/tailwindcss -i ./src/styles/globals.css -o ${cssOut} ${minifyFlag}`;
const widgetClass = project_name.toLowerCase();
const css = fs.readFileSync(cssOut, "utf-8");
return scopeTailwindThemeVariables(
css.replaceAll("__CUSTOM_NODE_CLASS__", widgetClass),
widgetClass,
);
} finally {
fs.rmSync(tempDir, { recursive: true, force: true });
}
}
async function build() {
const start = Date.now();
const globalCss = await buildCSS();
const result = await Bun.build({
entrypoints: ["./src/bench/main.tsx"],
outdir,
naming: {
chunk: "chunks/[name]-[hash].[ext]",
entry: "[name].[ext]",
asset: "assets/[name].[ext]",
},
splitting: true,
target: "browser",
minify: isRelease,
define: {
"process.env.NODE_ENV": isRelease ? '"production"' : '"development"',
"import.meta.env.PROJECT_NAME": `"${project_name}"`,
"__COMFY_EASY_MEDIA_GLOBAL_CSS__": JSON.stringify(globalCss),
},
css: false,
});
if (!result.success) {
for (const log of result.logs) {
console.error(log);
}
throw new Error("bench UI build failed");
}
// index.html referencing the built entry
const entryName = result.outputs
.map((o) => o.path)
.filter((p) => p.endsWith("main.js") || p.endsWith("main.js.map"))
.map((p) => path.basename(p))
.find((p) => p.endsWith(".js"));
const html = `<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<title>Moxie Multimedia Bench</title>
<style>
html, body, #root { height: 100%; margin: 0; }
body { background: #09090b; color: #fafafa; }
</style>
</head>
<body>
<div id="root"></div>
<script type="module" src="./${entryName ?? "main.js"}"></script>
</body>
</html>
`;
fs.writeFileSync(path.join(outdir, "index.html"), html);
console.log(
`[build:bench${isRelease ? ":release" : ""}] Done in ${Date.now() - start}ms (${result.outputs.length} outputs) -> ${outdir}`,
);
}
await build();

View File

@ -8,7 +8,9 @@
"build": "bun run build.ts", "build": "bun run build.ts",
"build:release": "bun run build.ts --release", "build:release": "bun run build.ts --release",
"test": "vitest run", "test": "vitest run",
"test:watch": "vitest" "test:watch": "vitest",
"build:bench": "bun run build.bench.ts",
"build:bench:release": "bun run build.bench.ts --release"
}, },
"dependencies": { "dependencies": {
"@base-ui/react": "^1.6.0", "@base-ui/react": "^1.6.0",

View File

@ -0,0 +1,268 @@
import { useEffect, useMemo, useReducer, useRef, useState } from 'react'
import { Loader2, Play, Radio, SquareTerminal } from 'lucide-react'
import { Button } from '@/components/ui/button'
import { Input } from '@/components/ui/input'
import { TooltipProvider } from '@/components/ui/tooltip'
import { MultiTrackWidget } from '@/components/widgets/MultiTrackWidget'
import type { TrackData } from '@/types/multitrack'
import {
benchApp,
benchState,
connectBenchSocket,
createBenchNode,
createBenchWidget,
fetchBenchConfig,
onBenchStateChange,
queueBenchRender,
type BenchOutputInfo,
} from './bench-runtime'
const RESOLUTION_PRESETS: Array<{ label: string; width: number; height: number }> = [
{ label: '1920 x 1080 (16:9)', width: 1920, height: 1080 },
{ label: '1088 x 1920 (9:16)', width: 1088, height: 1920 },
{ label: '1280 x 720 (16:9)', width: 1280, height: 720 },
{ label: '720 x 1280 (9:16)', width: 720, height: 1280 },
{ label: '832 x 480 (16:9)', width: 832, height: 480 },
{ label: '480 x 832 (9:16)', width: 480, height: 832 },
{ label: '1024 x 1024 (1:1)', width: 1024, height: 1024 },
]
function useBenchTick(): number {
const [, force] = useReducer((x: number) => x + 1, 0)
useEffect(() => onBenchStateChange(force), [force])
return 0
}
function OutputCard({ label, out }: { label: string; out: BenchOutputInfo }) {
return (
<div className="flex min-w-0 flex-1 flex-col gap-1 rounded-md border bg-background p-2">
<div className="flex items-center justify-between text-xs text-muted-foreground">
<span className="font-medium text-foreground">{label}</span>
<span>
{out.width}x{out.height} · {out.fps ?? '?'} fps · {Math.round((out.bytes ?? 0) / 1024)} KB
</span>
</div>
<video className="w-full rounded-sm" src={out.url} controls preload="metadata" />
</div>
)
}
export function BenchApp() {
useBenchTick()
const [trackData, setTrackData] = useState<TrackData | null>(null)
const [config, setConfig] = useState<Awaited<ReturnType<typeof fetchBenchConfig>>>(null)
const [backend, setBackend] = useState('auto')
const [resolution, setResolution] = useState('1920 x 1080 (16:9)')
const [steps, setSteps] = useState(4)
const [seed, setSeed] = useState(803100406447983)
const [projectName, setProjectName] = useState('MM-girlwaving')
const [queueing, setQueueing] = useState(false)
const node = useMemo(createBenchNode, [])
const widget = useMemo(createBenchWidget, [])
const consoleRef = useRef<HTMLDivElement>(null)
useEffect(() => {
connectBenchSocket()
void fetchBenchConfig().then(setConfig)
fetch('/api/workflow')
.then((r) => r.json())
.then((body: { track_data: string }) => {
try {
const parsed = JSON.parse(body.track_data) as TrackData
setTrackData(parsed)
setProjectName((current) => current || 'MM-girlwaving')
} catch {
/* server sent unusable track data; keep the empty editor */
}
})
.catch(() => undefined)
}, [])
useEffect(() => {
if (consoleRef.current) consoleRef.current.scrollTop = consoleRef.current.scrollHeight
})
const preset = RESOLUTION_PRESETS.find((p) => p.label === resolution)
const queue = async () => {
if (!trackData || queueing) return
setQueueing(true)
try {
await queueBenchRender({
track_data: trackData,
backend: backend === 'auto' ? undefined : backend,
overrides: {
width: preset?.width ?? 1920,
height: preset?.height ?? 1080,
steps,
seed,
project_name: projectName,
},
})
} catch (error) {
benchState.error = error instanceof Error ? error.message : String(error)
} finally {
setQueueing(false)
}
}
const busy = benchState.jobStatus === 'queued' || benchState.jobStatus === 'running'
const outputs = benchState.outputs ?? {}
return (
<TooltipProvider>
<div className="mmnode flex h-screen min-h-0 flex-col gap-2 bg-background p-3 text-foreground">
<header className="flex flex-wrap items-center gap-3">
<h1 className="text-lg font-semibold">Moxie Multimedia Bench</h1>
<span
className={`inline-flex items-center gap-1 rounded-full px-2 py-0.5 text-xs ${
benchState.connected
? 'bg-green-500/15 text-green-600'
: 'bg-red-500/15 text-red-600'
}`}
>
<Radio size={12} /> {benchState.connected ? 'server online' : 'offline'}
</span>
<span className="rounded-full bg-muted px-2 py-0.5 text-xs text-muted-foreground">
backend: {config?.backend ?? benchState.backend}
</span>
<span className="text-xs text-muted-foreground">v{config?.version ?? '?'}</span>
<div className="ml-auto" />
</header>
<div className="flex min-h-0 flex-1 gap-3">
<main className="mmnode flex min-h-0 min-w-0 flex-1 overflow-hidden rounded-md border">
{trackData ? (
<MultiTrackWidget
value={trackData}
onChange={(next) => setTrackData(next)}
inputName="track_data"
node={node}
widget={widget as never}
app={benchApp}
/>
) : (
<div className="flex h-full w-full items-center justify-center text-muted-foreground">
<Loader2 className="mr-2 animate-spin" size={18} /> loading timeline…
</div>
)}
</main>
<aside className="flex w-80 shrink-0 flex-col gap-3 overflow-y-auto rounded-md border p-3">
<div className="text-sm font-semibold">Render</div>
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
Resolution
<select
className="rounded-md border bg-background p-1.5 text-foreground"
value={resolution}
onChange={(e) => setResolution(e.target.value)}
>
{RESOLUTION_PRESETS.map((p) => (
<option key={p.label} value={p.label}>
{p.label}
</option>
))}
</select>
</label>
<div className="grid grid-cols-2 gap-2">
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
Steps
<Input
type="number"
min={1}
max={32}
value={steps}
onChange={(e) => setSteps(Math.max(1, Number(e.target.value) || 1))}
/>
</label>
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
Seed
<Input
type="number"
value={seed}
onChange={(e) => setSeed(Number(e.target.value) || 0)}
/>
</label>
</div>
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
Project name
<Input value={projectName} onChange={(e) => setProjectName(e.target.value)} />
</label>
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
Backend
<select
className="rounded-md border bg-background p-1.5 text-foreground"
value={backend}
onChange={(e) => setBackend(e.target.value)}
>
<option value="auto">auto</option>
<option value="mock">mock</option>
<option value="comfy">comfy (needs running ComfyUI)</option>
</select>
{config && !config.backends?.comfy?.available && (
<span className="text-[11px] text-muted-foreground">
comfy bridge not reachable — mock stays the default
</span>
)}
</label>
<Button onClick={queue} disabled={busy || queueing || !trackData}>
{busy ? <Loader2 className="mr-2 animate-spin" size={14} /> : <Play className="mr-2" size={14} />}
{busy ? 'Rendering…' : 'Queue render'}
</Button>
{busy && (
<div className="flex flex-col gap-1">
<div className="h-2 w-full overflow-hidden rounded-full bg-muted">
<div
className="h-full rounded-full bg-primary transition-all"
style={{ width: `${benchState.progress}%` }}
/>
</div>
<div className="text-xs text-muted-foreground">
{benchState.stage || 'waiting…'} — {benchState.message}
</div>
</div>
)}
{benchState.error && (
<div className="rounded-md bg-red-500/10 p-2 text-xs text-red-600">
{benchState.error}
</div>
)}
<div className="text-sm font-semibold">Outputs</div>
<div className="flex flex-col gap-2">
{outputs.native ? (
<OutputCard label="native (MoxieCombine)" out={outputs.native} />
) : null}
{outputs.upscaled ? (
<OutputCard label="upscaled (RTX branch)" out={outputs.upscaled} />
) : null}
{!outputs.native && !outputs.upscaled && (
<div className="text-xs text-muted-foreground">
queue a render — native and RTX-upscaled outputs appear here
</div>
)}
</div>
<div className="flex items-center gap-1 text-sm font-semibold">
<SquareTerminal size={14} /> Console
</div>
<div
ref={consoleRef}
className="h-40 overflow-y-auto rounded-md bg-muted/50 p-2 font-mono text-[11px] leading-relaxed"
>
{benchState.console.map((line, i) => (
<div key={i}>{line}</div>
))}
</div>
</aside>
</div>
</div>
</TooltipProvider>
)
}

View File

@ -0,0 +1,314 @@
/**
* Bench runtime: the ComfyUI-shaped shim that lets the unmodified editor
* components run against the bench server.
*
* This module MUST be imported before anything that pulls in `@/lib/comfy-api`
* (that module reads `globalThis.comfyAPI.app.app` at import time).
*/
import type { ComfyApp } from '@comfyorg/comfyui-frontend-types'
type CustomEventCallback = (event: CustomEvent<unknown>) => void
export interface BenchJobEvent {
type: string
job_id?: string
status?: string
stage?: string
progress?: number
message?: string
error?: string
outputs?: Record<string, BenchOutputInfo>
}
export interface BenchOutputInfo {
filename: string
url: string
width?: number
height?: number
fps?: number
bytes?: number
}
export interface BenchConfigInfo {
version: string
backend: string
backends: Record<string, { available: boolean; url?: string }>
}
class Hub {
private target = new EventTarget()
add(type: string, cb: CustomEventCallback): void {
this.target.addEventListener(type, cb as EventListener)
}
remove(type: string, cb: CustomEventCallback): void {
this.target.removeEventListener(type, cb as EventListener)
}
dispatch(type: string, detail: unknown): void {
this.target.dispatchEvent(new CustomEvent(type, { detail }))
}
}
export const benchHub = new Hub()
/** Live bench state the dock UI reads. */
export const benchState = {
connected: false,
backend: 'mock',
jobId: null as string | null,
jobStatus: 'idle',
stage: '',
progress: 0,
message: '',
error: null as string | null,
outputs: {} as Record<string, BenchOutputInfo>,
console: [] as string[],
}
const listeners = new Set<() => void>()
export function onBenchStateChange(cb: () => void): () => void {
listeners.add(cb)
return () => {
listeners.delete(cb)
}
}
function notify(): void {
listeners.forEach((cb) => cb())
}
function log(line: string): void {
benchState.console.push(`[${new Date().toLocaleTimeString()}] ${line}`)
if (benchState.console.length > 400) benchState.console.splice(0, benchState.console.length - 400)
notify()
}
function applyJobEvent(msg: BenchJobEvent): void {
benchState.jobId = msg.job_id ?? benchState.jobId
benchState.jobStatus = msg.status ?? benchState.jobStatus
benchState.stage = msg.stage ?? benchState.stage
benchState.progress = typeof msg.progress === 'number' ? msg.progress : benchState.progress
benchState.message = msg.message ?? benchState.message
if (msg.type === 'execution_error') {
benchState.error = msg.error ?? 'render failed'
log(`ERROR: ${benchState.error}`)
}
if (msg.type === 'execution_success' && msg.outputs) {
benchState.outputs = msg.outputs
benchState.error = null
log(`done: ${Object.keys(msg.outputs).join(', ')}`)
}
if (msg.type === 'job_progress') {
benchState.error = null
}
// Mirror ComfyUI's push-event model so any widget listening on app.api
// (executed / execution_success / execution_error) keeps working.
if (msg.type === 'execution_start') {
benchHub.dispatch('execution_start', { prompt_id: benchState.jobId })
} else if (msg.type === 'execution_success') {
benchHub.dispatch('executed', {
prompt_id: benchState.jobId,
output: msg.outputs,
})
benchHub.dispatch('execution_success', { prompt_id: benchState.jobId })
} else if (msg.type === 'execution_error') {
benchHub.dispatch('execution_error', {
prompt_id: benchState.jobId,
exception_message: benchState.error,
})
}
notify()
}
let socket: WebSocket | null = null
let reconnectTimer: number | null = null
let pingTimer: number | null = null
export function connectBenchSocket(): void {
if (typeof window === 'undefined') return
const proto = window.location.protocol === 'https:' ? 'wss' : 'ws'
const url = `${proto}://${window.location.host}/api/ws`
const open = () => {
socket = new WebSocket(url)
socket.onopen = () => {
benchState.connected = true
log('connected to bench server')
notify()
pingTimer = window.setInterval(() => {
try {
socket?.send('ping')
} catch {
/* ignore */
}
}, 20_000)
}
socket.onmessage = (event) => {
try {
const msg = JSON.parse(event.data as string) as BenchJobEvent
applyJobEvent(msg)
} catch {
/* non-JSON frames are keepalive noise */
}
}
socket.onclose = () => {
benchState.connected = false
socket = null
if (pingTimer) window.clearInterval(pingTimer)
notify()
reconnectTimer = window.setTimeout(open, 1500)
}
socket.onerror = () => socket?.close()
}
open()
}
export async function fetchBenchConfig(): Promise<BenchConfigInfo | null> {
try {
const res = await fetch('/api/config')
if (!res.ok) return null
const cfg = (await res.json()) as BenchConfigInfo
benchState.backend = cfg.backend
notify()
return cfg
} catch {
return null
}
}
export interface BenchRenderPayload {
track_data: unknown
overrides?: Record<string, unknown>
backend?: string
}
export async function queueBenchRender(payload: BenchRenderPayload): Promise<string> {
const res = await fetch('/api/render', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(payload),
})
if (!res.ok) {
const detail = await res.text()
throw new Error(`queue failed (${res.status}): ${detail.slice(0, 300)}`)
}
const body = (await res.json()) as { job_id: string }
benchState.jobId = body.job_id
benchState.jobStatus = 'queued'
benchState.error = null
benchState.outputs = {}
log(`queued render ${body.job_id}`)
notify()
return body.job_id
}
// ---------------------------------------------------------------------------
// The ComfyApp-shaped shim
// ---------------------------------------------------------------------------
const toastShim = {
add: (msg: { severity?: string; summary?: string; detail?: string; life?: number }) => {
const severity = (msg.severity ?? 'info').toString()
log(`toast.${severity}: ${msg.summary ?? ''} ${msg.detail ?? ''}`)
if (severity === 'error') console.error('[bench-toast]', msg.summary, msg.detail)
},
}
const dialogShim = {
prompt: async (opts: { title?: string; message?: string }) =>
window.prompt(opts?.title ?? '', opts?.message ?? ''),
confirm: async (opts: { title?: string; message?: string }) =>
window.confirm(`${opts?.title ?? ''}\n${opts?.message ?? ''}`),
}
export const benchApp = {
api: {
clientId: 'bench-client',
fetchApi: (path: string, options?: RequestInit) => fetch(path, options),
addEventListener: (_type: string, _cb: EventListener) => {},
removeEventListener: (_type: string, _cb: EventListener) => {},
addCustomEventListener: (type: string, cb: CustomEventCallback) => benchHub.add(type, cb),
removeCustomEventListener: (type: string, cb: CustomEventCallback) => benchHub.remove(type, cb),
queuePrompt: async () => {
throw new Error('queuePrompt is not used by the bench UI; use the render dock')
},
dispatchEvent: (ev: Event) => benchHub.dispatch(ev.type, ev),
},
extensionManager: {
toast: toastShim,
dialog: dialogShim,
workflow: {
activeWorkflow: { key: 'bench' },
isBusy: false,
$subscribe: (_cb: unknown) => () => {},
},
},
ui: {
settings: {
settingsValues: {
'Comfy.Locale': 'en',
'Comfy.VueNodes.Enabled': false,
},
},
},
canvas: null,
graph: {
getNodeById: () => null,
links: {},
},
queuePrompt: async () => {
throw new Error('queuePrompt is not used by the bench UI; use the render dock')
},
processingQueue: false,
} as unknown as ComfyApp
// comfy-api.ts reads this at module scope — must exist before that import runs.
;(globalThis as Record<string, unknown>).comfyAPI = { app: { app: benchApp } }
/** Structural stand-in for the LGraphNode the widget normally lives in. */
export function createBenchNode(): Record<string, unknown> {
return {
id: 81,
title: 'easy multiTrackEditor (bench)',
type: 'easy multiTrackEditor',
properties: { easyMediaTimelineHeight: 620 },
widgets: [],
widgets_values: [],
inputs: [],
outputs: [],
size: [1280, 640],
pos: [0, 0],
flags: {},
mode: 0,
order: 0,
setDirtyCanvas: () => {},
setSize: () => {},
onResize: () => {},
onNodeCreated: () => {},
addDOMWidget: () => ({
name: 'track_data',
type: 'react-widget',
element: document.createElement('div'),
inputEl: null,
width: undefined,
getValue: () => '',
setValue: () => {},
serializeValue: () => '',
options: {},
}),
}
}
export function createBenchWidget(): Record<string, unknown> {
return {
name: 'track_data',
width: undefined,
inputEl: null,
options: {},
getValue: () => '',
setValue: () => {},
}
}

View File

@ -0,0 +1,49 @@
/**
* Bench UI entry point.
*
* IMPORTANT: `./bench-runtime` must be evaluated first — it installs the
* `globalThis.comfyAPI` shim that `@/lib/comfy-api` reads at module scope.
*/
import './bench-runtime'
declare const __COMFY_EASY_MEDIA_GLOBAL_CSS__: string
import { createRoot } from 'react-dom/client'
import { BenchApp } from './BenchApp'
function injectGlobalCss(): void {
if (document.getElementById('mmnode-bench-globals')) return
const style = document.createElement('style')
style.id = 'mmnode-bench-globals'
style.textContent = __COMFY_EASY_MEDIA_GLOBAL_CSS__
document.head.appendChild(style)
}
function main(): void {
injectGlobalCss()
const host = document.getElementById('root')
if (!host) throw new Error('bench: missing #root host element')
// Surface fatal boot errors inside the page itself (sidecar windows have no
// devtools console by default).
const showFatal = (title: string, detail: string) => {
host.innerHTML = ''
const pre = document.createElement('pre')
pre.style.cssText = 'color:#f87171;padding:16px;white-space:pre-wrap;font:12px/1.5 monospace'
pre.textContent = `${title}\n\n${detail}`
host.appendChild(pre)
}
window.addEventListener('error', (e) => {
if (!host.childElementCount || host.querySelector('pre')) {
showFatal('Uncaught error', `${e.message}\n${e.error?.stack ?? ''}`)
}
})
window.addEventListener('unhandledrejection', (e) => {
showFatal('Unhandled rejection', `${e.reason?.stack || e.reason}`)
})
const root = createRoot(host)
root.render(<BenchApp />)
}
main()

View File

@ -19,6 +19,7 @@ SUBDIR_HINTS = {
"vae": "models/vae", "vae": "models/vae",
"audio_vae": "models/audio_vae", "audio_vae": "models/audio_vae",
"clip": "models/clip", "clip": "models/clip",
"loras": "models/loras",
} }
@ -52,11 +53,18 @@ def _resolve(subdir, name):
return path return path
# (path, sd, metadata) cache for the last parsed LoRA, mirroring the native
# LoraLoader. Kept at module level on purpose: newer ComfyUI builds lock node
# classes after registration, so writing class attributes on the node class
# itself raises "Cannot modify class attribute ... on locked class".
_loaded_lora = None
def _auto_pick(subdir): def _auto_pick(subdir):
"""Return the model file to use for a hidden picker. """Return the model file to use for a hidden picker.
End users only see the diffusion model widget; every auxiliary model End users only see the diffusion model widget; every auxiliary model
(video VAE, audio VAE, CLIP) is auto-selected from the pack's (video VAE, audio VAE, CLIP, LoRA) is auto-selected from the pack's
models/<subdir> folder - the first file in sorted order, which for the models/<subdir> folder - the first file in sorted order, which for the
published pack is exactly the bundled file. A missing file keeps the published pack is exactly the bundled file. A missing file keeps the
loud, actionable error from _resolve. loud, actionable error from _resolve.
@ -77,6 +85,14 @@ def _auto_pick(subdir):
return selected return selected
def _auto_lora():
"""Bundled LoRA file, or ``None`` when models/loras is empty."""
files = _list_files("loras")
if files[0].startswith("<no files found"):
return None
return files[0]
def _h3_clip_type(): def _h3_clip_type():
"""MiniMax H3 CLIP recipe. The Moxie model set is H3-only, so the text """MiniMax H3 CLIP recipe. The Moxie model set is H3-only, so the text
encoder type is intentionally NOT a user-facing choice.""" encoder type is intentionally NOT a user-facing choice."""
@ -94,12 +110,12 @@ class MoxieMultimediaLoader(io.ComfyNode):
"""All-in-one loader that looks like it loads a single model. """All-in-one loader that looks like it loads a single model.
The only visible picker is the diffusion model; the attention backend is The only visible picker is the diffusion model; the attention backend is
applied to it, and the video VAE, the audio VAE and the CLIP are applied to it, the bundled LoRA (if any), the video VAE, the audio VAE
auto-selected from the node pack's models/ folder. and the CLIP are auto-selected from the node pack's models/ folder.
The model set is MiniMax H3-only: the CLIP recipe is locked to The model set is MiniMax H3-only: the CLIP recipe is locked to
CLIPType.MINIMAX and outputs are exactly (model, clip, video_vae, audio_vae) CLIPType.MINIMAX and outputs are exactly (model, clip, video_vae, audio_vae)
with the attention backend already applied to the model.""" with the attention backend and LoRA already applied to the model."""
@classmethod @classmethod
def define_schema(cls) -> io.Schema: def define_schema(cls) -> io.Schema:
@ -107,7 +123,7 @@ class MoxieMultimediaLoader(io.ComfyNode):
node_id="MoxieMultimediaLoader", node_id="MoxieMultimediaLoader",
display_name="Moxie Multimedia Loader", display_name="Moxie Multimedia Loader",
category="loaders/moxie", category="loaders/moxie",
description="Loads the Moxie diffusion model (with the attention backend " description="Loads the Moxie diffusion model (with the attention backend and the bundled LoRA "
"already applied) plus the video VAE, audio VAE and text encoder that ship with " "already applied) plus the video VAE, audio VAE and text encoder that ship with "
"this custom node - one model picker, everything else is handled automatically. " "this custom node - one model picker, everything else is handled automatically. "
"Outputs model, clip, video_vae and audio_vae.", "Outputs model, clip, video_vae and audio_vae.",
@ -132,8 +148,8 @@ class MoxieMultimediaLoader(io.ComfyNode):
), ),
], ],
outputs=[ outputs=[
io.Model.Output(display_name="model", tooltip="Diffusion model with attention backend applied."), io.Model.Output(display_name="model", tooltip="Diffusion model with attention backend and LoRA applied."),
io.Clip.Output(display_name="clip", tooltip="CLIP text encoder."), io.Clip.Output(display_name="clip", tooltip="CLIP text encoder (with LoRA applied if present)."),
io.Vae.Output(display_name="video_vae", tooltip="Loaded video/image VAE."), io.Vae.Output(display_name="video_vae", tooltip="Loaded video/image VAE."),
io.Vae.Output(display_name="audio_vae", tooltip="Loaded audio VAE."), io.Vae.Output(display_name="audio_vae", tooltip="Loaded audio VAE."),
], ],
@ -173,6 +189,36 @@ class MoxieMultimediaLoader(io.ComfyNode):
m.set_model_optimized_attention(attention_function) m.set_model_optimized_attention(attention_function)
return m return m
@classmethod
def _apply_lora(cls, model, clip, lora_name, strength_model, strength_clip):
global _loaded_lora
if (
lora_name is None
or lora_name == "none"
or lora_name.startswith("<no files found")
or (strength_model == 0 and strength_clip == 0)
):
return model, clip
lora_path = _resolve("loras", lora_name)
lora = None
lora_metadata = None
if _loaded_lora is not None:
if _loaded_lora[0] == lora_path:
lora = _loaded_lora[1]
lora_metadata = _loaded_lora[2] if len(_loaded_lora) > 2 else None
else:
_loaded_lora = None
if lora is None:
lora, lora_metadata = comfy.utils.load_torch_file(lora_path, safe_load=True, return_metadata=True)
_loaded_lora = (lora_path, lora, lora_metadata)
model_lora, clip_lora = comfy.sd.load_lora_for_models(
model, clip, lora, strength_model, strength_clip, lora_metadata=lora_metadata
)
return model_lora, clip_lora
@classmethod @classmethod
def _load_vae(cls, vae_name, subdir="vae"): def _load_vae(cls, vae_name, subdir="vae"):
path = _resolve(subdir, vae_name) path = _resolve(subdir, vae_name)
@ -210,13 +256,15 @@ class MoxieMultimediaLoader(io.ComfyNode):
weight_dtype, weight_dtype,
attention_backend, attention_backend,
) -> io.NodeOutput: ) -> io.NodeOutput:
# Chain: Load Diffusion Model -> Model Attention Backend # Chain: Load Diffusion Model -> Model Attention Backend -> Lora Loader (needs both model and clip)
model = cls._load_diffusion_model(diffusion_model, weight_dtype) model = cls._load_diffusion_model(diffusion_model, weight_dtype)
model = cls._apply_attention_backend(model, attention_backend) model = cls._apply_attention_backend(model, attention_backend)
# Hidden pickers: auxiliary models are auto-selected from the pack's # Hidden pickers: auxiliary models are auto-selected from the pack's
# models/ folder so end users only ever see the diffusion model. # models/ folder so end users only ever see the diffusion model.
clip_obj = cls._load_clip(_auto_pick("clip"), "default") clip_obj = cls._load_clip(_auto_pick("clip"), "default")
lora_name = _auto_lora()
model, clip_obj = cls._apply_lora(model, clip_obj, lora_name, 1.0, 1.0)
video_vae_obj = cls._load_vae(_auto_pick("vae"), "vae") video_vae_obj = cls._load_vae(_auto_pick("vae"), "vae")
audio_vae_obj = cls._load_vae(_auto_pick("audio_vae"), "audio_vae") audio_vae_obj = cls._load_vae(_auto_pick("audio_vae"), "audio_vae")

View File

@ -1006,7 +1006,7 @@ class ModelPreviewOverrideKJ(io.ComfyNode):
inputs=[ inputs=[
io.Model.Input( io.Model.Input(
"model", "model",
tooltip="Diffusion model (with attention backend already applied).", tooltip="Diffusion model (with attention backend and LoRA already applied).",
), ),
io.Int.Input( io.Int.Input(
"max_resolution", "max_resolution",

View File

@ -5,8 +5,6 @@ from comfy_api.latest import io
from .nodes.project import ( from .nodes.project import (
EasyMultiTrackProject, EasyMultiTrackProject,
H3_CONTEXT_LENGTH_OPTIONS,
H3_CONTEXT_SOURCE_FRAMES,
TYPE_TRACKS_INFO, TYPE_TRACKS_INFO,
_first_input, _first_input,
) )
@ -54,7 +52,7 @@ class MoxieProject(EasyMultiTrackProject):
not_idempotent=True, not_idempotent=True,
inputs=[ inputs=[
TYPE_TRACKS_INFO.Input("tracks_info"), TYPE_TRACKS_INFO.Input("tracks_info"),
io.Model.Input("model", tooltip="Diffusion model from Moxie Multimedia Loader (attention backend already applied)."), io.Model.Input("model", tooltip="Diffusion model from Moxie Multimedia Loader (attention backend and LoRA already applied)."),
io.Clip.Input("clip", tooltip="CLIP text encoder from Moxie Multimedia Loader."), io.Clip.Input("clip", tooltip="CLIP text encoder from Moxie Multimedia Loader."),
io.Vae.Input("video_vae", tooltip="Video VAE from Moxie Multimedia Loader."), io.Vae.Input("video_vae", tooltip="Video VAE from Moxie Multimedia Loader."),
io.Vae.Input("audio_vae", tooltip="Audio VAE from Moxie Multimedia Loader (required to decode Moxie audio)."), io.Vae.Input("audio_vae", tooltip="Audio VAE from Moxie Multimedia Loader (required to decode Moxie audio)."),
@ -65,21 +63,8 @@ class MoxieProject(EasyMultiTrackProject):
default=3, default=3,
min=1, min=1,
max=10000, max=10000,
tooltip="Sampling steps for the beta scheduler (3 for the baked-in 3-step distill; " tooltip="Sampling steps for the beta scheduler (3 for the 3-step turbo LoRA; "
"8 for busy motion, 6 low motion, 4 runs/no dialogue).", "8 for busy motion with regular LoRAs, 6 low motion, 4 runs/no dialogue).",
),
io.Combo.Input(
"context_frames",
options=[str(option) for option in H3_CONTEXT_LENGTH_OPTIONS],
default=str(H3_CONTEXT_SOURCE_FRAMES),
tooltip=(
"Video frames copied from the previous segment as "
"continuation context for segments whose continuity "
"mode is Context or Context Swap. Longer values hand "
"off motion more smoothly but generate that many extra "
"frames per segment. Values are H3 temporal-grid "
"lengths shared with the Moxie context sampler nodes."
),
), ),
io.Int.Input( io.Int.Input(
"seed", "seed",

View File

@ -57,25 +57,6 @@ from ..utils.video import passthrough_video_media
CATEGORY_MINIMAX = "EasyUse/MiniMax" CATEGORY_MINIMAX = "EasyUse/MiniMax"
CANVAS_MULTIPLE = 32 CANVAS_MULTIPLE = 32
def _align_canvas_dimension(value: Any) -> int:
"""Floor a canvas side onto the H3 32px grid (CANVAS_MULTIPLE).
The video VAE latent grid is pixels // 16 and the diffusion model
patchifies that latent with a 2x2 patch, so every canvas side must be a
multiple of 32. Anything else produces an odd latent dimension and crashes
comfy's patchify_video reshape ("shape '[1, 24, ...]' is invalid for input
of size N"). Floors instead of rounds so alignment never upsamples past
the requested size.
"""
try:
number = int(value)
except (TypeError, ValueError):
return CANVAS_MULTIPLE
return max(CANVAS_MULTIPLE, number // CANVAS_MULTIPLE * CANVAS_MULTIPLE)
BASE_SHORT_EDGE = 768 BASE_SHORT_EDGE = 768
MAX_PIXELS = 768 * 1344 MAX_PIXELS = 768 * 1344
REF_IMAGE_SHORT_EDGE = 2048 REF_IMAGE_SHORT_EDGE = 2048
@ -508,8 +489,6 @@ class MiniMaxH3ImageToVideoFallback(io.ComfyNode):
first_frame: torch.Tensor | None = None, first_frame: torch.Tensor | None = None,
last_frame: torch.Tensor | None = None, last_frame: torch.Tensor | None = None,
) -> io.NodeOutput: ) -> io.NodeOutput:
width = _align_canvas_dimension(width)
height = _align_canvas_dimension(height)
latent, frame_count = _empty_av_latent(width, height, length) latent, frame_count = _empty_av_latent(width, height, length)
token_images: list[torch.Tensor] = [] token_images: list[torch.Tensor] = []
keyframes: list[dict[str, Any]] = [] keyframes: list[dict[str, Any]] = []
@ -638,8 +617,6 @@ class MiniMaxH3ReferenceToVideoFallback(io.ComfyNode):
ref_video_audios: dict[str, dict[str, Any] | None] | None = None, ref_video_audios: dict[str, dict[str, Any] | None] | None = None,
ref_audios: dict[str, dict[str, Any]] | None = None, ref_audios: dict[str, dict[str, Any]] | None = None,
) -> io.NodeOutput: ) -> io.NodeOutput:
width = _align_canvas_dimension(width)
height = _align_canvas_dimension(height)
latent, frame_count = _empty_av_latent(width, height, length) latent, frame_count = _empty_av_latent(width, height, length)
reference_items: list[dict[str, Any]] = [] reference_items: list[dict[str, Any]] = []
reference_blocks: list[dict[str, Any]] = [] reference_blocks: list[dict[str, Any]] = []
@ -685,20 +662,6 @@ class MiniMaxH3ReferenceToVideoFallback(io.ComfyNode):
continue continue
soundtrack = video_audios.get("ref_video_audio_" + name.rsplit("_", 1)[-1]) soundtrack = video_audios.get("ref_video_audio_" + name.rsplit("_", 1)[-1])
frames = _prepare_reference_video_frames(frames, frame_count) frames = _prepare_reference_video_frames(frames, frame_count)
# Grid-align the reference video canvas: comfy core's
# patchify_video floors each latent side into 2x2 patches, so an
# off-grid frame size (API callers can inject arbitrary media)
# would produce an odd latent dimension and crash sampling with
# "shape '[1, 24, ...]' is invalid for input of size N". ComfyUI
# builds with native H3 nodes do this alignment inside core's
# MiniMaxH3ReferenceToVideo; this fallback copy must match.
ref_height, ref_width = int(frames.shape[1]), int(frames.shape[2])
aligned_ref_width = _align_canvas_dimension(ref_width)
aligned_ref_height = _align_canvas_dimension(ref_height)
if (aligned_ref_width, aligned_ref_height) != (ref_width, ref_height):
frames = _resize(
frames, aligned_ref_width, aligned_ref_height, "disabled"
)
aligned_count = frames.shape[0] aligned_count = frames.shape[0]
canvas_height, canvas_width = frames.shape[1], frames.shape[2] canvas_height, canvas_width = frames.shape[1], frames.shape[2]
if aligned_count < 5: if aligned_count < 5:
@ -2779,8 +2742,8 @@ class EasyMiniMaxH3ToVideo(io.ComfyNode):
selected_vae = _first_input(vae) selected_vae = _first_input(vae)
selected_audio_vae = _first_input(audio_vae) selected_audio_vae = _first_input(audio_vae)
prompt_text = str(_first_input(prompt, "")) prompt_text = str(_first_input(prompt, ""))
target_width = _align_canvas_dimension(_first_input(width, 1344)) target_width = int(_first_input(width, 1344))
target_height = _align_canvas_dimension(_first_input(height, 768)) target_height = int(_first_input(height, 768))
target_length = int(_first_input(length, 124)) target_length = int(_first_input(length, 124))
locked_timing_frames = int(_first_input(locked_video_timing_frames, 0)) locked_timing_frames = int(_first_input(locked_video_timing_frames, 0))
expanded_images = expand_image_inputs(images) expanded_images = expand_image_inputs(images)

View File

@ -46,7 +46,6 @@ from ..utils.multitrack import (
MULTITRACK_RUNTIME_CACHE_KEY, MULTITRACK_RUNTIME_CACHE_KEY,
multitrack_runtime_cache, multitrack_runtime_cache,
) )
from .minimax import CANVAS_MULTIPLE, _align_canvas_dimension
TYPE_FAST_MODEL_LOADER = io.Custom(io_type="FAST_MODEL_LOADER") TYPE_FAST_MODEL_LOADER = io.Custom(io_type="FAST_MODEL_LOADER")
@ -55,12 +54,6 @@ TYPE_PROJECT_DATA = io.Custom(io_type="PROJECT_DATA")
TYPE_H3_PROJECT_STATIC_DATA = io.Custom(io_type="H3_PROJECT_STATIC_DATA") TYPE_H3_PROJECT_STATIC_DATA = io.Custom(io_type="H3_PROJECT_STATIC_DATA")
H3_CONTEXT_CONTINUITY_MODES = {"context", "context_swap"} H3_CONTEXT_CONTINUITY_MODES = {"context", "context_swap"}
H3_CONTEXT_SOURCE_FRAMES = 22 H3_CONTEXT_SOURCE_FRAMES = 22
# Context lengths supported by the H3 continuation sampler nodes; every
# value lands on the H3 temporal latent grid (17k + 5 frames).
H3_CONTEXT_LENGTH_OPTIONS = (5, 22, 39, 56)
# Extra frames generated past the context prefix so the decoded clip ends
# on a phase-aligned H3 grid boundary before the tail is trimmed.
H3_CONTEXT_GRID_TAIL_FRAMES = 12
def _first_input(value: Any, default: Any = None) -> Any: def _first_input(value: Any, default: Any = None) -> Any:
@ -69,18 +62,6 @@ def _first_input(value: Any, default: Any = None) -> Any:
return value if value is not None else default return value if value is not None else default
def _h3_context_frame_count(value: Any) -> int:
"""Resolve the context_frames widget to a supported H3 context length."""
frames = int(_first_input(value, str(H3_CONTEXT_SOURCE_FRAMES)))
if frames not in H3_CONTEXT_LENGTH_OPTIONS:
options = ", ".join(str(option) for option in H3_CONTEXT_LENGTH_OPTIONS)
raise ValueError(
"context_frames must be one of the supported H3 context lengths: "
f"{options} (got {frames})"
)
return frames
def _raw_project_input(value: Any) -> Any: def _raw_project_input(value: Any) -> Any:
"""Unwrap list inputs without splitting a raw graph link.""" """Unwrap list inputs without splitting a raw graph link."""
if is_link(value): if is_link(value):
@ -949,19 +930,6 @@ class EasyMultiTrackProject(io.ComfyNode):
"mode preserves existing video and latent files." "mode preserves existing video and latent files."
), ),
), ),
io.Combo.Input(
"context_frames",
options=[str(option) for option in H3_CONTEXT_LENGTH_OPTIONS],
default=str(H3_CONTEXT_SOURCE_FRAMES),
tooltip=(
"Video frames copied from the previous segment as "
"continuation context for segments whose continuity "
"mode is Context or Context Swap. Longer values hand "
"off motion more smoothly but generate that many extra "
"frames per segment. Values are H3 temporal-grid "
"lengths shared with the Moxie context sampler nodes."
),
),
io.Int.Input( io.Int.Input(
"seed", "seed",
default=42, default=42,
@ -1111,19 +1079,6 @@ class EasyMultiTrackProject(io.ComfyNode):
if not uses_linked_prepare: if not uses_linked_prepare:
_require_minimax_h3_model(model) _require_minimax_h3_model(model)
info = parse_tracks_info(kwargs.get("tracks_info")) info = parse_tracks_info(kwargs.get("tracks_info"))
# External app clients (the Moxie app drives this node through the
# ComfyUI API) can inject project dimensions that are not on the H3
# 32px canvas grid; the MultiTrack Editor UI only offers grid sizes.
# Every latent side is pixels // 16 and the diffusion model patchifies
# that latent 2x2, so any off-grid side eventually reaches comfy's
# patchify_video as an odd latent dimension and crashes first sampling
# with "shape '[1, 24, ...]' is invalid for input of size N". Floor
# both sides onto the grid once, here, so media loading, reference
# encoding, both sampling passes and the exported project all agree
# on the same aligned canvas. Editor-selected sizes are already
# multiples of 32, so this is a no-op for UI workflows.
info["width"] = _align_canvas_dimension(info["width"])
info["height"] = _align_canvas_dimension(info["height"])
hidden_inputs = getattr(cls, "hidden", None) hidden_inputs = getattr(cls, "hidden", None)
validate_h3_project_outputs( validate_h3_project_outputs(
info, info,
@ -1135,9 +1090,6 @@ class EasyMultiTrackProject(io.ComfyNode):
sampling_mode, sampling_config = _h3_sampling_mode_config( sampling_mode, sampling_config = _h3_sampling_mode_config(
kwargs.get("sampling_mode") kwargs.get("sampling_mode")
) )
context_source_frames = _h3_context_frame_count(
kwargs.get("context_frames")
)
is_passthrough = sampling_mode == "passthrough" is_passthrough = sampling_mode == "passthrough"
is_selflift = sampling_mode == "selflift" is_selflift = sampling_mode == "selflift"
has_second_pass = sampling_mode == "dual" has_second_pass = sampling_mode == "dual"
@ -1229,23 +1181,8 @@ class EasyMultiTrackProject(io.ComfyNode):
# keeps the editor size exactly as configured. # keeps the editor size exactly as configured.
first_pass_width, first_pass_height = editor_width, editor_height first_pass_width, first_pass_height = editor_width, editor_height
if sampling_mode == "single" and not audio_only: if sampling_mode == "single" and not audio_only:
# The half-resolution canvas must stay on the H3 32px grid half_width = max(32, editor_width // 2)
# (CANVAS_MULTIPLE): the video VAE downsamples by 16 and the half_height = max(32, editor_height // 2)
# diffusion model patchifies the latent with a 2x2 patch, so a
# first-pass side that is not a multiple of 32 yields an odd
# latent dimension and crashes comfy's patchify_video with
# "shape '[1, 24, 1, 1, ...]' is invalid for input of size N"
# (e.g. a 544x960 editor project halved to 272x480 produces a
# 17x30 latent). Floor to the grid instead of rounding so the
# first pass never upsamples past half the editor resolution.
half_width = max(
32,
(editor_width // 2) // CANVAS_MULTIPLE * CANVAS_MULTIPLE,
)
half_height = max(
32,
(editor_height // 2) // CANVAS_MULTIPLE * CANVAS_MULTIPLE,
)
if (half_width, half_height) != (32, 32): if (half_width, half_height) != (32, 32):
first_pass_width, first_pass_height = half_width, half_height first_pass_width, first_pass_height = half_width, half_height
target_width, target_height = h3_second_pass_dimensions( target_width, target_height = h3_second_pass_dimensions(
@ -1273,19 +1210,8 @@ class EasyMultiTrackProject(io.ComfyNode):
segment_count, segment_count,
) )
if not selected_entries: if not selected_entries:
if not all_entries:
raise ValueError(
"No H3 task segments are available: the MultiTrack timeline "
"does not contain any task-track segments. Add at least one "
"task segment in the MultiTrack Editor and run the project "
"again."
)
raise ValueError( raise ValueError(
"No H3 task segments are available from segment_start_number=" "No H3 task segments are available from segment_start_number."
f"{segment_start_number} (segment_count={segment_count}): the "
f"timeline only contains {len(all_entries)} task segment(s), "
"numbered from 1. Lower segment_start_number or add more task "
"segments in the MultiTrack Editor."
) )
first_selected_index, first_selected_entry = selected_entries[0] first_selected_index, first_selected_entry = selected_entries[0]
@ -1576,13 +1502,12 @@ class EasyMultiTrackProject(io.ComfyNode):
context_latent = _h3_encode_context_media( context_latent = _h3_encode_context_media(
graph, output_images, output_audio, vae, audio_vae, graph, output_images, output_audio, vae, audio_vae,
f"passthrough_context_{task_index}", f"passthrough_context_{task_index}",
context_frames=context_source_frames,
) )
runtime_context_latent = graph.node( runtime_context_latent = graph.node(
"easy h3MotionContextLatentTrim", "easy h3MotionContextLatentTrim",
id=f"trim_hires_context_latent_{task_index}", id=f"trim_hires_context_latent_{task_index}",
latent=context_latent, latent=context_latent,
context_length=str(context_source_frames), context_length=str(H3_CONTEXT_SOURCE_FRAMES),
).out(0) ).out(0)
saved_video = graph.node( saved_video = graph.node(
"easy saveVideo", "easy saveVideo",
@ -1643,9 +1568,8 @@ class EasyMultiTrackProject(io.ComfyNode):
uses_context uses_context
and (previous_hires_context_latent is not None or task_index > 0) and (previous_hires_context_latent is not None or task_index > 0)
) )
context_generation_frames = ( context_source_frames = H3_CONTEXT_SOURCE_FRAMES
context_source_frames + H3_CONTEXT_GRID_TAIL_FRAMES context_generation_frames = 34
)
if will_have_context_continuity: if will_have_context_continuity:
task_length = graph.node( task_length = graph.node(
"ComfyMathExpression", "ComfyMathExpression",
@ -2002,7 +1926,7 @@ class EasyMultiTrackProject(io.ComfyNode):
id=f"hires_continuity_{task_index}", id=f"hires_continuity_{task_index}",
current_hires_latent=upscaled_latent, current_hires_latent=upscaled_latent,
previous_hires_latent=previous_hires_context_latent, previous_hires_latent=previous_hires_context_latent,
context_length=str(context_source_frames), context_length=str(H3_CONTEXT_SOURCE_FRAMES),
video_transition_steps=4, video_transition_steps=4,
video_anchor_only=not audio_only, video_anchor_only=not audio_only,
) )
@ -2192,7 +2116,6 @@ class EasyMultiTrackProject(io.ComfyNode):
vae, vae,
audio_vae, audio_vae,
f"hires_context_{task_index}", f"hires_context_{task_index}",
context_frames=context_source_frames,
) )
hires_context_reencoded = True hires_context_reencoded = True
if (has_second_pass and run_second_pass) or ( if (has_second_pass and run_second_pass) or (
@ -2236,7 +2159,6 @@ class EasyMultiTrackProject(io.ComfyNode):
vae, vae,
audio_vae, audio_vae,
f"low_context_{task_index}", f"low_context_{task_index}",
context_frames=context_source_frames,
) )
low_context_reencoded = True low_context_reencoded = True
else: else:

View File

@ -1,59 +0,0 @@
# Moxie Multimedia Suite - Python dependencies
#
# ComfyUI-Manager installs this file automatically.
# If you cloned the pack manually, install it yourself - see README.md
# ("Installing dependencies") for the ComfyUI Portable command.
#
# This pack targets NVIDIA RTX 3060+ GPUs only, so every feature
# dependency ships as mandatory - speech, subtitle recognition, and the
# RTX Video Super Resolution upscale are all installed up front. The
# list is ordered light -> heavy: if a large package ever fails to
# install, everything above it is already in place and re-running the
# command resumes.
#
# Notes:
# - Core stack (torch, numpy, Pillow, PyYAML, aiohttp) comes with
# ComfyUI itself.
# - The two pip options below route nvidia-vfx to NVIDIA's own package
# index: PyPI only hosts a stub sdist for nvidia-vfx whose build
# backend (wheel_stub) fails on some setups, while pypi.nvidia.com
# hosts the real prebuilt wheels (Windows + Linux, python 3.10-3.13).
# --only-binary=nvidia-vfx guarantees the stub sdist is never built.
# - qwen-asr pins transformers==4.57.6 / accelerate==1.12.0, so pip may
# upgrade those two packages inside your ComfyUI environment.
# - voxcpm needs torch>=2.5.0 and torchcodec (wheels exist for Windows
# and Linux, python 3.10-3.14); keep ComfyUI reasonably current.
# - ffmpeg-python drives the FFmpeg binary - keep FFmpeg in PATH.
--extra-index-url https://pypi.nvidia.com/
--only-binary=nvidia-vfx
# Audio: reading/writing merged and generated audio
soundfile
# Subtitles: CJK-aware smart line splitting
jieba
# Model downloads from the media browser / helper routes
huggingface_hub
# Smart split (shot detection) - ffmpeg CLI wrapper
ffmpeg-python
# Smart split (shot detection helpers)
opencv-python
# Smart split (dataset loaders)
imageio
# RTX Video Super Resolution upscale (NVIDIA RTX GPUs only)
nvidia-vfx
# Subtitle recognition (local Whisper)
openai-whisper
# Subtitle recognition (Qwen ASR) - pins transformers==4.57.6
qwen-asr
# Speech / voiceover generation (VoxCPM)
voxcpm

View File

@ -0,0 +1,160 @@
#!/usr/bin/env python3
"""Standalone bench verification: API + WebSocket + mock pipeline end-to-end.
No ComfyUI, no torch. Exercises the FastAPI app exactly like a sidecar host
would: health/config, workflow & plan endpoints, a full render over the mock
backend (ffmpeg), WebSocket progress events, and output serving.
Run: python3 scripts/test_bench_pipeline.py
"""
from __future__ import annotations
import json
import os
import sys
import time
from pathlib import Path
ROOT = Path(__file__).resolve().parent.parent
BENCH = ROOT / "bench"
sys.path.insert(0, str(BENCH))
FAILED = []
def check(name: str, cond: bool, detail: str = "") -> None:
print(f"[{'PASS' if cond else 'FAIL'}] {name}" + (f" - {detail}" if detail and not cond else ""))
if not cond:
FAILED.append(name)
def main() -> int:
# py_compile every bench module first
import py_compile
py_files = sorted(BENCH.rglob("*.py"))
for f in py_files:
py_compile.compile(str(f), doraise=True)
check("py_compile all bench modules", True, f"({len(py_files)} files)")
from fastapi.testclient import TestClient
os.environ.setdefault("MOXIE_BENCH_WORK", str(BENCH / "work"))
from app.config import BenchConfig
from app.server import create_app
cfg = BenchConfig(backend="mock")
cfg.repo_root = ROOT
app = create_app(cfg)
with TestClient(app) as client: # context manager runs lifespan
r = client.get("/api/health")
check("GET /api/health", r.status_code == 200 and r.json()["ok"] is True)
check("health reports mock backend", r.json()["backend"] == "mock")
r = client.get("/api/config")
check("GET /api/config", r.status_code == 200 and "backends" in r.json())
r = client.get("/api/workflow")
body = r.json()
check("GET /api/workflow summary", r.status_code == 200
and body["summary"]["editor_resolution"] == {"width": 1920, "height": 1080})
track_data = json.loads(body["track_data"])
check("workflow carries editor track_data", track_data.get("total_length") == 240)
r = client.get("/api/plan")
plan = r.json()
check("GET /api/plan", r.status_code == 200
and [s["prompt"] for s in plan["segments"]] == ["scene1", "scene2"])
check("plan duration", plan["duration_seconds"] == 10.0, str(plan["duration_seconds"]))
# --- WebSocket + render e2e (small dims to keep it fast) ---
events = []
with client.websocket_connect("/api/ws") as ws:
# drain the initial connection by starting a render in parallel
render_payload = {
"track_data": body["track_data"],
"overrides": {
"width": 256, "height": 144, "frame_rate": 8,
"steps": 4, "seed": 42, "project_name": "MM-bench-e2e",
},
}
r = client.post("/api/render", json=render_payload)
check("POST /api/render accepted", r.status_code == 200, r.text[:200])
job_id = r.json()["job_id"]
deadline = time.time() + 120
got_success = False
while time.time() < deadline and not got_success:
msg = ws.receive_json()
events.append(msg)
if msg.get("job_id") != job_id:
continue
if msg.get("type") == "job_progress":
check("progress within pipeline stages", True) if len(events) == 2 else None
if msg.get("type") == "execution_success":
got_success = True
if msg.get("type") == "execution_error":
break
check("render completed over WebSocket", got_success,
f"events={ [e.get('type') for e in events][-6:] }")
r = client.get(f"/api/jobs/{job_id}")
job = r.json()
check("job result stored", job["status"] == "done" and job["result"] is not None)
outputs = (job["result"] or {}).get("outputs", {})
check("both branches produced", set(outputs) == {"native", "upscaled"}, str(list(outputs)))
for branch, out in outputs.items():
path = Path(out["path"])
check(f"{branch} file exists", path.is_file() and path.stat().st_size > 1000)
expected = {"native": (256, 144), "upscaled": (1920, 1080)}[branch]
check(f"{branch} served over /api/view", client.get(out["url"]).status_code == 200)
# output dims via ffprobe
import subprocess
probe = subprocess.run(
["ffprobe", "-v", "quiet", "-print_format", "json", "-show_streams",
outputs["upscaled"]["path"]],
capture_output=True, text=True)
streams = json.loads(probe.stdout).get("streams") or []
vs = next((s for s in streams if s.get("codec_type") == "video"), {})
check("upscaled dims match RTX target",
(vs.get("width"), vs.get("height")) == (1920, 1080),
f"{vs.get('width')}x{vs.get('height')}")
vs_n = json.loads(subprocess.run(
["ffprobe", "-v", "quiet", "-print_format", "json", "-show_streams",
outputs["native"]["path"]], capture_output=True, text=True).stdout
).get("streams", [{}])
vsn = next((s for s in vs_n if s.get("codec_type") == "video"), {})
check("native dims match editor canvas",
(vsn.get("width"), vsn.get("height")) == (256, 144),
f"{vsn.get('width')}x{vsn.get('height')}")
# progress stage ordering mirrors the pipeline
stages_seen = [e.get("stage") for e in events if e.get("type") == "job_progress"]
check("progress touches sampling stage", "MoxieProject" in stages_seen)
check("progress touches upscale stage", "RTXVideoSuperResolution" in stages_seen)
# /api/easy-media mirrors answer
r = client.get("/api/easy-media/media/list")
check("easy-media media/list mirror", r.status_code == 200 and "items" in r.json())
r = client.get("/api/easy-media/prompt/system-prompts")
check("easy-media system-prompts mirror", r.status_code == 200)
# static UI mount responds (built or explicit hint)
r = client.get("/")
check("root serves UI or build hint", r.status_code in (200, 404)
or "not built" in r.text)
print()
if FAILED:
print(f"FAILED: {len(FAILED)} -> {FAILED}")
return 1
print("ALL CHECKS PASSED")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@ -1,348 +0,0 @@
"""Standalone verification for merge_video_track_with_ffmpeg failure_reason.
Runs without ComfyUI/torch: stubs the heavy imports, then exercises every
``return None`` path of merge_video_track_with_ffmpeg and the
compose_h3_project_video error propagation, asserting each produces a
specific, human-readable reason.
Usage: python scripts/test_merge_failure_reason.py
"""
from __future__ import annotations
import sys
import types
from pathlib import Path
PACKAGE_ROOT = Path(__file__).resolve().parent.parent
def _install_stubs() -> None:
# Minimal torch stub: only attributes evaluated at IMPORT time are needed
# (annotations like `torch.Tensor`, `torch.device`, `torch.dtype` in
# image.py, and `torch.nn.functional` imported by image/audio/multitrack).
# No torch function actually runs on the paths under test.
if "torch" not in sys.modules:
torch = types.ModuleType("torch")
torch.__path__ = [] # mark as package so torch.nn.functional imports resolve
nn = types.ModuleType("torch.nn")
functional = types.ModuleType("torch.nn.functional")
functional.pad = lambda *a, **k: None
nn.functional = functional
torch.nn = nn
torch.Tensor = object
torch.device = object
torch.dtype = object
torch.float32 = object()
torch.float64 = object()
sys.modules["torch"] = torch
sys.modules["torch.nn"] = nn
sys.modules["torch.nn.functional"] = functional
if "folder_paths" not in sys.modules:
fp = types.ModuleType("folder_paths")
temp_dir = PACKAGE_ROOT / ".tmp-temp"
temp_dir.mkdir(exist_ok=True)
fp.get_output_directory = lambda: str(PACKAGE_ROOT / ".tmp-output")
fp.get_temp_directory = lambda: str(PACKAGE_ROOT / ".tmp-temp")
fp.get_input_directory = lambda: ""
fp.get_filename_list = lambda *a, **k: []
fp.get_full_path = lambda *a, **k: None
sys.modules["folder_paths"] = fp
if "server" not in sys.modules:
sys.modules["server"] = types.ModuleType("server")
_install_stubs()
sys.path.insert(0, str(PACKAGE_ROOT))
# Import the REAL utils package: with the stubs above, the eager `from .image
# import *` in utils/__init__.py resolves, so multitrack's `from . import
# FFMPEG_RESIZE_METHODS, ...` also works and h3_project loads untouched.
from utils.video import merge_video_track_with_ffmpeg # noqa: E402
import utils.video as video_mod # noqa: E402
import utils.h3_project as h3_project # noqa: E402
CHECKS: list[tuple[str, bool, str]] = []
def check(name: str, condition: bool, detail: str = "") -> None:
CHECKS.append((name, bool(condition), detail))
# ---------------------------------------------------------------------------
# 1. FFmpeg not found: point PATH lookup at an empty search space.
# ---------------------------------------------------------------------------
import shutil # noqa: E402
_real_which = shutil.which
shutil.which = lambda name, **k: None
try:
reasons: list[str] = []
out = merge_video_track_with_ffmpeg(
[{"source": "whatever.mp4", "start_frame": 0, "end_frame": 10}],
10,
24.0,
64,
64,
failure_reason=reasons,
)
check("ffmpeg-missing returns None", out is None)
check(
"ffmpeg-missing reason mentions FFmpeg executable",
len(reasons) == 1 and "FFmpeg executable not found" in reasons[0],
str(reasons),
)
finally:
shutil.which = _real_which
# ---------------------------------------------------------------------------
# 2. Without failure_reason the function still returns None silently and no
# caller-visible behavior changes (backwards compatibility).
# ---------------------------------------------------------------------------
shutil.which = lambda name, **k: None
try:
out = merge_video_track_with_ffmpeg(
[{"source": "whatever.mp4", "start_frame": 0, "end_frame": 10}],
10,
24.0,
64,
64,
)
check("no-collector path still returns None", out is None)
finally:
shutil.which = _real_which
# ---------------------------------------------------------------------------
# 3. ffmpeg "found": redirect get_ffmpeg_path via monkeypatched shutil.which to
# a real executable so we reach the precondition checks.
# ---------------------------------------------------------------------------
import os # noqa: E402
fake_ffmpeg = Path(PACKAGE_ROOT / ".tmp-fake-ffmpeg")
fake_ffmpeg.write_text("")
os.chmod(fake_ffmpeg, 0o755)
shutil.which = lambda name, **k: str(fake_ffmpeg) if name == "ffmpeg" else None
try:
# Empty timeline
reasons = []
out = merge_video_track_with_ffmpeg([], 0, 24.0, 64, 64, failure_reason=reasons)
check("empty timeline returns None", out is None)
check(
"empty timeline reason",
len(reasons) == 1 and "timeline is empty" in reasons[0],
str(reasons),
)
# Invalid frame rate
reasons = []
out = merge_video_track_with_ffmpeg(
[{"source": "x.mp4", "start_frame": 0, "end_frame": 1}],
1,
0.0,
64,
64,
failure_reason=reasons,
)
check("invalid fps returns None", out is None)
check("invalid fps reason", len(reasons) == 1 and "frame rate" in reasons[0], str(reasons))
# Odd dimensions
reasons = []
out = merge_video_track_with_ffmpeg(
[{"source": "x.mp4", "start_frame": 0, "end_frame": 1}],
1,
24.0,
63,
64,
failure_reason=reasons,
)
check("odd width returns None", out is None)
check(
"odd width reason names size",
len(reasons) == 1 and "63x64" in reasons[0] and "odd" in reasons[0],
str(reasons),
)
# Missing segment sources
reasons = []
out = merge_video_track_with_ffmpeg(
[
{"source": "/definitely/not/here/a.mp4", "start_frame": 0, "end_frame": 4},
{"source": "/definitely/not/here/b.mp4", "start_frame": 4, "end_frame": 8},
{"source": "/definitely/not/here/c.mp4", "start_frame": 8, "end_frame": 12},
],
12,
24.0,
64,
64,
failure_reason=reasons,
)
check("missing sources returns None", out is None)
check(
"missing sources reason lists each file",
len(reasons) == 1
and "a.mp4" in reasons[0]
and "b.mp4" in reasons[0]
and "c.mp4" in reasons[0]
and "missing" in reasons[0],
str(reasons),
)
# Missing locked audio
reasons = []
out = merge_video_track_with_ffmpeg(
[
{
"source": str(fake_ffmpeg),
"audio_source": "/definitely/not/here/locked.wav",
"start_frame": 0,
"end_frame": 4,
}
],
4,
24.0,
64,
64,
failure_reason=reasons,
)
check("missing locked audio returns None", out is None)
check(
"locked audio reason names file",
len(reasons) == 1 and "locked.wav" in reasons[0],
str(reasons),
)
# FFmpeg command failure: run the real (invalid) fake binary -> OSError or
# non-zero return. The fake file is not executable code; on POSIX exec
# raises OSError (ENOEXEC / format error), on the sandbox it surfaces as
# either the OSError path or a non-zero returncode path. Both must record.
reasons = []
out = merge_video_track_with_ffmpeg(
[{"source": str(fake_ffmpeg), "start_frame": 0, "end_frame": 1}],
1,
24.0,
64,
64,
failure_reason=reasons,
)
check("command failure returns None", out is None)
check(
"command failure records a reason",
len(reasons) == 1
and ("failed to start" in reasons[0] or "exited with code" in reasons[0]),
str(reasons),
)
finally:
shutil.which = _real_which
fake_ffmpeg.unlink(missing_ok=True)
# ---------------------------------------------------------------------------
# 3b. get_ffmpeg_path expanded Windows fallbacks (portable folder, winget/
# scoop/choco shims, imageio-ffmpeg bundled binary). On this POSIX sandbox
# os.name must be forced to "nt" so the fallback branches run.
# ---------------------------------------------------------------------------
import utils.video as _video_mod # noqa: E402
_real_os_name = os.name
_real_base = getattr(_video_mod.folder_paths, "base_path", "")
shutil.which = lambda name, **k: None
os.name = "nt"
try:
# Portable layout: <base>/ffmpeg/bin/ffmpeg.exe found via the portable
# root (dirname of folder_paths.base_path).
fake_base = PACKAGE_ROOT / ".tmp-fake-base" / "ComfyUI"
portable_exe = fake_base.parent / "ffmpeg" / "bin" / "ffmpeg.exe"
portable_exe.parent.mkdir(parents=True, exist_ok=True)
portable_exe.write_bytes(b"")
_video_mod.folder_paths.base_path = str(fake_base)
found = _video_mod.get_ffmpeg_path("ffmpeg")
check(
"portable ffmpeg/bin layout resolved",
found == str(portable_exe),
repr(found),
)
portable_exe.unlink()
shutil.rmtree(PACKAGE_ROOT / ".tmp-fake-base", ignore_errors=True)
_video_mod.folder_paths.base_path = _real_base
# ffmpeg (not ffprobe) falls back to the imageio-ffmpeg bundled binary.
fake_imageio = types.ModuleType("imageio_ffmpeg")
bundled = str(PACKAGE_ROOT / ".tmp-bundled-ffmpeg.exe")
fake_imageio.get_ffmpeg_exe = lambda: bundled
sys.modules["imageio_ffmpeg"] = fake_imageio
found = _video_mod.get_ffmpeg_path("ffmpeg")
check(
"imageio-ffmpeg bundled binary used as last resort",
found == bundled,
repr(found),
)
# ffprobe is NOT bundled by imageio-ffmpeg: must NOT resolve to it.
found = _video_mod.get_ffmpeg_path("ffprobe")
check(
"ffprobe never resolved from imageio-ffmpeg",
found is None,
repr(found),
)
del sys.modules["imageio_ffmpeg"]
finally:
os.name = _real_os_name
_video_mod.folder_paths.base_path = _real_base
shutil.which = _real_which
# ---------------------------------------------------------------------------
# 4. compose_h3_project_video propagates the reason into the RuntimeError.
# ---------------------------------------------------------------------------
def _fake_merge(segments, total_length, frame_rate, w, h, **kwargs):
reasons = kwargs.get("failure_reason")
if reasons is not None:
reasons.append("FFmpeg exited with code 1: Unknown encoder 'libx264'")
return None
# compose_h3_project_video imports merge_video_track_with_ffmpeg from the
# `utils.video` module at call time (function-local import), so patch the
# attribute on the already-loaded video module, not on h3_project.
h3_project.load_h3_project_data = lambda *a, **k: {
"width": 848,
"height": 480,
"frame_rate": 24.0,
"clips": [
{
"index": 0,
"enabled": True,
"file_path": "easy_media/projects/p/out/seg_000.mp4",
"updated_at": 1,
"source_frame_count": 16,
"source_start_frame": 0,
"source_end_frame": 16,
"video_files": [],
}
],
}
video_mod.merge_video_track_with_ffmpeg = _fake_merge
try:
h3_project.compose_h3_project_video("proj", None)
check("compose raises on None", False, "no exception raised")
except RuntimeError as error:
message = str(error)
check(
"compose error carries the real reason",
"could not compose the H3 project video" in message
and "Unknown encoder 'libx264'" in message,
message,
)
except Exception as error: # noqa: BLE001
check("compose raises RuntimeError only", False, repr(error))
# ---------------------------------------------------------------------------
passed = sum(1 for _, ok, _ in CHECKS if ok)
failed = len(CHECKS) - passed
for name, ok, detail in CHECKS:
print(f"{'PASS' if ok else 'FAIL'}: {name}" + (f" [{detail}]" if detail and not ok else ""))
print(f"\n{passed}/{len(CHECKS)} checks passed")
sys.exit(1 if failed else 0)

View File

@ -294,7 +294,8 @@ def _install_loader_stubs(state):
return (sd, {"meta": True}) if return_metadata else sd return (sd, {"meta": True}) if return_metadata else sd
def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None): def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None):
raise AssertionError("MoxieMultimediaLoader must not load LoRAs anymore") state.calls.append(("load_lora_for_models", strength_model, strength_clip))
return state.lora_model, clip
def get_attention_function(name, *args): def get_attention_function(name, *args):
state.calls.append(("get_attention_function", name)) state.calls.append(("get_attention_function", name))
@ -351,7 +352,7 @@ def _load_loader_module():
def verify_loader(): def verify_loader():
print("\n== 4. moxie_loader.py: single-model UI + auto-picked aux models ==") print("\n== 4. moxie_loader.py: single-model UI + auto-picked aux models ==")
state = types.SimpleNamespace( state = types.SimpleNamespace(
calls=[], clip=object(), attention_fn=object(), calls=[], clip=object(), attention_fn=object(), lora_model=object(),
) )
_install_loader_stubs(state) _install_loader_stubs(state)
loader = _load_loader_module() loader = _load_loader_module()
@ -376,14 +377,13 @@ def verify_loader():
("vae", "video_vae.safetensors"), ("vae", "video_vae.safetensors"),
("audio_vae", "audio_vae.safetensors"), ("audio_vae", "audio_vae.safetensors"),
("clip", "MM-VL.safetensors"), ("clip", "MM-VL.safetensors"),
# A leftover loras file must be ignored entirely: the distill
# weights ship baked into the diffusion model now.
("loras", "MM3step.safetensors"), ("loras", "MM3step.safetensors"),
): ):
os.makedirs(os.path.join(models, subdir), exist_ok=True) os.makedirs(os.path.join(models, subdir), exist_ok=True)
with open(os.path.join(models, subdir, name), "wb") as handle: with open(os.path.join(models, subdir, name), "wb") as handle:
handle.write(b"stub") handle.write(b"stub")
loader.MODELS_DIR = models loader.MODELS_DIR = models
loader._loaded_lora = None
result = loader.MoxieMultimediaLoader.execute( result = loader.MoxieMultimediaLoader.execute(
diffusion_model="moxie.safetensors", diffusion_model="moxie.safetensors",
@ -392,21 +392,37 @@ def verify_loader():
) )
names = [call[0] for call in state.calls] names = [call[0] for call in state.calls]
check("full chain runs: model -> attention -> clip -> vaes (no LoRA)", check("full chain runs: model -> attention -> clip -> lora -> vaes",
names == [ names == [
"load_diffusion_model", "get_attention_function", "load_diffusion_model", "get_attention_function",
"set_model_optimized_attention", "load_clip", "set_model_optimized_attention", "load_clip",
"load_torch_file", "load_lora_for_models",
"load_torch_file", "load_torch_file", "load_torch_file", "load_torch_file",
], repr(names)) ], repr(names))
lora_calls = [call for call in state.calls if call[0] == "load_lora_for_models"]
check("LoRA auto-applied at strength 1.0/1.0",
lora_calls == [("load_lora_for_models", 1.0, 1.0)], repr(lora_calls))
clip_calls = [call for call in state.calls if call[0] == "load_clip"] clip_calls = [call for call in state.calls if call[0] == "load_clip"]
check("CLIP auto-picked and loaded with the locked recipe", check("CLIP auto-picked and loaded with the locked recipe",
len(clip_calls) == 1 and clip_calls[0][1] is not None, repr(clip_calls)) len(clip_calls) == 1 and clip_calls[0][1] is not None, repr(clip_calls))
check("four outputs returned", check("four outputs returned",
isinstance(result, loader.io.NodeOutput) and len(result.values) == 4) isinstance(result, loader.io.NodeOutput) and len(result.values) == 4)
check("model output is the attention clone (no LoRA on top)", check("model output carries LoRA",
result.values[0] is not state.model) result.values[0] is state.lora_model)
check("clip output is the loaded clip", result.values[1] is state.clip) check("clip output is the loaded clip", result.values[1] is state.clip)
# Empty loras folder -> LoRA silently skipped, everything else loads.
os.remove(os.path.join(models, "loras", "MM3step.safetensors"))
state.calls.clear()
loader._loaded_lora = None
loader.MoxieMultimediaLoader.execute(
diffusion_model="moxie.safetensors",
weight_dtype="default",
attention_backend="pytorch_attention",
)
check("empty loras folder skips LoRA without breaking the load",
"load_lora_for_models" not in [call[0] for call in state.calls])
# Missing clip folder -> loud setup error (same message as before). # Missing clip folder -> loud setup error (same message as before).
state.calls.clear() state.calls.clear()
os.rename(os.path.join(models, "clip"), os.path.join(models, "clip_backup")) os.rename(os.path.join(models, "clip"), os.path.join(models, "clip_backup"))

View File

@ -266,8 +266,6 @@ custom 示例:
- `segment_count = -1` 表示从起始索引处理到末尾;非负数限制本次任务片段数量。 - `segment_count = -1` 表示从起始索引处理到末尾;非负数限制本次任务片段数量。
- `project_save`: `new` 或 `override`。在 `override` 且 count 为 `-1` 时,执行逻辑会删除起始索引之后的已保存片段再生成;修改前向用户清楚说明。 - `project_save`: `new` 或 `override`。在 `override` 且 count 为 `-1` 时,执行逻辑会删除起始索引之后的已保存片段再生成;修改前向用户清楚说明。
- `sampling_mode`: `single` 或 `dual`;`sampling_plan` 必须是当前节点可用预设。不要仅凭另一份流程中的值假设当前环境支持。 - `sampling_mode`: `single` 或 `dual`;`sampling_plan` 必须是当前节点可用预设。不要仅凭另一份流程中的值假设当前环境支持。
- `context_frames`(默认 `"22"`,可选 `"5"` / `"22"` / `"39"` / `"56"`)控制 Context / Context Swap 续接片段从上一段拷贝为延续上下文的视频帧数;取值必须落在 H3 时间潜空间网格(17k+5 帧)上,与 Moxie 上下文采样节点的选项一致。数值越大运动衔接越平滑,但每个续接片段会多生成等量帧(实际生成长度为 `任务长度 + context_frames + 12`,其中 12 帧为网格尾帧,解码后裁掉)。
- `Moxie MultiTrack Project`(`MoxieProject`)同样暴露同一个 `context_frames` 下拉,位于 `steps` 与 `seed` 之间;该节点的其余采样设置仍保持隐藏(euler + beta sigmas 由 `steps` 推导)。
- 编辑器分辨率是一采尺寸;`dual` 不会反算缩小一采。二采宽高分别为 `round(编辑器尺寸 * upscale_by / 32) * 32`,采用 Python `round` 就近对齐,恰好居中时取偶数,不是固定向上对齐。 - 编辑器分辨率是一采尺寸;`dual` 不会反算缩小一采。二采宽高分别为 `round(编辑器尺寸 * upscale_by / 32) * 32`,采用 Python `round` 就近对齐,恰好居中时取偶数,不是固定向上对齐。
- `upscale_by` 是 Project 的独立参数,默认 `1.250`,三位小数,步长 `0.001`;旧工作流保留已保存的值。两条放大路径及合并导出均使用放大并对齐后的尺寸。例如 `1344 × 768`、倍率 `1.250` 的二采为 `1664 × 960`。若用户指定最终输出尺寸,应同时核对编辑器尺寸与倍率,不能将编辑器尺寸直接当作双采样的最终尺寸。 - `upscale_by` 是 Project 的独立参数,默认 `1.250`,三位小数,步长 `0.001`;旧工作流保留已保存的值。两条放大路径及合并导出均使用放大并对齐后的尺寸。例如 `1344 × 768`、倍率 `1.250` 的二采为 `1664 × 960`。若用户指定最终输出尺寸,应同时核对编辑器尺寸与倍率,不能将编辑器尺寸直接当作双采样的最终尺寸。
- 单采样、仅一采预览保持编辑器尺寸;`upscale_by = 1.000` 不放大但可二采;纯音频项目(`32 × 32`)不放大。旧版反算缩小尺寸的一采检查点和上下文潜空间应重新生成后再按新规则续跑。 - 单采样、仅一采预览保持编辑器尺寸;`upscale_by = 1.000` 不放大但可二采;纯音频项目(`32 × 32`)不放大。旧版反算缩小尺寸的一采检查点和上下文潜空间应重新生成后再按新规则续跑。
@ -292,7 +290,6 @@ Plan 只包含实际变更。`node_id` 可省略并由拓扑定位;`editor.res
"project_save": "override", "project_save": "override",
"segment_start_number": 1, "segment_start_number": 1,
"segment_count": -1, "segment_count": -1,
"context_frames": "22",
"sampling_plan": "medium", "sampling_plan": "medium",
"sampling_mode": "single" "sampling_mode": "single"
} }

View File

@ -185,7 +185,7 @@ def _load_minimax_node(monkeypatch):
Combo=_PortType, Combo=_PortType,
ComfyNode=object, ComfyNode=object,
Conditioning=_PortType, Conditioning=_PortType,
ControlAfterGenerate=types.SimpleNamespace(fixed="fixed", randomize="randomize"), ControlAfterGenerate=types.SimpleNamespace(fixed="fixed"),
DynamicCombo=_DynamicCombo, DynamicCombo=_DynamicCombo,
Hidden=types.SimpleNamespace(prompt="PROMPT", unique_id="UNIQUE_ID"), Hidden=types.SimpleNamespace(prompt="PROMPT", unique_id="UNIQUE_ID"),
Custom=lambda **kwargs: _PortType, Custom=lambda **kwargs: _PortType,
@ -260,12 +260,6 @@ def _load_minimax_node(monkeypatch):
comfy.model_management = model_management comfy.model_management = model_management
comfy.nested_tensor = nested_tensor comfy.nested_tensor = nested_tensor
comfy.utils = comfy_utils comfy.utils = comfy_utils
comfy_samplers = types.ModuleType("comfy.samplers")
comfy_samplers.calculate_sigmas = (
lambda *args, **kwargs: torch.arange(8.0)
)
comfy_samplers.sampler_object = lambda name: {"sampler": name}
comfy.samplers = comfy_samplers
node_helpers = types.ModuleType("node_helpers") node_helpers = types.ModuleType("node_helpers")
@ -454,7 +448,6 @@ def _load_minimax_node(monkeypatch):
"comfy.model_management": model_management, "comfy.model_management": model_management,
"comfy.nested_tensor": nested_tensor, "comfy.nested_tensor": nested_tensor,
"comfy.utils": comfy_utils, "comfy.utils": comfy_utils,
"comfy.samplers": comfy_samplers,
"node_helpers": node_helpers, "node_helpers": node_helpers,
"torchaudio": torchaudio, "torchaudio": torchaudio,
"folder_paths": folder_paths, "folder_paths": folder_paths,
@ -523,21 +516,6 @@ def _load_moxie_combine(monkeypatch):
return module return module
def _load_moxie_project(monkeypatch):
"""Load moxie_project.py into the stub environment set up above."""
module = _load_minimax_node(monkeypatch)
assert module is not None
root = Path(__file__).parents[1]
spec = importlib.util.spec_from_file_location(
"easy_media.moxie_project", root / "moxie_project.py"
)
assert spec is not None and spec.loader is not None
moxie_module = importlib.util.module_from_spec(spec)
sys.modules[spec.name] = moxie_module
spec.loader.exec_module(moxie_module)
return moxie_module
def _image_values(*values): def _image_values(*values):
return torch.tensor(values, dtype=torch.float32).reshape(len(values), 1, 1, 1) return torch.tensor(values, dtype=torch.float32).reshape(len(values), 1, 1, 1)
@ -1260,7 +1238,6 @@ def test_multitrack_h3_project_schema_exposes_pipeline_configuration(monkeypatch
"project_save", "project_save",
"segment_start_number", "segment_start_number",
"segment_count", "segment_count",
"context_frames",
"seed", "seed",
"sampling_plan", "sampling_plan",
"sampling_mode", "sampling_mode",
@ -1282,8 +1259,6 @@ def test_multitrack_h3_project_schema_exposes_pipeline_configuration(monkeypatch
assert inputs["project_name"].kwargs["default"] == "" assert inputs["project_name"].kwargs["default"] == ""
assert inputs["project_save"].kwargs["options"] == ["new", "override"] assert inputs["project_save"].kwargs["options"] == ["new", "override"]
assert inputs["project_save"].kwargs["default"] == "override" assert inputs["project_save"].kwargs["default"] == "override"
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
assert inputs["context_frames"].kwargs["default"] == "22"
sampling_plan_options = inputs["sampling_plan"].kwargs["options"] sampling_plan_options = inputs["sampling_plan"].kwargs["options"]
assert sampling_plan_options == [ assert sampling_plan_options == [
"custom", "custom",
@ -2472,53 +2447,6 @@ def test_multitrack_h3_project_reads_ref_image_size_from_each_segment(monkeypatc
assert conditioning["ref_image_size"] == "max" assert conditioning["ref_image_size"] == "max"
def test_multitrack_h3_project_keeps_544x960_first_pass_on_grid(monkeypatch):
"""Regression for the 2026-10-16 user crash report.
A 544x960 single-pass project used to sample its hidden first pass at the
raw half resolution 272x480. The keyframe conditioned on that canvas is a
17-wide latent, and comfy's patchify_video floors it into 2x2 patches,
crashing with "shape '[1, 24, 1, 1, 15, 2, 8, 2]' is invalid for input of
size 12240". The first pass must land on the 32px canvas grid instead.
"""
module = _load_minimax_node(monkeypatch)
module.comfy_nodes.NODE_CLASS_MAPPINGS["ImageResizeKJv2"] = _ImageResizeKJWithNvidia
inputs = _h3_project_inputs(sampling_mode=_h3_sampling_mode("single"))
inputs["tracks_info"][0]["width"] = 544
inputs["tracks_info"][0]["height"] = 960
result = module.EasyMultiTrackProject.execute(**inputs)
conditioning = _graph_node(result, "easy minimaxH3ToVideo")["inputs"]
# half of 544x960 stays on the 32px grid: 256x480 -> latent 16x30, even.
assert conditioning["width"] == 256
assert conditioning["height"] == 480
def test_multitrack_h3_project_aligns_off_grid_api_dimensions(monkeypatch):
"""External app clients inject project dimensions through the ComfyUI API
that are not multiples of 32; the editor UI only offers grid sizes. The
project must run both sampling passes on the same grid-aligned canvas
instead of the raw injected size (odd latent -> patchify_video crash)."""
module = _load_minimax_node(monkeypatch)
module.comfy_nodes.NODE_CLASS_MAPPINGS["ImageResizeKJv2"] = _ImageResizeKJWithNvidia
inputs = _h3_project_inputs(sampling_mode=_h3_sampling_mode("dual"))
inputs["tracks_info"][0]["width"] = 530
inputs["tracks_info"][0]["height"] = 964
result = module.EasyMultiTrackProject.execute(**inputs)
conditionings = [
node["inputs"]
for node in result.expand.values()
if node["class_type"] == "easy minimaxH3ToVideo"
]
# 530x964 floors onto the 32px grid as 512x960; the default 1.25x second
# pass rounds to 640x1216 (960*1.25=1200 rounds up to the 32px grid).
assert sorted(item["width"] for item in conditionings) == [512, 640]
assert sorted(item["height"] for item in conditionings) == [960, 1216]
def test_multitrack_h3_project_logs_steps_and_reports_progress(monkeypatch): def test_multitrack_h3_project_logs_steps_and_reports_progress(monkeypatch):
module = _load_minimax_node(monkeypatch) module = _load_minimax_node(monkeypatch)
assert module is not None assert module is not None
@ -3821,89 +3749,6 @@ def test_multitrack_h3_context_chain_uses_previous_segment_latent(monkeypatch):
assert artifacts[1][1]["inputs"]["previous"] == [artifacts[0][0], 0] assert artifacts[1][1]["inputs"]["previous"] == [artifacts[0][0], 0]
def test_multitrack_context_frames_widget_propagates(monkeypatch):
module = _load_minimax_node(monkeypatch)
assert module is not None
module.comfy_nodes.NODE_CLASS_MAPPINGS["MiniMaxH3MotionContextTrim"] = (
_MiniMaxMotionContextTrim
)
info = _h3_project_inputs()["tracks_info"][0]
info["tracks"][0]["segments"].append(
{
"start_frame": 120,
"end_frame": 240,
"content": {
"task_mode": "l2v",
"continuity_mode": "context",
"images": [{"media_index": 0}],
"user_prompt": "continue",
},
}
)
info["tracks"].append(
{
"id": "locked-audio-track",
"type": "audio",
"audio_locked": True,
"segments": [
{
"id": "locked-audio",
"start_frame": 0,
"end_frame": 240,
"content": {"media_type": "audio"},
}
],
}
)
result = module.EasyMultiTrackProject.execute(
**_h3_project_inputs(tracks_info=[info], context_frames=["5"])
)
motion = next(
node
for node in result.expand.values()
if node["class_type"] == "easy MiniMaxH3MotionContextHard"
)
assert motion["inputs"]["context_length"] == "5"
context_trims = [
node
for node in result.expand.values()
if node["class_type"] == "easy h3MotionContextLatentTrim"
]
assert context_trims
assert {trim["inputs"]["context_length"] for trim in context_trims} == {"5"}
audio_locks = [
node
for node in result.expand.values()
if node["class_type"] == "easy minimaxH3AudioLock"
]
assert sorted(node["inputs"]["prepend_frames"] for node in audio_locks) == [0, 5]
math_node = next(
node
for node in result.expand.values()
if node["class_type"] == "ComfyMathExpression"
)
# The context continuation generates context_frames + 12 grid-tail frames.
assert math_node["inputs"]["expression"] == "a + 17"
context_encodes = [
node
for node in result.expand.values()
if node["class_type"] == "easy h3ContextMediaTrim"
and node["inputs"]["output_frames"] == 5
]
assert context_encodes
def test_multitrack_context_frames_widget_rejects_off_grid_values(monkeypatch):
module = _load_minimax_node(monkeypatch)
assert module is not None
with pytest.raises(ValueError, match="supported H3 context lengths"):
module.EasyMultiTrackProject.execute(
**_h3_project_inputs(context_frames=["18"])
)
def test_multitrack_h3_context_uses_embedded_five_frame_anchor(monkeypatch): def test_multitrack_h3_context_uses_embedded_five_frame_anchor(monkeypatch):
module = _load_minimax_node(monkeypatch) module = _load_minimax_node(monkeypatch)
info = _h3_project_inputs()["tracks_info"][0] info = _h3_project_inputs()["tracks_info"][0]
@ -5369,32 +5214,6 @@ def test_reference_encodes_images_video_audio_and_standalone_audio(monkeypatch):
assert conditioning["inputs"]["ref_video_0"][1] == 0 assert conditioning["inputs"]["ref_video_0"][1] == 0
def test_reference_fallback_grid_aligns_off_grid_ref_videos(monkeypatch):
"""ComfyUI builds with native H3 nodes grid-align reference videos inside
core's MiniMaxH3ReferenceToVideo; the fallback copy must do the same, or
an off-grid media size (the API can inject arbitrary media) yields an odd
latent dimension and crashes patchify_video during sampling."""
module = _load_minimax_node(monkeypatch)
assert module is not None
vae = _Vae()
module.MiniMaxH3ReferenceToVideoFallback.execute(
clip=_Clip(),
vae=vae,
audio_vae=None,
prompt="prompt",
width=32,
height=32,
length=5,
ref_videos={"ref_video_0": torch.zeros(6, 270, 480, 3)},
)
encoded_frames = vae.encoded[-1]
# Frames are [T, H, W, C]: 270 floors onto the 32px grid as 256; 480 is
# already aligned.
assert tuple(encoded_frames.shape[1:3]) == (256, 480)
def test_reference_bridge_groups_fixed_inputs_before_direct_execute(monkeypatch): def test_reference_bridge_groups_fixed_inputs_before_direct_execute(monkeypatch):
module = _load_minimax_node(monkeypatch) module = _load_minimax_node(monkeypatch)
assert module is not None assert module is not None
@ -6275,68 +6094,3 @@ def test_minimax_prompt_override_node_has_complete_chinese_localization():
assert translation["inputs"]["video_track_lock"]["name"] == "视频轨锁定" assert translation["inputs"]["video_track_lock"]["name"] == "视频轨锁定"
assert translation["inputs"]["audio_track_lock"]["name"] == "音频轨锁定" assert translation["inputs"]["audio_track_lock"]["name"] == "音频轨锁定"
assert translation["outputs"] == {"0": {"name": "提示词覆盖"}} assert translation["outputs"] == {"0": {"name": "提示词覆盖"}}
def test_moxie_project_schema_exposes_context_frames(monkeypatch):
module = _load_moxie_project(monkeypatch)
schema = module.MoxieProject.define_schema()
inputs = {port.name: port for port in schema.inputs}
assert schema.node_id == "MoxieProject"
assert schema.display_name == "Moxie MultiTrack Project"
assert schema.is_input_list is True
assert list(inputs) == [
"tracks_info",
"model",
"clip",
"video_vae",
"audio_vae",
"project_name",
"steps",
"context_frames",
"seed",
]
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
assert inputs["context_frames"].kwargs["default"] == "22"
assert inputs["steps"].kwargs["default"] == 3
assert inputs["seed"].kwargs["default"] == 42
def test_moxie_project_forwards_context_frames_to_engine(monkeypatch):
module = _load_moxie_project(monkeypatch)
captured = {}
def fake_execute(cls, **kwargs):
captured.update(kwargs)
return "sentinel-output"
monkeypatch.setattr(
module.EasyMultiTrackProject, "execute", classmethod(fake_execute)
)
model = types.SimpleNamespace(
get_model_object=lambda name: types.SimpleNamespace()
)
result = module.MoxieProject.execute(
tracks_info=[{"tracks": []}],
model=[model],
clip=[types.SimpleNamespace()],
video_vae=[types.SimpleNamespace()],
audio_vae=[types.SimpleNamespace()],
project_name=["demo"],
steps=["4"],
context_frames=["5"],
seed=["7"],
)
assert result == "sentinel-output"
assert captured["context_frames"] == ["5"]
assert captured["steps"] == ["4"]
assert captured["project_name"] == ["demo"]
assert captured["seed"] == ["7"]
# Baked settings survive the handoff and never clobber the widget.
assert captured["sampling_plan"] == "custom"
assert captured["sampling_mode"] == "single"
assert captured["sampler"] == {"sampler": "euler"}
assert captured["sigmas"].shape == (5,)

View File

@ -7,8 +7,7 @@ Verified contracts:
- MoxieMultimediaLoader: outputs exactly (model, clip, video_vae, audio_vae); - MoxieMultimediaLoader: outputs exactly (model, clip, video_vae, audio_vae);
no legacy FAST_MODEL_LOADER bundle output; no clip_type choice (the CLIP no legacy FAST_MODEL_LOADER bundle output; no clip_type choice (the CLIP
recipe is locked to CLIPType.MINIMAX); the model output has the attention recipe is locked to CLIPType.MINIMAX); the model output has the attention
backend applied, and any leftover file in models/loras is ignored (the backend and LoRA applied.
distill weights ship baked into the diffusion model).
- MoxieProject: takes model/clip/video_vae/audio_vae as required direct - MoxieProject: takes model/clip/video_vae/audio_vae as required direct
inputs instead of a bundled model_loader, and packs them into the loader inputs instead of a bundled model_loader, and packs them into the loader
dict consumed internally by EasyMultiTrackProject. dict consumed internally by EasyMultiTrackProject.
@ -123,6 +122,8 @@ class _StubState:
self.calls = [] self.calls = []
self.attention_fn = lambda pixels, *a, **k: pixels self.attention_fn = lambda pixels, *a, **k: pixels
self.clip = object() self.clip = object()
self.lora_model = None
self.lora_clip = None
def _register(state, io, monkeypatch=None): def _register(state, io, monkeypatch=None):
@ -157,7 +158,10 @@ def _register(state, io, monkeypatch=None):
return state.clip return state.clip
def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None): def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None):
raise AssertionError("MoxieMultimediaLoader must not load LoRAs anymore") state.calls.append(("load_lora_for_models", strength_model, strength_clip))
state.lora_model = model
state.lora_clip = clip
return model, clip
comfy_sd.CLIPType = CLIPType comfy_sd.CLIPType = CLIPType
comfy_sd.load_diffusion_model = load_diffusion_model comfy_sd.load_diffusion_model = load_diffusion_model
@ -299,7 +303,6 @@ def _build_moxie():
tiny_vae = types.ModuleType("MMnode.tiny_vae") tiny_vae = types.ModuleType("MMnode.tiny_vae")
tiny_vae.load_tiny_vae_decoder = lambda *a, **k: None tiny_vae.load_tiny_vae_decoder = lambda *a, **k: None
tiny_vae.pack_vae_approx_files = lambda *a, **k: []
added["MMnode.tiny_vae"] = sys.modules.get("MMnode.tiny_vae") added["MMnode.tiny_vae"] = sys.modules.get("MMnode.tiny_vae")
sys.modules["MMnode.tiny_vae"] = tiny_vae sys.modules["MMnode.tiny_vae"] = tiny_vae
@ -357,7 +360,7 @@ def test_loader_schema_shows_a_single_model(moxie):
inputs = _input_names(schema) inputs = _input_names(schema)
# End users see one model picker plus the attention backend; every # End users see one model picker plus the attention backend; every
# auxiliary model (video/audio VAE, CLIP) is auto-selected. # auxiliary model (LoRA, video/audio VAE, CLIP) is auto-selected.
assert inputs == ["diffusion_model", "weight_dtype", "attention_backend"] assert inputs == ["diffusion_model", "weight_dtype", "attention_backend"]
for hidden in ( for hidden in (
"lora_name", "lora_name",
@ -371,7 +374,7 @@ def test_loader_schema_shows_a_single_model(moxie):
assert hidden not in inputs assert hidden not in inputs
def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch, tmp_path): def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path):
state = moxie.state state = moxie.state
models = tmp_path / "models" models = tmp_path / "models"
for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"): for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"):
@ -386,6 +389,9 @@ def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch,
}[name] }[name]
(models / subdir / name).write_bytes(b"stub") (models / subdir / name).write_bytes(b"stub")
monkeypatch.setattr(moxie.loader, "MODELS_DIR", models) monkeypatch.setattr(moxie.loader, "MODELS_DIR", models)
# LoRA cache lives at module level: node classes are locked by newer
# ComfyUI builds and class attributes can no longer be written at runtime.
monkeypatch.setattr(moxie.loader, "_loaded_lora", None)
state.calls.clear() state.calls.clear()
result = moxie.loader.MoxieMultimediaLoader.execute( result = moxie.loader.MoxieMultimediaLoader.execute(
@ -394,15 +400,14 @@ def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch,
attention_backend="comfy_kitchen_attention", attention_backend="comfy_kitchen_attention",
) )
# The distill weights ship baked into the diffusion model: a leftover
# file in models/loras must be ignored entirely. The comfy.sd stub keeps
# a raising load_lora_for_models, so an accidental LoRA call fails loudly.
sequence = [call[0] for call in state.calls] sequence = [call[0] for call in state.calls]
assert sequence == [ assert sequence == [
"load_diffusion_model", "load_diffusion_model",
"get_attention_function", "get_attention_function",
"load_clip", "load_clip",
"load_torch_file", "load_torch_file",
"load_lora_for_models",
"load_torch_file",
"load_torch_file", "load_torch_file",
] ]
@ -410,22 +415,50 @@ def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch,
clip_calls = [call for call in state.calls if call[0] == "load_clip"] clip_calls = [call for call in state.calls if call[0] == "load_clip"]
assert clip_calls == [("load_clip", state.CLIPType.MINIMAX)] assert clip_calls == [("load_clip", state.CLIPType.MINIMAX)]
# Attention backend applied before the CLIP load. # LoRA applied on top of the attention-patched model, after the CLIP load.
assert state.calls.index(("get_attention_function", "comfy_kitchen_int8")) < state.calls.index( assert state.calls.index(("get_attention_function", "comfy_kitchen_int8")) < state.calls.index(
("load_clip", state.CLIPType.MINIMAX) ("load_clip", state.CLIPType.MINIMAX)
) ) < state.calls.index(("load_lora_for_models", 1.0, 1.0))
# Exactly four direct outputs, model carrying the attention backend. # Exactly four direct outputs, model carrying attention + LoRA.
assert isinstance(result, _NodeOutput) assert isinstance(result, _NodeOutput)
assert len(result.values) == 4 assert len(result.values) == 4
model_out, clip_out, video_vae_out, audio_vae_out = result.values model_out, clip_out, video_vae_out, audio_vae_out = result.values
assert model_out.label == "diffusion.clone" assert model_out is state.lora_model
assert model_out.attention_fn is state.attention_fn assert model_out.attention_fn is state.attention_fn
assert clip_out is state.clip assert clip_out is state.clip
assert isinstance(video_vae_out, _FakeVAE) assert isinstance(video_vae_out, _FakeVAE)
assert isinstance(audio_vae_out, _FakeVAE) assert isinstance(audio_vae_out, _FakeVAE)
def test_loader_execute_without_lora_skips_lora_loading(moxie, monkeypatch, tmp_path):
state = moxie.state
models = tmp_path / "models"
for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"):
(models / subdir).mkdir(parents=True, exist_ok=True)
for name in ("h3.safetensors", "video.safetensors", "audio.safetensors", "qwen.safetensors"):
subdir = {
"h3.safetensors": "diffusion",
"video.safetensors": "vae",
"audio.safetensors": "audio_vae",
"qwen.safetensors": "clip",
}[name]
(models / subdir / name).write_bytes(b"stub")
monkeypatch.setattr(moxie.loader, "MODELS_DIR", models)
state.calls.clear()
result = moxie.loader.MoxieMultimediaLoader.execute(
diffusion_model="h3.safetensors",
weight_dtype="default",
attention_backend="pytorch_attention",
)
assert "load_lora_for_models" not in [call[0] for call in state.calls]
assert result.values[0] is not state.clip # model output remains a model
assert result.values[0].attention_fn is state.attention_fn
assert result.values[1] is state.clip
def test_loader_requires_minimax_cliptype(moxie, monkeypatch, tmp_path): def test_loader_requires_minimax_cliptype(moxie, monkeypatch, tmp_path):
state = moxie.state state = moxie.state
models = tmp_path / "models" models = tmp_path / "models"
@ -462,12 +495,9 @@ def test_project_schema_takes_direct_model_inputs(moxie):
"audio_vae", "audio_vae",
"project_name", "project_name",
"steps", "steps",
"context_frames",
"seed", "seed",
] ]
assert "model_loader" not in inputs assert "model_loader" not in inputs
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
assert inputs["context_frames"].kwargs["default"] == "22"
# The four model components are required direct connections. # The four model components are required direct connections.
for name in ("model", "clip", "video_vae", "audio_vae"): for name in ("model", "clip", "video_vae", "audio_vae"):

View File

@ -1,138 +0,0 @@
@echo off
setlocal
title MMnode Updater
REM ============================================================
REM MMnode (Moxie-Multimedia) one-click updater
REM
REM Put this file INSIDE the pack folder:
REM ComfyUI\custom_nodes\<pack-folder>\
REM (next to __init__.py) and double-click it.
REM
REM It syncs code from HuggingFace, removes stale files from
REM old versions, downloads model weights via git-lfs, verifies
REM the build, and reminds you to restart ComfyUI.
REM ============================================================
cd /d "%~dp0"
echo.
echo ==================================================
echo MMnode updater - syncing from HuggingFace
echo ==================================================
echo.
if not exist "__init__.py" (
echo [ERROR] This is not the MMnode pack folder.
echo Move update.bat into ComfyUI\custom_nodes\^<pack-folder^>
echo next to __init__.py, then run it again.
goto end_fail
)
where git >nul 2>&1
if errorlevel 1 (
echo [ERROR] Git is not installed or not in PATH.
echo Get it from https://git-scm.com/download/win
goto end_fail
)
netstat -ano 2>nul | findstr /c:":8188" >nul 2>&1
if not errorlevel 1 (
echo [WARN] ComfyUI appears to be running on port 8188.
echo It must be fully closed for the update to take effect.
echo Continuing in 5 seconds - close it now...
timeout /t 5 /nobreak >nul
)
if not exist ".git" (
echo [*] No git repo here yet - initializing...
git init
if errorlevel 1 goto gitfail
)
git remote remove origin >nul 2>&1
git remote add origin https://huggingface.co/Moxiegen/Moxie-Multimedia
if errorlevel 1 goto gitfail
echo.
echo [*] Fetching latest version from HuggingFace...
git fetch origin --prune
if errorlevel 1 (
echo [ERROR] Could not reach HuggingFace - check your internet.
goto end_fail
)
echo.
echo [*] Syncing files...
git reset --hard origin/main
if errorlevel 1 goto gitfail
echo.
echo [*] Cleaning stale files from older versions...
git clean -fd -e update.bat >nul 2>&1
echo.
echo [*] Syncing model weights...
git lfs version >nul 2>&1
if errorlevel 1 goto nolfs
git lfs install >nul 2>&1
echo Downloading weights - first run can take several minutes...
git lfs fetch origin main
if errorlevel 1 echo [WARN] Weight download failed - check connection, re-run later.
git lfs checkout
goto after_lfs
:nolfs
echo [WARN] git-lfs is NOT installed on this PC.
echo Code was updated, but model weights may be broken placeholders.
echo Install git-lfs from https://git-lfs.com
echo or run: winget install GitHub.git-lfs
echo ...then run this updater again.
:after_lfs
REM ---- verify build ----
set FIXCOUNT=0
for /f %%A in ('findstr /c:"_align_canvas_dimension" "nodes\project.py" ^| find /c /v ""') do set FIXCOUNT=%%A
set POINTER=0
for /r models %%F in (*.safetensors) do (
if %%~zF LSS 1000 set /a POINTER+=1
)
echo.
echo ==================================================
for /f "delims=" %%L in ('git log -1 --oneline') do echo Build: %%L
echo Fix marker: %FIXCOUNT% found in nodes\project.py
if %FIXCOUNT% GEQ 3 (
echo [OK] This is the fixed build.
) else (
echo [ERROR] Fix marker missing - update did not complete!
)
if %POINTER% GTR 0 (
echo [WARN] %POINTER% model file^(s^) are broken placeholders.
echo Install git-lfs from https://git-lfs.com and re-run.
) else (
echo [OK] Model weights look good.
)
echo ==================================================
echo.
echo All done. RESTART COMFYUI COMPLETELY before testing.
echo Close this window and any ComfyUI console, then start ComfyUI.
goto end_ok
:gitfail
echo.
echo [ERROR] A git command failed - scroll up for the message.
goto end_fail
:end_fail
echo.
pause
exit /b 1
:end_ok
echo.
pause
exit /b 0

View File

@ -1327,18 +1327,15 @@ def compose_h3_project_video(project_name: Any, project_data: Any = None) -> Pat
ffprobe_info, ffprobe_info,
) )
failure_reasons: list[str] = []
temporary = merge_video_track_with_ffmpeg( temporary = merge_video_track_with_ffmpeg(
timeline_segments, timeline_segments,
cursor, cursor,
frame_rate, frame_rate,
compose_size[0] if compose_size else width, compose_size[0] if compose_size else width,
compose_size[1] if compose_size else height, compose_size[1] if compose_size else height,
failure_reason=failure_reasons,
) )
if temporary is None: if temporary is None:
detail = "; ".join(failure_reasons) if failure_reasons else "no reason was recorded" raise RuntimeError("FFmpeg could not compose the H3 project video")
raise RuntimeError(f"FFmpeg could not compose the H3 project video: {detail}")
return Path(temporary) return Path(temporary)

View File

@ -7,7 +7,6 @@ import math
import os import os
import shutil import shutil
import subprocess import subprocess
import sys
import tempfile import tempfile
import urllib.request import urllib.request
from .media import AUDIO_EXTENSIONS from .media import AUDIO_EXTENSIONS
@ -308,72 +307,21 @@ def merge_video_track_with_ffmpeg(
width: int, width: int,
height: int, height: int,
resize_method: str | None = None, resize_method: str | None = None,
*,
failure_reason: list[str] | None = None,
) -> str | None: ) -> str | None:
"""Compose file-backed segments on a black full-length timeline. """Compose file-backed segments on a black full-length timeline."""
Returns ``None`` when composition is impossible so callers can fall back
to another strategy. Callers that need to surface the real cause instead
pass a ``failure_reason`` list, which receives one human-readable entry
per failed precondition (missing FFmpeg, missing source files, ...) or
the FFmpeg stderr tail when the command itself fails.
"""
def record(reason: str) -> None:
if failure_reason is not None:
failure_reason.append(reason)
ffmpeg = get_ffmpeg_path("ffmpeg") ffmpeg = get_ffmpeg_path("ffmpeg")
resize_filter = _ffmpeg_resize_filter(width, height, resize_method) if resize_method else None resize_filter = _ffmpeg_resize_filter(width, height, resize_method) if resize_method else None
if ffmpeg is None: if ffmpeg is None or total_length <= 0 or frame_rate <= 0 or width % 2 or height % 2:
record(
"FFmpeg executable not found (checked PATH, common install folders, "
"package-manager shims and the imageio-ffmpeg bundled binary). "
"Install FFmpeg with ffmpeg.exe on PATH - or simply copy ffmpeg.exe "
"into C:\\ffmpeg\\bin - and restart ComfyUI from a NEW terminal so "
"the updated PATH is picked up"
)
return None
if total_length <= 0:
record(f"timeline is empty (total_length={total_length} frames)")
return None
if frame_rate <= 0:
record(f"invalid frame rate {frame_rate}")
return None
if width % 2 or height % 2:
record(
f"canvas size {width}x{height} has odd dimensions "
"(video encoders require even width and height)"
)
return None return None
if resize_method is not None and resize_filter is None: if resize_method is not None and resize_filter is None:
record(f"resize method '{resize_method}' is not supported by this FFmpeg build")
return None return None
missing_sources = [ if any(not os.path.isfile(str(segment.get("source", ""))) for segment in segments):
str(segment.get("source", ""))
for segment in segments
if not os.path.isfile(str(segment.get("source", "")))
]
if missing_sources:
record(
"segment source file(s) missing: "
+ " | ".join(missing_sources[:5])
+ (f" (+{len(missing_sources) - 5} more)" if len(missing_sources) > 5 else "")
)
return None return None
missing_audio = [ if any(
str(segment["audio_source"]) segment.get("audio_source") is not None
for segment in segments
if segment.get("audio_source") is not None
and not os.path.isfile(str(segment["audio_source"])) and not os.path.isfile(str(segment["audio_source"]))
] for segment in segments
if missing_audio: ):
record(
"locked audio file(s) missing: "
+ " | ".join(missing_audio[:5])
+ (f" (+{len(missing_audio) - 5} more)" if len(missing_audio) > 5 else "")
)
return None return None
total_seconds = total_length / frame_rate total_seconds = total_length / frame_rate
@ -547,7 +495,6 @@ def merge_video_track_with_ffmpeg(
) from exc ) from exc
except OSError as exc: except OSError as exc:
logger.warning("FFmpeg video track merge failed to start: %s", exc) logger.warning("FFmpeg video track merge failed to start: %s", exc)
record(f"FFmpeg failed to start ({exc})")
try: try:
os.unlink(output_path) os.unlink(output_path)
except OSError: except OSError:
@ -556,10 +503,9 @@ def merge_video_track_with_ffmpeg(
if result.returncode == 0: if result.returncode == 0:
return output_path return output_path
stderr_tail = result.stderr.decode(errors="replace")[-600:] logger.warning(
logger.warning("FFmpeg video track merge failed: %s", stderr_tail) "FFmpeg video track merge failed: %s",
record( result.stderr.decode(errors="replace")[-600:],
f"FFmpeg exited with code {result.returncode}: {stderr_tail.strip() or 'no stderr output'}"
) )
try: try:
os.unlink(output_path) os.unlink(output_path)
@ -569,58 +515,19 @@ def merge_video_track_with_ffmpeg(
def get_ffmpeg_path(name: str = "ffmpeg") -> str | None: def get_ffmpeg_path(name: str = "ffmpeg") -> str | None:
"""Find FFmpeg/ffprobe executable, with Windows-specific fallbacks. """Find FFmpeg/ffprobe executable, with Windows-specific fallbacks."""
PATH edits on Windows only reach processes started afterwards, and
Portable users routinely have ffmpeg.exe sitting next to the run bat or
installed via winget/scoop/choco without the ComfyUI process ever seeing
it. After ``which`` fails, probe the locations such installs actually
use, and finally fall back to the binary bundled with imageio-ffmpeg
(a ComfyUI core dependency) so a zero-setup machine can still encode.
"""
# Try standard which first (works on all platforms) # Try standard which first (works on all platforms)
ffmpeg = shutil.which(name) or shutil.which(f"{name}.exe") ffmpeg = shutil.which(name) or shutil.which(f"{name}.exe")
if ffmpeg: if ffmpeg:
return ffmpeg return ffmpeg
if os.name != "nt": # Windows fallback: common installation paths
return None if os.name == "nt":
exe = f"{name}.exe" for path in (
candidates: list[str] = [ r"C:\ffmpeg\bin\{}.exe".format(name),
r"C:\ffmpeg\bin\{}".format(name), os.path.expandvars(r"%LOCALAPPDATA%\Programs\{}\bin\{}.exe".format(name, name)),
os.path.expandvars(r"%LOCALAPPDATA%\Programs\{}\bin\{}".format(name, name)), ):
# Package-manager shims/links. if os.path.isfile(path):
os.path.expandvars(r"%LOCALAPPDATA%\Microsoft\WinGet\Links\{}".format(exe)), return path
os.path.expandvars(r"%USERPROFILE%\scoop\shims\{}".format(exe)),
r"C:\ProgramData\chocolatey\bin\{}".format(exe),
]
# Relative to the ComfyUI install: the portable root (parent of the
# ComfyUI folder) and the folder itself. ffmpeg.exe next to the run bat
# or in an ffmpeg/bin subfolder is a very common portable layout.
base = str(getattr(folder_paths, "base_path", "") or "")
roots: list[str] = []
if base:
roots.append(base)
roots.append(os.path.dirname(os.path.abspath(base)))
roots.append(os.path.dirname(os.path.abspath(sys.executable)))
roots.append(os.getcwd())
for root in roots:
candidates.extend([
os.path.join(root, exe),
os.path.join(root, "bin", exe),
os.path.join(root, "ffmpeg", "bin", exe),
])
for candidate in candidates:
if os.path.isfile(candidate):
return candidate
# Last resort: imageio-ffmpeg ships a full ffmpeg build (libx264
# included). It bundles no ffprobe, so only use it for ffmpeg itself.
if name == "ffmpeg":
try:
import imageio_ffmpeg
return imageio_ffmpeg.get_ffmpeg_exe()
except Exception:
pass
return None return None

View File

@ -39,12 +39,8 @@ if (!packName) {
} else { } else {
// Base prefix covers deployments behind a sub-path reverse proxy // Base prefix covers deployments behind a sub-path reverse proxy
// ("/comfyui/extensions/<pack>/..." -> "/comfyui/"); the pack's own // ("/comfyui/extensions/<pack>/..." -> "/comfyui/"); the pack's own
// static mount sits at "<base><pack>/". For the standard deployment // static mount sits at "<base><pack>/".
// there is no prefix and base is exactly "/" - it must NEVER be "//", const base = '/' + segments.slice(0, extensionsIndex).join('/') + '/';
// which the browser would parse as a protocol-relative URL and try to
// resolve "<pack>" as a DNS hostname (net::ERR_NAME_NOT_RESOLVED).
const prefix = segments.slice(0, extensionsIndex);
const base = prefix.length ? '/' + prefix.join('/') + '/' : '/';
// Top-level await: extension scripts are awaited before ComfyUI registers // Top-level await: extension scripts are awaited before ComfyUI registers
// node definitions, so the editor's registerExtension()/custom-widget // node definitions, so the editor's registerExtension()/custom-widget
// hooks stay in the normal loading order. // hooks stay in the normal loading order.