Compare commits
3 Commits
main
...
phase2-ben
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
033a59ece9 | ||
|
|
eefca88d35 | ||
|
|
5d81069400 |
3
.gitignore
vendored
3
.gitignore
vendored
@ -26,3 +26,6 @@ models/audio_vae/audio_vae_fp32.safetensors
|
||||
models/clip/MM-VL.safetensors
|
||||
models/diffusion/Moxie-Multimedia.safetensors
|
||||
models/vae/video_vae_.safetensors
|
||||
|
||||
# bench render outputs / job state
|
||||
bench/work/
|
||||
|
||||
16
CHANGELOG.md
16
CHANGELOG.md
@ -6,19 +6,9 @@
|
||||
|
||||
### ⚡ Improvements
|
||||
|
||||
- **Updater**: The pack now ships a **`update.bat`** one-click updater for Windows. Place it in the pack folder (next to `__init__.py`) and double-click: it points the folder's git remote at the HuggingFace repository (fixing installs whose `origin` still points at a stale URL), fetches and force-syncs `main`, clears leftover files from older versions, and downloads the model weights through git-lfs — with a clear warning and install pointer when git-lfs is missing. It then self-verifies: it prints the current build id (`git log -1 --oneline`), counts the odd-latent fix marker in `nodes/project.py` (3 hits = fixed build), and flags any model file that is still a git-lfs placeholder. ComfyUI must still be restarted after updating.
|
||||
|
||||
- **Moxie MultiTrack Project Slow**: Removed the experimental node again, together with its `upscale_model` / `tiling` widgets and the even-latent first-pass snap — the shared project-node internals are restored byte-for-byte to their certified behavior. Workflows that referenced the Slow node need rewiring to **Moxie MultiTrack Project**.
|
||||
|
||||
- **MultiTrack Project**: The inter-segment context overlap is now configurable. The new `context_frames` widget (5 / 22 / 39 / 56 frames, default 22) chooses how many frames of the previous segment are copied in as continuation context for segments using Context or Context Swap continuity — previously this was fixed at 22 frames in code. Longer values hand off motion more smoothly but generate that many extra frames per continued segment (each continuation also generates 12 grid-alignment tail frames that are trimmed after decode). Values are the H3 temporal-grid lengths already offered by the Moxie context sampler nodes; existing workflows keep 22 and behave exactly as before. The same widget (between `steps` and `seed`) is also exposed on **Moxie MultiTrack Project** — its baked euler/beta sampling stack is unchanged, so end users still only need `steps` and `seed` unless they touch the overlap.
|
||||
|
||||
- **Moxie Multimedia Loader**: Removed the automatic LoRA application entirely. The 3-step distill (speedup) weights are now baked into the diffusion model itself, so the loader no longer reads `models/loras` — a leftover file there is silently ignored, and the `models/loras` folder can be deleted. The node schema, outputs, and load order (diffusion model → attention backend → CLIP → VAEs) are unchanged; workflows saved earlier keep working. The `Moxie Project` steps tooltip now references the baked-in distill instead of the LoRA.
|
||||
- **FFmpeg resolution & diagnostics**: The pack now finds FFmpeg on far more Windows setups and tells you exactly what is wrong when it cannot. When `ffmpeg` is not on the ComfyUI process's PATH (PATH edits only reach processes started afterwards), it is additionally looked up in the classic `C:\ffmpeg\bin`, the ComfyUI/portable folders (including `ffmpeg\bin` next to the run bat), winget/scoop/chocolatey shim locations, and finally the ffmpeg binary bundled with ComfyUI's own `imageio-ffmpeg` dependency. When composing an H3 project still fails, the error dialog now names the concrete cause instead of a generic message: the missing executable, the exact missing segment/locked-audio file paths, odd canvas sizes, or the tail of FFmpeg's own stderr output (e.g. `Unknown encoder 'libx264'`).
|
||||
- **Installation**: The pack now ships a **`requirements.txt`**, so ComfyUI Manager installs the dependencies automatically and manual/portable installs have a one-line command (`python_embeded\python.exe -m pip install -r ...` for ComfyUI Portable — see the new "Installing dependencies" section in the README). Because this pack targets **NVIDIA RTX 3060+ GPUs only**, every feature dependency is mandatory: `soundfile`, `jieba`, `huggingface_hub`, `ffmpeg-python`, `opencv-python`, `imageio` (smart split), `nvidia-vfx` (RTX VSR upscale), `openai-whisper` and `qwen-asr` (subtitle recognition), and `voxcpm` (speech). No optional extras remain — previously these were installed on demand with error messages naming the missing package. The file also routes `nvidia-vfx` to NVIDIA's own index (`--extra-index-url https://pypi.nvidia.com/` + `--only-binary=nvidia-vfx`): the PyPI copy is a stub sdist whose `wheel_stub` build backend can abort the whole install (`Cannot import 'wheel_stub.buildapi'`), while NVIDIA's index serves real prebuilt wheels.
|
||||
- **Installation**: Removed the bundled `download_models.sh` downloader. All model files live in the Hugging Face version of the repository — cloning it downloads the complete model set, so there is no separate downloader step (the README's Models section now explains this).
|
||||
- **Moxie Bench (new)**: A standalone Python + web harness for the fixed Moxie pipeline (`bench/`) — the first step toward running the pack outside ComfyUI as a Tauri 2 sidecar. It locks to the base workflow's nine-stage graph (loader, LoRA, preview override, multitrack editor, project, combine, RTX upscale, create/save) and ships two interchangeable backends: a self-contained mock renderer (numpy + ffmpeg, deterministic per prompt/seed, real mp4 outputs on both the native and upscaled branches) and a ComfyUI bridge that drives a running ComfyUI over `/prompt` with the substituted workflow. The multitrack timeline editor UI is ported as-is (`frontend/src/bench/` shim mounts the same React components against a REST + WebSocket API), the server (FastAPI) exposes `/api/render`, `/api/jobs`, `WS /api/ws` progress plus ComfyUI-compatible `/view`, `/upload/image`, and `/easy-media/*` aliases, and `bench/moxie_bench.spec` + `bench/tauri/README.md` cover PyInstaller packaging and sidecar wiring. Run `python bench/run.py --port 8787`.
|
||||
- **MultiTrack Editor**: The editor frontend was **reverted to the pristine upstream v1.3.3 source** — the MiniMax/Moxie format display-label mapping and every workaround built around it were removed. The shipped editor bundle is now built from that pristine source, so its behavior is exactly the battle-tested upstream editor (Shot/Context continuity, reference-size, audio lock, and MiniMax prompt placeholders all key on the native `MiniMax` format value again).
|
||||
- **Editor delivery fix**: A fresh browser could previously end up without the editor bundle entirely. ComfyUI core lets a pack's `WEB_DIRECTORY` overwrite its extension web-dir registration, so the browser's `/extensions` listing only ever contained this pack's helper scripts — never the editor entry — and the editor only appeared from stale browser cache. A small static loader (`web/js/editor-loader.js`, served with the helpers) now imports the editor entry from the pack's own route, which serves `index.js` with `no-cache` so the entry is always revalidated while its content-hashed chunks stay cacheable. After updating: restart ComfyUI and reload the page once; no cache-clearing gymnastics required.
|
||||
- **Editor loader hotfix**: The loader's base-prefix computation produced `//` on standard installs (no reverse-proxy sub-path), which browsers parse as a protocol-relative URL — they then tried to resolve the pack folder name as a DNS hostname and the editor entry never loaded (`net::ERR_NAME_NOT_RESOLVED` on `moxie-multimedia/index.js` in the browser console). The base is now exactly `/` when there is no sub-path prefix. If the MultiTrack Editor node shows a plain placeholder instead of the timeline UI, update the pack and hard-refresh the browser.
|
||||
- **MultiTrack Editor / MultiTrack Project**: **T2V resolves natively again** (`t2v` stays `t2v`, matching upstream) — the model now handles text-to-video with the updated prompting, so the earlier T2V→R2V normalization was removed from the editor type computation, `h3_task_type`, `_multitrack_task_type`, and the generation-mode mapping. The reference-mode-with-zero-media cloud fallback was removed accordingly (reference mode raises for missing media again, as upstream).
|
||||
- **Moxie Multimedia Loader**: The loader now **looks like it loads a single model**. Only the diffusion-model picker (plus the attention backend) is visible; the LoRA, video VAE, audio VAE, and CLIP widgets were removed and their files are auto-selected from the pack's `models/loras`, `models/vae`, `models/audio_vae`, and `models/clip` folders (first file in sorted order, matching the bundled model set). The LoRA is applied automatically with strength 1.0; remove the file from `models/loras` to disable it. Missing auxiliary models raise the same clear setup error as before. Workflows saved before this change keep working — extra widget values are ignored.
|
||||
- **MultiTrack Project / Moxie Project Video Combine**: Single-pass projects now condition and sample at half the editor-selected resolution internally. The project timeline is composed at the segments' own size, and Moxie Project Video Combine restores the exact editor-selected resolution in pixel space before any output leaves the node (RTX Video Super Resolution via the optional `nvidia-vfx` package when available, with an automatic Lanczos fallback otherwise) — so every output carries the full editor resolution regardless of template wiring. The editor resolution therefore stays the final output resolution while the sampling chain runs smaller and faster. Dual-pass, SelfLift, and passthrough modes are unchanged.
|
||||
@ -29,10 +19,6 @@
|
||||
- **Moxie Multimedia Loader**: Locked the loader to the MiniMax H3 model set. The `clip_type` dropdown (which previously exposed every CLIP recipe in ComfyUI) was removed; the H3 Qwen3-VL text encoder is now always loaded with the `minimax` CLIP recipe. Any older ComfyUI build without `CLIPType.MINIMAX` raises a clear startup error instead of silently loading the wrong encoder.
|
||||
- **Moxie Multimedia Loader / Moxie MultiTrack Project**: Removed the legacy `FAST_MODEL_LOADER` (`model_loader`) bundle. The loader now outputs exactly `model`, `clip`, `video_vae`, and `audio_vae` (with the attention backend and LoRA already applied to the model), and the Moxie MultiTrack Project takes those four outputs as required direct inputs instead of a bundled loader. `Moxie Preview Override` likewise dropped its `model_loader` pass-through slots and now patches the `model` input directly. The generic `easy multitrackProject` node keeps its `model_loader` input for non-Moxie workflows.
|
||||
|
||||
### 🐛 Bug Fixes
|
||||
|
||||
- **MultiTrack Project / MiniMax H3 nodes**: Fixed the first-step `patchify_video` crash (`RuntimeError: shape '[1, 24, 1, 1, ...]' is invalid for input of size N`) that hit 11 of the 17 editor resolution dropdown entries in single-pass mode — including the app's default 960x544 and the auto fallback 544x960. The hidden half-resolution first pass floored off the H3 32px canvas grid (544x960 halved to 272x480 encodes to a 17x30 latent whose odd side comfy core cannot split into 2x2 patches); the first pass now floors onto the grid (544x960 samples at 256x480, and Moxie Project Video Combine still restores the exact editor resolution, so output size and quality are unchanged). The MiniMax H3 conditioning entry points (`MiniMaxH3ImageToVideoFallback`, `MiniMaxH3ReferenceToVideoFallback`, `EasyMiniMaxH3ToVideo`) align their canvas the same way, which also fixes dual-pass and SelfLift at any editor size that is not a multiple of 32. The misleading "No H3 task segments are available from segment_start_number." error now distinguishes an empty timeline from an out-of-range `segment_start_number` and says what to do about each. **After updating, restart ComfyUI once** — a stale off-grid latent cached in the temp directory otherwise survives the pack update and the crash appears to persist.
|
||||
|
||||
## [1.3.3] - 2026-09-25
|
||||
|
||||
### ✨ New Features
|
||||
|
||||
48
README.md
48
README.md
@ -1,17 +1,3 @@
|
||||
---
|
||||
license: gpl-3.0
|
||||
tags:
|
||||
- comfyui
|
||||
- text-to-video
|
||||
- video-generation
|
||||
- multi-track
|
||||
- audio
|
||||
- text-to-speech
|
||||
- automatic-speech-recognition
|
||||
- subtitles
|
||||
- nvidia-rtx
|
||||
---
|
||||
|
||||
<div align="center">
|
||||
|
||||
<img src="https://github.com/user-attachments/assets/fb602a3c-4a2a-48da-8c44-d36417f4633b" height="120">
|
||||
@ -21,6 +7,9 @@ Timeline-driven video generation for ComfyUI — arrange tasks, references, audi
|
||||
|
||||
</div>
|
||||
|
||||
> [!NOTE]
|
||||
> **About the model** — the Moxie Multimedia model is a version of the MiniMax H3 model that has gone through the Moxigen method.
|
||||
|
||||
## 📦 Installation
|
||||
|
||||
> [!IMPORTANT]
|
||||
@ -33,41 +22,24 @@ cd Your_ComfyUI_Path/custom_nodes
|
||||
git clone https://huggingface.co/turtle89431/Moxie-Multimedia
|
||||
```
|
||||
|
||||
### Installing dependencies
|
||||
|
||||
**ComfyUI Manager** installs `requirements.txt` automatically. If you cloned the repository manually, install the dependencies yourself:
|
||||
|
||||
**ComfyUI Portable (Windows)** — run from the portable root folder (the one containing `run_nvidia_gpu.bat`):
|
||||
|
||||
```bat
|
||||
cd Your_ComfyUI_windows_portable
|
||||
.\python_embeded\python.exe -m pip install -r .\ComfyUI\custom_nodes\Moxie-Multimedia\requirements.txt
|
||||
```
|
||||
|
||||
**Regular ComfyUI (venv / system Python):**
|
||||
|
||||
```bash
|
||||
pip install -r Your_ComfyUI_Path/custom_nodes/Moxie-Multimedia/requirements.txt
|
||||
```
|
||||
|
||||
Restart ComfyUI afterwards.
|
||||
|
||||
> [!NOTE]
|
||||
> The pack targets **NVIDIA RTX 3060+ GPUs only**, so every feature dependency is mandatory and installed by the command above — including speech generation (`voxcpm`), subtitle recognition (`openai-whisper`, `qwen-asr`), and the RTX Video Super Resolution upscale (`nvidia-vfx`). Expect a sizable first install: `qwen-asr` pins `transformers==4.57.6` (pip may upgrade it inside your ComfyUI environment) and `voxcpm` needs `torch>=2.5.0`, so keep ComfyUI reasonably current. `nvidia-vfx` is pulled from NVIDIA's own package index (`pypi.nvidia.com`, wired up inside `requirements.txt`) because the PyPI copy is a stub that fails to build on some setups — if the command is interrupted, re-run it and it resumes where it stopped.
|
||||
|
||||
### Models
|
||||
|
||||
All model files ship **inside the repo** — cloning it into `custom_nodes` is all that's needed. The loader finds everything automatically in the pack's `models/` folders:
|
||||
|
||||
| File | Location |
|
||||
|---|---|
|
||||
| Diffusion model (`Moxie-Multimedia.safetensors`, distill weights baked in) | `models/diffusion/` |
|
||||
| Diffusion model (`Moxie-Multimedia.safetensors`) | `models/diffusion/` |
|
||||
| Text encoder (`MM-VL.safetensors`) | `models/clip/` |
|
||||
| Video VAE (bundled file) | `models/vae/` |
|
||||
| Audio VAE (bundled file) | `models/audio_vae/` |
|
||||
| Turbo LoRA (`MM3step.safetensors`, applied by default) | `models/loras/` |
|
||||
| Preview tiny VAE (`MM-preview.safetensors`) | `models/vae_approx/` |
|
||||
|
||||
All model files live in the Hugging Face version of this repository — cloning it downloads the complete set, and there is no separate downloader step. If you obtained the pack from somewhere else and the `models/` folders are missing or incomplete, re-clone from Hugging Face to get the bundled files.
|
||||
If you obtained the pack without the bundled files, the included downloader fetches and places them (idempotent and resumable — re-run if interrupted):
|
||||
|
||||
```bash
|
||||
bash custom_nodes/MMnode/download_models.sh
|
||||
```
|
||||
|
||||
Restart ComfyUI afterwards.
|
||||
|
||||
|
||||
87
bench/README.md
Normal file
87
bench/README.md
Normal file
@ -0,0 +1,87 @@
|
||||
# Moxie Multimedia Bench
|
||||
|
||||
A standalone Python + web harness for the Moxie Multimedia pipeline — the
|
||||
first step toward running this node pack outside ComfyUI as a sidecar for a
|
||||
Tauri 2 desktop app.
|
||||
|
||||
The bench locks to the fixed pipeline of the base workflow
|
||||
(`bench/base_workflow.json`):
|
||||
|
||||
```
|
||||
MoxieMultimediaLoader -> LoraLoaderModelOnly -> MoxiePreviewOverride
|
||||
-> easy multiTrackEditor -> MoxieProject -> MoxieCombine
|
||||
-> [native branch] CreateVideo -> SaveVideo
|
||||
-> [upscaled branch] RTXVideoSuperResolution -> CreateVideo -> SaveVideo
|
||||
```
|
||||
|
||||
The multitrack timeline editor is the real thing: the bench bundles the same
|
||||
React editor components the ComfyUI node uses, mounted by a thin shim
|
||||
(`frontend/src/bench/`) that talks REST + WebSocket instead of the ComfyUI
|
||||
runtime.
|
||||
|
||||
## Run from source
|
||||
|
||||
```bash
|
||||
pip install -r bench/requirements.txt # fastapi, uvicorn, numpy, pillow
|
||||
python bench/run.py --port 8787 # auto backend (mock here)
|
||||
# open http://127.0.0.1:8787
|
||||
```
|
||||
|
||||
The UI needs a one-time build (a prebuilt copy ships in `bench/ui-dist/`):
|
||||
|
||||
```bash
|
||||
cd frontend && bun install && bun run build:bench
|
||||
```
|
||||
|
||||
## Backends
|
||||
|
||||
| backend | needs | behaviour |
|
||||
|---|---|---|
|
||||
| `mock` | ffmpeg, numpy, pillow | deterministic synthetic frames per (prompt, seed, frame), sine audio bed, lanczos "upscale" to the RTX target, h264 encode. Both output branches produce real mp4s. |
|
||||
| `comfy` | a running ComfyUI + downloaded models | substitutes the plan into the base workflow, converts it to the `/prompt` API format, submits to ComfyUI, polls progress, downloads outputs. |
|
||||
| `auto` | — | `comfy` when reachable, else `mock`. |
|
||||
|
||||
```bash
|
||||
python bench/run.py --backend comfy --comfyui-url http://127.0.0.1:8188
|
||||
```
|
||||
|
||||
## HTTP API
|
||||
|
||||
- `GET /api/health`, `GET /api/config` — liveness, backend availability
|
||||
- `GET /api/workflow` — base workflow summary + editor `track_data`
|
||||
- `GET /api/plan` — resolved render plan (segments, prompts, frames)
|
||||
- `POST /api/render` — queue a render:
|
||||
`{"track_data": <editor JSON>, "overrides": {"width": 1920, "height": 1080,
|
||||
"steps": 4, "seed": 123, "project_name": "MM-x", "backend": "auto"}}`
|
||||
- `GET /api/jobs`, `GET /api/jobs/{id}`, `DELETE /api/jobs/{id}`
|
||||
- `WS /api/ws` — push events: `job_queued`, `execution_start`, `job_progress`,
|
||||
`execution_success`, `execution_error` (mirrors ComfyUI's push model)
|
||||
- `GET /api/view?filename=&type=output&job=` — fetch rendered files
|
||||
- ComfyUI-compatible aliases: `/view`, `/upload/image`, `/easy-media/*`
|
||||
|
||||
Outputs land in `bench/work/outputs/<job_id>/` (`MOXIE_BENCH_WORK` overrides
|
||||
the root). Every render produces both branches: `native` (editor canvas) and
|
||||
`upscaled` (RTX target dimensions).
|
||||
|
||||
## Tauri 2 sidecar
|
||||
|
||||
See `bench/tauri/README.md` — PyInstaller onedir packaging
|
||||
(`bench/moxie_bench.spec`), `tauri.conf.json` resource wiring, spawn + health
|
||||
poll snippet, and the endpoint table.
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
python3 -m pytest bench/tests -q # contract tests (no ComfyUI needed)
|
||||
python3 scripts/test_bench_pipeline.py # full API + WS + mock render e2e
|
||||
cd frontend && bun run test # editor suite incl. bench shim
|
||||
```
|
||||
|
||||
## Phase map
|
||||
|
||||
- **phase 2 (this)** — bench: fixed pipeline, dual backend, timeline UI port,
|
||||
sidecar packaging.
|
||||
- **phase 3 (planned)** — in-process execution backend: load the pack's node
|
||||
classes directly (torch + ComfyUI modules on `sys.path`) instead of driving
|
||||
a separate ComfyUI server; `bench/app/backends/comfy.py` keeps the same
|
||||
interface.
|
||||
3
bench/app/__init__.py
Normal file
3
bench/app/__init__.py
Normal file
@ -0,0 +1,3 @@
|
||||
"""Moxie Multimedia bench."""
|
||||
|
||||
__version__ = "0.1.0"
|
||||
19
bench/app/backends/__init__.py
Normal file
19
bench/app/backends/__init__.py
Normal file
@ -0,0 +1,19 @@
|
||||
"""Backend registry."""
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
def comfy_bridge_available(cfg) -> bool:
|
||||
from . import comfy
|
||||
|
||||
return comfy.comfy_bridge_available(cfg)
|
||||
|
||||
|
||||
def available(cfg) -> dict:
|
||||
"""Probe both backends for /api/config."""
|
||||
info = {"mock": {"available": True}, "comfy": {"available": False}}
|
||||
try:
|
||||
ok = comfy_bridge_available(cfg)
|
||||
info["comfy"] = {"available": ok, "url": cfg.comfyui_url}
|
||||
except Exception as exc: # noqa: BLE001
|
||||
info["comfy"] = {"available": False, "error": str(exc)}
|
||||
return info
|
||||
351
bench/app/backends/comfy.py
Normal file
351
bench/app/backends/comfy.py
Normal file
@ -0,0 +1,351 @@
|
||||
"""ComfyUI bridge backend (real execution, phase-2 scope).
|
||||
|
||||
Drives a running ComfyUI over HTTP instead of reimplementing its executor
|
||||
in-process:
|
||||
|
||||
1. take the base workflow (frontend export format, checked in at
|
||||
bench/base_workflow.json),
|
||||
2. substitute the RenderPlan parameters (track_data, resolution, steps,
|
||||
seed, LoRA, upscale target, codecs, project_data),
|
||||
3. convert to the /prompt API format using the input names carried by the
|
||||
export itself (widget names via widgets_values_named, link inputs via
|
||||
the node's inputs[] entries),
|
||||
4. POST /prompt, poll /history/<id> for per-node progress and outputs,
|
||||
5. download produced files via /view into the job dir.
|
||||
|
||||
Phase 3 replaces this with an in-process executor (same interface).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from ..tracks import RenderPlan
|
||||
from ..workflow import EDITOR_TYPE, PipelineSpec
|
||||
from .mock import ProgressCb, RenderResult
|
||||
|
||||
_CONTROL_WIDGET_NAMES = {"randomize", "control_after_generate", "control_before_generate"}
|
||||
|
||||
|
||||
class ComfyBridgeError(RuntimeError):
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HTTP helpers (stdlib only)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _url(cfg_url: str, path: str, **params: Any) -> str:
|
||||
base = cfg_url.rstrip("/")
|
||||
url = f"{base}{path}"
|
||||
if params:
|
||||
url += "?" + urllib.parse.urlencode(params)
|
||||
return url
|
||||
|
||||
|
||||
def _get_json(url: str, timeout: float = 10.0) -> Any:
|
||||
with urllib.request.urlopen(url, timeout=timeout) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def _post_json(url: str, payload: dict, timeout: float = 30.0) -> Any:
|
||||
data = json.dumps(payload).encode("utf-8")
|
||||
req = urllib.request.Request(
|
||||
url, data=data, headers={"Content-Type": "application/json"}, method="POST"
|
||||
)
|
||||
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||
return json.loads(resp.read().decode("utf-8"))
|
||||
|
||||
|
||||
def comfy_bridge_available(cfg) -> bool:
|
||||
"""True when a ComfyUI answers at cfg.comfyui_url."""
|
||||
try:
|
||||
_get_json(_url(cfg.comfyui_url, "/system_stats"), timeout=3.0)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Workflow substitution + /prompt conversion
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_RESOLUTION_PRESETS = {
|
||||
(1920, 1080): "1920 x 1080 (16:9)",
|
||||
(1088, 1920): "1088 x 1920 (9:16)",
|
||||
(1080, 1920): "1080 x 1920 (9:16)",
|
||||
(1280, 720): "1280 x 720 (16:9)",
|
||||
(720, 1280): "720 x 1280 (9:16)",
|
||||
(832, 480): "832 x 480 (16:9)",
|
||||
(480, 832): "480 x 832 (9:16)",
|
||||
(1024, 1024): "1024 x 1024 (1:1)",
|
||||
}
|
||||
|
||||
|
||||
def substitute(spec: PipelineSpec, plan: RenderPlan) -> dict[str, Any]:
|
||||
"""Return a deep copy of the raw workflow with plan values applied."""
|
||||
raw = copy.deepcopy(spec.raw)
|
||||
nodes = {int(n["id"]): n for n in raw["nodes"]}
|
||||
|
||||
def named(nid: int) -> dict[str, Any]:
|
||||
n = nodes[nid]
|
||||
vals = n.setdefault("widgets_values_named", {})
|
||||
vals.setdefault("widgets_values", None)
|
||||
return vals
|
||||
|
||||
# editor: track_data + resolution
|
||||
named(spec.editor_node_id)["track_data"] = json.dumps(
|
||||
_track_data_for_submission(spec, plan)
|
||||
)
|
||||
preset = _RESOLUTION_PRESETS.get((plan.width, plan.height))
|
||||
named(spec.editor_node_id)["resolution"] = (
|
||||
preset if preset else {"width": plan.width, "height": plan.height}
|
||||
)
|
||||
|
||||
# project: name, steps, seed
|
||||
proj = spec.stages.get("MoxieProject")
|
||||
if proj:
|
||||
vals = named(proj.node_id)
|
||||
vals["project_name"] = plan.project_name
|
||||
vals["steps"] = plan.steps
|
||||
vals["seed"] = plan.seed
|
||||
|
||||
# lora
|
||||
lora = spec.stages.get("LoraLoaderModelOnly")
|
||||
if lora and plan.lora:
|
||||
vals = named(lora.node_id)
|
||||
vals["lora_name"] = plan.lora
|
||||
vals["strength_model"] = plan.lora_strength
|
||||
|
||||
# rtx upscale target
|
||||
rtx = spec.stages.get("RTXVideoSuperResolution")
|
||||
if rtx and plan.upscale_width:
|
||||
vals = named(rtx.node_id)
|
||||
vals["resize_type"] = "target dimensions"
|
||||
vals["resize_type.width"] = plan.upscale_width
|
||||
vals["resize_type.height"] = plan.upscale_height
|
||||
vals["quality"] = plan.upscale_quality
|
||||
|
||||
# combine project_data carries the canvas + frame rate
|
||||
combine = spec.stages.get("MoxieCombine")
|
||||
if combine:
|
||||
vals = named(combine.node_id)
|
||||
pd_raw = vals.get("project_data", "{}")
|
||||
pd = json.loads(pd_raw) if isinstance(pd_raw, str) else dict(pd_raw or {})
|
||||
pd.update({
|
||||
"project_name": plan.project_name,
|
||||
"width": plan.width,
|
||||
"height": plan.height,
|
||||
"frame_rate": plan.frame_rate,
|
||||
})
|
||||
vals["project_data"] = json.dumps(pd)
|
||||
|
||||
# save prefixes get a per-run folder
|
||||
for branch, stage in spec.save_nodes.items():
|
||||
vals = named(stage.node_id)
|
||||
if "filename_prefix" in vals:
|
||||
vals["filename_prefix"] = f"moxie-bench/{branch}"
|
||||
|
||||
return raw
|
||||
|
||||
|
||||
def _track_data_for_submission(spec: PipelineSpec, plan: RenderPlan) -> dict[str, Any]:
|
||||
"""Track data as edited in the bench UI; the editor widget value wins."""
|
||||
raw = spec.widget(EDITOR_TYPE, "track_data")
|
||||
data = json.loads(raw) if isinstance(raw, str) else dict(raw or {})
|
||||
data["frame_rate"] = plan.frame_rate
|
||||
data.setdefault("total_length", plan.total_length)
|
||||
return data
|
||||
|
||||
|
||||
def to_prompt_format(raw: dict[str, Any]) -> dict[str, dict[str, Any]]:
|
||||
"""Frontend-format workflow -> /prompt API format.
|
||||
|
||||
Widget values come from widgets_values_named (names are the schema input
|
||||
names); link inputs come from the node's inputs[] entries. A widget that
|
||||
is wired to a link (marker key "widget" plus a link id) takes the link.
|
||||
"""
|
||||
nodes = raw.get("nodes") or []
|
||||
links: dict[int, tuple[int, int]] = {}
|
||||
for lk in raw.get("links") or []:
|
||||
links[int(lk[0])] = (int(lk[1]), int(lk[2]))
|
||||
|
||||
prompt: dict[str, dict[str, Any]] = {}
|
||||
for n in nodes:
|
||||
nid = str(int(n["id"]))
|
||||
entry: dict[str, Any] = {}
|
||||
for inp in n.get("inputs") or []:
|
||||
if not isinstance(inp, dict):
|
||||
continue
|
||||
name = inp.get("name")
|
||||
link = inp.get("link")
|
||||
if name is None or link is None:
|
||||
continue
|
||||
frm = links.get(int(link))
|
||||
if frm is None:
|
||||
continue
|
||||
entry[name] = [str(frm[0]), frm[1]]
|
||||
named = n.get("widgets_values_named") or {}
|
||||
linked_widget_names = {
|
||||
inp.get("widget", {}).get("name")
|
||||
for inp in n.get("inputs") or []
|
||||
if isinstance(inp, dict) and inp.get("link") is not None
|
||||
and isinstance(inp.get("widget"), dict)
|
||||
}
|
||||
for name, value in named.items():
|
||||
if name in ("widgets_values",) or name in _CONTROL_WIDGET_NAMES:
|
||||
continue
|
||||
if name in linked_widget_names and name in entry:
|
||||
continue # wired input wins over the stored widget value
|
||||
entry[name] = value
|
||||
prompt[nid] = {"class_type": n.get("type"), "inputs": entry}
|
||||
return prompt
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Render
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def render(
|
||||
spec: PipelineSpec,
|
||||
plan: RenderPlan,
|
||||
cfg,
|
||||
job_dir: Path,
|
||||
progress: ProgressCb,
|
||||
) -> RenderResult:
|
||||
result = RenderResult()
|
||||
raw = substitute(spec, plan)
|
||||
prompt = to_prompt_format(raw)
|
||||
node_types = {str(int(n["id"])): n.get("type", "") for n in raw["nodes"]}
|
||||
|
||||
progress("MoxieMultimediaLoader", 5.0, f"comfy: submitting prompt to {cfg.comfyui_url}")
|
||||
try:
|
||||
resp = _post_json(_url(cfg.comfyui_url, "/prompt"), {"prompt": prompt})
|
||||
except urllib.error.HTTPError as exc:
|
||||
body = exc.read().decode("utf-8", errors="replace")
|
||||
raise ComfyBridgeError(f"ComfyUI rejected the prompt ({exc.code}): {body[:600]}") from exc
|
||||
except urllib.error.URLError as exc:
|
||||
raise ComfyBridgeError(
|
||||
f"Cannot reach ComfyUI at {cfg.comfyui_url} ({exc.reason}). "
|
||||
"Start it with: python main.py --listen 127.0.0.1 --port 8188"
|
||||
) from exc
|
||||
prompt_id = resp.get("prompt_id")
|
||||
if not prompt_id:
|
||||
raise ComfyBridgeError(f"ComfyUI returned no prompt_id: {resp}")
|
||||
|
||||
progress("MoxieProject", 10.0, f"comfy: queued ({prompt_id})")
|
||||
|
||||
stage_by_node = {str(nid): ntype for nid, ntype in node_types.items()}
|
||||
save_branches = {str(st.node_id): branch for branch, st in spec.save_nodes.items()}
|
||||
|
||||
deadline = time.time() + 60 * 60 # 1h safety stop
|
||||
poll = 1.0
|
||||
seen_progress_max = 0
|
||||
while time.time() < deadline:
|
||||
time.sleep(poll)
|
||||
try:
|
||||
hist = _get_json(_url(cfg.comfyui_url, f"/history/{prompt_id}"), timeout=15.0)
|
||||
except Exception as exc: # transient network hiccups
|
||||
result.log(f"history poll failed: {exc}")
|
||||
continue
|
||||
entry = hist.get(prompt_id)
|
||||
if entry is None:
|
||||
# still queued or running
|
||||
continue
|
||||
status = entry.get("status") or {}
|
||||
messages = status.get("messages") or []
|
||||
for msg in messages:
|
||||
mtype, mdata = msg[0], (msg[1] if len(msg) > 1 else {})
|
||||
if mtype == "executing" and mdata.get("node"):
|
||||
ntype = stage_by_node.get(str(mdata["node"]), "")
|
||||
if ntype:
|
||||
progress(ntype, max(12.0, result_progress(ntype)), f"comfy: {ntype}")
|
||||
elif mtype == "progress":
|
||||
try:
|
||||
val = float(mdata.get("value", 0)) / max(1.0, float(mdata.get("max", 1)))
|
||||
except (TypeError, ValueError):
|
||||
val = 0.0
|
||||
seen_progress_max = max(seen_progress_max, val)
|
||||
# sample/VAE progress mapped into the 15..92 window
|
||||
progress("MoxieProject", 15.0 + 70.0 * val,
|
||||
f"comfy: sampling {int(val * 100)}%",
|
||||
{"step": mdata.get("value"), "steps": mdata.get("max")})
|
||||
elif mtype == "execution_error":
|
||||
raise ComfyBridgeError(
|
||||
f"ComfyUI execution failed on node "
|
||||
f"{mdata.get('node_type')} ({mdata.get('node_id')}): "
|
||||
f"{mdata.get('exception_message')}"
|
||||
)
|
||||
if status.get("completed"):
|
||||
break
|
||||
if status.get("status_str") == "error":
|
||||
raise ComfyBridgeError("ComfyUI reported an error for this prompt")
|
||||
|
||||
if not status.get("completed"):
|
||||
raise ComfyBridgeError("ComfyUI did not finish in time")
|
||||
|
||||
# collect outputs
|
||||
outputs = entry.get("outputs") or {}
|
||||
files: dict[str, dict[str, Any]] = {}
|
||||
for nid, node_out in outputs.items():
|
||||
branch = save_branches.get(str(nid))
|
||||
for key, items in node_out.items():
|
||||
if not isinstance(items, list):
|
||||
continue
|
||||
for item in items:
|
||||
if not isinstance(item, dict) or "filename" not in item:
|
||||
continue
|
||||
if branch:
|
||||
files.setdefault(branch, dict(item))
|
||||
else:
|
||||
result.log(f"aux output from node {nid}: {key} {item.get('filename')}")
|
||||
|
||||
if not files:
|
||||
raise ComfyBridgeError(f"ComfyUI produced no video outputs: outputs={list(outputs)}")
|
||||
|
||||
for branch, item in files.items():
|
||||
params = {
|
||||
"filename": item["filename"],
|
||||
"subfolder": item.get("subfolder", ""),
|
||||
"type": item.get("type", "output"),
|
||||
}
|
||||
data = _download(_url(cfg.comfyui_url, "/view", **params))
|
||||
dst = job_dir / f"{branch}{Path(item['filename']).suffix or '.mp4'}"
|
||||
dst.write_bytes(data)
|
||||
result.outputs[branch] = {
|
||||
"filename": dst.name,
|
||||
"path": str(dst),
|
||||
"url": f"/api/view?filename={dst.name}&type=output&job={job_dir.name}",
|
||||
"source": item.get("subfolder", ""),
|
||||
"bytes": len(data),
|
||||
}
|
||||
result.log(f"{branch}: {item['filename']} ({len(data)} bytes)")
|
||||
|
||||
progress("SaveVideo", 100.0, "done", {"outputs": result.outputs})
|
||||
return result
|
||||
|
||||
|
||||
def result_progress(stage: str) -> float:
|
||||
order = {
|
||||
"MoxieMultimediaLoader": 8.0,
|
||||
"LoraLoaderModelOnly": 10.0,
|
||||
"MoxiePreviewOverride": 12.0,
|
||||
EDITOR_TYPE: 14.0,
|
||||
"MoxieProject": 40.0,
|
||||
"MoxieCombine": 78.0,
|
||||
"RTXVideoSuperResolution": 85.0,
|
||||
"CreateVideo": 92.0,
|
||||
"SaveVideo": 97.0,
|
||||
}
|
||||
return order.get(stage, 20.0)
|
||||
|
||||
|
||||
def _download(url: str, timeout: float = 120.0) -> bytes:
|
||||
with urllib.request.urlopen(url, timeout=timeout) as resp:
|
||||
return resp.read()
|
||||
324
bench/app/backends/mock.py
Normal file
324
bench/app/backends/mock.py
Normal file
@ -0,0 +1,324 @@
|
||||
"""Self-contained mock backend.
|
||||
|
||||
Renders the fixed pipeline without torch/ComfyUI: deterministic synthetic
|
||||
frames derived from (prompt, seed, frame index), a sine audio bed, PIL
|
||||
lanczos "upscale" for the RTX branch, and ffmpeg h264 encode. Exercises the
|
||||
exact same stage/progress contract as the ComfyUI bridge backend so the UI
|
||||
and the API cannot tell the difference.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import math
|
||||
import shutil
|
||||
import subprocess
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
|
||||
from ..tracks import RenderPlan, Segment
|
||||
|
||||
ProgressCb = Callable[[str, float, str, dict | None], None]
|
||||
|
||||
_VIDEO_CODECS = ("h264", "libx264", "mpeg4")
|
||||
|
||||
|
||||
@dataclass
|
||||
class RenderResult:
|
||||
outputs: dict[str, dict] = field(default_factory=dict)
|
||||
console: list[str] = field(default_factory=list)
|
||||
|
||||
def log(self, line: str) -> None:
|
||||
self.console.append(line)
|
||||
|
||||
|
||||
def _pick_encoder(codec: str) -> str:
|
||||
probe = subprocess.run(
|
||||
["ffmpeg", "-hide_banner", "-encoders"],
|
||||
capture_output=True, text=True, timeout=30,
|
||||
)
|
||||
names = probe.stdout.lower()
|
||||
if codec == "h264" and "libx264" in names:
|
||||
return "libx264"
|
||||
for cand in _VIDEO_CODECS:
|
||||
if cand in names:
|
||||
return cand
|
||||
return "mpeg4"
|
||||
|
||||
|
||||
def _audio_encoder() -> str:
|
||||
probe = subprocess.run(
|
||||
["ffmpeg", "-hide_banner", "-encoders"],
|
||||
capture_output=True, text=True, timeout=30,
|
||||
)
|
||||
return "aac" if " aac" in probe.stdout.lower() else "libmp3lame"
|
||||
|
||||
|
||||
def _segment_hue(segment: Segment, index: int) -> float:
|
||||
digest = hashlib.sha256(f"{segment.prompt}|{index}".encode()).digest()
|
||||
return digest[0] * 360.0 / 255.0
|
||||
|
||||
|
||||
def _segment_freq(segment: Segment, index: int) -> float:
|
||||
digest = hashlib.sha256(f"{segment.prompt}|{index}".encode()).digest()
|
||||
return 280.0 + digest[1] % 5 * 55.0 # 280..500 Hz
|
||||
|
||||
|
||||
def _seed_noise(seed: int, frame: int) -> float:
|
||||
digest = hashlib.sha256(f"{seed}:{frame}".encode()).digest()
|
||||
return digest[0] / 255.0 - 0.5
|
||||
|
||||
|
||||
def _synthesize_chunk(
|
||||
chunk_start: int,
|
||||
frames: int,
|
||||
plan: RenderPlan,
|
||||
w: int,
|
||||
h: int,
|
||||
) -> bytes:
|
||||
"""Deterministic RGB24 bytes for frames [chunk_start, chunk_start+frames)."""
|
||||
import numpy as np
|
||||
|
||||
out = bytearray()
|
||||
yy, xx = np.mgrid[0:h, 0:w].astype(np.float32)
|
||||
xn = xx / max(1, w - 1)
|
||||
yn = yy / max(1, h - 1)
|
||||
seg_by_frame = _frame_to_segment_map(plan)
|
||||
for f in range(chunk_start, chunk_start + frames):
|
||||
seg, seg_index = seg_by_frame.get(f, (None, 0))
|
||||
hue = _segment_hue(seg, seg_index) if seg else 210.0
|
||||
t = f / max(1, plan.frame_rate)
|
||||
# HSV-ish deterministic palette: hue rotates per segment prompt,
|
||||
# luminance carries a moving wave so motion is visible.
|
||||
base = np.zeros((h, w, 3), dtype=np.float32)
|
||||
h1 = hue % 360.0
|
||||
rgb = _hsv_to_rgb(h1 / 360.0, 0.55, 0.6)
|
||||
wave = 0.25 * np.sin(6.0 * math.pi * (xn + 0.35 * t)) + 0.15 * np.sin(
|
||||
4.0 * math.pi * (yn - 0.25 * t)
|
||||
)
|
||||
lum = 0.45 + wave + 0.06 * _seed_noise(plan.seed, f)
|
||||
base[..., 0] = rgb[0] * lum
|
||||
base[..., 1] = rgb[1] * lum
|
||||
base[..., 2] = rgb[2] * lum
|
||||
# progress bar strip at the bottom: frame position inside the timeline
|
||||
pos = f / max(1, plan.total_length or 1)
|
||||
bar_y = int(h * 0.95)
|
||||
base[bar_y:, : int(w * pos), 1] = 0.95
|
||||
out += (
|
||||
np.clip(base * 255.0, 0, 255).astype(np.uint8).tobytes()
|
||||
)
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _frame_to_segment_map(plan: RenderPlan) -> dict[int, tuple[Segment, int]]:
|
||||
mapping: dict[int, tuple[Segment, int]] = {}
|
||||
for idx, seg in enumerate(plan.segments):
|
||||
for f in range(seg.start_frame, seg.end_frame):
|
||||
mapping[f] = (seg, idx)
|
||||
return mapping
|
||||
|
||||
|
||||
def _hsv_to_rgb(h: float, s: float, v: float) -> tuple[float, float, float]:
|
||||
import colorsys
|
||||
|
||||
r, g, b = colorsys.hsv_to_rgb(h % 1.0, s, v)
|
||||
return r, g, b
|
||||
|
||||
|
||||
def _s16le_audio(plan: RenderPlan, sample_rate: int = 44100) -> bytes:
|
||||
"""Stereo s16le sine bed; per-segment tone, silence for muted segments."""
|
||||
import numpy as np
|
||||
|
||||
total = int(sample_rate * plan.duration_seconds)
|
||||
if total <= 0:
|
||||
return b""
|
||||
wave = np.zeros(total, dtype=np.float32)
|
||||
sr = sample_rate
|
||||
for idx, seg in enumerate(plan.segments):
|
||||
if seg.muted or not seg.prompt:
|
||||
continue
|
||||
a, b = seg.start_frame, seg.end_frame
|
||||
i0 = int(a / max(1, plan.frame_rate) * sr)
|
||||
i1 = min(total, int(b / max(1, plan.frame_rate) * sr))
|
||||
if i1 <= i0:
|
||||
continue
|
||||
t = np.arange(i1 - i0, dtype=np.float32) / sr
|
||||
freq = _segment_freq(seg, idx)
|
||||
gain = 0.12 * (10 ** (seg.volume_db / 20.0))
|
||||
wave[i0:i1] += gain * np.sin(2 * math.pi * freq * t)
|
||||
stereo = np.repeat(np.clip(wave, -1, 1)[:, None], 2, axis=1)
|
||||
return (stereo * 32767.0).astype("<i2").tobytes()
|
||||
|
||||
|
||||
def render(plan: RenderPlan, work_dir: Path, progress: ProgressCb) -> RenderResult:
|
||||
"""Run the mock pipeline. Returns native + upscaled output descriptors."""
|
||||
import numpy as np
|
||||
from PIL import Image
|
||||
|
||||
result = RenderResult()
|
||||
t0 = time.time()
|
||||
work_dir.mkdir(parents=True, exist_ok=True)
|
||||
if shutil.which("ffmpeg") is None:
|
||||
raise RuntimeError("ffmpeg not found on PATH - the mock backend needs it")
|
||||
|
||||
w, h = plan.width, plan.height
|
||||
frames = plan.total_length or sum(s.frame_count for s in plan.segments)
|
||||
if frames <= 0:
|
||||
raise RuntimeError("timeline has no frames to render")
|
||||
|
||||
def stage(name: str, pct: float, msg: str, data: dict | None = None) -> None:
|
||||
progress(name, pct, msg, data)
|
||||
|
||||
# stages 1-3: loader / lora / preview (contract parity with the real chain)
|
||||
stage("MoxieMultimediaLoader", 4.0, "mock: loading Moxie-Multimedia model set")
|
||||
stage("LoraLoaderModelOnly", 9.0, f"mock: LoRA {plan.lora or 'none'} @ {plan.lora_strength}")
|
||||
stage("MoxiePreviewOverride", 12.0, "mock: preview override active")
|
||||
|
||||
stage("easy multiTrackEditor", 15.0,
|
||||
f"mock: timeline {w}x{h} @ {plan.frame_rate}fps, "
|
||||
f"{len(plan.segments)} task segment(s)")
|
||||
|
||||
encoder = _pick_encoder(plan.codec)
|
||||
aenc = _audio_encoder()
|
||||
result.log(f"encoder: {encoder}, audio: {aenc}")
|
||||
|
||||
native_path = work_dir / "native.mp4"
|
||||
upscaled_path = work_dir / "upscaled.mp4"
|
||||
do_upscale = bool(plan.upscale_width and plan.upscale_height)
|
||||
|
||||
audio_bytes = _s16le_audio(plan)
|
||||
|
||||
def open_enc(out_path: Path, ow: int, oh: int) -> subprocess.Popen:
|
||||
import os as _os
|
||||
|
||||
cmd = [
|
||||
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
|
||||
"-f", "rawvideo", "-pix_fmt", "rgb24",
|
||||
"-s", f"{ow}x{oh}", "-r", str(plan.frame_rate), "-i", "pipe:0",
|
||||
"-f", "s16le", "-ar", "44100", "-ac", "2",
|
||||
]
|
||||
r, w = _os.pipe()
|
||||
# give ffmpeg the READ end (r); parent writes audio into w
|
||||
cmd += ["-i", f"/dev/fd/{r}", "-map", "0:v", "-map", "1:a",
|
||||
"-c:v", encoder, "-pix_fmt", "yuv420p",
|
||||
"-movflags", "+faststart",
|
||||
"-c:a", aenc, "-b:a", "128k", "-shortest",
|
||||
str(out_path)]
|
||||
proc = subprocess.Popen(
|
||||
cmd, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL,
|
||||
stderr=subprocess.PIPE, pass_fds=(r,),
|
||||
)
|
||||
proc._audio_wfd = w # type: ignore[attr-defined]
|
||||
_os.close(r)
|
||||
return proc
|
||||
|
||||
def feed_audio(proc: subprocess.Popen) -> None:
|
||||
# Audio may exceed the 64KB pipe buffer; pump it from a thread so the
|
||||
# video stdin feed never deadlocks against ffmpeg's startup probing.
|
||||
import os as _os
|
||||
import threading as _threading
|
||||
|
||||
def _pump() -> None:
|
||||
try:
|
||||
view = memoryview(audio_bytes)
|
||||
while view:
|
||||
n = _os.write(proc._audio_wfd, view[:65536]) # type: ignore[attr-defined]
|
||||
view = view[n:]
|
||||
except (BrokenPipeError, OSError):
|
||||
pass
|
||||
finally:
|
||||
try:
|
||||
_os.close(proc._audio_wfd) # type: ignore[attr-defined]
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
_threading.Thread(target=_pump, daemon=True).start()
|
||||
|
||||
seg_total = sum(s.frame_count for s in plan.segments)
|
||||
seg_done = 0
|
||||
chunk = 24
|
||||
|
||||
native_enc = open_enc(native_path, w, h)
|
||||
upscale_enc = open_enc(upscaled_path, plan.upscale_width, plan.upscale_height) if do_upscale else None
|
||||
|
||||
def _enc_err(proc: subprocess.Popen) -> str:
|
||||
try:
|
||||
return proc.stderr.read().decode(errors="replace")[-500:]
|
||||
except Exception: # noqa: BLE001
|
||||
return "(stderr unavailable)"
|
||||
|
||||
try:
|
||||
feed_audio(native_enc)
|
||||
if upscale_enc:
|
||||
feed_audio(upscale_enc)
|
||||
|
||||
f = 0
|
||||
while f < frames:
|
||||
n = min(chunk, frames - f)
|
||||
raw = _synthesize_chunk(f, n, plan, w, h)
|
||||
try:
|
||||
native_enc.stdin.write(raw)
|
||||
except BrokenPipeError as exc:
|
||||
raise RuntimeError(f"native encoder died: {_enc_err(native_enc)}") from exc
|
||||
if upscale_enc:
|
||||
# RTX-branch stand-in: lanczos resize per frame to target dims
|
||||
arr = np.frombuffer(raw, dtype=np.uint8).reshape(n, h, w, 3)
|
||||
imgs = [Image.fromarray(fr).resize(
|
||||
(plan.upscale_width, plan.upscale_height), Image.LANCZOS)
|
||||
for fr in arr]
|
||||
try:
|
||||
upscale_enc.stdin.write(np.stack([np.asarray(im) for im in imgs]).tobytes())
|
||||
except BrokenPipeError as exc:
|
||||
raise RuntimeError(f"upscale encoder died: {_enc_err(upscale_enc)}") from exc
|
||||
f += n
|
||||
seg_done += n
|
||||
pct = 15.0 + 60.0 * seg_done / seg_total
|
||||
# per-stage sub-progress: emulate per-segment sampling steps
|
||||
cur = next((s for s in plan.segments if s.start_frame <= f < s.end_frame), None)
|
||||
stage("MoxieProject", pct,
|
||||
f"mock: sampling frame {f}/{frames}"
|
||||
+ (f" ({cur.prompt or 'untitled'})" if cur else ""),
|
||||
{"frame": f, "frames": frames})
|
||||
native_enc.stdin.close()
|
||||
if upscale_enc:
|
||||
upscale_enc.stdin.close()
|
||||
rc = native_enc.wait(timeout=120)
|
||||
if rc != 0:
|
||||
raise RuntimeError(f"native encode failed: {native_enc.stderr.read().decode(errors='replace')[-400:]}")
|
||||
if upscale_enc:
|
||||
rc = upscale_enc.wait(timeout=180)
|
||||
if rc != 0:
|
||||
raise RuntimeError(f"upscale encode failed: {upscale_enc.stderr.read().decode(errors='replace')[-400:]}")
|
||||
finally:
|
||||
for proc in (native_enc, upscale_enc):
|
||||
if proc and proc.poll() is None:
|
||||
proc.kill()
|
||||
|
||||
stage("MoxieCombine", 78.0, "mock: combined images + audio + fps")
|
||||
if do_upscale:
|
||||
stage("RTXVideoSuperResolution", 88.0,
|
||||
f"mock: upscale {w}x{h} -> {plan.upscale_width}x{plan.upscale_height} "
|
||||
f"({plan.upscale_quality})")
|
||||
|
||||
stage("CreateVideo", 94.0, f"mock: encode {plan.codec}")
|
||||
stage("SaveVideo", 99.0, "mock: saved outputs")
|
||||
|
||||
def describe(path: Path, ow: int, oh: int) -> dict:
|
||||
return {
|
||||
"filename": path.name,
|
||||
"path": str(path),
|
||||
"url": f"/api/view?filename={path.name}&type=output",
|
||||
"width": ow, "height": oh,
|
||||
"fps": plan.frame_rate,
|
||||
"frames": frames,
|
||||
"bytes": path.stat().st_size if path.exists() else 0,
|
||||
}
|
||||
|
||||
result.outputs["native"] = describe(native_path, w, h)
|
||||
if do_upscale:
|
||||
result.outputs["upscaled"] = describe(
|
||||
upscaled_path, plan.upscale_width, plan.upscale_height)
|
||||
result.log(f"rendered {frames} frames in {time.time() - t0:.1f}s")
|
||||
stage("SaveVideo", 100.0, "done", {"outputs": result.outputs})
|
||||
return result
|
||||
113
bench/app/config.py
Normal file
113
bench/app/config.py
Normal file
@ -0,0 +1,113 @@
|
||||
"""Bench configuration: paths and runtime options.
|
||||
|
||||
Everything is derived from --flags / environment so the same code runs from
|
||||
source (repo checkout) and from a PyInstaller bundle (Tauri sidecar).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import sys
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _bundle_root() -> Path:
|
||||
"""Repo root / frozen asset root that contains bench/ assets."""
|
||||
if getattr(sys, "frozen", False): # PyInstaller (onedir: assets in _internal)
|
||||
return Path(getattr(sys, "_MEIPASS", Path(sys.executable).resolve().parent))
|
||||
return Path(__file__).resolve().parent.parent.parent
|
||||
|
||||
|
||||
@dataclass
|
||||
class BenchConfig:
|
||||
host: str = "127.0.0.1"
|
||||
port: int = 8787
|
||||
backend: str = "auto" # auto | mock | comfy
|
||||
repo_root: Path = field(default_factory=_bundle_root)
|
||||
|
||||
@property
|
||||
def bench_dir(self) -> Path:
|
||||
return self.repo_root / "bench"
|
||||
|
||||
@property
|
||||
def base_workflow_path(self) -> Path:
|
||||
return self.bench_dir / "base_workflow.json"
|
||||
|
||||
@property
|
||||
def ui_dist(self) -> Path:
|
||||
return self.bench_dir / "ui-dist"
|
||||
|
||||
@property
|
||||
def work_dir(self) -> Path:
|
||||
default = (
|
||||
# frozen sidecar: keep renders next to the binary, not in _internal
|
||||
Path(sys.executable).resolve().parent / "work"
|
||||
if getattr(sys, "frozen", False)
|
||||
else self.repo_root / "bench" / "work"
|
||||
)
|
||||
d = Path(os.environ.get("MOXIE_BENCH_WORK", str(default)))
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
@property
|
||||
def outputs_dir(self) -> Path:
|
||||
d = self.work_dir / "outputs"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
@property
|
||||
def inputs_dir(self) -> Path:
|
||||
d = self.work_dir / "inputs"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
@property
|
||||
def temp_dir(self) -> Path:
|
||||
d = self.work_dir / "temp"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
@property
|
||||
def pack_models_dir(self) -> Path:
|
||||
return self.repo_root / "models"
|
||||
|
||||
# --- Comfy bridge backend ---
|
||||
comfyui_url: str = os.environ.get("MOXIE_BENCH_COMFYUI_URL", "http://127.0.0.1:8188")
|
||||
|
||||
def resolve_backend(self) -> str:
|
||||
"""auto: comfy when torch + ComfyUI reachable, else mock."""
|
||||
if self.backend != "auto":
|
||||
return self.backend
|
||||
from .backends import comfy_bridge_available
|
||||
|
||||
return "comfy" if comfy_bridge_available(self) else "mock"
|
||||
|
||||
@classmethod
|
||||
def from_args(cls, argv: list[str] | None = None) -> "BenchConfig":
|
||||
p = argparse.ArgumentParser(prog="moxie-bench")
|
||||
p.add_argument("--host", default=cls.host)
|
||||
p.add_argument("--port", type=int, default=cls.port)
|
||||
p.add_argument(
|
||||
"--backend",
|
||||
choices=["auto", "mock", "comfy"],
|
||||
default=cls.backend,
|
||||
help="mock = self-contained synthetic renderer; comfy = drive a "
|
||||
"running ComfyUI at --comfyui-url with the base workflow",
|
||||
)
|
||||
p.add_argument("--comfyui-url", default=cls.comfyui_url)
|
||||
p.add_argument(
|
||||
"--repo-root",
|
||||
default=None,
|
||||
help="Root containing bench/ and models/ (default: auto-detect)",
|
||||
)
|
||||
args = p.parse_args(argv)
|
||||
cfg = cls(
|
||||
host=args.host,
|
||||
port=args.port,
|
||||
backend=args.backend,
|
||||
comfyui_url=args.comfyui_url,
|
||||
)
|
||||
if args.repo_root:
|
||||
cfg.repo_root = Path(args.repo_root).resolve()
|
||||
return cfg
|
||||
118
bench/app/jobs.py
Normal file
118
bench/app/jobs.py
Normal file
@ -0,0 +1,118 @@
|
||||
"""Render job queue: one worker thread, progress events, WS broadcast."""
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any, Callable
|
||||
|
||||
|
||||
@dataclass
|
||||
class Job:
|
||||
id: str
|
||||
created_at: float = field(default_factory=time.time)
|
||||
status: str = "queued" # queued | running | done | error
|
||||
backend: str = "mock"
|
||||
stage: str = ""
|
||||
progress: float = 0.0
|
||||
message: str = "queued"
|
||||
result: dict[str, Any] | None = None
|
||||
error: str | None = None
|
||||
events: list[dict[str, Any]] = field(default_factory=list)
|
||||
plan: dict[str, Any] | None = None
|
||||
|
||||
def snapshot(self) -> dict[str, Any]:
|
||||
return {
|
||||
"id": self.id,
|
||||
"status": self.status,
|
||||
"backend": self.backend,
|
||||
"stage": self.stage,
|
||||
"progress": round(self.progress, 1),
|
||||
"message": self.message,
|
||||
"result": self.result,
|
||||
"error": self.error,
|
||||
"plan": self.plan,
|
||||
}
|
||||
|
||||
|
||||
BroadcastFn = Callable[[dict[str, Any]], None]
|
||||
|
||||
|
||||
class JobManager:
|
||||
"""Serial render queue. Events mirror ComfyUI's push model so the UI shim
|
||||
can forward them onto app.api as execution events."""
|
||||
|
||||
def __init__(self, broadcast: BroadcastFn, max_workers: int = 1) -> None:
|
||||
self._pool = ThreadPoolExecutor(max_workers=max_workers)
|
||||
self._jobs: dict[str, Job] = {}
|
||||
self._order: list[str] = []
|
||||
self._lock = threading.Lock()
|
||||
self._broadcast = broadcast
|
||||
|
||||
# -- public -------------------------------------------------------------
|
||||
def create(self, backend: str, plan_summary: dict) -> Job:
|
||||
job = Job(id=uuid.uuid4().hex[:12], backend=backend, plan=plan_summary)
|
||||
with self._lock:
|
||||
self._jobs[job.id] = job
|
||||
self._order.append(job.id)
|
||||
self._push(job, event="job_queued")
|
||||
return job
|
||||
|
||||
def submit(self, job: Job, runner: Callable[[Job, BroadcastFn], dict]) -> None:
|
||||
self._pool.submit(self._run, job, runner)
|
||||
|
||||
def get(self, job_id: str) -> Job | None:
|
||||
return self._jobs.get(job_id)
|
||||
|
||||
def list(self, limit: int = 20) -> list[dict[str, Any]]:
|
||||
with self._lock:
|
||||
ids = list(reversed(self._order[-limit:]))
|
||||
return [self._jobs[i].snapshot() for i in ids if i in self._jobs]
|
||||
|
||||
# -- internals ------------------------------------------------------------
|
||||
def _run(self, job: Job, runner: Callable[[Job, BroadcastFn], dict]) -> None:
|
||||
job.status = "running"
|
||||
self._push(job, event="execution_start")
|
||||
|
||||
def progress(stage: str, pct: float, message: str, data: dict | None) -> None:
|
||||
job.stage = stage
|
||||
job.progress = max(0.0, min(100.0, pct))
|
||||
job.message = message
|
||||
evt = {
|
||||
"stage": stage,
|
||||
"progress": job.progress,
|
||||
"message": message,
|
||||
**({"data": data} if data else {}),
|
||||
}
|
||||
with self._lock:
|
||||
job.events.append({"time": time.time(), **evt})
|
||||
self._push(job, event="job_progress", **evt)
|
||||
|
||||
try:
|
||||
result = runner(job, progress)
|
||||
job.result = result
|
||||
job.status = "done"
|
||||
job.progress = 100.0
|
||||
job.message = "done"
|
||||
self._push(job, event="execution_success", outputs=result.get("outputs", {}))
|
||||
except Exception as exc: # noqa: BLE001 - surface everything to the UI
|
||||
job.status = "error"
|
||||
job.error = f"{type(exc).__name__}: {exc}"
|
||||
job.message = job.error
|
||||
self._push(job, event="execution_error", error=job.error)
|
||||
|
||||
def _push(self, job: Job, event: str, **extra: Any) -> None:
|
||||
try:
|
||||
self._broadcast({
|
||||
"type": event,
|
||||
"job_id": job.id,
|
||||
"status": job.status,
|
||||
"stage": job.stage,
|
||||
"progress": round(job.progress, 1),
|
||||
"message": job.message,
|
||||
**extra,
|
||||
})
|
||||
except Exception: # never let a dead WS client break a render
|
||||
pass
|
||||
50
bench/app/pipeline.py
Normal file
50
bench/app/pipeline.py
Normal file
@ -0,0 +1,50 @@
|
||||
"""Fixed-pipeline executor: binds a PipelineSpec + RenderPlan to a backend."""
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Callable
|
||||
|
||||
from .config import BenchConfig
|
||||
from .tracks import RenderPlan
|
||||
from .workflow import PipelineSpec
|
||||
|
||||
ProgressCb = Callable[[str, float, str, dict | None], None]
|
||||
|
||||
|
||||
def run_pipeline(
|
||||
spec: PipelineSpec,
|
||||
plan: RenderPlan,
|
||||
backend: str,
|
||||
cfg: BenchConfig,
|
||||
progress: ProgressCb,
|
||||
job_dir: Path,
|
||||
) -> dict:
|
||||
"""Run all stages; returns {outputs: {...}, console: [...]}."""
|
||||
job_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
if backend == "mock":
|
||||
from .backends import mock
|
||||
|
||||
result = mock.render(plan, job_dir, progress)
|
||||
elif backend == "comfy":
|
||||
from .backends import comfy
|
||||
|
||||
result = comfy.render(spec, plan, cfg, job_dir, progress)
|
||||
else:
|
||||
raise RuntimeError(f"unknown backend {backend!r}")
|
||||
|
||||
# Move finished files into the job's outputs folder served by /api/view.
|
||||
final: dict[str, dict] = {}
|
||||
for key, out in result.outputs.items():
|
||||
src = Path(out["path"])
|
||||
dst = job_dir / src.name
|
||||
if src.resolve() != dst.resolve():
|
||||
shutil.move(str(src), str(dst))
|
||||
out2 = dict(out)
|
||||
out2["path"] = str(dst)
|
||||
out2["filename"] = dst.name
|
||||
out2["url"] = f"/api/view?filename={dst.name}&type=output&job={job_dir.name}"
|
||||
final[key] = out2
|
||||
|
||||
return {"outputs": final, "console": result.console}
|
||||
421
bench/app/server.py
Normal file
421
bench/app/server.py
Normal file
@ -0,0 +1,421 @@
|
||||
"""Moxie bench FastAPI server.
|
||||
|
||||
Serves the timeline UI (bench/ui-dist), the fixed-pipeline REST API and a
|
||||
WebSocket that mirrors ComfyUI's push-event model (execution_start,
|
||||
job_progress, execution_success/execution_error) so the ported editor can
|
||||
subscribe with the same patterns it uses inside ComfyUI.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import FastAPI, File, Form, HTTPException, UploadFile, WebSocket, WebSocketDisconnect
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, JSONResponse, Response
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
|
||||
from . import __version__
|
||||
from .config import BenchConfig
|
||||
from .jobs import JobManager
|
||||
from .pipeline import run_pipeline
|
||||
from .tracks import TrackDataError, build_render_plan
|
||||
from .workflow import (
|
||||
EDITOR_TYPE,
|
||||
WorkflowError,
|
||||
describe,
|
||||
editor_resolution,
|
||||
load_base_workflow,
|
||||
parse_workflow_dict,
|
||||
project_data as spec_project_data,
|
||||
)
|
||||
|
||||
_MEDIA_KEYS = ("images", "videos", "gifs", "audio")
|
||||
|
||||
|
||||
def create_app(cfg: BenchConfig | None = None) -> FastAPI:
|
||||
cfg = cfg or BenchConfig.from_args()
|
||||
app = FastAPI(title="Moxie Multimedia Bench", version=__version__)
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
loop_holder: dict = {}
|
||||
subscribers: set[WebSocket] = set()
|
||||
|
||||
@app.on_event("startup")
|
||||
async def _capture_loop() -> None:
|
||||
loop_holder["loop"] = asyncio.get_running_loop()
|
||||
|
||||
def broadcast(payload: dict) -> None:
|
||||
loop = loop_holder.get("loop")
|
||||
if loop is None:
|
||||
return
|
||||
|
||||
def _publish() -> None:
|
||||
dead = []
|
||||
for ws in list(subscribers):
|
||||
asyncio.ensure_future(_send(ws, payload, dead))
|
||||
for ws in dead:
|
||||
subscribers.discard(ws)
|
||||
|
||||
try:
|
||||
loop.call_soon_threadsafe(_publish)
|
||||
except RuntimeError:
|
||||
pass
|
||||
|
||||
async def _send(ws: WebSocket, payload: dict, dead: list) -> None:
|
||||
try:
|
||||
await ws.send_json(payload)
|
||||
except Exception: # noqa: BLE001
|
||||
dead.append(ws)
|
||||
|
||||
jobs = JobManager(broadcast)
|
||||
|
||||
# -- spec helpers --------------------------------------------------------
|
||||
|
||||
def current_spec(raw_override: dict | None = None):
|
||||
if raw_override is not None:
|
||||
return parse_workflow_dict(raw_override)
|
||||
return load_base_workflow(cfg.base_workflow_path)
|
||||
|
||||
def build_plan(spec, overrides: dict):
|
||||
w, h = editor_resolution(spec)
|
||||
pd = spec_project_data(spec)
|
||||
ups = describe(spec)["upscale"] if "RTXVideoSuperResolution" in spec.stages else None
|
||||
ups_target = (ups or {}).get("target") or {}
|
||||
return build_render_plan(
|
||||
spec.widget(EDITOR_TYPE, "track_data", "{}"),
|
||||
width=int(overrides.get("width") or w),
|
||||
height=int(overrides.get("height") or h),
|
||||
frame_rate=overrides.get("frame_rate") or pd.get("frame_rate") or 24,
|
||||
steps=int(overrides.get("steps") or spec.widget("MoxieProject", "steps", 4)),
|
||||
seed=int(overrides.get("seed") or spec.widget("MoxieProject", "seed", 0) or 0),
|
||||
project_name=overrides.get("project_name")
|
||||
or spec.widget("MoxieProject", "project_name", "MoxieBench"),
|
||||
lora=overrides.get("lora") or spec.widget("LoraLoaderModelOnly", "lora_name"),
|
||||
lora_strength=float(
|
||||
overrides.get("lora_strength")
|
||||
or spec.widget("LoraLoaderModelOnly", "strength_model", 1.0)
|
||||
),
|
||||
upscale_width=ups_target.get("width"),
|
||||
upscale_height=ups_target.get("height"),
|
||||
upscale_quality=(ups or {}).get("quality", "ULTRA"),
|
||||
codec=overrides.get("codec", "h264"),
|
||||
)
|
||||
|
||||
# -- core API ------------------------------------------------------------
|
||||
|
||||
@app.get("/api/health")
|
||||
def health():
|
||||
return {"ok": True, "version": __version__, "backend": cfg.resolve_backend()}
|
||||
|
||||
@app.get("/api/config")
|
||||
def config():
|
||||
from .backends import available
|
||||
|
||||
return {
|
||||
"version": __version__,
|
||||
"backend": cfg.resolve_backend(),
|
||||
"backends": available(cfg),
|
||||
"work_dir": str(cfg.work_dir),
|
||||
"outputs_dir": str(cfg.outputs_dir),
|
||||
}
|
||||
|
||||
@app.get("/api/workflow")
|
||||
def get_workflow():
|
||||
spec = current_spec()
|
||||
return {
|
||||
"summary": describe(spec),
|
||||
"track_data": spec.widget(EDITOR_TYPE, "track_data"),
|
||||
"project_data": spec_project_data(spec),
|
||||
}
|
||||
|
||||
@app.get("/api/plan")
|
||||
def get_plan(overrides: str = "{}"):
|
||||
spec = current_spec()
|
||||
plan = build_plan(spec, json.loads(overrides or "{}"))
|
||||
return plan.summary()
|
||||
|
||||
@app.post("/api/render")
|
||||
async def render(body: dict):
|
||||
try:
|
||||
spec = current_spec(body.get("workflow"))
|
||||
except WorkflowError as exc:
|
||||
raise HTTPException(400, str(exc)) from exc
|
||||
|
||||
overrides = body.get("overrides") or {}
|
||||
# UI-submitted track data replaces the template's copy
|
||||
if body.get("track_data") is not None:
|
||||
editor = spec.stages[EDITOR_TYPE]
|
||||
editor.widget_values["track_data"] = (
|
||||
body["track_data"]
|
||||
if isinstance(body["track_data"], str)
|
||||
else json.dumps(body["track_data"])
|
||||
)
|
||||
for key in ("resolution",):
|
||||
if overrides.get(key):
|
||||
editor = spec.stages[EDITOR_TYPE]
|
||||
editor.widget_values[key] = overrides[key]
|
||||
|
||||
try:
|
||||
plan = build_plan(spec, overrides)
|
||||
except TrackDataError as exc:
|
||||
raise HTTPException(400, str(exc)) from exc
|
||||
|
||||
backend = overrides.get("backend") or body.get("backend") or cfg.resolve_backend()
|
||||
job = jobs.create(backend, plan.summary())
|
||||
|
||||
def runner(active: Job, progress) -> dict:
|
||||
job_dir = cfg.outputs_dir / active.id
|
||||
return run_pipeline(spec, plan, active.backend, cfg, progress, job_dir)
|
||||
|
||||
jobs.submit(job, runner)
|
||||
return {"job_id": job.id, "plan": plan.summary()}
|
||||
|
||||
@app.get("/api/jobs")
|
||||
def list_jobs():
|
||||
return {"jobs": jobs.list()}
|
||||
|
||||
@app.get("/api/jobs/{job_id}")
|
||||
def get_job(job_id: str):
|
||||
job = jobs.get(job_id)
|
||||
if not job:
|
||||
raise HTTPException(404, "no such job")
|
||||
snap = job.snapshot()
|
||||
snap["events"] = job.events[-200:]
|
||||
return snap
|
||||
|
||||
@app.delete("/api/jobs/{job_id}")
|
||||
def delete_job(job_id: str):
|
||||
job = jobs.get(job_id)
|
||||
if not job:
|
||||
raise HTTPException(404, "no such job")
|
||||
job.status = "deleted"
|
||||
return {"ok": True}
|
||||
|
||||
# -- media ----------------------------------------------------------------
|
||||
|
||||
def _safe(name: str) -> str:
|
||||
name = Path(name).name
|
||||
if not name or name.startswith("."):
|
||||
raise HTTPException(400, "bad filename")
|
||||
return name
|
||||
|
||||
@app.get("/api/view")
|
||||
def view(filename: str, type: str = "output", job: str | None = None):
|
||||
name = _safe(filename)
|
||||
if type == "input":
|
||||
path = cfg.inputs_dir / name
|
||||
elif type == "temp":
|
||||
path = cfg.temp_dir / name
|
||||
else:
|
||||
base = cfg.outputs_dir / _safe(job) if job else cfg.outputs_dir
|
||||
path = base / name
|
||||
if not path.is_file():
|
||||
raise HTTPException(404, "no such file")
|
||||
media = _guess_media_type(path)
|
||||
return FileResponse(path, media_type=media)
|
||||
|
||||
@app.post("/api/upload/image")
|
||||
async def upload_image(image: UploadFile = File(None), subfolder: str = Form("")):
|
||||
return await _save_upload(image, subfolder)
|
||||
|
||||
@app.post("/api/easy-media/upload")
|
||||
async def easy_upload(file: UploadFile = File(None), subfolder: str = Form("")):
|
||||
return await _save_upload(file, subfolder)
|
||||
|
||||
async def _save_upload(upload: UploadFile | None, subfolder: str) -> dict:
|
||||
if upload is None:
|
||||
raise HTTPException(400, "missing file")
|
||||
name = _safe(upload.filename or "upload.bin")
|
||||
target_dir = cfg.inputs_dir
|
||||
if subfolder:
|
||||
target_dir = target_dir / Path(subfolder).name
|
||||
target_dir.mkdir(parents=True, exist_ok=True)
|
||||
target = target_dir / name
|
||||
with open(target, "wb") as fh:
|
||||
shutil.copyfileobj(upload.file, fh)
|
||||
return {"name": name, "subfolder": str(target.parent.relative_to(cfg.inputs_dir)),
|
||||
"type": "input"}
|
||||
|
||||
@app.post("/api/easy-media/download-url")
|
||||
async def download_url(body: dict):
|
||||
import urllib.request
|
||||
|
||||
url = body.get("url")
|
||||
if not url:
|
||||
raise HTTPException(400, "missing url")
|
||||
name = _safe(body.get("name") or Path(url).name or "download.bin")
|
||||
dst = cfg.inputs_dir / name
|
||||
try:
|
||||
with urllib.request.urlopen(url, timeout=60) as resp, open(dst, "wb") as fh:
|
||||
shutil.copyfileobj(resp, fh)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
raise HTTPException(502, f"download failed: {exc}") from exc
|
||||
return {"name": name, "subfolder": "", "type": "input"}
|
||||
|
||||
@app.get("/api/easy-media/media/list")
|
||||
def media_list(kind: str = "input"):
|
||||
base = cfg.inputs_dir if kind == "input" else cfg.outputs_dir
|
||||
items = []
|
||||
for p in sorted(base.rglob("*")):
|
||||
if p.is_file() and not p.name.startswith("."):
|
||||
rel = str(p.relative_to(base).parent)
|
||||
items.append({
|
||||
"name": p.name,
|
||||
"path": str(p.relative_to(base)),
|
||||
"subfolder": "" if rel == "." else rel,
|
||||
"type": "input" if kind == "input" else "output",
|
||||
"size": p.stat().st_size,
|
||||
})
|
||||
return {"items": items}
|
||||
|
||||
@app.get("/api/easy-media/media/recent")
|
||||
def media_recent():
|
||||
return media_list("output")
|
||||
|
||||
@app.get("/api/easy-media/projects")
|
||||
def easy_projects():
|
||||
projects_dir = cfg.work_dir / "projects"
|
||||
projects_dir.mkdir(parents=True, exist_ok=True)
|
||||
return {"projects": sorted(
|
||||
p.stem for p in projects_dir.glob("*.json") if p.is_file())}
|
||||
|
||||
@app.get("/api/easy-media/project")
|
||||
def easy_project(project_name: str):
|
||||
path = cfg.work_dir / "projects" / f"{Path(project_name).stem}.json"
|
||||
if not path.is_file():
|
||||
raise HTTPException(404, "no such project")
|
||||
return JSONResponse(content=json.loads(path.read_text()))
|
||||
|
||||
@app.get("/api/easy-media/prompt/system-prompts")
|
||||
def system_prompts():
|
||||
return {"prompts": [
|
||||
{"id": "default", "name": "Default", "prompt": ""},
|
||||
]}
|
||||
|
||||
@app.post("/api/easy-media/video/source-fps")
|
||||
def source_fps(body: dict):
|
||||
name = _safe(body.get("filename") or "")
|
||||
path = cfg.inputs_dir / name
|
||||
if not path.is_file():
|
||||
raise HTTPException(404, "no such file")
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["ffprobe", "-v", "quiet", "-print_format", "json",
|
||||
"-show_streams", str(path)],
|
||||
capture_output=True, text=True, timeout=30,
|
||||
).stdout
|
||||
streams = json.loads(out).get("streams") or []
|
||||
rate = (streams[0].get("r_frame_rate") or "24/1") if streams else "24/1"
|
||||
num, _, den = rate.partition("/")
|
||||
fps = float(num) / float(den or 1)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
raise HTTPException(500, f"ffprobe failed: {exc}") from exc
|
||||
return {"fps": fps}
|
||||
|
||||
@app.post("/api/easy-media/video/preview-frame")
|
||||
def preview_frame(body: dict):
|
||||
name = _safe(body.get("filename") or "")
|
||||
path = cfg.inputs_dir / name
|
||||
if not path.is_file():
|
||||
raise HTTPException(404, "no such file")
|
||||
t = float(body.get("time") or 0)
|
||||
try:
|
||||
out = subprocess.run(
|
||||
["ffmpeg", "-v", "quiet", "-ss", str(t), "-i", str(path),
|
||||
"-frames:v", "1", "-f", "image2pipe", "-vcodec", "mjpeg", "pipe:1"],
|
||||
capture_output=True, timeout=60,
|
||||
)
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
raise HTTPException(504, "ffmpeg timeout") from exc
|
||||
if not out.stdout:
|
||||
raise HTTPException(500, "ffmpeg produced no frame")
|
||||
return Response(content=out.stdout, media_type="image/jpeg")
|
||||
|
||||
# -- ComfyUI-compatible root aliases --------------------------------------
|
||||
# The ported editor (and its media elements) call the exact ComfyUI paths:
|
||||
# /view, /upload/image, /easy-media/*. Register them alongside /api/* so
|
||||
# the UI shim needs no URL rewriting at all.
|
||||
app.add_api_route("/view", view, methods=["GET"], include_in_schema=False)
|
||||
app.add_api_route("/upload/image", _save_upload, methods=["POST"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/upload", _save_upload, methods=["POST"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/media/list", media_list, methods=["GET"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/media/recent", media_recent, methods=["GET"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/projects", easy_projects, methods=["GET"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/project", easy_project, methods=["GET"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/prompt/system-prompts", system_prompts, methods=["GET"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/video/source-fps", source_fps, methods=["POST"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/video/preview-frame", preview_frame, methods=["POST"], include_in_schema=False)
|
||||
app.add_api_route("/easy-media/download-url", download_url, methods=["POST"], include_in_schema=False)
|
||||
|
||||
# fallthrough for endpoints the editor may call in v1
|
||||
@app.api_route("/api/easy-media/{rest:path}", methods=["GET", "POST", "PATCH", "DELETE"])
|
||||
def easy_fallback(rest: str):
|
||||
return JSONResponse(
|
||||
status_code=501,
|
||||
content={"error": f"bench does not implement /api/easy-media/{rest} yet"},
|
||||
)
|
||||
|
||||
@app.api_route("/easy-media/{rest:path}", methods=["GET", "POST", "PATCH", "DELETE"])
|
||||
def easy_fallback_root(rest: str):
|
||||
return easy_fallback(rest)
|
||||
|
||||
# -- websocket -------------------------------------------------------------
|
||||
|
||||
@app.websocket("/api/ws")
|
||||
async def ws_endpoint(ws: WebSocket):
|
||||
await ws.accept()
|
||||
subscribers.add(ws)
|
||||
try:
|
||||
while True:
|
||||
await ws.receive_text() # keepalive pings from the UI
|
||||
except WebSocketDisconnect:
|
||||
pass
|
||||
finally:
|
||||
subscribers.discard(ws)
|
||||
|
||||
# -- static UI --------------------------------------------------------------
|
||||
|
||||
ui_dist = cfg.ui_dist
|
||||
if (ui_dist / "index.html").is_file():
|
||||
app.mount("/", StaticFiles(directory=str(ui_dist), html=True), name="ui")
|
||||
else:
|
||||
@app.get("/")
|
||||
def no_ui():
|
||||
return JSONResponse({
|
||||
"error": "bench UI not built",
|
||||
"hint": "cd frontend && bun run build:bench",
|
||||
})
|
||||
|
||||
# silence 404 noise for browsers requesting favicons
|
||||
@app.get("/favicon.ico")
|
||||
def favicon():
|
||||
return Response(status_code=204)
|
||||
|
||||
return app
|
||||
|
||||
|
||||
def _guess_media_type(path: Path) -> str:
|
||||
ext = path.suffix.lower()
|
||||
return {
|
||||
".mp4": "video/mp4", ".webm": "video/webm", ".mov": "video/quicktime",
|
||||
".mkv": "video/x-matroska", ".png": "image/png", ".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg", ".webp": "image/webp", ".gif": "image/gif",
|
||||
".wav": "audio/wav", ".mp3": "audio/mpeg", ".flac": "audio/flac",
|
||||
".srt": "text/plain",
|
||||
}.get(ext, "application/octet-stream")
|
||||
|
||||
|
||||
def main() -> None: # pragma: no cover - manual entry
|
||||
import uvicorn
|
||||
|
||||
cfg = BenchConfig.from_args()
|
||||
uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port, log_level="info")
|
||||
195
bench/app/tracks.py
Normal file
195
bench/app/tracks.py
Normal file
@ -0,0 +1,195 @@
|
||||
"""track_data (the multitrack editor JSON) -> RenderPlan.
|
||||
|
||||
Pure extraction, no ComfyUI. Mirrors what ``MultiTrackEditor.execute`` feeds
|
||||
into TRACKS_INFO for the fixed pipeline: task tracks carry the prompt segments
|
||||
that become generation tasks; video/audio tracks carry media references.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
|
||||
class TrackDataError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass
|
||||
class Segment:
|
||||
id: str
|
||||
start_frame: int
|
||||
end_frame: int
|
||||
prompt: str = ""
|
||||
task_mode: str = "default"
|
||||
continuity_mode: str = "shot"
|
||||
ref_image_size: str = "match"
|
||||
media_type: str = "none"
|
||||
images: list[Any] = field(default_factory=list)
|
||||
muted: bool = False
|
||||
volume_db: float = 0.0
|
||||
|
||||
@property
|
||||
def frame_count(self) -> int:
|
||||
return max(0, self.end_frame - self.start_frame)
|
||||
|
||||
|
||||
@dataclass
|
||||
class Track:
|
||||
id: str
|
||||
name: str
|
||||
type: str
|
||||
task_mode: str = "default"
|
||||
muted: bool = False
|
||||
solo: bool = False
|
||||
volume_db: float = 0.0
|
||||
locked: bool = False
|
||||
segments: list[Segment] = field(default_factory=list)
|
||||
|
||||
|
||||
@dataclass
|
||||
class RenderPlan:
|
||||
width: int
|
||||
height: int
|
||||
frame_rate: int
|
||||
total_length: int
|
||||
segments: list[Segment] = field(default_factory=list) # ordered task segments
|
||||
tracks: list[Track] = field(default_factory=list)
|
||||
project_name: str = "MoxieBench"
|
||||
steps: int = 4
|
||||
seed: int = 0
|
||||
lora: str | None = None
|
||||
lora_strength: float = 1.0
|
||||
upscale_width: int | None = None
|
||||
upscale_height: int | None = None
|
||||
upscale_quality: str = "ULTRA"
|
||||
codec: str = "h264"
|
||||
|
||||
@property
|
||||
def duration_seconds(self) -> float:
|
||||
return self.total_length / float(self.frame_rate or 24)
|
||||
|
||||
def summary(self) -> dict[str, Any]:
|
||||
return {
|
||||
"width": self.width,
|
||||
"height": self.height,
|
||||
"frame_rate": self.frame_rate,
|
||||
"total_length": self.total_length,
|
||||
"duration_seconds": round(self.duration_seconds, 3),
|
||||
"steps": self.steps,
|
||||
"seed": self.seed,
|
||||
"lora": self.lora,
|
||||
"project_name": self.project_name,
|
||||
"upscale": (
|
||||
{"width": self.upscale_width, "height": self.upscale_height,
|
||||
"quality": self.upscale_quality}
|
||||
if self.upscale_width else None
|
||||
),
|
||||
"segments": [
|
||||
{
|
||||
"id": s.id,
|
||||
"prompt": s.prompt,
|
||||
"start_frame": s.start_frame,
|
||||
"end_frame": s.end_frame,
|
||||
"frame_count": s.frame_count,
|
||||
"task_mode": s.task_mode,
|
||||
"continuity_mode": s.continuity_mode,
|
||||
}
|
||||
for s in self.segments
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def parse_track_data(raw: str | dict[str, Any]) -> dict[str, Any]:
|
||||
if isinstance(raw, str):
|
||||
try:
|
||||
raw = json.loads(raw)
|
||||
except json.JSONDecodeError as exc:
|
||||
raise TrackDataError(f"track_data is not valid JSON: {exc}") from exc
|
||||
if not isinstance(raw, dict):
|
||||
raise TrackDataError("track_data must be an object")
|
||||
if not isinstance(raw.get("tracks"), list):
|
||||
raise TrackDataError("track_data.tracks must be a list")
|
||||
return raw
|
||||
|
||||
|
||||
def build_render_plan(
|
||||
track_data: str | dict[str, Any],
|
||||
*,
|
||||
width: int,
|
||||
height: int,
|
||||
frame_rate: int | None = None,
|
||||
steps: int = 4,
|
||||
seed: int = 0,
|
||||
project_name: str = "MoxieBench",
|
||||
lora: str | None = None,
|
||||
lora_strength: float = 1.0,
|
||||
upscale_width: int | None = None,
|
||||
upscale_height: int | None = None,
|
||||
upscale_quality: str = "ULTRA",
|
||||
codec: str = "h264",
|
||||
) -> RenderPlan:
|
||||
data = parse_track_data(track_data)
|
||||
|
||||
# Timeline fps: editor stores it on track_data.frame_rate (24 in the base
|
||||
# workflow); the combine project_data may override it.
|
||||
if frame_rate is None:
|
||||
frame_rate = int(data.get("frame_rate") or 24)
|
||||
total = int(data.get("total_length") or 0)
|
||||
|
||||
tracks: list[Track] = []
|
||||
segments: list[Segment] = []
|
||||
for t in data["tracks"]:
|
||||
if not isinstance(t, dict):
|
||||
continue
|
||||
track = Track(
|
||||
id=str(t.get("id", "")),
|
||||
name=str(t.get("name", "")),
|
||||
type=str(t.get("type", "task")),
|
||||
task_mode=str(t.get("task_mode", "default")),
|
||||
muted=bool(t.get("muted", False)),
|
||||
solo=bool(t.get("solo", False)),
|
||||
volume_db=float(t.get("volume_db", 0)),
|
||||
locked=bool(t.get("locked", False)),
|
||||
)
|
||||
for seg in t.get("segments") or []:
|
||||
if not isinstance(seg, dict):
|
||||
continue
|
||||
content = seg.get("content") or {}
|
||||
s = Segment(
|
||||
id=str(seg.get("id", "")),
|
||||
start_frame=int(seg.get("start_frame", 0)),
|
||||
end_frame=int(seg.get("end_frame", 0)),
|
||||
prompt=str(content.get("user_prompt", "") or ""),
|
||||
task_mode=str(content.get("task_mode", track.task_mode or "default")),
|
||||
continuity_mode=str(content.get("continuity_mode", "shot")),
|
||||
ref_image_size=str(content.get("ref_image_size", "match")),
|
||||
media_type=str(content.get("media_type", "none")),
|
||||
images=list(content.get("images") or []),
|
||||
muted=bool(content.get("muted", False)),
|
||||
volume_db=float(content.get("volume_db", 0)),
|
||||
)
|
||||
track.segments.append(s)
|
||||
if track.type == "task" and s.frame_count > 0:
|
||||
segments.append(s)
|
||||
tracks.append(track)
|
||||
|
||||
segments.sort(key=lambda s: (s.start_frame, s.end_frame))
|
||||
|
||||
return RenderPlan(
|
||||
width=int(width),
|
||||
height=int(height),
|
||||
frame_rate=int(frame_rate),
|
||||
total_length=total,
|
||||
segments=segments,
|
||||
tracks=tracks,
|
||||
project_name=str(project_name),
|
||||
steps=int(steps),
|
||||
seed=int(seed),
|
||||
lora=lora,
|
||||
lora_strength=float(lora_strength),
|
||||
upscale_width=upscale_width,
|
||||
upscale_height=upscale_height,
|
||||
upscale_quality=str(upscale_quality),
|
||||
codec=str(codec),
|
||||
)
|
||||
204
bench/app/workflow.py
Normal file
204
bench/app/workflow.py
Normal file
@ -0,0 +1,204 @@
|
||||
"""Fixed-pipeline contract parsed from the base workflow JSON.
|
||||
|
||||
The bench locks to the Moxie pipeline used by the base workflow:
|
||||
|
||||
MoxieMultimediaLoader -> LoraLoaderModelOnly -> MoxiePreviewOverride
|
||||
-> easy multiTrackEditor -> MoxieProject -> MoxieCombine
|
||||
-> [branch A] CreateVideo(#45) -> SaveVideo(#46) (native)
|
||||
-> [branch B] RTXVideoSuperResolution -> CreateVideo(#68)
|
||||
-> SaveVideo(#69) (upscaled)
|
||||
|
||||
The parser walks the frontend-format workflow (nodes/links/widgets_values_named)
|
||||
that ComfyUI exports and needs no ComfyUI installation: link inputs carry their
|
||||
own names in the node's ``inputs`` array, widget values carry ``widgets_values_named``.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
import json
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
EDITOR_TYPE = "easy multiTrackEditor"
|
||||
STAGE_ORDER = (
|
||||
"MoxieMultimediaLoader",
|
||||
"LoraLoaderModelOnly",
|
||||
"MoxiePreviewOverride",
|
||||
EDITOR_TYPE,
|
||||
"MoxieProject",
|
||||
"MoxieCombine",
|
||||
"RTXVideoSuperResolution",
|
||||
"CreateVideo",
|
||||
) # SaveVideo x2 tracked via PipelineSpec.save_nodes (native/upscaled branches)
|
||||
|
||||
|
||||
class WorkflowError(ValueError):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass
|
||||
class Stage:
|
||||
node_id: int
|
||||
type: str
|
||||
widget_values: dict[str, Any] = field(default_factory=dict)
|
||||
link_inputs: dict[str, tuple[int, int]] = field(default_factory=dict) # name -> (node_id, slot)
|
||||
|
||||
|
||||
@dataclass
|
||||
class PipelineSpec:
|
||||
raw: dict[str, Any]
|
||||
stages: dict[str, Stage] # type -> Stage (types unique in the fixed graph)
|
||||
save_nodes: dict[str, Stage] = field(default_factory=dict) # native/upscaled -> SaveVideo
|
||||
|
||||
@property
|
||||
def editor_node_id(self) -> int:
|
||||
return self.stages[EDITOR_TYPE].node_id
|
||||
|
||||
def widget(self, stage_type: str, name: str, default: Any = None) -> Any:
|
||||
st = self.stages.get(stage_type)
|
||||
if not st:
|
||||
return default
|
||||
return st.widget_values.get(name, default)
|
||||
|
||||
|
||||
def _parse_workflow(raw: dict[str, Any]) -> PipelineSpec:
|
||||
nodes = raw.get("nodes")
|
||||
if not isinstance(nodes, list) or not nodes:
|
||||
raise WorkflowError("workflow has no nodes")
|
||||
links: dict[int, tuple[int, int, int]] = {}
|
||||
for lk in raw.get("links") or []:
|
||||
# [link_id, from_node, from_slot, to_node, to_slot, type]
|
||||
links[int(lk[0])] = (int(lk[1]), int(lk[2]), int(lk[4]))
|
||||
|
||||
stages: dict[str, Stage] = {}
|
||||
save_nodes: dict[str, Stage] = {}
|
||||
seen_create = 0
|
||||
for n in nodes:
|
||||
ntype = n.get("type")
|
||||
nid = int(n.get("id"))
|
||||
if ntype not in STAGE_ORDER and ntype != "SaveVideo":
|
||||
continue
|
||||
named = dict(n.get("widgets_values_named") or {})
|
||||
# Legacy export without named values: best effort positional mapping.
|
||||
if not named:
|
||||
named = {}
|
||||
stage = Stage(node_id=nid, type=ntype, widget_values=named)
|
||||
for inp in n.get("inputs") or []:
|
||||
if not isinstance(inp, dict):
|
||||
continue
|
||||
if "link" in inp and inp["link"] is not None and "widget" not in inp:
|
||||
frm = links.get(int(inp["link"]))
|
||||
if frm is None:
|
||||
raise WorkflowError(f"node {nid}: dangling link {inp['link']}")
|
||||
stage.link_inputs[inp["name"]] = (frm[0], frm[1])
|
||||
if ntype == "CreateVideo":
|
||||
seen_create += 1
|
||||
key = "CreateVideo"
|
||||
if seen_create == 2:
|
||||
stages.pop("CreateVideo#1", None)
|
||||
stages[f"CreateVideo#{seen_create}" if seen_create > 1 else key] = stage
|
||||
continue
|
||||
if ntype == "SaveVideo":
|
||||
src = stage.link_inputs.get("video", (None, None))[0]
|
||||
if src is not None and src in (
|
||||
stages.get("CreateVideo#2").node_id if stages.get("CreateVideo#2") else -1,
|
||||
):
|
||||
save_nodes["upscaled"] = stage
|
||||
else:
|
||||
save_nodes["native"] = stage
|
||||
continue
|
||||
if ntype in stages:
|
||||
raise WorkflowError(f"duplicate stage node type {ntype}")
|
||||
stages[ntype] = stage
|
||||
|
||||
missing = [t for t in STAGE_ORDER if t not in stages]
|
||||
if missing:
|
||||
raise WorkflowError(f"missing pipeline stages: {', '.join(missing)}")
|
||||
return PipelineSpec(raw=raw, stages=stages, save_nodes=save_nodes)
|
||||
|
||||
|
||||
def load_base_workflow(path: Path) -> PipelineSpec:
|
||||
with open(path, "r", encoding="utf-8") as fh:
|
||||
raw = json.load(fh)
|
||||
return _parse_workflow(raw)
|
||||
|
||||
|
||||
def parse_workflow_dict(raw: dict[str, Any]) -> PipelineSpec:
|
||||
return _parse_workflow(copy.deepcopy(raw))
|
||||
|
||||
|
||||
def editor_track_data(spec: PipelineSpec) -> dict[str, Any]:
|
||||
"""Parsed track_data JSON from the editor stage."""
|
||||
raw = spec.widget(EDITOR_TYPE, "track_data")
|
||||
if isinstance(raw, str):
|
||||
raw = json.loads(raw)
|
||||
if not isinstance(raw, dict):
|
||||
raise WorkflowError("editor track_data is not an object")
|
||||
return raw
|
||||
|
||||
|
||||
def editor_resolution(spec: PipelineSpec) -> tuple[int, int]:
|
||||
"""Width/height chosen on the editor, falling back to the project_data."""
|
||||
res = spec.widget(EDITOR_TYPE, "resolution", "")
|
||||
preset = _RESOLUTION_PRESETS.get(res)
|
||||
if preset:
|
||||
return preset
|
||||
combo = spec.stages[EDITOR_TYPE].widget_values.get("resolution")
|
||||
if isinstance(combo, dict): # custom: {"width": .., "height": ..}
|
||||
try:
|
||||
return int(combo.get("width")), int(combo.get("height"))
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
pd = _project_data(spec)
|
||||
return int(pd.get("width", 0)), int(pd.get("height", 0))
|
||||
|
||||
|
||||
def project_data(spec: PipelineSpec) -> dict[str, Any]:
|
||||
return _project_data(spec)
|
||||
|
||||
|
||||
def _project_data(spec: PipelineSpec) -> dict[str, Any]:
|
||||
raw = spec.widget("MoxieCombine", "project_data", "") or "{}"
|
||||
if isinstance(raw, str):
|
||||
raw = json.loads(raw)
|
||||
return raw if isinstance(raw, dict) else {}
|
||||
|
||||
|
||||
_RESOLUTION_PRESETS = {
|
||||
"1920 x 1080 (16:9)": (1920, 1080),
|
||||
"1088 x 1920 (9:16)": (1088, 1920),
|
||||
"1080 x 1920 (9:16)": (1080, 1920),
|
||||
"1280 x 720 (16:9)": (1280, 720),
|
||||
"720 x 1280 (9:16)": (720, 1280),
|
||||
"832 x 480 (16:9)": (832, 480),
|
||||
"480 x 832 (9:16)": (480, 832),
|
||||
"1024 x 1024 (1:1)": (1024, 1024),
|
||||
}
|
||||
|
||||
|
||||
def describe(spec: PipelineSpec) -> dict[str, Any]:
|
||||
"""Small human-readable summary used by /api/workflow."""
|
||||
w, h = editor_resolution(spec)
|
||||
pd = project_data(spec)
|
||||
return {
|
||||
"stages": [
|
||||
{"type": t, "node_id": s.node_id, "widgets": s.widget_values}
|
||||
for t, s in spec.stages.items()
|
||||
],
|
||||
"editor_resolution": {"width": w, "height": h},
|
||||
"frame_rate": pd.get("frame_rate", 24),
|
||||
"project_name": spec.widget("MoxieProject", "project_name", "MoxieBench"),
|
||||
"steps": spec.widget("MoxieProject", "steps", 4),
|
||||
"seed": spec.widget("MoxieProject", "seed", 0),
|
||||
"lora": spec.widget("LoraLoaderModelOnly", "lora_name"),
|
||||
"lora_strength": spec.widget("LoraLoaderModelOnly", "strength_model", 1.0),
|
||||
"upscale": {
|
||||
"enabled": "RTXVideoSuperResolution" in spec.stages,
|
||||
"target": {
|
||||
"width": spec.widget("RTXVideoSuperResolution", "resize_type.width"),
|
||||
"height": spec.widget("RTXVideoSuperResolution", "resize_type.height"),
|
||||
},
|
||||
"quality": spec.widget("RTXVideoSuperResolution", "quality", "ULTRA"),
|
||||
},
|
||||
}
|
||||
240
bench/base_workflow.json
Normal file
240
bench/base_workflow.json
Normal file
@ -0,0 +1,240 @@
|
||||
{
|
||||
"id": "74d5a00d-104c-439f-a307-874d829f8418",
|
||||
"revision": 0,
|
||||
"last_node_id": 81,
|
||||
"last_link_id": 166,
|
||||
"nodes": [
|
||||
{
|
||||
"id": 45,
|
||||
"type": "CreateVideo",
|
||||
"pos": [5303.990730085631, 2674.6532486606284],
|
||||
"size": [270, 150],
|
||||
"flags": {},
|
||||
"order": 7,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{"name": "images", "type": "IMAGE", "link": 151},
|
||||
{"name": "audio", "shape": 7, "type": "AUDIO", "link": 149},
|
||||
{"name": "fps", "type": "FLOAT", "widget": {"name": "fps"}, "link": 150}
|
||||
],
|
||||
"outputs": [{"name": "VIDEO", "type": "VIDEO", "links": [103]}],
|
||||
"properties": {"Node name for S&R": "CreateVideo"},
|
||||
"widgets_values": [30, "auto", "sRGB", "h264"],
|
||||
"widgets_values_named": {"fps": 30, "bit_depth": "auto", "color_space": "sRGB", "codec": "h264"}
|
||||
},
|
||||
{
|
||||
"id": 46,
|
||||
"type": "SaveVideo",
|
||||
"pos": [6536.102438951979, 2629.839158378452],
|
||||
"size": [1056.6122485107426, 1748.7755735581345],
|
||||
"flags": {},
|
||||
"order": 9,
|
||||
"mode": 0,
|
||||
"inputs": [{"name": "video", "type": "VIDEO", "link": 103}],
|
||||
"outputs": [{"name": "video", "type": "VIDEO", "links": null}],
|
||||
"properties": {"Node name for S&R": "SaveVideo"},
|
||||
"widgets_values": ["video/ComfyUI", "mp4", "h264", "auto", "auto"],
|
||||
"widgets_values_named": {"filename_prefix": "video/ComfyUI", "format": "mp4", "format.codec": "h264", "format.codec.encoding": "auto", "codec": "auto"}
|
||||
},
|
||||
{
|
||||
"id": 48,
|
||||
"type": "MoxiePreviewOverride",
|
||||
"pos": [3815.4265801150054, 2654.0916373620416],
|
||||
"size": [360, 480],
|
||||
"flags": {},
|
||||
"order": 3,
|
||||
"mode": 0,
|
||||
"inputs": [{"name": "model", "type": "MODEL", "link": 154}],
|
||||
"outputs": [{"name": "model", "type": "MODEL", "links": [144]}],
|
||||
"properties": {"Node name for S&R": "MoxiePreviewOverride"},
|
||||
"widgets_values": [1024, 80, true, 48, 24, "MM-preview.safetensors", ""],
|
||||
"widgets_values_named": {"max_resolution": 1024, "jpeg_quality": 80, "suppress_default_preview": true, "preview_frames": 48, "preview_fps": 24, "tiny_vae": "MM-preview.safetensors", "preview": ""}
|
||||
},
|
||||
{
|
||||
"id": 52,
|
||||
"type": "MoxieMultimediaLoader",
|
||||
"pos": [3000.4622746801124, 2645.6062887703747],
|
||||
"size": [370.66988849138306, 180.73178061607814],
|
||||
"flags": {},
|
||||
"order": 0,
|
||||
"mode": 0,
|
||||
"inputs": [],
|
||||
"outputs": [
|
||||
{"name": "model", "type": "MODEL", "links": [153]},
|
||||
{"name": "clip", "type": "CLIP", "links": [136]},
|
||||
{"name": "video_vae", "type": "VAE", "links": [137]},
|
||||
{"name": "audio_vae", "type": "VAE", "links": [138]}
|
||||
],
|
||||
"properties": {"Node name for S&R": "MoxieMultimediaLoader"},
|
||||
"widgets_values": ["Moxie-Multimedia.safetensors", "default", "comfy_kitchen_attention"],
|
||||
"widgets_values_named": {"diffusion_model": "Moxie-Multimedia.safetensors", "weight_dtype": "default", "attention_backend": "comfy_kitchen_attention"}
|
||||
},
|
||||
{
|
||||
"id": 57,
|
||||
"type": "MoxieProject",
|
||||
"pos": [4349.808755835772, 2799.703672942545],
|
||||
"size": [277.359375, 230],
|
||||
"flags": {},
|
||||
"order": 4,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{"name": "tracks_info", "type": "TRACKS_INFO", "link": 166},
|
||||
{"name": "model", "type": "MODEL", "link": 144},
|
||||
{"name": "clip", "type": "CLIP", "link": 136},
|
||||
{"name": "video_vae", "type": "VAE", "link": 137},
|
||||
{"name": "audio_vae", "type": "VAE", "link": 138}
|
||||
],
|
||||
"outputs": [
|
||||
{"name": "PROJECT_NAME", "type": "STRING", "links": [148]},
|
||||
{"name": "LOCKED_AUDIO", "type": "AUDIO", "links": null}
|
||||
],
|
||||
"properties": {"Node name for S&R": "MoxieProject"},
|
||||
"widgets_values": ["MM-girlwaving", 4, 803100406447983, "randomize"],
|
||||
"widgets_values_named": {"project_name": "MM-girlwaving", "steps": 4, "seed": 803100406447983, "randomize": "randomize"}
|
||||
},
|
||||
{
|
||||
"id": 63,
|
||||
"type": "MoxieCombine",
|
||||
"pos": [4734.228609743854, 2711.230251501094],
|
||||
"size": [295.59921875, 638],
|
||||
"flags": {},
|
||||
"order": 5,
|
||||
"mode": 0,
|
||||
"inputs": [{"name": "project_name", "type": "STRING", "link": 148}],
|
||||
"outputs": [
|
||||
{"name": "images", "type": "IMAGE", "links": [151, 155]},
|
||||
{"name": "audio", "type": "AUDIO", "links": [149, 157]},
|
||||
{"name": "fps", "type": "FLOAT", "links": [150, 159]}
|
||||
],
|
||||
"properties": {"Node name for S&R": "MoxieCombine"},
|
||||
"widgets_values": ["{\"project_name\":\"MM-girlwaving\",\"width\":1920,\"height\":1088,\"frame_rate\":24,\"clips\":[],\"auto_combine\":true,\"updated_at\":0}"],
|
||||
"widgets_values_named": {"project_data": "{\"project_name\":\"MM-girlwaving\",\"width\":1920,\"height\":1088,\"frame_rate\":24,\"clips\":[],\"auto_combine\":true,\"updated_at\":0}"}
|
||||
},
|
||||
{
|
||||
"id": 66,
|
||||
"type": "LoraLoaderModelOnly",
|
||||
"pos": [3539.407729088216, 2558.7373665366495],
|
||||
"size": [270, 82],
|
||||
"flags": {},
|
||||
"order": 2,
|
||||
"mode": 0,
|
||||
"inputs": [{"name": "model", "type": "MODEL", "link": 153}],
|
||||
"outputs": [{"name": "MODEL", "type": "MODEL", "links": [154]}],
|
||||
"properties": {"Node name for S&R": "LoraLoaderModelOnly"},
|
||||
"widgets_values": ["h3\\minimax_h3_t2v_mshandjob_audio_v1.safetensors", 1],
|
||||
"widgets_values_named": {"lora_name": "h3\\minimax_h3_t2v_mshandjob_audio_v1.safetensors", "strength_model": 1}
|
||||
},
|
||||
{
|
||||
"id": 67,
|
||||
"type": "RTXVideoSuperResolution",
|
||||
"pos": [5085.886101348601, 3012.843050491562],
|
||||
"size": [285.81015625, 130],
|
||||
"flags": {},
|
||||
"order": 6,
|
||||
"mode": 0,
|
||||
"inputs": [{"name": "images", "type": "IMAGE", "link": 155}],
|
||||
"outputs": [{"name": "upscaled_images", "type": "IMAGE", "links": [156]}],
|
||||
"properties": {"Node name for S&R": "RTXVideoSuperResolution"},
|
||||
"widgets_values": ["target dimensions", 1920, 1080, "ULTRA"],
|
||||
"widgets_values_named": {"resize_type": "target dimensions", "resize_type.width": 1920, "resize_type.height": 1080, "quality": "ULTRA"}
|
||||
},
|
||||
{
|
||||
"id": 68,
|
||||
"type": "CreateVideo",
|
||||
"pos": [5404.822730085631, 2911.779640536847],
|
||||
"size": [270, 150],
|
||||
"flags": {},
|
||||
"order": 8,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{"name": "images", "type": "IMAGE", "link": 156},
|
||||
{"name": "audio", "shape": 7, "type": "AUDIO", "link": 157},
|
||||
{"name": "fps", "type": "FLOAT", "widget": {"name": "fps"}, "link": 159}
|
||||
],
|
||||
"outputs": [{"name": "VIDEO", "type": "VIDEO", "links": [158]}],
|
||||
"properties": {"Node name for S&R": "CreateVideo"},
|
||||
"widgets_values": [30, "auto", "sRGB", "h264"],
|
||||
"widgets_values_named": {"fps": 30, "bit_depth": "auto", "color_space": "sRGB", "codec": "h264"}
|
||||
},
|
||||
{
|
||||
"id": 69,
|
||||
"type": "SaveVideo",
|
||||
"pos": [5174.9019386914315, 3158.8462584926756],
|
||||
"size": [270, 249.875],
|
||||
"flags": {},
|
||||
"order": 10,
|
||||
"mode": 0,
|
||||
"inputs": [{"name": "video", "type": "VIDEO", "link": 158}],
|
||||
"outputs": [{"name": "video", "type": "VIDEO", "links": null}],
|
||||
"properties": {"Node name for S&R": "SaveVideo"},
|
||||
"widgets_values": ["video/ComfyUI", "auto", "auto", "auto"],
|
||||
"widgets_values_named": {"filename_prefix": "video/ComfyUI", "format": "auto", "format.codec": "auto", "codec": "auto"}
|
||||
},
|
||||
{
|
||||
"id": 81,
|
||||
"type": "easy multiTrackEditor",
|
||||
"pos": [2903.9332978768557, 2791.1687123187226],
|
||||
"size": [800, 700],
|
||||
"flags": {},
|
||||
"order": 1,
|
||||
"mode": 0,
|
||||
"inputs": [
|
||||
{"name": "prompt_override", "shape": 7, "type": "*", "link": null},
|
||||
{"name": "image", "shape": 7, "type": "IMAGE", "link": null},
|
||||
{"name": "audio", "shape": 7, "type": "AUDIO", "link": null},
|
||||
{"name": "video", "shape": 7, "type": "VIDEO", "link": null}
|
||||
],
|
||||
"outputs": [
|
||||
{"name": "TRACKS_INFO", "type": "TRACKS_INFO", "links": [166]},
|
||||
{"name": "IMAGES", "shape": 6, "type": "IMAGE", "links": null},
|
||||
{"name": "AUDIO", "shape": 6, "type": "AUDIO", "links": null},
|
||||
{"name": "VIDEO", "shape": 6, "type": "VIDEO", "links": null}
|
||||
],
|
||||
"properties": {"easyMediaTimelineHeight": 700, "Node name for S&R": "easy multiTrackEditor"},
|
||||
"widgets_values": [
|
||||
"1920 x 1080 (16:9)",
|
||||
"stretch",
|
||||
"MiniMax",
|
||||
"{\"muted\":false,\"volume_db\":0,\"task_markers\":[],\"task_overview\":false,\"tracks\":[{\"id\":\"ac9830d5-1ffc-46a7-99ac-19989de1ec17\",\"name\":\"Task 0\",\"type\":\"task\",\"task_mode\":\"default\",\"color\":\"var(--multitrack-task-bg)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[{\"id\":\"389405fd-d576-45ab-b917-a4a4bbda7153\",\"start_frame\":0,\"end_frame\":120,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"shot\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene1\"}},{\"id\":\"b02ca679-4bb1-4758-86cc-a29d95d7d13a\",\"start_frame\":120,\"end_frame\":240,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"context\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene2\"}}]},{\"id\":\"49025400-b968-42bd-9e01-fda75549ca52\",\"name\":\"Video 0\",\"type\":\"video\",\"color\":\"var(--primary)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[],\"audio_locked\":false}],\"total_length\":240,\"frame_rate\":24}"
|
||||
],
|
||||
"widgets_values_named": {
|
||||
"resolution": "1920 x 1080 (16:9)",
|
||||
"resolution.resize_method": "stretch",
|
||||
"format": "MiniMax",
|
||||
"track_data": "{\"muted\":false,\"volume_db\":0,\"task_markers\":[],\"task_overview\":false,\"tracks\":[{\"id\":\"ac9830d5-1ffc-46a7-99ac-19989de1ec17\",\"name\":\"Task 0\",\"type\":\"task\",\"task_mode\":\"default\",\"color\":\"var(--multitrack-task-bg)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[{\"id\":\"389405fd-d576-45ab-b917-a4a4bbda7153\",\"start_frame\":0,\"end_frame\":120,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"shot\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene1\"}},{\"id\":\"b02ca679-4bb1-4758-86cc-a29d95d7d13a\",\"start_frame\":120,\"end_frame\":240,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"context\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene2\"}}]},{\"id\":\"49025400-b968-42bd-9e01-fda75549ca52\",\"name\":\"Video 0\",\"type\":\"video\",\"color\":\"var(--primary)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[],\"audio_locked\":false}],\"total_length\":240,\"frame_rate\":24}"
|
||||
}
|
||||
}
|
||||
],
|
||||
"links": [
|
||||
[136, 52, 1, 57, 2, "CLIP"],
|
||||
[137, 52, 2, 57, 3, "VAE"],
|
||||
[138, 52, 3, 57, 4, "VAE"],
|
||||
[144, 48, 0, 57, 1, "MODEL"],
|
||||
[148, 57, 0, 63, 0, "STRING"],
|
||||
[149, 63, 1, 45, 1, "AUDIO"],
|
||||
[150, 63, 2, 45, 2, "FLOAT"],
|
||||
[151, 63, 0, 45, 0, "IMAGE"],
|
||||
[153, 52, 0, 66, 0, "MODEL"],
|
||||
[154, 66, 0, 48, 0, "MODEL"],
|
||||
[156, 67, 0, 68, 0, "IMAGE"],
|
||||
[157, 63, 1, 68, 1, "AUDIO"],
|
||||
[159, 63, 2, 68, 2, "FLOAT"],
|
||||
[166, 81, 0, 57, 0, "TRACKS_INFO"],
|
||||
[103, 45, 0, 46, 0, "VIDEO"],
|
||||
[155, 63, 0, 67, 0, "IMAGE"],
|
||||
[158, 68, 0, 69, 0, "VIDEO"]
|
||||
],
|
||||
"groups": [],
|
||||
"config": {},
|
||||
"extra": {
|
||||
"ue_links": [],
|
||||
"ds": {"scale": 0.5131581182307068, "offset": [-2322.4149844419144, -2181.0583714987984]},
|
||||
"links_added_by_ue": [],
|
||||
"frontendVersion": "1.53.6",
|
||||
"VHS_latentpreview": false,
|
||||
"VHS_latentpreviewrate": 0,
|
||||
"VHS_MetadataImage": true,
|
||||
"VHS_KeepIntermediate": true
|
||||
},
|
||||
"version": 0.4
|
||||
}
|
||||
73
bench/moxie_bench.spec
Normal file
73
bench/moxie_bench.spec
Normal file
@ -0,0 +1,73 @@
|
||||
# -*- mode: python ; coding: utf-8 -*-
|
||||
"""PyInstaller spec for the Moxie bench sidecar.
|
||||
|
||||
Build (on the target machine, inside the repo root):
|
||||
|
||||
pip install -r bench/requirements.txt pyinstaller
|
||||
cd frontend && bun install && bun run build:bench:release && cd ..
|
||||
pyinstaller bench/moxie_bench.spec
|
||||
|
||||
Output: dist/moxie-bench/ (onedir - ships moxie-bench(.exe) plus _internal/)
|
||||
|
||||
Tauri 2 sidecar wiring (see bench/tauri/README.md):
|
||||
- copy dist/moxie-bench/ next to your Tauri binary as `moxie-bench/`
|
||||
- point externalBin / or spawn `moxie-bench/moxie-bench --port <free-port>`
|
||||
- poll http://127.0.0.1:<port>/api/health until {"ok": true}
|
||||
"""
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
repo_root = Path(SPECPATH).resolve().parent # bench/ -> repo root
|
||||
|
||||
hiddenimports = [
|
||||
"uvicorn.logging",
|
||||
"uvicorn.loops",
|
||||
"uvicorn.loops.auto",
|
||||
"uvicorn.protocols",
|
||||
"uvicorn.protocols.http",
|
||||
"uvicorn.protocols.http.auto",
|
||||
"uvicorn.protocols.websockets",
|
||||
"uvicorn.protocols.websockets.auto",
|
||||
"uvicorn.lifespan",
|
||||
"uvicorn.lifespan.on",
|
||||
"anyio._backends._asyncio",
|
||||
]
|
||||
|
||||
a = Analysis(
|
||||
["bench/run.py"],
|
||||
pathex=[str(repo_root / "bench")],
|
||||
binaries=[],
|
||||
datas=[
|
||||
# the prebuilt bench UI + base workflow travel with the binary
|
||||
(str(repo_root / "bench" / "ui-dist"), "bench/ui-dist"),
|
||||
(str(repo_root / "bench" / "base_workflow.json"), "bench"),
|
||||
],
|
||||
hiddenimports=hiddenimports,
|
||||
hookspath=[],
|
||||
hooksconfig={},
|
||||
runtime_hooks=[],
|
||||
excludes=["torch", "comfy", "comfy_execution", "folder_paths"],
|
||||
noarchive=False,
|
||||
)
|
||||
pyz = PYZ(a.pure)
|
||||
|
||||
exe = EXE(
|
||||
pyz,
|
||||
a.scripts,
|
||||
[],
|
||||
exclude_binaries=True,
|
||||
name="moxie-bench",
|
||||
debug=False,
|
||||
bootloader_ignore_signals=False,
|
||||
strip=False,
|
||||
upx=False,
|
||||
console=True, # sidecar logs are useful; Tauri can hide the window
|
||||
)
|
||||
coll = COLLECT(
|
||||
exe,
|
||||
a.binaries,
|
||||
a.datas,
|
||||
strip=False,
|
||||
upx=False,
|
||||
name="moxie-bench",
|
||||
)
|
||||
4
bench/requirements.txt
Normal file
4
bench/requirements.txt
Normal file
@ -0,0 +1,4 @@
|
||||
fastapi>=0.110
|
||||
uvicorn>=0.29
|
||||
numpy>=1.26
|
||||
pillow>=10.0
|
||||
33
bench/run.py
Normal file
33
bench/run.py
Normal file
@ -0,0 +1,33 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Moxie bench entry point.
|
||||
|
||||
python bench/run.py --port 8787 # auto backend (mock here)
|
||||
python bench/run.py --backend comfy --comfyui-url http://127.0.0.1:8188
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||
|
||||
from app.config import BenchConfig # noqa: E402
|
||||
|
||||
|
||||
def main() -> None:
|
||||
import uvicorn
|
||||
|
||||
from app.server import create_app
|
||||
|
||||
cfg = BenchConfig.from_args()
|
||||
backend = cfg.resolve_backend()
|
||||
print(f"[moxie-bench] backend={backend} http://{cfg.host}:{cfg.port}")
|
||||
if backend == "comfy":
|
||||
print(f"[moxie-bench] driving ComfyUI at {cfg.comfyui_url}")
|
||||
else:
|
||||
print("[moxie-bench] self-contained mock renderer (ffmpeg + numpy)")
|
||||
uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port, log_level="info")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
110
bench/tauri/README.md
Normal file
110
bench/tauri/README.md
Normal file
@ -0,0 +1,110 @@
|
||||
# Using the bench as a Tauri 2 sidecar
|
||||
|
||||
The bench is a self-contained Python HTTP server (FastAPI + uvicorn). Packaged
|
||||
with PyInstaller it becomes an `moxie-bench` binary your Tauri 2 app spawns at
|
||||
startup and talks to over `http://127.0.0.1:<port>`.
|
||||
|
||||
## 1. Build the sidecar
|
||||
|
||||
```bash
|
||||
# in the MMnode repo
|
||||
pip install -r bench/requirements.txt pyinstaller
|
||||
cd frontend && bun install && bun run build:bench:release && cd ..
|
||||
pyinstaller bench/moxie_bench.spec
|
||||
# -> dist/moxie-bench/moxie-bench(.exe) + _internal/
|
||||
```
|
||||
|
||||
Ship the whole `dist/moxie-bench/` folder (onedir layout: the binary needs
|
||||
`_internal/` beside it).
|
||||
|
||||
## 2. Bundle it with the Tauri app
|
||||
|
||||
`tauri.conf.json` (resources approach — simplest and cross-platform):
|
||||
|
||||
```json
|
||||
{
|
||||
"bundle": {
|
||||
"resources": {
|
||||
"../MMnode/dist/moxie-bench/": "moxie-bench/"
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The folder lands in the app resource dir; resolve it at runtime with
|
||||
`@tauri-apps/api/path` → `resourceDir()`.
|
||||
|
||||
## 3. Spawn and wait for health
|
||||
|
||||
Rust side (tauri plugin-shell or std::process):
|
||||
|
||||
```rust
|
||||
use std::process::{Command, Stdio};
|
||||
use std::time::Duration;
|
||||
|
||||
let sidecar = app.path().resource_dir()?.join("moxie-bench/moxie-bench");
|
||||
let port: u16 = 8787; // pick a free port, pass it through
|
||||
let mut child = Command::new(sidecar)
|
||||
.arg("--port").arg(port.to_string())
|
||||
.arg("--host").arg("127.0.0.1")
|
||||
.arg("--backend").arg("auto") // mock when no ComfyUI is reachable
|
||||
.stdout(Stdio::null())
|
||||
.stderr(Stdio::null())
|
||||
.spawn()?;
|
||||
|
||||
// poll until ready before loading the webview UI
|
||||
async fn wait_healthy(port: u16) -> bool {
|
||||
let url = format!("http://127.0.0.1:{port}/api/health");
|
||||
for _ in 0..60 {
|
||||
if let Ok(resp) = reqwest::get(&url).await {
|
||||
if resp.status().is_success() { return true; }
|
||||
}
|
||||
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||
}
|
||||
false
|
||||
}
|
||||
```
|
||||
|
||||
## 4. Drive it from the webview
|
||||
|
||||
Everything the bench UI does is plain HTTP/WS on the same origin — the Tauri
|
||||
webview can call the same endpoints:
|
||||
|
||||
| Action | Call |
|
||||
|---|---|
|
||||
| health/backend | `GET /api/health`, `GET /api/config` |
|
||||
| base workflow + timeline | `GET /api/workflow` |
|
||||
| render plan preview | `GET /api/plan` |
|
||||
| queue a render | `POST /api/render {track_data, overrides}` |
|
||||
| live progress | WebSocket `ws://127.0.0.1:<port>/api/ws` |
|
||||
| job list / detail | `GET /api/jobs`, `GET /api/jobs/{id}` |
|
||||
| play/save outputs | `GET /api/view?filename=...&type=output&job=<job_id>` |
|
||||
|
||||
`POST /api/render` body:
|
||||
|
||||
```json
|
||||
{
|
||||
"track_data": { "tracks": [ ... editor JSON ... ] },
|
||||
"overrides": {
|
||||
"width": 1920, "height": 1080, "frame_rate": 24,
|
||||
"steps": 4, "seed": 12345, "project_name": "MM-myproject",
|
||||
"backend": "auto"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## 5. Backends
|
||||
|
||||
- `mock` — self-contained synthetic renderer (numpy + ffmpeg). Deterministic
|
||||
per (prompt, seed); real mp4 + audio on both output branches. Use it to
|
||||
develop the Tauri integration with zero model downloads.
|
||||
- `comfy` — drives a running ComfyUI (`--comfyui-url`, default
|
||||
`http://127.0.0.1:8188`) with the exact base workflow. Real generation, same
|
||||
API surface. Start ComfyUI headless: `python main.py --listen 127.0.0.1`.
|
||||
- `auto` (default) — comfy when reachable, else mock.
|
||||
|
||||
## 6. Shutdown
|
||||
|
||||
Kill the spawned process on app exit (Rust `child.kill()` / `Drop`), or send
|
||||
SIGTERM. The server has no state beyond `work/` on disk; job history lives in
|
||||
memory.
|
||||
196
bench/tests/test_bench_contract.py
Normal file
196
bench/tests/test_bench_contract.py
Normal file
@ -0,0 +1,196 @@
|
||||
"""Bench contract tests: no ComfyUI, no torch required.
|
||||
|
||||
Run: python3 -m pytest bench/tests -q
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
BENCH_DIR = Path(__file__).resolve().parent.parent
|
||||
sys.path.insert(0, str(BENCH_DIR))
|
||||
|
||||
from app.tracks import TrackDataError, build_render_plan # noqa: E402
|
||||
from app.workflow import ( # noqa: E402
|
||||
EDITOR_TYPE,
|
||||
WorkflowError,
|
||||
describe,
|
||||
editor_track_data,
|
||||
load_base_workflow,
|
||||
parse_workflow_dict,
|
||||
)
|
||||
|
||||
BASE = load_base_workflow(BENCH_DIR / "base_workflow.json")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# workflow parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_all_nine_stage_types_present():
|
||||
types = set(BASE.stages)
|
||||
assert {
|
||||
"MoxieMultimediaLoader", "LoraLoaderModelOnly", "MoxiePreviewOverride",
|
||||
EDITOR_TYPE, "MoxieProject", "MoxieCombine", "RTXVideoSuperResolution",
|
||||
"CreateVideo",
|
||||
} <= types
|
||||
assert "CreateVideo#2" in types
|
||||
|
||||
|
||||
def test_save_nodes_classified_by_branch():
|
||||
assert set(BASE.save_nodes) == {"native", "upscaled"}
|
||||
# native save is fed by CreateVideo #45, upscaled by #68
|
||||
native_src = BASE.save_nodes["native"].link_inputs["video"][0]
|
||||
upscaled_src = BASE.save_nodes["upscaled"].link_inputs["video"][0]
|
||||
assert native_src == BASE.stages["CreateVideo"].node_id
|
||||
assert upscaled_src == BASE.stages["CreateVideo#2"].node_id
|
||||
|
||||
|
||||
def test_describe_summary():
|
||||
d = describe(BASE)
|
||||
assert d["editor_resolution"] == {"width": 1920, "height": 1080}
|
||||
assert d["frame_rate"] == 24
|
||||
assert d["steps"] == 4
|
||||
assert d["seed"] == 803100406447983
|
||||
assert d["project_name"] == "MM-girlwaving"
|
||||
assert d["upscale"]["target"] == {"width": 1920, "height": 1080}
|
||||
assert d["upscale"]["quality"] == "ULTRA"
|
||||
assert d["lora"].endswith(".safetensors")
|
||||
|
||||
|
||||
def test_editor_track_data_roundtrip():
|
||||
td = editor_track_data(BASE)
|
||||
assert td["frame_rate"] == 24
|
||||
assert td["total_length"] == 240
|
||||
task_tracks = [t for t in td["tracks"] if t["type"] == "task"]
|
||||
assert len(task_tracks) == 1
|
||||
assert len(task_tracks[0]["segments"]) == 2
|
||||
|
||||
|
||||
def test_missing_stage_rejected():
|
||||
raw = json.loads((BENCH_DIR / "base_workflow.json").read_text())
|
||||
raw["nodes"] = [n for n in raw["nodes"] if n["type"] != "MoxieProject"]
|
||||
with pytest.raises(WorkflowError):
|
||||
parse_workflow_dict(raw)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# track_data -> RenderPlan
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _plan(**kw):
|
||||
defaults = dict(width=1920, height=1080, frame_rate=24, steps=4, seed=7,
|
||||
project_name="MM-girlwaving", upscale_width=1920,
|
||||
upscale_height=1080)
|
||||
defaults.update(kw)
|
||||
return build_render_plan(editor_track_data(BASE), **defaults)
|
||||
|
||||
|
||||
def test_plan_segments_ordered_and_bounded():
|
||||
plan = _plan()
|
||||
assert [(s.prompt, s.start_frame, s.end_frame) for s in plan.segments] == [
|
||||
("scene1", 0, 120), ("scene2", 120, 240)]
|
||||
assert plan.duration_seconds == pytest.approx(10.0)
|
||||
|
||||
|
||||
def test_plan_honors_overrides():
|
||||
plan = _plan(width=1088, height=1920, steps=8, seed=99)
|
||||
assert (plan.width, plan.height) == (1088, 1920)
|
||||
assert plan.steps == 8 and plan.seed == 99
|
||||
|
||||
|
||||
def test_continuity_mode_carried_through():
|
||||
plan = _plan()
|
||||
modes = {s.prompt: s.continuity_mode for s in plan.segments}
|
||||
assert modes == {"scene1": "shot", "scene2": "context"}
|
||||
|
||||
|
||||
def test_invalid_track_data_raises():
|
||||
with pytest.raises(TrackDataError):
|
||||
build_render_plan("not json", width=1, height=1)
|
||||
with pytest.raises(TrackDataError):
|
||||
build_render_plan({"nope": 1}, width=1, height=1)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# /prompt conversion (comfy bridge)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def test_prompt_conversion_shape():
|
||||
from app.backends.comfy import to_prompt_format
|
||||
|
||||
prompt = to_prompt_format(BASE.raw)
|
||||
assert set(prompt) == {str(n["id"]) for n in BASE.raw["nodes"]}
|
||||
loader = prompt["52"]
|
||||
assert loader["class_type"] == "MoxieMultimediaLoader"
|
||||
assert loader["inputs"]["diffusion_model"] == "Moxie-Multimedia.safetensors"
|
||||
project = prompt["57"]
|
||||
assert project["inputs"]["tracks_info"] == ["81", 0]
|
||||
assert project["inputs"]["model"] == ["48", 0]
|
||||
assert project["inputs"]["clip"] == ["52", 1]
|
||||
assert "randomize" not in project["inputs"] # UI-only control widget
|
||||
|
||||
|
||||
def test_prompt_conversion_wired_widget_wins():
|
||||
from app.backends.comfy import to_prompt_format
|
||||
|
||||
prompt = to_prompt_format(BASE.raw)
|
||||
# CreateVideo #45: fps widget is wired to MoxieCombine slot 2
|
||||
create = prompt["45"]
|
||||
assert create["inputs"]["fps"] == ["63", 2]
|
||||
assert create["inputs"]["codec"] == "h264"
|
||||
|
||||
|
||||
def test_substitution_updates_all_parameters():
|
||||
from app.backends.comfy import substitute
|
||||
|
||||
plan = build_render_plan(
|
||||
editor_track_data(BASE), width=720, height=1280, frame_rate=16,
|
||||
steps=6, seed=1234, project_name="MM-bench", lora="MM3step.safetensors",
|
||||
lora_strength=0.8, upscale_width=1440, upscale_height=2560,
|
||||
)
|
||||
raw = substitute(BASE, plan)
|
||||
nodes = {n["id"]: n for n in raw["nodes"]}
|
||||
|
||||
editor = nodes[81]["widgets_values_named"]
|
||||
assert json.loads(editor["track_data"])["frame_rate"] == 16
|
||||
assert editor["resolution"] == "720 x 1280 (9:16)"
|
||||
|
||||
proj = nodes[57]["widgets_values_named"]
|
||||
assert proj["project_name"] == "MM-bench" and proj["steps"] == 6
|
||||
assert proj["seed"] == 1234
|
||||
|
||||
lora = nodes[66]["widgets_values_named"]
|
||||
assert lora["lora_name"] == "MM3step.safetensors" and lora["strength_model"] == 0.8
|
||||
|
||||
rtx = nodes[67]["widgets_values_named"]
|
||||
assert rtx["resize_type.width"] == 1440 and rtx["resize_type.height"] == 2560
|
||||
|
||||
combine = nodes[63]["widgets_values_named"]
|
||||
pd = json.loads(combine["project_data"])
|
||||
assert pd["width"] == 720 and pd["height"] == 1280 and pd["frame_rate"] == 16
|
||||
|
||||
saves = {n["id"]: n for n in raw["nodes"] if n["type"] == "SaveVideo"}
|
||||
assert saves[46]["widgets_values_named"]["filename_prefix"] == "moxie-bench/native"
|
||||
assert saves[69]["widgets_values_named"]["filename_prefix"] == "moxie-bench/upscaled"
|
||||
|
||||
|
||||
def test_substitution_custom_resolution_uses_dict():
|
||||
from app.backends.comfy import substitute
|
||||
|
||||
plan = build_render_plan(editor_track_data(BASE), width=998, height=742)
|
||||
raw = substitute(BASE, plan)
|
||||
editor = next(n for n in raw["nodes"] if n["id"] == 81)
|
||||
assert editor["widgets_values_named"]["resolution"] == {"width": 998, "height": 742}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# mock backend end-to-end (ffmpeg required)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@pytest.mark.skipif(True, reason="e2e covered by scripts/test_bench_pipeline.py; keep pytest fast")
|
||||
def test_mock_render_placeholder():
|
||||
pass
|
||||
16
bench/ui-dist/index.html
Normal file
16
bench/ui-dist/index.html
Normal file
@ -0,0 +1,16 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Moxie Multimedia Bench</title>
|
||||
<style>
|
||||
html, body, #root { height: 100%; margin: 0; }
|
||||
body { background: #09090b; color: #fafafa; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="./main.js"></script>
|
||||
</body>
|
||||
</html>
|
||||
71
bench/ui-dist/main.js
Normal file
71
bench/ui-dist/main.js
Normal file
File diff suppressed because one or more lines are too long
116
download_models.sh
Executable file
116
download_models.sh
Executable file
@ -0,0 +1,116 @@
|
||||
#!/usr/bin/env bash
|
||||
#
|
||||
# download_models.sh - fetch the MiniMax H3 model set for the Moxie Multimedia
|
||||
# Loader and place it (renamed) into this node pack's models/ folders.
|
||||
#
|
||||
# Usage (from anywhere - the script targets its own pack directory):
|
||||
# bash custom_nodes/MMnode/download_models.sh
|
||||
#
|
||||
# Files are downloaded to "<name>.part" first and moved into place when
|
||||
# complete, so an interrupted run can be re-run safely (it resumes).
|
||||
# Already-complete files are skipped, so the script is idempotent.
|
||||
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
MODELS_DIR="$SCRIPT_DIR/models"
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Model manifest: subdir/filename <- URL
|
||||
# Add the turbo LoRA here once a URL is available, e.g.:
|
||||
# "loras/Moxie-Turbo.safetensors|https://.../Moxie-Turbo.safetensors"
|
||||
# ---------------------------------------------------------------------------
|
||||
MANIFEST=(
|
||||
"diffusion/Moxie-Multimedia.safetensors|https://huggingface.co/WarmBloodAban/Minimax-h3_Singularity/resolve/main/Minimax-h3_Singularity_ref2va_v1.3_Pruned_w4a8.safetensors"
|
||||
"clip/MM-VL.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/text_encoders/qwen3vl_32b_minimax_h3_nvfp4_awq.safetensors"
|
||||
"vae/video_vae.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_video_vae_int8_convrot.safetensors"
|
||||
"audio_vae/audio_vae.safetensors|https://huggingface.co/Comfy-Org/MiniMax-H3/resolve/main/vae/minimax_h3_audio_vae_fp32.safetensors"
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Downloader: curl first (resume via -C -), wget as fallback.
|
||||
# ---------------------------------------------------------------------------
|
||||
curl_supports_retry_all() {
|
||||
# --retry-all-errors exists since curl 7.71.0; compare versions safely.
|
||||
local v
|
||||
v="$(curl --version 2>/dev/null | head -n1 | awk '{print $2}')"
|
||||
[[ "$(printf '%s\n7.71.0\n' "$v" | sort -V | head -n1)" == "7.71.0" ]]
|
||||
}
|
||||
|
||||
fetch() {
|
||||
local url="$1" output="$2"
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
local retry_all=""
|
||||
if curl_supports_retry_all; then
|
||||
retry_all="--retry-all-errors" # HuggingFace answers bursts with 429/503
|
||||
fi
|
||||
curl -L --fail --retry 5 --retry-delay 5 $retry_all \
|
||||
--connect-timeout 30 -C - -o "$output" "$url"
|
||||
elif command -v wget >/dev/null 2>&1; then
|
||||
wget -c -q --show-progress -O "$output" "$url"
|
||||
else
|
||||
echo "ERROR: neither curl nor wget is installed." >&2
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
download_entry() {
|
||||
local entry="$1"
|
||||
local rel="${entry%%|*}"
|
||||
local url="${entry#*|}"
|
||||
local subdir="${rel%%/*}"
|
||||
local filename="${rel#*/}"
|
||||
local target_dir="$MODELS_DIR/$subdir"
|
||||
local target="$target_dir/$filename"
|
||||
local partial="$target.part"
|
||||
|
||||
if [[ -f "$target" ]]; then
|
||||
echo "[skip] $subdir/$filename already present"
|
||||
return 0
|
||||
fi
|
||||
|
||||
mkdir -p "$target_dir"
|
||||
echo "[get ] $subdir/$filename"
|
||||
echo " from $url"
|
||||
|
||||
if ! fetch "$url" "$partial"; then
|
||||
# A stale/incomplete .part can make resume fail (e.g. HTTP 416 when the
|
||||
# file was already fully fetched but the move never happened).
|
||||
if [[ -f "$partial" ]]; then
|
||||
echo " resume failed - deleting partial and restarting download"
|
||||
rm -f "$partial"
|
||||
fetch "$url" "$partial"
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
fi
|
||||
|
||||
mv -f "$partial" "$target"
|
||||
echo "[done] $subdir/$filename ($(du -h "$target" | cut -f1))"
|
||||
}
|
||||
|
||||
echo "Moxie Multimedia - model downloader"
|
||||
echo " pack models dir: $MODELS_DIR"
|
||||
echo
|
||||
|
||||
failed=0
|
||||
for entry in "${MANIFEST[@]}"; do
|
||||
if ! download_entry "$entry"; then
|
||||
echo "ERROR: failed to download $entry" >&2
|
||||
failed=1
|
||||
fi
|
||||
done
|
||||
|
||||
echo
|
||||
if [[ "$failed" -eq 0 ]]; then
|
||||
echo "All models ready:"
|
||||
for entry in "${MANIFEST[@]}"; do
|
||||
local_rel="${entry%%|*}"
|
||||
printf ' %-40s %s\n' "$local_rel" "$(du -h "$MODELS_DIR/$local_rel" 2>/dev/null | cut -f1 || echo missing)"
|
||||
done
|
||||
echo
|
||||
echo "Restart ComfyUI so the loader picks the files up."
|
||||
else
|
||||
echo "Some downloads failed - re-run this script to resume." >&2
|
||||
exit 1
|
||||
fi
|
||||
103
frontend/build.bench.ts
Normal file
103
frontend/build.bench.ts
Normal file
@ -0,0 +1,103 @@
|
||||
/**
|
||||
* Bench UI build — mirrors build.ts but outputs a standalone web app into
|
||||
* ../bench/ui-dist (served by the bench FastAPI server).
|
||||
*
|
||||
* bun run build.bench.ts # dev build
|
||||
* bun run build.bench.ts --release # minified
|
||||
*/
|
||||
import { $ } from "bun";
|
||||
import path from "path";
|
||||
import fs from "fs";
|
||||
import os from "os";
|
||||
|
||||
const isRelease = Bun.argv.includes("--release");
|
||||
|
||||
const rootDir = path.resolve(import.meta.dir, "..");
|
||||
const outdir = path.join(rootDir, "bench", "ui-dist");
|
||||
const project_name = path.basename(rootDir);
|
||||
|
||||
fs.mkdirSync(outdir, { recursive: true });
|
||||
|
||||
function scopeTailwindThemeVariables(css: string, widgetClass: string): string {
|
||||
return css.replace(/:root\s*,\s*:host\s*\{/g, `.${widgetClass}{`);
|
||||
}
|
||||
|
||||
async function buildCSS(): Promise<string> {
|
||||
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "moxie-bench-css-"));
|
||||
const cssOut = path.join(tempDir, "globals.css");
|
||||
const minifyFlag = isRelease ? "--minify" : "";
|
||||
try {
|
||||
await $`./node_modules/.bin/tailwindcss -i ./src/styles/globals.css -o ${cssOut} ${minifyFlag}`;
|
||||
const widgetClass = project_name.toLowerCase();
|
||||
const css = fs.readFileSync(cssOut, "utf-8");
|
||||
return scopeTailwindThemeVariables(
|
||||
css.replaceAll("__CUSTOM_NODE_CLASS__", widgetClass),
|
||||
widgetClass,
|
||||
);
|
||||
} finally {
|
||||
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
|
||||
async function build() {
|
||||
const start = Date.now();
|
||||
const globalCss = await buildCSS();
|
||||
|
||||
const result = await Bun.build({
|
||||
entrypoints: ["./src/bench/main.tsx"],
|
||||
outdir,
|
||||
naming: {
|
||||
chunk: "chunks/[name]-[hash].[ext]",
|
||||
entry: "[name].[ext]",
|
||||
asset: "assets/[name].[ext]",
|
||||
},
|
||||
splitting: true,
|
||||
target: "browser",
|
||||
minify: isRelease,
|
||||
define: {
|
||||
"process.env.NODE_ENV": isRelease ? '"production"' : '"development"',
|
||||
"import.meta.env.PROJECT_NAME": `"${project_name}"`,
|
||||
"__COMFY_EASY_MEDIA_GLOBAL_CSS__": JSON.stringify(globalCss),
|
||||
},
|
||||
css: false,
|
||||
});
|
||||
|
||||
if (!result.success) {
|
||||
for (const log of result.logs) {
|
||||
console.error(log);
|
||||
}
|
||||
throw new Error("bench UI build failed");
|
||||
}
|
||||
|
||||
// index.html referencing the built entry
|
||||
const entryName = result.outputs
|
||||
.map((o) => o.path)
|
||||
.filter((p) => p.endsWith("main.js") || p.endsWith("main.js.map"))
|
||||
.map((p) => path.basename(p))
|
||||
.find((p) => p.endsWith(".js"));
|
||||
|
||||
const html = `<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Moxie Multimedia Bench</title>
|
||||
<style>
|
||||
html, body, #root { height: 100%; margin: 0; }
|
||||
body { background: #09090b; color: #fafafa; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div id="root"></div>
|
||||
<script type="module" src="./${entryName ?? "main.js"}"></script>
|
||||
</body>
|
||||
</html>
|
||||
`;
|
||||
fs.writeFileSync(path.join(outdir, "index.html"), html);
|
||||
|
||||
console.log(
|
||||
`[build:bench${isRelease ? ":release" : ""}] Done in ${Date.now() - start}ms (${result.outputs.length} outputs) -> ${outdir}`,
|
||||
);
|
||||
}
|
||||
|
||||
await build();
|
||||
@ -8,7 +8,9 @@
|
||||
"build": "bun run build.ts",
|
||||
"build:release": "bun run build.ts --release",
|
||||
"test": "vitest run",
|
||||
"test:watch": "vitest"
|
||||
"test:watch": "vitest",
|
||||
"build:bench": "bun run build.bench.ts",
|
||||
"build:bench:release": "bun run build.bench.ts --release"
|
||||
},
|
||||
"dependencies": {
|
||||
"@base-ui/react": "^1.6.0",
|
||||
@ -51,4 +53,4 @@
|
||||
"typescript": "^5.8.3",
|
||||
"vitest": "^3.1.4"
|
||||
}
|
||||
}
|
||||
}
|
||||
268
frontend/src/bench/BenchApp.tsx
Normal file
268
frontend/src/bench/BenchApp.tsx
Normal file
@ -0,0 +1,268 @@
|
||||
import { useEffect, useMemo, useReducer, useRef, useState } from 'react'
|
||||
import { Loader2, Play, Radio, SquareTerminal } from 'lucide-react'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Input } from '@/components/ui/input'
|
||||
import { TooltipProvider } from '@/components/ui/tooltip'
|
||||
import { MultiTrackWidget } from '@/components/widgets/MultiTrackWidget'
|
||||
import type { TrackData } from '@/types/multitrack'
|
||||
import {
|
||||
benchApp,
|
||||
benchState,
|
||||
connectBenchSocket,
|
||||
createBenchNode,
|
||||
createBenchWidget,
|
||||
fetchBenchConfig,
|
||||
onBenchStateChange,
|
||||
queueBenchRender,
|
||||
type BenchOutputInfo,
|
||||
} from './bench-runtime'
|
||||
|
||||
const RESOLUTION_PRESETS: Array<{ label: string; width: number; height: number }> = [
|
||||
{ label: '1920 x 1080 (16:9)', width: 1920, height: 1080 },
|
||||
{ label: '1088 x 1920 (9:16)', width: 1088, height: 1920 },
|
||||
{ label: '1280 x 720 (16:9)', width: 1280, height: 720 },
|
||||
{ label: '720 x 1280 (9:16)', width: 720, height: 1280 },
|
||||
{ label: '832 x 480 (16:9)', width: 832, height: 480 },
|
||||
{ label: '480 x 832 (9:16)', width: 480, height: 832 },
|
||||
{ label: '1024 x 1024 (1:1)', width: 1024, height: 1024 },
|
||||
]
|
||||
|
||||
function useBenchTick(): number {
|
||||
const [, force] = useReducer((x: number) => x + 1, 0)
|
||||
useEffect(() => onBenchStateChange(force), [force])
|
||||
return 0
|
||||
}
|
||||
|
||||
function OutputCard({ label, out }: { label: string; out: BenchOutputInfo }) {
|
||||
return (
|
||||
<div className="flex min-w-0 flex-1 flex-col gap-1 rounded-md border bg-background p-2">
|
||||
<div className="flex items-center justify-between text-xs text-muted-foreground">
|
||||
<span className="font-medium text-foreground">{label}</span>
|
||||
<span>
|
||||
{out.width}x{out.height} · {out.fps ?? '?'} fps · {Math.round((out.bytes ?? 0) / 1024)} KB
|
||||
</span>
|
||||
</div>
|
||||
<video className="w-full rounded-sm" src={out.url} controls preload="metadata" />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export function BenchApp() {
|
||||
useBenchTick()
|
||||
const [trackData, setTrackData] = useState<TrackData | null>(null)
|
||||
const [config, setConfig] = useState<Awaited<ReturnType<typeof fetchBenchConfig>>>(null)
|
||||
const [backend, setBackend] = useState('auto')
|
||||
const [resolution, setResolution] = useState('1920 x 1080 (16:9)')
|
||||
const [steps, setSteps] = useState(4)
|
||||
const [seed, setSeed] = useState(803100406447983)
|
||||
const [projectName, setProjectName] = useState('MM-girlwaving')
|
||||
const [queueing, setQueueing] = useState(false)
|
||||
const node = useMemo(createBenchNode, [])
|
||||
const widget = useMemo(createBenchWidget, [])
|
||||
const consoleRef = useRef<HTMLDivElement>(null)
|
||||
|
||||
useEffect(() => {
|
||||
connectBenchSocket()
|
||||
void fetchBenchConfig().then(setConfig)
|
||||
fetch('/api/workflow')
|
||||
.then((r) => r.json())
|
||||
.then((body: { track_data: string }) => {
|
||||
try {
|
||||
const parsed = JSON.parse(body.track_data) as TrackData
|
||||
setTrackData(parsed)
|
||||
setProjectName((current) => current || 'MM-girlwaving')
|
||||
} catch {
|
||||
/* server sent unusable track data; keep the empty editor */
|
||||
}
|
||||
})
|
||||
.catch(() => undefined)
|
||||
}, [])
|
||||
|
||||
useEffect(() => {
|
||||
if (consoleRef.current) consoleRef.current.scrollTop = consoleRef.current.scrollHeight
|
||||
})
|
||||
|
||||
const preset = RESOLUTION_PRESETS.find((p) => p.label === resolution)
|
||||
|
||||
const queue = async () => {
|
||||
if (!trackData || queueing) return
|
||||
setQueueing(true)
|
||||
try {
|
||||
await queueBenchRender({
|
||||
track_data: trackData,
|
||||
backend: backend === 'auto' ? undefined : backend,
|
||||
overrides: {
|
||||
width: preset?.width ?? 1920,
|
||||
height: preset?.height ?? 1080,
|
||||
steps,
|
||||
seed,
|
||||
project_name: projectName,
|
||||
},
|
||||
})
|
||||
} catch (error) {
|
||||
benchState.error = error instanceof Error ? error.message : String(error)
|
||||
} finally {
|
||||
setQueueing(false)
|
||||
}
|
||||
}
|
||||
|
||||
const busy = benchState.jobStatus === 'queued' || benchState.jobStatus === 'running'
|
||||
const outputs = benchState.outputs ?? {}
|
||||
|
||||
return (
|
||||
<TooltipProvider>
|
||||
<div className="mmnode flex h-screen min-h-0 flex-col gap-2 bg-background p-3 text-foreground">
|
||||
<header className="flex flex-wrap items-center gap-3">
|
||||
<h1 className="text-lg font-semibold">Moxie Multimedia Bench</h1>
|
||||
<span
|
||||
className={`inline-flex items-center gap-1 rounded-full px-2 py-0.5 text-xs ${
|
||||
benchState.connected
|
||||
? 'bg-green-500/15 text-green-600'
|
||||
: 'bg-red-500/15 text-red-600'
|
||||
}`}
|
||||
>
|
||||
<Radio size={12} /> {benchState.connected ? 'server online' : 'offline'}
|
||||
</span>
|
||||
<span className="rounded-full bg-muted px-2 py-0.5 text-xs text-muted-foreground">
|
||||
backend: {config?.backend ?? benchState.backend}
|
||||
</span>
|
||||
<span className="text-xs text-muted-foreground">v{config?.version ?? '?'}</span>
|
||||
<div className="ml-auto" />
|
||||
</header>
|
||||
|
||||
<div className="flex min-h-0 flex-1 gap-3">
|
||||
<main className="mmnode flex min-h-0 min-w-0 flex-1 overflow-hidden rounded-md border">
|
||||
{trackData ? (
|
||||
<MultiTrackWidget
|
||||
value={trackData}
|
||||
onChange={(next) => setTrackData(next)}
|
||||
inputName="track_data"
|
||||
node={node}
|
||||
widget={widget as never}
|
||||
app={benchApp}
|
||||
/>
|
||||
) : (
|
||||
<div className="flex h-full w-full items-center justify-center text-muted-foreground">
|
||||
<Loader2 className="mr-2 animate-spin" size={18} /> loading timeline…
|
||||
</div>
|
||||
)}
|
||||
</main>
|
||||
|
||||
<aside className="flex w-80 shrink-0 flex-col gap-3 overflow-y-auto rounded-md border p-3">
|
||||
<div className="text-sm font-semibold">Render</div>
|
||||
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Resolution
|
||||
<select
|
||||
className="rounded-md border bg-background p-1.5 text-foreground"
|
||||
value={resolution}
|
||||
onChange={(e) => setResolution(e.target.value)}
|
||||
>
|
||||
{RESOLUTION_PRESETS.map((p) => (
|
||||
<option key={p.label} value={p.label}>
|
||||
{p.label}
|
||||
</option>
|
||||
))}
|
||||
</select>
|
||||
</label>
|
||||
|
||||
<div className="grid grid-cols-2 gap-2">
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Steps
|
||||
<Input
|
||||
type="number"
|
||||
min={1}
|
||||
max={32}
|
||||
value={steps}
|
||||
onChange={(e) => setSteps(Math.max(1, Number(e.target.value) || 1))}
|
||||
/>
|
||||
</label>
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Seed
|
||||
<Input
|
||||
type="number"
|
||||
value={seed}
|
||||
onChange={(e) => setSeed(Number(e.target.value) || 0)}
|
||||
/>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Project name
|
||||
<Input value={projectName} onChange={(e) => setProjectName(e.target.value)} />
|
||||
</label>
|
||||
|
||||
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||
Backend
|
||||
<select
|
||||
className="rounded-md border bg-background p-1.5 text-foreground"
|
||||
value={backend}
|
||||
onChange={(e) => setBackend(e.target.value)}
|
||||
>
|
||||
<option value="auto">auto</option>
|
||||
<option value="mock">mock</option>
|
||||
<option value="comfy">comfy (needs running ComfyUI)</option>
|
||||
</select>
|
||||
{config && !config.backends?.comfy?.available && (
|
||||
<span className="text-[11px] text-muted-foreground">
|
||||
comfy bridge not reachable — mock stays the default
|
||||
</span>
|
||||
)}
|
||||
</label>
|
||||
|
||||
<Button onClick={queue} disabled={busy || queueing || !trackData}>
|
||||
{busy ? <Loader2 className="mr-2 animate-spin" size={14} /> : <Play className="mr-2" size={14} />}
|
||||
{busy ? 'Rendering…' : 'Queue render'}
|
||||
</Button>
|
||||
|
||||
{busy && (
|
||||
<div className="flex flex-col gap-1">
|
||||
<div className="h-2 w-full overflow-hidden rounded-full bg-muted">
|
||||
<div
|
||||
className="h-full rounded-full bg-primary transition-all"
|
||||
style={{ width: `${benchState.progress}%` }}
|
||||
/>
|
||||
</div>
|
||||
<div className="text-xs text-muted-foreground">
|
||||
{benchState.stage || 'waiting…'} — {benchState.message}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
{benchState.error && (
|
||||
<div className="rounded-md bg-red-500/10 p-2 text-xs text-red-600">
|
||||
{benchState.error}
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div className="text-sm font-semibold">Outputs</div>
|
||||
<div className="flex flex-col gap-2">
|
||||
{outputs.native ? (
|
||||
<OutputCard label="native (MoxieCombine)" out={outputs.native} />
|
||||
) : null}
|
||||
{outputs.upscaled ? (
|
||||
<OutputCard label="upscaled (RTX branch)" out={outputs.upscaled} />
|
||||
) : null}
|
||||
{!outputs.native && !outputs.upscaled && (
|
||||
<div className="text-xs text-muted-foreground">
|
||||
queue a render — native and RTX-upscaled outputs appear here
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
<div className="flex items-center gap-1 text-sm font-semibold">
|
||||
<SquareTerminal size={14} /> Console
|
||||
</div>
|
||||
<div
|
||||
ref={consoleRef}
|
||||
className="h-40 overflow-y-auto rounded-md bg-muted/50 p-2 font-mono text-[11px] leading-relaxed"
|
||||
>
|
||||
{benchState.console.map((line, i) => (
|
||||
<div key={i}>{line}</div>
|
||||
))}
|
||||
</div>
|
||||
</aside>
|
||||
</div>
|
||||
</div>
|
||||
</TooltipProvider>
|
||||
)
|
||||
}
|
||||
314
frontend/src/bench/bench-runtime.ts
Normal file
314
frontend/src/bench/bench-runtime.ts
Normal file
@ -0,0 +1,314 @@
|
||||
/**
|
||||
* Bench runtime: the ComfyUI-shaped shim that lets the unmodified editor
|
||||
* components run against the bench server.
|
||||
*
|
||||
* This module MUST be imported before anything that pulls in `@/lib/comfy-api`
|
||||
* (that module reads `globalThis.comfyAPI.app.app` at import time).
|
||||
*/
|
||||
import type { ComfyApp } from '@comfyorg/comfyui-frontend-types'
|
||||
|
||||
type CustomEventCallback = (event: CustomEvent<unknown>) => void
|
||||
|
||||
export interface BenchJobEvent {
|
||||
type: string
|
||||
job_id?: string
|
||||
status?: string
|
||||
stage?: string
|
||||
progress?: number
|
||||
message?: string
|
||||
error?: string
|
||||
outputs?: Record<string, BenchOutputInfo>
|
||||
}
|
||||
|
||||
export interface BenchOutputInfo {
|
||||
filename: string
|
||||
url: string
|
||||
width?: number
|
||||
height?: number
|
||||
fps?: number
|
||||
bytes?: number
|
||||
}
|
||||
|
||||
export interface BenchConfigInfo {
|
||||
version: string
|
||||
backend: string
|
||||
backends: Record<string, { available: boolean; url?: string }>
|
||||
}
|
||||
|
||||
class Hub {
|
||||
private target = new EventTarget()
|
||||
|
||||
add(type: string, cb: CustomEventCallback): void {
|
||||
this.target.addEventListener(type, cb as EventListener)
|
||||
}
|
||||
|
||||
remove(type: string, cb: CustomEventCallback): void {
|
||||
this.target.removeEventListener(type, cb as EventListener)
|
||||
}
|
||||
|
||||
dispatch(type: string, detail: unknown): void {
|
||||
this.target.dispatchEvent(new CustomEvent(type, { detail }))
|
||||
}
|
||||
}
|
||||
|
||||
export const benchHub = new Hub()
|
||||
|
||||
/** Live bench state the dock UI reads. */
|
||||
export const benchState = {
|
||||
connected: false,
|
||||
backend: 'mock',
|
||||
jobId: null as string | null,
|
||||
jobStatus: 'idle',
|
||||
stage: '',
|
||||
progress: 0,
|
||||
message: '',
|
||||
error: null as string | null,
|
||||
outputs: {} as Record<string, BenchOutputInfo>,
|
||||
console: [] as string[],
|
||||
}
|
||||
|
||||
const listeners = new Set<() => void>()
|
||||
|
||||
export function onBenchStateChange(cb: () => void): () => void {
|
||||
listeners.add(cb)
|
||||
return () => {
|
||||
listeners.delete(cb)
|
||||
}
|
||||
}
|
||||
|
||||
function notify(): void {
|
||||
listeners.forEach((cb) => cb())
|
||||
}
|
||||
|
||||
function log(line: string): void {
|
||||
benchState.console.push(`[${new Date().toLocaleTimeString()}] ${line}`)
|
||||
if (benchState.console.length > 400) benchState.console.splice(0, benchState.console.length - 400)
|
||||
notify()
|
||||
}
|
||||
|
||||
function applyJobEvent(msg: BenchJobEvent): void {
|
||||
benchState.jobId = msg.job_id ?? benchState.jobId
|
||||
benchState.jobStatus = msg.status ?? benchState.jobStatus
|
||||
benchState.stage = msg.stage ?? benchState.stage
|
||||
benchState.progress = typeof msg.progress === 'number' ? msg.progress : benchState.progress
|
||||
benchState.message = msg.message ?? benchState.message
|
||||
if (msg.type === 'execution_error') {
|
||||
benchState.error = msg.error ?? 'render failed'
|
||||
log(`ERROR: ${benchState.error}`)
|
||||
}
|
||||
if (msg.type === 'execution_success' && msg.outputs) {
|
||||
benchState.outputs = msg.outputs
|
||||
benchState.error = null
|
||||
log(`done: ${Object.keys(msg.outputs).join(', ')}`)
|
||||
}
|
||||
if (msg.type === 'job_progress') {
|
||||
benchState.error = null
|
||||
}
|
||||
// Mirror ComfyUI's push-event model so any widget listening on app.api
|
||||
// (executed / execution_success / execution_error) keeps working.
|
||||
if (msg.type === 'execution_start') {
|
||||
benchHub.dispatch('execution_start', { prompt_id: benchState.jobId })
|
||||
} else if (msg.type === 'execution_success') {
|
||||
benchHub.dispatch('executed', {
|
||||
prompt_id: benchState.jobId,
|
||||
output: msg.outputs,
|
||||
})
|
||||
benchHub.dispatch('execution_success', { prompt_id: benchState.jobId })
|
||||
} else if (msg.type === 'execution_error') {
|
||||
benchHub.dispatch('execution_error', {
|
||||
prompt_id: benchState.jobId,
|
||||
exception_message: benchState.error,
|
||||
})
|
||||
}
|
||||
notify()
|
||||
}
|
||||
|
||||
let socket: WebSocket | null = null
|
||||
let reconnectTimer: number | null = null
|
||||
let pingTimer: number | null = null
|
||||
|
||||
export function connectBenchSocket(): void {
|
||||
if (typeof window === 'undefined') return
|
||||
const proto = window.location.protocol === 'https:' ? 'wss' : 'ws'
|
||||
const url = `${proto}://${window.location.host}/api/ws`
|
||||
|
||||
const open = () => {
|
||||
socket = new WebSocket(url)
|
||||
socket.onopen = () => {
|
||||
benchState.connected = true
|
||||
log('connected to bench server')
|
||||
notify()
|
||||
pingTimer = window.setInterval(() => {
|
||||
try {
|
||||
socket?.send('ping')
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}, 20_000)
|
||||
}
|
||||
socket.onmessage = (event) => {
|
||||
try {
|
||||
const msg = JSON.parse(event.data as string) as BenchJobEvent
|
||||
applyJobEvent(msg)
|
||||
} catch {
|
||||
/* non-JSON frames are keepalive noise */
|
||||
}
|
||||
}
|
||||
socket.onclose = () => {
|
||||
benchState.connected = false
|
||||
socket = null
|
||||
if (pingTimer) window.clearInterval(pingTimer)
|
||||
notify()
|
||||
reconnectTimer = window.setTimeout(open, 1500)
|
||||
}
|
||||
socket.onerror = () => socket?.close()
|
||||
}
|
||||
open()
|
||||
}
|
||||
|
||||
export async function fetchBenchConfig(): Promise<BenchConfigInfo | null> {
|
||||
try {
|
||||
const res = await fetch('/api/config')
|
||||
if (!res.ok) return null
|
||||
const cfg = (await res.json()) as BenchConfigInfo
|
||||
benchState.backend = cfg.backend
|
||||
notify()
|
||||
return cfg
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
export interface BenchRenderPayload {
|
||||
track_data: unknown
|
||||
overrides?: Record<string, unknown>
|
||||
backend?: string
|
||||
}
|
||||
|
||||
export async function queueBenchRender(payload: BenchRenderPayload): Promise<string> {
|
||||
const res = await fetch('/api/render', {
|
||||
method: 'POST',
|
||||
headers: { 'Content-Type': 'application/json' },
|
||||
body: JSON.stringify(payload),
|
||||
})
|
||||
if (!res.ok) {
|
||||
const detail = await res.text()
|
||||
throw new Error(`queue failed (${res.status}): ${detail.slice(0, 300)}`)
|
||||
}
|
||||
const body = (await res.json()) as { job_id: string }
|
||||
benchState.jobId = body.job_id
|
||||
benchState.jobStatus = 'queued'
|
||||
benchState.error = null
|
||||
benchState.outputs = {}
|
||||
log(`queued render ${body.job_id}`)
|
||||
notify()
|
||||
return body.job_id
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// The ComfyApp-shaped shim
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const toastShim = {
|
||||
add: (msg: { severity?: string; summary?: string; detail?: string; life?: number }) => {
|
||||
const severity = (msg.severity ?? 'info').toString()
|
||||
log(`toast.${severity}: ${msg.summary ?? ''} ${msg.detail ?? ''}`)
|
||||
if (severity === 'error') console.error('[bench-toast]', msg.summary, msg.detail)
|
||||
},
|
||||
}
|
||||
|
||||
const dialogShim = {
|
||||
prompt: async (opts: { title?: string; message?: string }) =>
|
||||
window.prompt(opts?.title ?? '', opts?.message ?? ''),
|
||||
confirm: async (opts: { title?: string; message?: string }) =>
|
||||
window.confirm(`${opts?.title ?? ''}\n${opts?.message ?? ''}`),
|
||||
}
|
||||
|
||||
export const benchApp = {
|
||||
api: {
|
||||
clientId: 'bench-client',
|
||||
fetchApi: (path: string, options?: RequestInit) => fetch(path, options),
|
||||
addEventListener: (_type: string, _cb: EventListener) => {},
|
||||
removeEventListener: (_type: string, _cb: EventListener) => {},
|
||||
addCustomEventListener: (type: string, cb: CustomEventCallback) => benchHub.add(type, cb),
|
||||
removeCustomEventListener: (type: string, cb: CustomEventCallback) => benchHub.remove(type, cb),
|
||||
queuePrompt: async () => {
|
||||
throw new Error('queuePrompt is not used by the bench UI; use the render dock')
|
||||
},
|
||||
dispatchEvent: (ev: Event) => benchHub.dispatch(ev.type, ev),
|
||||
},
|
||||
extensionManager: {
|
||||
toast: toastShim,
|
||||
dialog: dialogShim,
|
||||
workflow: {
|
||||
activeWorkflow: { key: 'bench' },
|
||||
isBusy: false,
|
||||
$subscribe: (_cb: unknown) => () => {},
|
||||
},
|
||||
},
|
||||
ui: {
|
||||
settings: {
|
||||
settingsValues: {
|
||||
'Comfy.Locale': 'en',
|
||||
'Comfy.VueNodes.Enabled': false,
|
||||
},
|
||||
},
|
||||
},
|
||||
canvas: null,
|
||||
graph: {
|
||||
getNodeById: () => null,
|
||||
links: {},
|
||||
},
|
||||
queuePrompt: async () => {
|
||||
throw new Error('queuePrompt is not used by the bench UI; use the render dock')
|
||||
},
|
||||
processingQueue: false,
|
||||
} as unknown as ComfyApp
|
||||
|
||||
// comfy-api.ts reads this at module scope — must exist before that import runs.
|
||||
;(globalThis as Record<string, unknown>).comfyAPI = { app: { app: benchApp } }
|
||||
|
||||
/** Structural stand-in for the LGraphNode the widget normally lives in. */
|
||||
export function createBenchNode(): Record<string, unknown> {
|
||||
return {
|
||||
id: 81,
|
||||
title: 'easy multiTrackEditor (bench)',
|
||||
type: 'easy multiTrackEditor',
|
||||
properties: { easyMediaTimelineHeight: 620 },
|
||||
widgets: [],
|
||||
widgets_values: [],
|
||||
inputs: [],
|
||||
outputs: [],
|
||||
size: [1280, 640],
|
||||
pos: [0, 0],
|
||||
flags: {},
|
||||
mode: 0,
|
||||
order: 0,
|
||||
setDirtyCanvas: () => {},
|
||||
setSize: () => {},
|
||||
onResize: () => {},
|
||||
onNodeCreated: () => {},
|
||||
addDOMWidget: () => ({
|
||||
name: 'track_data',
|
||||
type: 'react-widget',
|
||||
element: document.createElement('div'),
|
||||
inputEl: null,
|
||||
width: undefined,
|
||||
getValue: () => '',
|
||||
setValue: () => {},
|
||||
serializeValue: () => '',
|
||||
options: {},
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
export function createBenchWidget(): Record<string, unknown> {
|
||||
return {
|
||||
name: 'track_data',
|
||||
width: undefined,
|
||||
inputEl: null,
|
||||
options: {},
|
||||
getValue: () => '',
|
||||
setValue: () => {},
|
||||
}
|
||||
}
|
||||
49
frontend/src/bench/main.tsx
Normal file
49
frontend/src/bench/main.tsx
Normal file
@ -0,0 +1,49 @@
|
||||
/**
|
||||
* Bench UI entry point.
|
||||
*
|
||||
* IMPORTANT: `./bench-runtime` must be evaluated first — it installs the
|
||||
* `globalThis.comfyAPI` shim that `@/lib/comfy-api` reads at module scope.
|
||||
*/
|
||||
import './bench-runtime'
|
||||
|
||||
declare const __COMFY_EASY_MEDIA_GLOBAL_CSS__: string
|
||||
|
||||
import { createRoot } from 'react-dom/client'
|
||||
import { BenchApp } from './BenchApp'
|
||||
|
||||
function injectGlobalCss(): void {
|
||||
if (document.getElementById('mmnode-bench-globals')) return
|
||||
const style = document.createElement('style')
|
||||
style.id = 'mmnode-bench-globals'
|
||||
style.textContent = __COMFY_EASY_MEDIA_GLOBAL_CSS__
|
||||
document.head.appendChild(style)
|
||||
}
|
||||
|
||||
function main(): void {
|
||||
injectGlobalCss()
|
||||
const host = document.getElementById('root')
|
||||
if (!host) throw new Error('bench: missing #root host element')
|
||||
|
||||
// Surface fatal boot errors inside the page itself (sidecar windows have no
|
||||
// devtools console by default).
|
||||
const showFatal = (title: string, detail: string) => {
|
||||
host.innerHTML = ''
|
||||
const pre = document.createElement('pre')
|
||||
pre.style.cssText = 'color:#f87171;padding:16px;white-space:pre-wrap;font:12px/1.5 monospace'
|
||||
pre.textContent = `${title}\n\n${detail}`
|
||||
host.appendChild(pre)
|
||||
}
|
||||
window.addEventListener('error', (e) => {
|
||||
if (!host.childElementCount || host.querySelector('pre')) {
|
||||
showFatal('Uncaught error', `${e.message}\n${e.error?.stack ?? ''}`)
|
||||
}
|
||||
})
|
||||
window.addEventListener('unhandledrejection', (e) => {
|
||||
showFatal('Unhandled rejection', `${e.reason?.stack || e.reason}`)
|
||||
})
|
||||
|
||||
const root = createRoot(host)
|
||||
root.render(<BenchApp />)
|
||||
}
|
||||
|
||||
main()
|
||||
@ -19,6 +19,7 @@ SUBDIR_HINTS = {
|
||||
"vae": "models/vae",
|
||||
"audio_vae": "models/audio_vae",
|
||||
"clip": "models/clip",
|
||||
"loras": "models/loras",
|
||||
}
|
||||
|
||||
|
||||
@ -52,11 +53,18 @@ def _resolve(subdir, name):
|
||||
return path
|
||||
|
||||
|
||||
# (path, sd, metadata) cache for the last parsed LoRA, mirroring the native
|
||||
# LoraLoader. Kept at module level on purpose: newer ComfyUI builds lock node
|
||||
# classes after registration, so writing class attributes on the node class
|
||||
# itself raises "Cannot modify class attribute ... on locked class".
|
||||
_loaded_lora = None
|
||||
|
||||
|
||||
def _auto_pick(subdir):
|
||||
"""Return the model file to use for a hidden picker.
|
||||
|
||||
End users only see the diffusion model widget; every auxiliary model
|
||||
(video VAE, audio VAE, CLIP) is auto-selected from the pack's
|
||||
(video VAE, audio VAE, CLIP, LoRA) is auto-selected from the pack's
|
||||
models/<subdir> folder - the first file in sorted order, which for the
|
||||
published pack is exactly the bundled file. A missing file keeps the
|
||||
loud, actionable error from _resolve.
|
||||
@ -77,6 +85,14 @@ def _auto_pick(subdir):
|
||||
return selected
|
||||
|
||||
|
||||
def _auto_lora():
|
||||
"""Bundled LoRA file, or ``None`` when models/loras is empty."""
|
||||
files = _list_files("loras")
|
||||
if files[0].startswith("<no files found"):
|
||||
return None
|
||||
return files[0]
|
||||
|
||||
|
||||
def _h3_clip_type():
|
||||
"""MiniMax H3 CLIP recipe. The Moxie model set is H3-only, so the text
|
||||
encoder type is intentionally NOT a user-facing choice."""
|
||||
@ -94,12 +110,12 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
"""All-in-one loader that looks like it loads a single model.
|
||||
|
||||
The only visible picker is the diffusion model; the attention backend is
|
||||
applied to it, and the video VAE, the audio VAE and the CLIP are
|
||||
auto-selected from the node pack's models/ folder.
|
||||
applied to it, the bundled LoRA (if any), the video VAE, the audio VAE
|
||||
and the CLIP are auto-selected from the node pack's models/ folder.
|
||||
|
||||
The model set is MiniMax H3-only: the CLIP recipe is locked to
|
||||
CLIPType.MINIMAX and outputs are exactly (model, clip, video_vae, audio_vae)
|
||||
with the attention backend already applied to the model."""
|
||||
with the attention backend and LoRA already applied to the model."""
|
||||
|
||||
@classmethod
|
||||
def define_schema(cls) -> io.Schema:
|
||||
@ -107,7 +123,7 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
node_id="MoxieMultimediaLoader",
|
||||
display_name="Moxie Multimedia Loader",
|
||||
category="loaders/moxie",
|
||||
description="Loads the Moxie diffusion model (with the attention backend "
|
||||
description="Loads the Moxie diffusion model (with the attention backend and the bundled LoRA "
|
||||
"already applied) plus the video VAE, audio VAE and text encoder that ship with "
|
||||
"this custom node - one model picker, everything else is handled automatically. "
|
||||
"Outputs model, clip, video_vae and audio_vae.",
|
||||
@ -132,8 +148,8 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
),
|
||||
],
|
||||
outputs=[
|
||||
io.Model.Output(display_name="model", tooltip="Diffusion model with attention backend applied."),
|
||||
io.Clip.Output(display_name="clip", tooltip="CLIP text encoder."),
|
||||
io.Model.Output(display_name="model", tooltip="Diffusion model with attention backend and LoRA applied."),
|
||||
io.Clip.Output(display_name="clip", tooltip="CLIP text encoder (with LoRA applied if present)."),
|
||||
io.Vae.Output(display_name="video_vae", tooltip="Loaded video/image VAE."),
|
||||
io.Vae.Output(display_name="audio_vae", tooltip="Loaded audio VAE."),
|
||||
],
|
||||
@ -173,6 +189,36 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
m.set_model_optimized_attention(attention_function)
|
||||
return m
|
||||
|
||||
@classmethod
|
||||
def _apply_lora(cls, model, clip, lora_name, strength_model, strength_clip):
|
||||
global _loaded_lora
|
||||
if (
|
||||
lora_name is None
|
||||
or lora_name == "none"
|
||||
or lora_name.startswith("<no files found")
|
||||
or (strength_model == 0 and strength_clip == 0)
|
||||
):
|
||||
return model, clip
|
||||
|
||||
lora_path = _resolve("loras", lora_name)
|
||||
lora = None
|
||||
lora_metadata = None
|
||||
if _loaded_lora is not None:
|
||||
if _loaded_lora[0] == lora_path:
|
||||
lora = _loaded_lora[1]
|
||||
lora_metadata = _loaded_lora[2] if len(_loaded_lora) > 2 else None
|
||||
else:
|
||||
_loaded_lora = None
|
||||
|
||||
if lora is None:
|
||||
lora, lora_metadata = comfy.utils.load_torch_file(lora_path, safe_load=True, return_metadata=True)
|
||||
_loaded_lora = (lora_path, lora, lora_metadata)
|
||||
|
||||
model_lora, clip_lora = comfy.sd.load_lora_for_models(
|
||||
model, clip, lora, strength_model, strength_clip, lora_metadata=lora_metadata
|
||||
)
|
||||
return model_lora, clip_lora
|
||||
|
||||
@classmethod
|
||||
def _load_vae(cls, vae_name, subdir="vae"):
|
||||
path = _resolve(subdir, vae_name)
|
||||
@ -210,13 +256,15 @@ class MoxieMultimediaLoader(io.ComfyNode):
|
||||
weight_dtype,
|
||||
attention_backend,
|
||||
) -> io.NodeOutput:
|
||||
# Chain: Load Diffusion Model -> Model Attention Backend
|
||||
# Chain: Load Diffusion Model -> Model Attention Backend -> Lora Loader (needs both model and clip)
|
||||
model = cls._load_diffusion_model(diffusion_model, weight_dtype)
|
||||
model = cls._apply_attention_backend(model, attention_backend)
|
||||
|
||||
# Hidden pickers: auxiliary models are auto-selected from the pack's
|
||||
# models/ folder so end users only ever see the diffusion model.
|
||||
clip_obj = cls._load_clip(_auto_pick("clip"), "default")
|
||||
lora_name = _auto_lora()
|
||||
model, clip_obj = cls._apply_lora(model, clip_obj, lora_name, 1.0, 1.0)
|
||||
|
||||
video_vae_obj = cls._load_vae(_auto_pick("vae"), "vae")
|
||||
audio_vae_obj = cls._load_vae(_auto_pick("audio_vae"), "audio_vae")
|
||||
|
||||
@ -1006,7 +1006,7 @@ class ModelPreviewOverrideKJ(io.ComfyNode):
|
||||
inputs=[
|
||||
io.Model.Input(
|
||||
"model",
|
||||
tooltip="Diffusion model (with attention backend already applied).",
|
||||
tooltip="Diffusion model (with attention backend and LoRA already applied).",
|
||||
),
|
||||
io.Int.Input(
|
||||
"max_resolution",
|
||||
|
||||
@ -5,8 +5,6 @@ from comfy_api.latest import io
|
||||
|
||||
from .nodes.project import (
|
||||
EasyMultiTrackProject,
|
||||
H3_CONTEXT_LENGTH_OPTIONS,
|
||||
H3_CONTEXT_SOURCE_FRAMES,
|
||||
TYPE_TRACKS_INFO,
|
||||
_first_input,
|
||||
)
|
||||
@ -54,7 +52,7 @@ class MoxieProject(EasyMultiTrackProject):
|
||||
not_idempotent=True,
|
||||
inputs=[
|
||||
TYPE_TRACKS_INFO.Input("tracks_info"),
|
||||
io.Model.Input("model", tooltip="Diffusion model from Moxie Multimedia Loader (attention backend already applied)."),
|
||||
io.Model.Input("model", tooltip="Diffusion model from Moxie Multimedia Loader (attention backend and LoRA already applied)."),
|
||||
io.Clip.Input("clip", tooltip="CLIP text encoder from Moxie Multimedia Loader."),
|
||||
io.Vae.Input("video_vae", tooltip="Video VAE from Moxie Multimedia Loader."),
|
||||
io.Vae.Input("audio_vae", tooltip="Audio VAE from Moxie Multimedia Loader (required to decode Moxie audio)."),
|
||||
@ -65,21 +63,8 @@ class MoxieProject(EasyMultiTrackProject):
|
||||
default=3,
|
||||
min=1,
|
||||
max=10000,
|
||||
tooltip="Sampling steps for the beta scheduler (3 for the baked-in 3-step distill; "
|
||||
"8 for busy motion, 6 low motion, 4 runs/no dialogue).",
|
||||
),
|
||||
io.Combo.Input(
|
||||
"context_frames",
|
||||
options=[str(option) for option in H3_CONTEXT_LENGTH_OPTIONS],
|
||||
default=str(H3_CONTEXT_SOURCE_FRAMES),
|
||||
tooltip=(
|
||||
"Video frames copied from the previous segment as "
|
||||
"continuation context for segments whose continuity "
|
||||
"mode is Context or Context Swap. Longer values hand "
|
||||
"off motion more smoothly but generate that many extra "
|
||||
"frames per segment. Values are H3 temporal-grid "
|
||||
"lengths shared with the Moxie context sampler nodes."
|
||||
),
|
||||
tooltip="Sampling steps for the beta scheduler (3 for the 3-step turbo LoRA; "
|
||||
"8 for busy motion with regular LoRAs, 6 low motion, 4 runs/no dialogue).",
|
||||
),
|
||||
io.Int.Input(
|
||||
"seed",
|
||||
|
||||
@ -57,25 +57,6 @@ from ..utils.video import passthrough_video_media
|
||||
|
||||
CATEGORY_MINIMAX = "EasyUse/MiniMax"
|
||||
CANVAS_MULTIPLE = 32
|
||||
|
||||
|
||||
def _align_canvas_dimension(value: Any) -> int:
|
||||
"""Floor a canvas side onto the H3 32px grid (CANVAS_MULTIPLE).
|
||||
|
||||
The video VAE latent grid is pixels // 16 and the diffusion model
|
||||
patchifies that latent with a 2x2 patch, so every canvas side must be a
|
||||
multiple of 32. Anything else produces an odd latent dimension and crashes
|
||||
comfy's patchify_video reshape ("shape '[1, 24, ...]' is invalid for input
|
||||
of size N"). Floors instead of rounds so alignment never upsamples past
|
||||
the requested size.
|
||||
"""
|
||||
try:
|
||||
number = int(value)
|
||||
except (TypeError, ValueError):
|
||||
return CANVAS_MULTIPLE
|
||||
return max(CANVAS_MULTIPLE, number // CANVAS_MULTIPLE * CANVAS_MULTIPLE)
|
||||
|
||||
|
||||
BASE_SHORT_EDGE = 768
|
||||
MAX_PIXELS = 768 * 1344
|
||||
REF_IMAGE_SHORT_EDGE = 2048
|
||||
@ -508,8 +489,6 @@ class MiniMaxH3ImageToVideoFallback(io.ComfyNode):
|
||||
first_frame: torch.Tensor | None = None,
|
||||
last_frame: torch.Tensor | None = None,
|
||||
) -> io.NodeOutput:
|
||||
width = _align_canvas_dimension(width)
|
||||
height = _align_canvas_dimension(height)
|
||||
latent, frame_count = _empty_av_latent(width, height, length)
|
||||
token_images: list[torch.Tensor] = []
|
||||
keyframes: list[dict[str, Any]] = []
|
||||
@ -638,8 +617,6 @@ class MiniMaxH3ReferenceToVideoFallback(io.ComfyNode):
|
||||
ref_video_audios: dict[str, dict[str, Any] | None] | None = None,
|
||||
ref_audios: dict[str, dict[str, Any]] | None = None,
|
||||
) -> io.NodeOutput:
|
||||
width = _align_canvas_dimension(width)
|
||||
height = _align_canvas_dimension(height)
|
||||
latent, frame_count = _empty_av_latent(width, height, length)
|
||||
reference_items: list[dict[str, Any]] = []
|
||||
reference_blocks: list[dict[str, Any]] = []
|
||||
@ -685,20 +662,6 @@ class MiniMaxH3ReferenceToVideoFallback(io.ComfyNode):
|
||||
continue
|
||||
soundtrack = video_audios.get("ref_video_audio_" + name.rsplit("_", 1)[-1])
|
||||
frames = _prepare_reference_video_frames(frames, frame_count)
|
||||
# Grid-align the reference video canvas: comfy core's
|
||||
# patchify_video floors each latent side into 2x2 patches, so an
|
||||
# off-grid frame size (API callers can inject arbitrary media)
|
||||
# would produce an odd latent dimension and crash sampling with
|
||||
# "shape '[1, 24, ...]' is invalid for input of size N". ComfyUI
|
||||
# builds with native H3 nodes do this alignment inside core's
|
||||
# MiniMaxH3ReferenceToVideo; this fallback copy must match.
|
||||
ref_height, ref_width = int(frames.shape[1]), int(frames.shape[2])
|
||||
aligned_ref_width = _align_canvas_dimension(ref_width)
|
||||
aligned_ref_height = _align_canvas_dimension(ref_height)
|
||||
if (aligned_ref_width, aligned_ref_height) != (ref_width, ref_height):
|
||||
frames = _resize(
|
||||
frames, aligned_ref_width, aligned_ref_height, "disabled"
|
||||
)
|
||||
aligned_count = frames.shape[0]
|
||||
canvas_height, canvas_width = frames.shape[1], frames.shape[2]
|
||||
if aligned_count < 5:
|
||||
@ -2779,8 +2742,8 @@ class EasyMiniMaxH3ToVideo(io.ComfyNode):
|
||||
selected_vae = _first_input(vae)
|
||||
selected_audio_vae = _first_input(audio_vae)
|
||||
prompt_text = str(_first_input(prompt, ""))
|
||||
target_width = _align_canvas_dimension(_first_input(width, 1344))
|
||||
target_height = _align_canvas_dimension(_first_input(height, 768))
|
||||
target_width = int(_first_input(width, 1344))
|
||||
target_height = int(_first_input(height, 768))
|
||||
target_length = int(_first_input(length, 124))
|
||||
locked_timing_frames = int(_first_input(locked_video_timing_frames, 0))
|
||||
expanded_images = expand_image_inputs(images)
|
||||
|
||||
@ -46,7 +46,6 @@ from ..utils.multitrack import (
|
||||
MULTITRACK_RUNTIME_CACHE_KEY,
|
||||
multitrack_runtime_cache,
|
||||
)
|
||||
from .minimax import CANVAS_MULTIPLE, _align_canvas_dimension
|
||||
|
||||
|
||||
TYPE_FAST_MODEL_LOADER = io.Custom(io_type="FAST_MODEL_LOADER")
|
||||
@ -55,12 +54,6 @@ TYPE_PROJECT_DATA = io.Custom(io_type="PROJECT_DATA")
|
||||
TYPE_H3_PROJECT_STATIC_DATA = io.Custom(io_type="H3_PROJECT_STATIC_DATA")
|
||||
H3_CONTEXT_CONTINUITY_MODES = {"context", "context_swap"}
|
||||
H3_CONTEXT_SOURCE_FRAMES = 22
|
||||
# Context lengths supported by the H3 continuation sampler nodes; every
|
||||
# value lands on the H3 temporal latent grid (17k + 5 frames).
|
||||
H3_CONTEXT_LENGTH_OPTIONS = (5, 22, 39, 56)
|
||||
# Extra frames generated past the context prefix so the decoded clip ends
|
||||
# on a phase-aligned H3 grid boundary before the tail is trimmed.
|
||||
H3_CONTEXT_GRID_TAIL_FRAMES = 12
|
||||
|
||||
|
||||
def _first_input(value: Any, default: Any = None) -> Any:
|
||||
@ -69,18 +62,6 @@ def _first_input(value: Any, default: Any = None) -> Any:
|
||||
return value if value is not None else default
|
||||
|
||||
|
||||
def _h3_context_frame_count(value: Any) -> int:
|
||||
"""Resolve the context_frames widget to a supported H3 context length."""
|
||||
frames = int(_first_input(value, str(H3_CONTEXT_SOURCE_FRAMES)))
|
||||
if frames not in H3_CONTEXT_LENGTH_OPTIONS:
|
||||
options = ", ".join(str(option) for option in H3_CONTEXT_LENGTH_OPTIONS)
|
||||
raise ValueError(
|
||||
"context_frames must be one of the supported H3 context lengths: "
|
||||
f"{options} (got {frames})"
|
||||
)
|
||||
return frames
|
||||
|
||||
|
||||
def _raw_project_input(value: Any) -> Any:
|
||||
"""Unwrap list inputs without splitting a raw graph link."""
|
||||
if is_link(value):
|
||||
@ -949,19 +930,6 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
"mode preserves existing video and latent files."
|
||||
),
|
||||
),
|
||||
io.Combo.Input(
|
||||
"context_frames",
|
||||
options=[str(option) for option in H3_CONTEXT_LENGTH_OPTIONS],
|
||||
default=str(H3_CONTEXT_SOURCE_FRAMES),
|
||||
tooltip=(
|
||||
"Video frames copied from the previous segment as "
|
||||
"continuation context for segments whose continuity "
|
||||
"mode is Context or Context Swap. Longer values hand "
|
||||
"off motion more smoothly but generate that many extra "
|
||||
"frames per segment. Values are H3 temporal-grid "
|
||||
"lengths shared with the Moxie context sampler nodes."
|
||||
),
|
||||
),
|
||||
io.Int.Input(
|
||||
"seed",
|
||||
default=42,
|
||||
@ -1111,19 +1079,6 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
if not uses_linked_prepare:
|
||||
_require_minimax_h3_model(model)
|
||||
info = parse_tracks_info(kwargs.get("tracks_info"))
|
||||
# External app clients (the Moxie app drives this node through the
|
||||
# ComfyUI API) can inject project dimensions that are not on the H3
|
||||
# 32px canvas grid; the MultiTrack Editor UI only offers grid sizes.
|
||||
# Every latent side is pixels // 16 and the diffusion model patchifies
|
||||
# that latent 2x2, so any off-grid side eventually reaches comfy's
|
||||
# patchify_video as an odd latent dimension and crashes first sampling
|
||||
# with "shape '[1, 24, ...]' is invalid for input of size N". Floor
|
||||
# both sides onto the grid once, here, so media loading, reference
|
||||
# encoding, both sampling passes and the exported project all agree
|
||||
# on the same aligned canvas. Editor-selected sizes are already
|
||||
# multiples of 32, so this is a no-op for UI workflows.
|
||||
info["width"] = _align_canvas_dimension(info["width"])
|
||||
info["height"] = _align_canvas_dimension(info["height"])
|
||||
hidden_inputs = getattr(cls, "hidden", None)
|
||||
validate_h3_project_outputs(
|
||||
info,
|
||||
@ -1135,9 +1090,6 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
sampling_mode, sampling_config = _h3_sampling_mode_config(
|
||||
kwargs.get("sampling_mode")
|
||||
)
|
||||
context_source_frames = _h3_context_frame_count(
|
||||
kwargs.get("context_frames")
|
||||
)
|
||||
is_passthrough = sampling_mode == "passthrough"
|
||||
is_selflift = sampling_mode == "selflift"
|
||||
has_second_pass = sampling_mode == "dual"
|
||||
@ -1229,23 +1181,8 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
# keeps the editor size exactly as configured.
|
||||
first_pass_width, first_pass_height = editor_width, editor_height
|
||||
if sampling_mode == "single" and not audio_only:
|
||||
# The half-resolution canvas must stay on the H3 32px grid
|
||||
# (CANVAS_MULTIPLE): the video VAE downsamples by 16 and the
|
||||
# diffusion model patchifies the latent with a 2x2 patch, so a
|
||||
# first-pass side that is not a multiple of 32 yields an odd
|
||||
# latent dimension and crashes comfy's patchify_video with
|
||||
# "shape '[1, 24, 1, 1, ...]' is invalid for input of size N"
|
||||
# (e.g. a 544x960 editor project halved to 272x480 produces a
|
||||
# 17x30 latent). Floor to the grid instead of rounding so the
|
||||
# first pass never upsamples past half the editor resolution.
|
||||
half_width = max(
|
||||
32,
|
||||
(editor_width // 2) // CANVAS_MULTIPLE * CANVAS_MULTIPLE,
|
||||
)
|
||||
half_height = max(
|
||||
32,
|
||||
(editor_height // 2) // CANVAS_MULTIPLE * CANVAS_MULTIPLE,
|
||||
)
|
||||
half_width = max(32, editor_width // 2)
|
||||
half_height = max(32, editor_height // 2)
|
||||
if (half_width, half_height) != (32, 32):
|
||||
first_pass_width, first_pass_height = half_width, half_height
|
||||
target_width, target_height = h3_second_pass_dimensions(
|
||||
@ -1273,19 +1210,8 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
segment_count,
|
||||
)
|
||||
if not selected_entries:
|
||||
if not all_entries:
|
||||
raise ValueError(
|
||||
"No H3 task segments are available: the MultiTrack timeline "
|
||||
"does not contain any task-track segments. Add at least one "
|
||||
"task segment in the MultiTrack Editor and run the project "
|
||||
"again."
|
||||
)
|
||||
raise ValueError(
|
||||
"No H3 task segments are available from segment_start_number="
|
||||
f"{segment_start_number} (segment_count={segment_count}): the "
|
||||
f"timeline only contains {len(all_entries)} task segment(s), "
|
||||
"numbered from 1. Lower segment_start_number or add more task "
|
||||
"segments in the MultiTrack Editor."
|
||||
"No H3 task segments are available from segment_start_number."
|
||||
)
|
||||
|
||||
first_selected_index, first_selected_entry = selected_entries[0]
|
||||
@ -1576,13 +1502,12 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
context_latent = _h3_encode_context_media(
|
||||
graph, output_images, output_audio, vae, audio_vae,
|
||||
f"passthrough_context_{task_index}",
|
||||
context_frames=context_source_frames,
|
||||
)
|
||||
runtime_context_latent = graph.node(
|
||||
"easy h3MotionContextLatentTrim",
|
||||
id=f"trim_hires_context_latent_{task_index}",
|
||||
latent=context_latent,
|
||||
context_length=str(context_source_frames),
|
||||
context_length=str(H3_CONTEXT_SOURCE_FRAMES),
|
||||
).out(0)
|
||||
saved_video = graph.node(
|
||||
"easy saveVideo",
|
||||
@ -1643,9 +1568,8 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
uses_context
|
||||
and (previous_hires_context_latent is not None or task_index > 0)
|
||||
)
|
||||
context_generation_frames = (
|
||||
context_source_frames + H3_CONTEXT_GRID_TAIL_FRAMES
|
||||
)
|
||||
context_source_frames = H3_CONTEXT_SOURCE_FRAMES
|
||||
context_generation_frames = 34
|
||||
if will_have_context_continuity:
|
||||
task_length = graph.node(
|
||||
"ComfyMathExpression",
|
||||
@ -2002,7 +1926,7 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
id=f"hires_continuity_{task_index}",
|
||||
current_hires_latent=upscaled_latent,
|
||||
previous_hires_latent=previous_hires_context_latent,
|
||||
context_length=str(context_source_frames),
|
||||
context_length=str(H3_CONTEXT_SOURCE_FRAMES),
|
||||
video_transition_steps=4,
|
||||
video_anchor_only=not audio_only,
|
||||
)
|
||||
@ -2192,7 +2116,6 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
vae,
|
||||
audio_vae,
|
||||
f"hires_context_{task_index}",
|
||||
context_frames=context_source_frames,
|
||||
)
|
||||
hires_context_reencoded = True
|
||||
if (has_second_pass and run_second_pass) or (
|
||||
@ -2236,7 +2159,6 @@ class EasyMultiTrackProject(io.ComfyNode):
|
||||
vae,
|
||||
audio_vae,
|
||||
f"low_context_{task_index}",
|
||||
context_frames=context_source_frames,
|
||||
)
|
||||
low_context_reencoded = True
|
||||
else:
|
||||
|
||||
@ -1,59 +0,0 @@
|
||||
# Moxie Multimedia Suite - Python dependencies
|
||||
#
|
||||
# ComfyUI-Manager installs this file automatically.
|
||||
# If you cloned the pack manually, install it yourself - see README.md
|
||||
# ("Installing dependencies") for the ComfyUI Portable command.
|
||||
#
|
||||
# This pack targets NVIDIA RTX 3060+ GPUs only, so every feature
|
||||
# dependency ships as mandatory - speech, subtitle recognition, and the
|
||||
# RTX Video Super Resolution upscale are all installed up front. The
|
||||
# list is ordered light -> heavy: if a large package ever fails to
|
||||
# install, everything above it is already in place and re-running the
|
||||
# command resumes.
|
||||
#
|
||||
# Notes:
|
||||
# - Core stack (torch, numpy, Pillow, PyYAML, aiohttp) comes with
|
||||
# ComfyUI itself.
|
||||
# - The two pip options below route nvidia-vfx to NVIDIA's own package
|
||||
# index: PyPI only hosts a stub sdist for nvidia-vfx whose build
|
||||
# backend (wheel_stub) fails on some setups, while pypi.nvidia.com
|
||||
# hosts the real prebuilt wheels (Windows + Linux, python 3.10-3.13).
|
||||
# --only-binary=nvidia-vfx guarantees the stub sdist is never built.
|
||||
# - qwen-asr pins transformers==4.57.6 / accelerate==1.12.0, so pip may
|
||||
# upgrade those two packages inside your ComfyUI environment.
|
||||
# - voxcpm needs torch>=2.5.0 and torchcodec (wheels exist for Windows
|
||||
# and Linux, python 3.10-3.14); keep ComfyUI reasonably current.
|
||||
# - ffmpeg-python drives the FFmpeg binary - keep FFmpeg in PATH.
|
||||
|
||||
--extra-index-url https://pypi.nvidia.com/
|
||||
--only-binary=nvidia-vfx
|
||||
|
||||
# Audio: reading/writing merged and generated audio
|
||||
soundfile
|
||||
|
||||
# Subtitles: CJK-aware smart line splitting
|
||||
jieba
|
||||
|
||||
# Model downloads from the media browser / helper routes
|
||||
huggingface_hub
|
||||
|
||||
# Smart split (shot detection) - ffmpeg CLI wrapper
|
||||
ffmpeg-python
|
||||
|
||||
# Smart split (shot detection helpers)
|
||||
opencv-python
|
||||
|
||||
# Smart split (dataset loaders)
|
||||
imageio
|
||||
|
||||
# RTX Video Super Resolution upscale (NVIDIA RTX GPUs only)
|
||||
nvidia-vfx
|
||||
|
||||
# Subtitle recognition (local Whisper)
|
||||
openai-whisper
|
||||
|
||||
# Subtitle recognition (Qwen ASR) - pins transformers==4.57.6
|
||||
qwen-asr
|
||||
|
||||
# Speech / voiceover generation (VoxCPM)
|
||||
voxcpm
|
||||
160
scripts/test_bench_pipeline.py
Normal file
160
scripts/test_bench_pipeline.py
Normal file
@ -0,0 +1,160 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Standalone bench verification: API + WebSocket + mock pipeline end-to-end.
|
||||
|
||||
No ComfyUI, no torch. Exercises the FastAPI app exactly like a sidecar host
|
||||
would: health/config, workflow & plan endpoints, a full render over the mock
|
||||
backend (ffmpeg), WebSocket progress events, and output serving.
|
||||
|
||||
Run: python3 scripts/test_bench_pipeline.py
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parent.parent
|
||||
BENCH = ROOT / "bench"
|
||||
sys.path.insert(0, str(BENCH))
|
||||
|
||||
FAILED = []
|
||||
|
||||
|
||||
def check(name: str, cond: bool, detail: str = "") -> None:
|
||||
print(f"[{'PASS' if cond else 'FAIL'}] {name}" + (f" - {detail}" if detail and not cond else ""))
|
||||
if not cond:
|
||||
FAILED.append(name)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
# py_compile every bench module first
|
||||
import py_compile
|
||||
|
||||
py_files = sorted(BENCH.rglob("*.py"))
|
||||
for f in py_files:
|
||||
py_compile.compile(str(f), doraise=True)
|
||||
check("py_compile all bench modules", True, f"({len(py_files)} files)")
|
||||
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
os.environ.setdefault("MOXIE_BENCH_WORK", str(BENCH / "work"))
|
||||
from app.config import BenchConfig
|
||||
from app.server import create_app
|
||||
|
||||
cfg = BenchConfig(backend="mock")
|
||||
cfg.repo_root = ROOT
|
||||
app = create_app(cfg)
|
||||
|
||||
with TestClient(app) as client: # context manager runs lifespan
|
||||
r = client.get("/api/health")
|
||||
check("GET /api/health", r.status_code == 200 and r.json()["ok"] is True)
|
||||
check("health reports mock backend", r.json()["backend"] == "mock")
|
||||
|
||||
r = client.get("/api/config")
|
||||
check("GET /api/config", r.status_code == 200 and "backends" in r.json())
|
||||
|
||||
r = client.get("/api/workflow")
|
||||
body = r.json()
|
||||
check("GET /api/workflow summary", r.status_code == 200
|
||||
and body["summary"]["editor_resolution"] == {"width": 1920, "height": 1080})
|
||||
track_data = json.loads(body["track_data"])
|
||||
check("workflow carries editor track_data", track_data.get("total_length") == 240)
|
||||
|
||||
r = client.get("/api/plan")
|
||||
plan = r.json()
|
||||
check("GET /api/plan", r.status_code == 200
|
||||
and [s["prompt"] for s in plan["segments"]] == ["scene1", "scene2"])
|
||||
check("plan duration", plan["duration_seconds"] == 10.0, str(plan["duration_seconds"]))
|
||||
|
||||
# --- WebSocket + render e2e (small dims to keep it fast) ---
|
||||
events = []
|
||||
with client.websocket_connect("/api/ws") as ws:
|
||||
# drain the initial connection by starting a render in parallel
|
||||
render_payload = {
|
||||
"track_data": body["track_data"],
|
||||
"overrides": {
|
||||
"width": 256, "height": 144, "frame_rate": 8,
|
||||
"steps": 4, "seed": 42, "project_name": "MM-bench-e2e",
|
||||
},
|
||||
}
|
||||
r = client.post("/api/render", json=render_payload)
|
||||
check("POST /api/render accepted", r.status_code == 200, r.text[:200])
|
||||
job_id = r.json()["job_id"]
|
||||
|
||||
deadline = time.time() + 120
|
||||
got_success = False
|
||||
while time.time() < deadline and not got_success:
|
||||
msg = ws.receive_json()
|
||||
events.append(msg)
|
||||
if msg.get("job_id") != job_id:
|
||||
continue
|
||||
if msg.get("type") == "job_progress":
|
||||
check("progress within pipeline stages", True) if len(events) == 2 else None
|
||||
if msg.get("type") == "execution_success":
|
||||
got_success = True
|
||||
if msg.get("type") == "execution_error":
|
||||
break
|
||||
|
||||
check("render completed over WebSocket", got_success,
|
||||
f"events={ [e.get('type') for e in events][-6:] }")
|
||||
|
||||
r = client.get(f"/api/jobs/{job_id}")
|
||||
job = r.json()
|
||||
check("job result stored", job["status"] == "done" and job["result"] is not None)
|
||||
outputs = (job["result"] or {}).get("outputs", {})
|
||||
check("both branches produced", set(outputs) == {"native", "upscaled"}, str(list(outputs)))
|
||||
for branch, out in outputs.items():
|
||||
path = Path(out["path"])
|
||||
check(f"{branch} file exists", path.is_file() and path.stat().st_size > 1000)
|
||||
|
||||
expected = {"native": (256, 144), "upscaled": (1920, 1080)}[branch]
|
||||
check(f"{branch} served over /api/view", client.get(out["url"]).status_code == 200)
|
||||
|
||||
# output dims via ffprobe
|
||||
import subprocess
|
||||
probe = subprocess.run(
|
||||
["ffprobe", "-v", "quiet", "-print_format", "json", "-show_streams",
|
||||
outputs["upscaled"]["path"]],
|
||||
capture_output=True, text=True)
|
||||
streams = json.loads(probe.stdout).get("streams") or []
|
||||
vs = next((s for s in streams if s.get("codec_type") == "video"), {})
|
||||
check("upscaled dims match RTX target",
|
||||
(vs.get("width"), vs.get("height")) == (1920, 1080),
|
||||
f"{vs.get('width')}x{vs.get('height')}")
|
||||
vs_n = json.loads(subprocess.run(
|
||||
["ffprobe", "-v", "quiet", "-print_format", "json", "-show_streams",
|
||||
outputs["native"]["path"]], capture_output=True, text=True).stdout
|
||||
).get("streams", [{}])
|
||||
vsn = next((s for s in vs_n if s.get("codec_type") == "video"), {})
|
||||
check("native dims match editor canvas",
|
||||
(vsn.get("width"), vsn.get("height")) == (256, 144),
|
||||
f"{vsn.get('width')}x{vsn.get('height')}")
|
||||
|
||||
# progress stage ordering mirrors the pipeline
|
||||
stages_seen = [e.get("stage") for e in events if e.get("type") == "job_progress"]
|
||||
check("progress touches sampling stage", "MoxieProject" in stages_seen)
|
||||
check("progress touches upscale stage", "RTXVideoSuperResolution" in stages_seen)
|
||||
|
||||
# /api/easy-media mirrors answer
|
||||
r = client.get("/api/easy-media/media/list")
|
||||
check("easy-media media/list mirror", r.status_code == 200 and "items" in r.json())
|
||||
r = client.get("/api/easy-media/prompt/system-prompts")
|
||||
check("easy-media system-prompts mirror", r.status_code == 200)
|
||||
|
||||
# static UI mount responds (built or explicit hint)
|
||||
r = client.get("/")
|
||||
check("root serves UI or build hint", r.status_code in (200, 404)
|
||||
or "not built" in r.text)
|
||||
|
||||
print()
|
||||
if FAILED:
|
||||
print(f"FAILED: {len(FAILED)} -> {FAILED}")
|
||||
return 1
|
||||
print("ALL CHECKS PASSED")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
@ -1,348 +0,0 @@
|
||||
"""Standalone verification for merge_video_track_with_ffmpeg failure_reason.
|
||||
|
||||
Runs without ComfyUI/torch: stubs the heavy imports, then exercises every
|
||||
``return None`` path of merge_video_track_with_ffmpeg and the
|
||||
compose_h3_project_video error propagation, asserting each produces a
|
||||
specific, human-readable reason.
|
||||
|
||||
Usage: python scripts/test_merge_failure_reason.py
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import types
|
||||
from pathlib import Path
|
||||
|
||||
PACKAGE_ROOT = Path(__file__).resolve().parent.parent
|
||||
|
||||
|
||||
def _install_stubs() -> None:
|
||||
# Minimal torch stub: only attributes evaluated at IMPORT time are needed
|
||||
# (annotations like `torch.Tensor`, `torch.device`, `torch.dtype` in
|
||||
# image.py, and `torch.nn.functional` imported by image/audio/multitrack).
|
||||
# No torch function actually runs on the paths under test.
|
||||
if "torch" not in sys.modules:
|
||||
torch = types.ModuleType("torch")
|
||||
torch.__path__ = [] # mark as package so torch.nn.functional imports resolve
|
||||
nn = types.ModuleType("torch.nn")
|
||||
functional = types.ModuleType("torch.nn.functional")
|
||||
functional.pad = lambda *a, **k: None
|
||||
nn.functional = functional
|
||||
torch.nn = nn
|
||||
torch.Tensor = object
|
||||
torch.device = object
|
||||
torch.dtype = object
|
||||
torch.float32 = object()
|
||||
torch.float64 = object()
|
||||
sys.modules["torch"] = torch
|
||||
sys.modules["torch.nn"] = nn
|
||||
sys.modules["torch.nn.functional"] = functional
|
||||
|
||||
if "folder_paths" not in sys.modules:
|
||||
fp = types.ModuleType("folder_paths")
|
||||
temp_dir = PACKAGE_ROOT / ".tmp-temp"
|
||||
temp_dir.mkdir(exist_ok=True)
|
||||
fp.get_output_directory = lambda: str(PACKAGE_ROOT / ".tmp-output")
|
||||
fp.get_temp_directory = lambda: str(PACKAGE_ROOT / ".tmp-temp")
|
||||
fp.get_input_directory = lambda: ""
|
||||
fp.get_filename_list = lambda *a, **k: []
|
||||
fp.get_full_path = lambda *a, **k: None
|
||||
sys.modules["folder_paths"] = fp
|
||||
|
||||
if "server" not in sys.modules:
|
||||
sys.modules["server"] = types.ModuleType("server")
|
||||
|
||||
|
||||
_install_stubs()
|
||||
sys.path.insert(0, str(PACKAGE_ROOT))
|
||||
|
||||
# Import the REAL utils package: with the stubs above, the eager `from .image
|
||||
# import *` in utils/__init__.py resolves, so multitrack's `from . import
|
||||
# FFMPEG_RESIZE_METHODS, ...` also works and h3_project loads untouched.
|
||||
from utils.video import merge_video_track_with_ffmpeg # noqa: E402
|
||||
import utils.video as video_mod # noqa: E402
|
||||
import utils.h3_project as h3_project # noqa: E402
|
||||
|
||||
CHECKS: list[tuple[str, bool, str]] = []
|
||||
|
||||
|
||||
def check(name: str, condition: bool, detail: str = "") -> None:
|
||||
CHECKS.append((name, bool(condition), detail))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1. FFmpeg not found: point PATH lookup at an empty search space.
|
||||
# ---------------------------------------------------------------------------
|
||||
import shutil # noqa: E402
|
||||
|
||||
_real_which = shutil.which
|
||||
shutil.which = lambda name, **k: None
|
||||
try:
|
||||
reasons: list[str] = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": "whatever.mp4", "start_frame": 0, "end_frame": 10}],
|
||||
10,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("ffmpeg-missing returns None", out is None)
|
||||
check(
|
||||
"ffmpeg-missing reason mentions FFmpeg executable",
|
||||
len(reasons) == 1 and "FFmpeg executable not found" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
finally:
|
||||
shutil.which = _real_which
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 2. Without failure_reason the function still returns None silently and no
|
||||
# caller-visible behavior changes (backwards compatibility).
|
||||
# ---------------------------------------------------------------------------
|
||||
shutil.which = lambda name, **k: None
|
||||
try:
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": "whatever.mp4", "start_frame": 0, "end_frame": 10}],
|
||||
10,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
)
|
||||
check("no-collector path still returns None", out is None)
|
||||
finally:
|
||||
shutil.which = _real_which
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. ffmpeg "found": redirect get_ffmpeg_path via monkeypatched shutil.which to
|
||||
# a real executable so we reach the precondition checks.
|
||||
# ---------------------------------------------------------------------------
|
||||
import os # noqa: E402
|
||||
|
||||
fake_ffmpeg = Path(PACKAGE_ROOT / ".tmp-fake-ffmpeg")
|
||||
fake_ffmpeg.write_text("")
|
||||
os.chmod(fake_ffmpeg, 0o755)
|
||||
shutil.which = lambda name, **k: str(fake_ffmpeg) if name == "ffmpeg" else None
|
||||
try:
|
||||
# Empty timeline
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg([], 0, 24.0, 64, 64, failure_reason=reasons)
|
||||
check("empty timeline returns None", out is None)
|
||||
check(
|
||||
"empty timeline reason",
|
||||
len(reasons) == 1 and "timeline is empty" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
|
||||
# Invalid frame rate
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": "x.mp4", "start_frame": 0, "end_frame": 1}],
|
||||
1,
|
||||
0.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("invalid fps returns None", out is None)
|
||||
check("invalid fps reason", len(reasons) == 1 and "frame rate" in reasons[0], str(reasons))
|
||||
|
||||
# Odd dimensions
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": "x.mp4", "start_frame": 0, "end_frame": 1}],
|
||||
1,
|
||||
24.0,
|
||||
63,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("odd width returns None", out is None)
|
||||
check(
|
||||
"odd width reason names size",
|
||||
len(reasons) == 1 and "63x64" in reasons[0] and "odd" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
|
||||
# Missing segment sources
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[
|
||||
{"source": "/definitely/not/here/a.mp4", "start_frame": 0, "end_frame": 4},
|
||||
{"source": "/definitely/not/here/b.mp4", "start_frame": 4, "end_frame": 8},
|
||||
{"source": "/definitely/not/here/c.mp4", "start_frame": 8, "end_frame": 12},
|
||||
],
|
||||
12,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("missing sources returns None", out is None)
|
||||
check(
|
||||
"missing sources reason lists each file",
|
||||
len(reasons) == 1
|
||||
and "a.mp4" in reasons[0]
|
||||
and "b.mp4" in reasons[0]
|
||||
and "c.mp4" in reasons[0]
|
||||
and "missing" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
|
||||
# Missing locked audio
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[
|
||||
{
|
||||
"source": str(fake_ffmpeg),
|
||||
"audio_source": "/definitely/not/here/locked.wav",
|
||||
"start_frame": 0,
|
||||
"end_frame": 4,
|
||||
}
|
||||
],
|
||||
4,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("missing locked audio returns None", out is None)
|
||||
check(
|
||||
"locked audio reason names file",
|
||||
len(reasons) == 1 and "locked.wav" in reasons[0],
|
||||
str(reasons),
|
||||
)
|
||||
|
||||
# FFmpeg command failure: run the real (invalid) fake binary -> OSError or
|
||||
# non-zero return. The fake file is not executable code; on POSIX exec
|
||||
# raises OSError (ENOEXEC / format error), on the sandbox it surfaces as
|
||||
# either the OSError path or a non-zero returncode path. Both must record.
|
||||
reasons = []
|
||||
out = merge_video_track_with_ffmpeg(
|
||||
[{"source": str(fake_ffmpeg), "start_frame": 0, "end_frame": 1}],
|
||||
1,
|
||||
24.0,
|
||||
64,
|
||||
64,
|
||||
failure_reason=reasons,
|
||||
)
|
||||
check("command failure returns None", out is None)
|
||||
check(
|
||||
"command failure records a reason",
|
||||
len(reasons) == 1
|
||||
and ("failed to start" in reasons[0] or "exited with code" in reasons[0]),
|
||||
str(reasons),
|
||||
)
|
||||
finally:
|
||||
shutil.which = _real_which
|
||||
fake_ffmpeg.unlink(missing_ok=True)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3b. get_ffmpeg_path expanded Windows fallbacks (portable folder, winget/
|
||||
# scoop/choco shims, imageio-ffmpeg bundled binary). On this POSIX sandbox
|
||||
# os.name must be forced to "nt" so the fallback branches run.
|
||||
# ---------------------------------------------------------------------------
|
||||
import utils.video as _video_mod # noqa: E402
|
||||
|
||||
_real_os_name = os.name
|
||||
_real_base = getattr(_video_mod.folder_paths, "base_path", "")
|
||||
shutil.which = lambda name, **k: None
|
||||
os.name = "nt"
|
||||
try:
|
||||
# Portable layout: <base>/ffmpeg/bin/ffmpeg.exe found via the portable
|
||||
# root (dirname of folder_paths.base_path).
|
||||
fake_base = PACKAGE_ROOT / ".tmp-fake-base" / "ComfyUI"
|
||||
portable_exe = fake_base.parent / "ffmpeg" / "bin" / "ffmpeg.exe"
|
||||
portable_exe.parent.mkdir(parents=True, exist_ok=True)
|
||||
portable_exe.write_bytes(b"")
|
||||
_video_mod.folder_paths.base_path = str(fake_base)
|
||||
found = _video_mod.get_ffmpeg_path("ffmpeg")
|
||||
check(
|
||||
"portable ffmpeg/bin layout resolved",
|
||||
found == str(portable_exe),
|
||||
repr(found),
|
||||
)
|
||||
portable_exe.unlink()
|
||||
shutil.rmtree(PACKAGE_ROOT / ".tmp-fake-base", ignore_errors=True)
|
||||
_video_mod.folder_paths.base_path = _real_base
|
||||
|
||||
# ffmpeg (not ffprobe) falls back to the imageio-ffmpeg bundled binary.
|
||||
fake_imageio = types.ModuleType("imageio_ffmpeg")
|
||||
bundled = str(PACKAGE_ROOT / ".tmp-bundled-ffmpeg.exe")
|
||||
fake_imageio.get_ffmpeg_exe = lambda: bundled
|
||||
sys.modules["imageio_ffmpeg"] = fake_imageio
|
||||
found = _video_mod.get_ffmpeg_path("ffmpeg")
|
||||
check(
|
||||
"imageio-ffmpeg bundled binary used as last resort",
|
||||
found == bundled,
|
||||
repr(found),
|
||||
)
|
||||
# ffprobe is NOT bundled by imageio-ffmpeg: must NOT resolve to it.
|
||||
found = _video_mod.get_ffmpeg_path("ffprobe")
|
||||
check(
|
||||
"ffprobe never resolved from imageio-ffmpeg",
|
||||
found is None,
|
||||
repr(found),
|
||||
)
|
||||
del sys.modules["imageio_ffmpeg"]
|
||||
finally:
|
||||
os.name = _real_os_name
|
||||
_video_mod.folder_paths.base_path = _real_base
|
||||
shutil.which = _real_which
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. compose_h3_project_video propagates the reason into the RuntimeError.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _fake_merge(segments, total_length, frame_rate, w, h, **kwargs):
|
||||
reasons = kwargs.get("failure_reason")
|
||||
if reasons is not None:
|
||||
reasons.append("FFmpeg exited with code 1: Unknown encoder 'libx264'")
|
||||
return None
|
||||
|
||||
|
||||
# compose_h3_project_video imports merge_video_track_with_ffmpeg from the
|
||||
# `utils.video` module at call time (function-local import), so patch the
|
||||
# attribute on the already-loaded video module, not on h3_project.
|
||||
h3_project.load_h3_project_data = lambda *a, **k: {
|
||||
"width": 848,
|
||||
"height": 480,
|
||||
"frame_rate": 24.0,
|
||||
"clips": [
|
||||
{
|
||||
"index": 0,
|
||||
"enabled": True,
|
||||
"file_path": "easy_media/projects/p/out/seg_000.mp4",
|
||||
"updated_at": 1,
|
||||
"source_frame_count": 16,
|
||||
"source_start_frame": 0,
|
||||
"source_end_frame": 16,
|
||||
"video_files": [],
|
||||
}
|
||||
],
|
||||
}
|
||||
video_mod.merge_video_track_with_ffmpeg = _fake_merge
|
||||
|
||||
try:
|
||||
h3_project.compose_h3_project_video("proj", None)
|
||||
check("compose raises on None", False, "no exception raised")
|
||||
except RuntimeError as error:
|
||||
message = str(error)
|
||||
check(
|
||||
"compose error carries the real reason",
|
||||
"could not compose the H3 project video" in message
|
||||
and "Unknown encoder 'libx264'" in message,
|
||||
message,
|
||||
)
|
||||
except Exception as error: # noqa: BLE001
|
||||
check("compose raises RuntimeError only", False, repr(error))
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
passed = sum(1 for _, ok, _ in CHECKS if ok)
|
||||
failed = len(CHECKS) - passed
|
||||
for name, ok, detail in CHECKS:
|
||||
print(f"{'PASS' if ok else 'FAIL'}: {name}" + (f" [{detail}]" if detail and not ok else ""))
|
||||
print(f"\n{passed}/{len(CHECKS)} checks passed")
|
||||
sys.exit(1 if failed else 0)
|
||||
@ -294,7 +294,8 @@ def _install_loader_stubs(state):
|
||||
return (sd, {"meta": True}) if return_metadata else sd
|
||||
|
||||
def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None):
|
||||
raise AssertionError("MoxieMultimediaLoader must not load LoRAs anymore")
|
||||
state.calls.append(("load_lora_for_models", strength_model, strength_clip))
|
||||
return state.lora_model, clip
|
||||
|
||||
def get_attention_function(name, *args):
|
||||
state.calls.append(("get_attention_function", name))
|
||||
@ -351,7 +352,7 @@ def _load_loader_module():
|
||||
def verify_loader():
|
||||
print("\n== 4. moxie_loader.py: single-model UI + auto-picked aux models ==")
|
||||
state = types.SimpleNamespace(
|
||||
calls=[], clip=object(), attention_fn=object(),
|
||||
calls=[], clip=object(), attention_fn=object(), lora_model=object(),
|
||||
)
|
||||
_install_loader_stubs(state)
|
||||
loader = _load_loader_module()
|
||||
@ -376,14 +377,13 @@ def verify_loader():
|
||||
("vae", "video_vae.safetensors"),
|
||||
("audio_vae", "audio_vae.safetensors"),
|
||||
("clip", "MM-VL.safetensors"),
|
||||
# A leftover loras file must be ignored entirely: the distill
|
||||
# weights ship baked into the diffusion model now.
|
||||
("loras", "MM3step.safetensors"),
|
||||
):
|
||||
os.makedirs(os.path.join(models, subdir), exist_ok=True)
|
||||
with open(os.path.join(models, subdir, name), "wb") as handle:
|
||||
handle.write(b"stub")
|
||||
loader.MODELS_DIR = models
|
||||
loader._loaded_lora = None
|
||||
|
||||
result = loader.MoxieMultimediaLoader.execute(
|
||||
diffusion_model="moxie.safetensors",
|
||||
@ -392,21 +392,37 @@ def verify_loader():
|
||||
)
|
||||
|
||||
names = [call[0] for call in state.calls]
|
||||
check("full chain runs: model -> attention -> clip -> vaes (no LoRA)",
|
||||
check("full chain runs: model -> attention -> clip -> lora -> vaes",
|
||||
names == [
|
||||
"load_diffusion_model", "get_attention_function",
|
||||
"set_model_optimized_attention", "load_clip",
|
||||
"load_torch_file", "load_lora_for_models",
|
||||
"load_torch_file", "load_torch_file",
|
||||
], repr(names))
|
||||
lora_calls = [call for call in state.calls if call[0] == "load_lora_for_models"]
|
||||
check("LoRA auto-applied at strength 1.0/1.0",
|
||||
lora_calls == [("load_lora_for_models", 1.0, 1.0)], repr(lora_calls))
|
||||
clip_calls = [call for call in state.calls if call[0] == "load_clip"]
|
||||
check("CLIP auto-picked and loaded with the locked recipe",
|
||||
len(clip_calls) == 1 and clip_calls[0][1] is not None, repr(clip_calls))
|
||||
check("four outputs returned",
|
||||
isinstance(result, loader.io.NodeOutput) and len(result.values) == 4)
|
||||
check("model output is the attention clone (no LoRA on top)",
|
||||
result.values[0] is not state.model)
|
||||
check("model output carries LoRA",
|
||||
result.values[0] is state.lora_model)
|
||||
check("clip output is the loaded clip", result.values[1] is state.clip)
|
||||
|
||||
# Empty loras folder -> LoRA silently skipped, everything else loads.
|
||||
os.remove(os.path.join(models, "loras", "MM3step.safetensors"))
|
||||
state.calls.clear()
|
||||
loader._loaded_lora = None
|
||||
loader.MoxieMultimediaLoader.execute(
|
||||
diffusion_model="moxie.safetensors",
|
||||
weight_dtype="default",
|
||||
attention_backend="pytorch_attention",
|
||||
)
|
||||
check("empty loras folder skips LoRA without breaking the load",
|
||||
"load_lora_for_models" not in [call[0] for call in state.calls])
|
||||
|
||||
# Missing clip folder -> loud setup error (same message as before).
|
||||
state.calls.clear()
|
||||
os.rename(os.path.join(models, "clip"), os.path.join(models, "clip_backup"))
|
||||
|
||||
@ -266,8 +266,6 @@ custom 示例:
|
||||
- `segment_count = -1` 表示从起始索引处理到末尾;非负数限制本次任务片段数量。
|
||||
- `project_save`: `new` 或 `override`。在 `override` 且 count 为 `-1` 时,执行逻辑会删除起始索引之后的已保存片段再生成;修改前向用户清楚说明。
|
||||
- `sampling_mode`: `single` 或 `dual`;`sampling_plan` 必须是当前节点可用预设。不要仅凭另一份流程中的值假设当前环境支持。
|
||||
- `context_frames`(默认 `"22"`,可选 `"5"` / `"22"` / `"39"` / `"56"`)控制 Context / Context Swap 续接片段从上一段拷贝为延续上下文的视频帧数;取值必须落在 H3 时间潜空间网格(17k+5 帧)上,与 Moxie 上下文采样节点的选项一致。数值越大运动衔接越平滑,但每个续接片段会多生成等量帧(实际生成长度为 `任务长度 + context_frames + 12`,其中 12 帧为网格尾帧,解码后裁掉)。
|
||||
- `Moxie MultiTrack Project`(`MoxieProject`)同样暴露同一个 `context_frames` 下拉,位于 `steps` 与 `seed` 之间;该节点的其余采样设置仍保持隐藏(euler + beta sigmas 由 `steps` 推导)。
|
||||
- 编辑器分辨率是一采尺寸;`dual` 不会反算缩小一采。二采宽高分别为 `round(编辑器尺寸 * upscale_by / 32) * 32`,采用 Python `round` 就近对齐,恰好居中时取偶数,不是固定向上对齐。
|
||||
- `upscale_by` 是 Project 的独立参数,默认 `1.250`,三位小数,步长 `0.001`;旧工作流保留已保存的值。两条放大路径及合并导出均使用放大并对齐后的尺寸。例如 `1344 × 768`、倍率 `1.250` 的二采为 `1664 × 960`。若用户指定最终输出尺寸,应同时核对编辑器尺寸与倍率,不能将编辑器尺寸直接当作双采样的最终尺寸。
|
||||
- 单采样、仅一采预览保持编辑器尺寸;`upscale_by = 1.000` 不放大但可二采;纯音频项目(`32 × 32`)不放大。旧版反算缩小尺寸的一采检查点和上下文潜空间应重新生成后再按新规则续跑。
|
||||
@ -292,7 +290,6 @@ Plan 只包含实际变更。`node_id` 可省略并由拓扑定位;`editor.res
|
||||
"project_save": "override",
|
||||
"segment_start_number": 1,
|
||||
"segment_count": -1,
|
||||
"context_frames": "22",
|
||||
"sampling_plan": "medium",
|
||||
"sampling_mode": "single"
|
||||
}
|
||||
|
||||
@ -185,7 +185,7 @@ def _load_minimax_node(monkeypatch):
|
||||
Combo=_PortType,
|
||||
ComfyNode=object,
|
||||
Conditioning=_PortType,
|
||||
ControlAfterGenerate=types.SimpleNamespace(fixed="fixed", randomize="randomize"),
|
||||
ControlAfterGenerate=types.SimpleNamespace(fixed="fixed"),
|
||||
DynamicCombo=_DynamicCombo,
|
||||
Hidden=types.SimpleNamespace(prompt="PROMPT", unique_id="UNIQUE_ID"),
|
||||
Custom=lambda **kwargs: _PortType,
|
||||
@ -260,12 +260,6 @@ def _load_minimax_node(monkeypatch):
|
||||
comfy.model_management = model_management
|
||||
comfy.nested_tensor = nested_tensor
|
||||
comfy.utils = comfy_utils
|
||||
comfy_samplers = types.ModuleType("comfy.samplers")
|
||||
comfy_samplers.calculate_sigmas = (
|
||||
lambda *args, **kwargs: torch.arange(8.0)
|
||||
)
|
||||
comfy_samplers.sampler_object = lambda name: {"sampler": name}
|
||||
comfy.samplers = comfy_samplers
|
||||
|
||||
node_helpers = types.ModuleType("node_helpers")
|
||||
|
||||
@ -454,7 +448,6 @@ def _load_minimax_node(monkeypatch):
|
||||
"comfy.model_management": model_management,
|
||||
"comfy.nested_tensor": nested_tensor,
|
||||
"comfy.utils": comfy_utils,
|
||||
"comfy.samplers": comfy_samplers,
|
||||
"node_helpers": node_helpers,
|
||||
"torchaudio": torchaudio,
|
||||
"folder_paths": folder_paths,
|
||||
@ -523,21 +516,6 @@ def _load_moxie_combine(monkeypatch):
|
||||
return module
|
||||
|
||||
|
||||
def _load_moxie_project(monkeypatch):
|
||||
"""Load moxie_project.py into the stub environment set up above."""
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
root = Path(__file__).parents[1]
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"easy_media.moxie_project", root / "moxie_project.py"
|
||||
)
|
||||
assert spec is not None and spec.loader is not None
|
||||
moxie_module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = moxie_module
|
||||
spec.loader.exec_module(moxie_module)
|
||||
return moxie_module
|
||||
|
||||
|
||||
def _image_values(*values):
|
||||
return torch.tensor(values, dtype=torch.float32).reshape(len(values), 1, 1, 1)
|
||||
|
||||
@ -1260,7 +1238,6 @@ def test_multitrack_h3_project_schema_exposes_pipeline_configuration(monkeypatch
|
||||
"project_save",
|
||||
"segment_start_number",
|
||||
"segment_count",
|
||||
"context_frames",
|
||||
"seed",
|
||||
"sampling_plan",
|
||||
"sampling_mode",
|
||||
@ -1282,8 +1259,6 @@ def test_multitrack_h3_project_schema_exposes_pipeline_configuration(monkeypatch
|
||||
assert inputs["project_name"].kwargs["default"] == ""
|
||||
assert inputs["project_save"].kwargs["options"] == ["new", "override"]
|
||||
assert inputs["project_save"].kwargs["default"] == "override"
|
||||
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
|
||||
assert inputs["context_frames"].kwargs["default"] == "22"
|
||||
sampling_plan_options = inputs["sampling_plan"].kwargs["options"]
|
||||
assert sampling_plan_options == [
|
||||
"custom",
|
||||
@ -2472,53 +2447,6 @@ def test_multitrack_h3_project_reads_ref_image_size_from_each_segment(monkeypatc
|
||||
assert conditioning["ref_image_size"] == "max"
|
||||
|
||||
|
||||
def test_multitrack_h3_project_keeps_544x960_first_pass_on_grid(monkeypatch):
|
||||
"""Regression for the 2026-10-16 user crash report.
|
||||
|
||||
A 544x960 single-pass project used to sample its hidden first pass at the
|
||||
raw half resolution 272x480. The keyframe conditioned on that canvas is a
|
||||
17-wide latent, and comfy's patchify_video floors it into 2x2 patches,
|
||||
crashing with "shape '[1, 24, 1, 1, 15, 2, 8, 2]' is invalid for input of
|
||||
size 12240". The first pass must land on the 32px canvas grid instead.
|
||||
"""
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
module.comfy_nodes.NODE_CLASS_MAPPINGS["ImageResizeKJv2"] = _ImageResizeKJWithNvidia
|
||||
inputs = _h3_project_inputs(sampling_mode=_h3_sampling_mode("single"))
|
||||
inputs["tracks_info"][0]["width"] = 544
|
||||
inputs["tracks_info"][0]["height"] = 960
|
||||
|
||||
result = module.EasyMultiTrackProject.execute(**inputs)
|
||||
conditioning = _graph_node(result, "easy minimaxH3ToVideo")["inputs"]
|
||||
|
||||
# half of 544x960 stays on the 32px grid: 256x480 -> latent 16x30, even.
|
||||
assert conditioning["width"] == 256
|
||||
assert conditioning["height"] == 480
|
||||
|
||||
|
||||
def test_multitrack_h3_project_aligns_off_grid_api_dimensions(monkeypatch):
|
||||
"""External app clients inject project dimensions through the ComfyUI API
|
||||
that are not multiples of 32; the editor UI only offers grid sizes. The
|
||||
project must run both sampling passes on the same grid-aligned canvas
|
||||
instead of the raw injected size (odd latent -> patchify_video crash)."""
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
module.comfy_nodes.NODE_CLASS_MAPPINGS["ImageResizeKJv2"] = _ImageResizeKJWithNvidia
|
||||
inputs = _h3_project_inputs(sampling_mode=_h3_sampling_mode("dual"))
|
||||
inputs["tracks_info"][0]["width"] = 530
|
||||
inputs["tracks_info"][0]["height"] = 964
|
||||
|
||||
result = module.EasyMultiTrackProject.execute(**inputs)
|
||||
conditionings = [
|
||||
node["inputs"]
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy minimaxH3ToVideo"
|
||||
]
|
||||
|
||||
# 530x964 floors onto the 32px grid as 512x960; the default 1.25x second
|
||||
# pass rounds to 640x1216 (960*1.25=1200 rounds up to the 32px grid).
|
||||
assert sorted(item["width"] for item in conditionings) == [512, 640]
|
||||
assert sorted(item["height"] for item in conditionings) == [960, 1216]
|
||||
|
||||
|
||||
def test_multitrack_h3_project_logs_steps_and_reports_progress(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
@ -3821,89 +3749,6 @@ def test_multitrack_h3_context_chain_uses_previous_segment_latent(monkeypatch):
|
||||
assert artifacts[1][1]["inputs"]["previous"] == [artifacts[0][0], 0]
|
||||
|
||||
|
||||
def test_multitrack_context_frames_widget_propagates(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
module.comfy_nodes.NODE_CLASS_MAPPINGS["MiniMaxH3MotionContextTrim"] = (
|
||||
_MiniMaxMotionContextTrim
|
||||
)
|
||||
info = _h3_project_inputs()["tracks_info"][0]
|
||||
info["tracks"][0]["segments"].append(
|
||||
{
|
||||
"start_frame": 120,
|
||||
"end_frame": 240,
|
||||
"content": {
|
||||
"task_mode": "l2v",
|
||||
"continuity_mode": "context",
|
||||
"images": [{"media_index": 0}],
|
||||
"user_prompt": "continue",
|
||||
},
|
||||
}
|
||||
)
|
||||
info["tracks"].append(
|
||||
{
|
||||
"id": "locked-audio-track",
|
||||
"type": "audio",
|
||||
"audio_locked": True,
|
||||
"segments": [
|
||||
{
|
||||
"id": "locked-audio",
|
||||
"start_frame": 0,
|
||||
"end_frame": 240,
|
||||
"content": {"media_type": "audio"},
|
||||
}
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
result = module.EasyMultiTrackProject.execute(
|
||||
**_h3_project_inputs(tracks_info=[info], context_frames=["5"])
|
||||
)
|
||||
|
||||
motion = next(
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy MiniMaxH3MotionContextHard"
|
||||
)
|
||||
assert motion["inputs"]["context_length"] == "5"
|
||||
context_trims = [
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy h3MotionContextLatentTrim"
|
||||
]
|
||||
assert context_trims
|
||||
assert {trim["inputs"]["context_length"] for trim in context_trims} == {"5"}
|
||||
audio_locks = [
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy minimaxH3AudioLock"
|
||||
]
|
||||
assert sorted(node["inputs"]["prepend_frames"] for node in audio_locks) == [0, 5]
|
||||
math_node = next(
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "ComfyMathExpression"
|
||||
)
|
||||
# The context continuation generates context_frames + 12 grid-tail frames.
|
||||
assert math_node["inputs"]["expression"] == "a + 17"
|
||||
context_encodes = [
|
||||
node
|
||||
for node in result.expand.values()
|
||||
if node["class_type"] == "easy h3ContextMediaTrim"
|
||||
and node["inputs"]["output_frames"] == 5
|
||||
]
|
||||
assert context_encodes
|
||||
|
||||
|
||||
def test_multitrack_context_frames_widget_rejects_off_grid_values(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
with pytest.raises(ValueError, match="supported H3 context lengths"):
|
||||
module.EasyMultiTrackProject.execute(
|
||||
**_h3_project_inputs(context_frames=["18"])
|
||||
)
|
||||
|
||||
|
||||
def test_multitrack_h3_context_uses_embedded_five_frame_anchor(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
info = _h3_project_inputs()["tracks_info"][0]
|
||||
@ -5369,32 +5214,6 @@ def test_reference_encodes_images_video_audio_and_standalone_audio(monkeypatch):
|
||||
assert conditioning["inputs"]["ref_video_0"][1] == 0
|
||||
|
||||
|
||||
def test_reference_fallback_grid_aligns_off_grid_ref_videos(monkeypatch):
|
||||
"""ComfyUI builds with native H3 nodes grid-align reference videos inside
|
||||
core's MiniMaxH3ReferenceToVideo; the fallback copy must do the same, or
|
||||
an off-grid media size (the API can inject arbitrary media) yields an odd
|
||||
latent dimension and crashes patchify_video during sampling."""
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
vae = _Vae()
|
||||
|
||||
module.MiniMaxH3ReferenceToVideoFallback.execute(
|
||||
clip=_Clip(),
|
||||
vae=vae,
|
||||
audio_vae=None,
|
||||
prompt="prompt",
|
||||
width=32,
|
||||
height=32,
|
||||
length=5,
|
||||
ref_videos={"ref_video_0": torch.zeros(6, 270, 480, 3)},
|
||||
)
|
||||
|
||||
encoded_frames = vae.encoded[-1]
|
||||
# Frames are [T, H, W, C]: 270 floors onto the 32px grid as 256; 480 is
|
||||
# already aligned.
|
||||
assert tuple(encoded_frames.shape[1:3]) == (256, 480)
|
||||
|
||||
|
||||
def test_reference_bridge_groups_fixed_inputs_before_direct_execute(monkeypatch):
|
||||
module = _load_minimax_node(monkeypatch)
|
||||
assert module is not None
|
||||
@ -6275,68 +6094,3 @@ def test_minimax_prompt_override_node_has_complete_chinese_localization():
|
||||
assert translation["inputs"]["video_track_lock"]["name"] == "视频轨锁定"
|
||||
assert translation["inputs"]["audio_track_lock"]["name"] == "音频轨锁定"
|
||||
assert translation["outputs"] == {"0": {"name": "提示词覆盖"}}
|
||||
|
||||
|
||||
def test_moxie_project_schema_exposes_context_frames(monkeypatch):
|
||||
module = _load_moxie_project(monkeypatch)
|
||||
schema = module.MoxieProject.define_schema()
|
||||
inputs = {port.name: port for port in schema.inputs}
|
||||
|
||||
assert schema.node_id == "MoxieProject"
|
||||
assert schema.display_name == "Moxie MultiTrack Project"
|
||||
assert schema.is_input_list is True
|
||||
assert list(inputs) == [
|
||||
"tracks_info",
|
||||
"model",
|
||||
"clip",
|
||||
"video_vae",
|
||||
"audio_vae",
|
||||
"project_name",
|
||||
"steps",
|
||||
"context_frames",
|
||||
"seed",
|
||||
]
|
||||
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
|
||||
assert inputs["context_frames"].kwargs["default"] == "22"
|
||||
assert inputs["steps"].kwargs["default"] == 3
|
||||
assert inputs["seed"].kwargs["default"] == 42
|
||||
|
||||
|
||||
def test_moxie_project_forwards_context_frames_to_engine(monkeypatch):
|
||||
module = _load_moxie_project(monkeypatch)
|
||||
|
||||
captured = {}
|
||||
|
||||
def fake_execute(cls, **kwargs):
|
||||
captured.update(kwargs)
|
||||
return "sentinel-output"
|
||||
|
||||
monkeypatch.setattr(
|
||||
module.EasyMultiTrackProject, "execute", classmethod(fake_execute)
|
||||
)
|
||||
|
||||
model = types.SimpleNamespace(
|
||||
get_model_object=lambda name: types.SimpleNamespace()
|
||||
)
|
||||
result = module.MoxieProject.execute(
|
||||
tracks_info=[{"tracks": []}],
|
||||
model=[model],
|
||||
clip=[types.SimpleNamespace()],
|
||||
video_vae=[types.SimpleNamespace()],
|
||||
audio_vae=[types.SimpleNamespace()],
|
||||
project_name=["demo"],
|
||||
steps=["4"],
|
||||
context_frames=["5"],
|
||||
seed=["7"],
|
||||
)
|
||||
|
||||
assert result == "sentinel-output"
|
||||
assert captured["context_frames"] == ["5"]
|
||||
assert captured["steps"] == ["4"]
|
||||
assert captured["project_name"] == ["demo"]
|
||||
assert captured["seed"] == ["7"]
|
||||
# Baked settings survive the handoff and never clobber the widget.
|
||||
assert captured["sampling_plan"] == "custom"
|
||||
assert captured["sampling_mode"] == "single"
|
||||
assert captured["sampler"] == {"sampler": "euler"}
|
||||
assert captured["sigmas"].shape == (5,)
|
||||
|
||||
@ -7,8 +7,7 @@ Verified contracts:
|
||||
- MoxieMultimediaLoader: outputs exactly (model, clip, video_vae, audio_vae);
|
||||
no legacy FAST_MODEL_LOADER bundle output; no clip_type choice (the CLIP
|
||||
recipe is locked to CLIPType.MINIMAX); the model output has the attention
|
||||
backend applied, and any leftover file in models/loras is ignored (the
|
||||
distill weights ship baked into the diffusion model).
|
||||
backend and LoRA applied.
|
||||
- MoxieProject: takes model/clip/video_vae/audio_vae as required direct
|
||||
inputs instead of a bundled model_loader, and packs them into the loader
|
||||
dict consumed internally by EasyMultiTrackProject.
|
||||
@ -123,6 +122,8 @@ class _StubState:
|
||||
self.calls = []
|
||||
self.attention_fn = lambda pixels, *a, **k: pixels
|
||||
self.clip = object()
|
||||
self.lora_model = None
|
||||
self.lora_clip = None
|
||||
|
||||
|
||||
def _register(state, io, monkeypatch=None):
|
||||
@ -157,7 +158,10 @@ def _register(state, io, monkeypatch=None):
|
||||
return state.clip
|
||||
|
||||
def load_lora_for_models(model, clip, lora_sd, strength_model, strength_clip, lora_metadata=None):
|
||||
raise AssertionError("MoxieMultimediaLoader must not load LoRAs anymore")
|
||||
state.calls.append(("load_lora_for_models", strength_model, strength_clip))
|
||||
state.lora_model = model
|
||||
state.lora_clip = clip
|
||||
return model, clip
|
||||
|
||||
comfy_sd.CLIPType = CLIPType
|
||||
comfy_sd.load_diffusion_model = load_diffusion_model
|
||||
@ -299,7 +303,6 @@ def _build_moxie():
|
||||
|
||||
tiny_vae = types.ModuleType("MMnode.tiny_vae")
|
||||
tiny_vae.load_tiny_vae_decoder = lambda *a, **k: None
|
||||
tiny_vae.pack_vae_approx_files = lambda *a, **k: []
|
||||
added["MMnode.tiny_vae"] = sys.modules.get("MMnode.tiny_vae")
|
||||
sys.modules["MMnode.tiny_vae"] = tiny_vae
|
||||
|
||||
@ -357,7 +360,7 @@ def test_loader_schema_shows_a_single_model(moxie):
|
||||
inputs = _input_names(schema)
|
||||
|
||||
# End users see one model picker plus the attention backend; every
|
||||
# auxiliary model (video/audio VAE, CLIP) is auto-selected.
|
||||
# auxiliary model (LoRA, video/audio VAE, CLIP) is auto-selected.
|
||||
assert inputs == ["diffusion_model", "weight_dtype", "attention_backend"]
|
||||
for hidden in (
|
||||
"lora_name",
|
||||
@ -371,7 +374,7 @@ def test_loader_schema_shows_a_single_model(moxie):
|
||||
assert hidden not in inputs
|
||||
|
||||
|
||||
def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch, tmp_path):
|
||||
def test_loader_execute_applies_attention_then_lora(moxie, monkeypatch, tmp_path):
|
||||
state = moxie.state
|
||||
models = tmp_path / "models"
|
||||
for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"):
|
||||
@ -386,6 +389,9 @@ def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch,
|
||||
}[name]
|
||||
(models / subdir / name).write_bytes(b"stub")
|
||||
monkeypatch.setattr(moxie.loader, "MODELS_DIR", models)
|
||||
# LoRA cache lives at module level: node classes are locked by newer
|
||||
# ComfyUI builds and class attributes can no longer be written at runtime.
|
||||
monkeypatch.setattr(moxie.loader, "_loaded_lora", None)
|
||||
state.calls.clear()
|
||||
|
||||
result = moxie.loader.MoxieMultimediaLoader.execute(
|
||||
@ -394,15 +400,14 @@ def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch,
|
||||
attention_backend="comfy_kitchen_attention",
|
||||
)
|
||||
|
||||
# The distill weights ship baked into the diffusion model: a leftover
|
||||
# file in models/loras must be ignored entirely. The comfy.sd stub keeps
|
||||
# a raising load_lora_for_models, so an accidental LoRA call fails loudly.
|
||||
sequence = [call[0] for call in state.calls]
|
||||
assert sequence == [
|
||||
"load_diffusion_model",
|
||||
"get_attention_function",
|
||||
"load_clip",
|
||||
"load_torch_file",
|
||||
"load_lora_for_models",
|
||||
"load_torch_file",
|
||||
"load_torch_file",
|
||||
]
|
||||
|
||||
@ -410,22 +415,50 @@ def test_loader_execute_applies_attention_and_ignores_loras(moxie, monkeypatch,
|
||||
clip_calls = [call for call in state.calls if call[0] == "load_clip"]
|
||||
assert clip_calls == [("load_clip", state.CLIPType.MINIMAX)]
|
||||
|
||||
# Attention backend applied before the CLIP load.
|
||||
# LoRA applied on top of the attention-patched model, after the CLIP load.
|
||||
assert state.calls.index(("get_attention_function", "comfy_kitchen_int8")) < state.calls.index(
|
||||
("load_clip", state.CLIPType.MINIMAX)
|
||||
)
|
||||
) < state.calls.index(("load_lora_for_models", 1.0, 1.0))
|
||||
|
||||
# Exactly four direct outputs, model carrying the attention backend.
|
||||
# Exactly four direct outputs, model carrying attention + LoRA.
|
||||
assert isinstance(result, _NodeOutput)
|
||||
assert len(result.values) == 4
|
||||
model_out, clip_out, video_vae_out, audio_vae_out = result.values
|
||||
assert model_out.label == "diffusion.clone"
|
||||
assert model_out is state.lora_model
|
||||
assert model_out.attention_fn is state.attention_fn
|
||||
assert clip_out is state.clip
|
||||
assert isinstance(video_vae_out, _FakeVAE)
|
||||
assert isinstance(audio_vae_out, _FakeVAE)
|
||||
|
||||
|
||||
def test_loader_execute_without_lora_skips_lora_loading(moxie, monkeypatch, tmp_path):
|
||||
state = moxie.state
|
||||
models = tmp_path / "models"
|
||||
for subdir in ("diffusion", "vae", "audio_vae", "clip", "loras"):
|
||||
(models / subdir).mkdir(parents=True, exist_ok=True)
|
||||
for name in ("h3.safetensors", "video.safetensors", "audio.safetensors", "qwen.safetensors"):
|
||||
subdir = {
|
||||
"h3.safetensors": "diffusion",
|
||||
"video.safetensors": "vae",
|
||||
"audio.safetensors": "audio_vae",
|
||||
"qwen.safetensors": "clip",
|
||||
}[name]
|
||||
(models / subdir / name).write_bytes(b"stub")
|
||||
monkeypatch.setattr(moxie.loader, "MODELS_DIR", models)
|
||||
state.calls.clear()
|
||||
|
||||
result = moxie.loader.MoxieMultimediaLoader.execute(
|
||||
diffusion_model="h3.safetensors",
|
||||
weight_dtype="default",
|
||||
attention_backend="pytorch_attention",
|
||||
)
|
||||
|
||||
assert "load_lora_for_models" not in [call[0] for call in state.calls]
|
||||
assert result.values[0] is not state.clip # model output remains a model
|
||||
assert result.values[0].attention_fn is state.attention_fn
|
||||
assert result.values[1] is state.clip
|
||||
|
||||
|
||||
def test_loader_requires_minimax_cliptype(moxie, monkeypatch, tmp_path):
|
||||
state = moxie.state
|
||||
models = tmp_path / "models"
|
||||
@ -462,12 +495,9 @@ def test_project_schema_takes_direct_model_inputs(moxie):
|
||||
"audio_vae",
|
||||
"project_name",
|
||||
"steps",
|
||||
"context_frames",
|
||||
"seed",
|
||||
]
|
||||
assert "model_loader" not in inputs
|
||||
assert inputs["context_frames"].kwargs["options"] == ["5", "22", "39", "56"]
|
||||
assert inputs["context_frames"].kwargs["default"] == "22"
|
||||
|
||||
# The four model components are required direct connections.
|
||||
for name in ("model", "clip", "video_vae", "audio_vae"):
|
||||
|
||||
138
update.bat
138
update.bat
@ -1,138 +0,0 @@
|
||||
@echo off
|
||||
setlocal
|
||||
title MMnode Updater
|
||||
|
||||
REM ============================================================
|
||||
REM MMnode (Moxie-Multimedia) one-click updater
|
||||
REM
|
||||
REM Put this file INSIDE the pack folder:
|
||||
REM ComfyUI\custom_nodes\<pack-folder>\
|
||||
REM (next to __init__.py) and double-click it.
|
||||
REM
|
||||
REM It syncs code from HuggingFace, removes stale files from
|
||||
REM old versions, downloads model weights via git-lfs, verifies
|
||||
REM the build, and reminds you to restart ComfyUI.
|
||||
REM ============================================================
|
||||
|
||||
cd /d "%~dp0"
|
||||
|
||||
echo.
|
||||
echo ==================================================
|
||||
echo MMnode updater - syncing from HuggingFace
|
||||
echo ==================================================
|
||||
echo.
|
||||
|
||||
if not exist "__init__.py" (
|
||||
echo [ERROR] This is not the MMnode pack folder.
|
||||
echo Move update.bat into ComfyUI\custom_nodes\^<pack-folder^>
|
||||
echo next to __init__.py, then run it again.
|
||||
goto end_fail
|
||||
)
|
||||
|
||||
where git >nul 2>&1
|
||||
if errorlevel 1 (
|
||||
echo [ERROR] Git is not installed or not in PATH.
|
||||
echo Get it from https://git-scm.com/download/win
|
||||
goto end_fail
|
||||
)
|
||||
|
||||
netstat -ano 2>nul | findstr /c:":8188" >nul 2>&1
|
||||
if not errorlevel 1 (
|
||||
echo [WARN] ComfyUI appears to be running on port 8188.
|
||||
echo It must be fully closed for the update to take effect.
|
||||
echo Continuing in 5 seconds - close it now...
|
||||
timeout /t 5 /nobreak >nul
|
||||
)
|
||||
|
||||
if not exist ".git" (
|
||||
echo [*] No git repo here yet - initializing...
|
||||
git init
|
||||
if errorlevel 1 goto gitfail
|
||||
)
|
||||
|
||||
git remote remove origin >nul 2>&1
|
||||
git remote add origin https://huggingface.co/Moxiegen/Moxie-Multimedia
|
||||
if errorlevel 1 goto gitfail
|
||||
|
||||
echo.
|
||||
echo [*] Fetching latest version from HuggingFace...
|
||||
git fetch origin --prune
|
||||
if errorlevel 1 (
|
||||
echo [ERROR] Could not reach HuggingFace - check your internet.
|
||||
goto end_fail
|
||||
)
|
||||
|
||||
echo.
|
||||
echo [*] Syncing files...
|
||||
git reset --hard origin/main
|
||||
if errorlevel 1 goto gitfail
|
||||
|
||||
echo.
|
||||
echo [*] Cleaning stale files from older versions...
|
||||
git clean -fd -e update.bat >nul 2>&1
|
||||
|
||||
echo.
|
||||
echo [*] Syncing model weights...
|
||||
git lfs version >nul 2>&1
|
||||
if errorlevel 1 goto nolfs
|
||||
git lfs install >nul 2>&1
|
||||
echo Downloading weights - first run can take several minutes...
|
||||
git lfs fetch origin main
|
||||
if errorlevel 1 echo [WARN] Weight download failed - check connection, re-run later.
|
||||
git lfs checkout
|
||||
goto after_lfs
|
||||
|
||||
:nolfs
|
||||
echo [WARN] git-lfs is NOT installed on this PC.
|
||||
echo Code was updated, but model weights may be broken placeholders.
|
||||
echo Install git-lfs from https://git-lfs.com
|
||||
echo or run: winget install GitHub.git-lfs
|
||||
echo ...then run this updater again.
|
||||
|
||||
:after_lfs
|
||||
|
||||
REM ---- verify build ----
|
||||
set FIXCOUNT=0
|
||||
for /f %%A in ('findstr /c:"_align_canvas_dimension" "nodes\project.py" ^| find /c /v ""') do set FIXCOUNT=%%A
|
||||
|
||||
set POINTER=0
|
||||
for /r models %%F in (*.safetensors) do (
|
||||
if %%~zF LSS 1000 set /a POINTER+=1
|
||||
)
|
||||
|
||||
echo.
|
||||
echo ==================================================
|
||||
for /f "delims=" %%L in ('git log -1 --oneline') do echo Build: %%L
|
||||
echo Fix marker: %FIXCOUNT% found in nodes\project.py
|
||||
if %FIXCOUNT% GEQ 3 (
|
||||
echo [OK] This is the fixed build.
|
||||
) else (
|
||||
echo [ERROR] Fix marker missing - update did not complete!
|
||||
)
|
||||
if %POINTER% GTR 0 (
|
||||
echo [WARN] %POINTER% model file^(s^) are broken placeholders.
|
||||
echo Install git-lfs from https://git-lfs.com and re-run.
|
||||
) else (
|
||||
echo [OK] Model weights look good.
|
||||
)
|
||||
echo ==================================================
|
||||
|
||||
echo.
|
||||
echo All done. RESTART COMFYUI COMPLETELY before testing.
|
||||
echo Close this window and any ComfyUI console, then start ComfyUI.
|
||||
goto end_ok
|
||||
|
||||
:gitfail
|
||||
echo.
|
||||
echo [ERROR] A git command failed - scroll up for the message.
|
||||
goto end_fail
|
||||
|
||||
:end_fail
|
||||
echo.
|
||||
pause
|
||||
exit /b 1
|
||||
|
||||
:end_ok
|
||||
echo.
|
||||
pause
|
||||
exit /b 0
|
||||
@ -1327,18 +1327,15 @@ def compose_h3_project_video(project_name: Any, project_data: Any = None) -> Pat
|
||||
ffprobe_info,
|
||||
)
|
||||
|
||||
failure_reasons: list[str] = []
|
||||
temporary = merge_video_track_with_ffmpeg(
|
||||
timeline_segments,
|
||||
cursor,
|
||||
frame_rate,
|
||||
compose_size[0] if compose_size else width,
|
||||
compose_size[1] if compose_size else height,
|
||||
failure_reason=failure_reasons,
|
||||
)
|
||||
if temporary is None:
|
||||
detail = "; ".join(failure_reasons) if failure_reasons else "no reason was recorded"
|
||||
raise RuntimeError(f"FFmpeg could not compose the H3 project video: {detail}")
|
||||
raise RuntimeError("FFmpeg could not compose the H3 project video")
|
||||
return Path(temporary)
|
||||
|
||||
|
||||
|
||||
131
utils/video.py
131
utils/video.py
@ -7,7 +7,6 @@ import math
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import urllib.request
|
||||
from .media import AUDIO_EXTENSIONS
|
||||
@ -308,72 +307,21 @@ def merge_video_track_with_ffmpeg(
|
||||
width: int,
|
||||
height: int,
|
||||
resize_method: str | None = None,
|
||||
*,
|
||||
failure_reason: list[str] | None = None,
|
||||
) -> str | None:
|
||||
"""Compose file-backed segments on a black full-length timeline.
|
||||
|
||||
Returns ``None`` when composition is impossible so callers can fall back
|
||||
to another strategy. Callers that need to surface the real cause instead
|
||||
pass a ``failure_reason`` list, which receives one human-readable entry
|
||||
per failed precondition (missing FFmpeg, missing source files, ...) or
|
||||
the FFmpeg stderr tail when the command itself fails.
|
||||
"""
|
||||
|
||||
def record(reason: str) -> None:
|
||||
if failure_reason is not None:
|
||||
failure_reason.append(reason)
|
||||
|
||||
"""Compose file-backed segments on a black full-length timeline."""
|
||||
ffmpeg = get_ffmpeg_path("ffmpeg")
|
||||
resize_filter = _ffmpeg_resize_filter(width, height, resize_method) if resize_method else None
|
||||
if ffmpeg is None:
|
||||
record(
|
||||
"FFmpeg executable not found (checked PATH, common install folders, "
|
||||
"package-manager shims and the imageio-ffmpeg bundled binary). "
|
||||
"Install FFmpeg with ffmpeg.exe on PATH - or simply copy ffmpeg.exe "
|
||||
"into C:\\ffmpeg\\bin - and restart ComfyUI from a NEW terminal so "
|
||||
"the updated PATH is picked up"
|
||||
)
|
||||
return None
|
||||
if total_length <= 0:
|
||||
record(f"timeline is empty (total_length={total_length} frames)")
|
||||
return None
|
||||
if frame_rate <= 0:
|
||||
record(f"invalid frame rate {frame_rate}")
|
||||
return None
|
||||
if width % 2 or height % 2:
|
||||
record(
|
||||
f"canvas size {width}x{height} has odd dimensions "
|
||||
"(video encoders require even width and height)"
|
||||
)
|
||||
if ffmpeg is None or total_length <= 0 or frame_rate <= 0 or width % 2 or height % 2:
|
||||
return None
|
||||
if resize_method is not None and resize_filter is None:
|
||||
record(f"resize method '{resize_method}' is not supported by this FFmpeg build")
|
||||
return None
|
||||
missing_sources = [
|
||||
str(segment.get("source", ""))
|
||||
for segment in segments
|
||||
if not os.path.isfile(str(segment.get("source", "")))
|
||||
]
|
||||
if missing_sources:
|
||||
record(
|
||||
"segment source file(s) missing: "
|
||||
+ " | ".join(missing_sources[:5])
|
||||
+ (f" (+{len(missing_sources) - 5} more)" if len(missing_sources) > 5 else "")
|
||||
)
|
||||
if any(not os.path.isfile(str(segment.get("source", ""))) for segment in segments):
|
||||
return None
|
||||
missing_audio = [
|
||||
str(segment["audio_source"])
|
||||
for segment in segments
|
||||
if segment.get("audio_source") is not None
|
||||
if any(
|
||||
segment.get("audio_source") is not None
|
||||
and not os.path.isfile(str(segment["audio_source"]))
|
||||
]
|
||||
if missing_audio:
|
||||
record(
|
||||
"locked audio file(s) missing: "
|
||||
+ " | ".join(missing_audio[:5])
|
||||
+ (f" (+{len(missing_audio) - 5} more)" if len(missing_audio) > 5 else "")
|
||||
)
|
||||
for segment in segments
|
||||
):
|
||||
return None
|
||||
|
||||
total_seconds = total_length / frame_rate
|
||||
@ -547,7 +495,6 @@ def merge_video_track_with_ffmpeg(
|
||||
) from exc
|
||||
except OSError as exc:
|
||||
logger.warning("FFmpeg video track merge failed to start: %s", exc)
|
||||
record(f"FFmpeg failed to start ({exc})")
|
||||
try:
|
||||
os.unlink(output_path)
|
||||
except OSError:
|
||||
@ -556,10 +503,9 @@ def merge_video_track_with_ffmpeg(
|
||||
if result.returncode == 0:
|
||||
return output_path
|
||||
|
||||
stderr_tail = result.stderr.decode(errors="replace")[-600:]
|
||||
logger.warning("FFmpeg video track merge failed: %s", stderr_tail)
|
||||
record(
|
||||
f"FFmpeg exited with code {result.returncode}: {stderr_tail.strip() or 'no stderr output'}"
|
||||
logger.warning(
|
||||
"FFmpeg video track merge failed: %s",
|
||||
result.stderr.decode(errors="replace")[-600:],
|
||||
)
|
||||
try:
|
||||
os.unlink(output_path)
|
||||
@ -569,58 +515,19 @@ def merge_video_track_with_ffmpeg(
|
||||
|
||||
|
||||
def get_ffmpeg_path(name: str = "ffmpeg") -> str | None:
|
||||
"""Find FFmpeg/ffprobe executable, with Windows-specific fallbacks.
|
||||
|
||||
PATH edits on Windows only reach processes started afterwards, and
|
||||
Portable users routinely have ffmpeg.exe sitting next to the run bat or
|
||||
installed via winget/scoop/choco without the ComfyUI process ever seeing
|
||||
it. After ``which`` fails, probe the locations such installs actually
|
||||
use, and finally fall back to the binary bundled with imageio-ffmpeg
|
||||
(a ComfyUI core dependency) so a zero-setup machine can still encode.
|
||||
"""
|
||||
"""Find FFmpeg/ffprobe executable, with Windows-specific fallbacks."""
|
||||
# Try standard which first (works on all platforms)
|
||||
ffmpeg = shutil.which(name) or shutil.which(f"{name}.exe")
|
||||
if ffmpeg:
|
||||
return ffmpeg
|
||||
if os.name != "nt":
|
||||
return None
|
||||
exe = f"{name}.exe"
|
||||
candidates: list[str] = [
|
||||
r"C:\ffmpeg\bin\{}".format(name),
|
||||
os.path.expandvars(r"%LOCALAPPDATA%\Programs\{}\bin\{}".format(name, name)),
|
||||
# Package-manager shims/links.
|
||||
os.path.expandvars(r"%LOCALAPPDATA%\Microsoft\WinGet\Links\{}".format(exe)),
|
||||
os.path.expandvars(r"%USERPROFILE%\scoop\shims\{}".format(exe)),
|
||||
r"C:\ProgramData\chocolatey\bin\{}".format(exe),
|
||||
]
|
||||
# Relative to the ComfyUI install: the portable root (parent of the
|
||||
# ComfyUI folder) and the folder itself. ffmpeg.exe next to the run bat
|
||||
# or in an ffmpeg/bin subfolder is a very common portable layout.
|
||||
base = str(getattr(folder_paths, "base_path", "") or "")
|
||||
roots: list[str] = []
|
||||
if base:
|
||||
roots.append(base)
|
||||
roots.append(os.path.dirname(os.path.abspath(base)))
|
||||
roots.append(os.path.dirname(os.path.abspath(sys.executable)))
|
||||
roots.append(os.getcwd())
|
||||
for root in roots:
|
||||
candidates.extend([
|
||||
os.path.join(root, exe),
|
||||
os.path.join(root, "bin", exe),
|
||||
os.path.join(root, "ffmpeg", "bin", exe),
|
||||
])
|
||||
for candidate in candidates:
|
||||
if os.path.isfile(candidate):
|
||||
return candidate
|
||||
# Last resort: imageio-ffmpeg ships a full ffmpeg build (libx264
|
||||
# included). It bundles no ffprobe, so only use it for ffmpeg itself.
|
||||
if name == "ffmpeg":
|
||||
try:
|
||||
import imageio_ffmpeg
|
||||
|
||||
return imageio_ffmpeg.get_ffmpeg_exe()
|
||||
except Exception:
|
||||
pass
|
||||
# Windows fallback: common installation paths
|
||||
if os.name == "nt":
|
||||
for path in (
|
||||
r"C:\ffmpeg\bin\{}.exe".format(name),
|
||||
os.path.expandvars(r"%LOCALAPPDATA%\Programs\{}\bin\{}.exe".format(name, name)),
|
||||
):
|
||||
if os.path.isfile(path):
|
||||
return path
|
||||
return None
|
||||
|
||||
|
||||
|
||||
@ -39,12 +39,8 @@ if (!packName) {
|
||||
} else {
|
||||
// Base prefix covers deployments behind a sub-path reverse proxy
|
||||
// ("/comfyui/extensions/<pack>/..." -> "/comfyui/"); the pack's own
|
||||
// static mount sits at "<base><pack>/". For the standard deployment
|
||||
// there is no prefix and base is exactly "/" - it must NEVER be "//",
|
||||
// which the browser would parse as a protocol-relative URL and try to
|
||||
// resolve "<pack>" as a DNS hostname (net::ERR_NAME_NOT_RESOLVED).
|
||||
const prefix = segments.slice(0, extensionsIndex);
|
||||
const base = prefix.length ? '/' + prefix.join('/') + '/' : '/';
|
||||
// static mount sits at "<base><pack>/".
|
||||
const base = '/' + segments.slice(0, extensionsIndex).join('/') + '/';
|
||||
// Top-level await: extension scripts are awaited before ComfyUI registers
|
||||
// node definitions, so the editor's registerExtension()/custom-widget
|
||||
// hooks stay in the normal loading order.
|
||||
|
||||
Loading…
Reference in New Issue
Block a user