feat: Moxie bench - standalone python/web harness for the fixed pipeline
- bench/ FastAPI server: fixed 9-stage pipeline parsed from the base workflow, serial render jobs, WS progress mirroring ComfyUI's push events, /api/view + ComfyUI-compatible /view, /upload/image, /easy-media/* aliases for the ported editor - dual backend: self-contained mock renderer (numpy+ffmpeg, deterministic per prompt/seed, native + RTX-upscaled mp4 branches) and a ComfyUI bridge (substitutes the plan into base_workflow.json, converts to the /prompt API format, polls history, downloads outputs) - timeline port: frontend/src/bench mounts the unmodified MultiTrack editor against a comfyAPI shim (fetchApi/toast/dialog/custom events); build.bench.ts emits bench/ui-dist served by the server - packaging: PyInstaller onedir spec + Tauri2 sidecar docs - tests: bench/tests contract suite + scripts/test_bench_pipeline.py e2e (API+WS+render); prebuilt bench UI committed
This commit is contained in:
parent
5d81069400
commit
eefca88d35
@ -6,6 +6,7 @@
|
|||||||
|
|
||||||
### ⚡ Improvements
|
### ⚡ Improvements
|
||||||
|
|
||||||
|
- **Moxie Bench (new)**: A standalone Python + web harness for the fixed Moxie pipeline (`bench/`) — the first step toward running the pack outside ComfyUI as a Tauri 2 sidecar. It locks to the base workflow's nine-stage graph (loader, LoRA, preview override, multitrack editor, project, combine, RTX upscale, create/save) and ships two interchangeable backends: a self-contained mock renderer (numpy + ffmpeg, deterministic per prompt/seed, real mp4 outputs on both the native and upscaled branches) and a ComfyUI bridge that drives a running ComfyUI over `/prompt` with the substituted workflow. The multitrack timeline editor UI is ported as-is (`frontend/src/bench/` shim mounts the same React components against a REST + WebSocket API), the server (FastAPI) exposes `/api/render`, `/api/jobs`, `WS /api/ws` progress plus ComfyUI-compatible `/view`, `/upload/image`, and `/easy-media/*` aliases, and `bench/moxie_bench.spec` + `bench/tauri/README.md` cover PyInstaller packaging and sidecar wiring. Run `python bench/run.py --port 8787`.
|
||||||
- **MultiTrack Editor**: The editor frontend was **reverted to the pristine upstream v1.3.3 source** — the MiniMax/Moxie format display-label mapping and every workaround built around it were removed. The shipped editor bundle is now built from that pristine source, so its behavior is exactly the battle-tested upstream editor (Shot/Context continuity, reference-size, audio lock, and MiniMax prompt placeholders all key on the native `MiniMax` format value again).
|
- **MultiTrack Editor**: The editor frontend was **reverted to the pristine upstream v1.3.3 source** — the MiniMax/Moxie format display-label mapping and every workaround built around it were removed. The shipped editor bundle is now built from that pristine source, so its behavior is exactly the battle-tested upstream editor (Shot/Context continuity, reference-size, audio lock, and MiniMax prompt placeholders all key on the native `MiniMax` format value again).
|
||||||
- **Editor delivery fix**: A fresh browser could previously end up without the editor bundle entirely. ComfyUI core lets a pack's `WEB_DIRECTORY` overwrite its extension web-dir registration, so the browser's `/extensions` listing only ever contained this pack's helper scripts — never the editor entry — and the editor only appeared from stale browser cache. A small static loader (`web/js/editor-loader.js`, served with the helpers) now imports the editor entry from the pack's own route, which serves `index.js` with `no-cache` so the entry is always revalidated while its content-hashed chunks stay cacheable. After updating: restart ComfyUI and reload the page once; no cache-clearing gymnastics required.
|
- **Editor delivery fix**: A fresh browser could previously end up without the editor bundle entirely. ComfyUI core lets a pack's `WEB_DIRECTORY` overwrite its extension web-dir registration, so the browser's `/extensions` listing only ever contained this pack's helper scripts — never the editor entry — and the editor only appeared from stale browser cache. A small static loader (`web/js/editor-loader.js`, served with the helpers) now imports the editor entry from the pack's own route, which serves `index.js` with `no-cache` so the entry is always revalidated while its content-hashed chunks stay cacheable. After updating: restart ComfyUI and reload the page once; no cache-clearing gymnastics required.
|
||||||
- **MultiTrack Editor / MultiTrack Project**: **T2V resolves natively again** (`t2v` stays `t2v`, matching upstream) — the model now handles text-to-video with the updated prompting, so the earlier T2V→R2V normalization was removed from the editor type computation, `h3_task_type`, `_multitrack_task_type`, and the generation-mode mapping. The reference-mode-with-zero-media cloud fallback was removed accordingly (reference mode raises for missing media again, as upstream).
|
- **MultiTrack Editor / MultiTrack Project**: **T2V resolves natively again** (`t2v` stays `t2v`, matching upstream) — the model now handles text-to-video with the updated prompting, so the earlier T2V→R2V normalization was removed from the editor type computation, `h3_task_type`, `_multitrack_task_type`, and the generation-mode mapping. The reference-mode-with-zero-media cloud fallback was removed accordingly (reference mode raises for missing media again, as upstream).
|
||||||
|
|||||||
87
bench/README.md
Normal file
87
bench/README.md
Normal file
@ -0,0 +1,87 @@
|
|||||||
|
# Moxie Multimedia Bench
|
||||||
|
|
||||||
|
A standalone Python + web harness for the Moxie Multimedia pipeline — the
|
||||||
|
first step toward running this node pack outside ComfyUI as a sidecar for a
|
||||||
|
Tauri 2 desktop app.
|
||||||
|
|
||||||
|
The bench locks to the fixed pipeline of the base workflow
|
||||||
|
(`bench/base_workflow.json`):
|
||||||
|
|
||||||
|
```
|
||||||
|
MoxieMultimediaLoader -> LoraLoaderModelOnly -> MoxiePreviewOverride
|
||||||
|
-> easy multiTrackEditor -> MoxieProject -> MoxieCombine
|
||||||
|
-> [native branch] CreateVideo -> SaveVideo
|
||||||
|
-> [upscaled branch] RTXVideoSuperResolution -> CreateVideo -> SaveVideo
|
||||||
|
```
|
||||||
|
|
||||||
|
The multitrack timeline editor is the real thing: the bench bundles the same
|
||||||
|
React editor components the ComfyUI node uses, mounted by a thin shim
|
||||||
|
(`frontend/src/bench/`) that talks REST + WebSocket instead of the ComfyUI
|
||||||
|
runtime.
|
||||||
|
|
||||||
|
## Run from source
|
||||||
|
|
||||||
|
```bash
|
||||||
|
pip install -r bench/requirements.txt # fastapi, uvicorn, numpy, pillow
|
||||||
|
python bench/run.py --port 8787 # auto backend (mock here)
|
||||||
|
# open http://127.0.0.1:8787
|
||||||
|
```
|
||||||
|
|
||||||
|
The UI needs a one-time build (a prebuilt copy ships in `bench/ui-dist/`):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd frontend && bun install && bun run build:bench
|
||||||
|
```
|
||||||
|
|
||||||
|
## Backends
|
||||||
|
|
||||||
|
| backend | needs | behaviour |
|
||||||
|
|---|---|---|
|
||||||
|
| `mock` | ffmpeg, numpy, pillow | deterministic synthetic frames per (prompt, seed, frame), sine audio bed, lanczos "upscale" to the RTX target, h264 encode. Both output branches produce real mp4s. |
|
||||||
|
| `comfy` | a running ComfyUI + downloaded models | substitutes the plan into the base workflow, converts it to the `/prompt` API format, submits to ComfyUI, polls progress, downloads outputs. |
|
||||||
|
| `auto` | — | `comfy` when reachable, else `mock`. |
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python bench/run.py --backend comfy --comfyui-url http://127.0.0.1:8188
|
||||||
|
```
|
||||||
|
|
||||||
|
## HTTP API
|
||||||
|
|
||||||
|
- `GET /api/health`, `GET /api/config` — liveness, backend availability
|
||||||
|
- `GET /api/workflow` — base workflow summary + editor `track_data`
|
||||||
|
- `GET /api/plan` — resolved render plan (segments, prompts, frames)
|
||||||
|
- `POST /api/render` — queue a render:
|
||||||
|
`{"track_data": <editor JSON>, "overrides": {"width": 1920, "height": 1080,
|
||||||
|
"steps": 4, "seed": 123, "project_name": "MM-x", "backend": "auto"}}`
|
||||||
|
- `GET /api/jobs`, `GET /api/jobs/{id}`, `DELETE /api/jobs/{id}`
|
||||||
|
- `WS /api/ws` — push events: `job_queued`, `execution_start`, `job_progress`,
|
||||||
|
`execution_success`, `execution_error` (mirrors ComfyUI's push model)
|
||||||
|
- `GET /api/view?filename=&type=output&job=` — fetch rendered files
|
||||||
|
- ComfyUI-compatible aliases: `/view`, `/upload/image`, `/easy-media/*`
|
||||||
|
|
||||||
|
Outputs land in `bench/work/outputs/<job_id>/` (`MOXIE_BENCH_WORK` overrides
|
||||||
|
the root). Every render produces both branches: `native` (editor canvas) and
|
||||||
|
`upscaled` (RTX target dimensions).
|
||||||
|
|
||||||
|
## Tauri 2 sidecar
|
||||||
|
|
||||||
|
See `bench/tauri/README.md` — PyInstaller onedir packaging
|
||||||
|
(`bench/moxie_bench.spec`), `tauri.conf.json` resource wiring, spawn + health
|
||||||
|
poll snippet, and the endpoint table.
|
||||||
|
|
||||||
|
## Tests
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 -m pytest bench/tests -q # contract tests (no ComfyUI needed)
|
||||||
|
python3 scripts/test_bench_pipeline.py # full API + WS + mock render e2e
|
||||||
|
cd frontend && bun run test # editor suite incl. bench shim
|
||||||
|
```
|
||||||
|
|
||||||
|
## Phase map
|
||||||
|
|
||||||
|
- **phase 2 (this)** — bench: fixed pipeline, dual backend, timeline UI port,
|
||||||
|
sidecar packaging.
|
||||||
|
- **phase 3 (planned)** — in-process execution backend: load the pack's node
|
||||||
|
classes directly (torch + ComfyUI modules on `sys.path`) instead of driving
|
||||||
|
a separate ComfyUI server; `bench/app/backends/comfy.py` keeps the same
|
||||||
|
interface.
|
||||||
3
bench/app/__init__.py
Normal file
3
bench/app/__init__.py
Normal file
@ -0,0 +1,3 @@
|
|||||||
|
"""Moxie Multimedia bench."""
|
||||||
|
|
||||||
|
__version__ = "0.1.0"
|
||||||
19
bench/app/backends/__init__.py
Normal file
19
bench/app/backends/__init__.py
Normal file
@ -0,0 +1,19 @@
|
|||||||
|
"""Backend registry."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
|
||||||
|
def comfy_bridge_available(cfg) -> bool:
|
||||||
|
from . import comfy
|
||||||
|
|
||||||
|
return comfy.comfy_bridge_available(cfg)
|
||||||
|
|
||||||
|
|
||||||
|
def available(cfg) -> dict:
|
||||||
|
"""Probe both backends for /api/config."""
|
||||||
|
info = {"mock": {"available": True}, "comfy": {"available": False}}
|
||||||
|
try:
|
||||||
|
ok = comfy_bridge_available(cfg)
|
||||||
|
info["comfy"] = {"available": ok, "url": cfg.comfyui_url}
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
info["comfy"] = {"available": False, "error": str(exc)}
|
||||||
|
return info
|
||||||
351
bench/app/backends/comfy.py
Normal file
351
bench/app/backends/comfy.py
Normal file
@ -0,0 +1,351 @@
|
|||||||
|
"""ComfyUI bridge backend (real execution, phase-2 scope).
|
||||||
|
|
||||||
|
Drives a running ComfyUI over HTTP instead of reimplementing its executor
|
||||||
|
in-process:
|
||||||
|
|
||||||
|
1. take the base workflow (frontend export format, checked in at
|
||||||
|
bench/base_workflow.json),
|
||||||
|
2. substitute the RenderPlan parameters (track_data, resolution, steps,
|
||||||
|
seed, LoRA, upscale target, codecs, project_data),
|
||||||
|
3. convert to the /prompt API format using the input names carried by the
|
||||||
|
export itself (widget names via widgets_values_named, link inputs via
|
||||||
|
the node's inputs[] entries),
|
||||||
|
4. POST /prompt, poll /history/<id> for per-node progress and outputs,
|
||||||
|
5. download produced files via /view into the job dir.
|
||||||
|
|
||||||
|
Phase 3 replaces this with an in-process executor (same interface).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import json
|
||||||
|
import time
|
||||||
|
import urllib.error
|
||||||
|
import urllib.parse
|
||||||
|
import urllib.request
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from ..tracks import RenderPlan
|
||||||
|
from ..workflow import EDITOR_TYPE, PipelineSpec
|
||||||
|
from .mock import ProgressCb, RenderResult
|
||||||
|
|
||||||
|
_CONTROL_WIDGET_NAMES = {"randomize", "control_after_generate", "control_before_generate"}
|
||||||
|
|
||||||
|
|
||||||
|
class ComfyBridgeError(RuntimeError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# HTTP helpers (stdlib only)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _url(cfg_url: str, path: str, **params: Any) -> str:
|
||||||
|
base = cfg_url.rstrip("/")
|
||||||
|
url = f"{base}{path}"
|
||||||
|
if params:
|
||||||
|
url += "?" + urllib.parse.urlencode(params)
|
||||||
|
return url
|
||||||
|
|
||||||
|
|
||||||
|
def _get_json(url: str, timeout: float = 10.0) -> Any:
|
||||||
|
with urllib.request.urlopen(url, timeout=timeout) as resp:
|
||||||
|
return json.loads(resp.read().decode("utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def _post_json(url: str, payload: dict, timeout: float = 30.0) -> Any:
|
||||||
|
data = json.dumps(payload).encode("utf-8")
|
||||||
|
req = urllib.request.Request(
|
||||||
|
url, data=data, headers={"Content-Type": "application/json"}, method="POST"
|
||||||
|
)
|
||||||
|
with urllib.request.urlopen(req, timeout=timeout) as resp:
|
||||||
|
return json.loads(resp.read().decode("utf-8"))
|
||||||
|
|
||||||
|
|
||||||
|
def comfy_bridge_available(cfg) -> bool:
|
||||||
|
"""True when a ComfyUI answers at cfg.comfyui_url."""
|
||||||
|
try:
|
||||||
|
_get_json(_url(cfg.comfyui_url, "/system_stats"), timeout=3.0)
|
||||||
|
return True
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Workflow substitution + /prompt conversion
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
_RESOLUTION_PRESETS = {
|
||||||
|
(1920, 1080): "1920 x 1080 (16:9)",
|
||||||
|
(1088, 1920): "1088 x 1920 (9:16)",
|
||||||
|
(1080, 1920): "1080 x 1920 (9:16)",
|
||||||
|
(1280, 720): "1280 x 720 (16:9)",
|
||||||
|
(720, 1280): "720 x 1280 (9:16)",
|
||||||
|
(832, 480): "832 x 480 (16:9)",
|
||||||
|
(480, 832): "480 x 832 (9:16)",
|
||||||
|
(1024, 1024): "1024 x 1024 (1:1)",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def substitute(spec: PipelineSpec, plan: RenderPlan) -> dict[str, Any]:
|
||||||
|
"""Return a deep copy of the raw workflow with plan values applied."""
|
||||||
|
raw = copy.deepcopy(spec.raw)
|
||||||
|
nodes = {int(n["id"]): n for n in raw["nodes"]}
|
||||||
|
|
||||||
|
def named(nid: int) -> dict[str, Any]:
|
||||||
|
n = nodes[nid]
|
||||||
|
vals = n.setdefault("widgets_values_named", {})
|
||||||
|
vals.setdefault("widgets_values", None)
|
||||||
|
return vals
|
||||||
|
|
||||||
|
# editor: track_data + resolution
|
||||||
|
named(spec.editor_node_id)["track_data"] = json.dumps(
|
||||||
|
_track_data_for_submission(spec, plan)
|
||||||
|
)
|
||||||
|
preset = _RESOLUTION_PRESETS.get((plan.width, plan.height))
|
||||||
|
named(spec.editor_node_id)["resolution"] = (
|
||||||
|
preset if preset else {"width": plan.width, "height": plan.height}
|
||||||
|
)
|
||||||
|
|
||||||
|
# project: name, steps, seed
|
||||||
|
proj = spec.stages.get("MoxieProject")
|
||||||
|
if proj:
|
||||||
|
vals = named(proj.node_id)
|
||||||
|
vals["project_name"] = plan.project_name
|
||||||
|
vals["steps"] = plan.steps
|
||||||
|
vals["seed"] = plan.seed
|
||||||
|
|
||||||
|
# lora
|
||||||
|
lora = spec.stages.get("LoraLoaderModelOnly")
|
||||||
|
if lora and plan.lora:
|
||||||
|
vals = named(lora.node_id)
|
||||||
|
vals["lora_name"] = plan.lora
|
||||||
|
vals["strength_model"] = plan.lora_strength
|
||||||
|
|
||||||
|
# rtx upscale target
|
||||||
|
rtx = spec.stages.get("RTXVideoSuperResolution")
|
||||||
|
if rtx and plan.upscale_width:
|
||||||
|
vals = named(rtx.node_id)
|
||||||
|
vals["resize_type"] = "target dimensions"
|
||||||
|
vals["resize_type.width"] = plan.upscale_width
|
||||||
|
vals["resize_type.height"] = plan.upscale_height
|
||||||
|
vals["quality"] = plan.upscale_quality
|
||||||
|
|
||||||
|
# combine project_data carries the canvas + frame rate
|
||||||
|
combine = spec.stages.get("MoxieCombine")
|
||||||
|
if combine:
|
||||||
|
vals = named(combine.node_id)
|
||||||
|
pd_raw = vals.get("project_data", "{}")
|
||||||
|
pd = json.loads(pd_raw) if isinstance(pd_raw, str) else dict(pd_raw or {})
|
||||||
|
pd.update({
|
||||||
|
"project_name": plan.project_name,
|
||||||
|
"width": plan.width,
|
||||||
|
"height": plan.height,
|
||||||
|
"frame_rate": plan.frame_rate,
|
||||||
|
})
|
||||||
|
vals["project_data"] = json.dumps(pd)
|
||||||
|
|
||||||
|
# save prefixes get a per-run folder
|
||||||
|
for branch, stage in spec.save_nodes.items():
|
||||||
|
vals = named(stage.node_id)
|
||||||
|
if "filename_prefix" in vals:
|
||||||
|
vals["filename_prefix"] = f"moxie-bench/{branch}"
|
||||||
|
|
||||||
|
return raw
|
||||||
|
|
||||||
|
|
||||||
|
def _track_data_for_submission(spec: PipelineSpec, plan: RenderPlan) -> dict[str, Any]:
|
||||||
|
"""Track data as edited in the bench UI; the editor widget value wins."""
|
||||||
|
raw = spec.widget(EDITOR_TYPE, "track_data")
|
||||||
|
data = json.loads(raw) if isinstance(raw, str) else dict(raw or {})
|
||||||
|
data["frame_rate"] = plan.frame_rate
|
||||||
|
data.setdefault("total_length", plan.total_length)
|
||||||
|
return data
|
||||||
|
|
||||||
|
|
||||||
|
def to_prompt_format(raw: dict[str, Any]) -> dict[str, dict[str, Any]]:
|
||||||
|
"""Frontend-format workflow -> /prompt API format.
|
||||||
|
|
||||||
|
Widget values come from widgets_values_named (names are the schema input
|
||||||
|
names); link inputs come from the node's inputs[] entries. A widget that
|
||||||
|
is wired to a link (marker key "widget" plus a link id) takes the link.
|
||||||
|
"""
|
||||||
|
nodes = raw.get("nodes") or []
|
||||||
|
links: dict[int, tuple[int, int]] = {}
|
||||||
|
for lk in raw.get("links") or []:
|
||||||
|
links[int(lk[0])] = (int(lk[1]), int(lk[2]))
|
||||||
|
|
||||||
|
prompt: dict[str, dict[str, Any]] = {}
|
||||||
|
for n in nodes:
|
||||||
|
nid = str(int(n["id"]))
|
||||||
|
entry: dict[str, Any] = {}
|
||||||
|
for inp in n.get("inputs") or []:
|
||||||
|
if not isinstance(inp, dict):
|
||||||
|
continue
|
||||||
|
name = inp.get("name")
|
||||||
|
link = inp.get("link")
|
||||||
|
if name is None or link is None:
|
||||||
|
continue
|
||||||
|
frm = links.get(int(link))
|
||||||
|
if frm is None:
|
||||||
|
continue
|
||||||
|
entry[name] = [str(frm[0]), frm[1]]
|
||||||
|
named = n.get("widgets_values_named") or {}
|
||||||
|
linked_widget_names = {
|
||||||
|
inp.get("widget", {}).get("name")
|
||||||
|
for inp in n.get("inputs") or []
|
||||||
|
if isinstance(inp, dict) and inp.get("link") is not None
|
||||||
|
and isinstance(inp.get("widget"), dict)
|
||||||
|
}
|
||||||
|
for name, value in named.items():
|
||||||
|
if name in ("widgets_values",) or name in _CONTROL_WIDGET_NAMES:
|
||||||
|
continue
|
||||||
|
if name in linked_widget_names and name in entry:
|
||||||
|
continue # wired input wins over the stored widget value
|
||||||
|
entry[name] = value
|
||||||
|
prompt[nid] = {"class_type": n.get("type"), "inputs": entry}
|
||||||
|
return prompt
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# Render
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def render(
|
||||||
|
spec: PipelineSpec,
|
||||||
|
plan: RenderPlan,
|
||||||
|
cfg,
|
||||||
|
job_dir: Path,
|
||||||
|
progress: ProgressCb,
|
||||||
|
) -> RenderResult:
|
||||||
|
result = RenderResult()
|
||||||
|
raw = substitute(spec, plan)
|
||||||
|
prompt = to_prompt_format(raw)
|
||||||
|
node_types = {str(int(n["id"])): n.get("type", "") for n in raw["nodes"]}
|
||||||
|
|
||||||
|
progress("MoxieMultimediaLoader", 5.0, f"comfy: submitting prompt to {cfg.comfyui_url}")
|
||||||
|
try:
|
||||||
|
resp = _post_json(_url(cfg.comfyui_url, "/prompt"), {"prompt": prompt})
|
||||||
|
except urllib.error.HTTPError as exc:
|
||||||
|
body = exc.read().decode("utf-8", errors="replace")
|
||||||
|
raise ComfyBridgeError(f"ComfyUI rejected the prompt ({exc.code}): {body[:600]}") from exc
|
||||||
|
except urllib.error.URLError as exc:
|
||||||
|
raise ComfyBridgeError(
|
||||||
|
f"Cannot reach ComfyUI at {cfg.comfyui_url} ({exc.reason}). "
|
||||||
|
"Start it with: python main.py --listen 127.0.0.1 --port 8188"
|
||||||
|
) from exc
|
||||||
|
prompt_id = resp.get("prompt_id")
|
||||||
|
if not prompt_id:
|
||||||
|
raise ComfyBridgeError(f"ComfyUI returned no prompt_id: {resp}")
|
||||||
|
|
||||||
|
progress("MoxieProject", 10.0, f"comfy: queued ({prompt_id})")
|
||||||
|
|
||||||
|
stage_by_node = {str(nid): ntype for nid, ntype in node_types.items()}
|
||||||
|
save_branches = {str(st.node_id): branch for branch, st in spec.save_nodes.items()}
|
||||||
|
|
||||||
|
deadline = time.time() + 60 * 60 # 1h safety stop
|
||||||
|
poll = 1.0
|
||||||
|
seen_progress_max = 0
|
||||||
|
while time.time() < deadline:
|
||||||
|
time.sleep(poll)
|
||||||
|
try:
|
||||||
|
hist = _get_json(_url(cfg.comfyui_url, f"/history/{prompt_id}"), timeout=15.0)
|
||||||
|
except Exception as exc: # transient network hiccups
|
||||||
|
result.log(f"history poll failed: {exc}")
|
||||||
|
continue
|
||||||
|
entry = hist.get(prompt_id)
|
||||||
|
if entry is None:
|
||||||
|
# still queued or running
|
||||||
|
continue
|
||||||
|
status = entry.get("status") or {}
|
||||||
|
messages = status.get("messages") or []
|
||||||
|
for msg in messages:
|
||||||
|
mtype, mdata = msg[0], (msg[1] if len(msg) > 1 else {})
|
||||||
|
if mtype == "executing" and mdata.get("node"):
|
||||||
|
ntype = stage_by_node.get(str(mdata["node"]), "")
|
||||||
|
if ntype:
|
||||||
|
progress(ntype, max(12.0, result_progress(ntype)), f"comfy: {ntype}")
|
||||||
|
elif mtype == "progress":
|
||||||
|
try:
|
||||||
|
val = float(mdata.get("value", 0)) / max(1.0, float(mdata.get("max", 1)))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
val = 0.0
|
||||||
|
seen_progress_max = max(seen_progress_max, val)
|
||||||
|
# sample/VAE progress mapped into the 15..92 window
|
||||||
|
progress("MoxieProject", 15.0 + 70.0 * val,
|
||||||
|
f"comfy: sampling {int(val * 100)}%",
|
||||||
|
{"step": mdata.get("value"), "steps": mdata.get("max")})
|
||||||
|
elif mtype == "execution_error":
|
||||||
|
raise ComfyBridgeError(
|
||||||
|
f"ComfyUI execution failed on node "
|
||||||
|
f"{mdata.get('node_type')} ({mdata.get('node_id')}): "
|
||||||
|
f"{mdata.get('exception_message')}"
|
||||||
|
)
|
||||||
|
if status.get("completed"):
|
||||||
|
break
|
||||||
|
if status.get("status_str") == "error":
|
||||||
|
raise ComfyBridgeError("ComfyUI reported an error for this prompt")
|
||||||
|
|
||||||
|
if not status.get("completed"):
|
||||||
|
raise ComfyBridgeError("ComfyUI did not finish in time")
|
||||||
|
|
||||||
|
# collect outputs
|
||||||
|
outputs = entry.get("outputs") or {}
|
||||||
|
files: dict[str, dict[str, Any]] = {}
|
||||||
|
for nid, node_out in outputs.items():
|
||||||
|
branch = save_branches.get(str(nid))
|
||||||
|
for key, items in node_out.items():
|
||||||
|
if not isinstance(items, list):
|
||||||
|
continue
|
||||||
|
for item in items:
|
||||||
|
if not isinstance(item, dict) or "filename" not in item:
|
||||||
|
continue
|
||||||
|
if branch:
|
||||||
|
files.setdefault(branch, dict(item))
|
||||||
|
else:
|
||||||
|
result.log(f"aux output from node {nid}: {key} {item.get('filename')}")
|
||||||
|
|
||||||
|
if not files:
|
||||||
|
raise ComfyBridgeError(f"ComfyUI produced no video outputs: outputs={list(outputs)}")
|
||||||
|
|
||||||
|
for branch, item in files.items():
|
||||||
|
params = {
|
||||||
|
"filename": item["filename"],
|
||||||
|
"subfolder": item.get("subfolder", ""),
|
||||||
|
"type": item.get("type", "output"),
|
||||||
|
}
|
||||||
|
data = _download(_url(cfg.comfyui_url, "/view", **params))
|
||||||
|
dst = job_dir / f"{branch}{Path(item['filename']).suffix or '.mp4'}"
|
||||||
|
dst.write_bytes(data)
|
||||||
|
result.outputs[branch] = {
|
||||||
|
"filename": dst.name,
|
||||||
|
"path": str(dst),
|
||||||
|
"url": f"/api/view?filename={dst.name}&type=output&job={job_dir.name}",
|
||||||
|
"source": item.get("subfolder", ""),
|
||||||
|
"bytes": len(data),
|
||||||
|
}
|
||||||
|
result.log(f"{branch}: {item['filename']} ({len(data)} bytes)")
|
||||||
|
|
||||||
|
progress("SaveVideo", 100.0, "done", {"outputs": result.outputs})
|
||||||
|
return result
|
||||||
|
|
||||||
|
|
||||||
|
def result_progress(stage: str) -> float:
|
||||||
|
order = {
|
||||||
|
"MoxieMultimediaLoader": 8.0,
|
||||||
|
"LoraLoaderModelOnly": 10.0,
|
||||||
|
"MoxiePreviewOverride": 12.0,
|
||||||
|
EDITOR_TYPE: 14.0,
|
||||||
|
"MoxieProject": 40.0,
|
||||||
|
"MoxieCombine": 78.0,
|
||||||
|
"RTXVideoSuperResolution": 85.0,
|
||||||
|
"CreateVideo": 92.0,
|
||||||
|
"SaveVideo": 97.0,
|
||||||
|
}
|
||||||
|
return order.get(stage, 20.0)
|
||||||
|
|
||||||
|
|
||||||
|
def _download(url: str, timeout: float = 120.0) -> bytes:
|
||||||
|
with urllib.request.urlopen(url, timeout=timeout) as resp:
|
||||||
|
return resp.read()
|
||||||
324
bench/app/backends/mock.py
Normal file
324
bench/app/backends/mock.py
Normal file
@ -0,0 +1,324 @@
|
|||||||
|
"""Self-contained mock backend.
|
||||||
|
|
||||||
|
Renders the fixed pipeline without torch/ComfyUI: deterministic synthetic
|
||||||
|
frames derived from (prompt, seed, frame index), a sine audio bed, PIL
|
||||||
|
lanczos "upscale" for the RTX branch, and ffmpeg h264 encode. Exercises the
|
||||||
|
exact same stage/progress contract as the ComfyUI bridge backend so the UI
|
||||||
|
and the API cannot tell the difference.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import math
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
import time
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
from ..tracks import RenderPlan, Segment
|
||||||
|
|
||||||
|
ProgressCb = Callable[[str, float, str, dict | None], None]
|
||||||
|
|
||||||
|
_VIDEO_CODECS = ("h264", "libx264", "mpeg4")
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RenderResult:
|
||||||
|
outputs: dict[str, dict] = field(default_factory=dict)
|
||||||
|
console: list[str] = field(default_factory=list)
|
||||||
|
|
||||||
|
def log(self, line: str) -> None:
|
||||||
|
self.console.append(line)
|
||||||
|
|
||||||
|
|
||||||
|
def _pick_encoder(codec: str) -> str:
|
||||||
|
probe = subprocess.run(
|
||||||
|
["ffmpeg", "-hide_banner", "-encoders"],
|
||||||
|
capture_output=True, text=True, timeout=30,
|
||||||
|
)
|
||||||
|
names = probe.stdout.lower()
|
||||||
|
if codec == "h264" and "libx264" in names:
|
||||||
|
return "libx264"
|
||||||
|
for cand in _VIDEO_CODECS:
|
||||||
|
if cand in names:
|
||||||
|
return cand
|
||||||
|
return "mpeg4"
|
||||||
|
|
||||||
|
|
||||||
|
def _audio_encoder() -> str:
|
||||||
|
probe = subprocess.run(
|
||||||
|
["ffmpeg", "-hide_banner", "-encoders"],
|
||||||
|
capture_output=True, text=True, timeout=30,
|
||||||
|
)
|
||||||
|
return "aac" if " aac" in probe.stdout.lower() else "libmp3lame"
|
||||||
|
|
||||||
|
|
||||||
|
def _segment_hue(segment: Segment, index: int) -> float:
|
||||||
|
digest = hashlib.sha256(f"{segment.prompt}|{index}".encode()).digest()
|
||||||
|
return digest[0] * 360.0 / 255.0
|
||||||
|
|
||||||
|
|
||||||
|
def _segment_freq(segment: Segment, index: int) -> float:
|
||||||
|
digest = hashlib.sha256(f"{segment.prompt}|{index}".encode()).digest()
|
||||||
|
return 280.0 + digest[1] % 5 * 55.0 # 280..500 Hz
|
||||||
|
|
||||||
|
|
||||||
|
def _seed_noise(seed: int, frame: int) -> float:
|
||||||
|
digest = hashlib.sha256(f"{seed}:{frame}".encode()).digest()
|
||||||
|
return digest[0] / 255.0 - 0.5
|
||||||
|
|
||||||
|
|
||||||
|
def _synthesize_chunk(
|
||||||
|
chunk_start: int,
|
||||||
|
frames: int,
|
||||||
|
plan: RenderPlan,
|
||||||
|
w: int,
|
||||||
|
h: int,
|
||||||
|
) -> bytes:
|
||||||
|
"""Deterministic RGB24 bytes for frames [chunk_start, chunk_start+frames)."""
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
out = bytearray()
|
||||||
|
yy, xx = np.mgrid[0:h, 0:w].astype(np.float32)
|
||||||
|
xn = xx / max(1, w - 1)
|
||||||
|
yn = yy / max(1, h - 1)
|
||||||
|
seg_by_frame = _frame_to_segment_map(plan)
|
||||||
|
for f in range(chunk_start, chunk_start + frames):
|
||||||
|
seg, seg_index = seg_by_frame.get(f, (None, 0))
|
||||||
|
hue = _segment_hue(seg, seg_index) if seg else 210.0
|
||||||
|
t = f / max(1, plan.frame_rate)
|
||||||
|
# HSV-ish deterministic palette: hue rotates per segment prompt,
|
||||||
|
# luminance carries a moving wave so motion is visible.
|
||||||
|
base = np.zeros((h, w, 3), dtype=np.float32)
|
||||||
|
h1 = hue % 360.0
|
||||||
|
rgb = _hsv_to_rgb(h1 / 360.0, 0.55, 0.6)
|
||||||
|
wave = 0.25 * np.sin(6.0 * math.pi * (xn + 0.35 * t)) + 0.15 * np.sin(
|
||||||
|
4.0 * math.pi * (yn - 0.25 * t)
|
||||||
|
)
|
||||||
|
lum = 0.45 + wave + 0.06 * _seed_noise(plan.seed, f)
|
||||||
|
base[..., 0] = rgb[0] * lum
|
||||||
|
base[..., 1] = rgb[1] * lum
|
||||||
|
base[..., 2] = rgb[2] * lum
|
||||||
|
# progress bar strip at the bottom: frame position inside the timeline
|
||||||
|
pos = f / max(1, plan.total_length or 1)
|
||||||
|
bar_y = int(h * 0.95)
|
||||||
|
base[bar_y:, : int(w * pos), 1] = 0.95
|
||||||
|
out += (
|
||||||
|
np.clip(base * 255.0, 0, 255).astype(np.uint8).tobytes()
|
||||||
|
)
|
||||||
|
return bytes(out)
|
||||||
|
|
||||||
|
|
||||||
|
def _frame_to_segment_map(plan: RenderPlan) -> dict[int, tuple[Segment, int]]:
|
||||||
|
mapping: dict[int, tuple[Segment, int]] = {}
|
||||||
|
for idx, seg in enumerate(plan.segments):
|
||||||
|
for f in range(seg.start_frame, seg.end_frame):
|
||||||
|
mapping[f] = (seg, idx)
|
||||||
|
return mapping
|
||||||
|
|
||||||
|
|
||||||
|
def _hsv_to_rgb(h: float, s: float, v: float) -> tuple[float, float, float]:
|
||||||
|
import colorsys
|
||||||
|
|
||||||
|
r, g, b = colorsys.hsv_to_rgb(h % 1.0, s, v)
|
||||||
|
return r, g, b
|
||||||
|
|
||||||
|
|
||||||
|
def _s16le_audio(plan: RenderPlan, sample_rate: int = 44100) -> bytes:
|
||||||
|
"""Stereo s16le sine bed; per-segment tone, silence for muted segments."""
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
total = int(sample_rate * plan.duration_seconds)
|
||||||
|
if total <= 0:
|
||||||
|
return b""
|
||||||
|
wave = np.zeros(total, dtype=np.float32)
|
||||||
|
sr = sample_rate
|
||||||
|
for idx, seg in enumerate(plan.segments):
|
||||||
|
if seg.muted or not seg.prompt:
|
||||||
|
continue
|
||||||
|
a, b = seg.start_frame, seg.end_frame
|
||||||
|
i0 = int(a / max(1, plan.frame_rate) * sr)
|
||||||
|
i1 = min(total, int(b / max(1, plan.frame_rate) * sr))
|
||||||
|
if i1 <= i0:
|
||||||
|
continue
|
||||||
|
t = np.arange(i1 - i0, dtype=np.float32) / sr
|
||||||
|
freq = _segment_freq(seg, idx)
|
||||||
|
gain = 0.12 * (10 ** (seg.volume_db / 20.0))
|
||||||
|
wave[i0:i1] += gain * np.sin(2 * math.pi * freq * t)
|
||||||
|
stereo = np.repeat(np.clip(wave, -1, 1)[:, None], 2, axis=1)
|
||||||
|
return (stereo * 32767.0).astype("<i2").tobytes()
|
||||||
|
|
||||||
|
|
||||||
|
def render(plan: RenderPlan, work_dir: Path, progress: ProgressCb) -> RenderResult:
|
||||||
|
"""Run the mock pipeline. Returns native + upscaled output descriptors."""
|
||||||
|
import numpy as np
|
||||||
|
from PIL import Image
|
||||||
|
|
||||||
|
result = RenderResult()
|
||||||
|
t0 = time.time()
|
||||||
|
work_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
if shutil.which("ffmpeg") is None:
|
||||||
|
raise RuntimeError("ffmpeg not found on PATH - the mock backend needs it")
|
||||||
|
|
||||||
|
w, h = plan.width, plan.height
|
||||||
|
frames = plan.total_length or sum(s.frame_count for s in plan.segments)
|
||||||
|
if frames <= 0:
|
||||||
|
raise RuntimeError("timeline has no frames to render")
|
||||||
|
|
||||||
|
def stage(name: str, pct: float, msg: str, data: dict | None = None) -> None:
|
||||||
|
progress(name, pct, msg, data)
|
||||||
|
|
||||||
|
# stages 1-3: loader / lora / preview (contract parity with the real chain)
|
||||||
|
stage("MoxieMultimediaLoader", 4.0, "mock: loading Moxie-Multimedia model set")
|
||||||
|
stage("LoraLoaderModelOnly", 9.0, f"mock: LoRA {plan.lora or 'none'} @ {plan.lora_strength}")
|
||||||
|
stage("MoxiePreviewOverride", 12.0, "mock: preview override active")
|
||||||
|
|
||||||
|
stage("easy multiTrackEditor", 15.0,
|
||||||
|
f"mock: timeline {w}x{h} @ {plan.frame_rate}fps, "
|
||||||
|
f"{len(plan.segments)} task segment(s)")
|
||||||
|
|
||||||
|
encoder = _pick_encoder(plan.codec)
|
||||||
|
aenc = _audio_encoder()
|
||||||
|
result.log(f"encoder: {encoder}, audio: {aenc}")
|
||||||
|
|
||||||
|
native_path = work_dir / "native.mp4"
|
||||||
|
upscaled_path = work_dir / "upscaled.mp4"
|
||||||
|
do_upscale = bool(plan.upscale_width and plan.upscale_height)
|
||||||
|
|
||||||
|
audio_bytes = _s16le_audio(plan)
|
||||||
|
|
||||||
|
def open_enc(out_path: Path, ow: int, oh: int) -> subprocess.Popen:
|
||||||
|
import os as _os
|
||||||
|
|
||||||
|
cmd = [
|
||||||
|
"ffmpeg", "-hide_banner", "-loglevel", "error", "-y",
|
||||||
|
"-f", "rawvideo", "-pix_fmt", "rgb24",
|
||||||
|
"-s", f"{ow}x{oh}", "-r", str(plan.frame_rate), "-i", "pipe:0",
|
||||||
|
"-f", "s16le", "-ar", "44100", "-ac", "2",
|
||||||
|
]
|
||||||
|
r, w = _os.pipe()
|
||||||
|
# give ffmpeg the READ end (r); parent writes audio into w
|
||||||
|
cmd += ["-i", f"/dev/fd/{r}", "-map", "0:v", "-map", "1:a",
|
||||||
|
"-c:v", encoder, "-pix_fmt", "yuv420p",
|
||||||
|
"-movflags", "+faststart",
|
||||||
|
"-c:a", aenc, "-b:a", "128k", "-shortest",
|
||||||
|
str(out_path)]
|
||||||
|
proc = subprocess.Popen(
|
||||||
|
cmd, stdin=subprocess.PIPE, stdout=subprocess.DEVNULL,
|
||||||
|
stderr=subprocess.PIPE, pass_fds=(r,),
|
||||||
|
)
|
||||||
|
proc._audio_wfd = w # type: ignore[attr-defined]
|
||||||
|
_os.close(r)
|
||||||
|
return proc
|
||||||
|
|
||||||
|
def feed_audio(proc: subprocess.Popen) -> None:
|
||||||
|
# Audio may exceed the 64KB pipe buffer; pump it from a thread so the
|
||||||
|
# video stdin feed never deadlocks against ffmpeg's startup probing.
|
||||||
|
import os as _os
|
||||||
|
import threading as _threading
|
||||||
|
|
||||||
|
def _pump() -> None:
|
||||||
|
try:
|
||||||
|
view = memoryview(audio_bytes)
|
||||||
|
while view:
|
||||||
|
n = _os.write(proc._audio_wfd, view[:65536]) # type: ignore[attr-defined]
|
||||||
|
view = view[n:]
|
||||||
|
except (BrokenPipeError, OSError):
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
try:
|
||||||
|
_os.close(proc._audio_wfd) # type: ignore[attr-defined]
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
_threading.Thread(target=_pump, daemon=True).start()
|
||||||
|
|
||||||
|
seg_total = sum(s.frame_count for s in plan.segments)
|
||||||
|
seg_done = 0
|
||||||
|
chunk = 24
|
||||||
|
|
||||||
|
native_enc = open_enc(native_path, w, h)
|
||||||
|
upscale_enc = open_enc(upscaled_path, plan.upscale_width, plan.upscale_height) if do_upscale else None
|
||||||
|
|
||||||
|
def _enc_err(proc: subprocess.Popen) -> str:
|
||||||
|
try:
|
||||||
|
return proc.stderr.read().decode(errors="replace")[-500:]
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
return "(stderr unavailable)"
|
||||||
|
|
||||||
|
try:
|
||||||
|
feed_audio(native_enc)
|
||||||
|
if upscale_enc:
|
||||||
|
feed_audio(upscale_enc)
|
||||||
|
|
||||||
|
f = 0
|
||||||
|
while f < frames:
|
||||||
|
n = min(chunk, frames - f)
|
||||||
|
raw = _synthesize_chunk(f, n, plan, w, h)
|
||||||
|
try:
|
||||||
|
native_enc.stdin.write(raw)
|
||||||
|
except BrokenPipeError as exc:
|
||||||
|
raise RuntimeError(f"native encoder died: {_enc_err(native_enc)}") from exc
|
||||||
|
if upscale_enc:
|
||||||
|
# RTX-branch stand-in: lanczos resize per frame to target dims
|
||||||
|
arr = np.frombuffer(raw, dtype=np.uint8).reshape(n, h, w, 3)
|
||||||
|
imgs = [Image.fromarray(fr).resize(
|
||||||
|
(plan.upscale_width, plan.upscale_height), Image.LANCZOS)
|
||||||
|
for fr in arr]
|
||||||
|
try:
|
||||||
|
upscale_enc.stdin.write(np.stack([np.asarray(im) for im in imgs]).tobytes())
|
||||||
|
except BrokenPipeError as exc:
|
||||||
|
raise RuntimeError(f"upscale encoder died: {_enc_err(upscale_enc)}") from exc
|
||||||
|
f += n
|
||||||
|
seg_done += n
|
||||||
|
pct = 15.0 + 60.0 * seg_done / seg_total
|
||||||
|
# per-stage sub-progress: emulate per-segment sampling steps
|
||||||
|
cur = next((s for s in plan.segments if s.start_frame <= f < s.end_frame), None)
|
||||||
|
stage("MoxieProject", pct,
|
||||||
|
f"mock: sampling frame {f}/{frames}"
|
||||||
|
+ (f" ({cur.prompt or 'untitled'})" if cur else ""),
|
||||||
|
{"frame": f, "frames": frames})
|
||||||
|
native_enc.stdin.close()
|
||||||
|
if upscale_enc:
|
||||||
|
upscale_enc.stdin.close()
|
||||||
|
rc = native_enc.wait(timeout=120)
|
||||||
|
if rc != 0:
|
||||||
|
raise RuntimeError(f"native encode failed: {native_enc.stderr.read().decode(errors='replace')[-400:]}")
|
||||||
|
if upscale_enc:
|
||||||
|
rc = upscale_enc.wait(timeout=180)
|
||||||
|
if rc != 0:
|
||||||
|
raise RuntimeError(f"upscale encode failed: {upscale_enc.stderr.read().decode(errors='replace')[-400:]}")
|
||||||
|
finally:
|
||||||
|
for proc in (native_enc, upscale_enc):
|
||||||
|
if proc and proc.poll() is None:
|
||||||
|
proc.kill()
|
||||||
|
|
||||||
|
stage("MoxieCombine", 78.0, "mock: combined images + audio + fps")
|
||||||
|
if do_upscale:
|
||||||
|
stage("RTXVideoSuperResolution", 88.0,
|
||||||
|
f"mock: upscale {w}x{h} -> {plan.upscale_width}x{plan.upscale_height} "
|
||||||
|
f"({plan.upscale_quality})")
|
||||||
|
|
||||||
|
stage("CreateVideo", 94.0, f"mock: encode {plan.codec}")
|
||||||
|
stage("SaveVideo", 99.0, "mock: saved outputs")
|
||||||
|
|
||||||
|
def describe(path: Path, ow: int, oh: int) -> dict:
|
||||||
|
return {
|
||||||
|
"filename": path.name,
|
||||||
|
"path": str(path),
|
||||||
|
"url": f"/api/view?filename={path.name}&type=output",
|
||||||
|
"width": ow, "height": oh,
|
||||||
|
"fps": plan.frame_rate,
|
||||||
|
"frames": frames,
|
||||||
|
"bytes": path.stat().st_size if path.exists() else 0,
|
||||||
|
}
|
||||||
|
|
||||||
|
result.outputs["native"] = describe(native_path, w, h)
|
||||||
|
if do_upscale:
|
||||||
|
result.outputs["upscaled"] = describe(
|
||||||
|
upscaled_path, plan.upscale_width, plan.upscale_height)
|
||||||
|
result.log(f"rendered {frames} frames in {time.time() - t0:.1f}s")
|
||||||
|
stage("SaveVideo", 100.0, "done", {"outputs": result.outputs})
|
||||||
|
return result
|
||||||
113
bench/app/config.py
Normal file
113
bench/app/config.py
Normal file
@ -0,0 +1,113 @@
|
|||||||
|
"""Bench configuration: paths and runtime options.
|
||||||
|
|
||||||
|
Everything is derived from --flags / environment so the same code runs from
|
||||||
|
source (repo checkout) and from a PyInstaller bundle (Tauri sidecar).
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def _bundle_root() -> Path:
|
||||||
|
"""Repo root / frozen asset root that contains bench/ assets."""
|
||||||
|
if getattr(sys, "frozen", False): # PyInstaller (onedir: assets in _internal)
|
||||||
|
return Path(getattr(sys, "_MEIPASS", Path(sys.executable).resolve().parent))
|
||||||
|
return Path(__file__).resolve().parent.parent.parent
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class BenchConfig:
|
||||||
|
host: str = "127.0.0.1"
|
||||||
|
port: int = 8787
|
||||||
|
backend: str = "auto" # auto | mock | comfy
|
||||||
|
repo_root: Path = field(default_factory=_bundle_root)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def bench_dir(self) -> Path:
|
||||||
|
return self.repo_root / "bench"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def base_workflow_path(self) -> Path:
|
||||||
|
return self.bench_dir / "base_workflow.json"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def ui_dist(self) -> Path:
|
||||||
|
return self.bench_dir / "ui-dist"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def work_dir(self) -> Path:
|
||||||
|
default = (
|
||||||
|
# frozen sidecar: keep renders next to the binary, not in _internal
|
||||||
|
Path(sys.executable).resolve().parent / "work"
|
||||||
|
if getattr(sys, "frozen", False)
|
||||||
|
else self.repo_root / "bench" / "work"
|
||||||
|
)
|
||||||
|
d = Path(os.environ.get("MOXIE_BENCH_WORK", str(default)))
|
||||||
|
d.mkdir(parents=True, exist_ok=True)
|
||||||
|
return d
|
||||||
|
|
||||||
|
@property
|
||||||
|
def outputs_dir(self) -> Path:
|
||||||
|
d = self.work_dir / "outputs"
|
||||||
|
d.mkdir(parents=True, exist_ok=True)
|
||||||
|
return d
|
||||||
|
|
||||||
|
@property
|
||||||
|
def inputs_dir(self) -> Path:
|
||||||
|
d = self.work_dir / "inputs"
|
||||||
|
d.mkdir(parents=True, exist_ok=True)
|
||||||
|
return d
|
||||||
|
|
||||||
|
@property
|
||||||
|
def temp_dir(self) -> Path:
|
||||||
|
d = self.work_dir / "temp"
|
||||||
|
d.mkdir(parents=True, exist_ok=True)
|
||||||
|
return d
|
||||||
|
|
||||||
|
@property
|
||||||
|
def pack_models_dir(self) -> Path:
|
||||||
|
return self.repo_root / "models"
|
||||||
|
|
||||||
|
# --- Comfy bridge backend ---
|
||||||
|
comfyui_url: str = os.environ.get("MOXIE_BENCH_COMFYUI_URL", "http://127.0.0.1:8188")
|
||||||
|
|
||||||
|
def resolve_backend(self) -> str:
|
||||||
|
"""auto: comfy when torch + ComfyUI reachable, else mock."""
|
||||||
|
if self.backend != "auto":
|
||||||
|
return self.backend
|
||||||
|
from .backends import comfy_bridge_available
|
||||||
|
|
||||||
|
return "comfy" if comfy_bridge_available(self) else "mock"
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def from_args(cls, argv: list[str] | None = None) -> "BenchConfig":
|
||||||
|
p = argparse.ArgumentParser(prog="moxie-bench")
|
||||||
|
p.add_argument("--host", default=cls.host)
|
||||||
|
p.add_argument("--port", type=int, default=cls.port)
|
||||||
|
p.add_argument(
|
||||||
|
"--backend",
|
||||||
|
choices=["auto", "mock", "comfy"],
|
||||||
|
default=cls.backend,
|
||||||
|
help="mock = self-contained synthetic renderer; comfy = drive a "
|
||||||
|
"running ComfyUI at --comfyui-url with the base workflow",
|
||||||
|
)
|
||||||
|
p.add_argument("--comfyui-url", default=cls.comfyui_url)
|
||||||
|
p.add_argument(
|
||||||
|
"--repo-root",
|
||||||
|
default=None,
|
||||||
|
help="Root containing bench/ and models/ (default: auto-detect)",
|
||||||
|
)
|
||||||
|
args = p.parse_args(argv)
|
||||||
|
cfg = cls(
|
||||||
|
host=args.host,
|
||||||
|
port=args.port,
|
||||||
|
backend=args.backend,
|
||||||
|
comfyui_url=args.comfyui_url,
|
||||||
|
)
|
||||||
|
if args.repo_root:
|
||||||
|
cfg.repo_root = Path(args.repo_root).resolve()
|
||||||
|
return cfg
|
||||||
118
bench/app/jobs.py
Normal file
118
bench/app/jobs.py
Normal file
@ -0,0 +1,118 @@
|
|||||||
|
"""Render job queue: one worker thread, progress events, WS broadcast."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import threading
|
||||||
|
import time
|
||||||
|
import uuid
|
||||||
|
from concurrent.futures import ThreadPoolExecutor
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Job:
|
||||||
|
id: str
|
||||||
|
created_at: float = field(default_factory=time.time)
|
||||||
|
status: str = "queued" # queued | running | done | error
|
||||||
|
backend: str = "mock"
|
||||||
|
stage: str = ""
|
||||||
|
progress: float = 0.0
|
||||||
|
message: str = "queued"
|
||||||
|
result: dict[str, Any] | None = None
|
||||||
|
error: str | None = None
|
||||||
|
events: list[dict[str, Any]] = field(default_factory=list)
|
||||||
|
plan: dict[str, Any] | None = None
|
||||||
|
|
||||||
|
def snapshot(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"id": self.id,
|
||||||
|
"status": self.status,
|
||||||
|
"backend": self.backend,
|
||||||
|
"stage": self.stage,
|
||||||
|
"progress": round(self.progress, 1),
|
||||||
|
"message": self.message,
|
||||||
|
"result": self.result,
|
||||||
|
"error": self.error,
|
||||||
|
"plan": self.plan,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
BroadcastFn = Callable[[dict[str, Any]], None]
|
||||||
|
|
||||||
|
|
||||||
|
class JobManager:
|
||||||
|
"""Serial render queue. Events mirror ComfyUI's push model so the UI shim
|
||||||
|
can forward them onto app.api as execution events."""
|
||||||
|
|
||||||
|
def __init__(self, broadcast: BroadcastFn, max_workers: int = 1) -> None:
|
||||||
|
self._pool = ThreadPoolExecutor(max_workers=max_workers)
|
||||||
|
self._jobs: dict[str, Job] = {}
|
||||||
|
self._order: list[str] = []
|
||||||
|
self._lock = threading.Lock()
|
||||||
|
self._broadcast = broadcast
|
||||||
|
|
||||||
|
# -- public -------------------------------------------------------------
|
||||||
|
def create(self, backend: str, plan_summary: dict) -> Job:
|
||||||
|
job = Job(id=uuid.uuid4().hex[:12], backend=backend, plan=plan_summary)
|
||||||
|
with self._lock:
|
||||||
|
self._jobs[job.id] = job
|
||||||
|
self._order.append(job.id)
|
||||||
|
self._push(job, event="job_queued")
|
||||||
|
return job
|
||||||
|
|
||||||
|
def submit(self, job: Job, runner: Callable[[Job, BroadcastFn], dict]) -> None:
|
||||||
|
self._pool.submit(self._run, job, runner)
|
||||||
|
|
||||||
|
def get(self, job_id: str) -> Job | None:
|
||||||
|
return self._jobs.get(job_id)
|
||||||
|
|
||||||
|
def list(self, limit: int = 20) -> list[dict[str, Any]]:
|
||||||
|
with self._lock:
|
||||||
|
ids = list(reversed(self._order[-limit:]))
|
||||||
|
return [self._jobs[i].snapshot() for i in ids if i in self._jobs]
|
||||||
|
|
||||||
|
# -- internals ------------------------------------------------------------
|
||||||
|
def _run(self, job: Job, runner: Callable[[Job, BroadcastFn], dict]) -> None:
|
||||||
|
job.status = "running"
|
||||||
|
self._push(job, event="execution_start")
|
||||||
|
|
||||||
|
def progress(stage: str, pct: float, message: str, data: dict | None) -> None:
|
||||||
|
job.stage = stage
|
||||||
|
job.progress = max(0.0, min(100.0, pct))
|
||||||
|
job.message = message
|
||||||
|
evt = {
|
||||||
|
"stage": stage,
|
||||||
|
"progress": job.progress,
|
||||||
|
"message": message,
|
||||||
|
**({"data": data} if data else {}),
|
||||||
|
}
|
||||||
|
with self._lock:
|
||||||
|
job.events.append({"time": time.time(), **evt})
|
||||||
|
self._push(job, event="job_progress", **evt)
|
||||||
|
|
||||||
|
try:
|
||||||
|
result = runner(job, progress)
|
||||||
|
job.result = result
|
||||||
|
job.status = "done"
|
||||||
|
job.progress = 100.0
|
||||||
|
job.message = "done"
|
||||||
|
self._push(job, event="execution_success", outputs=result.get("outputs", {}))
|
||||||
|
except Exception as exc: # noqa: BLE001 - surface everything to the UI
|
||||||
|
job.status = "error"
|
||||||
|
job.error = f"{type(exc).__name__}: {exc}"
|
||||||
|
job.message = job.error
|
||||||
|
self._push(job, event="execution_error", error=job.error)
|
||||||
|
|
||||||
|
def _push(self, job: Job, event: str, **extra: Any) -> None:
|
||||||
|
try:
|
||||||
|
self._broadcast({
|
||||||
|
"type": event,
|
||||||
|
"job_id": job.id,
|
||||||
|
"status": job.status,
|
||||||
|
"stage": job.stage,
|
||||||
|
"progress": round(job.progress, 1),
|
||||||
|
"message": job.message,
|
||||||
|
**extra,
|
||||||
|
})
|
||||||
|
except Exception: # never let a dead WS client break a render
|
||||||
|
pass
|
||||||
50
bench/app/pipeline.py
Normal file
50
bench/app/pipeline.py
Normal file
@ -0,0 +1,50 @@
|
|||||||
|
"""Fixed-pipeline executor: binds a PipelineSpec + RenderPlan to a backend."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Callable
|
||||||
|
|
||||||
|
from .config import BenchConfig
|
||||||
|
from .tracks import RenderPlan
|
||||||
|
from .workflow import PipelineSpec
|
||||||
|
|
||||||
|
ProgressCb = Callable[[str, float, str, dict | None], None]
|
||||||
|
|
||||||
|
|
||||||
|
def run_pipeline(
|
||||||
|
spec: PipelineSpec,
|
||||||
|
plan: RenderPlan,
|
||||||
|
backend: str,
|
||||||
|
cfg: BenchConfig,
|
||||||
|
progress: ProgressCb,
|
||||||
|
job_dir: Path,
|
||||||
|
) -> dict:
|
||||||
|
"""Run all stages; returns {outputs: {...}, console: [...]}."""
|
||||||
|
job_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
if backend == "mock":
|
||||||
|
from .backends import mock
|
||||||
|
|
||||||
|
result = mock.render(plan, job_dir, progress)
|
||||||
|
elif backend == "comfy":
|
||||||
|
from .backends import comfy
|
||||||
|
|
||||||
|
result = comfy.render(spec, plan, cfg, job_dir, progress)
|
||||||
|
else:
|
||||||
|
raise RuntimeError(f"unknown backend {backend!r}")
|
||||||
|
|
||||||
|
# Move finished files into the job's outputs folder served by /api/view.
|
||||||
|
final: dict[str, dict] = {}
|
||||||
|
for key, out in result.outputs.items():
|
||||||
|
src = Path(out["path"])
|
||||||
|
dst = job_dir / src.name
|
||||||
|
if src.resolve() != dst.resolve():
|
||||||
|
shutil.move(str(src), str(dst))
|
||||||
|
out2 = dict(out)
|
||||||
|
out2["path"] = str(dst)
|
||||||
|
out2["filename"] = dst.name
|
||||||
|
out2["url"] = f"/api/view?filename={dst.name}&type=output&job={job_dir.name}"
|
||||||
|
final[key] = out2
|
||||||
|
|
||||||
|
return {"outputs": final, "console": result.console}
|
||||||
421
bench/app/server.py
Normal file
421
bench/app/server.py
Normal file
@ -0,0 +1,421 @@
|
|||||||
|
"""Moxie bench FastAPI server.
|
||||||
|
|
||||||
|
Serves the timeline UI (bench/ui-dist), the fixed-pipeline REST API and a
|
||||||
|
WebSocket that mirrors ComfyUI's push-event model (execution_start,
|
||||||
|
job_progress, execution_success/execution_error) so the ported editor can
|
||||||
|
subscribe with the same patterns it uses inside ComfyUI.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import shutil
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from fastapi import FastAPI, File, Form, HTTPException, UploadFile, WebSocket, WebSocketDisconnect
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
from fastapi.responses import FileResponse, JSONResponse, Response
|
||||||
|
from fastapi.staticfiles import StaticFiles
|
||||||
|
|
||||||
|
from . import __version__
|
||||||
|
from .config import BenchConfig
|
||||||
|
from .jobs import JobManager
|
||||||
|
from .pipeline import run_pipeline
|
||||||
|
from .tracks import TrackDataError, build_render_plan
|
||||||
|
from .workflow import (
|
||||||
|
EDITOR_TYPE,
|
||||||
|
WorkflowError,
|
||||||
|
describe,
|
||||||
|
editor_resolution,
|
||||||
|
load_base_workflow,
|
||||||
|
parse_workflow_dict,
|
||||||
|
project_data as spec_project_data,
|
||||||
|
)
|
||||||
|
|
||||||
|
_MEDIA_KEYS = ("images", "videos", "gifs", "audio")
|
||||||
|
|
||||||
|
|
||||||
|
def create_app(cfg: BenchConfig | None = None) -> FastAPI:
|
||||||
|
cfg = cfg or BenchConfig.from_args()
|
||||||
|
app = FastAPI(title="Moxie Multimedia Bench", version=__version__)
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=["*"],
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
|
||||||
|
loop_holder: dict = {}
|
||||||
|
subscribers: set[WebSocket] = set()
|
||||||
|
|
||||||
|
@app.on_event("startup")
|
||||||
|
async def _capture_loop() -> None:
|
||||||
|
loop_holder["loop"] = asyncio.get_running_loop()
|
||||||
|
|
||||||
|
def broadcast(payload: dict) -> None:
|
||||||
|
loop = loop_holder.get("loop")
|
||||||
|
if loop is None:
|
||||||
|
return
|
||||||
|
|
||||||
|
def _publish() -> None:
|
||||||
|
dead = []
|
||||||
|
for ws in list(subscribers):
|
||||||
|
asyncio.ensure_future(_send(ws, payload, dead))
|
||||||
|
for ws in dead:
|
||||||
|
subscribers.discard(ws)
|
||||||
|
|
||||||
|
try:
|
||||||
|
loop.call_soon_threadsafe(_publish)
|
||||||
|
except RuntimeError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def _send(ws: WebSocket, payload: dict, dead: list) -> None:
|
||||||
|
try:
|
||||||
|
await ws.send_json(payload)
|
||||||
|
except Exception: # noqa: BLE001
|
||||||
|
dead.append(ws)
|
||||||
|
|
||||||
|
jobs = JobManager(broadcast)
|
||||||
|
|
||||||
|
# -- spec helpers --------------------------------------------------------
|
||||||
|
|
||||||
|
def current_spec(raw_override: dict | None = None):
|
||||||
|
if raw_override is not None:
|
||||||
|
return parse_workflow_dict(raw_override)
|
||||||
|
return load_base_workflow(cfg.base_workflow_path)
|
||||||
|
|
||||||
|
def build_plan(spec, overrides: dict):
|
||||||
|
w, h = editor_resolution(spec)
|
||||||
|
pd = spec_project_data(spec)
|
||||||
|
ups = describe(spec)["upscale"] if "RTXVideoSuperResolution" in spec.stages else None
|
||||||
|
ups_target = (ups or {}).get("target") or {}
|
||||||
|
return build_render_plan(
|
||||||
|
spec.widget(EDITOR_TYPE, "track_data", "{}"),
|
||||||
|
width=int(overrides.get("width") or w),
|
||||||
|
height=int(overrides.get("height") or h),
|
||||||
|
frame_rate=overrides.get("frame_rate") or pd.get("frame_rate") or 24,
|
||||||
|
steps=int(overrides.get("steps") or spec.widget("MoxieProject", "steps", 4)),
|
||||||
|
seed=int(overrides.get("seed") or spec.widget("MoxieProject", "seed", 0) or 0),
|
||||||
|
project_name=overrides.get("project_name")
|
||||||
|
or spec.widget("MoxieProject", "project_name", "MoxieBench"),
|
||||||
|
lora=overrides.get("lora") or spec.widget("LoraLoaderModelOnly", "lora_name"),
|
||||||
|
lora_strength=float(
|
||||||
|
overrides.get("lora_strength")
|
||||||
|
or spec.widget("LoraLoaderModelOnly", "strength_model", 1.0)
|
||||||
|
),
|
||||||
|
upscale_width=ups_target.get("width"),
|
||||||
|
upscale_height=ups_target.get("height"),
|
||||||
|
upscale_quality=(ups or {}).get("quality", "ULTRA"),
|
||||||
|
codec=overrides.get("codec", "h264"),
|
||||||
|
)
|
||||||
|
|
||||||
|
# -- core API ------------------------------------------------------------
|
||||||
|
|
||||||
|
@app.get("/api/health")
|
||||||
|
def health():
|
||||||
|
return {"ok": True, "version": __version__, "backend": cfg.resolve_backend()}
|
||||||
|
|
||||||
|
@app.get("/api/config")
|
||||||
|
def config():
|
||||||
|
from .backends import available
|
||||||
|
|
||||||
|
return {
|
||||||
|
"version": __version__,
|
||||||
|
"backend": cfg.resolve_backend(),
|
||||||
|
"backends": available(cfg),
|
||||||
|
"work_dir": str(cfg.work_dir),
|
||||||
|
"outputs_dir": str(cfg.outputs_dir),
|
||||||
|
}
|
||||||
|
|
||||||
|
@app.get("/api/workflow")
|
||||||
|
def get_workflow():
|
||||||
|
spec = current_spec()
|
||||||
|
return {
|
||||||
|
"summary": describe(spec),
|
||||||
|
"track_data": spec.widget(EDITOR_TYPE, "track_data"),
|
||||||
|
"project_data": spec_project_data(spec),
|
||||||
|
}
|
||||||
|
|
||||||
|
@app.get("/api/plan")
|
||||||
|
def get_plan(overrides: str = "{}"):
|
||||||
|
spec = current_spec()
|
||||||
|
plan = build_plan(spec, json.loads(overrides or "{}"))
|
||||||
|
return plan.summary()
|
||||||
|
|
||||||
|
@app.post("/api/render")
|
||||||
|
async def render(body: dict):
|
||||||
|
try:
|
||||||
|
spec = current_spec(body.get("workflow"))
|
||||||
|
except WorkflowError as exc:
|
||||||
|
raise HTTPException(400, str(exc)) from exc
|
||||||
|
|
||||||
|
overrides = body.get("overrides") or {}
|
||||||
|
# UI-submitted track data replaces the template's copy
|
||||||
|
if body.get("track_data") is not None:
|
||||||
|
editor = spec.stages[EDITOR_TYPE]
|
||||||
|
editor.widget_values["track_data"] = (
|
||||||
|
body["track_data"]
|
||||||
|
if isinstance(body["track_data"], str)
|
||||||
|
else json.dumps(body["track_data"])
|
||||||
|
)
|
||||||
|
for key in ("resolution",):
|
||||||
|
if overrides.get(key):
|
||||||
|
editor = spec.stages[EDITOR_TYPE]
|
||||||
|
editor.widget_values[key] = overrides[key]
|
||||||
|
|
||||||
|
try:
|
||||||
|
plan = build_plan(spec, overrides)
|
||||||
|
except TrackDataError as exc:
|
||||||
|
raise HTTPException(400, str(exc)) from exc
|
||||||
|
|
||||||
|
backend = overrides.get("backend") or body.get("backend") or cfg.resolve_backend()
|
||||||
|
job = jobs.create(backend, plan.summary())
|
||||||
|
|
||||||
|
def runner(active: Job, progress) -> dict:
|
||||||
|
job_dir = cfg.outputs_dir / active.id
|
||||||
|
return run_pipeline(spec, plan, active.backend, cfg, progress, job_dir)
|
||||||
|
|
||||||
|
jobs.submit(job, runner)
|
||||||
|
return {"job_id": job.id, "plan": plan.summary()}
|
||||||
|
|
||||||
|
@app.get("/api/jobs")
|
||||||
|
def list_jobs():
|
||||||
|
return {"jobs": jobs.list()}
|
||||||
|
|
||||||
|
@app.get("/api/jobs/{job_id}")
|
||||||
|
def get_job(job_id: str):
|
||||||
|
job = jobs.get(job_id)
|
||||||
|
if not job:
|
||||||
|
raise HTTPException(404, "no such job")
|
||||||
|
snap = job.snapshot()
|
||||||
|
snap["events"] = job.events[-200:]
|
||||||
|
return snap
|
||||||
|
|
||||||
|
@app.delete("/api/jobs/{job_id}")
|
||||||
|
def delete_job(job_id: str):
|
||||||
|
job = jobs.get(job_id)
|
||||||
|
if not job:
|
||||||
|
raise HTTPException(404, "no such job")
|
||||||
|
job.status = "deleted"
|
||||||
|
return {"ok": True}
|
||||||
|
|
||||||
|
# -- media ----------------------------------------------------------------
|
||||||
|
|
||||||
|
def _safe(name: str) -> str:
|
||||||
|
name = Path(name).name
|
||||||
|
if not name or name.startswith("."):
|
||||||
|
raise HTTPException(400, "bad filename")
|
||||||
|
return name
|
||||||
|
|
||||||
|
@app.get("/api/view")
|
||||||
|
def view(filename: str, type: str = "output", job: str | None = None):
|
||||||
|
name = _safe(filename)
|
||||||
|
if type == "input":
|
||||||
|
path = cfg.inputs_dir / name
|
||||||
|
elif type == "temp":
|
||||||
|
path = cfg.temp_dir / name
|
||||||
|
else:
|
||||||
|
base = cfg.outputs_dir / _safe(job) if job else cfg.outputs_dir
|
||||||
|
path = base / name
|
||||||
|
if not path.is_file():
|
||||||
|
raise HTTPException(404, "no such file")
|
||||||
|
media = _guess_media_type(path)
|
||||||
|
return FileResponse(path, media_type=media)
|
||||||
|
|
||||||
|
@app.post("/api/upload/image")
|
||||||
|
async def upload_image(image: UploadFile = File(None), subfolder: str = Form("")):
|
||||||
|
return await _save_upload(image, subfolder)
|
||||||
|
|
||||||
|
@app.post("/api/easy-media/upload")
|
||||||
|
async def easy_upload(file: UploadFile = File(None), subfolder: str = Form("")):
|
||||||
|
return await _save_upload(file, subfolder)
|
||||||
|
|
||||||
|
async def _save_upload(upload: UploadFile | None, subfolder: str) -> dict:
|
||||||
|
if upload is None:
|
||||||
|
raise HTTPException(400, "missing file")
|
||||||
|
name = _safe(upload.filename or "upload.bin")
|
||||||
|
target_dir = cfg.inputs_dir
|
||||||
|
if subfolder:
|
||||||
|
target_dir = target_dir / Path(subfolder).name
|
||||||
|
target_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
target = target_dir / name
|
||||||
|
with open(target, "wb") as fh:
|
||||||
|
shutil.copyfileobj(upload.file, fh)
|
||||||
|
return {"name": name, "subfolder": str(target.parent.relative_to(cfg.inputs_dir)),
|
||||||
|
"type": "input"}
|
||||||
|
|
||||||
|
@app.post("/api/easy-media/download-url")
|
||||||
|
async def download_url(body: dict):
|
||||||
|
import urllib.request
|
||||||
|
|
||||||
|
url = body.get("url")
|
||||||
|
if not url:
|
||||||
|
raise HTTPException(400, "missing url")
|
||||||
|
name = _safe(body.get("name") or Path(url).name or "download.bin")
|
||||||
|
dst = cfg.inputs_dir / name
|
||||||
|
try:
|
||||||
|
with urllib.request.urlopen(url, timeout=60) as resp, open(dst, "wb") as fh:
|
||||||
|
shutil.copyfileobj(resp, fh)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
raise HTTPException(502, f"download failed: {exc}") from exc
|
||||||
|
return {"name": name, "subfolder": "", "type": "input"}
|
||||||
|
|
||||||
|
@app.get("/api/easy-media/media/list")
|
||||||
|
def media_list(kind: str = "input"):
|
||||||
|
base = cfg.inputs_dir if kind == "input" else cfg.outputs_dir
|
||||||
|
items = []
|
||||||
|
for p in sorted(base.rglob("*")):
|
||||||
|
if p.is_file() and not p.name.startswith("."):
|
||||||
|
rel = str(p.relative_to(base).parent)
|
||||||
|
items.append({
|
||||||
|
"name": p.name,
|
||||||
|
"path": str(p.relative_to(base)),
|
||||||
|
"subfolder": "" if rel == "." else rel,
|
||||||
|
"type": "input" if kind == "input" else "output",
|
||||||
|
"size": p.stat().st_size,
|
||||||
|
})
|
||||||
|
return {"items": items}
|
||||||
|
|
||||||
|
@app.get("/api/easy-media/media/recent")
|
||||||
|
def media_recent():
|
||||||
|
return media_list("output")
|
||||||
|
|
||||||
|
@app.get("/api/easy-media/projects")
|
||||||
|
def easy_projects():
|
||||||
|
projects_dir = cfg.work_dir / "projects"
|
||||||
|
projects_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
return {"projects": sorted(
|
||||||
|
p.stem for p in projects_dir.glob("*.json") if p.is_file())}
|
||||||
|
|
||||||
|
@app.get("/api/easy-media/project")
|
||||||
|
def easy_project(project_name: str):
|
||||||
|
path = cfg.work_dir / "projects" / f"{Path(project_name).stem}.json"
|
||||||
|
if not path.is_file():
|
||||||
|
raise HTTPException(404, "no such project")
|
||||||
|
return JSONResponse(content=json.loads(path.read_text()))
|
||||||
|
|
||||||
|
@app.get("/api/easy-media/prompt/system-prompts")
|
||||||
|
def system_prompts():
|
||||||
|
return {"prompts": [
|
||||||
|
{"id": "default", "name": "Default", "prompt": ""},
|
||||||
|
]}
|
||||||
|
|
||||||
|
@app.post("/api/easy-media/video/source-fps")
|
||||||
|
def source_fps(body: dict):
|
||||||
|
name = _safe(body.get("filename") or "")
|
||||||
|
path = cfg.inputs_dir / name
|
||||||
|
if not path.is_file():
|
||||||
|
raise HTTPException(404, "no such file")
|
||||||
|
try:
|
||||||
|
out = subprocess.run(
|
||||||
|
["ffprobe", "-v", "quiet", "-print_format", "json",
|
||||||
|
"-show_streams", str(path)],
|
||||||
|
capture_output=True, text=True, timeout=30,
|
||||||
|
).stdout
|
||||||
|
streams = json.loads(out).get("streams") or []
|
||||||
|
rate = (streams[0].get("r_frame_rate") or "24/1") if streams else "24/1"
|
||||||
|
num, _, den = rate.partition("/")
|
||||||
|
fps = float(num) / float(den or 1)
|
||||||
|
except Exception as exc: # noqa: BLE001
|
||||||
|
raise HTTPException(500, f"ffprobe failed: {exc}") from exc
|
||||||
|
return {"fps": fps}
|
||||||
|
|
||||||
|
@app.post("/api/easy-media/video/preview-frame")
|
||||||
|
def preview_frame(body: dict):
|
||||||
|
name = _safe(body.get("filename") or "")
|
||||||
|
path = cfg.inputs_dir / name
|
||||||
|
if not path.is_file():
|
||||||
|
raise HTTPException(404, "no such file")
|
||||||
|
t = float(body.get("time") or 0)
|
||||||
|
try:
|
||||||
|
out = subprocess.run(
|
||||||
|
["ffmpeg", "-v", "quiet", "-ss", str(t), "-i", str(path),
|
||||||
|
"-frames:v", "1", "-f", "image2pipe", "-vcodec", "mjpeg", "pipe:1"],
|
||||||
|
capture_output=True, timeout=60,
|
||||||
|
)
|
||||||
|
except subprocess.TimeoutExpired as exc:
|
||||||
|
raise HTTPException(504, "ffmpeg timeout") from exc
|
||||||
|
if not out.stdout:
|
||||||
|
raise HTTPException(500, "ffmpeg produced no frame")
|
||||||
|
return Response(content=out.stdout, media_type="image/jpeg")
|
||||||
|
|
||||||
|
# -- ComfyUI-compatible root aliases --------------------------------------
|
||||||
|
# The ported editor (and its media elements) call the exact ComfyUI paths:
|
||||||
|
# /view, /upload/image, /easy-media/*. Register them alongside /api/* so
|
||||||
|
# the UI shim needs no URL rewriting at all.
|
||||||
|
app.add_api_route("/view", view, methods=["GET"], include_in_schema=False)
|
||||||
|
app.add_api_route("/upload/image", _save_upload, methods=["POST"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/upload", _save_upload, methods=["POST"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/media/list", media_list, methods=["GET"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/media/recent", media_recent, methods=["GET"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/projects", easy_projects, methods=["GET"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/project", easy_project, methods=["GET"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/prompt/system-prompts", system_prompts, methods=["GET"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/video/source-fps", source_fps, methods=["POST"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/video/preview-frame", preview_frame, methods=["POST"], include_in_schema=False)
|
||||||
|
app.add_api_route("/easy-media/download-url", download_url, methods=["POST"], include_in_schema=False)
|
||||||
|
|
||||||
|
# fallthrough for endpoints the editor may call in v1
|
||||||
|
@app.api_route("/api/easy-media/{rest:path}", methods=["GET", "POST", "PATCH", "DELETE"])
|
||||||
|
def easy_fallback(rest: str):
|
||||||
|
return JSONResponse(
|
||||||
|
status_code=501,
|
||||||
|
content={"error": f"bench does not implement /api/easy-media/{rest} yet"},
|
||||||
|
)
|
||||||
|
|
||||||
|
@app.api_route("/easy-media/{rest:path}", methods=["GET", "POST", "PATCH", "DELETE"])
|
||||||
|
def easy_fallback_root(rest: str):
|
||||||
|
return easy_fallback(rest)
|
||||||
|
|
||||||
|
# -- websocket -------------------------------------------------------------
|
||||||
|
|
||||||
|
@app.websocket("/api/ws")
|
||||||
|
async def ws_endpoint(ws: WebSocket):
|
||||||
|
await ws.accept()
|
||||||
|
subscribers.add(ws)
|
||||||
|
try:
|
||||||
|
while True:
|
||||||
|
await ws.receive_text() # keepalive pings from the UI
|
||||||
|
except WebSocketDisconnect:
|
||||||
|
pass
|
||||||
|
finally:
|
||||||
|
subscribers.discard(ws)
|
||||||
|
|
||||||
|
# -- static UI --------------------------------------------------------------
|
||||||
|
|
||||||
|
ui_dist = cfg.ui_dist
|
||||||
|
if (ui_dist / "index.html").is_file():
|
||||||
|
app.mount("/", StaticFiles(directory=str(ui_dist), html=True), name="ui")
|
||||||
|
else:
|
||||||
|
@app.get("/")
|
||||||
|
def no_ui():
|
||||||
|
return JSONResponse({
|
||||||
|
"error": "bench UI not built",
|
||||||
|
"hint": "cd frontend && bun run build:bench",
|
||||||
|
})
|
||||||
|
|
||||||
|
# silence 404 noise for browsers requesting favicons
|
||||||
|
@app.get("/favicon.ico")
|
||||||
|
def favicon():
|
||||||
|
return Response(status_code=204)
|
||||||
|
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
def _guess_media_type(path: Path) -> str:
|
||||||
|
ext = path.suffix.lower()
|
||||||
|
return {
|
||||||
|
".mp4": "video/mp4", ".webm": "video/webm", ".mov": "video/quicktime",
|
||||||
|
".mkv": "video/x-matroska", ".png": "image/png", ".jpg": "image/jpeg",
|
||||||
|
".jpeg": "image/jpeg", ".webp": "image/webp", ".gif": "image/gif",
|
||||||
|
".wav": "audio/wav", ".mp3": "audio/mpeg", ".flac": "audio/flac",
|
||||||
|
".srt": "text/plain",
|
||||||
|
}.get(ext, "application/octet-stream")
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None: # pragma: no cover - manual entry
|
||||||
|
import uvicorn
|
||||||
|
|
||||||
|
cfg = BenchConfig.from_args()
|
||||||
|
uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port, log_level="info")
|
||||||
195
bench/app/tracks.py
Normal file
195
bench/app/tracks.py
Normal file
@ -0,0 +1,195 @@
|
|||||||
|
"""track_data (the multitrack editor JSON) -> RenderPlan.
|
||||||
|
|
||||||
|
Pure extraction, no ComfyUI. Mirrors what ``MultiTrackEditor.execute`` feeds
|
||||||
|
into TRACKS_INFO for the fixed pipeline: task tracks carry the prompt segments
|
||||||
|
that become generation tasks; video/audio tracks carry media references.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
class TrackDataError(ValueError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Segment:
|
||||||
|
id: str
|
||||||
|
start_frame: int
|
||||||
|
end_frame: int
|
||||||
|
prompt: str = ""
|
||||||
|
task_mode: str = "default"
|
||||||
|
continuity_mode: str = "shot"
|
||||||
|
ref_image_size: str = "match"
|
||||||
|
media_type: str = "none"
|
||||||
|
images: list[Any] = field(default_factory=list)
|
||||||
|
muted: bool = False
|
||||||
|
volume_db: float = 0.0
|
||||||
|
|
||||||
|
@property
|
||||||
|
def frame_count(self) -> int:
|
||||||
|
return max(0, self.end_frame - self.start_frame)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Track:
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
type: str
|
||||||
|
task_mode: str = "default"
|
||||||
|
muted: bool = False
|
||||||
|
solo: bool = False
|
||||||
|
volume_db: float = 0.0
|
||||||
|
locked: bool = False
|
||||||
|
segments: list[Segment] = field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class RenderPlan:
|
||||||
|
width: int
|
||||||
|
height: int
|
||||||
|
frame_rate: int
|
||||||
|
total_length: int
|
||||||
|
segments: list[Segment] = field(default_factory=list) # ordered task segments
|
||||||
|
tracks: list[Track] = field(default_factory=list)
|
||||||
|
project_name: str = "MoxieBench"
|
||||||
|
steps: int = 4
|
||||||
|
seed: int = 0
|
||||||
|
lora: str | None = None
|
||||||
|
lora_strength: float = 1.0
|
||||||
|
upscale_width: int | None = None
|
||||||
|
upscale_height: int | None = None
|
||||||
|
upscale_quality: str = "ULTRA"
|
||||||
|
codec: str = "h264"
|
||||||
|
|
||||||
|
@property
|
||||||
|
def duration_seconds(self) -> float:
|
||||||
|
return self.total_length / float(self.frame_rate or 24)
|
||||||
|
|
||||||
|
def summary(self) -> dict[str, Any]:
|
||||||
|
return {
|
||||||
|
"width": self.width,
|
||||||
|
"height": self.height,
|
||||||
|
"frame_rate": self.frame_rate,
|
||||||
|
"total_length": self.total_length,
|
||||||
|
"duration_seconds": round(self.duration_seconds, 3),
|
||||||
|
"steps": self.steps,
|
||||||
|
"seed": self.seed,
|
||||||
|
"lora": self.lora,
|
||||||
|
"project_name": self.project_name,
|
||||||
|
"upscale": (
|
||||||
|
{"width": self.upscale_width, "height": self.upscale_height,
|
||||||
|
"quality": self.upscale_quality}
|
||||||
|
if self.upscale_width else None
|
||||||
|
),
|
||||||
|
"segments": [
|
||||||
|
{
|
||||||
|
"id": s.id,
|
||||||
|
"prompt": s.prompt,
|
||||||
|
"start_frame": s.start_frame,
|
||||||
|
"end_frame": s.end_frame,
|
||||||
|
"frame_count": s.frame_count,
|
||||||
|
"task_mode": s.task_mode,
|
||||||
|
"continuity_mode": s.continuity_mode,
|
||||||
|
}
|
||||||
|
for s in self.segments
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def parse_track_data(raw: str | dict[str, Any]) -> dict[str, Any]:
|
||||||
|
if isinstance(raw, str):
|
||||||
|
try:
|
||||||
|
raw = json.loads(raw)
|
||||||
|
except json.JSONDecodeError as exc:
|
||||||
|
raise TrackDataError(f"track_data is not valid JSON: {exc}") from exc
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
raise TrackDataError("track_data must be an object")
|
||||||
|
if not isinstance(raw.get("tracks"), list):
|
||||||
|
raise TrackDataError("track_data.tracks must be a list")
|
||||||
|
return raw
|
||||||
|
|
||||||
|
|
||||||
|
def build_render_plan(
|
||||||
|
track_data: str | dict[str, Any],
|
||||||
|
*,
|
||||||
|
width: int,
|
||||||
|
height: int,
|
||||||
|
frame_rate: int | None = None,
|
||||||
|
steps: int = 4,
|
||||||
|
seed: int = 0,
|
||||||
|
project_name: str = "MoxieBench",
|
||||||
|
lora: str | None = None,
|
||||||
|
lora_strength: float = 1.0,
|
||||||
|
upscale_width: int | None = None,
|
||||||
|
upscale_height: int | None = None,
|
||||||
|
upscale_quality: str = "ULTRA",
|
||||||
|
codec: str = "h264",
|
||||||
|
) -> RenderPlan:
|
||||||
|
data = parse_track_data(track_data)
|
||||||
|
|
||||||
|
# Timeline fps: editor stores it on track_data.frame_rate (24 in the base
|
||||||
|
# workflow); the combine project_data may override it.
|
||||||
|
if frame_rate is None:
|
||||||
|
frame_rate = int(data.get("frame_rate") or 24)
|
||||||
|
total = int(data.get("total_length") or 0)
|
||||||
|
|
||||||
|
tracks: list[Track] = []
|
||||||
|
segments: list[Segment] = []
|
||||||
|
for t in data["tracks"]:
|
||||||
|
if not isinstance(t, dict):
|
||||||
|
continue
|
||||||
|
track = Track(
|
||||||
|
id=str(t.get("id", "")),
|
||||||
|
name=str(t.get("name", "")),
|
||||||
|
type=str(t.get("type", "task")),
|
||||||
|
task_mode=str(t.get("task_mode", "default")),
|
||||||
|
muted=bool(t.get("muted", False)),
|
||||||
|
solo=bool(t.get("solo", False)),
|
||||||
|
volume_db=float(t.get("volume_db", 0)),
|
||||||
|
locked=bool(t.get("locked", False)),
|
||||||
|
)
|
||||||
|
for seg in t.get("segments") or []:
|
||||||
|
if not isinstance(seg, dict):
|
||||||
|
continue
|
||||||
|
content = seg.get("content") or {}
|
||||||
|
s = Segment(
|
||||||
|
id=str(seg.get("id", "")),
|
||||||
|
start_frame=int(seg.get("start_frame", 0)),
|
||||||
|
end_frame=int(seg.get("end_frame", 0)),
|
||||||
|
prompt=str(content.get("user_prompt", "") or ""),
|
||||||
|
task_mode=str(content.get("task_mode", track.task_mode or "default")),
|
||||||
|
continuity_mode=str(content.get("continuity_mode", "shot")),
|
||||||
|
ref_image_size=str(content.get("ref_image_size", "match")),
|
||||||
|
media_type=str(content.get("media_type", "none")),
|
||||||
|
images=list(content.get("images") or []),
|
||||||
|
muted=bool(content.get("muted", False)),
|
||||||
|
volume_db=float(content.get("volume_db", 0)),
|
||||||
|
)
|
||||||
|
track.segments.append(s)
|
||||||
|
if track.type == "task" and s.frame_count > 0:
|
||||||
|
segments.append(s)
|
||||||
|
tracks.append(track)
|
||||||
|
|
||||||
|
segments.sort(key=lambda s: (s.start_frame, s.end_frame))
|
||||||
|
|
||||||
|
return RenderPlan(
|
||||||
|
width=int(width),
|
||||||
|
height=int(height),
|
||||||
|
frame_rate=int(frame_rate),
|
||||||
|
total_length=total,
|
||||||
|
segments=segments,
|
||||||
|
tracks=tracks,
|
||||||
|
project_name=str(project_name),
|
||||||
|
steps=int(steps),
|
||||||
|
seed=int(seed),
|
||||||
|
lora=lora,
|
||||||
|
lora_strength=float(lora_strength),
|
||||||
|
upscale_width=upscale_width,
|
||||||
|
upscale_height=upscale_height,
|
||||||
|
upscale_quality=str(upscale_quality),
|
||||||
|
codec=str(codec),
|
||||||
|
)
|
||||||
204
bench/app/workflow.py
Normal file
204
bench/app/workflow.py
Normal file
@ -0,0 +1,204 @@
|
|||||||
|
"""Fixed-pipeline contract parsed from the base workflow JSON.
|
||||||
|
|
||||||
|
The bench locks to the Moxie pipeline used by the base workflow:
|
||||||
|
|
||||||
|
MoxieMultimediaLoader -> LoraLoaderModelOnly -> MoxiePreviewOverride
|
||||||
|
-> easy multiTrackEditor -> MoxieProject -> MoxieCombine
|
||||||
|
-> [branch A] CreateVideo(#45) -> SaveVideo(#46) (native)
|
||||||
|
-> [branch B] RTXVideoSuperResolution -> CreateVideo(#68)
|
||||||
|
-> SaveVideo(#69) (upscaled)
|
||||||
|
|
||||||
|
The parser walks the frontend-format workflow (nodes/links/widgets_values_named)
|
||||||
|
that ComfyUI exports and needs no ComfyUI installation: link inputs carry their
|
||||||
|
own names in the node's ``inputs`` array, widget values carry ``widgets_values_named``.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import copy
|
||||||
|
import json
|
||||||
|
from dataclasses import dataclass, field
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
EDITOR_TYPE = "easy multiTrackEditor"
|
||||||
|
STAGE_ORDER = (
|
||||||
|
"MoxieMultimediaLoader",
|
||||||
|
"LoraLoaderModelOnly",
|
||||||
|
"MoxiePreviewOverride",
|
||||||
|
EDITOR_TYPE,
|
||||||
|
"MoxieProject",
|
||||||
|
"MoxieCombine",
|
||||||
|
"RTXVideoSuperResolution",
|
||||||
|
"CreateVideo",
|
||||||
|
) # SaveVideo x2 tracked via PipelineSpec.save_nodes (native/upscaled branches)
|
||||||
|
|
||||||
|
|
||||||
|
class WorkflowError(ValueError):
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class Stage:
|
||||||
|
node_id: int
|
||||||
|
type: str
|
||||||
|
widget_values: dict[str, Any] = field(default_factory=dict)
|
||||||
|
link_inputs: dict[str, tuple[int, int]] = field(default_factory=dict) # name -> (node_id, slot)
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class PipelineSpec:
|
||||||
|
raw: dict[str, Any]
|
||||||
|
stages: dict[str, Stage] # type -> Stage (types unique in the fixed graph)
|
||||||
|
save_nodes: dict[str, Stage] = field(default_factory=dict) # native/upscaled -> SaveVideo
|
||||||
|
|
||||||
|
@property
|
||||||
|
def editor_node_id(self) -> int:
|
||||||
|
return self.stages[EDITOR_TYPE].node_id
|
||||||
|
|
||||||
|
def widget(self, stage_type: str, name: str, default: Any = None) -> Any:
|
||||||
|
st = self.stages.get(stage_type)
|
||||||
|
if not st:
|
||||||
|
return default
|
||||||
|
return st.widget_values.get(name, default)
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_workflow(raw: dict[str, Any]) -> PipelineSpec:
|
||||||
|
nodes = raw.get("nodes")
|
||||||
|
if not isinstance(nodes, list) or not nodes:
|
||||||
|
raise WorkflowError("workflow has no nodes")
|
||||||
|
links: dict[int, tuple[int, int, int]] = {}
|
||||||
|
for lk in raw.get("links") or []:
|
||||||
|
# [link_id, from_node, from_slot, to_node, to_slot, type]
|
||||||
|
links[int(lk[0])] = (int(lk[1]), int(lk[2]), int(lk[4]))
|
||||||
|
|
||||||
|
stages: dict[str, Stage] = {}
|
||||||
|
save_nodes: dict[str, Stage] = {}
|
||||||
|
seen_create = 0
|
||||||
|
for n in nodes:
|
||||||
|
ntype = n.get("type")
|
||||||
|
nid = int(n.get("id"))
|
||||||
|
if ntype not in STAGE_ORDER and ntype != "SaveVideo":
|
||||||
|
continue
|
||||||
|
named = dict(n.get("widgets_values_named") or {})
|
||||||
|
# Legacy export without named values: best effort positional mapping.
|
||||||
|
if not named:
|
||||||
|
named = {}
|
||||||
|
stage = Stage(node_id=nid, type=ntype, widget_values=named)
|
||||||
|
for inp in n.get("inputs") or []:
|
||||||
|
if not isinstance(inp, dict):
|
||||||
|
continue
|
||||||
|
if "link" in inp and inp["link"] is not None and "widget" not in inp:
|
||||||
|
frm = links.get(int(inp["link"]))
|
||||||
|
if frm is None:
|
||||||
|
raise WorkflowError(f"node {nid}: dangling link {inp['link']}")
|
||||||
|
stage.link_inputs[inp["name"]] = (frm[0], frm[1])
|
||||||
|
if ntype == "CreateVideo":
|
||||||
|
seen_create += 1
|
||||||
|
key = "CreateVideo"
|
||||||
|
if seen_create == 2:
|
||||||
|
stages.pop("CreateVideo#1", None)
|
||||||
|
stages[f"CreateVideo#{seen_create}" if seen_create > 1 else key] = stage
|
||||||
|
continue
|
||||||
|
if ntype == "SaveVideo":
|
||||||
|
src = stage.link_inputs.get("video", (None, None))[0]
|
||||||
|
if src is not None and src in (
|
||||||
|
stages.get("CreateVideo#2").node_id if stages.get("CreateVideo#2") else -1,
|
||||||
|
):
|
||||||
|
save_nodes["upscaled"] = stage
|
||||||
|
else:
|
||||||
|
save_nodes["native"] = stage
|
||||||
|
continue
|
||||||
|
if ntype in stages:
|
||||||
|
raise WorkflowError(f"duplicate stage node type {ntype}")
|
||||||
|
stages[ntype] = stage
|
||||||
|
|
||||||
|
missing = [t for t in STAGE_ORDER if t not in stages]
|
||||||
|
if missing:
|
||||||
|
raise WorkflowError(f"missing pipeline stages: {', '.join(missing)}")
|
||||||
|
return PipelineSpec(raw=raw, stages=stages, save_nodes=save_nodes)
|
||||||
|
|
||||||
|
|
||||||
|
def load_base_workflow(path: Path) -> PipelineSpec:
|
||||||
|
with open(path, "r", encoding="utf-8") as fh:
|
||||||
|
raw = json.load(fh)
|
||||||
|
return _parse_workflow(raw)
|
||||||
|
|
||||||
|
|
||||||
|
def parse_workflow_dict(raw: dict[str, Any]) -> PipelineSpec:
|
||||||
|
return _parse_workflow(copy.deepcopy(raw))
|
||||||
|
|
||||||
|
|
||||||
|
def editor_track_data(spec: PipelineSpec) -> dict[str, Any]:
|
||||||
|
"""Parsed track_data JSON from the editor stage."""
|
||||||
|
raw = spec.widget(EDITOR_TYPE, "track_data")
|
||||||
|
if isinstance(raw, str):
|
||||||
|
raw = json.loads(raw)
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
raise WorkflowError("editor track_data is not an object")
|
||||||
|
return raw
|
||||||
|
|
||||||
|
|
||||||
|
def editor_resolution(spec: PipelineSpec) -> tuple[int, int]:
|
||||||
|
"""Width/height chosen on the editor, falling back to the project_data."""
|
||||||
|
res = spec.widget(EDITOR_TYPE, "resolution", "")
|
||||||
|
preset = _RESOLUTION_PRESETS.get(res)
|
||||||
|
if preset:
|
||||||
|
return preset
|
||||||
|
combo = spec.stages[EDITOR_TYPE].widget_values.get("resolution")
|
||||||
|
if isinstance(combo, dict): # custom: {"width": .., "height": ..}
|
||||||
|
try:
|
||||||
|
return int(combo.get("width")), int(combo.get("height"))
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
pass
|
||||||
|
pd = _project_data(spec)
|
||||||
|
return int(pd.get("width", 0)), int(pd.get("height", 0))
|
||||||
|
|
||||||
|
|
||||||
|
def project_data(spec: PipelineSpec) -> dict[str, Any]:
|
||||||
|
return _project_data(spec)
|
||||||
|
|
||||||
|
|
||||||
|
def _project_data(spec: PipelineSpec) -> dict[str, Any]:
|
||||||
|
raw = spec.widget("MoxieCombine", "project_data", "") or "{}"
|
||||||
|
if isinstance(raw, str):
|
||||||
|
raw = json.loads(raw)
|
||||||
|
return raw if isinstance(raw, dict) else {}
|
||||||
|
|
||||||
|
|
||||||
|
_RESOLUTION_PRESETS = {
|
||||||
|
"1920 x 1080 (16:9)": (1920, 1080),
|
||||||
|
"1088 x 1920 (9:16)": (1088, 1920),
|
||||||
|
"1080 x 1920 (9:16)": (1080, 1920),
|
||||||
|
"1280 x 720 (16:9)": (1280, 720),
|
||||||
|
"720 x 1280 (9:16)": (720, 1280),
|
||||||
|
"832 x 480 (16:9)": (832, 480),
|
||||||
|
"480 x 832 (9:16)": (480, 832),
|
||||||
|
"1024 x 1024 (1:1)": (1024, 1024),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def describe(spec: PipelineSpec) -> dict[str, Any]:
|
||||||
|
"""Small human-readable summary used by /api/workflow."""
|
||||||
|
w, h = editor_resolution(spec)
|
||||||
|
pd = project_data(spec)
|
||||||
|
return {
|
||||||
|
"stages": [
|
||||||
|
{"type": t, "node_id": s.node_id, "widgets": s.widget_values}
|
||||||
|
for t, s in spec.stages.items()
|
||||||
|
],
|
||||||
|
"editor_resolution": {"width": w, "height": h},
|
||||||
|
"frame_rate": pd.get("frame_rate", 24),
|
||||||
|
"project_name": spec.widget("MoxieProject", "project_name", "MoxieBench"),
|
||||||
|
"steps": spec.widget("MoxieProject", "steps", 4),
|
||||||
|
"seed": spec.widget("MoxieProject", "seed", 0),
|
||||||
|
"lora": spec.widget("LoraLoaderModelOnly", "lora_name"),
|
||||||
|
"lora_strength": spec.widget("LoraLoaderModelOnly", "strength_model", 1.0),
|
||||||
|
"upscale": {
|
||||||
|
"enabled": "RTXVideoSuperResolution" in spec.stages,
|
||||||
|
"target": {
|
||||||
|
"width": spec.widget("RTXVideoSuperResolution", "resize_type.width"),
|
||||||
|
"height": spec.widget("RTXVideoSuperResolution", "resize_type.height"),
|
||||||
|
},
|
||||||
|
"quality": spec.widget("RTXVideoSuperResolution", "quality", "ULTRA"),
|
||||||
|
},
|
||||||
|
}
|
||||||
240
bench/base_workflow.json
Normal file
240
bench/base_workflow.json
Normal file
@ -0,0 +1,240 @@
|
|||||||
|
{
|
||||||
|
"id": "74d5a00d-104c-439f-a307-874d829f8418",
|
||||||
|
"revision": 0,
|
||||||
|
"last_node_id": 81,
|
||||||
|
"last_link_id": 166,
|
||||||
|
"nodes": [
|
||||||
|
{
|
||||||
|
"id": 45,
|
||||||
|
"type": "CreateVideo",
|
||||||
|
"pos": [5303.990730085631, 2674.6532486606284],
|
||||||
|
"size": [270, 150],
|
||||||
|
"flags": {},
|
||||||
|
"order": 7,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [
|
||||||
|
{"name": "images", "type": "IMAGE", "link": 151},
|
||||||
|
{"name": "audio", "shape": 7, "type": "AUDIO", "link": 149},
|
||||||
|
{"name": "fps", "type": "FLOAT", "widget": {"name": "fps"}, "link": 150}
|
||||||
|
],
|
||||||
|
"outputs": [{"name": "VIDEO", "type": "VIDEO", "links": [103]}],
|
||||||
|
"properties": {"Node name for S&R": "CreateVideo"},
|
||||||
|
"widgets_values": [30, "auto", "sRGB", "h264"],
|
||||||
|
"widgets_values_named": {"fps": 30, "bit_depth": "auto", "color_space": "sRGB", "codec": "h264"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 46,
|
||||||
|
"type": "SaveVideo",
|
||||||
|
"pos": [6536.102438951979, 2629.839158378452],
|
||||||
|
"size": [1056.6122485107426, 1748.7755735581345],
|
||||||
|
"flags": {},
|
||||||
|
"order": 9,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [{"name": "video", "type": "VIDEO", "link": 103}],
|
||||||
|
"outputs": [{"name": "video", "type": "VIDEO", "links": null}],
|
||||||
|
"properties": {"Node name for S&R": "SaveVideo"},
|
||||||
|
"widgets_values": ["video/ComfyUI", "mp4", "h264", "auto", "auto"],
|
||||||
|
"widgets_values_named": {"filename_prefix": "video/ComfyUI", "format": "mp4", "format.codec": "h264", "format.codec.encoding": "auto", "codec": "auto"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 48,
|
||||||
|
"type": "MoxiePreviewOverride",
|
||||||
|
"pos": [3815.4265801150054, 2654.0916373620416],
|
||||||
|
"size": [360, 480],
|
||||||
|
"flags": {},
|
||||||
|
"order": 3,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [{"name": "model", "type": "MODEL", "link": 154}],
|
||||||
|
"outputs": [{"name": "model", "type": "MODEL", "links": [144]}],
|
||||||
|
"properties": {"Node name for S&R": "MoxiePreviewOverride"},
|
||||||
|
"widgets_values": [1024, 80, true, 48, 24, "MM-preview.safetensors", ""],
|
||||||
|
"widgets_values_named": {"max_resolution": 1024, "jpeg_quality": 80, "suppress_default_preview": true, "preview_frames": 48, "preview_fps": 24, "tiny_vae": "MM-preview.safetensors", "preview": ""}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 52,
|
||||||
|
"type": "MoxieMultimediaLoader",
|
||||||
|
"pos": [3000.4622746801124, 2645.6062887703747],
|
||||||
|
"size": [370.66988849138306, 180.73178061607814],
|
||||||
|
"flags": {},
|
||||||
|
"order": 0,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [],
|
||||||
|
"outputs": [
|
||||||
|
{"name": "model", "type": "MODEL", "links": [153]},
|
||||||
|
{"name": "clip", "type": "CLIP", "links": [136]},
|
||||||
|
{"name": "video_vae", "type": "VAE", "links": [137]},
|
||||||
|
{"name": "audio_vae", "type": "VAE", "links": [138]}
|
||||||
|
],
|
||||||
|
"properties": {"Node name for S&R": "MoxieMultimediaLoader"},
|
||||||
|
"widgets_values": ["Moxie-Multimedia.safetensors", "default", "comfy_kitchen_attention"],
|
||||||
|
"widgets_values_named": {"diffusion_model": "Moxie-Multimedia.safetensors", "weight_dtype": "default", "attention_backend": "comfy_kitchen_attention"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 57,
|
||||||
|
"type": "MoxieProject",
|
||||||
|
"pos": [4349.808755835772, 2799.703672942545],
|
||||||
|
"size": [277.359375, 230],
|
||||||
|
"flags": {},
|
||||||
|
"order": 4,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [
|
||||||
|
{"name": "tracks_info", "type": "TRACKS_INFO", "link": 166},
|
||||||
|
{"name": "model", "type": "MODEL", "link": 144},
|
||||||
|
{"name": "clip", "type": "CLIP", "link": 136},
|
||||||
|
{"name": "video_vae", "type": "VAE", "link": 137},
|
||||||
|
{"name": "audio_vae", "type": "VAE", "link": 138}
|
||||||
|
],
|
||||||
|
"outputs": [
|
||||||
|
{"name": "PROJECT_NAME", "type": "STRING", "links": [148]},
|
||||||
|
{"name": "LOCKED_AUDIO", "type": "AUDIO", "links": null}
|
||||||
|
],
|
||||||
|
"properties": {"Node name for S&R": "MoxieProject"},
|
||||||
|
"widgets_values": ["MM-girlwaving", 4, 803100406447983, "randomize"],
|
||||||
|
"widgets_values_named": {"project_name": "MM-girlwaving", "steps": 4, "seed": 803100406447983, "randomize": "randomize"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 63,
|
||||||
|
"type": "MoxieCombine",
|
||||||
|
"pos": [4734.228609743854, 2711.230251501094],
|
||||||
|
"size": [295.59921875, 638],
|
||||||
|
"flags": {},
|
||||||
|
"order": 5,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [{"name": "project_name", "type": "STRING", "link": 148}],
|
||||||
|
"outputs": [
|
||||||
|
{"name": "images", "type": "IMAGE", "links": [151, 155]},
|
||||||
|
{"name": "audio", "type": "AUDIO", "links": [149, 157]},
|
||||||
|
{"name": "fps", "type": "FLOAT", "links": [150, 159]}
|
||||||
|
],
|
||||||
|
"properties": {"Node name for S&R": "MoxieCombine"},
|
||||||
|
"widgets_values": ["{\"project_name\":\"MM-girlwaving\",\"width\":1920,\"height\":1088,\"frame_rate\":24,\"clips\":[],\"auto_combine\":true,\"updated_at\":0}"],
|
||||||
|
"widgets_values_named": {"project_data": "{\"project_name\":\"MM-girlwaving\",\"width\":1920,\"height\":1088,\"frame_rate\":24,\"clips\":[],\"auto_combine\":true,\"updated_at\":0}"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 66,
|
||||||
|
"type": "LoraLoaderModelOnly",
|
||||||
|
"pos": [3539.407729088216, 2558.7373665366495],
|
||||||
|
"size": [270, 82],
|
||||||
|
"flags": {},
|
||||||
|
"order": 2,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [{"name": "model", "type": "MODEL", "link": 153}],
|
||||||
|
"outputs": [{"name": "MODEL", "type": "MODEL", "links": [154]}],
|
||||||
|
"properties": {"Node name for S&R": "LoraLoaderModelOnly"},
|
||||||
|
"widgets_values": ["h3\\minimax_h3_t2v_mshandjob_audio_v1.safetensors", 1],
|
||||||
|
"widgets_values_named": {"lora_name": "h3\\minimax_h3_t2v_mshandjob_audio_v1.safetensors", "strength_model": 1}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 67,
|
||||||
|
"type": "RTXVideoSuperResolution",
|
||||||
|
"pos": [5085.886101348601, 3012.843050491562],
|
||||||
|
"size": [285.81015625, 130],
|
||||||
|
"flags": {},
|
||||||
|
"order": 6,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [{"name": "images", "type": "IMAGE", "link": 155}],
|
||||||
|
"outputs": [{"name": "upscaled_images", "type": "IMAGE", "links": [156]}],
|
||||||
|
"properties": {"Node name for S&R": "RTXVideoSuperResolution"},
|
||||||
|
"widgets_values": ["target dimensions", 1920, 1080, "ULTRA"],
|
||||||
|
"widgets_values_named": {"resize_type": "target dimensions", "resize_type.width": 1920, "resize_type.height": 1080, "quality": "ULTRA"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 68,
|
||||||
|
"type": "CreateVideo",
|
||||||
|
"pos": [5404.822730085631, 2911.779640536847],
|
||||||
|
"size": [270, 150],
|
||||||
|
"flags": {},
|
||||||
|
"order": 8,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [
|
||||||
|
{"name": "images", "type": "IMAGE", "link": 156},
|
||||||
|
{"name": "audio", "shape": 7, "type": "AUDIO", "link": 157},
|
||||||
|
{"name": "fps", "type": "FLOAT", "widget": {"name": "fps"}, "link": 159}
|
||||||
|
],
|
||||||
|
"outputs": [{"name": "VIDEO", "type": "VIDEO", "links": [158]}],
|
||||||
|
"properties": {"Node name for S&R": "CreateVideo"},
|
||||||
|
"widgets_values": [30, "auto", "sRGB", "h264"],
|
||||||
|
"widgets_values_named": {"fps": 30, "bit_depth": "auto", "color_space": "sRGB", "codec": "h264"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 69,
|
||||||
|
"type": "SaveVideo",
|
||||||
|
"pos": [5174.9019386914315, 3158.8462584926756],
|
||||||
|
"size": [270, 249.875],
|
||||||
|
"flags": {},
|
||||||
|
"order": 10,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [{"name": "video", "type": "VIDEO", "link": 158}],
|
||||||
|
"outputs": [{"name": "video", "type": "VIDEO", "links": null}],
|
||||||
|
"properties": {"Node name for S&R": "SaveVideo"},
|
||||||
|
"widgets_values": ["video/ComfyUI", "auto", "auto", "auto"],
|
||||||
|
"widgets_values_named": {"filename_prefix": "video/ComfyUI", "format": "auto", "format.codec": "auto", "codec": "auto"}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"id": 81,
|
||||||
|
"type": "easy multiTrackEditor",
|
||||||
|
"pos": [2903.9332978768557, 2791.1687123187226],
|
||||||
|
"size": [800, 700],
|
||||||
|
"flags": {},
|
||||||
|
"order": 1,
|
||||||
|
"mode": 0,
|
||||||
|
"inputs": [
|
||||||
|
{"name": "prompt_override", "shape": 7, "type": "*", "link": null},
|
||||||
|
{"name": "image", "shape": 7, "type": "IMAGE", "link": null},
|
||||||
|
{"name": "audio", "shape": 7, "type": "AUDIO", "link": null},
|
||||||
|
{"name": "video", "shape": 7, "type": "VIDEO", "link": null}
|
||||||
|
],
|
||||||
|
"outputs": [
|
||||||
|
{"name": "TRACKS_INFO", "type": "TRACKS_INFO", "links": [166]},
|
||||||
|
{"name": "IMAGES", "shape": 6, "type": "IMAGE", "links": null},
|
||||||
|
{"name": "AUDIO", "shape": 6, "type": "AUDIO", "links": null},
|
||||||
|
{"name": "VIDEO", "shape": 6, "type": "VIDEO", "links": null}
|
||||||
|
],
|
||||||
|
"properties": {"easyMediaTimelineHeight": 700, "Node name for S&R": "easy multiTrackEditor"},
|
||||||
|
"widgets_values": [
|
||||||
|
"1920 x 1080 (16:9)",
|
||||||
|
"stretch",
|
||||||
|
"MiniMax",
|
||||||
|
"{\"muted\":false,\"volume_db\":0,\"task_markers\":[],\"task_overview\":false,\"tracks\":[{\"id\":\"ac9830d5-1ffc-46a7-99ac-19989de1ec17\",\"name\":\"Task 0\",\"type\":\"task\",\"task_mode\":\"default\",\"color\":\"var(--multitrack-task-bg)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[{\"id\":\"389405fd-d576-45ab-b917-a4a4bbda7153\",\"start_frame\":0,\"end_frame\":120,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"shot\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene1\"}},{\"id\":\"b02ca679-4bb1-4758-86cc-a29d95d7d13a\",\"start_frame\":120,\"end_frame\":240,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"context\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene2\"}}]},{\"id\":\"49025400-b968-42bd-9e01-fda75549ca52\",\"name\":\"Video 0\",\"type\":\"video\",\"color\":\"var(--primary)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[],\"audio_locked\":false}],\"total_length\":240,\"frame_rate\":24}"
|
||||||
|
],
|
||||||
|
"widgets_values_named": {
|
||||||
|
"resolution": "1920 x 1080 (16:9)",
|
||||||
|
"resolution.resize_method": "stretch",
|
||||||
|
"format": "MiniMax",
|
||||||
|
"track_data": "{\"muted\":false,\"volume_db\":0,\"task_markers\":[],\"task_overview\":false,\"tracks\":[{\"id\":\"ac9830d5-1ffc-46a7-99ac-19989de1ec17\",\"name\":\"Task 0\",\"type\":\"task\",\"task_mode\":\"default\",\"color\":\"var(--multitrack-task-bg)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[{\"id\":\"389405fd-d576-45ab-b917-a4a4bbda7153\",\"start_frame\":0,\"end_frame\":120,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"shot\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene1\"}},{\"id\":\"b02ca679-4bb1-4758-86cc-a29d95d7d13a\",\"start_frame\":120,\"end_frame\":240,\"color\":\"var(--multitrack-task-bg)\",\"content\":{\"media_type\":\"none\",\"task_mode\":\"default\",\"continuity_mode\":\"context\",\"ref_image_size\":\"match\",\"images\":[],\"muted\":false,\"volume_db\":0,\"user_prompt\":\"scene2\"}}]},{\"id\":\"49025400-b968-42bd-9e01-fda75549ca52\",\"name\":\"Video 0\",\"type\":\"video\",\"color\":\"var(--primary)\",\"muted\":false,\"solo\":false,\"volume_db\":0,\"locked\":false,\"segments\":[],\"audio_locked\":false}],\"total_length\":240,\"frame_rate\":24}"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"links": [
|
||||||
|
[136, 52, 1, 57, 2, "CLIP"],
|
||||||
|
[137, 52, 2, 57, 3, "VAE"],
|
||||||
|
[138, 52, 3, 57, 4, "VAE"],
|
||||||
|
[144, 48, 0, 57, 1, "MODEL"],
|
||||||
|
[148, 57, 0, 63, 0, "STRING"],
|
||||||
|
[149, 63, 1, 45, 1, "AUDIO"],
|
||||||
|
[150, 63, 2, 45, 2, "FLOAT"],
|
||||||
|
[151, 63, 0, 45, 0, "IMAGE"],
|
||||||
|
[153, 52, 0, 66, 0, "MODEL"],
|
||||||
|
[154, 66, 0, 48, 0, "MODEL"],
|
||||||
|
[156, 67, 0, 68, 0, "IMAGE"],
|
||||||
|
[157, 63, 1, 68, 1, "AUDIO"],
|
||||||
|
[159, 63, 2, 68, 2, "FLOAT"],
|
||||||
|
[166, 81, 0, 57, 0, "TRACKS_INFO"],
|
||||||
|
[103, 45, 0, 46, 0, "VIDEO"],
|
||||||
|
[155, 63, 0, 67, 0, "IMAGE"],
|
||||||
|
[158, 68, 0, 69, 0, "VIDEO"]
|
||||||
|
],
|
||||||
|
"groups": [],
|
||||||
|
"config": {},
|
||||||
|
"extra": {
|
||||||
|
"ue_links": [],
|
||||||
|
"ds": {"scale": 0.5131581182307068, "offset": [-2322.4149844419144, -2181.0583714987984]},
|
||||||
|
"links_added_by_ue": [],
|
||||||
|
"frontendVersion": "1.53.6",
|
||||||
|
"VHS_latentpreview": false,
|
||||||
|
"VHS_latentpreviewrate": 0,
|
||||||
|
"VHS_MetadataImage": true,
|
||||||
|
"VHS_KeepIntermediate": true
|
||||||
|
},
|
||||||
|
"version": 0.4
|
||||||
|
}
|
||||||
73
bench/moxie_bench.spec
Normal file
73
bench/moxie_bench.spec
Normal file
@ -0,0 +1,73 @@
|
|||||||
|
# -*- mode: python ; coding: utf-8 -*-
|
||||||
|
"""PyInstaller spec for the Moxie bench sidecar.
|
||||||
|
|
||||||
|
Build (on the target machine, inside the repo root):
|
||||||
|
|
||||||
|
pip install -r bench/requirements.txt pyinstaller
|
||||||
|
cd frontend && bun install && bun run build:bench:release && cd ..
|
||||||
|
pyinstaller bench/moxie_bench.spec
|
||||||
|
|
||||||
|
Output: dist/moxie-bench/ (onedir - ships moxie-bench(.exe) plus _internal/)
|
||||||
|
|
||||||
|
Tauri 2 sidecar wiring (see bench/tauri/README.md):
|
||||||
|
- copy dist/moxie-bench/ next to your Tauri binary as `moxie-bench/`
|
||||||
|
- point externalBin / or spawn `moxie-bench/moxie-bench --port <free-port>`
|
||||||
|
- poll http://127.0.0.1:<port>/api/health until {"ok": true}
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
repo_root = Path(SPECPATH).resolve().parent # bench/ -> repo root
|
||||||
|
|
||||||
|
hiddenimports = [
|
||||||
|
"uvicorn.logging",
|
||||||
|
"uvicorn.loops",
|
||||||
|
"uvicorn.loops.auto",
|
||||||
|
"uvicorn.protocols",
|
||||||
|
"uvicorn.protocols.http",
|
||||||
|
"uvicorn.protocols.http.auto",
|
||||||
|
"uvicorn.protocols.websockets",
|
||||||
|
"uvicorn.protocols.websockets.auto",
|
||||||
|
"uvicorn.lifespan",
|
||||||
|
"uvicorn.lifespan.on",
|
||||||
|
"anyio._backends._asyncio",
|
||||||
|
]
|
||||||
|
|
||||||
|
a = Analysis(
|
||||||
|
["bench/run.py"],
|
||||||
|
pathex=[str(repo_root / "bench")],
|
||||||
|
binaries=[],
|
||||||
|
datas=[
|
||||||
|
# the prebuilt bench UI + base workflow travel with the binary
|
||||||
|
(str(repo_root / "bench" / "ui-dist"), "bench/ui-dist"),
|
||||||
|
(str(repo_root / "bench" / "base_workflow.json"), "bench"),
|
||||||
|
],
|
||||||
|
hiddenimports=hiddenimports,
|
||||||
|
hookspath=[],
|
||||||
|
hooksconfig={},
|
||||||
|
runtime_hooks=[],
|
||||||
|
excludes=["torch", "comfy", "comfy_execution", "folder_paths"],
|
||||||
|
noarchive=False,
|
||||||
|
)
|
||||||
|
pyz = PYZ(a.pure)
|
||||||
|
|
||||||
|
exe = EXE(
|
||||||
|
pyz,
|
||||||
|
a.scripts,
|
||||||
|
[],
|
||||||
|
exclude_binaries=True,
|
||||||
|
name="moxie-bench",
|
||||||
|
debug=False,
|
||||||
|
bootloader_ignore_signals=False,
|
||||||
|
strip=False,
|
||||||
|
upx=False,
|
||||||
|
console=True, # sidecar logs are useful; Tauri can hide the window
|
||||||
|
)
|
||||||
|
coll = COLLECT(
|
||||||
|
exe,
|
||||||
|
a.binaries,
|
||||||
|
a.datas,
|
||||||
|
strip=False,
|
||||||
|
upx=False,
|
||||||
|
name="moxie-bench",
|
||||||
|
)
|
||||||
4
bench/requirements.txt
Normal file
4
bench/requirements.txt
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
fastapi>=0.110
|
||||||
|
uvicorn>=0.29
|
||||||
|
numpy>=1.26
|
||||||
|
pillow>=10.0
|
||||||
33
bench/run.py
Normal file
33
bench/run.py
Normal file
@ -0,0 +1,33 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Moxie bench entry point.
|
||||||
|
|
||||||
|
python bench/run.py --port 8787 # auto backend (mock here)
|
||||||
|
python bench/run.py --backend comfy --comfyui-url http://127.0.0.1:8188
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).resolve().parent))
|
||||||
|
|
||||||
|
from app.config import BenchConfig # noqa: E402
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> None:
|
||||||
|
import uvicorn
|
||||||
|
|
||||||
|
from app.server import create_app
|
||||||
|
|
||||||
|
cfg = BenchConfig.from_args()
|
||||||
|
backend = cfg.resolve_backend()
|
||||||
|
print(f"[moxie-bench] backend={backend} http://{cfg.host}:{cfg.port}")
|
||||||
|
if backend == "comfy":
|
||||||
|
print(f"[moxie-bench] driving ComfyUI at {cfg.comfyui_url}")
|
||||||
|
else:
|
||||||
|
print("[moxie-bench] self-contained mock renderer (ffmpeg + numpy)")
|
||||||
|
uvicorn.run(create_app(cfg), host=cfg.host, port=cfg.port, log_level="info")
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
main()
|
||||||
110
bench/tauri/README.md
Normal file
110
bench/tauri/README.md
Normal file
@ -0,0 +1,110 @@
|
|||||||
|
# Using the bench as a Tauri 2 sidecar
|
||||||
|
|
||||||
|
The bench is a self-contained Python HTTP server (FastAPI + uvicorn). Packaged
|
||||||
|
with PyInstaller it becomes an `moxie-bench` binary your Tauri 2 app spawns at
|
||||||
|
startup and talks to over `http://127.0.0.1:<port>`.
|
||||||
|
|
||||||
|
## 1. Build the sidecar
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in the MMnode repo
|
||||||
|
pip install -r bench/requirements.txt pyinstaller
|
||||||
|
cd frontend && bun install && bun run build:bench:release && cd ..
|
||||||
|
pyinstaller bench/moxie_bench.spec
|
||||||
|
# -> dist/moxie-bench/moxie-bench(.exe) + _internal/
|
||||||
|
```
|
||||||
|
|
||||||
|
Ship the whole `dist/moxie-bench/` folder (onedir layout: the binary needs
|
||||||
|
`_internal/` beside it).
|
||||||
|
|
||||||
|
## 2. Bundle it with the Tauri app
|
||||||
|
|
||||||
|
`tauri.conf.json` (resources approach — simplest and cross-platform):
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"bundle": {
|
||||||
|
"resources": {
|
||||||
|
"../MMnode/dist/moxie-bench/": "moxie-bench/"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
The folder lands in the app resource dir; resolve it at runtime with
|
||||||
|
`@tauri-apps/api/path` → `resourceDir()`.
|
||||||
|
|
||||||
|
## 3. Spawn and wait for health
|
||||||
|
|
||||||
|
Rust side (tauri plugin-shell or std::process):
|
||||||
|
|
||||||
|
```rust
|
||||||
|
use std::process::{Command, Stdio};
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
|
let sidecar = app.path().resource_dir()?.join("moxie-bench/moxie-bench");
|
||||||
|
let port: u16 = 8787; // pick a free port, pass it through
|
||||||
|
let mut child = Command::new(sidecar)
|
||||||
|
.arg("--port").arg(port.to_string())
|
||||||
|
.arg("--host").arg("127.0.0.1")
|
||||||
|
.arg("--backend").arg("auto") // mock when no ComfyUI is reachable
|
||||||
|
.stdout(Stdio::null())
|
||||||
|
.stderr(Stdio::null())
|
||||||
|
.spawn()?;
|
||||||
|
|
||||||
|
// poll until ready before loading the webview UI
|
||||||
|
async fn wait_healthy(port: u16) -> bool {
|
||||||
|
let url = format!("http://127.0.0.1:{port}/api/health");
|
||||||
|
for _ in 0..60 {
|
||||||
|
if let Ok(resp) = reqwest::get(&url).await {
|
||||||
|
if resp.status().is_success() { return true; }
|
||||||
|
}
|
||||||
|
tokio::time::sleep(Duration::from_millis(500)).await;
|
||||||
|
}
|
||||||
|
false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 4. Drive it from the webview
|
||||||
|
|
||||||
|
Everything the bench UI does is plain HTTP/WS on the same origin — the Tauri
|
||||||
|
webview can call the same endpoints:
|
||||||
|
|
||||||
|
| Action | Call |
|
||||||
|
|---|---|
|
||||||
|
| health/backend | `GET /api/health`, `GET /api/config` |
|
||||||
|
| base workflow + timeline | `GET /api/workflow` |
|
||||||
|
| render plan preview | `GET /api/plan` |
|
||||||
|
| queue a render | `POST /api/render {track_data, overrides}` |
|
||||||
|
| live progress | WebSocket `ws://127.0.0.1:<port>/api/ws` |
|
||||||
|
| job list / detail | `GET /api/jobs`, `GET /api/jobs/{id}` |
|
||||||
|
| play/save outputs | `GET /api/view?filename=...&type=output&job=<job_id>` |
|
||||||
|
|
||||||
|
`POST /api/render` body:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"track_data": { "tracks": [ ... editor JSON ... ] },
|
||||||
|
"overrides": {
|
||||||
|
"width": 1920, "height": 1080, "frame_rate": 24,
|
||||||
|
"steps": 4, "seed": 12345, "project_name": "MM-myproject",
|
||||||
|
"backend": "auto"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## 5. Backends
|
||||||
|
|
||||||
|
- `mock` — self-contained synthetic renderer (numpy + ffmpeg). Deterministic
|
||||||
|
per (prompt, seed); real mp4 + audio on both output branches. Use it to
|
||||||
|
develop the Tauri integration with zero model downloads.
|
||||||
|
- `comfy` — drives a running ComfyUI (`--comfyui-url`, default
|
||||||
|
`http://127.0.0.1:8188`) with the exact base workflow. Real generation, same
|
||||||
|
API surface. Start ComfyUI headless: `python main.py --listen 127.0.0.1`.
|
||||||
|
- `auto` (default) — comfy when reachable, else mock.
|
||||||
|
|
||||||
|
## 6. Shutdown
|
||||||
|
|
||||||
|
Kill the spawned process on app exit (Rust `child.kill()` / `Drop`), or send
|
||||||
|
SIGTERM. The server has no state beyond `work/` on disk; job history lives in
|
||||||
|
memory.
|
||||||
196
bench/tests/test_bench_contract.py
Normal file
196
bench/tests/test_bench_contract.py
Normal file
@ -0,0 +1,196 @@
|
|||||||
|
"""Bench contract tests: no ComfyUI, no torch required.
|
||||||
|
|
||||||
|
Run: python3 -m pytest bench/tests -q
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
BENCH_DIR = Path(__file__).resolve().parent.parent
|
||||||
|
sys.path.insert(0, str(BENCH_DIR))
|
||||||
|
|
||||||
|
from app.tracks import TrackDataError, build_render_plan # noqa: E402
|
||||||
|
from app.workflow import ( # noqa: E402
|
||||||
|
EDITOR_TYPE,
|
||||||
|
WorkflowError,
|
||||||
|
describe,
|
||||||
|
editor_track_data,
|
||||||
|
load_base_workflow,
|
||||||
|
parse_workflow_dict,
|
||||||
|
)
|
||||||
|
|
||||||
|
BASE = load_base_workflow(BENCH_DIR / "base_workflow.json")
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# workflow parsing
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def test_all_nine_stage_types_present():
|
||||||
|
types = set(BASE.stages)
|
||||||
|
assert {
|
||||||
|
"MoxieMultimediaLoader", "LoraLoaderModelOnly", "MoxiePreviewOverride",
|
||||||
|
EDITOR_TYPE, "MoxieProject", "MoxieCombine", "RTXVideoSuperResolution",
|
||||||
|
"CreateVideo",
|
||||||
|
} <= types
|
||||||
|
assert "CreateVideo#2" in types
|
||||||
|
|
||||||
|
|
||||||
|
def test_save_nodes_classified_by_branch():
|
||||||
|
assert set(BASE.save_nodes) == {"native", "upscaled"}
|
||||||
|
# native save is fed by CreateVideo #45, upscaled by #68
|
||||||
|
native_src = BASE.save_nodes["native"].link_inputs["video"][0]
|
||||||
|
upscaled_src = BASE.save_nodes["upscaled"].link_inputs["video"][0]
|
||||||
|
assert native_src == BASE.stages["CreateVideo"].node_id
|
||||||
|
assert upscaled_src == BASE.stages["CreateVideo#2"].node_id
|
||||||
|
|
||||||
|
|
||||||
|
def test_describe_summary():
|
||||||
|
d = describe(BASE)
|
||||||
|
assert d["editor_resolution"] == {"width": 1920, "height": 1080}
|
||||||
|
assert d["frame_rate"] == 24
|
||||||
|
assert d["steps"] == 4
|
||||||
|
assert d["seed"] == 803100406447983
|
||||||
|
assert d["project_name"] == "MM-girlwaving"
|
||||||
|
assert d["upscale"]["target"] == {"width": 1920, "height": 1080}
|
||||||
|
assert d["upscale"]["quality"] == "ULTRA"
|
||||||
|
assert d["lora"].endswith(".safetensors")
|
||||||
|
|
||||||
|
|
||||||
|
def test_editor_track_data_roundtrip():
|
||||||
|
td = editor_track_data(BASE)
|
||||||
|
assert td["frame_rate"] == 24
|
||||||
|
assert td["total_length"] == 240
|
||||||
|
task_tracks = [t for t in td["tracks"] if t["type"] == "task"]
|
||||||
|
assert len(task_tracks) == 1
|
||||||
|
assert len(task_tracks[0]["segments"]) == 2
|
||||||
|
|
||||||
|
|
||||||
|
def test_missing_stage_rejected():
|
||||||
|
raw = json.loads((BENCH_DIR / "base_workflow.json").read_text())
|
||||||
|
raw["nodes"] = [n for n in raw["nodes"] if n["type"] != "MoxieProject"]
|
||||||
|
with pytest.raises(WorkflowError):
|
||||||
|
parse_workflow_dict(raw)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# track_data -> RenderPlan
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _plan(**kw):
|
||||||
|
defaults = dict(width=1920, height=1080, frame_rate=24, steps=4, seed=7,
|
||||||
|
project_name="MM-girlwaving", upscale_width=1920,
|
||||||
|
upscale_height=1080)
|
||||||
|
defaults.update(kw)
|
||||||
|
return build_render_plan(editor_track_data(BASE), **defaults)
|
||||||
|
|
||||||
|
|
||||||
|
def test_plan_segments_ordered_and_bounded():
|
||||||
|
plan = _plan()
|
||||||
|
assert [(s.prompt, s.start_frame, s.end_frame) for s in plan.segments] == [
|
||||||
|
("scene1", 0, 120), ("scene2", 120, 240)]
|
||||||
|
assert plan.duration_seconds == pytest.approx(10.0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_plan_honors_overrides():
|
||||||
|
plan = _plan(width=1088, height=1920, steps=8, seed=99)
|
||||||
|
assert (plan.width, plan.height) == (1088, 1920)
|
||||||
|
assert plan.steps == 8 and plan.seed == 99
|
||||||
|
|
||||||
|
|
||||||
|
def test_continuity_mode_carried_through():
|
||||||
|
plan = _plan()
|
||||||
|
modes = {s.prompt: s.continuity_mode for s in plan.segments}
|
||||||
|
assert modes == {"scene1": "shot", "scene2": "context"}
|
||||||
|
|
||||||
|
|
||||||
|
def test_invalid_track_data_raises():
|
||||||
|
with pytest.raises(TrackDataError):
|
||||||
|
build_render_plan("not json", width=1, height=1)
|
||||||
|
with pytest.raises(TrackDataError):
|
||||||
|
build_render_plan({"nope": 1}, width=1, height=1)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# /prompt conversion (comfy bridge)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
def test_prompt_conversion_shape():
|
||||||
|
from app.backends.comfy import to_prompt_format
|
||||||
|
|
||||||
|
prompt = to_prompt_format(BASE.raw)
|
||||||
|
assert set(prompt) == {str(n["id"]) for n in BASE.raw["nodes"]}
|
||||||
|
loader = prompt["52"]
|
||||||
|
assert loader["class_type"] == "MoxieMultimediaLoader"
|
||||||
|
assert loader["inputs"]["diffusion_model"] == "Moxie-Multimedia.safetensors"
|
||||||
|
project = prompt["57"]
|
||||||
|
assert project["inputs"]["tracks_info"] == ["81", 0]
|
||||||
|
assert project["inputs"]["model"] == ["48", 0]
|
||||||
|
assert project["inputs"]["clip"] == ["52", 1]
|
||||||
|
assert "randomize" not in project["inputs"] # UI-only control widget
|
||||||
|
|
||||||
|
|
||||||
|
def test_prompt_conversion_wired_widget_wins():
|
||||||
|
from app.backends.comfy import to_prompt_format
|
||||||
|
|
||||||
|
prompt = to_prompt_format(BASE.raw)
|
||||||
|
# CreateVideo #45: fps widget is wired to MoxieCombine slot 2
|
||||||
|
create = prompt["45"]
|
||||||
|
assert create["inputs"]["fps"] == ["63", 2]
|
||||||
|
assert create["inputs"]["codec"] == "h264"
|
||||||
|
|
||||||
|
|
||||||
|
def test_substitution_updates_all_parameters():
|
||||||
|
from app.backends.comfy import substitute
|
||||||
|
|
||||||
|
plan = build_render_plan(
|
||||||
|
editor_track_data(BASE), width=720, height=1280, frame_rate=16,
|
||||||
|
steps=6, seed=1234, project_name="MM-bench", lora="MM3step.safetensors",
|
||||||
|
lora_strength=0.8, upscale_width=1440, upscale_height=2560,
|
||||||
|
)
|
||||||
|
raw = substitute(BASE, plan)
|
||||||
|
nodes = {n["id"]: n for n in raw["nodes"]}
|
||||||
|
|
||||||
|
editor = nodes[81]["widgets_values_named"]
|
||||||
|
assert json.loads(editor["track_data"])["frame_rate"] == 16
|
||||||
|
assert editor["resolution"] == "720 x 1280 (9:16)"
|
||||||
|
|
||||||
|
proj = nodes[57]["widgets_values_named"]
|
||||||
|
assert proj["project_name"] == "MM-bench" and proj["steps"] == 6
|
||||||
|
assert proj["seed"] == 1234
|
||||||
|
|
||||||
|
lora = nodes[66]["widgets_values_named"]
|
||||||
|
assert lora["lora_name"] == "MM3step.safetensors" and lora["strength_model"] == 0.8
|
||||||
|
|
||||||
|
rtx = nodes[67]["widgets_values_named"]
|
||||||
|
assert rtx["resize_type.width"] == 1440 and rtx["resize_type.height"] == 2560
|
||||||
|
|
||||||
|
combine = nodes[63]["widgets_values_named"]
|
||||||
|
pd = json.loads(combine["project_data"])
|
||||||
|
assert pd["width"] == 720 and pd["height"] == 1280 and pd["frame_rate"] == 16
|
||||||
|
|
||||||
|
saves = {n["id"]: n for n in raw["nodes"] if n["type"] == "SaveVideo"}
|
||||||
|
assert saves[46]["widgets_values_named"]["filename_prefix"] == "moxie-bench/native"
|
||||||
|
assert saves[69]["widgets_values_named"]["filename_prefix"] == "moxie-bench/upscaled"
|
||||||
|
|
||||||
|
|
||||||
|
def test_substitution_custom_resolution_uses_dict():
|
||||||
|
from app.backends.comfy import substitute
|
||||||
|
|
||||||
|
plan = build_render_plan(editor_track_data(BASE), width=998, height=742)
|
||||||
|
raw = substitute(BASE, plan)
|
||||||
|
editor = next(n for n in raw["nodes"] if n["id"] == 81)
|
||||||
|
assert editor["widgets_values_named"]["resolution"] == {"width": 998, "height": 742}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# mock backend end-to-end (ffmpeg required)
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
@pytest.mark.skipif(True, reason="e2e covered by scripts/test_bench_pipeline.py; keep pytest fast")
|
||||||
|
def test_mock_render_placeholder():
|
||||||
|
pass
|
||||||
16
bench/ui-dist/index.html
Normal file
16
bench/ui-dist/index.html
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||||
|
<title>Moxie Multimedia Bench</title>
|
||||||
|
<style>
|
||||||
|
html, body, #root { height: 100%; margin: 0; }
|
||||||
|
body { background: #09090b; color: #fafafa; }
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="root"></div>
|
||||||
|
<script type="module" src="./main.js"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
71
bench/ui-dist/main.js
Normal file
71
bench/ui-dist/main.js
Normal file
File diff suppressed because one or more lines are too long
BIN
bench/work/outputs/7017c50576ec/native.mp4
Normal file
BIN
bench/work/outputs/7017c50576ec/native.mp4
Normal file
Binary file not shown.
BIN
bench/work/outputs/7017c50576ec/upscaled.mp4
Normal file
BIN
bench/work/outputs/7017c50576ec/upscaled.mp4
Normal file
Binary file not shown.
BIN
bench/work/outputs/8647494f0d4e/native.mp4
Normal file
BIN
bench/work/outputs/8647494f0d4e/native.mp4
Normal file
Binary file not shown.
BIN
bench/work/outputs/f3acb27415ad/native.mp4
Normal file
BIN
bench/work/outputs/f3acb27415ad/native.mp4
Normal file
Binary file not shown.
BIN
bench/work/outputs/f3acb27415ad/upscaled.mp4
Normal file
BIN
bench/work/outputs/f3acb27415ad/upscaled.mp4
Normal file
Binary file not shown.
103
frontend/build.bench.ts
Normal file
103
frontend/build.bench.ts
Normal file
@ -0,0 +1,103 @@
|
|||||||
|
/**
|
||||||
|
* Bench UI build — mirrors build.ts but outputs a standalone web app into
|
||||||
|
* ../bench/ui-dist (served by the bench FastAPI server).
|
||||||
|
*
|
||||||
|
* bun run build.bench.ts # dev build
|
||||||
|
* bun run build.bench.ts --release # minified
|
||||||
|
*/
|
||||||
|
import { $ } from "bun";
|
||||||
|
import path from "path";
|
||||||
|
import fs from "fs";
|
||||||
|
import os from "os";
|
||||||
|
|
||||||
|
const isRelease = Bun.argv.includes("--release");
|
||||||
|
|
||||||
|
const rootDir = path.resolve(import.meta.dir, "..");
|
||||||
|
const outdir = path.join(rootDir, "bench", "ui-dist");
|
||||||
|
const project_name = path.basename(rootDir);
|
||||||
|
|
||||||
|
fs.mkdirSync(outdir, { recursive: true });
|
||||||
|
|
||||||
|
function scopeTailwindThemeVariables(css: string, widgetClass: string): string {
|
||||||
|
return css.replace(/:root\s*,\s*:host\s*\{/g, `.${widgetClass}{`);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function buildCSS(): Promise<string> {
|
||||||
|
const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), "moxie-bench-css-"));
|
||||||
|
const cssOut = path.join(tempDir, "globals.css");
|
||||||
|
const minifyFlag = isRelease ? "--minify" : "";
|
||||||
|
try {
|
||||||
|
await $`./node_modules/.bin/tailwindcss -i ./src/styles/globals.css -o ${cssOut} ${minifyFlag}`;
|
||||||
|
const widgetClass = project_name.toLowerCase();
|
||||||
|
const css = fs.readFileSync(cssOut, "utf-8");
|
||||||
|
return scopeTailwindThemeVariables(
|
||||||
|
css.replaceAll("__CUSTOM_NODE_CLASS__", widgetClass),
|
||||||
|
widgetClass,
|
||||||
|
);
|
||||||
|
} finally {
|
||||||
|
fs.rmSync(tempDir, { recursive: true, force: true });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function build() {
|
||||||
|
const start = Date.now();
|
||||||
|
const globalCss = await buildCSS();
|
||||||
|
|
||||||
|
const result = await Bun.build({
|
||||||
|
entrypoints: ["./src/bench/main.tsx"],
|
||||||
|
outdir,
|
||||||
|
naming: {
|
||||||
|
chunk: "chunks/[name]-[hash].[ext]",
|
||||||
|
entry: "[name].[ext]",
|
||||||
|
asset: "assets/[name].[ext]",
|
||||||
|
},
|
||||||
|
splitting: true,
|
||||||
|
target: "browser",
|
||||||
|
minify: isRelease,
|
||||||
|
define: {
|
||||||
|
"process.env.NODE_ENV": isRelease ? '"production"' : '"development"',
|
||||||
|
"import.meta.env.PROJECT_NAME": `"${project_name}"`,
|
||||||
|
"__COMFY_EASY_MEDIA_GLOBAL_CSS__": JSON.stringify(globalCss),
|
||||||
|
},
|
||||||
|
css: false,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!result.success) {
|
||||||
|
for (const log of result.logs) {
|
||||||
|
console.error(log);
|
||||||
|
}
|
||||||
|
throw new Error("bench UI build failed");
|
||||||
|
}
|
||||||
|
|
||||||
|
// index.html referencing the built entry
|
||||||
|
const entryName = result.outputs
|
||||||
|
.map((o) => o.path)
|
||||||
|
.filter((p) => p.endsWith("main.js") || p.endsWith("main.js.map"))
|
||||||
|
.map((p) => path.basename(p))
|
||||||
|
.find((p) => p.endsWith(".js"));
|
||||||
|
|
||||||
|
const html = `<!doctype html>
|
||||||
|
<html lang="en">
|
||||||
|
<head>
|
||||||
|
<meta charset="utf-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||||
|
<title>Moxie Multimedia Bench</title>
|
||||||
|
<style>
|
||||||
|
html, body, #root { height: 100%; margin: 0; }
|
||||||
|
body { background: #09090b; color: #fafafa; }
|
||||||
|
</style>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<div id="root"></div>
|
||||||
|
<script type="module" src="./${entryName ?? "main.js"}"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
|
`;
|
||||||
|
fs.writeFileSync(path.join(outdir, "index.html"), html);
|
||||||
|
|
||||||
|
console.log(
|
||||||
|
`[build:bench${isRelease ? ":release" : ""}] Done in ${Date.now() - start}ms (${result.outputs.length} outputs) -> ${outdir}`,
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
await build();
|
||||||
@ -8,7 +8,9 @@
|
|||||||
"build": "bun run build.ts",
|
"build": "bun run build.ts",
|
||||||
"build:release": "bun run build.ts --release",
|
"build:release": "bun run build.ts --release",
|
||||||
"test": "vitest run",
|
"test": "vitest run",
|
||||||
"test:watch": "vitest"
|
"test:watch": "vitest",
|
||||||
|
"build:bench": "bun run build.bench.ts",
|
||||||
|
"build:bench:release": "bun run build.bench.ts --release"
|
||||||
},
|
},
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@base-ui/react": "^1.6.0",
|
"@base-ui/react": "^1.6.0",
|
||||||
|
|||||||
268
frontend/src/bench/BenchApp.tsx
Normal file
268
frontend/src/bench/BenchApp.tsx
Normal file
@ -0,0 +1,268 @@
|
|||||||
|
import { useEffect, useMemo, useReducer, useRef, useState } from 'react'
|
||||||
|
import { Loader2, Play, Radio, SquareTerminal } from 'lucide-react'
|
||||||
|
import { Button } from '@/components/ui/button'
|
||||||
|
import { Input } from '@/components/ui/input'
|
||||||
|
import { TooltipProvider } from '@/components/ui/tooltip'
|
||||||
|
import { MultiTrackWidget } from '@/components/widgets/MultiTrackWidget'
|
||||||
|
import type { TrackData } from '@/types/multitrack'
|
||||||
|
import {
|
||||||
|
benchApp,
|
||||||
|
benchState,
|
||||||
|
connectBenchSocket,
|
||||||
|
createBenchNode,
|
||||||
|
createBenchWidget,
|
||||||
|
fetchBenchConfig,
|
||||||
|
onBenchStateChange,
|
||||||
|
queueBenchRender,
|
||||||
|
type BenchOutputInfo,
|
||||||
|
} from './bench-runtime'
|
||||||
|
|
||||||
|
const RESOLUTION_PRESETS: Array<{ label: string; width: number; height: number }> = [
|
||||||
|
{ label: '1920 x 1080 (16:9)', width: 1920, height: 1080 },
|
||||||
|
{ label: '1088 x 1920 (9:16)', width: 1088, height: 1920 },
|
||||||
|
{ label: '1280 x 720 (16:9)', width: 1280, height: 720 },
|
||||||
|
{ label: '720 x 1280 (9:16)', width: 720, height: 1280 },
|
||||||
|
{ label: '832 x 480 (16:9)', width: 832, height: 480 },
|
||||||
|
{ label: '480 x 832 (9:16)', width: 480, height: 832 },
|
||||||
|
{ label: '1024 x 1024 (1:1)', width: 1024, height: 1024 },
|
||||||
|
]
|
||||||
|
|
||||||
|
function useBenchTick(): number {
|
||||||
|
const [, force] = useReducer((x: number) => x + 1, 0)
|
||||||
|
useEffect(() => onBenchStateChange(force), [force])
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
function OutputCard({ label, out }: { label: string; out: BenchOutputInfo }) {
|
||||||
|
return (
|
||||||
|
<div className="flex min-w-0 flex-1 flex-col gap-1 rounded-md border bg-background p-2">
|
||||||
|
<div className="flex items-center justify-between text-xs text-muted-foreground">
|
||||||
|
<span className="font-medium text-foreground">{label}</span>
|
||||||
|
<span>
|
||||||
|
{out.width}x{out.height} · {out.fps ?? '?'} fps · {Math.round((out.bytes ?? 0) / 1024)} KB
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<video className="w-full rounded-sm" src={out.url} controls preload="metadata" />
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
export function BenchApp() {
|
||||||
|
useBenchTick()
|
||||||
|
const [trackData, setTrackData] = useState<TrackData | null>(null)
|
||||||
|
const [config, setConfig] = useState<Awaited<ReturnType<typeof fetchBenchConfig>>>(null)
|
||||||
|
const [backend, setBackend] = useState('auto')
|
||||||
|
const [resolution, setResolution] = useState('1920 x 1080 (16:9)')
|
||||||
|
const [steps, setSteps] = useState(4)
|
||||||
|
const [seed, setSeed] = useState(803100406447983)
|
||||||
|
const [projectName, setProjectName] = useState('MM-girlwaving')
|
||||||
|
const [queueing, setQueueing] = useState(false)
|
||||||
|
const node = useMemo(createBenchNode, [])
|
||||||
|
const widget = useMemo(createBenchWidget, [])
|
||||||
|
const consoleRef = useRef<HTMLDivElement>(null)
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
connectBenchSocket()
|
||||||
|
void fetchBenchConfig().then(setConfig)
|
||||||
|
fetch('/api/workflow')
|
||||||
|
.then((r) => r.json())
|
||||||
|
.then((body: { track_data: string }) => {
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(body.track_data) as TrackData
|
||||||
|
setTrackData(parsed)
|
||||||
|
setProjectName((current) => current || 'MM-girlwaving')
|
||||||
|
} catch {
|
||||||
|
/* server sent unusable track data; keep the empty editor */
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.catch(() => undefined)
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
if (consoleRef.current) consoleRef.current.scrollTop = consoleRef.current.scrollHeight
|
||||||
|
})
|
||||||
|
|
||||||
|
const preset = RESOLUTION_PRESETS.find((p) => p.label === resolution)
|
||||||
|
|
||||||
|
const queue = async () => {
|
||||||
|
if (!trackData || queueing) return
|
||||||
|
setQueueing(true)
|
||||||
|
try {
|
||||||
|
await queueBenchRender({
|
||||||
|
track_data: trackData,
|
||||||
|
backend: backend === 'auto' ? undefined : backend,
|
||||||
|
overrides: {
|
||||||
|
width: preset?.width ?? 1920,
|
||||||
|
height: preset?.height ?? 1080,
|
||||||
|
steps,
|
||||||
|
seed,
|
||||||
|
project_name: projectName,
|
||||||
|
},
|
||||||
|
})
|
||||||
|
} catch (error) {
|
||||||
|
benchState.error = error instanceof Error ? error.message : String(error)
|
||||||
|
} finally {
|
||||||
|
setQueueing(false)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
const busy = benchState.jobStatus === 'queued' || benchState.jobStatus === 'running'
|
||||||
|
const outputs = benchState.outputs ?? {}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<TooltipProvider>
|
||||||
|
<div className="mmnode flex h-screen min-h-0 flex-col gap-2 bg-background p-3 text-foreground">
|
||||||
|
<header className="flex flex-wrap items-center gap-3">
|
||||||
|
<h1 className="text-lg font-semibold">Moxie Multimedia Bench</h1>
|
||||||
|
<span
|
||||||
|
className={`inline-flex items-center gap-1 rounded-full px-2 py-0.5 text-xs ${
|
||||||
|
benchState.connected
|
||||||
|
? 'bg-green-500/15 text-green-600'
|
||||||
|
: 'bg-red-500/15 text-red-600'
|
||||||
|
}`}
|
||||||
|
>
|
||||||
|
<Radio size={12} /> {benchState.connected ? 'server online' : 'offline'}
|
||||||
|
</span>
|
||||||
|
<span className="rounded-full bg-muted px-2 py-0.5 text-xs text-muted-foreground">
|
||||||
|
backend: {config?.backend ?? benchState.backend}
|
||||||
|
</span>
|
||||||
|
<span className="text-xs text-muted-foreground">v{config?.version ?? '?'}</span>
|
||||||
|
<div className="ml-auto" />
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<div className="flex min-h-0 flex-1 gap-3">
|
||||||
|
<main className="mmnode flex min-h-0 min-w-0 flex-1 overflow-hidden rounded-md border">
|
||||||
|
{trackData ? (
|
||||||
|
<MultiTrackWidget
|
||||||
|
value={trackData}
|
||||||
|
onChange={(next) => setTrackData(next)}
|
||||||
|
inputName="track_data"
|
||||||
|
node={node}
|
||||||
|
widget={widget as never}
|
||||||
|
app={benchApp}
|
||||||
|
/>
|
||||||
|
) : (
|
||||||
|
<div className="flex h-full w-full items-center justify-center text-muted-foreground">
|
||||||
|
<Loader2 className="mr-2 animate-spin" size={18} /> loading timeline…
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</main>
|
||||||
|
|
||||||
|
<aside className="flex w-80 shrink-0 flex-col gap-3 overflow-y-auto rounded-md border p-3">
|
||||||
|
<div className="text-sm font-semibold">Render</div>
|
||||||
|
|
||||||
|
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||||
|
Resolution
|
||||||
|
<select
|
||||||
|
className="rounded-md border bg-background p-1.5 text-foreground"
|
||||||
|
value={resolution}
|
||||||
|
onChange={(e) => setResolution(e.target.value)}
|
||||||
|
>
|
||||||
|
{RESOLUTION_PRESETS.map((p) => (
|
||||||
|
<option key={p.label} value={p.label}>
|
||||||
|
{p.label}
|
||||||
|
</option>
|
||||||
|
))}
|
||||||
|
</select>
|
||||||
|
</label>
|
||||||
|
|
||||||
|
<div className="grid grid-cols-2 gap-2">
|
||||||
|
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||||
|
Steps
|
||||||
|
<Input
|
||||||
|
type="number"
|
||||||
|
min={1}
|
||||||
|
max={32}
|
||||||
|
value={steps}
|
||||||
|
onChange={(e) => setSteps(Math.max(1, Number(e.target.value) || 1))}
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||||
|
Seed
|
||||||
|
<Input
|
||||||
|
type="number"
|
||||||
|
value={seed}
|
||||||
|
onChange={(e) => setSeed(Number(e.target.value) || 0)}
|
||||||
|
/>
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||||
|
Project name
|
||||||
|
<Input value={projectName} onChange={(e) => setProjectName(e.target.value)} />
|
||||||
|
</label>
|
||||||
|
|
||||||
|
<label className="flex flex-col gap-1 text-xs text-muted-foreground">
|
||||||
|
Backend
|
||||||
|
<select
|
||||||
|
className="rounded-md border bg-background p-1.5 text-foreground"
|
||||||
|
value={backend}
|
||||||
|
onChange={(e) => setBackend(e.target.value)}
|
||||||
|
>
|
||||||
|
<option value="auto">auto</option>
|
||||||
|
<option value="mock">mock</option>
|
||||||
|
<option value="comfy">comfy (needs running ComfyUI)</option>
|
||||||
|
</select>
|
||||||
|
{config && !config.backends?.comfy?.available && (
|
||||||
|
<span className="text-[11px] text-muted-foreground">
|
||||||
|
comfy bridge not reachable — mock stays the default
|
||||||
|
</span>
|
||||||
|
)}
|
||||||
|
</label>
|
||||||
|
|
||||||
|
<Button onClick={queue} disabled={busy || queueing || !trackData}>
|
||||||
|
{busy ? <Loader2 className="mr-2 animate-spin" size={14} /> : <Play className="mr-2" size={14} />}
|
||||||
|
{busy ? 'Rendering…' : 'Queue render'}
|
||||||
|
</Button>
|
||||||
|
|
||||||
|
{busy && (
|
||||||
|
<div className="flex flex-col gap-1">
|
||||||
|
<div className="h-2 w-full overflow-hidden rounded-full bg-muted">
|
||||||
|
<div
|
||||||
|
className="h-full rounded-full bg-primary transition-all"
|
||||||
|
style={{ width: `${benchState.progress}%` }}
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
<div className="text-xs text-muted-foreground">
|
||||||
|
{benchState.stage || 'waiting…'} — {benchState.message}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
{benchState.error && (
|
||||||
|
<div className="rounded-md bg-red-500/10 p-2 text-xs text-red-600">
|
||||||
|
{benchState.error}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
|
||||||
|
<div className="text-sm font-semibold">Outputs</div>
|
||||||
|
<div className="flex flex-col gap-2">
|
||||||
|
{outputs.native ? (
|
||||||
|
<OutputCard label="native (MoxieCombine)" out={outputs.native} />
|
||||||
|
) : null}
|
||||||
|
{outputs.upscaled ? (
|
||||||
|
<OutputCard label="upscaled (RTX branch)" out={outputs.upscaled} />
|
||||||
|
) : null}
|
||||||
|
{!outputs.native && !outputs.upscaled && (
|
||||||
|
<div className="text-xs text-muted-foreground">
|
||||||
|
queue a render — native and RTX-upscaled outputs appear here
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div className="flex items-center gap-1 text-sm font-semibold">
|
||||||
|
<SquareTerminal size={14} /> Console
|
||||||
|
</div>
|
||||||
|
<div
|
||||||
|
ref={consoleRef}
|
||||||
|
className="h-40 overflow-y-auto rounded-md bg-muted/50 p-2 font-mono text-[11px] leading-relaxed"
|
||||||
|
>
|
||||||
|
{benchState.console.map((line, i) => (
|
||||||
|
<div key={i}>{line}</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
</aside>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</TooltipProvider>
|
||||||
|
)
|
||||||
|
}
|
||||||
314
frontend/src/bench/bench-runtime.ts
Normal file
314
frontend/src/bench/bench-runtime.ts
Normal file
@ -0,0 +1,314 @@
|
|||||||
|
/**
|
||||||
|
* Bench runtime: the ComfyUI-shaped shim that lets the unmodified editor
|
||||||
|
* components run against the bench server.
|
||||||
|
*
|
||||||
|
* This module MUST be imported before anything that pulls in `@/lib/comfy-api`
|
||||||
|
* (that module reads `globalThis.comfyAPI.app.app` at import time).
|
||||||
|
*/
|
||||||
|
import type { ComfyApp } from '@comfyorg/comfyui-frontend-types'
|
||||||
|
|
||||||
|
type CustomEventCallback = (event: CustomEvent<unknown>) => void
|
||||||
|
|
||||||
|
export interface BenchJobEvent {
|
||||||
|
type: string
|
||||||
|
job_id?: string
|
||||||
|
status?: string
|
||||||
|
stage?: string
|
||||||
|
progress?: number
|
||||||
|
message?: string
|
||||||
|
error?: string
|
||||||
|
outputs?: Record<string, BenchOutputInfo>
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface BenchOutputInfo {
|
||||||
|
filename: string
|
||||||
|
url: string
|
||||||
|
width?: number
|
||||||
|
height?: number
|
||||||
|
fps?: number
|
||||||
|
bytes?: number
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface BenchConfigInfo {
|
||||||
|
version: string
|
||||||
|
backend: string
|
||||||
|
backends: Record<string, { available: boolean; url?: string }>
|
||||||
|
}
|
||||||
|
|
||||||
|
class Hub {
|
||||||
|
private target = new EventTarget()
|
||||||
|
|
||||||
|
add(type: string, cb: CustomEventCallback): void {
|
||||||
|
this.target.addEventListener(type, cb as EventListener)
|
||||||
|
}
|
||||||
|
|
||||||
|
remove(type: string, cb: CustomEventCallback): void {
|
||||||
|
this.target.removeEventListener(type, cb as EventListener)
|
||||||
|
}
|
||||||
|
|
||||||
|
dispatch(type: string, detail: unknown): void {
|
||||||
|
this.target.dispatchEvent(new CustomEvent(type, { detail }))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export const benchHub = new Hub()
|
||||||
|
|
||||||
|
/** Live bench state the dock UI reads. */
|
||||||
|
export const benchState = {
|
||||||
|
connected: false,
|
||||||
|
backend: 'mock',
|
||||||
|
jobId: null as string | null,
|
||||||
|
jobStatus: 'idle',
|
||||||
|
stage: '',
|
||||||
|
progress: 0,
|
||||||
|
message: '',
|
||||||
|
error: null as string | null,
|
||||||
|
outputs: {} as Record<string, BenchOutputInfo>,
|
||||||
|
console: [] as string[],
|
||||||
|
}
|
||||||
|
|
||||||
|
const listeners = new Set<() => void>()
|
||||||
|
|
||||||
|
export function onBenchStateChange(cb: () => void): () => void {
|
||||||
|
listeners.add(cb)
|
||||||
|
return () => {
|
||||||
|
listeners.delete(cb)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function notify(): void {
|
||||||
|
listeners.forEach((cb) => cb())
|
||||||
|
}
|
||||||
|
|
||||||
|
function log(line: string): void {
|
||||||
|
benchState.console.push(`[${new Date().toLocaleTimeString()}] ${line}`)
|
||||||
|
if (benchState.console.length > 400) benchState.console.splice(0, benchState.console.length - 400)
|
||||||
|
notify()
|
||||||
|
}
|
||||||
|
|
||||||
|
function applyJobEvent(msg: BenchJobEvent): void {
|
||||||
|
benchState.jobId = msg.job_id ?? benchState.jobId
|
||||||
|
benchState.jobStatus = msg.status ?? benchState.jobStatus
|
||||||
|
benchState.stage = msg.stage ?? benchState.stage
|
||||||
|
benchState.progress = typeof msg.progress === 'number' ? msg.progress : benchState.progress
|
||||||
|
benchState.message = msg.message ?? benchState.message
|
||||||
|
if (msg.type === 'execution_error') {
|
||||||
|
benchState.error = msg.error ?? 'render failed'
|
||||||
|
log(`ERROR: ${benchState.error}`)
|
||||||
|
}
|
||||||
|
if (msg.type === 'execution_success' && msg.outputs) {
|
||||||
|
benchState.outputs = msg.outputs
|
||||||
|
benchState.error = null
|
||||||
|
log(`done: ${Object.keys(msg.outputs).join(', ')}`)
|
||||||
|
}
|
||||||
|
if (msg.type === 'job_progress') {
|
||||||
|
benchState.error = null
|
||||||
|
}
|
||||||
|
// Mirror ComfyUI's push-event model so any widget listening on app.api
|
||||||
|
// (executed / execution_success / execution_error) keeps working.
|
||||||
|
if (msg.type === 'execution_start') {
|
||||||
|
benchHub.dispatch('execution_start', { prompt_id: benchState.jobId })
|
||||||
|
} else if (msg.type === 'execution_success') {
|
||||||
|
benchHub.dispatch('executed', {
|
||||||
|
prompt_id: benchState.jobId,
|
||||||
|
output: msg.outputs,
|
||||||
|
})
|
||||||
|
benchHub.dispatch('execution_success', { prompt_id: benchState.jobId })
|
||||||
|
} else if (msg.type === 'execution_error') {
|
||||||
|
benchHub.dispatch('execution_error', {
|
||||||
|
prompt_id: benchState.jobId,
|
||||||
|
exception_message: benchState.error,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
notify()
|
||||||
|
}
|
||||||
|
|
||||||
|
let socket: WebSocket | null = null
|
||||||
|
let reconnectTimer: number | null = null
|
||||||
|
let pingTimer: number | null = null
|
||||||
|
|
||||||
|
export function connectBenchSocket(): void {
|
||||||
|
if (typeof window === 'undefined') return
|
||||||
|
const proto = window.location.protocol === 'https:' ? 'wss' : 'ws'
|
||||||
|
const url = `${proto}://${window.location.host}/api/ws`
|
||||||
|
|
||||||
|
const open = () => {
|
||||||
|
socket = new WebSocket(url)
|
||||||
|
socket.onopen = () => {
|
||||||
|
benchState.connected = true
|
||||||
|
log('connected to bench server')
|
||||||
|
notify()
|
||||||
|
pingTimer = window.setInterval(() => {
|
||||||
|
try {
|
||||||
|
socket?.send('ping')
|
||||||
|
} catch {
|
||||||
|
/* ignore */
|
||||||
|
}
|
||||||
|
}, 20_000)
|
||||||
|
}
|
||||||
|
socket.onmessage = (event) => {
|
||||||
|
try {
|
||||||
|
const msg = JSON.parse(event.data as string) as BenchJobEvent
|
||||||
|
applyJobEvent(msg)
|
||||||
|
} catch {
|
||||||
|
/* non-JSON frames are keepalive noise */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
socket.onclose = () => {
|
||||||
|
benchState.connected = false
|
||||||
|
socket = null
|
||||||
|
if (pingTimer) window.clearInterval(pingTimer)
|
||||||
|
notify()
|
||||||
|
reconnectTimer = window.setTimeout(open, 1500)
|
||||||
|
}
|
||||||
|
socket.onerror = () => socket?.close()
|
||||||
|
}
|
||||||
|
open()
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function fetchBenchConfig(): Promise<BenchConfigInfo | null> {
|
||||||
|
try {
|
||||||
|
const res = await fetch('/api/config')
|
||||||
|
if (!res.ok) return null
|
||||||
|
const cfg = (await res.json()) as BenchConfigInfo
|
||||||
|
benchState.backend = cfg.backend
|
||||||
|
notify()
|
||||||
|
return cfg
|
||||||
|
} catch {
|
||||||
|
return null
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export interface BenchRenderPayload {
|
||||||
|
track_data: unknown
|
||||||
|
overrides?: Record<string, unknown>
|
||||||
|
backend?: string
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function queueBenchRender(payload: BenchRenderPayload): Promise<string> {
|
||||||
|
const res = await fetch('/api/render', {
|
||||||
|
method: 'POST',
|
||||||
|
headers: { 'Content-Type': 'application/json' },
|
||||||
|
body: JSON.stringify(payload),
|
||||||
|
})
|
||||||
|
if (!res.ok) {
|
||||||
|
const detail = await res.text()
|
||||||
|
throw new Error(`queue failed (${res.status}): ${detail.slice(0, 300)}`)
|
||||||
|
}
|
||||||
|
const body = (await res.json()) as { job_id: string }
|
||||||
|
benchState.jobId = body.job_id
|
||||||
|
benchState.jobStatus = 'queued'
|
||||||
|
benchState.error = null
|
||||||
|
benchState.outputs = {}
|
||||||
|
log(`queued render ${body.job_id}`)
|
||||||
|
notify()
|
||||||
|
return body.job_id
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
// The ComfyApp-shaped shim
|
||||||
|
// ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
const toastShim = {
|
||||||
|
add: (msg: { severity?: string; summary?: string; detail?: string; life?: number }) => {
|
||||||
|
const severity = (msg.severity ?? 'info').toString()
|
||||||
|
log(`toast.${severity}: ${msg.summary ?? ''} ${msg.detail ?? ''}`)
|
||||||
|
if (severity === 'error') console.error('[bench-toast]', msg.summary, msg.detail)
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
const dialogShim = {
|
||||||
|
prompt: async (opts: { title?: string; message?: string }) =>
|
||||||
|
window.prompt(opts?.title ?? '', opts?.message ?? ''),
|
||||||
|
confirm: async (opts: { title?: string; message?: string }) =>
|
||||||
|
window.confirm(`${opts?.title ?? ''}\n${opts?.message ?? ''}`),
|
||||||
|
}
|
||||||
|
|
||||||
|
export const benchApp = {
|
||||||
|
api: {
|
||||||
|
clientId: 'bench-client',
|
||||||
|
fetchApi: (path: string, options?: RequestInit) => fetch(path, options),
|
||||||
|
addEventListener: (_type: string, _cb: EventListener) => {},
|
||||||
|
removeEventListener: (_type: string, _cb: EventListener) => {},
|
||||||
|
addCustomEventListener: (type: string, cb: CustomEventCallback) => benchHub.add(type, cb),
|
||||||
|
removeCustomEventListener: (type: string, cb: CustomEventCallback) => benchHub.remove(type, cb),
|
||||||
|
queuePrompt: async () => {
|
||||||
|
throw new Error('queuePrompt is not used by the bench UI; use the render dock')
|
||||||
|
},
|
||||||
|
dispatchEvent: (ev: Event) => benchHub.dispatch(ev.type, ev),
|
||||||
|
},
|
||||||
|
extensionManager: {
|
||||||
|
toast: toastShim,
|
||||||
|
dialog: dialogShim,
|
||||||
|
workflow: {
|
||||||
|
activeWorkflow: { key: 'bench' },
|
||||||
|
isBusy: false,
|
||||||
|
$subscribe: (_cb: unknown) => () => {},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
ui: {
|
||||||
|
settings: {
|
||||||
|
settingsValues: {
|
||||||
|
'Comfy.Locale': 'en',
|
||||||
|
'Comfy.VueNodes.Enabled': false,
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
canvas: null,
|
||||||
|
graph: {
|
||||||
|
getNodeById: () => null,
|
||||||
|
links: {},
|
||||||
|
},
|
||||||
|
queuePrompt: async () => {
|
||||||
|
throw new Error('queuePrompt is not used by the bench UI; use the render dock')
|
||||||
|
},
|
||||||
|
processingQueue: false,
|
||||||
|
} as unknown as ComfyApp
|
||||||
|
|
||||||
|
// comfy-api.ts reads this at module scope — must exist before that import runs.
|
||||||
|
;(globalThis as Record<string, unknown>).comfyAPI = { app: { app: benchApp } }
|
||||||
|
|
||||||
|
/** Structural stand-in for the LGraphNode the widget normally lives in. */
|
||||||
|
export function createBenchNode(): Record<string, unknown> {
|
||||||
|
return {
|
||||||
|
id: 81,
|
||||||
|
title: 'easy multiTrackEditor (bench)',
|
||||||
|
type: 'easy multiTrackEditor',
|
||||||
|
properties: { easyMediaTimelineHeight: 620 },
|
||||||
|
widgets: [],
|
||||||
|
widgets_values: [],
|
||||||
|
inputs: [],
|
||||||
|
outputs: [],
|
||||||
|
size: [1280, 640],
|
||||||
|
pos: [0, 0],
|
||||||
|
flags: {},
|
||||||
|
mode: 0,
|
||||||
|
order: 0,
|
||||||
|
setDirtyCanvas: () => {},
|
||||||
|
setSize: () => {},
|
||||||
|
onResize: () => {},
|
||||||
|
onNodeCreated: () => {},
|
||||||
|
addDOMWidget: () => ({
|
||||||
|
name: 'track_data',
|
||||||
|
type: 'react-widget',
|
||||||
|
element: document.createElement('div'),
|
||||||
|
inputEl: null,
|
||||||
|
width: undefined,
|
||||||
|
getValue: () => '',
|
||||||
|
setValue: () => {},
|
||||||
|
serializeValue: () => '',
|
||||||
|
options: {},
|
||||||
|
}),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createBenchWidget(): Record<string, unknown> {
|
||||||
|
return {
|
||||||
|
name: 'track_data',
|
||||||
|
width: undefined,
|
||||||
|
inputEl: null,
|
||||||
|
options: {},
|
||||||
|
getValue: () => '',
|
||||||
|
setValue: () => {},
|
||||||
|
}
|
||||||
|
}
|
||||||
49
frontend/src/bench/main.tsx
Normal file
49
frontend/src/bench/main.tsx
Normal file
@ -0,0 +1,49 @@
|
|||||||
|
/**
|
||||||
|
* Bench UI entry point.
|
||||||
|
*
|
||||||
|
* IMPORTANT: `./bench-runtime` must be evaluated first — it installs the
|
||||||
|
* `globalThis.comfyAPI` shim that `@/lib/comfy-api` reads at module scope.
|
||||||
|
*/
|
||||||
|
import './bench-runtime'
|
||||||
|
|
||||||
|
declare const __COMFY_EASY_MEDIA_GLOBAL_CSS__: string
|
||||||
|
|
||||||
|
import { createRoot } from 'react-dom/client'
|
||||||
|
import { BenchApp } from './BenchApp'
|
||||||
|
|
||||||
|
function injectGlobalCss(): void {
|
||||||
|
if (document.getElementById('mmnode-bench-globals')) return
|
||||||
|
const style = document.createElement('style')
|
||||||
|
style.id = 'mmnode-bench-globals'
|
||||||
|
style.textContent = __COMFY_EASY_MEDIA_GLOBAL_CSS__
|
||||||
|
document.head.appendChild(style)
|
||||||
|
}
|
||||||
|
|
||||||
|
function main(): void {
|
||||||
|
injectGlobalCss()
|
||||||
|
const host = document.getElementById('root')
|
||||||
|
if (!host) throw new Error('bench: missing #root host element')
|
||||||
|
|
||||||
|
// Surface fatal boot errors inside the page itself (sidecar windows have no
|
||||||
|
// devtools console by default).
|
||||||
|
const showFatal = (title: string, detail: string) => {
|
||||||
|
host.innerHTML = ''
|
||||||
|
const pre = document.createElement('pre')
|
||||||
|
pre.style.cssText = 'color:#f87171;padding:16px;white-space:pre-wrap;font:12px/1.5 monospace'
|
||||||
|
pre.textContent = `${title}\n\n${detail}`
|
||||||
|
host.appendChild(pre)
|
||||||
|
}
|
||||||
|
window.addEventListener('error', (e) => {
|
||||||
|
if (!host.childElementCount || host.querySelector('pre')) {
|
||||||
|
showFatal('Uncaught error', `${e.message}\n${e.error?.stack ?? ''}`)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
window.addEventListener('unhandledrejection', (e) => {
|
||||||
|
showFatal('Unhandled rejection', `${e.reason?.stack || e.reason}`)
|
||||||
|
})
|
||||||
|
|
||||||
|
const root = createRoot(host)
|
||||||
|
root.render(<BenchApp />)
|
||||||
|
}
|
||||||
|
|
||||||
|
main()
|
||||||
160
scripts/test_bench_pipeline.py
Normal file
160
scripts/test_bench_pipeline.py
Normal file
@ -0,0 +1,160 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""Standalone bench verification: API + WebSocket + mock pipeline end-to-end.
|
||||||
|
|
||||||
|
No ComfyUI, no torch. Exercises the FastAPI app exactly like a sidecar host
|
||||||
|
would: health/config, workflow & plan endpoints, a full render over the mock
|
||||||
|
backend (ffmpeg), WebSocket progress events, and output serving.
|
||||||
|
|
||||||
|
Run: python3 scripts/test_bench_pipeline.py
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import time
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ROOT = Path(__file__).resolve().parent.parent
|
||||||
|
BENCH = ROOT / "bench"
|
||||||
|
sys.path.insert(0, str(BENCH))
|
||||||
|
|
||||||
|
FAILED = []
|
||||||
|
|
||||||
|
|
||||||
|
def check(name: str, cond: bool, detail: str = "") -> None:
|
||||||
|
print(f"[{'PASS' if cond else 'FAIL'}] {name}" + (f" - {detail}" if detail and not cond else ""))
|
||||||
|
if not cond:
|
||||||
|
FAILED.append(name)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
# py_compile every bench module first
|
||||||
|
import py_compile
|
||||||
|
|
||||||
|
py_files = sorted(BENCH.rglob("*.py"))
|
||||||
|
for f in py_files:
|
||||||
|
py_compile.compile(str(f), doraise=True)
|
||||||
|
check("py_compile all bench modules", True, f"({len(py_files)} files)")
|
||||||
|
|
||||||
|
from fastapi.testclient import TestClient
|
||||||
|
|
||||||
|
os.environ.setdefault("MOXIE_BENCH_WORK", str(BENCH / "work"))
|
||||||
|
from app.config import BenchConfig
|
||||||
|
from app.server import create_app
|
||||||
|
|
||||||
|
cfg = BenchConfig(backend="mock")
|
||||||
|
cfg.repo_root = ROOT
|
||||||
|
app = create_app(cfg)
|
||||||
|
|
||||||
|
with TestClient(app) as client: # context manager runs lifespan
|
||||||
|
r = client.get("/api/health")
|
||||||
|
check("GET /api/health", r.status_code == 200 and r.json()["ok"] is True)
|
||||||
|
check("health reports mock backend", r.json()["backend"] == "mock")
|
||||||
|
|
||||||
|
r = client.get("/api/config")
|
||||||
|
check("GET /api/config", r.status_code == 200 and "backends" in r.json())
|
||||||
|
|
||||||
|
r = client.get("/api/workflow")
|
||||||
|
body = r.json()
|
||||||
|
check("GET /api/workflow summary", r.status_code == 200
|
||||||
|
and body["summary"]["editor_resolution"] == {"width": 1920, "height": 1080})
|
||||||
|
track_data = json.loads(body["track_data"])
|
||||||
|
check("workflow carries editor track_data", track_data.get("total_length") == 240)
|
||||||
|
|
||||||
|
r = client.get("/api/plan")
|
||||||
|
plan = r.json()
|
||||||
|
check("GET /api/plan", r.status_code == 200
|
||||||
|
and [s["prompt"] for s in plan["segments"]] == ["scene1", "scene2"])
|
||||||
|
check("plan duration", plan["duration_seconds"] == 10.0, str(plan["duration_seconds"]))
|
||||||
|
|
||||||
|
# --- WebSocket + render e2e (small dims to keep it fast) ---
|
||||||
|
events = []
|
||||||
|
with client.websocket_connect("/api/ws") as ws:
|
||||||
|
# drain the initial connection by starting a render in parallel
|
||||||
|
render_payload = {
|
||||||
|
"track_data": body["track_data"],
|
||||||
|
"overrides": {
|
||||||
|
"width": 256, "height": 144, "frame_rate": 8,
|
||||||
|
"steps": 4, "seed": 42, "project_name": "MM-bench-e2e",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
r = client.post("/api/render", json=render_payload)
|
||||||
|
check("POST /api/render accepted", r.status_code == 200, r.text[:200])
|
||||||
|
job_id = r.json()["job_id"]
|
||||||
|
|
||||||
|
deadline = time.time() + 120
|
||||||
|
got_success = False
|
||||||
|
while time.time() < deadline and not got_success:
|
||||||
|
msg = ws.receive_json()
|
||||||
|
events.append(msg)
|
||||||
|
if msg.get("job_id") != job_id:
|
||||||
|
continue
|
||||||
|
if msg.get("type") == "job_progress":
|
||||||
|
check("progress within pipeline stages", True) if len(events) == 2 else None
|
||||||
|
if msg.get("type") == "execution_success":
|
||||||
|
got_success = True
|
||||||
|
if msg.get("type") == "execution_error":
|
||||||
|
break
|
||||||
|
|
||||||
|
check("render completed over WebSocket", got_success,
|
||||||
|
f"events={ [e.get('type') for e in events][-6:] }")
|
||||||
|
|
||||||
|
r = client.get(f"/api/jobs/{job_id}")
|
||||||
|
job = r.json()
|
||||||
|
check("job result stored", job["status"] == "done" and job["result"] is not None)
|
||||||
|
outputs = (job["result"] or {}).get("outputs", {})
|
||||||
|
check("both branches produced", set(outputs) == {"native", "upscaled"}, str(list(outputs)))
|
||||||
|
for branch, out in outputs.items():
|
||||||
|
path = Path(out["path"])
|
||||||
|
check(f"{branch} file exists", path.is_file() and path.stat().st_size > 1000)
|
||||||
|
|
||||||
|
expected = {"native": (256, 144), "upscaled": (1920, 1080)}[branch]
|
||||||
|
check(f"{branch} served over /api/view", client.get(out["url"]).status_code == 200)
|
||||||
|
|
||||||
|
# output dims via ffprobe
|
||||||
|
import subprocess
|
||||||
|
probe = subprocess.run(
|
||||||
|
["ffprobe", "-v", "quiet", "-print_format", "json", "-show_streams",
|
||||||
|
outputs["upscaled"]["path"]],
|
||||||
|
capture_output=True, text=True)
|
||||||
|
streams = json.loads(probe.stdout).get("streams") or []
|
||||||
|
vs = next((s for s in streams if s.get("codec_type") == "video"), {})
|
||||||
|
check("upscaled dims match RTX target",
|
||||||
|
(vs.get("width"), vs.get("height")) == (1920, 1080),
|
||||||
|
f"{vs.get('width')}x{vs.get('height')}")
|
||||||
|
vs_n = json.loads(subprocess.run(
|
||||||
|
["ffprobe", "-v", "quiet", "-print_format", "json", "-show_streams",
|
||||||
|
outputs["native"]["path"]], capture_output=True, text=True).stdout
|
||||||
|
).get("streams", [{}])
|
||||||
|
vsn = next((s for s in vs_n if s.get("codec_type") == "video"), {})
|
||||||
|
check("native dims match editor canvas",
|
||||||
|
(vsn.get("width"), vsn.get("height")) == (256, 144),
|
||||||
|
f"{vsn.get('width')}x{vsn.get('height')}")
|
||||||
|
|
||||||
|
# progress stage ordering mirrors the pipeline
|
||||||
|
stages_seen = [e.get("stage") for e in events if e.get("type") == "job_progress"]
|
||||||
|
check("progress touches sampling stage", "MoxieProject" in stages_seen)
|
||||||
|
check("progress touches upscale stage", "RTXVideoSuperResolution" in stages_seen)
|
||||||
|
|
||||||
|
# /api/easy-media mirrors answer
|
||||||
|
r = client.get("/api/easy-media/media/list")
|
||||||
|
check("easy-media media/list mirror", r.status_code == 200 and "items" in r.json())
|
||||||
|
r = client.get("/api/easy-media/prompt/system-prompts")
|
||||||
|
check("easy-media system-prompts mirror", r.status_code == 200)
|
||||||
|
|
||||||
|
# static UI mount responds (built or explicit hint)
|
||||||
|
r = client.get("/")
|
||||||
|
check("root serves UI or build hint", r.status_code in (200, 404)
|
||||||
|
or "not built" in r.text)
|
||||||
|
|
||||||
|
print()
|
||||||
|
if FAILED:
|
||||||
|
print(f"FAILED: {len(FAILED)} -> {FAILED}")
|
||||||
|
return 1
|
||||||
|
print("ALL CHECKS PASSED")
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
Loading…
Reference in New Issue
Block a user