- bench/ FastAPI server: fixed 9-stage pipeline parsed from the base workflow, serial render jobs, WS progress mirroring ComfyUI's push events, /api/view + ComfyUI-compatible /view, /upload/image, /easy-media/* aliases for the ported editor - dual backend: self-contained mock renderer (numpy+ffmpeg, deterministic per prompt/seed, native + RTX-upscaled mp4 branches) and a ComfyUI bridge (substitutes the plan into base_workflow.json, converts to the /prompt API format, polls history, downloads outputs) - timeline port: frontend/src/bench mounts the unmodified MultiTrack editor against a comfyAPI shim (fetchApi/toast/dialog/custom events); build.bench.ts emits bench/ui-dist served by the server - packaging: PyInstaller onedir spec + Tauri2 sidecar docs - tests: bench/tests contract suite + scripts/test_bench_pipeline.py e2e (API+WS+render); prebuilt bench UI committed
88 lines
3.4 KiB
Markdown
88 lines
3.4 KiB
Markdown
# Moxie Multimedia Bench
|
|
|
|
A standalone Python + web harness for the Moxie Multimedia pipeline — the
|
|
first step toward running this node pack outside ComfyUI as a sidecar for a
|
|
Tauri 2 desktop app.
|
|
|
|
The bench locks to the fixed pipeline of the base workflow
|
|
(`bench/base_workflow.json`):
|
|
|
|
```
|
|
MoxieMultimediaLoader -> LoraLoaderModelOnly -> MoxiePreviewOverride
|
|
-> easy multiTrackEditor -> MoxieProject -> MoxieCombine
|
|
-> [native branch] CreateVideo -> SaveVideo
|
|
-> [upscaled branch] RTXVideoSuperResolution -> CreateVideo -> SaveVideo
|
|
```
|
|
|
|
The multitrack timeline editor is the real thing: the bench bundles the same
|
|
React editor components the ComfyUI node uses, mounted by a thin shim
|
|
(`frontend/src/bench/`) that talks REST + WebSocket instead of the ComfyUI
|
|
runtime.
|
|
|
|
## Run from source
|
|
|
|
```bash
|
|
pip install -r bench/requirements.txt # fastapi, uvicorn, numpy, pillow
|
|
python bench/run.py --port 8787 # auto backend (mock here)
|
|
# open http://127.0.0.1:8787
|
|
```
|
|
|
|
The UI needs a one-time build (a prebuilt copy ships in `bench/ui-dist/`):
|
|
|
|
```bash
|
|
cd frontend && bun install && bun run build:bench
|
|
```
|
|
|
|
## Backends
|
|
|
|
| backend | needs | behaviour |
|
|
|---|---|---|
|
|
| `mock` | ffmpeg, numpy, pillow | deterministic synthetic frames per (prompt, seed, frame), sine audio bed, lanczos "upscale" to the RTX target, h264 encode. Both output branches produce real mp4s. |
|
|
| `comfy` | a running ComfyUI + downloaded models | substitutes the plan into the base workflow, converts it to the `/prompt` API format, submits to ComfyUI, polls progress, downloads outputs. |
|
|
| `auto` | — | `comfy` when reachable, else `mock`. |
|
|
|
|
```bash
|
|
python bench/run.py --backend comfy --comfyui-url http://127.0.0.1:8188
|
|
```
|
|
|
|
## HTTP API
|
|
|
|
- `GET /api/health`, `GET /api/config` — liveness, backend availability
|
|
- `GET /api/workflow` — base workflow summary + editor `track_data`
|
|
- `GET /api/plan` — resolved render plan (segments, prompts, frames)
|
|
- `POST /api/render` — queue a render:
|
|
`{"track_data": <editor JSON>, "overrides": {"width": 1920, "height": 1080,
|
|
"steps": 4, "seed": 123, "project_name": "MM-x", "backend": "auto"}}`
|
|
- `GET /api/jobs`, `GET /api/jobs/{id}`, `DELETE /api/jobs/{id}`
|
|
- `WS /api/ws` — push events: `job_queued`, `execution_start`, `job_progress`,
|
|
`execution_success`, `execution_error` (mirrors ComfyUI's push model)
|
|
- `GET /api/view?filename=&type=output&job=` — fetch rendered files
|
|
- ComfyUI-compatible aliases: `/view`, `/upload/image`, `/easy-media/*`
|
|
|
|
Outputs land in `bench/work/outputs/<job_id>/` (`MOXIE_BENCH_WORK` overrides
|
|
the root). Every render produces both branches: `native` (editor canvas) and
|
|
`upscaled` (RTX target dimensions).
|
|
|
|
## Tauri 2 sidecar
|
|
|
|
See `bench/tauri/README.md` — PyInstaller onedir packaging
|
|
(`bench/moxie_bench.spec`), `tauri.conf.json` resource wiring, spawn + health
|
|
poll snippet, and the endpoint table.
|
|
|
|
## Tests
|
|
|
|
```bash
|
|
python3 -m pytest bench/tests -q # contract tests (no ComfyUI needed)
|
|
python3 scripts/test_bench_pipeline.py # full API + WS + mock render e2e
|
|
cd frontend && bun run test # editor suite incl. bench shim
|
|
```
|
|
|
|
## Phase map
|
|
|
|
- **phase 2 (this)** — bench: fixed pipeline, dual backend, timeline UI port,
|
|
sidecar packaging.
|
|
- **phase 3 (planned)** — in-process execution backend: load the pack's node
|
|
classes directly (torch + ComfyUI modules on `sys.path`) instead of driving
|
|
a separate ComfyUI server; `bench/app/backends/comfy.py` keeps the same
|
|
interface.
|