The deliverable is the single merged plan.json — the same shape the single-pass system prompt emits — and the multi-clip video is what the node itself produces from it: each shot renders as its own clip, chained through the H3 context loop into one continuous video. Removed the --clips per-scene export (external stitching is NOT the design). Workflow restated everywhere: outline turn -> the LLM writes each scene one at a time (one focused turn per scene) -> build_plan.py puts it all together at the end into plan.json -> paste once, queue once. build_plan.py: --clips option and export block removed; docstring now describes the node-side chaining. README: intro/example/validating/ wiring rewritten around the single deliverable. SKILL.md: goal, Step 3 (Put it all together at the end), Step 4 (the node makes the multi-clip video), and the revising loop updated. Suite + --check green.
29 lines
891 B
Python
Executable File
29 lines
891 B
Python
Executable File
"""Count o200k tokens (proxy for Llama/Qwen tokenizers) of every ```text fence
|
|
in the given markdown files; .json args are counted as-is.
|
|
|
|
Usage: python count_prompt_tokens.py FILE.md [FILE.md ...] [FILE.json ...]
|
|
"""
|
|
import re
|
|
import sys
|
|
|
|
import tiktoken
|
|
|
|
enc = tiktoken.get_encoding("o200k_base")
|
|
|
|
for path in sys.argv[1:]:
|
|
with open(path, encoding="utf-8") as fh:
|
|
text = fh.read()
|
|
print(path)
|
|
if path.endswith(".json"):
|
|
n = len(enc.encode(text))
|
|
print(f" whole file: {n:5d} tok ({len(text)} chars)")
|
|
continue
|
|
fences = re.findall(r"```text\n(.*?)\n```", text, re.S)
|
|
total = 0
|
|
for i, fence in enumerate(fences, 1):
|
|
t = len(enc.encode(fence))
|
|
total += t
|
|
first = fence.lstrip().splitlines()[0][:60]
|
|
print(f" fence {i}: {t:5d} tok ({len(fence):6d} chars) | {first}")
|
|
print(f" total: {total} tok")
|