1movmedia/pyworker
2
0
Fork 0
Code Pull Requests Activity

Add well-known fallback path for benchmark.json

Browse Source 
Read /opt/comfyui-api-wrapper/workflows/pyworker_benchmark.json when
neither misc/benchmark.json nor $BENCHMARK_JSON_PATH yields a usable
file. The vast.ai ComfyUI base image's convert-workflows.sh maintains
that path as a symlink to the first provisioned workflow, so on that
image the operator does not need to set BENCHMARK_JSON_PATH at all.

A set-but-broken $BENCHMARK_JSON_PATH now warns and falls through to
the well-known path instead of dropping straight to the SD1.5 fallback,
so a typo in the env var doesn't mask an otherwise-working benchmark.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rob Ballantyne
2026-05-07 11:54:20 +01:00
parent a634ba07a6
commit 381a39f201
2 changed files with 30 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
workers/comfyui-json/README.md
+7 -5
View File
@@ -106,11 +106,13 @@ Images will be saved locally AND uploaded to `s3://{bucket}/comfyui/{filename}`.
You can provide a custom ComfyUI workflow for benchmarking. This allows you to test performance using your preferred models and workflow complexity.
**Ways to provide the benchmark file:**
- **Fork this repository** and commit your workflow to `workers/comfyui-json/misc/benchmark.json`.
- **Write the file during provisioning** to a path *outside* the pyworker tree (e.g. `/workspace/benchmark.json`) and export `BENCHMARK_JSON_PATH` so the worker can find it. The pyworker repo is cloned by `start_server.sh` *after* provisioning runs, so provisioning cannot write into `misc/` directly — the destination would be clobbered, or the clone would fail.
**Ways to provide the benchmark file** (in resolution order — first match wins):
If both are present, the in-tree `misc/benchmark.json` wins; `BENCHMARK_JSON_PATH` is consulted only when no in-tree file exists. If the env var is set but points at a missing or unreadable file, the worker logs a warning and falls back to the default benchmark.
1. **Fork this repository** and commit your workflow to `workers/comfyui-json/misc/benchmark.json`.
2. **Write the file during provisioning** to a path *outside* the pyworker tree (e.g. `/workspace/benchmark.json`) and export `BENCHMARK_JSON_PATH` so the worker can find it. The pyworker repo is cloned by `start_server.sh` *after* provisioning runs, so provisioning cannot write into `misc/` directly — the destination would be clobbered, or the clone would fail.
3. **Run on the vast.ai ComfyUI base image.** Its `convert-workflows.sh` maintains `/opt/comfyui-api-wrapper/workflows/pyworker_benchmark.json` as a symlink to the first provisioned workflow; the worker reads this automatically when neither of the above is set. No env var required.
If `BENCHMARK_JSON_PATH` is set but points at a missing or unreadable file, the worker logs a warning and falls through to the next tier rather than going straight to the SD1.5 fallback.
An example workflow is provided at `workers/comfyui-json/misc/benchmark.json.example`. To ensure varied generations, use the placeholder `__RANDOM_INT__` in place of static seed values — it will be replaced with a random integer for each benchmark run.
@@ -122,7 +124,7 @@ The default benchmark uses Stable Diffusion v1.5 with ComfyUI's standard text-to
| Environment Variable | Default Value | Description |
| -------------------- | ------------- | ----------- |
| BENCHMARK_JSON_PATH | (unset) | Path to a custom workflow file outside the pyworker tree. Used only if `misc/benchmark.json` is absent. |
| BENCHMARK_JSON_PATH | (unset) | Path to a custom workflow file outside the pyworker tree. Used if `misc/benchmark.json` is absent. Falls through to `/opt/comfyui-api-wrapper/workflows/pyworker_benchmark.json` if set but missing. |
| BENCHMARK_TEST_WIDTH | 512 | Fallback benchmark: image width (pixels) |
| BENCHMARK_TEST_HEIGHT | 512 | Fallback benchmark: image height (pixels) |
| BENCHMARK_TEST_STEPS | 20 | Fallback benchmark: number of denoising steps |