comfyui-json: address PR #85 review

Five issues raised by Copilot's review:

1. _resolve_benchmark_path's docstring/README claim that a set-but-
   broken BENCHMARK_JSON_PATH falls through to the well-known tier,
   but the implementation only handled "file missing". A path
   pointing at a directory or holding malformed JSON dropped
   straight to the SD1.5 fallback without consulting tier 3.
   Replaced with a true tiered try-and-load: walk
   (misc, env, well-known), attempt to load each, and fall through
   to the next on any failure (missing, not a regular file,
   unreadable, invalid JSON). The env-var case still surfaces a
   warning so a typo doesn't fail silently.

2. int(os.getenv("BENCHMARK_TEST_WIDTH", ...)) crashed on non-int
   values. Added _env_int helper that warns + returns default on
   ValueError. Empty string also handled.

3. random.choice([]) on an empty test_prompts.txt raised IndexError.
   _load_prompts now warns + uses a built-in _FALLBACK_PROMPT when
   the file is missing or yields no non-blank lines.

4. README already claimed "missing or unreadable" fall-through; the
   refactor in (1) makes the code match. No README change needed.

5. test_prompts.txt restored verbatim from the pre-rewrite tree
   carried real-person and IP-laden prompts (Pope Francis, Iron Man,
   Luke Skywalker, "Disney socialite"). Used automatically during
   warm-up they're a reputational/safety-filter risk for the worker.
   Replaced with generic equivalents that exercise the same workload
   characteristics (1 elderly figure on motorcycle, 1 armoured hero
   with axe, etc.).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rob Ballantyne
2026-05-07 18:25:21 +01:00
parent cecf0236fa
commit a5bcc3de5e
2 changed files with 82 additions and 41 deletions
+5 -5
View File
@@ -7,13 +7,13 @@ inspired by realflow-cinema4d editor features, create image of a transparent lux
biker with backpack on his back riding a motorcycle, Style by Ade Santora, Oilpunk, Cover photo, craig mullins style, on the cover of a magazine, Outdoor Magazine, inspired by Alex Petruk APe, image of a male biker, Cover of an award-winning magazine, the man has a backpack, photo for magazine, with a backpack, magazine cover biker with backpack on his back riding a motorcycle, Style by Ade Santora, Oilpunk, Cover photo, craig mullins style, on the cover of a magazine, Outdoor Magazine, inspired by Alex Petruk APe, image of a male biker, Cover of an award-winning magazine, the man has a backpack, photo for magazine, with a backpack, magazine cover
generate a collage-style illustration inspired by the Procreate raster graphic editor, photographic illustration with the theme, 2D vector, art for textile sublimation, containing surrealistic cartoon cat wearing a baseball cap and jeans standing in front of a poster, inspired by Sadao Watanabe, Doraemon, Japanese cartoon style, Eichiro Oda, Iconic high detail character, Director: Nakahara Nantenbō, Kastuhiro Otomo, image detailed, by Miyamoto, Hidetaka Miyazaki, Katsuhiro illustration, 8k, masterpiece, Minimize noise and grain in photo quality without lose quality and increase brightness and lighting,Symmetry and Alignment, Avoid asymmetrical shapes and out-of-focus points. Focus and Sharpness: Make sure the image is focused and sharp and encourages the viewer to see it as a work of art printed on fabric. generate a collage-style illustration inspired by the Procreate raster graphic editor, photographic illustration with the theme, 2D vector, art for textile sublimation, containing surrealistic cartoon cat wearing a baseball cap and jeans standing in front of a poster, inspired by Sadao Watanabe, Doraemon, Japanese cartoon style, Eichiro Oda, Iconic high detail character, Director: Nakahara Nantenbō, Kastuhiro Otomo, image detailed, by Miyamoto, Hidetaka Miyazaki, Katsuhiro illustration, 8k, masterpiece, Minimize noise and grain in photo quality without lose quality and increase brightness and lighting,Symmetry and Alignment, Avoid asymmetrical shapes and out-of-focus points. Focus and Sharpness: Make sure the image is focused and sharp and encourages the viewer to see it as a work of art printed on fabric.
fantasy medieval village world inside a glass sphere , high detail, fantasy, realistic, light effect, hyper detail, volumetric lighting, cinematic, macro, depth of field, blur, red light and clouds from the back, highly detailed epic cinematic concept art cg render made in maya, blender and photoshop, octane render, excellent composition, dynamic dramatic cinematic lighting, aesthetic, very inspirational, world inside a glass sphere by james gurney by artgerm with james jean, joe fenton and tristan eaton by ross tran, fine details fantasy medieval village world inside a glass sphere , high detail, fantasy, realistic, light effect, hyper detail, volumetric lighting, cinematic, macro, depth of field, blur, red light and clouds from the back, highly detailed epic cinematic concept art cg render made in maya, blender and photoshop, octane render, excellent composition, dynamic dramatic cinematic lighting, aesthetic, very inspirational, world inside a glass sphere by james gurney by artgerm with james jean, joe fenton and tristan eaton by ross tran, fine details
Iron Man, (Arnold Tsang, Toru Nakayama), Masterpiece, Studio Quality, 6k , toa, toaair, 1boy, glowing, axe, mecha, science_fiction, solo, weapon, jungle , green_background, nature, outdoors, solo, tree, weapon, mask, dynamic lighting, detailed shading, digital texture painting armored hero with a glowing axe, mecha science_fiction, jungle background, dynamic lighting, detailed shading, digital texture painting, masterpiece, studio quality, 6k
(Pope Francis) wearing leather jacket is a DJ in a nightclub, mixing live on stage, giant mixing table, a masterpiece elderly figure in a leather jacket DJing in a smoky nightclub, mixing live on a giant console, dramatic stage lighting, a masterpiece
Pope Francis wearing biker (leather jacket), a masterpiece elderly figure in a leather jacket on a motorcycle, magazine cover lighting, a masterpiece
Luke Skywalker ordering a burger and fries from the Death Star canteen. a young pilot ordering a burger and fries from a futuristic space cantina
I want to generate a group avatar for a Feishu group chat. The role of this group is daily software technical communication. Now the subject technology stacks that members of this group discuss daily include: algorithms, data structures, optimization, functional programming, and the programming languages often discussed are: TypeScript, Java, python, etc. I hope this avatar has a simple aesthetic, this avatar is a single person avatar I want to generate a group avatar for a Feishu group chat. The role of this group is daily software technical communication. Now the subject technology stacks that members of this group discuss daily include: algorithms, data structures, optimization, functional programming, and the programming languages often discussed are: TypeScript, Java, python, etc. I hope this avatar has a simple aesthetic, this avatar is a single person avatar
portrait Anime black girl cute-fine-face, pretty face, realistic shaded Perfect face, fine details. Anime. realistic shaded lighting by Ilya Kuvshinov Giuseppe Dangelico Pino and Michael Garmash and Rob Rey, IAMAG premiere, WLOP matte print, cute freckles, masterpiece portrait Anime black girl cute-fine-face, pretty face, realistic shaded Perfect face, fine details. Anime. realistic shaded lighting by Ilya Kuvshinov Giuseppe Dangelico Pino and Michael Garmash and Rob Rey, IAMAG premiere, WLOP matte print, cute freckles, masterpiece
young Disney socialite wearing a beige miniskirt, dark brown turtleneck sweater, small neckless, cute-fine-face, anime. illustration, realistic shaded perfect face, brown hair, grey eyes, fine details, realistic shaded lighting by ilya kuvshinov giuseppe dangelico pino and michael garmash and rob rey, iamag premiere, wlop matte print, a masterpiece young woman in modern fashion editorial, beige miniskirt and dark brown turtleneck sweater, soft studio lighting, brown hair, grey eyes, fine details, magazine cover style, a masterpiece
Cute small cat sitting in a movie theater eating chicken wiggs watching a movie ,unreal engine, cozy indoor lighting, artstation, detailed, digital painting,cinematic,character design by mark ryden and pixar and hayao miyazaki, unreal 5, daz, hyperrealistic, octane render Cute small cat sitting in a movie theater eating chicken wiggs watching a movie ,unreal engine, cozy indoor lighting, artstation, detailed, digital painting,cinematic,character design by mark ryden and pixar and hayao miyazaki, unreal 5, daz, hyperrealistic, octane render
Cute small dog sitting in a movie theater eating popcorn watching a movie ,unreal engine, cozy indoor lighting, artstation, detailed, digital painting,cinematic,character design by mark ryden and pixar and hayao miyazaki, unreal 5, daz, hyperrealistic, octane render Cute small dog sitting in a movie theater eating popcorn watching a movie ,unreal engine, cozy indoor lighting, artstation, detailed, digital painting,cinematic,character design by mark ryden and pixar and hayao miyazaki, unreal 5, daz, hyperrealistic, octane render
fox bracelet made of buckskin with fox features, rich details, fine carvings, studio lighting fox bracelet made of buckskin with fox features, rich details, fine carvings, studio lighting
+73 -32
View File
@@ -92,61 +92,102 @@ WELLKNOWN_BENCHMARK = Path("/opt/comfyui-api-wrapper/workflows/pyworker_benchmar
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
# Used when test_prompts.txt is unreadable or empty. Bare and generic
# on purpose — this is a benchmark seed, not a creative output.
_FALLBACK_PROMPT = "a still life on a wooden table, soft daylight"
def _resolve_benchmark_path() -> Path | None:
"""Return the path to the custom benchmark workflow, or None if absent.
See module docstring for the precedence rule. A set-but-broken def _env_int(name: str, default: int) -> int:
``$BENCHMARK_JSON_PATH`` logs a warning then falls through to the """Read an integer env var, warning + falling back on bad values."""
well-known path, so a typo in the env var doesn't silently mask a raw = os.getenv(name)
provisioned benchmark sitting at the standard location. if raw is None or raw == "":
return default
try:
return int(raw)
except ValueError:
log.warning("ignoring %s=%r (not an int); using default %d", name, raw, default)
return default
def _try_load_workflow(path: Path) -> dict | None:
"""Load and return a benchmark workflow from ``path``.
Returns None on any failure (path missing, not a regular file,
unreadable, invalid JSON) so the caller can fall through to the
next tier rather than dropping straight to the SD1.5 default.
""" """
if BENCHMARK_FILE.exists(): if not path.is_file():
return BENCHMARK_FILE return None
env_path = os.getenv("BENCHMARK_JSON_PATH") try:
if env_path: with open(path) as f:
path = Path(env_path) return json.load(f)
if path.exists(): except (json.JSONDecodeError, OSError) as e:
return path log.warning("Failed to load %s: %s; trying next tier", path, e)
log.warning("BENCHMARK_JSON_PATH=%s does not exist; trying fallbacks", path)
if WELLKNOWN_BENCHMARK.exists():
return WELLKNOWN_BENCHMARK
return None return None
def _custom_workflow_payload() -> dict | None: def _custom_workflow_payload() -> dict | None:
"""Build a payload from a custom benchmark workflow JSON, or None if unavailable.""" """Try each benchmark workflow tier in order; return the first one
path = _resolve_benchmark_path() that loads cleanly as a payload, or None if every tier is absent /
if path is None: unreadable. Tiers (in order): in-tree ``misc/benchmark.json``,
return None ``$BENCHMARK_JSON_PATH``, well-known base-image symlink.
try: """
with open(path) as f: env_path = os.getenv("BENCHMARK_JSON_PATH")
workflow = json.load(f) candidates = [("misc", BENCHMARK_FILE)]
except (json.JSONDecodeError, OSError) as e: if env_path:
log.error("Failed to load %s: %s; falling back to default benchmark", path, e) candidates.append(("env", Path(env_path)))
return None candidates.append(("well-known", WELLKNOWN_BENCHMARK))
log.info("Using custom benchmark workflow from %s", path)
for label, path in candidates:
# Surface a warning specifically when the operator pointed
# BENCHMARK_JSON_PATH at something we can't use — silent
# fall-through there is a footgun (typo => SD1.5 fallback,
# operator wonders why custom benchmark didn't take).
if not path.is_file():
if label == "env":
log.warning(
"BENCHMARK_JSON_PATH=%s is not a readable file; trying fallbacks", path
)
continue
workflow = _try_load_workflow(path)
if workflow is None:
continue
log.info("Using custom benchmark workflow from %s (%s)", path, label)
return { return {
"input": { "input": {
"request_id": f"test-{random.randint(1000, 99999)}", "request_id": f"test-{random.randint(1000, 99999)}",
"workflow_json": workflow, "workflow_json": workflow,
} }
} }
return None
def _load_prompts() -> list[str]:
"""Read misc/test_prompts.txt; defensive against missing/empty file."""
try:
with open(TEST_PROMPTS) as f:
prompts = [line.strip() for line in f if line.strip()]
except OSError as e:
log.warning("could not read %s: %s; using built-in fallback prompt", TEST_PROMPTS, e)
return [_FALLBACK_PROMPT]
if not prompts:
log.warning("%s is empty; using built-in fallback prompt", TEST_PROMPTS)
return [_FALLBACK_PROMPT]
return prompts
def _default_payload() -> dict: def _default_payload() -> dict:
"""Build the SD1.5 Text2Image fallback payload.""" """Build the SD1.5 Text2Image fallback payload."""
with open(TEST_PROMPTS) as f: prompts = _load_prompts()
prompts = [line.strip() for line in f if line.strip()]
return { return {
"input": { "input": {
"request_id": f"test-{random.randint(1000, 99999)}", "request_id": f"test-{random.randint(1000, 99999)}",
"modifier": "Text2Image", "modifier": "Text2Image",
"modifications": { "modifications": {
"prompt": random.choice(prompts), "prompt": random.choice(prompts),
"width": int(os.getenv("BENCHMARK_TEST_WIDTH", 512)), "width": _env_int("BENCHMARK_TEST_WIDTH", 512),
"height": int(os.getenv("BENCHMARK_TEST_HEIGHT", 512)), "height": _env_int("BENCHMARK_TEST_HEIGHT", 512),
"steps": int(os.getenv("BENCHMARK_TEST_STEPS", 20)), "steps": _env_int("BENCHMARK_TEST_STEPS", 20),
"seed": random.randint(0, sys.maxsize), "seed": random.randint(0, sys.maxsize),
} }
} }