comfyui-json: address PR #85 review
Five issues raised by Copilot's review:
1. _resolve_benchmark_path's docstring/README claim that a set-but-
broken BENCHMARK_JSON_PATH falls through to the well-known tier,
but the implementation only handled "file missing". A path
pointing at a directory or holding malformed JSON dropped
straight to the SD1.5 fallback without consulting tier 3.
Replaced with a true tiered try-and-load: walk
(misc, env, well-known), attempt to load each, and fall through
to the next on any failure (missing, not a regular file,
unreadable, invalid JSON). The env-var case still surfaces a
warning so a typo doesn't fail silently.
2. int(os.getenv("BENCHMARK_TEST_WIDTH", ...)) crashed on non-int
values. Added _env_int helper that warns + returns default on
ValueError. Empty string also handled.
3. random.choice([]) on an empty test_prompts.txt raised IndexError.
_load_prompts now warns + uses a built-in _FALLBACK_PROMPT when
the file is missing or yields no non-blank lines.
4. README already claimed "missing or unreadable" fall-through; the
refactor in (1) makes the code match. No README change needed.
5. test_prompts.txt restored verbatim from the pre-rewrite tree
carried real-person and IP-laden prompts (Pope Francis, Iron Man,
Luke Skywalker, "Disney socialite"). Used automatically during
warm-up they're a reputational/safety-filter risk for the worker.
Replaced with generic equivalents that exercise the same workload
characteristics (1 elderly figure on motorcycle, 1 armoured hero
with axe, etc.).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -7,13 +7,13 @@ inspired by realflow-cinema4d editor features, create image of a transparent lux
|
||||
biker with backpack on his back riding a motorcycle, Style by Ade Santora, Oilpunk, Cover photo, craig mullins style, on the cover of a magazine, Outdoor Magazine, inspired by Alex Petruk APe, image of a male biker, Cover of an award-winning magazine, the man has a backpack, photo for magazine, with a backpack, magazine cover
|
||||
generate a collage-style illustration inspired by the Procreate raster graphic editor, photographic illustration with the theme, 2D vector, art for textile sublimation, containing surrealistic cartoon cat wearing a baseball cap and jeans standing in front of a poster, inspired by Sadao Watanabe, Doraemon, Japanese cartoon style, Eichiro Oda, Iconic high detail character, Director: Nakahara Nantenbō, Kastuhiro Otomo, image detailed, by Miyamoto, Hidetaka Miyazaki, Katsuhiro illustration, 8k, masterpiece, Minimize noise and grain in photo quality without lose quality and increase brightness and lighting,Symmetry and Alignment, Avoid asymmetrical shapes and out-of-focus points. Focus and Sharpness: Make sure the image is focused and sharp and encourages the viewer to see it as a work of art printed on fabric.
|
||||
fantasy medieval village world inside a glass sphere , high detail, fantasy, realistic, light effect, hyper detail, volumetric lighting, cinematic, macro, depth of field, blur, red light and clouds from the back, highly detailed epic cinematic concept art cg render made in maya, blender and photoshop, octane render, excellent composition, dynamic dramatic cinematic lighting, aesthetic, very inspirational, world inside a glass sphere by james gurney by artgerm with james jean, joe fenton and tristan eaton by ross tran, fine details
|
||||
Iron Man, (Arnold Tsang, Toru Nakayama), Masterpiece, Studio Quality, 6k , toa, toaair, 1boy, glowing, axe, mecha, science_fiction, solo, weapon, jungle , green_background, nature, outdoors, solo, tree, weapon, mask, dynamic lighting, detailed shading, digital texture painting
|
||||
(Pope Francis) wearing leather jacket is a DJ in a nightclub, mixing live on stage, giant mixing table, a masterpiece
|
||||
Pope Francis wearing biker (leather jacket), a masterpiece
|
||||
Luke Skywalker ordering a burger and fries from the Death Star canteen.
|
||||
armored hero with a glowing axe, mecha science_fiction, jungle background, dynamic lighting, detailed shading, digital texture painting, masterpiece, studio quality, 6k
|
||||
elderly figure in a leather jacket DJing in a smoky nightclub, mixing live on a giant console, dramatic stage lighting, a masterpiece
|
||||
elderly figure in a leather jacket on a motorcycle, magazine cover lighting, a masterpiece
|
||||
a young pilot ordering a burger and fries from a futuristic space cantina
|
||||
I want to generate a group avatar for a Feishu group chat. The role of this group is daily software technical communication. Now the subject technology stacks that members of this group discuss daily include: algorithms, data structures, optimization, functional programming, and the programming languages often discussed are: TypeScript, Java, python, etc. I hope this avatar has a simple aesthetic, this avatar is a single person avatar
|
||||
portrait Anime black girl cute-fine-face, pretty face, realistic shaded Perfect face, fine details. Anime. realistic shaded lighting by Ilya Kuvshinov Giuseppe Dangelico Pino and Michael Garmash and Rob Rey, IAMAG premiere, WLOP matte print, cute freckles, masterpiece
|
||||
young Disney socialite wearing a beige miniskirt, dark brown turtleneck sweater, small neckless, cute-fine-face, anime. illustration, realistic shaded perfect face, brown hair, grey eyes, fine details, realistic shaded lighting by ilya kuvshinov giuseppe dangelico pino and michael garmash and rob rey, iamag premiere, wlop matte print, a masterpiece
|
||||
young woman in modern fashion editorial, beige miniskirt and dark brown turtleneck sweater, soft studio lighting, brown hair, grey eyes, fine details, magazine cover style, a masterpiece
|
||||
Cute small cat sitting in a movie theater eating chicken wiggs watching a movie ,unreal engine, cozy indoor lighting, artstation, detailed, digital painting,cinematic,character design by mark ryden and pixar and hayao miyazaki, unreal 5, daz, hyperrealistic, octane render
|
||||
Cute small dog sitting in a movie theater eating popcorn watching a movie ,unreal engine, cozy indoor lighting, artstation, detailed, digital painting,cinematic,character design by mark ryden and pixar and hayao miyazaki, unreal 5, daz, hyperrealistic, octane render
|
||||
fox bracelet made of buckskin with fox features, rich details, fine carvings, studio lighting
|
||||
|
||||
@@ -92,61 +92,102 @@ WELLKNOWN_BENCHMARK = Path("/opt/comfyui-api-wrapper/workflows/pyworker_benchmar
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# Used when test_prompts.txt is unreadable or empty. Bare and generic
|
||||
# on purpose — this is a benchmark seed, not a creative output.
|
||||
_FALLBACK_PROMPT = "a still life on a wooden table, soft daylight"
|
||||
|
||||
def _resolve_benchmark_path() -> Path | None:
|
||||
"""Return the path to the custom benchmark workflow, or None if absent.
|
||||
|
||||
See module docstring for the precedence rule. A set-but-broken
|
||||
``$BENCHMARK_JSON_PATH`` logs a warning then falls through to the
|
||||
well-known path, so a typo in the env var doesn't silently mask a
|
||||
provisioned benchmark sitting at the standard location.
|
||||
def _env_int(name: str, default: int) -> int:
|
||||
"""Read an integer env var, warning + falling back on bad values."""
|
||||
raw = os.getenv(name)
|
||||
if raw is None or raw == "":
|
||||
return default
|
||||
try:
|
||||
return int(raw)
|
||||
except ValueError:
|
||||
log.warning("ignoring %s=%r (not an int); using default %d", name, raw, default)
|
||||
return default
|
||||
|
||||
|
||||
def _try_load_workflow(path: Path) -> dict | None:
|
||||
"""Load and return a benchmark workflow from ``path``.
|
||||
|
||||
Returns None on any failure (path missing, not a regular file,
|
||||
unreadable, invalid JSON) so the caller can fall through to the
|
||||
next tier rather than dropping straight to the SD1.5 default.
|
||||
"""
|
||||
if BENCHMARK_FILE.exists():
|
||||
return BENCHMARK_FILE
|
||||
env_path = os.getenv("BENCHMARK_JSON_PATH")
|
||||
if env_path:
|
||||
path = Path(env_path)
|
||||
if path.exists():
|
||||
return path
|
||||
log.warning("BENCHMARK_JSON_PATH=%s does not exist; trying fallbacks", path)
|
||||
if WELLKNOWN_BENCHMARK.exists():
|
||||
return WELLKNOWN_BENCHMARK
|
||||
return None
|
||||
|
||||
|
||||
def _custom_workflow_payload() -> dict | None:
|
||||
"""Build a payload from a custom benchmark workflow JSON, or None if unavailable."""
|
||||
path = _resolve_benchmark_path()
|
||||
if path is None:
|
||||
if not path.is_file():
|
||||
return None
|
||||
try:
|
||||
with open(path) as f:
|
||||
workflow = json.load(f)
|
||||
return json.load(f)
|
||||
except (json.JSONDecodeError, OSError) as e:
|
||||
log.error("Failed to load %s: %s; falling back to default benchmark", path, e)
|
||||
log.warning("Failed to load %s: %s; trying next tier", path, e)
|
||||
return None
|
||||
log.info("Using custom benchmark workflow from %s", path)
|
||||
return {
|
||||
"input": {
|
||||
"request_id": f"test-{random.randint(1000, 99999)}",
|
||||
"workflow_json": workflow,
|
||||
|
||||
|
||||
def _custom_workflow_payload() -> dict | None:
|
||||
"""Try each benchmark workflow tier in order; return the first one
|
||||
that loads cleanly as a payload, or None if every tier is absent /
|
||||
unreadable. Tiers (in order): in-tree ``misc/benchmark.json``,
|
||||
``$BENCHMARK_JSON_PATH``, well-known base-image symlink.
|
||||
"""
|
||||
env_path = os.getenv("BENCHMARK_JSON_PATH")
|
||||
candidates = [("misc", BENCHMARK_FILE)]
|
||||
if env_path:
|
||||
candidates.append(("env", Path(env_path)))
|
||||
candidates.append(("well-known", WELLKNOWN_BENCHMARK))
|
||||
|
||||
for label, path in candidates:
|
||||
# Surface a warning specifically when the operator pointed
|
||||
# BENCHMARK_JSON_PATH at something we can't use — silent
|
||||
# fall-through there is a footgun (typo => SD1.5 fallback,
|
||||
# operator wonders why custom benchmark didn't take).
|
||||
if not path.is_file():
|
||||
if label == "env":
|
||||
log.warning(
|
||||
"BENCHMARK_JSON_PATH=%s is not a readable file; trying fallbacks", path
|
||||
)
|
||||
continue
|
||||
workflow = _try_load_workflow(path)
|
||||
if workflow is None:
|
||||
continue
|
||||
log.info("Using custom benchmark workflow from %s (%s)", path, label)
|
||||
return {
|
||||
"input": {
|
||||
"request_id": f"test-{random.randint(1000, 99999)}",
|
||||
"workflow_json": workflow,
|
||||
}
|
||||
}
|
||||
}
|
||||
return None
|
||||
|
||||
|
||||
def _load_prompts() -> list[str]:
|
||||
"""Read misc/test_prompts.txt; defensive against missing/empty file."""
|
||||
try:
|
||||
with open(TEST_PROMPTS) as f:
|
||||
prompts = [line.strip() for line in f if line.strip()]
|
||||
except OSError as e:
|
||||
log.warning("could not read %s: %s; using built-in fallback prompt", TEST_PROMPTS, e)
|
||||
return [_FALLBACK_PROMPT]
|
||||
if not prompts:
|
||||
log.warning("%s is empty; using built-in fallback prompt", TEST_PROMPTS)
|
||||
return [_FALLBACK_PROMPT]
|
||||
return prompts
|
||||
|
||||
|
||||
def _default_payload() -> dict:
|
||||
"""Build the SD1.5 Text2Image fallback payload."""
|
||||
with open(TEST_PROMPTS) as f:
|
||||
prompts = [line.strip() for line in f if line.strip()]
|
||||
prompts = _load_prompts()
|
||||
return {
|
||||
"input": {
|
||||
"request_id": f"test-{random.randint(1000, 99999)}",
|
||||
"modifier": "Text2Image",
|
||||
"modifications": {
|
||||
"prompt": random.choice(prompts),
|
||||
"width": int(os.getenv("BENCHMARK_TEST_WIDTH", 512)),
|
||||
"height": int(os.getenv("BENCHMARK_TEST_HEIGHT", 512)),
|
||||
"steps": int(os.getenv("BENCHMARK_TEST_STEPS", 20)),
|
||||
"width": _env_int("BENCHMARK_TEST_WIDTH", 512),
|
||||
"height": _env_int("BENCHMARK_TEST_HEIGHT", 512),
|
||||
"steps": _env_int("BENCHMARK_TEST_STEPS", 20),
|
||||
"seed": random.randint(0, sys.maxsize),
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user