pyworker/workers/comfyui-json/README.md

ComfyUI PyWorker

This is the base PyWorker for ComfyUI. It provides a unified interface for running any ComfyUI workflow through a proxy-based architecture. See the Serverless documentation for guides and how-to's.

The cost for each request has a static value of 100. ComfyUI does not handle concurrent workloads and there is no current provision to load multiple instances of ComfyUI per worker node.

Instance Setup

1. Pick a template

• ComfyUI (Serverless)

2. Follow the getting started guide for help with configuring your serverless setup. For testing, we recommend that you use the default options presented by the web interface.

Requirements

This worker requires both ComfyUI and ComfyUI API Wrapper.

A docker image is provided but you may use any if the above requirements are met.

Client

The client demonstrates how to use the Vast Serverless SDK to generate images, save them locally, and optionally upload to S3-compatible storage.

Setup

1. Clone the PyWorker repository to your local machine and install the necessary requirements for running the test client.

git clone https://github.com/vast-ai/pyworker
cd pyworker
pip install uv
uv venv -p 3.12
source .venv/bin/activate
uv pip install -r requirements.txt

2. Set your API key:

export VAST_API_KEY=<your_api_key>

Usage

# Default prompt
python -m workers.comfyui-json.client

# Custom prompt
python -m workers.comfyui-json.client --prompt "a cat sitting on a rainbow"

# With options
python -m workers.comfyui-json.client --prompt "sunset" --width 1024 --height 1024 --steps 30

# Using a custom workflow file
python -m workers.comfyui-json.client --workflow my_workflow.json

# With S3 upload
python -m workers.comfyui-json.client --s3

CLI Flags

Flag	Default	Description
--endpoint	my-comfyui-endpoint	Vast endpoint name
--prompt	(default)	Text prompt for image generation
--workflow	(none)	Path to custom workflow JSON file
--width	512	Image width in pixels
--height	512	Image height in pixels
--steps	20	Number of denoising steps
--seed	(random)	Random seed for reproducibility
--s3	(disabled)	Upload generated images to S3

Output

Images are saved to ./generated_images/comfy_{seed}.png.

S3 Upload (Optional)

You can optionally upload generated images to an S3-compatible storage service (AWS S3, Cloudflare R2, Backblaze B2, etc.) by using the --s3 flag.

1. Set environment variables:

export S3_ENDPOINT_URL="https://your-account.r2.cloudflarestorage.com"
export S3_BUCKET_NAME="my-bucket"
export S3_ACCESS_KEY_ID="your-access-key-id"
export S3_SECRET_ACCESS_KEY="your-secret-access-key"

2. Run with S3 upload enabled:

python -m workers.comfyui-json.client --prompt "a beautiful landscape" --s3

Images will be saved locally AND uploaded to s3://{bucket}/comfyui/{filename}.

Note: Requires boto3 (pip install boto3).

Benchmarking

Custom Benchmark Workflows

You can provide a custom ComfyUI workflow for benchmarking. This allows you to test performance using your preferred models and workflow complexity.

Ways to provide the benchmark file (in resolution order — first match wins):

1. Fork this repository and commit your workflow to workers/comfyui-json/misc/benchmark.json.

2. Write the file during provisioning to a path outside the pyworker tree (e.g. /workspace/benchmark.json) and export BENCHMARK_JSON_PATH so the worker can find it. The pyworker repo is cloned by start_server.sh after provisioning runs, so provisioning cannot write into misc/ directly — the destination would be clobbered, or the clone would fail.

3. Run on the vast.ai ComfyUI base image. Its convert-workflows.sh maintains /opt/comfyui-api-wrapper/workflows/pyworker_benchmark.json as a symlink to the first provisioned workflow; the worker reads this automatically when neither of the above is set. No env var required.

   Note on timing: the symlink is created only after convert-workflows.sh finishes converting workflows, which races with the pyworker's own warm-up — both unblock when ComfyUI is ready, but conversion takes a few seconds longer. To avoid losing the race, the worker waits up to 30 seconds for the symlink before falling through. Override the timeout with BENCHMARK_WAIT_TIMEOUT if conversion is slow on your setup. The wait fires only on the well-known tier and only once per worker process; non-base-image deployments skip it (the parent directory /opt/comfyui-api-wrapper/workflows/ doesn't exist, so there's nothing to wait for).

If BENCHMARK_JSON_PATH is set but points at a missing or unreadable file, the worker logs a warning and falls through to the next tier rather than going straight to the SD1.5 fallback.

An example workflow is provided at workers/comfyui-json/misc/benchmark.json.example. To ensure varied generations, use the placeholder __RANDOM_INT__ in place of static seed values — it will be replaced with a random integer for each benchmark run.

Default Benchmark (Fallback)

If benchmark.json is not available, a simple image generation benchmark runs when each worker initializes. This validates GPU performance and helps identify underperforming machines.

The default benchmark uses Stable Diffusion v1.5 with ComfyUI's standard text-to-image workflow. Configure it using these environment variables:

Environment Variable	Default Value	Description
BENCHMARK_JSON_PATH	(unset)	Path to a custom workflow file outside the pyworker tree. Used if misc/benchmark.json is absent. Falls through to /opt/comfyui-api-wrapper/workflows/pyworker_benchmark.json if set but missing.
BENCHMARK_WAIT_TIMEOUT	30	Seconds to wait for the well-known symlink before giving up (base image only).
BENCHMARK_TEST_WIDTH	512	Fallback benchmark: image width (pixels)
BENCHMARK_TEST_HEIGHT	512	Fallback benchmark: image height (pixels)
BENCHMARK_TEST_STEPS	20	Fallback benchmark: number of denoising steps

Each benchmark run uses a random prompt from misc/test_prompts.txt and a random seed to ensure consistent GPU load patterns.

Calibrating Fallback Benchmark Duration

To screen for underperforming hardware, set BENCHMARK_TEST_STEPS to match your expected production workflow duration. This allows you to identify machines that won't meet performance requirements.

Example: If your typical workflow should complete in 90 seconds on acceptable hardware:

# 1. Measure it/sec on your reference machine
# RTX 4090 typically achieves ~43 it/sec with SD1.5

# 2. Calculate required steps
# 90 seconds × 43 it/sec = 3870 steps

# 3. Configure benchmark
export BENCHMARK_TEST_STEPS=3870

# 4. Machines completing significantly slower than 90s indicate hardware issues

Performance expectations:

• Benchmark duration should remain consistent across identical GPU models
• Significant variation (>20%) may indicate thermal, power, or configuration issues

Endpoint

The worker provides a single endpoint:

• /generate/sync: Processes ComfyUI workflows using either predefined modifiers or custom workflow JSON

Request Format

The worker accepts requests in the following format. Choose either modifier mode OR custom workflow mode:

Modifier Mode:

{
  "input": {
    "request_id": "uuid-string",    // optional - UUID generated if not provided
    "modifier": "RawWorkflow",
    "modifications": {
      "prompt": "a beautiful landscape",
      "width": 1024,
      "height": 1024,
      "steps": 20,
      "seed": 123456789
    },
    "s3": { ... },       // optional
    "webhook": { ... }   // optional
  }
}

Custom Workflow Mode:

{
  "input": {
    "request_id": "uuid-string",    // optional - UUID generated if not provided
    "workflow_json": {
      // Complete ComfyUI workflow JSON
    },
    "s3": { ... },       // optional
    "webhook": { ... }   // optional
  }
}

Request Fields

Required Fields

• input: Contains the main workflow data
• input.request_id: Unique identifier for the request

Workflow Mode (Choose One)

You must provide either modifier OR workflow_json, but not both:

Option 1: Modifier Mode

• input.modifier: Name of the predefined workflow modifier (e.g., "Text2Image")
• input.modifications: Parameters to pass to the modifier

Option 2: Custom Workflow Mode

• input.workflow_json: Complete ComfyUI workflow JSON

Optional Fields

• input.s3: S3 configuration for file storage
• input.webhook: Webhook configuration for notifications

These configurations can be provided in the request JSON or via environment variables. Request-level configuration takes precedence over environment variables.

S3 Configuration

Via Request JSON:

"s3": {
  "access_key_id": "your-s3-access-key",
  "secret_access_key": "your-s3-secret-access-key", 
  "endpoint_url": "https://my-endpoint.backblaze.com",
  "bucket_name": "your-bucket",
  "region": "us-east-1"
}

Via Environment Variables:

S3_ACCESS_KEY_ID=your-key
S3_SECRET_ACCESS_KEY=your-secret
S3_BUCKET_NAME=your-bucket
S3_ENDPOINT_URL=https://s3.amazonaws.com
S3_REGION=us-east-1

Webhook Configuration

Via Request JSON:

"webhook": {
  "url": "your-webhook-url",
  "extra_params": {
    "custom_field": "value"
  }
}

Via Environment Variables:

WEBHOOK_URL=https://your-webhook.com  # Default webhook URL
WEBHOOK_TIMEOUT=30                   # Webhook timeout in seconds

Examples

Basic Text-to-Image (Modifier Mode)

{
  "input": {
    "modifier": "Text2Image",
    "modifications": {
      "prompt": "a cat sitting on a windowsill",
      "width": 512,
      "height": 512,
      "steps": 20,
      "seed": 42
    }
  }
}

Custom Workflow Mode

{
  "input": {
    "request_id": "67890",    // optional - using custom ID for tracking
    "workflow_json": {
      "3": {
        "inputs": {
          "seed": 42,
          "steps": 20,
          "cfg": 8,
          "sampler_name": "euler",
          "scheduler": "normal",
          "denoise": 1,
          "model": ["4", 0],
          "positive": ["6", 0],
          "negative": ["7", 0],
          "latent_image": ["5", 0]
        },
        "class_type": "KSampler"
      }
    }
  }
}

Client Libraries

See the client example for implementation details on how to integrate with the ComfyUI worker.

---

See Vast's serverless documentation for more details on how to use ComfyUI with autoscaler.