Reject queued /reserve immediately on busy null workers

A held reservation runs for up to MAX_RESERVATION_SECONDS (default 1h), so
queueing a second /reserve behind it makes no sense — the wait would dwarf
any sane timeout. Set max_queue_time=0.0 so the framework rejects 429 as
soon as another reservation is in flight, and serverless routes the request
to a free worker or scales a new one up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Rob Ballantyne
2026-05-11 17:05:02 +01:00
parent 3668d948be
commit ed0db198c3
2 changed files with 12 additions and 7 deletions
+6 -1
View File
@@ -159,7 +159,12 @@ worker_config = WorkerConfig(
HandlerConfig(
route="/reserve",
allow_parallel_requests=False,
max_queue_time=30.0,
# Reject (429) any /reserve that arrives while the worker is
# already busy. A held reservation lasts up to MAX_RESERVATION_
# SECONDS, so queueing behind it would mean hours of wait —
# better to bounce the request immediately so serverless routes
# it to a free worker (or spins up a new one).
max_queue_time=0.0,
remote_function=reserve_worker,
workload_calculator=lambda _payload: 100.0,
benchmark_config=BenchmarkConfig(