Reject queued /reserve immediately on busy null workers
A held reservation runs for up to MAX_RESERVATION_SECONDS (default 1h), so queueing a second /reserve behind it makes no sense — the wait would dwarf any sane timeout. Set max_queue_time=0.0 so the framework rejects 429 as soon as another reservation is in flight, and serverless routes the request to a free worker or scales a new one up. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -159,7 +159,12 @@ worker_config = WorkerConfig(
|
||||
HandlerConfig(
|
||||
route="/reserve",
|
||||
allow_parallel_requests=False,
|
||||
max_queue_time=30.0,
|
||||
# Reject (429) any /reserve that arrives while the worker is
|
||||
# already busy. A held reservation lasts up to MAX_RESERVATION_
|
||||
# SECONDS, so queueing behind it would mean hours of wait —
|
||||
# better to bounce the request immediately so serverless routes
|
||||
# it to a free worker (or spins up a new one).
|
||||
max_queue_time=0.0,
|
||||
remote_function=reserve_worker,
|
||||
workload_calculator=lambda _payload: 100.0,
|
||||
benchmark_config=BenchmarkConfig(
|
||||
|
||||
Reference in New Issue
Block a user