Commit Graph

24 Commits

Author SHA1 Message Date
Colter Downing 8d9ffb3a6c removed 5 sec sleep and warmup request on load 2025-10-28 15:28:30 -07:00
Colter Downing 5d5bc197d7 adding timings for cold start 2025-10-26 18:44:23 -07:00
Colter Downing bcecd6df40 Suppress matplot debug logs 2025-10-25 16:18:02 -07:00
Lucas Armand 37ad3f8d46 asyncio in metrics 2025-10-23 10:18:31 -07:00
Lucas Armand 5edfa968ca async sleep 2025-10-21 18:49:48 -07:00
Lucas Armand 5b5ef7227a nvm moved it here 2025-10-21 18:20:11 -07:00
Lucas Armand 16990ff8ff move start request 2025-10-21 18:18:44 -07:00
Lucas Armand 9748176366 fixed semaphore acquire bool 2025-10-21 18:12:23 -07:00
Lucas Armand b39193ae70 check for sem acquire 2025-10-21 18:02:14 -07:00
Lucas Armand 9a6ca5d412 added versioning 2025-10-21 15:42:43 -07:00
Lucas Armand e9ba1b03e4 Use delete_requests and track request_idxs 2025-10-21 11:59:35 -07:00
Colter Downing 25db78e39d Fix healthcheck with separate session 2025-10-01 18:04:31 -07:00
Rob Ballantyne 1e4fa87437 Prevent timeout and allow long running connections 2025-08-28 15:48:57 +01:00
Rob Ballantyne 4c5fa03c7b adds import for ClientTimeout 2025-08-27 20:54:27 +01:00
Rob Ballantyne a8fe74f771 Remove default 300s timeout 2025-08-27 18:34:45 +01:00
Nader Arbabian c595b42410 for benchmarking, use concurrent requests (#26) 2025-08-11 12:39:28 -07:00
Nader Arbabian 4016cf9a53 redo metrics tracking for requests, fixes bug wherere some requests were marked as pending, even though they had finished (#24) 2025-08-08 17:01:21 -07:00
Nader Arbabian be2aafdb1f fix pyright errors + revert to old way of handling cancelled api requests (#23) 2025-07-17 16:59:06 -07:00
Nader Arbabian 6fb610cb5b fix pyworker miscounting active connections (#20)
* fix pyworker miscounting active connections

* clean up some issues

* add option to skip auth
2025-07-15 15:33:27 -07:00
Nader Arbabian 9ebf1924ea don't healthcheck endpoints until model is loaded and benchmarks have run 2025-06-11 15:26:50 -07:00
Abiola Akinnubi 71ed54ebe4 Endpoint update pr one (#1)
* Added endpoint flexibility along with existing log. extended the log support

* Switched Endpoint back to vast-ai, Added endpoint flexibility along with existing log. extended the log support

* Modify the endpoint return type as optional and check via pyright to ensure there are not compilation/type errors

* Added endpoint flexibility along with existing log. extended the log support

Switched Endpoint back to vast-ai, Added endpoint flexibility along with existing log. extended the log support

Modify the endpoint return type as optional and check via pyright to ensure there are not compilation/type errors

* Endpoint Utils and API changes
2025-06-02 18:43:27 -07:00
Abiola Akinnubi b1ca68c349 Added endpoint flexibility along with existing log. extended the log support
Switched Endpoint back to vast-ai, Added endpoint flexibility along with existing log. extended the log support

Modify the endpoint return type as optional and check via pyright to ensure there are not compilation/type errors
2025-05-30 14:40:42 -07:00
Nader Arbabian f15c7a3540 fix wrong aiohttp version and add pubkey fetch retry 2025-04-02 17:01:45 -07:00
Nader Arbabian 589216d15f initial commit 2024-09-04 11:19:30 -07:00