andreas/llama-swap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Improve timeout and exit handling of child processes. fix #3 and #5

Browse Source 
llama-swap only waited a maximum of 5 seconds for an upstream
HTTP server to be available. If it took longer than that it will error
out the request. Now it will wait up to the configured healthCheckTimeout
or the upstream process unexpectedly exits.
This commit is contained in:
Benson Wong
2024-11-01 14:32:39 -07:00
parent 8448efa7fc
commit 34f9fd7340
2 changed files with 35 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
config.example.yaml
View File

@@ -1,6 +1,6 @@
# Seconds to wait for llama.cpp to be available to serve requests
# Default (and minimum): 15 seconds
healthCheckTimeout: 60
healthCheckTimeout: 15
models:
"llama":