andreas/llama-swap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

revise health check logic to not error on 5 second timeout

Browse Source
This commit is contained in:
Benson Wong
2024-11-01 09:42:37 -07:00
parent 8cf2a389d8
commit 8448efa7fc
2 changed files with 18 additions and 12 deletions
Download Patch File Download Diff File Expand all files Collapse all files

7
config.example.yaml
View File

@@ -35,7 +35,10 @@ models:
# until the upstream server is ready for traffic
checkEndpoint: none
# don't use this, just for testing if things are broken
# don't use these, just for testing if things are broken
"broken":
cmd: models/llama-server-osx --port 8999 -m models/doesnotexist.gguf
proxy: http://127.0.0.1:8999
proxy: http://127.0.0.1:8999
"broken_timeout":
cmd: models/llama-server-osx --port 8999 -m models/Qwen2.5-1.5B-Instruct-Q4_K_M.gguf
proxy: http://127.0.0.1:9000