Benson Wong
34f9fd7340
Improve timeout and exit handling of child processes. fix #3 and #5
...
llama-swap only waited a maximum of 5 seconds for an upstream
HTTP server to be available. If it took longer than that it will error
out the request. Now it will wait up to the configured healthCheckTimeout
or the upstream process unexpectedly exits.
2024-11-01 14:32:39 -07:00
Benson Wong
8448efa7fc
revise health check logic to not error on 5 second timeout
2024-11-01 09:42:37 -07:00
Benson Wong
8cf2a389d8
Refactor log implementation
...
- use []byte instead of unnecessary string conversions
- make LogManager.Broadcast private
- make LogManager.GetHistory public
- add tests
2024-10-31 12:16:54 -07:00
Benson Wong
0f133f5b74
Add /logs endpoint to monitor upstream processes
...
- outputs last 10KB of logs from upstream processes
- supports streaming
2024-10-30 21:02:30 -07:00
Benson Wong
6c3819022c
Add compatibility with OpenAI /v1/models endpoint to list models
2024-10-21 15:38:12 -07:00
Benson Wong
be82d1a6a0
Support multiline cmds in YAML configuration
...
Add support for multiline `cmd` configurations allowing for nicer looking configuration YAML files.
2024-10-19 20:06:59 -07:00
Benson Wong
8eb5b7b6c4
Add custom check endpoint
...
Replace previously hardcoded value for `/health` to check when the
server became ready to serve traffic. With this the server can support
any server that provides an an OpenAI compatible inference endpoint.
2024-10-11 21:59:21 -07:00
Benson Wong
476086c066
Add Cmd.Wait() to prevent creation of zombie child processes see: #1
2024-10-04 21:38:29 -07:00
Benson Wong
85743ad914
remove the v1/models endpoint, needs improvement
2024-10-04 12:33:41 -07:00
Benson Wong
3e90f8328d
add /v1/models endpoint and proxy everything to llama-server
2024-10-04 12:28:50 -07:00
Benson Wong
d682589fb1
support environment variables
2024-10-04 11:55:27 -07:00
Benson Wong
bfdba43bd8
improve error handling
2024-10-04 10:55:02 -07:00
Benson Wong
2d387cf373
rename proxy.go to manager.go
2024-10-04 09:39:10 -07:00
Benson Wong
d061819fb1
moved config into proxy package
2024-10-04 09:38:30 -07:00
Benson Wong
83415430ba
move proxy logic into the proxy package
2024-10-03 21:35:33 -07:00
