andreas/llama-swap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
52 Commits 1 Branch 0 Tags
533162ce6af65e5cd67fba573a311d0855f0d858
Commit Graph

11 Commits

Author SHA1 Message Date
Benson Wong
533162ce6a add support for automatically unloading a model (#10) (#14) 
* Make starting upstream process on-demand (#10)
* Add automatic unload of model after TTL is reached
* add `ttl` configuration parameter to models in seconds, default is 0 (never unload)
 2024-11-19 16:32:51 -08:00
Benson Wong
a33ac6f8fb update README 2024-11-18 15:37:50 -08:00
Benson Wong
a8e5ee13b9 Add logging with pipes example to README 2024-11-15 09:10:43 -08:00
Benson Wong
0f133f5b74 Add /logs endpoint to monitor upstream processes 
- outputs last 10KB of logs from upstream processes
- supports streaming
 2024-10-30 21:02:30 -07:00
Benson Wong
1510b3fbd9 clean up README 2024-10-22 10:37:45 -07:00
Benson Wong
0f8a8e70f1 add header image 2024-10-22 10:30:30 -07:00
Benson Wong
8eb5b7b6c4 Add custom check endpoint 
Replace previously hardcoded value for `/health` to check when the
server became ready to serve traffic. With this the server can support
any server that provides an an OpenAI compatible inference endpoint.
 2024-10-11 21:59:21 -07:00
Benson Wong
4fae7cf946 update docs 2024-10-04 21:11:08 -07:00
Benson Wong
cc944251df update README 2024-10-04 20:43:48 -07:00
Benson Wong
d682589fb1 support environment variables 2024-10-04 11:55:27 -07:00
Benson Wong
43119e807f add README 2024-10-04 11:37:51 -07:00