Benson Wong
533162ce6a
add support for automatically unloading a model ( #10 ) ( #14 )
...
* Make starting upstream process on-demand (#10 )
* Add automatic unload of model after TTL is reached
* add `ttl` configuration parameter to models in seconds, default is 0 (never unload)
2024-11-19 16:32:51 -08:00
Benson Wong
ba39ed4c18
Add support for legacy v1/completions API ( #12 )
2024-11-19 09:57:39 -08:00
Benson Wong
7eec51f3f2
Dechunk HTTP requests by default ( #11 )
...
ProxyManager already has all the Request body's data. There is no never
a need to use chunked transfer encoding to the upstream process.
2024-11-19 09:40:44 -08:00
Benson Wong
c9233d2c9a
use gin instead of standard http lib in main
2024-11-18 15:58:28 -08:00
Benson Wong
401aa88949
move log handlers to separate file
2024-11-18 15:33:06 -08:00
Benson Wong
e9e88fd229
rename proxy.go to proxymanager.go
2024-11-18 15:30:34 -08:00
