Benson Wong
9fc0431531
Clean up and Documentation ( #347 ) [skip ci]
...
* cmd,misc: move misc binaries to cmd/
* docs: add docs and move examples/ there
* misc: remove unused misc/assets dir
* docs: add configuration.md
* update README with better structure
Updates: #334
2025-10-19 14:53:13 -07:00
Benson Wong
caf9e98b1e
Fix race conditions in proxy.Process ( #349 )
...
- Fix data races found in proxy.Process by go's race detector.
- Add data race detection to the CI tests.
Fixes #348
2025-10-13 16:42:49 -07:00
Benson Wong
70930e4e91
proxy: add support for user defined metadata in model configs ( #333 )
...
Changes:
- add Metadata key to ModelConfig
- include metadata in /v1/models under meta.llamaswap key
- add recursive macro substitution into Metadata
- change macros at global and model level to be any scalar type
Note:
This is the first mostly AI generated change to llama-swap. See #333 for notes about the workflow and approach to AI going forward.
2025-10-04 19:56:41 -07:00
Benson Wong
216c40b951
proxy/config: create config package and migrate configuration ( #329 )
...
* proxy/config: create config package and migrate configuration
The configuration is become more complex as llama-swap adds more
advanced features. This commit moves config to its own package so it can
be developed independently of the proxy package.
Additionally, enforcing a public API for a configuration will allow
downstream usage to be more decoupled.
2025-09-28 16:50:06 -07:00
Aaron Ang
6307bd3205
Add support for building Linux ARM64 binary in Makefile ( #221 )
2025-08-05 16:26:06 -07:00
Benson Wong
54c519e365
update Makefile to install ui deps
2025-06-17 09:54:01 -07:00
Benson Wong
9a3c656738
New UI ( #157 , #164 )
...
- Add a react UI to replace the plain HTML one.
- Serve as a foundation for better GUI interactions
2025-06-16 16:45:19 -07:00
Benson Wong
d7b390df74
Add GH Action for Testing on Windows ( #132 )
...
* Add windows specific test changes
* Change the command line parsing library - Possible breaking changes for windows users!
2025-05-14 21:51:53 -07:00
Benson Wong
7f37bcc6eb
Improve testing around using SIGKILL ( #127 )
...
* Add test for SIGKILL of process
* silent TestProxyManager_RunningEndpoint debug output
* Ref #125
2025-05-13 21:21:52 -07:00
Benson Wong
0815bb4cc3
Add windows to goreleaser #54
2025-02-18 17:26:43 -08:00
daschiller
7187cfe52e
add Windows build support to Makefile ( #54 )
2025-02-18 17:24:31 -08:00
Benson Wong
13d4552edc
Add FreeBSD/amd64 to auto built releases ( #51 )
2025-02-13 16:44:31 -08:00
Benson Wong
d6ca535939
tweak release tagging so it is not based on number of commits
2024-12-14 15:46:10 -08:00
Benson Wong
27302c0c02
change llama-swap to use goreleaser default ldflag values
2024-12-14 10:30:06 -08:00
Benson Wong
4c94927658
Move release to Makefile out of goreleaser
...
- less complexity
- easier
- goreleaser, github, pipelines: 1... mostlygeek: 0
2024-12-14 10:16:46 -08:00
Benson Wong
22d3f1a4f9
Change versioning to use git commits counts instead of semver
...
- less work for me
- more frequent releases
2024-12-14 09:53:13 -08:00
Benson Wong
533162ce6a
add support for automatically unloading a model ( #10 ) ( #14 )
...
* Make starting upstream process on-demand (#10 )
* Add automatic unload of model after TTL is reached
* add `ttl` configuration parameter to models in seconds, default is 0 (never unload)
2024-11-19 16:32:51 -08:00
Benson Wong
e5c909ddf7
add tests for proxy.Process
2024-11-17 20:49:14 -08:00
Benson Wong
8cf2a389d8
Refactor log implementation
...
- use []byte instead of unnecessary string conversions
- make LogManager.Broadcast private
- make LogManager.GetHistory public
- add tests
2024-10-31 12:16:54 -07:00
Benson Wong
ef05c05f9c
renaming to llama-swap
2024-10-04 20:21:11 -07:00
Benson Wong
e0103d1884
build simple-responder with make all
2024-10-04 12:14:10 -07:00
Benson Wong
d682589fb1
support environment variables
2024-10-04 11:55:27 -07:00
Benson Wong
aaca9d889b
add Makefile
2024-10-04 11:07:00 -07:00
