llama-swap

Files

Benson Wong cf82b3c633 Improve Concurrency and Parallel Request Handling (#19 )

Rewrite the swap behaviour so that in-flight requests block process swapping until they are completed. 

Additionally: 

- add tests for parallel requests with proxy.ProxyManager and proxy.Process
- improve Process startup behaviour and simplified the code 
- stopping of processes are sent SIGTERM and have 5 seconds to terminate, before they are killed

2024-11-30 15:24:42 -08:00

config_test.go

Implement Multi-Process Handling (#7 )

2024-11-23 19:45:13 -08:00

config.go

Implement Multi-Process Handling (#7 )

2024-11-23 19:45:13 -08:00

helpers_test.go

Improve Concurrency and Parallel Request Handling (#19 )

2024-11-30 15:24:42 -08:00

logMonitor_test.go

Improve LogMonitor to handle empty writes and ensure buffer immutability