Introduce Groups Feature (#107)

Groups allows more control over swapping behaviour when a model is requested. The new groups feature provides three ways to control swapping: within the group, swapping out other groups or keep the models in the group loaded persistently (never swapped out). 

Closes #96, #99 and #106.
This commit is contained in:
Benson Wong
2025-05-02 22:35:38 -07:00
committed by GitHub
parent ec0348e431
commit 448ccae959
10 changed files with 754 additions and 482 deletions

View File

@@ -301,6 +301,10 @@ func (p *Process) start() error {
}
func (p *Process) Stop() {
if !isValidTransition(p.CurrentState(), StateStopping) {
return
}
// wait for any inflight requests before proceeding
p.inFlightRequests.Wait()
p.proxyLogger.Debugf("<%s> Stopping process", p.ID)