andreas/llama-swap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

proxy: add unload of single model (#318)

Browse Source 
This adds a new API endpoint, /api/models/unload/*model, that unloads a single model. In the UI when a model is in a ReadyState it will have a new button to unload it. 

Fixes #312
This commit is contained in:
Benson Wong
2025-09-24 20:53:48 -07:00
committed by GitHub
parent fc3bb716df
commit 1a84926505
7 changed files with 138 additions and 14 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
proxy/proxymanager.go
View File

@@ -228,7 +228,6 @@ func (pm *ProxyManager) setupGinEngine() {
c.Redirect(http.StatusFound, "/ui/models")
})
pm.ginEngine.Any("/upstream/*upstreamPath", pm.proxyToUpstream)
pm.ginEngine.GET("/unload", pm.unloadAllModelsHandler)
pm.ginEngine.GET("/running", pm.listRunningProcessesHandler)
pm.ginEngine.GET("/health", func(c *gin.Context) {