From 6667e307a2b918d8fa131baaec5f7f996712f3a0 Mon Sep 17 00:00:00 2001 From: Benson Wong Date: Sat, 8 Feb 2025 10:28:35 -0800 Subject: [PATCH] Update README.md --- README.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index dee7a17..80d11dc 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,9 @@ When a request is made to an OpenAI compatible endpoint, lama-swap will extract In the most basic configuration llama-swap handles one model at a time. For more advanced use cases, the `profiles` feature can load multiple models at the same time. You have complete control over how your system resources are used. ## Do I need to use llama.cpp's server (llama-server)? -Any OpenAI compatible server would work. llama-swap was originally designed for llama-server and it is the best supported. For Python based inference servers like vllm or tabbyAPI it is recommended to run them via podman. This provides clean environment isolation as well as responding correctly to `SIGTERM` signals to shutdown. +Any OpenAI compatible server would work. llama-swap was originally designed for llama-server and it is the best supported. + +For Python based inference servers like vllm or tabbyAPI it is recommended to run them via podman or docker. This provides clean environment isolation as well as responding correctly to `SIGTERM` signals to shutdown. ## Features: