Commit d80619397 for llama.cpp
commit d80619397c2ba5cf1366c29f7d43699ca3a159b9
Author: Peter van Velzen <info@pcvanvelzen.nl>
Date: Tue Oct 20 13:23:41 2026 +0200
server : pass id_slot through the Anthropic /v1/messages conversion (#28554)
* server : pass id_slot through the Anthropic /v1/messages conversion
server_chat_convert_anthropic_to_oai copies a small whitelist of common
params from the Anthropic request into the internal OAI-style body and
left id_slot out. Every other completion endpoint honours id_slot; a
proxy that pins a session to the slot holding its KV cache needs the
same on /v1/messages, or it silently falls back to automatic slot
selection and, on hybrid/recurrent models, pays a full re-prefill.
Adds a test asserting a request with id_slot set is served by that
slot, observed via GET /slots.
* server : shorten comments
* rm test
---------
Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
diff --git a/tools/server/server-chat.cpp b/tools/server/server-chat.cpp
index a3a4ea152..c5473417c 100644
--- a/tools/server/server-chat.cpp
+++ b/tools/server/server-chat.cpp
@@ -590,7 +590,7 @@ json server_chat_convert_anthropic_to_oai(const json & body) {
}
// Pass through common params
- for (const auto & key : {"temperature", "top_p", "top_k", "stream", "chat_template_kwargs"}) {
+ for (const auto & key : {"temperature", "top_p", "top_k", "stream", "chat_template_kwargs", "id_slot"}) {
if (body.contains(key)) {
oai_body[key] = body.at(key);
}