Claude Claude Sonnet 5 commited on
Commit
490468e
·
unverified ·
1 Parent(s): 30eec79

Use LLAMA_API_KEY (not gateway's own API_KEY) to authenticate to llama-server

Browse files

The previous fix reused the gateway's own API_KEY secret, but that's
unset on this Space and, more fundamentally, is the wrong secret:
llama.cpp maps --api-key to the env var LLAMA_API_KEY specifically
(not the generic LLAMA_ARG_* pattern used for other flags), and this
Space already has a LLAMA_API_KEY secret configured for llama-server
that nothing was wiring up. Read that env var directly and attach it
to every gateway -> llama-server request instead.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XvAfE7bsSLrQm1VVBswYnx

Files changed (1) hide show
  1. gateway.py +8 -1
gateway.py CHANGED
@@ -22,6 +22,7 @@ DEFAULT_LANGUAGE = os.getenv("DEFAULT_LANGUAGE", "en")
22
  MAX_QUEUE_SIZE = int(os.getenv("MAX_QUEUE_SIZE", "32"))
23
  REQUEST_TIMEOUT = float(os.getenv("REQUEST_TIMEOUT_SECONDS", "900"))
24
  API_KEY = os.getenv("API_KEY", "").strip()
 
25
  ENGLISH_GUARD = (
26
  "Respond entirely in English unless the user explicitly requests another "
27
  "language. Preserve code, identifiers, and quoted source material exactly."
@@ -123,7 +124,13 @@ async def wait_for_llama() -> None:
123
 
124
 
125
  def llama_auth_headers() -> dict[str, str]:
126
- return {"Authorization": f"Bearer {API_KEY}"} if API_KEY else {}
 
 
 
 
 
 
127
 
128
 
129
  async def execute_completion(payload: dict[str, Any]) -> dict[str, Any]:
 
22
  MAX_QUEUE_SIZE = int(os.getenv("MAX_QUEUE_SIZE", "32"))
23
  REQUEST_TIMEOUT = float(os.getenv("REQUEST_TIMEOUT_SECONDS", "900"))
24
  API_KEY = os.getenv("API_KEY", "").strip()
25
+ LLAMA_API_KEY = os.getenv("LLAMA_API_KEY", "").strip()
26
  ENGLISH_GUARD = (
27
  "Respond entirely in English unless the user explicitly requests another "
28
  "language. Preserve code, identifiers, and quoted source material exactly."
 
124
 
125
 
126
  def llama_auth_headers() -> dict[str, str]:
127
+ # llama-server reads its own API key from the LLAMA_API_KEY env var
128
+ # (llama.cpp maps --api-key to that specific name, not the generic
129
+ # LLAMA_ARG_* pattern), independently of this gateway's own API_KEY.
130
+ # The gateway holds the same secret (it's set on the whole
131
+ # container), so attach it on every request we make to llama-server
132
+ # rather than relying on whatever the external caller supplied to us.
133
+ return {"Authorization": f"Bearer {LLAMA_API_KEY}"} if LLAMA_API_KEY else {}
134
 
135
 
136
  async def execute_completion(payload: dict[str, Any]) -> dict[str, Any]: