litellm: re-index models with local_/proxy_/direct_ prefixes + scaffold OpenAI+Gemini
Backend-prefix taxonomy so the Open WebUI picker is self-documenting and a model name can't lie about where it routes: local_* -> Anvil/Ollama (free) e.g. local_qwen2.5-72b proxy_* -> Claude via Meridian/Max e.g. proxy_claude-sonnet-4-6 direct_* -> metered OpenAI/Gemini e.g. direct_gpt-4o, direct_gemini-2.0-flash Drops the redundant -max suffix (proxy_ already implies Max). api_base is now emitted only when a model defines it, so direct_* hit the provider default endpoint instead of Meridian. direct_* are SCAFFOLDED (no live keys): litellm.env writes a placeholder so the proxy boots; deploy.sh pulls OPENAI_API_KEY/ GEMINI_API_KEY from Infisical /meridian if present (non-fatal). They 401 until real keys land. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -30,6 +30,25 @@ if [[ -z "${LITELLM_MASTER_KEY:-}" ]]; then
|
||||
export LITELLM_MASTER_KEY
|
||||
fi
|
||||
|
||||
# Optional provider keys for direct_* models. Non-fatal: if a key isn't in
|
||||
# Infisical /meridian yet, litellm.env falls back to a placeholder and the
|
||||
# direct_* model 401s on call (proxy_* + local_* keep working). Drop the secret
|
||||
# into Infisical /meridian to activate, then re-deploy.
|
||||
for keyvar in OPENAI_API_KEY GEMINI_API_KEY; do
|
||||
if [[ -z "${!keyvar:-}" ]]; then
|
||||
secret_name="vault_$(echo "$keyvar" | tr '[:upper:]' '[:lower:]')"
|
||||
val="$(infisical secrets get "$secret_name" \
|
||||
--projectId 50062d7c-06ff-4d5c-8ca3-6c0cdba9f270 \
|
||||
--env prod --path /meridian --plain 2>/dev/null || true)"
|
||||
if [[ -n "$val" ]]; then
|
||||
echo "==> Pulled ${keyvar} from Infisical (direct_* enabled)."
|
||||
export "$keyvar=$val"
|
||||
else
|
||||
echo "==> ${keyvar} not in Infisical /meridian — direct_* for this provider stays scaffolded (401 until set)."
|
||||
fi
|
||||
fi
|
||||
done
|
||||
|
||||
echo "==> Checking connectivity to ${HOST_USER}@${HOST_IP} ..."
|
||||
if ! ssh -o ConnectTimeout=5 -o BatchMode=yes "${HOST_USER}@${HOST_IP}" true 2>/dev/null; then
|
||||
echo " Cannot SSH to ${HOST_IP} — refreshing host key ..."
|
||||
|
||||
Reference in New Issue
Block a user