cbalders/homelab-ansible-lxc-meridian
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

litellm: re-index models with local_/proxy_/direct_ prefixes + scaffold OpenAI+Gemini

Browse Source 
Backend-prefix taxonomy so the Open WebUI picker is self-documenting and a
model name can't lie about where it routes:
  local_*  -> Anvil/Ollama (free)        e.g. local_qwen2.5-72b
  proxy_*  -> Claude via Meridian/Max     e.g. proxy_claude-sonnet-4-6
  direct_* -> metered OpenAI/Gemini       e.g. direct_gpt-4o, direct_gemini-2.0-flash

Drops the redundant -max suffix (proxy_ already implies Max). api_base is now
emitted only when a model defines it, so direct_* hit the provider default
endpoint instead of Meridian. direct_* are SCAFFOLDED (no live keys): litellm.env
writes a placeholder so the proxy boots; deploy.sh pulls OPENAI_API_KEY/
GEMINI_API_KEY from Infisical /meridian if present (non-fatal). They 401 until
real keys land.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Your Name
2026-06-05 12:05:55 -04:00
parent b64a95a71b
commit 211d26cc63
4 changed files with 96 additions and 24 deletions
Download Patch File Download Diff File Expand all files Collapse all files
roles/litellm/templates/litellm-config.yaml.j2
+13 -6
View File
@@ -1,17 +1,24 @@
# {{ ansible_managed }}
#
# LiteLLM proxy config. Routes OpenAI-shaped requests to backends:
# - Claude models → Meridian's /v1/messages (same host, :3456), which
# ignores the upstream API key (placeholder passed below).
# - Local models → Anvil's Ollama (openai/ provider, OpenAI-compatible
# endpoint at http://192.168.1.150:11434). Set per-model api_base in vars.
# LiteLLM proxy config. Routes OpenAI-shaped requests to backends by the
# model-name prefix set in vars (litellm_models):
# - proxy_* → Meridian's /v1/messages (same host, :3456), which ignores the
# upstream API key (placeholder); the Max-OAuth sub pays. Explicit api_base.
# - local_* → Anvil's Ollama (OpenAI-compatible, http://192.168.1.150:11434).
# Explicit api_base.
# - direct_* → a public provider (OpenAI/Gemini). NO api_base → LiteLLM uses
# the provider default endpoint; api_key reads os.environ/<PROVIDER>_API_KEY.
# api_base is emitted only when a model defines it; omit it to reach a provider
# default.
model_list:
{% for m in litellm_models %}
- model_name: {{ m.name }}
litellm_params:
model: {{ m.backend }}
api_base: {{ m.api_base | default('http://127.0.0.1:' ~ meridian_port) }}
{% if m.api_base is defined %}
api_base: {{ m.api_base }}
{% endif %}
api_key: {{ m.api_key | default('placeholder-meridian-ignores-this') }}
{% endfor %}