cbalders/homelab-ansible-lxc-meridian
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(litellm): re-add gpt-4o-mini probe alias → free local

Browse Source 
paperless-ai's Settings validation hard-codes a gpt-4o-mini probe
(independent of OPENAI_MODEL); the bare alias was removed in the
2026-05-29 prefix-taxonomy refactor, so the probe 400s and blocks
saving settings. Re-add a single narrowly-scoped shadow alias routed to
free local llama3.1:8b (the one probe name unreachable via a prefix).
Trade-off note updated so this isn't read as a regression.

Co-Authored-By: Claude Opus 4.8 (1M context) <[email protected]>
This commit is contained in:
Your Name
2026-06-21 15:57:05 -04:00
co-authored by Claude Opus 4.8
parent 02c2f4ee2d
commit f33aad89a2
1 changed files with 12 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
vars/main.yml
+12
View File
@@ -72,6 +72,11 @@ anvil_ollama_base: "http://192.168.1.150:11434"
# Trade-off: clients that hard-code a bare model name in a probe (paperless-ai # Trade-off: clients that hard-code a bare model name in a probe (paperless-ai
# wizard hits gpt-4o-mini) 400 until pointed at the prefixed name — or, for # wizard hits gpt-4o-mini) 400 until pointed at the prefixed name — or, for
# OpenAI specifically, at direct_gpt-4o-mini once a key is in Infisical. # OpenAI specifically, at direct_gpt-4o-mini once a key is in Infisical.
# Narrow exception (re-added 2026-06-21): a single `gpt-4o-mini` shadow alias
# is kept below SOLELY for paperless-ai's hardcoded validation probe, routed to
# free local llama3.1:8b. This is NOT a return to the bare-name scheme — it's
# the one probe name we can't reach via a prefix (paperless-ai ignores
# OPENAI_MODEL for the probe). Don't add more bare aliases.
# #
# Single GPU, OLLAMA_NUM_PARALLEL=1 — concurrent/mixed requests queue and the # Single GPU, OLLAMA_NUM_PARALLEL=1 — concurrent/mixed requests queue and the
# 70B+8B can't both stay resident in the ~62 GB budget (expect model swaps). # 70B+8B can't both stay resident in the ~62 GB budget (expect model swaps).
@@ -89,6 +94,13 @@ litellm_models:
backend: ollama_chat/llama3.1:8b backend: ollama_chat/llama3.1:8b
api_base: "{{ anvil_ollama_base }}" api_base: "{{ anvil_ollama_base }}"
api_key: ollama-no-auth api_key: ollama-no-auth
# Shadow alias SOLELY for paperless-ai's hardcoded gpt-4o-mini validation
# probe (see the "Narrow exception" note above). Routed to free local
# llama3.1:8b so the probe costs nothing. Do not point real traffic here.
- name: gpt-4o-mini
backend: ollama_chat/llama3.1:8b
api_base: "{{ anvil_ollama_base }}"
api_key: ollama-no-auth
# Embeddings don't actually route through LiteLLM 1.55.10 (broken async ollama # Embeddings don't actually route through LiteLLM 1.55.10 (broken async ollama
# handler → 500); consumers hit Anvil direct. Kept as a catalog entry only. # handler → 500); consumers hit Anvil direct. Kept as a catalog entry only.
- name: local_nomic-embed-text - name: local_nomic-embed-text