paperless-ai kept returning "Invalid JSON response from API" after the
real cause of the large failures was fixed (TOKEN_LIMIT=128000 against a
16386 window; requests fell from 44k to 2.1k tokens and truncation
stopped). Something in the hop was still mangling the reply.
Isolated by replaying paperless-ai's exact system prompt 3x per path,
identical model, temperature and context:
Anvil /v1/chat/completions (OpenAI-compat) -> 3/3 VALID
Anvil /api/chat (native) -> 3/3 VALID
via LiteLLM ollama_chat/ -> invalid every time
Both ollama endpoints are clean, so the fault is the ollama_chat provider
transformation. openai/ is a near-passthrough to the endpoint already
proven good.
Deliberately NOT enabling LiteLLM verbose logging to confirm this from the
inside: those request bodies are tax and medical documents, and writing
them to Loki would undo the privacy property that put paperless-ai on a
local model in the first place.
Scoped to this one route to keep the blast radius small; the other local_*
entries are untouched.
Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
