LiteLLM
A LiteLLM proxy exposing an OpenAI-compatible endpoint, selected with LLM_PROVIDER=litellm, used to extract facts from conversations.
Enable it
bash
LLM_PROVIDER=litellm
OPENAI_BASE_URL=http://localhost:4000
OPENAI_API_KEY=<litellm-proxy-key>
OPENAI_LLM_MODEL=<model-name-configured-in-litellm>Configuration
| Variable | Default | Description |
|---|---|---|
LLM_PROVIDER | openai | Set to litellm. |
OPENAI_BASE_URL | https://api.openai.com/v1 | Point at your LiteLLM proxy: http://localhost:4000. |
OPENAI_API_KEY | (none) | Proxy key, sent as a bearer token. |
OPENAI_LLM_MODEL | gpt-4.1-nano | A model name routed by your LiteLLM config. |
Notes
- OpenAI-compatible: the provider reuses the OpenAI request handling and only differs by base URL, key, and model.
- The provider also reads
LITELLM_BASE_URL(defaulthttp://localhost:4000) andLITELLM_API_KEYas its own fallbacks. - LiteLLM itself fans out to whatever downstream models its proxy config defines.