update litellm docs
All checks were successful
Podman DDNS Image / build-and-push-ddns (push) Successful in 1m22s

This commit is contained in:
2026-05-11 18:22:14 -04:00
parent f2015e2c71
commit a56402c2cc
9 changed files with 169 additions and 147 deletions

View File

@@ -9,9 +9,8 @@
- [Convert litellm compose spec to quadlets](#convert-litellm-compose-spec-to-quadlets)
- [Create the litellm.env file](#create-the-litellmenv-file)
- [Start and enable your systemd quadlet](#start-and-enable-your-systemd-quadlet)
- [Install via Ansible](#install-via-ansible)
- [Expose litellm](#expose-litellm)
- [Using LiteLLM](#using-litellm)
- [Adding Models](#adding-models)
- [Testing Models](#testing-models)
- [Backup litellm](#backup-litellm)
- [Upgrade litellm](#upgrade-litellm)
@@ -110,63 +109,28 @@ journalctl --user -u litellm -f
systemctl --user enable --now podman-auto-update.timer
```
### Install via Ansible
Preview changes with a dry run:
```bash
ansible-playbook -i ansible/inventory.yaml active/container_litellm/playbook.yml --check --diff
```
Run the playbook from the Homelab root:
```bash
ansible-playbook -i ansible/inventory.yaml active/container_litellm/playbook.yml
```
This copies the quadlets, config, reloads the systemd user daemon, and starts both `litellm-db` and `litellm` services as the `ai` user.
### Expose litellm
1. If you need a domain, follow the [DDNS instructions](/active/container_ddns/ddns.md#install-a-new-ddns-service)
2. For a web service, follow the [Caddy instructions](/active/container_caddy/caddy.md#adding-a-new-caddy-record)
3. Finally, follow your OS's guide for opening ports via its firewall service.
## Using LiteLLM
### Adding Models
```json
// qwen3.5-35b-a3b-thinking
{
"temperature": 1,
"top_p": 0.95,
"presence_penalty": 1.5,
"extra_body": {
"top_k": 20,
"min_p": 0,
"repetition_penalty": 1,
"chat_template_kwargs": {
"enable_thinking": true
}
}
}
// qwen3.5-35b-a3b-coding
{
"temperature": 0.6,
"top_p": 0.95,
"presence_penalty": 0,
"extra_body": {
"top_k": 20,
"min_p": 0,
"repetition_penalty": 1,
"chat_template_kwargs": {
"enable_thinking": true
}
}
}
// qwen3.5-35b-a3b-instruct
{
"temperature": 0.7,
"top_p": 0.8,
"presence_penalty": 1.5,
"extra_body": {
"top_k": 20,
"min_p": 0,
"repetition_penalty": 1,
"chat_template_kwargs": {
"enable_thinking": false
}
}
}
```
### Testing Models
```bash
@@ -191,7 +155,7 @@ curl -L -X POST 'https://aipi.reeseapps.com/v1/chat/completions' \
## Backup litellm
Follow the [Borg Backup instructions](/active/systemd_borg/borg.md#set-up-a-client-for-backup)
Follow the [Borg Backup instructions](/active/software_borg/borg.md#set-up-a-client-for-backup)
## Upgrade litellm