update ai stack with turbo

This commit is contained in:
2026-05-25 21:40:56 -04:00
parent a56402c2cc
commit b531e75413
8 changed files with 74 additions and 27 deletions
+32
View File
@@ -31,7 +31,10 @@
- [Nomic Embedding](#nomic-embedding)
- [llama.cpp](#llamacpp)
- [stable-diffusion.cpp](#stable-diffusioncpp)
- [STT whisper.cpp](#stt-whispercpp)
- [TTS coqui-tts](#tts-coqui-tts)
- [open-webui](#open-webui)
- [Upgrade Open Webui](#upgrade-open-webui)
- [lite-llm](#lite-llm)
- [Install Services with Quadlets](#install-services-with-quadlets)
- [API Keys](#api-keys)
@@ -526,6 +529,28 @@ localhost/stable-diffusion-cpp:latest \
-p "A normal business meeting. People discuss business for 2 seconds. Suddenly, a horde of furries carrying assault rifles bursts into the room and causes a panic. Hatsune Miku leads the charge screaming in rage."
```
## STT whisper.cpp
Whisper stt service.
```bash
podman run -it --rm -p "8081:8080" \
-v /home/ai/models/stt:/models:z \
--device /dev/kfd \
--device /dev/dri \
ghcr.io/ggml-org/whisper.cpp:main-vulkan \
"whisper-server \
--host 0.0.0.0 \
-m /models/ggml-large-v3-turbo/ggml-large-v3-turbo-q5_0.bin \
--convert"
```
## TTS coqui-tts
```bash
podman run --rm -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/coqui-ai/tts-cpu
```
## open-webui
```bash
@@ -552,6 +577,13 @@ Use the following connections:
| stable-diffusion.cpp | <http://host.containers.internal:1234/v1> |
| stable-diffusion.cpp edit | <http://host.containers.internal:1235/v1> |
### Upgrade Open Webui
```bash
podman image pull ghcr.io/open-webui/open-webui:main
systemctl --user restart open-webui.service
```
## lite-llm
<https://docs.litellm.ai/docs/proxy/configs>