update ai stack with turbo
This commit is contained in:
@@ -31,7 +31,10 @@
|
||||
- [Nomic Embedding](#nomic-embedding)
|
||||
- [llama.cpp](#llamacpp)
|
||||
- [stable-diffusion.cpp](#stable-diffusioncpp)
|
||||
- [STT whisper.cpp](#stt-whispercpp)
|
||||
- [TTS coqui-tts](#tts-coqui-tts)
|
||||
- [open-webui](#open-webui)
|
||||
- [Upgrade Open Webui](#upgrade-open-webui)
|
||||
- [lite-llm](#lite-llm)
|
||||
- [Install Services with Quadlets](#install-services-with-quadlets)
|
||||
- [API Keys](#api-keys)
|
||||
@@ -526,6 +529,28 @@ localhost/stable-diffusion-cpp:latest \
|
||||
-p "A normal business meeting. People discuss business for 2 seconds. Suddenly, a horde of furries carrying assault rifles bursts into the room and causes a panic. Hatsune Miku leads the charge screaming in rage."
|
||||
```
|
||||
|
||||
## STT whisper.cpp
|
||||
|
||||
Whisper stt service.
|
||||
|
||||
```bash
|
||||
podman run -it --rm -p "8081:8080" \
|
||||
-v /home/ai/models/stt:/models:z \
|
||||
--device /dev/kfd \
|
||||
--device /dev/dri \
|
||||
ghcr.io/ggml-org/whisper.cpp:main-vulkan \
|
||||
"whisper-server \
|
||||
--host 0.0.0.0 \
|
||||
-m /models/ggml-large-v3-turbo/ggml-large-v3-turbo-q5_0.bin \
|
||||
--convert"
|
||||
```
|
||||
|
||||
## TTS coqui-tts
|
||||
|
||||
```bash
|
||||
podman run --rm -it -p 5002:5002 --entrypoint /bin/bash ghcr.io/coqui-ai/tts-cpu
|
||||
```
|
||||
|
||||
## open-webui
|
||||
|
||||
```bash
|
||||
@@ -552,6 +577,13 @@ Use the following connections:
|
||||
| stable-diffusion.cpp | <http://host.containers.internal:1234/v1> |
|
||||
| stable-diffusion.cpp edit | <http://host.containers.internal:1235/v1> |
|
||||
|
||||
### Upgrade Open Webui
|
||||
|
||||
```bash
|
||||
podman image pull ghcr.io/open-webui/open-webui:main
|
||||
systemctl --user restart open-webui.service
|
||||
```
|
||||
|
||||
## lite-llm
|
||||
|
||||
<https://docs.litellm.ai/docs/proxy/configs>
|
||||
|
||||
Reference in New Issue
Block a user