393 lines
18 KiB
Markdown
393 lines
18 KiB
Markdown
# Vibe Discord Bot with RAG Chat History
|
||
|
||
A Discord bot that stores long-term chat history using SQLite with RAG (Retrieval-Augmented Generation) capabilities. It supports custom bots with personalities, text-to-speech via Kokoro, image generation, and image editing.
|
||
|
||
- [Vibe Discord Bot with RAG Chat History](#vibe-discord-bot-with-rag-chat-history)
|
||
- [Available Commands](#available-commands)
|
||
- [Custom Bot Management](#custom-bot-management)
|
||
- [Using Custom Bots](#using-custom-bots)
|
||
- [Text-to-Speech](#text-to-speech)
|
||
- [Image Commands](#image-commands)
|
||
- [Bot Conversations](#bot-conversations)
|
||
- [Admin & Debug](#admin--debug)
|
||
- [Design Notes](#design-notes)
|
||
- [Limits & Cooldowns](#limits--cooldowns)
|
||
- [Features](#features)
|
||
- [Setup](#setup)
|
||
- [Prerequisites](#prerequisites)
|
||
- [Environment Variables](#environment-variables)
|
||
- [Installation](#installation)
|
||
- [Running the Bot](#running-the-bot)
|
||
- [How It Works](#how-it-works)
|
||
- [Database Structure](#database-structure)
|
||
- [RAG Process](#rag-process)
|
||
- [File Structure](#file-structure)
|
||
- [Building](#building)
|
||
- [Local](#local)
|
||
- [Container](#container)
|
||
- [Testing](#testing)
|
||
- [Configuration](#configuration)
|
||
|
||
## Available Commands
|
||
|
||
### Custom Bot Management
|
||
|
||
| Command | Description | Example Usage |
|
||
| ---------------------------------- | -------------------------------------- | ---------------------------------------------------- |
|
||
| `!custom-bot <name> <personality>` | Create a custom bot with a personality | `!custom-bot alfred you are a proper british butler` |
|
||
| `!list-custom-bots` | List all available custom bots | `!list-custom-bots` |
|
||
| `!delete-custom-bot <name>` | Delete your custom bot (owner only) | `!delete-custom-bot alfred` |
|
||
|
||
### Using Custom Bots
|
||
|
||
Once you create a custom bot, interact with it by prefixing your message with the bot name:
|
||
|
||
```text
|
||
!<bot_name> <your message>
|
||
```
|
||
|
||
**Example:**
|
||
|
||
1. Create a bot: `!custom-bot alfred you are a proper british butler`
|
||
2. Use the bot: `!alfred Could you fetch me some tea?`
|
||
3. The bot will respond in character as a British butler
|
||
|
||
### Text-to-Speech
|
||
|
||
| Command | Description | Example Usage |
|
||
| ------------------------------------ | ----------------------------------------------------- | ----------------------------------------- |
|
||
| `!speak <text>` | Convert text to speech (MP3 attachment) | `!speak hello world` |
|
||
| `!speak <text> --voice <voice>` | Convert text to speech with a specific voice | `!speak hello world --voice af_bella` |
|
||
| `!speak <bot_name> <text>` | Have a custom bot respond and speak | `!speak alfred what time is it` |
|
||
| `!speak <bot_name> <text> --voice` | Have a custom bot respond and speak with a voice | `!speak alfred what time is it --voice am_puck` |
|
||
| `!voices` | List all available TTS voices by category | `!voices` |
|
||
|
||
### Image Commands
|
||
|
||
| Command | Description | Example Usage |
|
||
| ------------ | ------------------------------------ | ----------------------------------------- |
|
||
| `!doodlebob` | Generate an image from a text prompt | `!doodlebob a cat sitting on the moon` |
|
||
| `!retcon` | Edit an attached image with text | `!retcon <image attachment> Make it sunny` |
|
||
|
||
### Bot Conversations
|
||
|
||
| Command | Description | Example Usage |
|
||
| -------------------------------------- | ------------------------------------------- | ------------------------------------------------ |
|
||
| `!talkforme <bot1> <bot2> <n> <topic>` | Have two bots discuss a topic for n replies | `!talkforme alfred jarvis 4 the meaning of life` |
|
||
|
||
### Admin & Debug
|
||
|
||
| Command | Description | Example Usage |
|
||
| --------------------- | ------------------------------------------------------- | --------------------- |
|
||
| `!lobotomize` | Clear all conversation history and memory for all bots | `!lobotomize` |
|
||
| `!debug` | Show the debug menu | `!debug` |
|
||
| `!debug members` | List all members in the current channel | `!debug members` |
|
||
| `!debug whoami` | Show all information the bot has about you | `!debug whoami` |
|
||
| `!debug tools` | Show the LLM's available tools | `!debug tools` |
|
||
| `!history <bot_name>` | View the chat history of a custom bot | `!history alfred` |
|
||
|
||
## Design Notes
|
||
|
||
These semantics are intentional:
|
||
|
||
- **Shared memory**: message history is not siloed per bot. Any user's history can inform
|
||
any bot's RAG context — the semantic (embedding) search runs over the whole message
|
||
store, so a fact one user told bot A can resurface in a later conversation with bot B.
|
||
- **Shared bot namespace**: bot names are global, not per-user. Creating a bot with a
|
||
name that already exists **replaces** it (you will be told it was replaced); only
|
||
deletion is restricted to the bot's creator.
|
||
- **Any member may use any bot**: custom-bot chat (`!<bot_name> <message>`) is open to
|
||
every member of the server, regardless of who created the bot.
|
||
|
||
## Limits & Cooldowns
|
||
|
||
**Input limits** (over-limit input is rejected with a friendly message before any
|
||
LLM/DB work):
|
||
|
||
| Input | Limit |
|
||
| ------------------------- | ------------------------------------------------ |
|
||
| Custom bot name | 2–50 characters |
|
||
| Custom bot personality | 10–1000 characters |
|
||
| `!speak` text | 5000 characters (after `--voice` parsing) |
|
||
| `!doodlebob` / `!retcon` prompt | 2000 characters |
|
||
| `!talkforme` topic | 500 characters |
|
||
| `!talkforme` reply count | Capped at 20 replies per invocation |
|
||
| LLM responses | `MAX_COMPLETION_TOKENS` tokens (default 1000) |
|
||
| `!retcon` downloads | Discord CDN hosts only, 8 MB per image |
|
||
|
||
**Cooldowns** (per user; a friendly "try again in Ns" message is sent when hit):
|
||
|
||
| Command | Cooldown |
|
||
| ------------- | ----------- |
|
||
| `!doodlebob` | 1 / 60 s |
|
||
| `!talkforme` | 1 / 120 s |
|
||
| `!speak` | 3 / 30 s |
|
||
|
||
## Features
|
||
|
||
- **Long-term chat history storage**: Persistent storage of all bot interactions in SQLite
|
||
- **RAG-based context retrieval**: Smart retrieval of relevant conversation history using vector embeddings
|
||
- **Custom bots**: Create unlimited bots with unique personalities
|
||
- **Text-to-speech**: Kokoro TTS engine converts bot responses to MP3 audio
|
||
- **Image generation**: Generate images from text prompts via OpenAI-compatible API
|
||
- **Image editing**: Edit uploaded images with text instructions
|
||
- **Bot conversations**: Two custom bots can discuss a topic autonomously
|
||
- **Chat history**: View the full conversation history of any custom bot with `!history`
|
||
- **Automatic message cleanup**: Configurable limits on stored messages
|
||
|
||
## Setup
|
||
|
||
### Prerequisites
|
||
|
||
- Python 3.13 or higher
|
||
- [uv](https://docs.astral.sh/uv/) package manager
|
||
- Discord bot token
|
||
- OpenAI-compatible API endpoints (for chat, embeddings, and image generation)
|
||
|
||
### Environment Variables
|
||
|
||
Create a `.env` file with the following variables:
|
||
|
||
```bash
|
||
# Discord Bot Token (required)
|
||
DISCORD_TOKEN=your_discord_bot_token
|
||
|
||
# Chat/Completion API (required)
|
||
CHAT_ENDPOINT=https://your-api.com/v1
|
||
CHAT_ENDPOINT_KEY=your_api_key
|
||
CHAT_MODEL=your_model_name
|
||
|
||
# Image Generation (required)
|
||
IMAGE_GEN_ENDPOINT=https://your-api.com/v1
|
||
IMAGE_EDIT_ENDPOINT=https://your-api.com/v1
|
||
IMAGE_GEN_ENDPOINT_KEY=your_api_key
|
||
IMAGE_EDIT_ENDPOINT_KEY=your_api_key
|
||
IMAGE_GEN_MODEL=gen
|
||
IMAGE_EDIT_MODEL=edit
|
||
IMAGE_GEN_SIZE_SQUARE=1024x1024
|
||
IMAGE_GEN_SIZE_PORTRAIT=1024x1536
|
||
IMAGE_GEN_SIZE_LANDSCAPE=1536x1024
|
||
|
||
# Embedding API (required)
|
||
EMBEDDING_ENDPOINT=https://your-api.com/v1
|
||
EMBEDDING_ENDPOINT_KEY=your_api_key
|
||
EMBEDDING_MODEL=your_embed_model
|
||
|
||
# Optional: TTS Configuration
|
||
TTS_MODEL_PATH=kokoro-v1.0.onnx
|
||
TTS_VOICES_PATH=voices-v1.0.bin
|
||
TTS_VOICE=af_sarah
|
||
TTS_SPEED=1.0
|
||
|
||
# Optional: Database/Chat Settings
|
||
DB_PATH=chat_history.db
|
||
MAX_COMPLETION_TOKENS=1000
|
||
MAX_HISTORY_MESSAGES=1000
|
||
SIMILARITY_THRESHOLD=0.7
|
||
TOP_K_RESULTS=5
|
||
```
|
||
|
||
### Installation
|
||
|
||
1. Clone the repository and sync dependencies:
|
||
|
||
```bash
|
||
uv sync
|
||
```
|
||
|
||
2. Ensure the TTS model files are present in the project root:
|
||
|
||
- `kokoro-v1.0.onnx`
|
||
- `voices-v1.0.bin`
|
||
|
||
### Running the Bot
|
||
|
||
```bash
|
||
uv run python -m vibe_bot.main
|
||
```
|
||
|
||
## How It Works
|
||
|
||
### Database Structure
|
||
|
||
The system uses SQLite with four tables:
|
||
|
||
1. **chat_messages**: Stores message metadata
|
||
- `message_id`, `user_id`, `username`, `content`, `bot_name`, `role`, `timestamp`, `channel_id`, `guild_id`
|
||
2. **message_embeddings**: Stores vector embeddings for RAG
|
||
- `message_id` (PK), `embedding` (binary blob of float32 values)
|
||
3. **custom_bots**: Stores custom bot configurations
|
||
- `bot_name` (PK), `system_prompt`, `created_by`, `created_at`, `is_active`
|
||
4. **image_generation_times**: Stores per-generation durations for the ETA estimate
|
||
- `id` (PK), `duration_seconds`, `generated_at`
|
||
|
||
### RAG Process
|
||
|
||
1. When a message is sent to a custom bot, it's stored in `chat_messages`
|
||
2. An embedding is generated via the configured embedding API and stored in `message_embeddings`
|
||
3. When a new message is sent:
|
||
- The system retrieves recent messages from the same user
|
||
- It searches for semantically similar messages using cosine similarity on embeddings
|
||
- Relevant context (user + bot message pairs) is prepended to the prompt
|
||
- The LLM generates a response with awareness of past conversations
|
||
|
||
## File Structure
|
||
|
||
```text
|
||
vibe_discord_bots/
|
||
├── vibe_bot/
|
||
│ ├── __init__.py # Package marker
|
||
│ ├── app.py # Composition root: App, services, bot event handlers
|
||
│ ├── commands/
|
||
│ │ ├── __init__.py # register_all: wires every command group
|
||
│ │ ├── _state.py # Shared App holder for command handlers
|
||
│ │ ├── admin.py # !lobotomize, !debug, !history
|
||
│ │ ├── chat.py # Custom-bot chat group (dispatched via on_message)
|
||
│ │ ├── conversation.py # !talkforme
|
||
│ │ ├── custom_bots.py # !custom-bot, !list-custom-bots, !delete-custom-bot
|
||
│ │ ├── images.py # !doodlebob, !retcon
|
||
│ │ └── speech.py # !speak, !voices
|
||
│ ├── config.py # Environment variable loading and validation
|
||
│ ├── database.py # Facade over the db/ package
|
||
│ ├── db/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── bots.py # CustomBotManager (create/read/list/delete)
|
||
│ │ ├── connection.py # Shared SQLite connect() helper
|
||
│ │ ├── messages.py # ChatDatabase: persistence, cleanup, RAG context
|
||
│ │ ├── schema.py # Table creation and migrations
|
||
│ │ ├── search.py # Similarity search and history lookups
|
||
│ │ ├── timing.py # Image-generation timing statistics
|
||
│ │ └── vectors.py # Embedding vector helpers
|
||
│ ├── llm/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── chat.py # Chat completion clients (instruct/tools/history)
|
||
│ │ ├── images.py # Image gen/edit clients
|
||
│ │ └── registry.py # Tool registry (schema + dispatch)
|
||
│ ├── llm_client.py # Public LLM/image/embedding client facade
|
||
│ ├── main.py # Entrypoint
|
||
│ ├── prompts.py # LLM prompt constants
|
||
│ ├── services/
|
||
│ │ ├── __init__.py
|
||
│ │ ├── chat_service.py # Custom-bot chat turn (context -> LLM -> persist)
|
||
│ │ ├── conversation_service.py # Bot-vs-bot conversation runner
|
||
│ │ ├── image_service.py # !doodlebob / !retcon logic
|
||
│ │ └── speech_service.py # !speak logic
|
||
│ ├── textutil.py # Message splitting and user-info helpers
|
||
│ ├── tools.py # LLM tool stubs and channel-member helper
|
||
│ ├── tts.py # Kokoro TTS engine
|
||
│ └── tests/
|
||
│ ├── __init__.py
|
||
│ ├── _helpers.py # Shared test helpers
|
||
│ ├── conftest.py # Shared test fixtures
|
||
│ ├── test_app.py # Composition root and bot handler tests
|
||
│ ├── test_commands.py # Discord command tests
|
||
│ ├── test_config.py # Config loading tests
|
||
│ ├── test_database.py # Database + CustomBotManager tests
|
||
│ ├── test_docs.py # README sync tests (commands + file tree)
|
||
│ ├── test_llm_client.py # LLM client tests
|
||
│ ├── test_logging.py # No-content logging tests
|
||
│ ├── test_prompts.py # Prompt constant tests
|
||
│ ├── test_services.py # Service-layer tests
|
||
│ ├── test_textutil.py # Text utility tests
|
||
│ ├── test_tools.py # Tool stub tests
|
||
│ └── test_tts.py # TTS engine tests
|
||
├── .gitea/
|
||
│ └── workflows/
|
||
│ └── build-push.yml # CI: build and push the container image
|
||
├── AGENTS.md # Agent instructions
|
||
├── Containerfile # Podman/Docker build file
|
||
├── MANUAL_TESTS.md # Manual Discord testing guide
|
||
├── README.md # This file
|
||
├── build.sh # Podman build wrapper
|
||
├── kokoro-v1.0.onnx # Kokoro TTS model
|
||
├── pyproject.toml # Project metadata and dependencies (uv)
|
||
├── scripts/
|
||
│ └── bench_rag.py # RAG p95 latency benchmark (standalone)
|
||
├── uv.lock # Locked dependency versions
|
||
├── vibe-bot.container # systemd unit for running the bot container
|
||
└── voices-v1.0.bin # Kokoro voice definitions
|
||
```
|
||
|
||
## Building
|
||
|
||
### Local
|
||
|
||
```bash
|
||
# Sync dependencies
|
||
uv sync
|
||
|
||
# Run the bot
|
||
uv run python -m vibe_bot.main
|
||
```
|
||
|
||
### Container
|
||
|
||
```bash
|
||
# Build the container image
|
||
podman build -t localhost/vibe-bot:latest -f Containerfile .
|
||
|
||
# Run with environment file
|
||
podman run --env-file .env localhost/vibe-bot:latest
|
||
```
|
||
|
||
## Testing
|
||
|
||
Run the full test suite (hermetic by default; network-dependent `live` tests are
|
||
deselected):
|
||
|
||
```bash
|
||
uv run pytest vibe_bot/tests/ -v
|
||
```
|
||
|
||
Run the network-dependent live tests explicitly:
|
||
|
||
```bash
|
||
uv run pytest -m live
|
||
```
|
||
|
||
Run linters:
|
||
|
||
```bash
|
||
# Ruff (linter + formatter)
|
||
uv run ruff check vibe_bot/
|
||
|
||
# Mypy (type checking)
|
||
uv run mypy vibe_bot/
|
||
|
||
# Pyright (type checking)
|
||
uv run pyright vibe_bot/
|
||
|
||
# Black (formatter check)
|
||
uv run black --check vibe_bot/
|
||
```
|
||
|
||
## Configuration
|
||
|
||
| Variable | Default | Description |
|
||
| -------------------------- | ------------------ | --------------------------------------------- |
|
||
| `DISCORD_TOKEN` | *(required)* | Discord bot authentication token |
|
||
| `CHAT_ENDPOINT` | *(required)* | OpenAI-compatible chat API URL |
|
||
| `CHAT_ENDPOINT_KEY` | `placeholder` | API key for the chat endpoint |
|
||
| `CHAT_MODEL` | *(required)* | Model name for chat completions |
|
||
| `IMAGE_GEN_ENDPOINT` | *(required)* | Image generation API URL |
|
||
| `IMAGE_GEN_ENDPOINT_KEY` | `placeholder` | API key for the image generation endpoint |
|
||
| `IMAGE_GEN_MODEL` | *(required)* | Model name for image generation |
|
||
| `IMAGE_GEN_SIZE_SQUARE` | `1024x1024` | Square canvas size for image generation |
|
||
| `IMAGE_GEN_SIZE_PORTRAIT` | `1024x1536` | Portrait (tall) canvas size for image generation |
|
||
| `IMAGE_GEN_SIZE_LANDSCAPE` | `1536x1024` | Landscape (wide) canvas size for image generation |
|
||
| `IMAGE_EDIT_ENDPOINT` | *(required)* | Image editing API URL |
|
||
| `IMAGE_EDIT_ENDPOINT_KEY` | `placeholder` | API key for the image editing endpoint |
|
||
| `IMAGE_EDIT_MODEL` | *(required)* | Model name for image editing |
|
||
| `EMBEDDING_ENDPOINT` | *(required)* | Embedding API URL |
|
||
| `EMBEDDING_ENDPOINT_KEY` | `placeholder` | API key for the embedding endpoint |
|
||
| `EMBEDDING_MODEL` | *(required)* | Model name for text embeddings |
|
||
| `MAX_COMPLETION_TOKENS` | `1000` | Max tokens in LLM responses |
|
||
| `DB_PATH` | `chat_history.db` | SQLite database file path |
|
||
| `MAX_HISTORY_MESSAGES` | `1000` | Max messages kept in the database |
|
||
| `SIMILARITY_THRESHOLD` | `0.7` | Min cosine similarity for RAG context |
|
||
| `TOP_K_RESULTS` | `5` | Number of similar messages retrieved |
|
||
| `TTS_MODEL_PATH` | `kokoro-v1.0.onnx` | Path to Kokoro ONNX model file |
|
||
| `TTS_VOICES_PATH` | `voices-v1.0.bin` | Path to Kokoro voices binary file |
|
||
| `TTS_VOICE` | `af_sarah` | Default voice for TTS |
|
||
| `TTS_SPEED` | `1.0` | Speech speed multiplier |
|