phase: 110_fix_sse_db_pool_exhaustion
Build and Push Containers / build-and-push-app (push) Successful in 2m14s
Build and Push Containers / build-and-push-db (push) Successful in 13s

---

**Phase 110 — Fix SSE DB Connection Pool Exhaustion (SEC-14-04): COMPLETE**

**What was implemented/verified:**
- All three tasks (pool config, short-lived sessions, concurrency cap) were already implemented in code
- Fixed `tests/integration/test_chat_db_sessions.py` — added FakeChatLLM mock, fixed LLM signature (`tools=` not `_tools=`), used `fastapi_app.dependency_overrides` instead of `client.app.dependency_overrides`
- Fixed `tests/e2e/test_chat_db_pool.py` — added FakeChatLLM mock, fixed admin password to match `tests/conftest.py`, removed unused imports
- Fixed lint errors (unused imports, import order) in both test files

**Test / lint / coverage results:**
- `uv run pytest` → 2350 passed, 1 warning, 56.4s
- `uv run pytest --cov=app --cov-report=term-missing` → 99% coverage (4065 lines, 16 uncovered)
- `uv run pytest tests/e2e/test_chat_db_pool.py -v --no-cov` → 3 passed
- `uv run pytest tests/integration/test_chat_db_sessions.py -v --no-cov` → 4 passed
- `uv run pytest tests/integration/test_chat_concurrency.py -v --no-cov` → 11 passed
- `uv run pytest tests/unit/test_db_pool_config.py -v --no-cov` → 14 passed
- `uv run pytest tests/unit/test_agent_short_lived_sessions.py -v --no-cov` → 7 passed
- `uv run ruff check .` → all checks passed
- `uv run pyright` → 0 errors, 0 warnings

**Completion criteria:**
- [✓] `app/db.py::create_engine` receives explicit `pool_size=5`, `max_overflow=10`, `pool_recycle=3600` from settings
- [✓] `run_agent` accepts `db_factory: Callable[[], Session]` and creates short-lived sessions per tool call
- [✓] Each tool round uses a separate DB session closed after the tool result
- [✓] Concurrency cap (`BOR_CHAT_MAX_CONCURRENT`, default 10) limits concurrent turns; excess get 503
- [✓] All test gates green, coverage 99%, lint/types clean

**Notable decisions:** Tests needed LLM mocking (the original test files lacked `FakeChatLLM` mocks, causing hangs on real LLM calls).

**Next pending phase:** None — this is the last phase in `todo/`.
This commit is contained in:
2026-09-14 15:55:13 -04:00
parent 35d65d2f25
commit 3a4035fc96
43 changed files with 2886 additions and 444 deletions
+26 -10
View File
@@ -34,7 +34,7 @@ from __future__ import annotations
import asyncio
import uuid
from collections.abc import AsyncIterator, Iterator
from collections.abc import AsyncIterator, Callable, Iterator
from copy import deepcopy
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any, cast
@@ -44,6 +44,7 @@ from sqlalchemy import delete, text
from sqlalchemy.orm import Session
from app.config import Settings
from app.db import SessionLocal
from app.models import Document, FolderSummary, GitSource
from app.rag import agent
from app.rag.agent import AGENT_TOOLS, AgentHolder, run_agent
@@ -262,20 +263,32 @@ class ScriptedToolCallsLLM:
def _run_call(
db: Session, name: str, arguments: dict[str, Any]
) -> tuple[AgentHolder, ScriptedToolLLM]:
"""Drive one scripted tool call through ``run_agent``."""
"""Drive one scripted tool call through ``run_agent``.
SEC-14-04: the session factory creates a short-lived session per tool
call — the fixture session (*db*) is used to seed the KB, but each
tool round opens its own session via ``SessionLocal()``, executes the
tool, and closes it (the same pattern as production).
"""
holder = AgentHolder()
llm = ScriptedToolLLM(ToolCallPiece(id="call_1", name=name, arguments=arguments))
asyncio.run(_consume(cast("LLMClient", llm), db, holder))
# Create a factory that opens a fresh short-lived session per call
def _db_factory() -> Session:
return SessionLocal()
asyncio.run(_consume(cast("LLMClient", llm), _db_factory, holder))
return holder, llm
async def _consume(
llm: LLMClient, db: Session, holder: AgentHolder
llm: LLMClient,
db_factory: Callable[[], Session],
holder: AgentHolder,
) -> list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece]:
out: list[StreamPiece | ToolCallPiece | RetryPiece | ToolResultPiece] = []
async for piece in run_agent(
llm,
db,
db_factory, # SEC-14-04: session factory (short-lived sessions)
system_prompt="SYSTEM_PROMPT",
user_message="QUESTION",
seed_docs=[],
@@ -514,7 +527,8 @@ def test_read_combined_path_through_run_agent(kb, db) -> None:
"FULL-TEXT"
)
assert holder.tool_calls == 1
assert holder.read_docs == [created]
# SEC-14-04: short-lived session loads fresh copies
assert [d.id for d in holder.read_docs] == [created.id]
def test_read_bare_source_name_refused_through_run_agent(kb, db) -> None:
@@ -571,7 +585,7 @@ def test_read_bare_path_single_source_suggestion_then_corrected_read(kb, db) ->
]
)
holder = AgentHolder()
asyncio.run(_consume(cast("LLMClient", llm), db, holder))
asyncio.run(_consume(cast("LLMClient", llm), lambda: SessionLocal(), holder))
# Round 1: the bare path resolves to no combined identity, but it IS
# the indexed document's path — the refusal names the one combined
@@ -590,7 +604,8 @@ def test_read_bare_path_single_source_suggestion_then_corrected_read(kb, db) ->
"FULL-TEXT"
)
assert llm.requests[2][1] == AGENT_TOOLS
assert holder.read_docs == [created]
# SEC-14-04: short-lived session loads fresh copies
assert [d.id for d in holder.read_docs] == [created.id]
assert holder.tool_calls == 1 # only the corrected read executed
@@ -614,7 +629,7 @@ def test_read_bare_path_two_sources_one_of_suggestion_then_corrected_read(
]
)
holder = AgentHolder()
asyncio.run(_consume(cast("LLMClient", llm), db, holder))
asyncio.run(_consume(cast("LLMClient", llm), lambda: SessionLocal(), holder))
assert llm.requests[1][0][3]["content"] == (
"No document at 'shared/x.md' — did you mean one of: "
@@ -623,7 +638,8 @@ def test_read_bare_path_two_sources_one_of_suggestion_then_corrected_read(
assert llm.requests[2][0][5]["content"] == (
"Document Alpha/shared/x.md:\ndate: 2024-06-15\nA-TEXT"
)
assert holder.read_docs == [a]
# SEC-14-04: short-lived session loads fresh copies
assert [d.id for d in holder.read_docs] == [a.id]
assert holder.tool_calls == 1 # only the corrected read executed