feat(agent): align the document tools with the harness-trained shape — ls, read(path), grep(pattern, path?)

This commit is contained in:
2026-09-03 11:17:47 -04:00
parent 16f1cfbcaf
commit 801639efcc
55 changed files with 4031 additions and 1466 deletions
+21 -17
View File
@@ -9,18 +9,20 @@ MOCK-ONLY suite: ``E2E_REAL_LLM=1`` is not supported — the real ``turbo``
does whatever it does with the tools, while this story's gate is the
deterministic marker flow in ``tests/e2e/mock_llm.py`` (user message
contains ``use your tools`` **and** the system prompt carries the
``<tools>`` section of the HIGH prompt):
``<tools>`` section of the HIGH prompt; phase 70: the flow emits the
harness-aligned names — ``ls`` / ``read`` with the combined
``source/path`` identity):
1. request 1 (``tools`` offered, no tool results yet) → streams ONLY
``tool_calls`` deltas calling ``list_documents`` (id ``call_0``, no
arguments, ``finish_reason: "tool_calls"``);
``tool_calls`` deltas calling ``ls`` (id ``call_0``, no arguments,
``finish_reason: "tool_calls"``);
2. request 2 (a ``tool``-role catalog result in the messages) → streams a
``tool_calls`` delta calling ``read_document`` on the FIRST catalog
line (id ``call_1``);
3. request 3 (no ``tools`` parameter, the read result in the messages) →
the content answer ``Read <source/path>. <first 80 chars of the read
document's content>`` — so the suite can assert the read document
reached the model and landed in the answer.
``tool_calls`` delta calling ``read`` on the JOINED combined
``source/path`` of the FIRST catalog line (id ``call_1``);
3. request 3 (a ``tool``-role read result in the messages) → the content
answer ``Read <source/path>. <first 80 chars of the read document's
content>`` — so the suite can assert the read document reached the
model and landed in the answer.
KB fixture — reproduces the TODO failure (``aws-route53.md`` references
``example-record-file.json`` "for the exact JSON shape of
@@ -43,10 +45,11 @@ reseelink.json" but does not include it):
Test → story mapping (Playwright Mapping Rule):
1. ``test_marker_question_lists_reads_and_quotes`` — the SSE carries
``tool`` frames (list, then read, ahead of any delta), the UI shows
the "calling tool" label while a tool runs, the bubble shows both
tool lines, the final answer quotes the read document, and the
source chips include the read document (viewer link).
``tool`` frames (``ls``, then ``read`` with the combined path, ahead
of any delta), the UI shows the transient calling-tool status while a
tool runs, the bubble shows both tool lines, the final answer quotes
the read document, and the source chips include the read document
(viewer link).
2. ``test_tool_lines_re_render_after_reload`` — the persisted record
(phase 14) re-renders the tool lines.
3. ``test_plain_grounded_question_has_no_tool_frames`` — no marker → no
@@ -385,12 +388,13 @@ def test_marker_question_lists_reads_and_quotes(
assert i_list is not None and i_read is not None, statuses
assert i_list < i_read, statuses
# Wire level: exactly two `tool` frames — list then read — and both
# ahead of the first `delta` frame.
# Wire level: exactly two `tool` frames — ``ls`` then ``read`` (the
# combined source/path as the model passed it) — and both ahead of
# the first `delta` frame.
frames = _frames(page)
assert _tool_frames(frames) == [
{"type": "tool", "name": "list_documents", "argument": None},
{"type": "tool", "name": "read_document", "argument": READ_SP},
{"type": "tool", "name": "ls", "argument": None},
{"type": "tool", "name": "read", "argument": READ_SP},
]
first_delta = next(i for i, f in enumerate(frames) if f.get("type") == "delta")
assert all(