diff --git a/TOOL_CALLING_TESTING.md b/TOOL_CALLING_TESTING.md index c30927e..05c9645 100644 --- a/TOOL_CALLING_TESTING.md +++ b/TOOL_CALLING_TESTING.md @@ -203,9 +203,11 @@ there is to **not** re-read what is already in the prompt. gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/11 executed (73%) contract 11/11 (100%) (wall 43.4s) gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 8/13 executed (62%) contract 12/13 (92%) (wall 50.6s) gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 7/11 executed (64%) contract 11/11 (100%) (wall 46.8s) +gate: lite PASS turns=10 answered=10 caps=0 tool-turns=9 calls 9/12 executed (75%) contract 11/12 (92%) 2026-09-06 (wall 40.4s) +gate: lite PASS turns=10 answered=10 caps=0 tool-turns=10 calls 9/14 executed (64%) contract 13/14 (93%) 2026-09-06 (wall 40.5s) ``` -Contract accuracy ≥ 90 %: **met** (100 / 92 / 100 / 93). The executed +Contract accuracy ≥ 90 %: **met** (100 / 92 / 100 / 93 / 92 / 93). The executed ratio sits at 58–73 % for the reason documented in §5 — an app semantics choice, not a model defect, and the open design question in §7. @@ -340,6 +342,7 @@ executed/emitted ≥ 0.90. Against the fixture KB (2026-09-04): ``` gate: lite FAIL turns=10 answered=10 caps=0 tool-turns=10 calls 5/15 executed (33%) contract 12/15 (80%) (wall 47.7s) +gate: lite FAIL turns=10 answered=10 caps=0 tool-turns=10 calls 5/14 executed (36%) contract 10/14 (71%) 2026-09-06 (wall 38.3s) ``` Reading that result: the teaching works — **every bare-path trap