ducoterra
b78afc08f2
Added 6 rows for chat model results (lite + turbo, fixture + derived) to benchmarks/model_benchmarks.csv.
1.2 KiB
1.2 KiB
| 1 | date | script | model | mode | gate_status | turns | answered | caps | tool_turns | emitted | executed | contract | wall_s | contract_denom | executed_denom | tool_turns_denom | extra1_col | extra1_val | extra2_col | extra2_val |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2 | 2026-09-06 | summary | lite | quality | FAIL | 8 | 8 | 0 | 0 | 0 | 0 | 58 | 26.9 | 58 | 0 | 8 | coherence | 4.0/5 | hallucinations | 4/8 |
| 3 | 2026-09-06 | summary | lite | quality | FAIL | 8 | 8 | 0 | 0 | 0 | 0 | 53 | 25.4 | 53 | 0 | 8 | coherence | 4.0/5 | hallucinations | 5/8 |
| 4 | 2026-09-06 | embed | embed | dimension | PASS | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0.1 | 1 | 0 | 1 | actual_dim | 768 | expected_dim | 768 |
| 5 | 2026-09-06 | embed | embed | cosine | PASS | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0.2 | 1 | 0 | 1 | margin | 0.283 | quality | 56 |
| 6 | 2026-09-06 | embed | embed | speed | PASS | 1 | 1 | 0 | 0 | 0 | 0 | 1 | 0.5 | 1 | 0 | 1 | rate | 93.8 vec/s | bad_vectors | 0 |
| 7 | 2026-09-06 | summary | turbo | quality | FAIL | 8 | 8 | 0 | 0 | 0 | 0 | 63 | 25.3 | 63 | 0 | 8 | coherence | 4.0/5 | hallucinations | 3/8 |
| 8 | 2026-09-06 | chat | lite | fixture | PASS | 10 | 10 | 0 | 9 | 9 | 9 | 11 | 40.4 | 12 | 12 | 10 | ||||
| 9 | 2026-09-06 | chat | lite | fixture | PASS | 10 | 10 | 0 | 10 | 9 | 9 | 13 | 40.5 | 14 | 14 | 10 | ||||
| 10 | 2026-09-06 | chat | lite | derived | FAIL | 10 | 10 | 0 | 10 | 5 | 5 | 10 | 38.3 | 14 | 14 | 10 | ||||
| 11 | 2026-09-06 | chat | turbo | fixture | PASS | 10 | 10 | 0 | 7 | 9 | 9 | 9 | 113.7 | 9 | 9 | 10 | ||||
| 12 | 2026-09-06 | chat | turbo | fixture | PASS | 10 | 10 | 0 | 7 | 7 | 7 | 7 | 112.8 | 7 | 7 | 10 | ||||
| 13 | 2026-09-06 | chat | turbo | derived | FAIL | 10 | 10 | 0 | 5 | 5 | 5 | 5 | 97.2 | 5 | 5 | 10 |