ducoterra
542e568d25
The embed alias was reverted to the previous backing model and re-benchmarked over 3 runs. All gates pass and the numbers reproduce the original baseline exactly: 768 dims, cosine margin 0.283 (quality 56) versus the 0.217 measured on the swapped-in model, with throughput back to 109.7-123.3 vec/s. The margin returning to the identical 0.283 confirms the locked semantic-pair corpus is a reliable fingerprint for this endpoint, and a raw-vector probe shows the two models are genuinely different spaces despite both being unit-normalized 768-dim (max component diff 0.035 on the same input). Any content embedded while the swapped-in model was live is therefore in an incompatible space and needs re-embedding. LLM-Generated: true
41 lines
3.5 KiB
CSV
41 lines
3.5 KiB
CSV
date,script,model,mode,gate_status,turns,answered,caps,tool_turns,emitted,executed,contract,wall_s,contract_denom,executed_denom,tool_turns_denom,extra1_col,extra1_val,extra2_col,extra2_val
|
|
2026-09-06,summary,lite,quality,FAIL,8,8,0,0,0,0,58,26.9,58,0,8,coherence,4.0/5,hallucinations,4/8
|
|
2026-09-06,summary,lite,quality,FAIL,8,8,0,0,0,0,53,25.4,53,0,8,coherence,4.0/5,hallucinations,5/8
|
|
2026-09-06,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.1,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-09-06,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.2,1,0,1,margin,0.283,quality,56
|
|
2026-09-06,embed,embed,speed,PASS,1,1,0,0,0,0,1,0.5,1,0,1,rate,93.8 vec/s,bad_vectors,0
|
|
2026-09-06,summary,turbo,quality,FAIL,8,8,0,0,0,0,63,25.3,63,0,8,coherence,4.0/5,hallucinations,3/8
|
|
2026-09-06,chat,lite,fixture,PASS,10,10,0,9,9,9,11,40.4,12,12,10,,,,
|
|
2026-09-06,chat,lite,fixture,PASS,10,10,0,10,9,9,13,40.5,14,14,10,,,,
|
|
2026-09-06,chat,lite,derived,FAIL,10,10,0,10,5,5,10,38.3,14,14,10,,,,
|
|
2026-09-06,chat,turbo,fixture,PASS,10,10,0,7,9,9,9,113.7,9,9,10,,,,
|
|
2026-09-06,chat,turbo,fixture,PASS,10,10,0,7,7,7,7,112.8,7,7,10,,,,
|
|
2026-09-06,chat,turbo,derived,FAIL,10,10,0,5,5,5,5,97.2,5,5,10,,,,
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.2,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.6,1,0,1,margin,0.217,quality,43
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,1.4,1,0,1,rate,35.6 vec/s,bad_vectors,0
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.0,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.6,1,0,1,margin,0.217,quality,43
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,1.4,1,0,1,rate,35.5 vec/s,bad_vectors,0
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.0,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.6,1,0,1,margin,0.217,quality,43
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,1.4,1,0,1,rate,35.6 vec/s,bad_vectors,0
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.1,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.3,1,0,1,margin,0.217,quality,43
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,0.8,1,0,1,rate,59.1 vec/s,bad_vectors,0
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.0,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.4,1,0,1,margin,0.217,quality,43
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,0.8,1,0,1,rate,61.4 vec/s,bad_vectors,0
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.0,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.3,1,0,1,margin,0.217,quality,43
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,0.9,1,0,1,rate,57.4 vec/s,bad_vectors,0
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.2,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.3,1,0,1,margin,0.283,quality,56
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,0.5,1,0,1,rate,109.7 vec/s,bad_vectors,0
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.0,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.1,1,0,1,margin,0.283,quality,56
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,0.4,1,0,1,rate,123.3 vec/s,bad_vectors,0
|
|
2026-10-09,embed,embed,dimension,PASS,1,1,0,0,0,0,1,0.0,1,0,1,actual_dim,768,expected_dim,768
|
|
2026-10-09,embed,embed,cosine,PASS,1,1,0,0,0,0,1,0.2,1,0,1,margin,0.283,quality,56
|
|
2026-10-09,embed,embed,speed,PASS,1,1,0,0,0,0,1,0.4,1,0,1,rate,119.0 vec/s,bad_vectors,0
|