Run archive
Every run in results/, newest first. Accuracy is the mean over questions of each question's mean over repeats; correct = 1, partial = ½. Runs still going show what's graded so far.
| Run | Domain | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|---|
| 2026-11-06-0935-sonnet-query | Credit ratings: the base data | claude:sonnet | query | 48 | D1 sql 84% 73–93D2 sql+markdown 100% 99–100D3 sql+executed 99% 98–100D4 sparql 82% 71–91D5 sparql+ontology 100% 100–100D6 sparql+axioms 90% 81–98 |
done |
| 2026-11-05-1233-haiku | Credit ratings: the base data | claude:haiku | read | 48 | R1 csv 78% 67–88R2 rdf 72% 60–83R3 csv+padding 76% 64–87R4 csv+markdown 98% 94–100R5 csv+ontology 99% 96–100R6 rdf+ontology 97% 92–100R7 csv+neutral 100% 99–100R8 csv+axioms 99% 98–100R9 csv+axioms+markdown 99% 97–100R10 executed 97% 92–100 |
done |
| 2026-11-05-1227-sonnet | Credit ratings: the base data | claude:sonnet | read | 48 | R1 csv 84% 73–93R2 rdf 85% 74–94R3 csv+padding 84% 73–94R4 csv+markdown 100% 100–100R5 csv+ontology 99% 98–100R6 rdf+ontology 100% 100–100R7 csv+neutral 100% 99–100R8 csv+axioms 99% 98–100R9 csv+axioms+markdown 100% 99–100R10 executed 99% 98–100 |
done |
| 2026-11-04-2028-haiku | Credit ratings: the base data | claude:haiku | read | 48 | R1 csv 78% 67–88R4 csv+markdown 99% 96–100R5 csv+ontology 99% 96–100R8 csv+axioms 100% 99–100 |
done |
| 2026-11-04-2013-sonnet | Credit ratings: the base data | claude:sonnet | read | 48 | R1 csv 85% 75–94R4 csv+markdown 100% 100–100R5 csv+ontology 99% 98–100R8 csv+axioms 99% 98–100 |
done |
| 2026-11-04-1825-sonnet | Credit ratings: the base data | claude:sonnet | read | 48 | R1 csv 89% 79–97R5 csv+ontology 100% 99–100R8 csv+axioms 97% 92–100R9 csv+axioms+markdown 100% 99–100 |
done |
| 2026-11-04-1824-haiku | Credit ratings: the base data | claude:haiku | read | 48 | R1 csv 78% 66–87R5 csv+ontology 100% 99–100R8 csv+axioms 98% 93–100R9 csv+axioms+markdown 98% 94–100 |
done |
| 2026-11-04-1723-haiku | Credit ratings: the base data | claude:haiku | read | 48 | R5 csv+ontology 98% 95–100R7 csv+neutral 100% 100–100 |
done |
| 2026-11-04-1712-sonnet | Credit ratings: the base data | claude:sonnet | read | 48 | R1 csv 89% 80–98R10 executed 98% 96–100 |
done |
| 2026-11-04-1711-sonnet | Credit ratings: the base data | claude:sonnet | read | 48 | R1 csv 89% 79–97R2 rdf 86% 77–94R3 csv+padding 89% 80–98R4 csv+markdown 100% 100–100R5 csv+ontology 100% 99–100R6 rdf+ontology 100% 100–100 |
done |
| 2026-11-04-1644-haiku | Credit ratings: the base data | claude:haiku | read | 48 | R1 csv 79% 68–89R2 rdf 75% 64–85R3 csv+padding 74% 62–85R4 csv+markdown 99% 98–100R5 csv+ontology 98% 94–100R6 rdf+ontology 97% 93–100 |
done |
| 2026-11-04-1643-sonnet | Credit ratings: the base data | claude:sonnet | read | 48 | R1 csv 88% 78–96R2 rdf 89% 80–97R3 csv+padding 86% 76–95R4 csv+markdown 100% 100–100R5 csv+ontology 99% 98–100R6 rdf+ontology 100% 100–100 |
done |
| 2026-11-04-1616-sonnet | Credit ratings: the base data | claude:sonnet | read | 40 | R1 csv 99% 98–100R2 rdf 98% 95–100R5 csv+ontology 99% 97–100R6 rdf+ontology 100% 100–100 |
done |
| 2026-11-04-1526-sonnet | Credit ratings: the base data | claude:sonnet | read | 10 | R1 csv 95% 85–100R2 rdf 93% 79–100R5 csv+ontology 96% 88–100R6 rdf+ontology 99% 97–100 |
done |
| 2026-11-06-0041-sonnet-query | Credit ratings: at 300 issuers | claude:sonnet | query | 48 | D1 sql 80% 69–90D2 sql+markdown 99% 98–100D3 sql+executed 99% 97–100D4 sparql 77% 66–87D5 sparql+ontology 97% 94–99D6 sparql+axioms 88% 80–95 |
done |
| 2026-11-05-2135-sonnet | Credit ratings: at 300 issuers | claude:sonnet | read | 48 | R1 csv 83% 72–92R2 rdf 79% 68–89R3 csv+padding 83% 73–92R4 csv+markdown 95% 89–99R5 csv+ontology 95% 89–99R6 rdf+ontology 96% 92–99R7 csv+neutral 97% 93–100R8 csv+axioms 94% 88–99R9 csv+axioms+markdown 95% 90–99R10 executed 95% 90–99 |
done |
| 2026-11-05-2135-haiku | Credit ratings: at 300 issuers | claude:haiku | read | 48 | R1 csv 74% 62–85R2 rdf 64% 51–76R3 csv+padding 69% 57–81R4 csv+markdown 88% 79–95R5 csv+ontology 89% 80–96R6 rdf+ontology 84% 74–92R7 csv+neutral 88% 80–95R8 csv+axioms 89% 82–95R9 csv+axioms+markdown 85% 75–93R10 executed 93% 87–98 |
done |
| 2026-11-04-1738-sonnet-query | Credit ratings: at 300 issuers | claude:sonnet | query | 48 | D1 sql 87% 77–95D2 sql+markdown 98% 96–100D4 sparql 80% 70–89D5 sparql+ontology 97% 94–100 |
done |
| 2026-11-04-1738-sonnet | Credit ratings: at 300 issuers | claude:sonnet | read | 48 | R1 csv 82% 71–91R4 csv+markdown 97% 92–100R10 executed 96% 92–100 |
done |
| 2026-11-05-1819-sonnet-query-graphdb | Credit ratings: ownership paths | claude:sonnet | query | 61 | D4 sparql 69% 59–79D5 sparql+ontology 83% 78–89D6 sparql+axioms 80% 72–87 |
done |
| 2026-11-05-1515-sonnet-query | Credit ratings: ownership paths | claude:sonnet | query | 61 | D1 sql 75% 65–85D2 sql+markdown 99% 98–100D3 sql+executed 97% 94–100D4 sparql 59% 50–69D5 sparql+ontology 77% 69–85D6 sparql+axioms 85% 79–91 |
done |
| 2026-11-04-2024-sonnet-query | Credit ratings: ownership paths | claude:sonnet | query | 61 | D1 sql 74% 63–84D2 sql+markdown 86% 82–90D3 sql+executed 95% 91–98D4 sparql 66% 55–76D5 sparql+ontology 96% 92–99D6 sparql+axioms 82% 73–90 |
done |
| 2026-11-04-2022-sonnet | Credit ratings: ownership paths | claude:sonnet | read | 61 | R1 csv 83% 75–91R4 csv+markdown 96% 92–99R8 csv+axioms 94% 89–98R10 executed 95% 90–99 |
done |
| 2026-11-06-0855-sonnet | Research analytics: the base data | claude:sonnet | read | 48 | R8 csv+axioms 97% 92–100R9 csv+axioms+markdown 98% 93–100 |
done |
| 2026-11-06-0855-haiku | Research analytics: the base data | claude:haiku | read | 48 | R8 csv+axioms 97% 92–100R9 csv+axioms+markdown 98% 94–100 |
done |
| 2026-11-06-0808-sonnet-query | Research analytics: the base data | claude:sonnet | query | 48 | D1 sql 52% 39–65D2 sql+markdown 96% 91–100D3 sql+executed 93% 87–98D4 sparql 40% 29–51D5 sparql+ontology 81% 77–85 |
done |
| 2026-11-05-1235-haiku | Research analytics: the base data | claude:haiku | read | 48 | R1 csv 45% 32–59R2 rdf 43% 30–57R3 csv+padding 44% 31–57R4 csv+markdown 99% 96–100R5 csv+ontology 96% 92–99R6 rdf+ontology 98% 95–100R10 executed 92% 84–98 |
done |
| 2026-11-05-1234-sonnet | Research analytics: the base data | claude:sonnet | read | 48 | R1 csv 57% 44–69R2 rdf 47% 34–61R3 csv+padding 57% 44–70R4 csv+markdown 98% 94–100R5 csv+ontology 98% 94–100R6 rdf+ontology 98% 94–100R10 executed 90% 84–96 |
done |
| 2026-11-04-2053-sonnet | Research analytics: the base data | claude:sonnet | read | 48 | R1 csv 58% 46–70R10 executed 93% 87–98 |
done |
| 2026-11-04-2035-haiku | Research analytics: the base data | claude:haiku | read | 48 | R1 csv 42% 29–55R10 executed 90% 82–97 |
done |
| 2026-11-04-2010-sonnet | Research analytics: the base data | claude:sonnet | read | 48 | R1 csv 56% 42–68R2 rdf 50% 36–64R3 csv+padding 55% 41–68R4 csv+markdown 97% 92–100R5 csv+ontology 97% 92–100R6 rdf+ontology 98% 95–100 |
done |
| 2026-11-04-1752-haiku | Research analytics: the base data | claude:haiku | read | 48 | R1 csv 44% 30–57R2 rdf 44% 30–58R3 csv+padding 45% 32–59R4 csv+markdown 96% 92–99R5 csv+ontology 97% 92–100R6 rdf+ontology 97% 93–100 |
done |
| 2026-11-06-0855-sonnet-query | Research analytics: at 850 works | claude:sonnet | query | 48 | D6 sparql+axioms 91% 85–96 |
done |
| 2026-11-06-0519-sonnet-query | Research analytics: at 850 works | claude:sonnet | query | 48 | D1 sql 38% 26–50D2 sql+markdown 80% 74–85D3 sql+executed 92% 84–98D4 sparql 30% 20–41D5 sparql+ontology 85% 79–90 |
done |
| 2026-11-06-0206-sonnet | Research analytics: at 850 works | claude:sonnet | read | 48 | R1 csv 42% 30–54R3 csv+padding 42% 30–54R4 csv+markdown 87% 78–94R5 csv+ontology 89% 82–95R10 executed 79% 69–89 |
done |
| 2026-11-06-0206-haiku | Research analytics: at 850 works | claude:haiku | read | 48 | R1 csv 21% 12–31R3 csv+padding 30% 19–41R4 csv+markdown 43% 34–51R5 csv+ontology 49% 40–58R10 executed 69% 58–79 |
done |