Scores
Each table puts a dataset's latest finished runs side by side: a column per run, a row per condition, with accuracy and its 95% interval, as the method defines them. Only runs on the dataset's current questions are shown, and a newer run replaces an older one that asked fewer of the same conditions. Click a column's heading for the run. Under each table are notes on what the runs show, every run on the dataset, and the latest reading run question by question.
What the numbers mean for each goal is in the conclusions. Every run, on every dataset, is in the run archive.
Credit ratings
The base data
| Condition | claude:haiku, reading 48 questions, 5 repeats | claude:sonnet, reading 48 questions, 5 repeats | claude:sonnet, querying 48 questions, 5 repeats |
|---|---|---|---|
R1 csv | 78% 67–88 | 84% 73–93 | |
R2 rdf | 72% 60–83 | 85% 74–94 | |
R3 csv+padding | 76% 64–87 | 84% 73–94 | |
R4 csv+markdown | 98% 94–100 | 100% 100–100 | |
R5 csv+ontology | 99% 96–100 | 99% 98–100 | |
R6 rdf+ontology | 97% 92–100 | 100% 100–100 | |
R7 csv+neutral | 100% 99–100 | 100% 99–100 | |
R8 csv+axioms | 99% 98–100 | 99% 98–100 | |
R9 csv+axioms+markdown | 99% 97–100 | 100% 99–100 | |
R10 executed | 97% 92–100 | 99% 98–100 | |
D1 sql | 84% 73–93 | ||
D2 sql+markdown | 100% 99–100 | ||
D3 sql+executed | 99% 98–100 | ||
D4 sparql | 82% 71–91 | ||
D5 sparql+ontology | 100% 100–100 | ||
D6 sparql+axioms | 90% 81–98 |
The table shows the current rules (version 2), with every reading condition in one run per model (Sonnet, Haiku).
- Written-down meaning helps, and the gain sits in the local rules. The definitions add +15 to +16 points for Sonnet and +20 to +21 for Haiku. On the local rules (Q41 to Q48), R1
csvscores 57% and 40%; every condition with the meaning reaches 88–100%. - The form matters little. Markdown scores the same as Turtle. RDF alone adds nothing, and costs Haiku 6 points. The decoy glossary (R3
csv+padding: bond-market terms, as long as the definitions, none of them relevant) stays within 2 points of R1csv. - Logic works as well as prose. With only the axioms, both models score 99%. An axiom is followed to the letter, mistakes included: in version 1 the axioms made
TestRecorda subclass ofEntity, and both models counted the test record (Q37). The fix made Q37 right. - One sentence can slant an answer. The ontology calls the foreign-currency rating "the headline rating". With it, Haiku gets Q10 ("is Valdoria Telecom investment grade?") right in 2 of 5 repeats with the Turtle and 1 of 5 with the Markdown. Made neutral, it's right in 5 of 5, and so it is with the axioms, which carry no prose.
- Executed rules help as much as definitions on this data: +15 points for Sonnet and +19 for Haiku. What they don't give is the reason: the one "why" question (Q6) is answered only partly in most repeats.
The list of every run below has version 1 and version 2 together. Each run is graded against the keys it ran with.
Querying (Sonnet), five repeats, against reading with the same model:
| No definitions | With the definitions | Rules run | |
|---|---|---|---|
| Reading | R1 csv 84% |
R4 csv+markdown 100% |
R10 executed 99% |
| SQL | D1 sql 84% |
D2 sql+markdown 100% |
D3 sql+executed 99% |
| SPARQL | D4 sparql 82% |
D5 sparql+ontology 100% |
D6 sparql+axioms 90% |
- At this size, querying and reading end up level: 100% with the definitions either way, and 82–84% without them. The data is small enough that the model fetches whole tables in a few queries and then reads them.
- SPARQL with the ontology is as good as SQL with the definitions: 100% in all five sessions.
- D6
sparql+axiomsmisses the test record. Its five misses (Q7, Q9, Q37, Q40, Q41) all come from E16, Brindle Capital, aQAtest record that describes no real entity. The executed tables drop test records. The graph after the axioms' rules keeps it, unmarked, and the note on what the rules derived doesn't mention it, so the model counts it, and its 2026 downgrade, as real. The same misses explain the 300-issuer run's empty or wrong answers to Q7. A rule that marks test records, and a line in the note, would close the gap.
Every run on this dataset
Reading the data
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-05-1233-haiku | claude:haiku | read | 48 | R1 csv 78% 67–88R2 rdf 72% 60–83R3 csv+padding 76% 64–87R4 csv+markdown 98% 94–100R5 csv+ontology 99% 96–100R6 rdf+ontology 97% 92–100R7 csv+neutral 100% 99–100R8 csv+axioms 99% 98–100R9 csv+axioms+markdown 99% 97–100R10 executed 97% 92–100 |
done |
| 2026-11-05-1227-sonnet | claude:sonnet | read | 48 | R1 csv 84% 73–93R2 rdf 85% 74–94R3 csv+padding 84% 73–94R4 csv+markdown 100% 100–100R5 csv+ontology 99% 98–100R6 rdf+ontology 100% 100–100R7 csv+neutral 100% 99–100R8 csv+axioms 99% 98–100R9 csv+axioms+markdown 100% 99–100R10 executed 99% 98–100 |
done |
| 2026-11-04-2028-haiku | claude:haiku | read | 48 | R1 csv 78% 67–88R4 csv+markdown 99% 96–100R5 csv+ontology 99% 96–100R8 csv+axioms 100% 99–100 |
done |
| 2026-11-04-2013-sonnet | claude:sonnet | read | 48 | R1 csv 85% 75–94R4 csv+markdown 100% 100–100R5 csv+ontology 99% 98–100R8 csv+axioms 99% 98–100 |
done |
| 2026-11-04-1825-sonnet | claude:sonnet | read | 48 | R1 csv 89% 79–97R5 csv+ontology 100% 99–100R8 csv+axioms 97% 92–100R9 csv+axioms+markdown 100% 99–100 |
done |
| 2026-11-04-1824-haiku | claude:haiku | read | 48 | R1 csv 78% 66–87R5 csv+ontology 100% 99–100R8 csv+axioms 98% 93–100R9 csv+axioms+markdown 98% 94–100 |
done |
| 2026-11-04-1723-haiku | claude:haiku | read | 48 | R5 csv+ontology 98% 95–100R7 csv+neutral 100% 100–100 |
done |
| 2026-11-04-1712-sonnet | claude:sonnet | read | 48 | R1 csv 89% 80–98R10 executed 98% 96–100 |
done |
| 2026-11-04-1711-sonnet | claude:sonnet | read | 48 | R1 csv 89% 79–97R2 rdf 86% 77–94R3 csv+padding 89% 80–98R4 csv+markdown 100% 100–100R5 csv+ontology 100% 99–100R6 rdf+ontology 100% 100–100 |
done |
| 2026-11-04-1644-haiku | claude:haiku | read | 48 | R1 csv 79% 68–89R2 rdf 75% 64–85R3 csv+padding 74% 62–85R4 csv+markdown 99% 98–100R5 csv+ontology 98% 94–100R6 rdf+ontology 97% 93–100 |
done |
| 2026-11-04-1643-sonnet | claude:sonnet | read | 48 | R1 csv 88% 78–96R2 rdf 89% 80–97R3 csv+padding 86% 76–95R4 csv+markdown 100% 100–100R5 csv+ontology 99% 98–100R6 rdf+ontology 100% 100–100 |
done |
| 2026-11-04-1616-sonnet | claude:sonnet | read | 40 | R1 csv 99% 98–100R2 rdf 98% 95–100R5 csv+ontology 99% 97–100R6 rdf+ontology 100% 100–100 |
done |
| 2026-11-04-1526-sonnet | claude:sonnet | read | 10 | R1 csv 95% 85–100R2 rdf 93% 79–100R5 csv+ontology 96% 88–100R6 rdf+ontology 99% 97–100 |
done |
Writing queries
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-06-0935-sonnet-query | claude:sonnet | query | 48 | D1 sql 84% 73–93D2 sql+markdown 100% 99–100D3 sql+executed 99% 98–100D4 sparql 82% 71–91D5 sparql+ontology 100% 100–100D6 sparql+axioms 90% 81–98 |
done |
Question by question: the latest finished reading run, 2026-11-05-1233-haiku
One mark per repeat; hover for the grader's reason, click a question for every answer to it. The run's page.
| Question | R1 csv | R2 rdf | R3 csv+padding | R4 csv+markdown | R5 csv+ontology | R6 rdf+ontology | R7 csv+neutral | R8 csv+axioms | R9 csv+axioms+markdown | R10 executed |
|---|---|---|---|---|---|---|---|---|---|---|
| Q1 What was Kestrel Energy's rating on 1 January 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q2 Has Kestrel Energy been downgraded in 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q3 Is Kestrel Energy's 2031 note investment grade? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q4 What is Northwind Holdings' current outlook? | ✅✅✅✅❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q5 Whose credit risk does a holder of the Northwind Finance 2030 notes take, and how is it rated? | ✅✅✅✅✅ | 🟡🟡🟡🟡🟡 | ✅🟡✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q6 Why is Valdoria Telecom's foreign-currency rating lower than its local-currency rating? | 🟡🟡🟡🟡✅ | 🟡🟡🟡🟡🟡 | 🟡🟡🟡🟡❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅🟡✅🟡✅ | ✅✅✅✅✅ | 🟡🟡✅🟡🟡 |
| Q7 Which issuers were downgraded in 2026? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q8 Which entities were in default on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q9 As of 1 October 2026, how many distinct issuers hold an issuer rating? | ✅❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅❌❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q10 Is Valdoria Telecom investment grade? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅❌❌❌❌ | ❌❌❌✅✅ | ❌❌❌✅❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ❌✅✅❌✅ | ✅✅✅✅✅ |
| Q11 What was Marlow Utilities' rating on 1 June 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q12 What was Marlow Utilities' rating on 1 January 2024? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q13 How many times has Marlow Utilities been downgraded? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q14 What was Halden Foods' rating on 1 January 2024? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q15 What was Halden Bank's rating on 1 January 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q16 Has Marlow Utilities' rating changed since 1 January 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q17 Was Orrin Bank downgraded in 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q18 Which issuers were on Rating Watch Negative on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅❌✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q19 Which issuers had a Negative outlook on 1 October 2026? | ✅✅✅✅❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅❌✅✅ | ✅✅✅✅✅ |
| Q20 Is Pellam Retail in default? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q21 Which entities had an issuer rating at some point but no longer had one on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q22 Which entities have ever been in default? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q23 What is Kestrel Energy's short-term rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q24 Is Kestrel Energy's issuer rating investment grade? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q25 How many instruments held an investment-grade rating on 1 October 2026? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q26 What is Corvenia Power's local-currency rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q27 How is Corvenia Power's 2032 note rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q28 What is the Republic of Corvenia's foreign-currency issuer rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q29 What is the Republic of Valdoria's issuer rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q30 Which entities had a foreign-currency issuer rating exactly at their country's ceiling on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅❌✅ | ✅✅✅✅✅ | ✅✅❌✅✅ | ✅✅✅❌✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q31 Did any company have a foreign-currency issuer rating above its country's ceiling on 1 October 2026? | ✅✅❌✅✅ | ✅✅✅✅✅ | ❌✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q32 Who is Halden Leasing's ultimate parent? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q33 Whose credit risk does a holder of the Halden Leasing 2028 notes take, and how is it rated? | ✅✅✅✅✅ | 🟡🟡🟡🟡🟡 | ✅🟡✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q34 Is Halden Foods part of the same group as Halden Bank? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q35 How many entities are in the Halden group, the parent included? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q36 How many entities are in the Northwind group, the parent included? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q37 How many distinct legal entities does the data describe? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q38 Are Orrin Bank AG and Orrin Bancorp Inc the same entity? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q39 What is Northwind Holdings' current rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q40 How many distinct issuers domiciled in GB held an issuer rating on 1 October 2026? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q41 What is Brindle Capital's current rating? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q42 In which country is Halden Bank domiciled? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q43 How many distinct entities are domiciled in Luxembourg (LU)? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q44 Is Valdoria Telecom's 2027 note a local-currency obligation? | ✅✅✅✅✅ | ❌❌✅❌❌ | ✅✅❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q45 Is Valdoria Telecom's 2027 note rated above what Valdoria's Country Ceiling allows? | ❌✅✅❌✅ | ❌❌✅❌❌ | ✅❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ❌❌❌❌❌ |
| Q46 Does Halden Leasing have an issuer rating? | ❌❌✅✅✅ | ✅✅✅✅❌ | ✅❌✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q47 How is Corvenia Power's 2034 note rated? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q48 How many instruments held a rating on 1 October 2026? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
At 300 issuers
| Condition | claude:haiku, reading 48 questions, 5 repeats | claude:sonnet, reading 48 questions, 5 repeats | claude:sonnet, querying 48 questions, 5 repeats |
|---|---|---|---|
R1 csv | 74% 62–85 | 83% 72–92 | |
R2 rdf | 64% 51–76 | 79% 68–89 | |
R3 csv+padding | 69% 57–81 | 83% 73–92 | |
R4 csv+markdown | 88% 79–95 | 95% 89–99 | |
R5 csv+ontology | 89% 80–96 | 95% 89–99 | |
R6 rdf+ontology | 84% 74–92 | 96% 92–99 | |
R7 csv+neutral | 88% 80–95 | 97% 93–100 | |
R8 csv+axioms | 89% 82–95 | 94% 88–99 | |
R9 csv+axioms+markdown | 85% 75–93 | 95% 90–99 | |
R10 executed | 93% 87–98 | 95% 90–99 | |
D1 sql | 80% 69–90 | ||
D2 sql+markdown | 99% 98–100 | ||
D3 sql+executed | 99% 97–100 | ||
D4 sparql | 77% 66–87 | ||
D5 sparql+ontology | 97% 94–99 | ||
D6 sparql+axioms | 88% 80–95 |
Reading, every condition, on the version 2 rules (Sonnet, Haiku), five repeats:
| Condition | Sonnet | Haiku |
|---|---|---|
R1 csv |
83% | 74% |
R2 rdf |
79% (−4, −8 to −0) | 64% (−10, −17 to −3) |
R3 csv+padding |
83% (+0) | 69% (−4, −8 to −1) |
R4 csv+markdown |
95% (+12) | 88% (+14) |
R5 csv+ontology |
95% (+12) | 89% (+15) |
R6 rdf+ontology |
96% (+13) | 84% (+10) |
R7 csv+neutral |
97% (+14) | 88% (+15) |
R8 csv+axioms |
94% (+11) | 89% (+15) |
R9 csv+axioms+markdown |
95% (+12) | 85% (+11) |
R10 executed |
95% (+12) | 93% (+19) |
- Definitions matter as much at scale: +11 to +14 points for Sonnet and +10 to +19 for Haiku, in every form.
- Size costs Haiku more than Sonnet: with the definitions, Sonnet reaches 94–97% and Haiku 84–93%, against 96–100% for both on the base data. The misses are mostly long lists of 28 to 32 issuers (Q7, Q18, Q19), where an answer is wrong if it misses or adds a single one.
- Raw RDF costs more at this size: −4 for Sonnet and −10 for Haiku. The decoy glossary costs Haiku 4 points.
- Q18 counts a Watch on either currency. Its key first counted only Watches on the foreign-currency rating, and missed two issuers here (E1019 and E1059) on Watch Negative on their local-currency rating. The key was fixed and the runs regraded. The base data has no such case.
Querying, the six conditions, on the version 2 rules (Sonnet), five repeats, against reading with the same model:
| No definitions | With the definitions | Rules executed | |
|---|---|---|---|
| Reading | R1 csv 83% |
R4 csv+markdown 95% |
R10 executed 95% |
| SQL | D1 sql 80% |
D2 sql+markdown 99% |
D3 sql+executed 99% |
| SPARQL | D4 sparql 77% |
D5 sparql+ontology 97% |
D6 sparql+axioms 88% |
- Definitions matter as much when querying: +19 points with SQL and +20 with SPARQL, against +12 reading.
- Querying beats reading a little, with the definitions: 97–99% against 95%. The gain is on the long lists, which SQL gets entirely right more often than reading does. Without the definitions, querying is no better than reading.
- A graph store doesn't help with these questions: SPARQL is 3 points below SQL without the definitions and 2 below with them. These questions need joins, not paths.
- The rules run over the graph help less than the definitions: D6
sparql+axiomsreaches 88%. It answers Q7, the 2026 downgrades, with an empty list in all five sessions. - Writing queries was rarely the problem: 17 of 578 queries failed, and 3 timed out.
- One D6
sparql+axiomssession was refused by the API ("safeguards flagged this message") and asked again on resume.
Every run on this dataset
Reading the data
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-05-2135-sonnet | claude:sonnet | read | 48 | R1 csv 83% 72–92R2 rdf 79% 68–89R3 csv+padding 83% 73–92R4 csv+markdown 95% 89–99R5 csv+ontology 95% 89–99R6 rdf+ontology 96% 92–99R7 csv+neutral 97% 93–100R8 csv+axioms 94% 88–99R9 csv+axioms+markdown 95% 90–99R10 executed 95% 90–99 |
done |
| 2026-11-05-2135-haiku | claude:haiku | read | 48 | R1 csv 74% 62–85R2 rdf 64% 51–76R3 csv+padding 69% 57–81R4 csv+markdown 88% 79–95R5 csv+ontology 89% 80–96R6 rdf+ontology 84% 74–92R7 csv+neutral 88% 80–95R8 csv+axioms 89% 82–95R9 csv+axioms+markdown 85% 75–93R10 executed 93% 87–98 |
done |
| 2026-11-04-1738-sonnet | claude:sonnet | read | 48 | R1 csv 82% 71–91R4 csv+markdown 97% 92–100R10 executed 96% 92–100 |
done |
Writing queries
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-06-0041-sonnet-query | claude:sonnet | query | 48 | D1 sql 80% 69–90D2 sql+markdown 99% 98–100D3 sql+executed 99% 97–100D4 sparql 77% 66–87D5 sparql+ontology 97% 94–99D6 sparql+axioms 88% 80–95 |
done |
| 2026-11-04-1738-sonnet-query | claude:sonnet | query | 48 | D1 sql 87% 77–95D2 sql+markdown 98% 96–100D4 sparql 80% 70–89D5 sparql+ontology 97% 94–100 |
done |
Question by question: the latest finished reading run, 2026-11-05-2135-sonnet
One mark per repeat; hover for the grader's reason, click a question for every answer to it. The run's page.
| Question | R1 csv | R2 rdf | R3 csv+padding | R4 csv+markdown | R5 csv+ontology | R6 rdf+ontology | R7 csv+neutral | R8 csv+axioms | R9 csv+axioms+markdown | R10 executed |
|---|---|---|---|---|---|---|---|---|---|---|
| Q1 What was Kestrel Energy's rating on 1 January 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q2 Has Kestrel Energy been downgraded in 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q3 Is Kestrel Energy's 2031 note investment grade? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q4 What is Northwind Holdings' current outlook? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q5 Whose credit risk does a holder of the Northwind Finance 2030 notes take, and how is it rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q6 Why is Valdoria Telecom's foreign-currency rating lower than its local-currency rating? | ✅🟡🟡✅✅ | 🟡🟡✅🟡✅ | 🟡🟡✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅🟡✅✅✅ | ✅✅✅✅✅ | 🟡✅🟡✅✅ | ✅✅✅✅✅ | 🟡🟡🟡✅🟡 |
| Q7 Which issuers were downgraded in 2026? | ❌❌❌❌✅ | ❌❌❌❌❌ | ❌❌❌✅❌ | ❌✅❌❌❌ | ✅❌❌❌✅ | ✅✅❌✅✅ | ✅❌✅✅❌ | ❌❌❌❌❌ | ✅✅❌✅❌ | ✅❌✅✅✅ |
| Q8 Which entities were in default on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q9 As of 1 October 2026, how many distinct issuers hold an issuer rating? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌✅❌❌ | ❌✅✅✅✅ | ✅❌✅❌✅ | ✅✅✅❌✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q10 Is Valdoria Telecom investment grade? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅❌✅ | ✅✅✅✅✅ | ✅❌✅❌✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅❌✅✅✅ | ✅✅✅✅✅ |
| Q11 What was Marlow Utilities' rating on 1 June 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q12 What was Marlow Utilities' rating on 1 January 2024? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q13 How many times has Marlow Utilities been downgraded? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q14 What was Halden Foods' rating on 1 January 2024? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q15 What was Halden Bank's rating on 1 January 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q16 Has Marlow Utilities' rating changed since 1 January 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q17 Was Orrin Bank downgraded in 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q18 Which issuers were on Rating Watch Negative on 1 October 2026? | ✅✅✅❌✅ | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅❌✅✅✅ | ✅✅✅✅✅ | ✅❌✅✅✅ | ❌✅✅❌✅ | ✅❌✅❌✅ |
| Q19 Which issuers had a Negative outlook on 1 October 2026? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌✅❌❌✅ | ❌✅❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ |
| Q20 Is Pellam Retail in default? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q21 Which entities had an issuer rating at some point but no longer had one on 1 October 2026? | ✅❌❌✅❌ | ❌✅❌✅✅ | ❌❌✅❌❌ | ✅✅✅✅✅ | ✅❌✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q22 Which entities have ever been in default? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q23 What is Kestrel Energy's short-term rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q24 Is Kestrel Energy's issuer rating investment grade? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q25 How many instruments held an investment-grade rating on 1 October 2026? | ✅✅✅✅✅ | ✅❌✅❌❌ | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q26 What is Corvenia Power's local-currency rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q27 How is Corvenia Power's 2032 note rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q28 What is the Republic of Corvenia's foreign-currency issuer rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q29 What is the Republic of Valdoria's issuer rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q30 Which entities had a foreign-currency issuer rating exactly at their country's ceiling on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q31 Did any company have a foreign-currency issuer rating above its country's ceiling on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q32 Who is Halden Leasing's ultimate parent? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q33 Whose credit risk does a holder of the Halden Leasing 2028 notes take, and how is it rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q34 Is Halden Foods part of the same group as Halden Bank? | ✅✅✅✅✅ | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q35 How many entities are in the Halden group, the parent included? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q36 How many entities are in the Northwind group, the parent included? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q37 How many distinct legal entities does the data describe? | ❌❌❌❌✅ | ❌❌❌❌❌ | ❌❌❌✅❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅❌✅✅✅ | ✅✅✅✅✅ |
| Q38 Are Orrin Bank AG and Orrin Bancorp Inc the same entity? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q39 What is Northwind Holdings' current rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q40 How many distinct issuers domiciled in GB held an issuer rating on 1 October 2026? | ❌❌❌❌✅ | ❌❌❌❌❌ | ❌❌✅✅❌ | ❌✅✅✅✅ | ✅✅✅❌❌ | ✅✅✅✅✅ | ❌✅✅✅✅ | ❌❌✅✅✅ | ✅✅✅❌✅ | ✅✅❌✅✅ |
| Q41 What is Brindle Capital's current rating? | ❌❌❌❌✅ | ❌❌❌❌❌ | ❌❌✅✅❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q42 In which country is Halden Bank domiciled? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q43 How many distinct entities are domiciled in Luxembourg (LU)? | ❌❌❌❌❌ | ❌✅❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q44 Is Valdoria Telecom's 2027 note a local-currency obligation? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q45 Is Valdoria Telecom's 2027 note rated above what Valdoria's Country Ceiling allows? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q46 Does Halden Leasing have an issuer rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q47 How is Corvenia Power's 2034 note rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q48 How many instruments held a rating on 1 October 2026? | ✅✅✅✅✅ | ✅❌✅❌❌ | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
Ownership paths
| Condition | claude:sonnet, reading 61 questions, 5 repeats | claude:sonnet, querying 61 questions, 5 repeats | claude:sonnet, querying, GraphDB 61 questions, 5 repeats |
|---|---|---|---|
R1 csv | 83% 75–91 | ||
R4 csv+markdown | 96% 92–99 | ||
R8 csv+axioms | 94% 89–98 | ||
R10 executed | 95% 90–99 | ||
D1 sql | 75% 65–85 | ||
D2 sql+markdown | 99% 98–100 | ||
D3 sql+executed | 97% 94–100 | ||
D4 sparql | 59% 50–69 | 69% 59–79 | |
D5 sparql+ontology | 77% 69–85 | 83% 78–89 | |
D6 sparql+axioms | 85% 79–91 | 80% 72–87 |
Sonnet, five repeats: reading (run), querying with SPARQL on rdflib (run), and the SPARQL conditions again on GraphDB (run).
| All 61 questions | The 13 path questions | Path score in each session | |
|---|---|---|---|
Reading: R1 csv |
83% | 77% | 69, 85, 62, 85, 85 |
Reading: R4 csv+markdown |
96% | 100% | 100 in all five |
Reading: R10 executed |
95% | 97% | 92, 100, 92, 100, 100 |
Reading: R8 csv+axioms |
94% | 97% | 92, 100, 100, 92, 100 |
SQL: D1 sql |
75% | 60% | 38, 62, 62, 69, 69 |
SQL: D2 sql+markdown |
99% | 100% | 100 in all five |
SQL: D3 sql+executed |
97% | 100% | 100 in all five |
SPARQL: D4 sparql |
59% | 32% | 46, 62, 0, 31, 23 |
SPARQL: D5 sparql+ontology |
77% | 42% | 4, 8, 46, 77, 77 |
SPARQL: D6 sparql+axioms |
85% | 77% | 100, 100, 100, 8, 77 |
SPARQL on GraphDB: D4 sparql |
69% | 40% | 23, 38, 38, 38, 62 |
SPARQL on GraphDB: D5 sparql+ontology |
83% | 60% | 85, 85, 0, 100, 31 |
SPARQL on GraphDB: D6 sparql+axioms |
80% | 68% | 92, 0, 85, 92, 69 |
- A graph store doesn't help with paths, with or without definitions. Without them, SPARQL gets 32% of the path questions and SQL 60%. With them, SQL gets every path right in all five sessions, and SPARQL swings from 4% to 77%. The chains run through a RESEARCH record, and a test record sits inside a group, so neither store can be walked correctly without knowing what those records mean.
- SPARQL is the less steady store here, and not because of the engine. On rdflib, the model's large queries with the ontology were stopped after 60 seconds, up to three a session. On GraphDB, the same triples with no inference, no query timed out, and accuracy against rdflib went up 10 points without definitions and 6 with the ontology, but down 5 over the graph after the rules. Its poor sessions still happen: the model stops after three of its eight rounds, saying results were malformed when none were, or spends its rounds fetching whole classes of records, 60 rows at a time, and runs out before the path questions. With SQL, the same model asks for what each question needs, and no session fails that way.
- Precomputed chains win. With an
ancestorstable, SQL gets every path question right in every session, and so do three of the five sessions over the graph after the SHACL rules. Once rules have computed the chains, a table does as well as a graph. - Reading does as well at this size. With the definitions, reading the 100 KB of CSV gets all 13 right in all five sessions, as SQL with the definitions does. Querying should pay off when the data is too big to read, which this isn't.
An earlier querying run with three repeats (run) had SPARQL with the ontology as the steadiest condition (92% on paths) and SQL with the definitions at 71%. With five repeats the order is reversed, which shows how much a single session moves these scores.
Every run on this dataset
Reading the data
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-04-2022-sonnet | claude:sonnet | read | 61 | R1 csv 83% 75–91R4 csv+markdown 96% 92–99R8 csv+axioms 94% 89–98R10 executed 95% 90–99 |
done |
Writing queries
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-05-1819-sonnet-query-graphdb | claude:sonnet | query | 61 | D4 sparql 69% 59–79D5 sparql+ontology 83% 78–89D6 sparql+axioms 80% 72–87 |
done |
| 2026-11-05-1515-sonnet-query | claude:sonnet | query | 61 | D1 sql 75% 65–85D2 sql+markdown 99% 98–100D3 sql+executed 97% 94–100D4 sparql 59% 50–69D5 sparql+ontology 77% 69–85D6 sparql+axioms 85% 79–91 |
done |
| 2026-11-04-2024-sonnet-query | claude:sonnet | query | 61 | D1 sql 74% 63–84D2 sql+markdown 86% 82–90D3 sql+executed 95% 91–98D4 sparql 66% 55–76D5 sparql+ontology 96% 92–99D6 sparql+axioms 82% 73–90 |
done |
Question by question: the latest finished reading run, 2026-11-04-2022-sonnet
One mark per repeat; hover for the grader's reason, click a question for every answer to it. The run's page.
| Question | R1 csv | R4 csv+markdown | R8 csv+axioms | R10 executed |
|---|---|---|---|---|
| Q1 What was Kestrel Energy's rating on 1 January 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q2 Has Kestrel Energy been downgraded in 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q3 Is Kestrel Energy's 2031 note investment grade? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q4 What is Northwind Holdings' current outlook? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q5 Whose credit risk does a holder of the Northwind Finance 2030 notes take, and how is it rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q6 Why is Valdoria Telecom's foreign-currency rating lower than its local-currency rating? | ✅✅🟡🟡🟡 | ✅✅🟡✅✅ | ✅✅✅🟡✅ | ✅🟡🟡🟡🟡 |
| Q7 Which issuers were downgraded in 2026? | ❌❌❌❌❌ | ✅✅❌✅✅ | ❌❌❌❌❌ | ❌❌❌❌❌ |
| Q8 Which entities were in default on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q9 As of 1 October 2026, how many distinct issuers hold an issuer rating? | ❌✅❌✅❌ | ✅✅❌✅✅ | ✅❌✅✅✅ | ✅✅✅✅✅ |
| Q10 Is Valdoria Telecom investment grade? | ✅✅✅✅✅ | ❌✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q11 What was Marlow Utilities' rating on 1 June 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q12 What was Marlow Utilities' rating on 1 January 2024? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q13 How many times has Marlow Utilities been downgraded? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q14 What was Halden Foods' rating on 1 January 2024? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q15 What was Halden Bank's rating on 1 January 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q16 Has Marlow Utilities' rating changed since 1 January 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q17 Was Orrin Bank downgraded in 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q18 Which issuers were on Rating Watch Negative on 1 October 2026? | ✅✅✅✅✅ | ❌✅❌✅✅ | ❌✅✅❌✅ | ❌✅✅✅✅ |
| Q19 Which issuers had a Negative outlook on 1 October 2026? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅❌❌❌ |
| Q20 Is Pellam Retail in default? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q21 Which entities had an issuer rating at some point but no longer had one on 1 October 2026? | ✅✅❌✅❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q22 Which entities have ever been in default? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q23 What is Kestrel Energy's short-term rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q24 Is Kestrel Energy's issuer rating investment grade? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q25 How many instruments held an investment-grade rating on 1 October 2026? | ✅✅✅❌✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q26 What is Corvenia Power's local-currency rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q27 How is Corvenia Power's 2032 note rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q28 What is the Republic of Corvenia's foreign-currency issuer rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q29 What is the Republic of Valdoria's issuer rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q30 Which entities had a foreign-currency issuer rating exactly at their country's ceiling on 1 October 2026? | ✅✅❌✅✅ | ✅✅✅❌✅ | ✅✅❌✅❌ | ✅✅✅✅❌ |
| Q31 Did any company have a foreign-currency issuer rating above its country's ceiling on 1 October 2026? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q32 Who is Halden Leasing's ultimate parent? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q33 Whose credit risk does a holder of the Halden Leasing 2028 notes take, and how is it rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q34 Is Halden Foods part of the same group as Halden Bank? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q35 How many entities are in the Halden group, the parent included? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q36 How many entities are in the Northwind group, the parent included? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q37 How many distinct legal entities does the data describe? | ❌✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q38 Are Orrin Bank AG and Orrin Bancorp Inc the same entity? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q39 What is Northwind Holdings' current rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q40 How many distinct issuers domiciled in GB held an issuer rating on 1 October 2026? | ❌✅❌✅❌ | ✅✅✅✅✅ | ❌✅✅✅✅ | ✅❌✅✅✅ |
| Q41 What is Brindle Capital's current rating? | ❌✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q42 In which country is Halden Bank domiciled? | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q43 How many distinct entities are domiciled in Luxembourg (LU)? | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q44 Is Valdoria Telecom's 2027 note a local-currency obligation? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q45 Is Valdoria Telecom's 2027 note rated above what Valdoria's Country Ceiling allows? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q46 Does Halden Leasing have an issuer rating? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q47 How is Corvenia Power's 2034 note rated? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q48 How many instruments held a rating on 1 October 2026? | ✅✅✅❌✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q49 Who is Cerid Rosor Holdings SA's ultimate parent? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q50 How many levels of ownership separate Cerid Rosor Holdings SA from its ultimate parent? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q51 How many entities are in the Cerid Group plc group, the parent included? | ❌✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q52 How many entities are in the Rotam Group Inc group, the parent included? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q53 Are Belir Mining SA and Rotam Group Inc in the same group? | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q54 Which members of the Wonod Group SA group are domiciled in Valdoria (VD)? | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q55 If Valdoria (VD) came under stress, how many entities with an issuer rating in force have an entity domiciled in VD somewhere above them in their chain of parents? | ❌❌❌❌❌ | ✅✅✅✅✅ | ❌✅✅❌✅ | ❌✅❌✅✅ |
| Q56 What is the lowest current foreign-currency issuer rating held by any member of the Vesol Group BV group? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q57 Which is the nearest entity that Cerid Guven Energy SA and Cerid Telecom SA both have in their chains of parents? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q58 Which entities guarantee a note issued by an entity outside their own group? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q59 How many entities in the Cerid Group plc group sit exactly three levels below the top? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q60 Whose credit risk does a holder of the Cross-group 4.75% notes due 2031 take, and is that entity in the same group as the issuer? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q61 How many members of the Cerid Group plc group, the top included, hold a foreign-currency issuer rating below investment grade? | ❌✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
Research analytics
The base data
| Condition | claude:haiku, reading 48 questions, 5 repeats | claude:haiku, reading 48 questions, 5 repeats | claude:sonnet, reading 48 questions, 5 repeats | claude:sonnet, reading 48 questions, 5 repeats | claude:sonnet, querying 48 questions, 5 repeats |
|---|---|---|---|---|---|
R1 csv | 45% 32–59 | 57% 44–69 | |||
R2 rdf | 43% 30–57 | 47% 34–61 | |||
R3 csv+padding | 44% 31–57 | 57% 44–70 | |||
R4 csv+markdown | 99% 96–100 | 98% 94–100 | |||
R5 csv+ontology | 96% 92–99 | 98% 94–100 | |||
R6 rdf+ontology | 98% 95–100 | 98% 94–100 | |||
R8 csv+axioms | 97% 92–100 | 97% 92–100 | |||
R9 csv+axioms+markdown | 98% 94–100 | 98% 93–100 | |||
R10 executed | 92% 84–98 | 90% 84–96 | |||
D1 sql | 52% 39–65 | ||||
D2 sql+markdown | 96% 91–100 | ||||
D3 sql+executed | 93% 87–98 | ||||
D4 sparql | 40% 29–51 | ||||
D5 sparql+ontology | 81% 77–85 |
The table shows every reading condition in one run per model (Sonnet, Haiku). The axioms came later, and were read in their own runs (Sonnet, Haiku): R8 csv+axioms scores 97% with both models and R9 csv+axioms+markdown 98%, as the prose definitions do. The misses are Q1, which most conditions read as the three name forms, and partial answers to the "why" questions (Q47, Q48): the logic says what counts, not why.
- The same pattern as credit, and larger. The definitions add +41 points for Sonnet and +51 to +53 for Haiku. The decoy glossary (R3
csv+padding) adds nothing, Markdown matches Turtle, and RDF alone costs Sonnet 9 points (−17 to −3). - The gain sits in the dataset's own rules. On the local rules (Q33 to Q48), R1
csvscores 16% for Sonnet and 18% for Haiku, and the definitions take them to 97–99%. On the general questions (Q1 to Q32), R1csvscores 77% and 59%. Most of that general gap also rests on rules that belong to the dataset: retracted works aren't research output (Q24 to Q26), and the hierarchy counts depend on the affiliation and affiliate rules (Q7, Q9, Q10). Without the ontology, the model has noticed the retraction flag and said dropping retracted works "would be a policy choice the data doesn't make". That's a fair reading, and exactly what an ontology is for. - Executed rules get most of the way, but not all: 90% for Sonnet and 92% for Haiku, 6 to 9 points short of the definitions. The misses come from the tables' design. Merging a preprint into its published version hides it (Q2, Q20). The counting questions lose the per-author shares (Q12, Q13). There's no ultimate-parent column (Q46). And executed rules don't say why (Q47, Q48).
Querying (Sonnet), five repeats, against reading with the same model. The domain had no axioms when this ran, so no D6 sparql+axioms.
| No definitions | With the definitions | Rules executed | |
|---|---|---|---|
| Reading | R1 csv 57% |
R4 csv+markdown 98% |
R10 executed 90% |
| SQL | D1 sql 52% |
D2 sql+markdown 96% |
D3 sql+executed 93% |
| SPARQL | D4 sparql 40% |
D5 sparql+ontology 81% |
- At this size, querying with the definitions does as well as reading with them: 96% with SQL, every session between 96% and 97%, against 98% reading. The local rules are right in 99% of answers.
- Executed tables don't beat the definitions here: 93% with SQL. At 850 works the order reverses: SQL with the definitions drops to 80% and SQL over executed tables holds at 92% (scores at 850 works). So the case for running the rules first grows with the data.
- SPARQL sessions break down in a way SQL sessions don't. D5
sparql+ontologyscores 97–98% in four sessions and 17% in the fifth, where the model gave up after one round, saying the store returned "repeated, contradictory rows" when no query had failed. One D4sparqlsession scored 0%: the model began writing tool-call markup instead of query blocks and ended without answers. - Without the definitions, querying is no better than reading: 52% with SQL and 40% with SPARQL, against 57%.
Every run on this dataset
Reading the data
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-06-0855-sonnet | claude:sonnet | read | 48 | R8 csv+axioms 97% 92–100R9 csv+axioms+markdown 98% 93–100 |
done |
| 2026-11-06-0855-haiku | claude:haiku | read | 48 | R8 csv+axioms 97% 92–100R9 csv+axioms+markdown 98% 94–100 |
done |
| 2026-11-05-1235-haiku | claude:haiku | read | 48 | R1 csv 45% 32–59R2 rdf 43% 30–57R3 csv+padding 44% 31–57R4 csv+markdown 99% 96–100R5 csv+ontology 96% 92–99R6 rdf+ontology 98% 95–100R10 executed 92% 84–98 |
done |
| 2026-11-05-1234-sonnet | claude:sonnet | read | 48 | R1 csv 57% 44–69R2 rdf 47% 34–61R3 csv+padding 57% 44–70R4 csv+markdown 98% 94–100R5 csv+ontology 98% 94–100R6 rdf+ontology 98% 94–100R10 executed 90% 84–96 |
done |
| 2026-11-04-2053-sonnet | claude:sonnet | read | 48 | R1 csv 58% 46–70R10 executed 93% 87–98 |
done |
| 2026-11-04-2035-haiku | claude:haiku | read | 48 | R1 csv 42% 29–55R10 executed 90% 82–97 |
done |
| 2026-11-04-2010-sonnet | claude:sonnet | read | 48 | R1 csv 56% 42–68R2 rdf 50% 36–64R3 csv+padding 55% 41–68R4 csv+markdown 97% 92–100R5 csv+ontology 97% 92–100R6 rdf+ontology 98% 95–100 |
done |
| 2026-11-04-1752-haiku | claude:haiku | read | 48 | R1 csv 44% 30–57R2 rdf 44% 30–58R3 csv+padding 45% 32–59R4 csv+markdown 96% 92–99R5 csv+ontology 97% 92–100R6 rdf+ontology 97% 93–100 |
done |
Writing queries
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-06-0808-sonnet-query | claude:sonnet | query | 48 | D1 sql 52% 39–65D2 sql+markdown 96% 91–100D3 sql+executed 93% 87–98D4 sparql 40% 29–51D5 sparql+ontology 81% 77–85 |
done |
Question by question: the latest finished reading run, 2026-11-06-0855-sonnet
One mark per repeat; hover for the grader's reason, click a question for every answer to it. The run's page.
| Question | R8 csv+axioms | R9 csv+axioms+markdown |
|---|---|---|
| Q1 How many distinct people do the author records named "Mara Ellison", "M. Ellison" and "Ellison, Mara J." stand for? | ❌❌❌❌❌ | ❌❌❌❌❌ |
| Q2 List every work, of any type, that Mara Ellison (A01) is an author of. | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q3 Is the Jonas Brandt who wrote W37 the same person as the Jonas Brandt who wrote W38? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q4 How many research works has the Jonas Brandt of Albury Metropolitan University (A05) authored? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q5 Is the M. Ellison who wrote W08 the same person as the Mara Ellison who wrote W01? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q6 Which author has the most research works credited to the University of Albury, its units included? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q7 How many research works has Mara Ellison authored? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q8 What is the ultimate parent of the Tessel Laboratory for Ocean Sensing (I04)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q9 How many research works is the University of Albury (I02) credited with, its units included? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q10 How many research works is the Albury University System (I01) credited with, all its units included? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q11 How many research works is Kesting University (I07) credited with, its units included? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q12 How many research works did Kesting University (I07, units included) publish in 2024, under full counting and under fractional counting (each work split equally among its authors)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q13 In 2024, which had more research works, Kesting University (I07) or Rostval Technical University (I09): under full counting, and under fractional counting? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q14 How many research works did GB institutions publish in 2024, under full counting and under fractional counting (each work split equally among its authors, each author's share going to their institution's country)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q15 How many research works appeared in Ocean Sensing Letters (S02) in 2024? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q16 How many research works appeared in the Journal of Marine Autonomy (S01)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q17 Which of Mara Ellison's works are not research output because of their document type? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q18 How many distinct research works has Priya Ramanathan (A07) authored? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q19 How many distinct works cite "Drift-aware mapping with low-cost sonar", in either of its versions? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q20 Which works in the data are preprints of another work that is also in the data? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q21 How many research works is Oskel Polytechnic (I14) credited with? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q22 Which has more citations, W30 or W31 (counting both versions of each): in total, and within its publication year and the two years after? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q23 How many citations has W30 received in total, and how many within its publication year and the two years after? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q24 How many research works has Lena Vossberg (A09) authored? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q25 What is the total of the citation counts of Lena Vossberg's research works? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q26 Does W23 count towards Kesting University's research output? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q27 Which other institution's works cite Kesting University's research works the most, each citing work counted once per institution, and how many of its works do? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q28 How many distinct works from Rostval Technical University (I09) cite Kesting University's research works? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q29 Of the works citing "Drift-aware mapping with low-cost sonar" (either version), how many are self-citations at the level of authors, of the University of Albury (units included), and of countries? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q30 Of the works citing "Drift-aware mapping with low-cost sonar" (either version), how many are self-citations at the level of the Albury University System? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q31 Which works cite at least one of the works that cite W01? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q32 How many distinct works are within two citation hops of W01 (citing it, or citing a work that cites it)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q33 Which of Kesting University's research works has the most citations? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q34 How many research works was Kesting University (units included) credited with in 2025? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q35 How many research works is the Kesting Institute of Hydrodynamics (I08) credited with? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q36 How many research works appeared in the Proceedings of the Hydrodynamics Forum (S06)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q37 Which institution is W70 credited to? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q38 Which institutions is W71 credited to, at every level? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q39 How many of the University of Albury's research works (units included) appeared in a source of the highest tier? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q40 Which source is in the highest tier? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q41 Is Coastal Engineering Review (S03) in a higher tier than Ocean Sensing Letters (S02)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q42 How many research works appeared in Global Marine Science Reports (S04)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q43 Is W07 research output? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q44 Which institutions are part of the Albury University System, at any level below it? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q45 How many research works is the Halloran Institute for Coastal Studies (I06) credited with? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q46 What is the ultimate parent of the Halloran Institute for Coastal Studies (I06)? | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q47 Why do Kesting University and Rostval Technical University swap places in 2024 between full and fractional counting? | 🟡🟡🟡🟡✅ | ✅✅✅🟡✅ |
| Q48 Why is W70 not one of the University of Albury's research works, though its author lists the Tessel Laboratory? | 🟡✅🟡✅✅ | ✅✅✅✅✅ |
At 850 works
| Condition | claude:haiku, reading 48 questions, 5 repeats | claude:sonnet, reading 48 questions, 5 repeats | claude:sonnet, querying 48 questions, 5 repeats | claude:sonnet, querying 48 questions, 5 repeats |
|---|---|---|---|---|
R1 csv | 21% 12–31 | 42% 30–54 | ||
R2 rdf | left out: the prompt, about 770 KB, is too large for the model's context | |||
R3 csv+padding | 30% 19–41 | 42% 30–54 | ||
R4 csv+markdown | 43% 34–51 | 87% 78–94 | ||
R5 csv+ontology | 49% 40–58 | 89% 82–95 | ||
R6 rdf+ontology | left out: the prompt, about 770 KB, is too large for the model's context | |||
R10 executed | 69% 58–79 | 79% 69–89 | ||
D1 sql | 38% 26–50 | |||
D2 sql+markdown | 80% 74–85 | |||
D3 sql+executed | 92% 84–98 | |||
D4 sparql | 30% 20–41 | |||
D5 sparql+ontology | 85% 79–90 | |||
D6 sparql+axioms | 91% 85–96 | |||
Reading, at ten times the base data (Sonnet, Haiku), five repeats. The prompts are 110 to 160 KB, against 19 to 39 KB for the base data. R2 rdf and R6 rdf+ontology are left out: as Turtle, the data alone is about 770 KB, too large for the model's context.
| Condition | Sonnet | Haiku | On the base data: Sonnet, Haiku |
|---|---|---|---|
R1 csv |
42% | 21% | 57%, 45% |
R3 csv+padding |
42% (+0) | 30% (+9) | 57%, 44% |
R4 csv+markdown |
87% (+45) | 43% (+22) | 98%, 99% |
R5 csv+ontology |
89% (+47) | 49% (+28) | 98%, 96% |
R10 executed |
79% (+37) | 69% (+48) | 90%, 92% |
- For Sonnet, the definitions gain as much as on the base data, +45 to +47 points, and the local rules stay right: 97% on Q33 to Q48 with the definitions, against 21% without. What drops is the general questions, from 97% to 82–85%. These are the counts and roll-ups over the whole data (Q9, Q10, Q11, Q14, Q16, Q19, Q21), which are now ten times longer to tally by reading. Q1 is the odd one: with the definitions, Sonnet answers that the three name forms stand for one person, where the records with those names carry three ORCIDs.
- Haiku starts to give up. In 5 of its 25 sessions it answers only some of the questions, sometimes saying so: "Q9–Q48: not yet answered… I'll continue". Four of the five are with the definitions, whose prompts are the largest. Counting only the sessions that answered everything, it scores 23% with R1
csv, 58% with R5csv+ontologyand 74% with R4csv+markdown(2 sessions). - Executed rules matter more as the data grows, most of all for the weaker model. For Haiku, R10
executedis now the best condition: the merging and rolling up are done, and its prompt is the smallest. For Sonnet it is 8 to 10 points short of the definitions, as on the base data. - The decoy glossary helps Haiku by 9 points (+4 to +15). That's the first time it has helped either model, and it may be noise in a run where Haiku answers so unevenly.
Querying, at the same size (Sonnet), five repeats, against reading with the same model. The domain had no axioms when this ran, so no D6 sparql+axioms.
| No definitions | With the definitions | Rules executed | |
|---|---|---|---|
| Reading | R1 csv 42% |
R4 csv+markdown 87% |
R10 executed 79% |
| SQL | D1 sql 38% |
D2 sql+markdown 80% |
D3 sql+executed 92% |
| SPARQL | D4 sparql 30% |
D5 sparql+ontology 85% |
- Querying doesn't recover the counts by itself. With the definitions, SQL and SPARQL score 80% and 85%, no better than reading. On the general questions they get 75% and 80%, against reading's 82–85%. The model writes the queries, but has to get every join and filter of each count right, and the definitions don't make that any easier.
- Querying over executed tables is the best condition at this size: 92%, in every session between 90% and 94%, against 79% when reading those same tables. The merging and roll-ups are done, and the model only has to count what the tables already hold.
- The definitions are steady when reading, not when querying. D2
sql+markdownranges from 41% to 97% across the five sessions, and D5sparql+ontologyfrom 73% to 95%. In the weakest session, the model stopped after four rounds and answered from what it had seen. - SPARQL alone is the weakest condition again, at 30%, 7 points below SQL (−13 to −2). Plain D4
sparqlused all eight rounds in every session. - 34 of 643 queries failed, and 5 timed out on rdflib.
Querying the graph after the axioms' rules have run (Sonnet), five repeats, added once the domain had axioms:
| Score | In each session | |
|---|---|---|
D2 sql+markdown |
80% | 97, 84, 41, 80, 97 |
D5 sparql+ontology |
85% | 74, 73, 95, 93, 91 |
D3 sql+executed |
92% | 92, 92, 92, 94, 90 |
D6 sparql+axioms |
91% | 82, 96, 93, 97, 89 |
- A graph whose rules have run does as well as executed tables: 91% against 92%, and 6 to 11 points above querying with only the definitions. The rules have merged the people and the preprints, credited each person's institution and rolled it up the hierarchy, so each count is a short query over facts the store already holds.
- It's what makes querying pay off at this size. Reading with the definitions scores 87–89% here; querying the graph after the rules, 91%. The general questions, the counts and roll-ups reading got wrong, are right 90% of the time, against 82–85% reading.
- The misses are Q1 (the name forms again) and the "why" questions (Q47, Q48), which the derived facts don't explain. 11 of 174 queries failed; none timed out.
Every run on this dataset
Reading the data
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-06-0206-sonnet | claude:sonnet | read | 48 | R1 csv 42% 30–54R3 csv+padding 42% 30–54R4 csv+markdown 87% 78–94R5 csv+ontology 89% 82–95R10 executed 79% 69–89 |
done |
| 2026-11-06-0206-haiku | claude:haiku | read | 48 | R1 csv 21% 12–31R3 csv+padding 30% 19–41R4 csv+markdown 43% 34–51R5 csv+ontology 49% 40–58R10 executed 69% 58–79 |
done |
Writing queries
| Run | Model | Mode | Questions | Accuracy by condition (95% interval) | Status |
|---|---|---|---|---|---|
| 2026-11-06-0855-sonnet-query | claude:sonnet | query | 48 | D6 sparql+axioms 91% 85–96 |
done |
| 2026-11-06-0519-sonnet-query | claude:sonnet | query | 48 | D1 sql 38% 26–50D2 sql+markdown 80% 74–85D3 sql+executed 92% 84–98D4 sparql 30% 20–41D5 sparql+ontology 85% 79–90 |
done |
Question by question: the latest finished reading run, 2026-11-06-0206-sonnet
One mark per repeat; hover for the grader's reason, click a question for every answer to it. The run's page.
| Question | R1 csv | R3 csv+padding | R4 csv+markdown | R5 csv+ontology | R10 executed |
|---|---|---|---|---|---|
| Q1 How many distinct people do the author records named "Mara Ellison", "M. Ellison" and "Ellison, Mara J." stand for? | ✅✅✅✅✅ | ✅❌✅✅✅ | ✅❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ |
| Q2 List every work, of any type, that Mara Ellison (A01) is an author of. | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ❌❌❌❌❌ |
| Q3 Is the Jonas Brandt who wrote W37 the same person as the Jonas Brandt who wrote W38? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q4 How many research works has the Jonas Brandt of Albury Metropolitan University (A05) authored? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅❌✅ | ✅✅✅✅✅ |
| Q5 Is the M. Ellison who wrote W08 the same person as the Mara Ellison who wrote W01? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q6 Which author has the most research works credited to the University of Albury, its units included? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q7 How many research works has Mara Ellison authored? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ❌✅✅✅✅ | ✅❌❌❌❌ |
| Q8 What is the ultimate parent of the Tessel Laboratory for Ocean Sensing (I04)? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q9 How many research works is the University of Albury (I02) credited with, its units included? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌✅❌✅ | ❌✅✅✅✅ | ❌❌❌❌❌ |
| Q10 How many research works is the Albury University System (I01) credited with, all its units included? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅❌❌✅ | ❌❌❌❌❌ |
| Q11 How many research works is Kesting University (I07) credited with, its units included? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌✅✅ | ✅✅❌✅❌ | ✅✅✅✅✅ |
| Q12 How many research works did Kesting University (I07, units included) publish in 2024, under full counting and under fractional counting (each work split equally among its authors)? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q13 In 2024, which had more research works, Kesting University (I07) or Rostval Technical University (I09): under full counting, and under fractional counting? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q14 How many research works did GB institutions publish in 2024, under full counting and under fractional counting (each work split equally among its authors, each author's share going to their institution's country)? | ❌❌❌❌❌ | ❌❌❌❌❌ | 🟡✅❌❌✅ | ❌❌❌🟡🟡 | ❌✅❌✅✅ |
| Q15 How many research works appeared in Ocean Sensing Letters (S02) in 2024? | ❌❌❌❌✅ | ✅❌❌❌❌ | ✅✅✅✅✅ | ✅❌✅✅✅ | ❌❌✅❌✅ |
| Q16 How many research works appeared in the Journal of Marine Autonomy (S01)? | ❌❌❌✅❌ | ❌✅✅❌❌ | ❌✅❌❌❌ | ✅✅❌❌✅ | ❌❌❌❌❌ |
| Q17 Which of Mara Ellison's works are not research output because of their document type? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q18 How many distinct research works has Priya Ramanathan (A07) authored? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q19 How many distinct works cite "Drift-aware mapping with low-cost sonar", in either of its versions? | ❌❌❌❌❌ | ✅❌❌✅❌ | ❌✅❌✅❌ | ✅❌❌✅❌ | ❌✅❌❌❌ |
| Q20 Which works in the data are preprints of another work that is also in the data? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ❌❌❌❌❌ |
| Q21 How many research works is Oskel Polytechnic (I14) credited with? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌❌✅❌✅ | ✅❌❌✅✅ |
| Q22 Which has more citations, W30 or W31 (counting both versions of each): in total, and within its publication year and the two years after? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q23 How many citations has W30 received in total, and how many within its publication year and the two years after? | 🟡🟡🟡🟡🟡 | 🟡🟡🟡🟡🟡 | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q24 How many research works has Lena Vossberg (A09) authored? | ✅❌❌✅✅ | ✅❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q25 What is the total of the citation counts of Lena Vossberg's research works? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q26 Does W23 count towards Kesting University's research output? | ✅❌❌✅✅ | ✅❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q27 Which other institution's works cite Kesting University's research works the most, each citing work counted once per institution, and how many of its works do? | 🟡🟡🟡🟡🟡 | 🟡🟡🟡🟡🟡 | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q28 How many distinct works from Rostval Technical University (I09) cite Kesting University's research works? | ✅✅✅✅✅ | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q29 Of the works citing "Drift-aware mapping with low-cost sonar" (either version), how many are self-citations at the level of authors, of the University of Albury (units included), and of countries? | 🟡🟡🟡🟡🟡 | 🟡🟡🟡🟡🟡 | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q30 Of the works citing "Drift-aware mapping with low-cost sonar" (either version), how many are self-citations at the level of the Albury University System? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q31 Which works cite at least one of the works that cite W01? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q32 How many distinct works are within two citation hops of W01 (citing it, or citing a work that cites it)? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q33 Which of Kesting University's research works has the most citations? | ✅❌❌❌✅ | ✅✅✅✅❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q34 How many research works was Kesting University (units included) credited with in 2025? | ❌✅✅❌❌ | ❌✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q35 How many research works is the Kesting Institute of Hydrodynamics (I08) credited with? | ❌✅✅❌❌ | ❌❌✅✅✅ | ✅✅✅✅✅ | ✅✅❌✅✅ | ✅✅✅✅✅ |
| Q36 How many research works appeared in the Proceedings of the Hydrodynamics Forum (S06)? | ❌❌❌❌❌ | ❌❌❌❌❌ | ❌✅✅✅✅ | ✅❌✅✅✅ | ✅✅✅❌✅ |
| Q37 Which institution is W70 credited to? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q38 Which institutions is W71 credited to, at every level? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q39 How many of the University of Albury's research works (units included) appeared in a source of the highest tier? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅❌✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q40 Which source is in the highest tier? | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q41 Is Coastal Engineering Review (S03) in a higher tier than Ocean Sensing Letters (S02)? | ✅✅✅✅✅ | ✅✅✅✅❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q42 How many research works appeared in Global Marine Science Reports (S04)? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q43 Is W07 research output? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q44 Which institutions are part of the Albury University System, at any level below it? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q45 How many research works is the Halloran Institute for Coastal Studies (I06) credited with? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ✅✅✅✅✅ |
| Q46 What is the ultimate parent of the Halloran Institute for Coastal Studies (I06)? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | ❌❌❌❌❌ |
| Q47 Why do Kesting University and Rostval Technical University swap places in 2024 between full and fractional counting? | 🟡✅✅🟡✅ | 🟡🟡✅✅🟡 | ✅✅✅✅🟡 | ✅✅✅✅✅ | ✅✅🟡🟡🟡 |
| Q48 Why is W70 not one of the University of Albury's research works, though its author lists the Tessel Laboratory? | ❌❌❌❌❌ | ❌❌❌❌❌ | ✅✅✅✅✅ | ✅✅✅✅✅ | 🟡🟡🟡🟡🟡 |