Run 2026-11-06-0935-sonnet-query

Credit ratings: the base data claude:sonnet query 48 questions started 2026-11-06 09:35 commit fddbced done

Accuracy

ConditionRepeatsAccuracy95% interval
D1 sql5 84%73–93%
D2 sql+markdown5 100%99–100%
D3 sql+executed5 99%98–100%
D4 sparql5 82%71–91%
D5 sparql+ontology5 100%100–100%
D6 sparql+axioms5 90%81–98%

Queries

ConditionQueries runFailedRounds, on average
D1 sql1501.0
D2 sql+markdown1501.2
D3 sql+executed2501.0
D4 sparql3413.0
D5 sparql+ontology1701.6
D6 sparql+axioms3303.4

Question by question

One mark per repeat; hover for the grader's reason, click a question for every answer to it.

QuestionD1 sqlD2 sql+markdownD3 sql+executedD4 sparqlD5 sparql+ontologyD6 sparql+axioms
Q1 What was Kestrel Energy's rating on 1 January 2025? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q2 Has Kestrel Energy been downgraded in 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q3 Is Kestrel Energy's 2031 note investment grade? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q4 What is Northwind Holdings' current outlook? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q5 Whose credit risk does a holder of the Northwind Finance 2030 notes take, and how is it rated? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q6 Why is Valdoria Telecom's foreign-currency rating lower than its local-currency rating? 🟡✅✅🟡✅🟡✅✅✅✅✅🟡🟡✅🟡🟡🟡🟡🟡🟡✅✅✅✅✅🟡✅✅✅✅
Q7 Which issuers were downgraded in 2026? ❌❌❌❌❌✅✅✅✅✅✅✅✅✅✅❌❌❌❌❌✅✅✅✅✅❌❌❌❌❌
Q8 Which entities were in default on 1 October 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q9 As of 1 October 2026, how many distinct issuers hold an issuer rating? ❌❌❌❌❌✅✅✅✅✅✅✅✅✅✅❌❌❌❌❌✅✅✅✅✅❌❌❌✅❌
Q10 Is Valdoria Telecom investment grade? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q11 What was Marlow Utilities' rating on 1 June 2025? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q12 What was Marlow Utilities' rating on 1 January 2024? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q13 How many times has Marlow Utilities been downgraded? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q14 What was Halden Foods' rating on 1 January 2024? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q15 What was Halden Bank's rating on 1 January 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q16 Has Marlow Utilities' rating changed since 1 January 2025? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q17 Was Orrin Bank downgraded in 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q18 Which issuers were on Rating Watch Negative on 1 October 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q19 Which issuers had a Negative outlook on 1 October 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q20 Is Pellam Retail in default? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q21 Which entities had an issuer rating at some point but no longer had one on 1 October 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q22 Which entities have ever been in default? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q23 What is Kestrel Energy's short-term rating? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q24 Is Kestrel Energy's issuer rating investment grade? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q25 How many instruments held an investment-grade rating on 1 October 2026? ❌✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅❌❌✅✅✅✅✅✅✅✅✅✅
Q26 What is Corvenia Power's local-currency rating? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q27 How is Corvenia Power's 2032 note rated? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q28 What is the Republic of Corvenia's foreign-currency issuer rating? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q29 What is the Republic of Valdoria's issuer rating? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q30 Which entities had a foreign-currency issuer rating exactly at their country's ceiling on 1 October 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q31 Did any company have a foreign-currency issuer rating above its country's ceiling on 1 October 2026? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q32 Who is Halden Leasing's ultimate parent? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q33 Whose credit risk does a holder of the Halden Leasing 2028 notes take, and how is it rated? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q34 Is Halden Foods part of the same group as Halden Bank? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q35 How many entities are in the Halden group, the parent included? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q36 How many entities are in the Northwind group, the parent included? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q37 How many distinct legal entities does the data describe? ❌❌❌❌❌✅✅✅✅✅✅✅✅✅✅❌❌❌❌❌✅✅✅✅✅❌❌❌❌❌
Q38 Are Orrin Bank AG and Orrin Bancorp Inc the same entity? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q39 What is Northwind Holdings' current rating? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q40 How many distinct issuers domiciled in GB held an issuer rating on 1 October 2026? ❌❌❌❌❌✅✅✅✅✅✅✅✅✅✅❌❌❌❌❌✅✅✅✅✅❌❌❌✅❌
Q41 What is Brindle Capital's current rating? ❌❌❌❌❌✅✅✅✅✅✅✅✅✅✅❌❌❌❌❌✅✅✅✅✅❌❌❌❌❌
Q42 In which country is Halden Bank domiciled? ❌❌❌❌❌✅✅✅✅✅✅✅✅✅✅❌❌❌❌❌✅✅✅✅✅✅✅✅✅✅
Q43 How many distinct entities are domiciled in Luxembourg (LU)? ❌❌❌❌❌✅✅✅✅✅✅✅✅✅✅❌✅❌❌❌✅✅✅✅✅✅✅✅✅✅
Q44 Is Valdoria Telecom's 2027 note a local-currency obligation? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q45 Is Valdoria Telecom's 2027 note rated above what Valdoria's Country Ceiling allows? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅❌✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q46 Does Halden Leasing have an issuer rating? ✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅
Q47 How is Corvenia Power's 2034 note rated? ❌✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅❌❌✅✅✅✅✅✅✅✅✅✅
Q48 How many instruments held a rating on 1 October 2026? ❌✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅✅❌❌✅✅✅✅✅✅✅✅✅✅

The prompts

The report, as report.md

credit: claude:sonnet, 2026-11-06 09:35

48 questions, 5 repeats per condition. Correct = 1, partial = ½. Graded by code where a question has a typed key, by claude:sonnet with a checklist otherwise.

Accuracy

Condition Accuracy 95% interval Questions unstable across repeats
D1 sql 84% 73% to 93% 4 of 48
D2 sql+markdown 100% 99% to 100% 1 of 48
D4 sparql 82% 71% to 91% 5 of 48
D5 sparql+ontology 100% 100% to 100% 0 of 48
D3 sql+executed 99% 98% to 100% 1 of 48
D6 sparql+axioms 90% 81% to 98% 3 of 48

Against D1 sql, question by question

Condition Difference 95% interval
D2 sql+markdown +16 points +7 to +27
D4 sparql -2 points -4 to +0
D5 sparql+ontology +16 points +7 to +27
D3 sql+executed +16 points +6 to +27
D6 sparql+axioms +6 points +2 to +13

By question

Question Graded by D1 sql D2 sql+markdown D4 sparql D5 sparql+ontology D3 sql+executed D6 sparql+axioms
Q1 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q2 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q3 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q4 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q5 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q6 llm 🟡 ✅ ✅ 🟡 ✅ 🟡 ✅ ✅ ✅ ✅ 🟡 🟡 🟡 🟡 🟡 ✅ ✅ ✅ ✅ ✅ ✅ 🟡 🟡 ✅ 🟡 🟡 ✅ ✅ ✅ ✅
Q7 code ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
Q8 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q9 code ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ✅ ❌
Q10 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q11 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q12 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q13 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q14 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q15 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q16 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q17 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q18 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q19 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q20 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q21 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q22 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q23 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q24 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q25 code ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q26 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q27 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q28 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q29 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q30 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q31 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q32 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q33 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q34 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q35 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q36 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q37 code ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
Q38 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q39 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q40 code ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ✅ ❌
Q41 code ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌
Q42 code ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q43 code ❌ ❌ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ❌ ✅ ❌ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q44 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q45 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q46 code ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q47 code ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅
Q48 code ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ❌ ❌ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅ ✅

Lists, with partial credit

The 7 questions answered with a list of entities: strict accuracy (the whole list right) against the mean F1 of the lists given, which credits a list that is nearly right.

Condition Strict Mean F1
D1 sql 86% 97%
D2 sql+markdown 100% 100%
D4 sparql 86% 97%
D5 sparql+ontology 100% 100%
D3 sql+executed 100% 100%
D6 sparql+axioms 86% 97%

The checklist grader agreed with itself on 30 of 30 answers graded twice.

Why each miss was marked

Q6. Why is Valdoria Telecom's foreign-currency rating lower than its local-currency rating?

  • D1 sql run 1: Names the BB Country Ceiling as the likely cause but does not say it caps foreign-currency (not local-currency) ratings, and hedges that the reason isn't stated.
  • D1 sql run 4: Names the BB Country Ceiling as the cap on foreign-currency rating, but never says the ceiling applies only to foreign-currency and not local-currency ratings.
  • D2 sql+markdown run 1: Names the BB Country Ceiling as the cap on the FC rating, but never states that it applies only to foreign-currency ratings and not local-currency ones (only implied by the LC rating being higher).
  • D4 sparql run 1: It names the BB Country Ceiling and shows FC equals it while LC is higher, but it hedges that the data states no reason instead of asserting the ceiling as the cause, and it never explicitly says the ceiling caps FC and not LC.
  • D4 sparql run 2: Identifies the BB country ceiling matching the FC rating as the likely cause, but hedges as unconfirmed and never states that the ceiling caps foreign-currency ratings and not local-currency ones.
  • D4 sparql run 3: Names the BB Country Ceiling as the cause, but never says the ceiling caps foreign-currency ratings and not local-currency ones; it only notes the FC rating matches the ceiling.
  • D4 sparql run 4: Mentions the BB Country Ceiling alongside the FC rating of BB but states no reason and doesn't say the ceiling caps FC and not LC ratings, so it never names the ceiling as the cause.
  • D4 sparql run 5: Names the BB country ceiling as the likely cause, but never says the ceiling caps foreign-currency ratings and not local-currency ones.
  • D3 sql+executed run 2: Names the BB Country Ceiling and links it to the FC rating being capped, but hedges ('consistent with') and never states that the ceiling applies to foreign-currency and not local-currency ratings.
  • D3 sql+executed run 3: Names the BB Country Ceiling as consistent with the gap, but hedges that the data states no reason and never says the ceiling caps foreign-currency ratings and not local-currency ones.
  • D3 sql+executed run 5: Names the BB Country Ceiling as the likely cause, but never states that it caps foreign-currency ratings and not local-currency ones, and hedges that the cause isn't explicit.
  • D6 sparql+axioms run 1: Identifies the BB Country Ceiling as matching the foreign-currency rating and implies a cap, but hedges and never says the ceiling applies to foreign-currency and not local-currency ratings.

Q7. Which issuers were downgraded in 2026?

  • D1 sql run 1: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D1 sql run 2: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D1 sql run 3: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D1 sql run 4: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D1 sql run 5: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D4 sparql run 1: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D4 sparql run 2: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D4 sparql run 3: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D4 sparql run 4: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D4 sparql run 5: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D6 sparql+axioms run 1: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D6 sparql+axioms run 2: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D6 sparql+axioms run 3: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D6 sparql+axioms run 4: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']
  • D6 sparql+axioms run 5: issuers: expected ['E08', 'E10'], got ['E08', 'E10', 'E16']

Q9. As of 1 October 2026, how many distinct issuers hold an issuer rating?

  • D1 sql run 1: count: expected 12, got 13
  • D1 sql run 2: count: expected 12, got 13
  • D1 sql run 3: count: expected 12, got 13
  • D1 sql run 4: count: expected 12, got 13
  • D1 sql run 5: count: expected 12, got 13
  • D4 sparql run 1: count: expected 12, got 13
  • D4 sparql run 2: count: expected 12, got 13
  • D4 sparql run 3: count: expected 12, got 13
  • D4 sparql run 4: count: expected 12, got 13
  • D4 sparql run 5: count: expected 12, got 13
  • D6 sparql+axioms run 1: count: expected 12, got 13
  • D6 sparql+axioms run 2: count: expected 12, got 13
  • D6 sparql+axioms run 3: count: expected 12, got 13
  • D6 sparql+axioms run 5: count: expected 12, got 13

Q25. How many instruments held an investment-grade rating on 1 October 2026?

  • D1 sql run 1: count: expected 5, got 6
  • D4 sparql run 4: count: expected 5, got 6
  • D4 sparql run 5: count: expected 5, got 6

Q37. How many distinct legal entities does the data describe?

  • D1 sql run 1: count: expected 16, got 17
  • D1 sql run 2: count: expected 16, got 17
  • D1 sql run 3: count: expected 16, got 17
  • D1 sql run 4: count: expected 16, got 17
  • D1 sql run 5: count: expected 16, got 17
  • D4 sparql run 1: count: expected 16, got 17
  • D4 sparql run 2: count: expected 16, got 17
  • D4 sparql run 3: count: expected 16, got 17
  • D4 sparql run 4: count: expected 16, got 17
  • D4 sparql run 5: count: expected 16, got 17
  • D6 sparql+axioms run 1: count: expected 16, got 17
  • D6 sparql+axioms run 2: count: expected 16, got 17
  • D6 sparql+axioms run 3: count: expected 16, got 18
  • D6 sparql+axioms run 4: count: expected 16, got 17
  • D6 sparql+axioms run 5: count: expected 16, got 17

Q40. How many distinct issuers domiciled in GB held an issuer rating on 1 October 2026?

  • D1 sql run 1: count: expected 5, got 6
  • D1 sql run 2: count: expected 5, got 6
  • D1 sql run 3: count: expected 5, got 6
  • D1 sql run 4: count: expected 5, got 6
  • D1 sql run 5: count: expected 5, got 6
  • D4 sparql run 1: count: expected 5, got 6
  • D4 sparql run 2: count: expected 5, got 6
  • D4 sparql run 3: count: expected 5, got 6
  • D4 sparql run 4: count: expected 5, got 6
  • D4 sparql run 5: count: expected 5, got 6
  • D6 sparql+axioms run 1: count: expected 5, got 6
  • D6 sparql+axioms run 2: count: expected 5, got 6
  • D6 sparql+axioms run 3: count: expected 5, got 6
  • D6 sparql+axioms run 5: count: expected 5, got 6

Q41. What is Brindle Capital's current rating?

  • D1 sql run 1: rating: expected none, got BB+
  • D1 sql run 2: rating: expected none, got BB+
  • D1 sql run 3: rating: expected none, got BB+
  • D1 sql run 4: rating: expected none, got BB+
  • D1 sql run 5: rating: expected none, got BB+
  • D4 sparql run 1: rating: expected none, got BB+
  • D4 sparql run 2: rating: expected none, got BB+
  • D4 sparql run 3: rating: expected none, got BB+
  • D4 sparql run 4: rating: expected none, got BB+
  • D4 sparql run 5: rating: expected none, got BB+
  • D6 sparql+axioms run 1: rating: expected none, got BB+
  • D6 sparql+axioms run 2: rating: expected none, got BB+
  • D6 sparql+axioms run 3: rating: expected none, got BB+
  • D6 sparql+axioms run 4: rating: expected none, got BB+
  • D6 sparql+axioms run 5: rating: expected none, got BB+

Q42. In which country is Halden Bank domiciled?

  • D1 sql run 1: country: expected LU, got DE
  • D1 sql run 2: country: expected LU, got DE
  • D1 sql run 3: country: expected LU, got DE
  • D1 sql run 4: country: expected LU, got DE
  • D1 sql run 5: country: expected LU, got DE
  • D4 sparql run 1: country: expected LU, got DE
  • D4 sparql run 2: country: expected LU, got DE
  • D4 sparql run 3: country: expected LU, got DE
  • D4 sparql run 4: country: expected LU, got DE
  • D4 sparql run 5: country: expected LU, got DE

Q43. How many distinct entities are domiciled in Luxembourg (LU)?

  • D1 sql run 1: count: expected 1, got 0
  • D1 sql run 2: count: expected 1, got 0
  • D1 sql run 3: count: expected 1, got 0
  • D1 sql run 4: count: expected 1, got 0
  • D1 sql run 5: count: expected 1, got 0
  • D4 sparql run 1: count: expected 1, got 0
  • D4 sparql run 3: count: expected 1, got 0
  • D4 sparql run 4: count: expected 1, got 0
  • D4 sparql run 5: count: expected 1, got 0

Q45. Is Valdoria Telecom's 2027 note rated above what Valdoria's Country Ceiling allows?

  • D4 sparql run 1: above_ceiling: expected no, got yes

Q47. How is Corvenia Power's 2034 note rated?

  • D1 sql run 1: rating: expected none, got A
  • D4 sparql run 4: rating: expected none, got A
  • D4 sparql run 5: rating: expected none, got A

Q48. How many instruments held a rating on 1 October 2026?

  • D1 sql run 1: count: expected 7, got 8
  • D4 sparql run 4: count: expected 7, got 8
  • D4 sparql run 5: count: expected 7, got 8