Semantics ladder

How much does an LLM gain from being told what data means?

The experiment

Goals
Method

Credit ratings

Research analytics

The results

All data is invented: the companies, LEIs, ratings, works, people and institutions. Nothing here is any rating agency's or bibliographic database's data or definitions.

Last updated 6 November 2026.

Q27, run 2026-11-04-1752-haiku

← back to the run

Which other institution's works cite Kesting University's research works the most, each citing work counted once per institution, and how many of its works do?

Tests: Citations rolled up: Rostval's one citing work has four Rostval authors; counted per author it would win.

Expected: institution = I12 citing_works = 3

✅ R1 csv run 1 graded by code

I12, with 3 citing works (W33, W34, W35). W32 cites Kesting from I09 (1 work), W36 from I11 (1), and W39 from I14 (1). W90–W92 are Kesting's own works.

JSON: {"institution": "I12", "citing_works": 3}

✅ R1 csv run 2 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. Other institutions have 1 each (I09, I11, I14).

JSON: {"institution": "I12", "citing_works": 3}

✅ R1 csv run 3 graded by code

I12, 3 works. W33, W34 and W35 cite Kesting's works, and each of W32 (I09), W36 (I11) and W39 (I14) is only one work.

JSON: {"institution": "I12", "citing_works": 3}

✅ R1 csv run 4 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. Kesting's own W90–W92 are excluded as self-citations.

JSON: {"institution": "I12", "citing_works": 3}

✅ R1 csv run 5 graded by code

I12 (Université de Lorvain), with 3 citing works. W33, W34 and W35 cite Kesting's W21 and W22 and W20, and no other institution reaches 3.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 1 graded by code

I12 (Université de Lorvain), with 3 works (W33, W34, W35). Kesting's own works W90, W91 and W92 are excluded as self-citations.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 2 graded by code

I12 (Université de Lorvain), with 3 works (W33, W34, W35). I09, I11 and I14 each have 1, and W90–W92 are I07's own.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 3 graded by code

I12 (Université de Lorvain), with 3 citing works. W33, W34 and W35 are all I12 works, and they cite W21 and W22 (W33, W34) or W20 (W35). Note that these citations come from I12, not from the Kesting-affiliated W90, W91 and W92.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 4 graded by code

I12 (Université de Lorvain), citing 3 works: W33, W34 and W35. These cite Kesting works W21 and W22 and W20. Kesting's own works W90–W92 are excluded.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 5 graded by code

I12 (Université de Lorvain), with 3 works (W33, W34, W35) citing Kesting's W21 and W22. Kesting's own W90–W92 are excluded as self-citations.

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 1 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35). W32 is credited to I09 and W36 to I11, each with one work.

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 2 graded by code

I12, 3 works. W33, W34 and W35 are the citing works from I12, each counted once for I12. I09 has only 1 citing work, and I11 has 1.

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 3 graded by code

I12 (Université de Lorvain), with 3 works. W33, W34 and W35 are its citing works; I09 and I11 each have only one.

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 4 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35). These are the citing works of Kesting's research works that are credited to I12, counted once each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 5 graded by code

I12, with 3 citing works. W33, W34 and W35 cite Kesting works and are all credited to I12; I09 and I11 each have one.

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 1 graded by code

I12 (Université de Lorvain), with 3 citing works. Its W33, W34 and W35 cite Kesting's W21 and W22 and W20 respectively; I09 (W32) and I11 (W36) have one each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 2 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35). Kesting's cited works are cited by W32 (I09), W33–W35 (I12) and W36 (I11).

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 3 graded by code

I12 (Université de Lorvain), with 3 citing works. W33, W34 and W35 cite Kesting's research works, and all three are credited to I12.

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 4 graded by code

I12 (Université de Lorvain), citing 3 works (W33, W34, W35). Each citing work is counted once per institution.

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 5 graded by code

I12 (Université de Lorvain), citing 3 Kesting works through W33, W34 and W35. The other citing institutions are I09 and I11, with one work each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 1 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35).

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 2 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. W32 is credited to I09 and W36 to I11, one each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 3 graded by code

I12, 3 works. W33, W34 and W35 cite Kesting's research works, and each is credited to I12 (Université de Lorvain).

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 4 graded by code

I12, 3 citing works. W33, W34 and W35 are each from I12 (Université de Lorvain), and no other outside institution cites Kesting's works more often.

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 5 graded by code

I12 (Université de Lorvain), with 3 works: W33, W34 and W35. Rostval (I09) and Oskel's I11 each have 1.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 1 graded by code

I12 (Université de Lorvain), with 3 citing works. W33, W34 and W35 cite Kesting's W21, W22 and W20. Rostval, Brevin and Oskel each have only 1 citing work, and Kesting's own W90–W92 are self-citations.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 2 graded by code

I12, with 3 citing works (W33, W34, W35). Those are the most among other institutions.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 3 graded by code

I12 (Université de Lorvain), with 3 works. W33, W34 and W35 are the citing works from I12; Rostval (I09), Brevin (I11) and Oskel (I14) each have 1.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 4 graded by code

I12 (Université de Lorvain), with 3 works: W33, W34, W35.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 5 graded by code

I12 (Université de Lorvain), with 3 works: W33, W34 and W35.

JSON: {"institution": "I12", "citing_works": 3}