Semantics ladder

How much does an LLM gain from being told what data means?

The experiment

Goals
Method

Credit ratings

Research analytics

The results

All data is invented: the companies, LEIs, ratings, works, people and institutions. Nothing here is any rating agency's or bibliographic database's data or definitions.

Last updated 6 November 2026.

Q27, run 2026-11-05-1235-haiku

← back to the run

Which other institution's works cite Kesting University's research works the most, each citing work counted once per institution, and how many of its works do?

Tests: Citations rolled up: Rostval's one citing work has four Rostval authors; counted per author it would win.

Expected: institution = I12 citing_works = 3

✅ R1 csv run 1 graded by code

I12, with 3 citing works (W33, W34, W35). Kesting's works are cited by W32 (I09), W33/W34/W35 (I12), W36 (I11) and W39 (I14).

JSON: {"institution": "I12", "citing_works": 3}

✅ R1 csv run 2 graded by code

I12, citing 3 works.
*Reasoning:* I12 authors W33, W34 and W35, which cite Kesting works W21 and W22 and W20. The other institutions have one each (I09 via W32, I11 via W36, I14 via W39).

JSON: {"institution": "I12", "citing_works": 3}

✅ R1 csv run 3 graded by code

I12 (Université de Lorvain), with 3 citing works. W33, W34 and W35 cite Kesting's works and have I12 authors. The other citing institutions have one work each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R1 csv run 4 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35.

JSON: {"institution": "I12", "citing_works": 3}

✅ R1 csv run 5 graded by code

I12 (Université de Lorvain), with 3 citing works. W33, W34 and W35 are all I12-affiliated and cite Kesting work (W21, W22 and W20). W32 (I09) and W36 (I11) each give 1.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 1 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 2 graded by code

I12 (Université de Lorvain), 3 citing works. W33, W34 and W35 cite Kesting's works; no other institution exceeds one citing work.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 3 graded by code

I12 (Université de Lorvain), citing works 3: W33, W34 and W35 each cite a Kesting work.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 4 graded by code

I12, 3 citing works. Works from I12 (W33, W34, W35) cite Kesting research works. I09 (W32) and I11 (W36) each have one citing work, and I14 (W39) has one.

JSON: {"institution": "I12", "citing_works": 3}

✅ R2 rdf run 5 graded by code

I12, with 3 citing works. W33, W34 and W35 are all from I12 and cite Kesting works (I07's works are excluded as self-citations).

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 1 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35.

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 2 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. These cite W22 and W21, which are Kesting research works.

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 3 graded by code

I12, 3. Its works W33, W34 and W35 cite Kesting's research works, against 1 each for I09 (W32) and I11 (W36).

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 4 graded by code

I12 (Université de Lorvain), 3 works. Its works W33, W34 and W35 cite Kesting's W21 or W22, and W36 (I11) and W32 (I09) each account for only one.

JSON: {"institution": "I12", "citing_works": 3}

✅ R6 rdf+ontology run 5 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35.

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 1 graded by code

I12 (Université de Lorvain), with 3 works. W33, W34 and W35 each cite a Kesting research work and are credited to I12.

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 2 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35). Kesting's research works are cited by W32 (I09), W33, W34 and W35 (I12), and W36 (I11).

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 3 graded by code

I12, 3 works. Université de Lorvain (I12) cites Kesting's research works through W33, W34 and W35; Rostval (I09) has 1 (W32) and Brevin (I11) has 1 (W36).

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 4 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35). Kesting's research works are cited by W32 (I09), W33, W34, W35 (I12) and W36 (I11), with each work counted once per institution.

JSON: {"institution": "I12", "citing_works": 3}

✅ R5 csv+ontology run 5 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. Other institutions have fewer: I09 has 1 (W32) and I11 has 1 (W36).

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 1 graded by code

I12 (Université de Lorvain), with 3 citing works. W33, W34 and W35 cite Kesting's works, and each is credited to I12.

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 2 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. These cite Kesting's W21 and W22 and W20 respectively.

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 3 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. These cite Kesting's W22, W21 and W20.

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 4 graded by code

I12, with 3 citing works. W33, W34 and W35 cite Kesting works. W32 (I09) and W36 (I11) each contribute one work.

JSON: {"institution": "I12", "citing_works": 3}

✅ R4 csv+markdown run 5 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35).

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 1 graded by code

I12 (Université de Lorvain), with 3 citing works. W33, W34 and W35 each cite a Kesting work.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 2 graded by code

I12 (Université de Lorvain), with 3 works. W33, W34 and W35 each cite Kesting's work, and each citing work is counted once.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 3 graded by code

I12 (Université de Lorvain), with 3 works (W33, W34, W35). Kesting-authored citers W90–W92 are excluded.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 4 graded by code

Université de Lorvain (I12), with 3 works: W33, W34, W35 cite W21, W22, W20.

JSON: {"institution": "I12", "citing_works": 3}

✅ R3 csv+padding run 5 graded by code

I12 (Université de Lorvain), with 3 works (W33, W34, W35) citing Kesting's works.

JSON: {"institution": "I12", "citing_works": 3}

✅ R10 executed run 1 graded by code

I12 (Université de Lorvain), with 3 citing works. Kesting's research works (W08, W16, W20, W21, W22, W24) are cited by W32 (I09), W33, W34, W35 (I12) and W36 (I11).

JSON: {"institution": "I12", "citing_works": 3}

✅ R10 executed run 2 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. Citations to Kesting's research works come from W32 (I09), W33/W34/W35 (I12) and W36 (I11).

JSON: {"institution": "I12", "citing_works": 3}

✅ R10 executed run 3 graded by code

I12 (Université de Lorvain), with three citing works: W33, W34 and W35. The other citing institutions are I09 (W32) and I11 (W36), with one each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R10 executed run 4 graded by code

Université de Lorvain (I12), 3 works (W33, W34, W35). These cite Kesting's W20, W21 and W22.

JSON: {"institution": "I12", "citing_works": 3}

✅ R10 executed run 5 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. W32 (I09) and W36 (I11) each cite Kesting once.

JSON: {"institution": "I12", "citing_works": 3}