Semantics ladder

How much does an LLM gain from being told what data means?

The experiment

Goals
Method

Credit ratings

Research analytics

The results

All data is invented: the companies, LEIs, ratings, works, people and institutions. Nothing here is any rating agency's or bibliographic database's data or definitions.

Last updated 6 November 2026.

Q27, run 2026-11-06-0855-haiku

← back to the run

Which other institution's works cite Kesting University's research works the most, each citing work counted once per institution, and how many of its works do?

Tests: Citations rolled up: Rostval's one citing work has four Rostval authors; counted per author it would win.

Expected: institution = I12 citing_works = 3

✅ R8 csv+axioms run 1 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35. W36 (I11) and W32 (I09) each contribute 1.

JSON: {"institution": "I12", "citing_works": 3}

✅ R8 csv+axioms run 2 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35.

JSON: {"institution": "I12", "citing_works": 3}

✅ R8 csv+axioms run 3 graded by code

I12, with 3 citing works (W33, W34, W35). Kesting's research works are cited by W32 (I09), W33, W34 and W35 (I12), and W36 (I11).

JSON: {"institution": "I12", "citing_works": 3}

✅ R8 csv+axioms run 4 graded by code

I12 (Université de Lorvain), with 3 citing works: W33, W34 and W35.

JSON: {"institution": "I12", "citing_works": 3}

✅ R8 csv+axioms run 5 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35) that cite Kesting's research works.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 1 graded by code

I12 (Université de Lorvain), with 3 citing works (W33, W34, W35). Those works cite W20, W21 or W22.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 2 graded by code

I12, with 3 citing works. W33, W34 and W35 are credited to I12, while W32 (I09) and W36 (I11) each count once for their own institution.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 3 graded by code

I12, with 3 citing works (W33, W34, W35). I09 has 1 (W32) and I11 has 1 (W36).

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 4 graded by code

I12 (Université de Lorvain), with 3 works. Its citing works W33, W34 and W35 cite Kesting's W21, W22 or W20; W32 (I09) and W36 (I11) each count once.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 5 graded by code

I12, with 3 citing works (W33, W34, W35). W32 (I09) and W36 (I11) each cite Kesting's work once.

JSON: {"institution": "I12", "citing_works": 3}