Semantics ladder

How much does an LLM gain from being told what data means?

The experiment

Goals
Method

Credit ratings

Research analytics

The results

All data is invented: the companies, LEIs, ratings, works, people and institutions. Nothing here is any rating agency's or bibliographic database's data or definitions.

Last updated 6 November 2026.

Q27, run 2026-11-06-0855-sonnet

← back to the run

Which other institution's works cite Kesting University's research works the most, each citing work counted once per institution, and how many of its works do?

Tests: Citations rolled up: Rostval's one citing work has four Rostval authors; counted per author it would win.

Expected: institution = I12 citing_works = 3

✅ R8 csv+axioms run 1 graded by code

I12, with 3 citing works (W33, W34, W35). I09 and I11 have 1 each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R8 csv+axioms run 2 graded by code

I12, with 3 citing works: W33, W34, W35. I09 and I11 have 1 each, and the staging works are excluded.

JSON: {"institution": "I12", "citing_works": 3}

✅ R8 csv+axioms run 3 graded by code

I12, with 3 citing works (W33, W34, W35). I11 has 1 (W36) and I09 has 1 (W32).

JSON: {"institution": "I12", "citing_works": 3}

✅ R8 csv+axioms run 4 graded by code

I12, with 3 citing works (W33, W34, W35). I09 and I11 have 1 each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R8 csv+axioms run 5 graded by code

I12, with 3 citing works (W33, W34, W35). I09 and I11 have 1 each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 1 graded by code

I12, with 3 citing works: W33, W34, W35. I11 and I09 have 1 each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 2 graded by code

I12, with 3 citing works (W33, W34, W35). The other institutions each have 1.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 3 graded by code

I12, with 3 citing works: W33, W34 and W35. I09 and I11 have 1 each (W32 and W36). The staging works W90–92 are ignored.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 4 graded by code

I12, with 3 citing works (W33, W34, W35). The other citing works are W32 (I09) and W36 (I11), one each.

JSON: {"institution": "I12", "citing_works": 3}

✅ R9 csv+axioms+markdown run 5 graded by code

I12, with 3 citing works (W33, W34, W35).

JSON: {"institution": "I12", "citing_works": 3}