Semantics ladder

How much does an LLM gain from being told what data means?

The experiment

Goals
Method

Credit ratings

Research analytics

The results

All data is invented: the companies, LEIs, ratings, works, people and institutions. Nothing here is any rating agency's or bibliographic database's data or definitions.

Last updated 6 November 2026.

Q13, run 2026-11-04-1752-haiku

← back to the run

In 2024, which had more research works, Kesting University (I07) or Rostval Technical University (I09): under full counting, and under fractional counting?

Tests: Counting method: the ranking flips.

Expected: full = I09 fractional = I07

✅ R1 csv run 1 graded by code

Full: I09 (4 works, W21, W27, W28, W29, against 3). Fractional: I07 (2.0, against 1.0 for I09). The result is the same whether or not Kesting's unit I08 is included.

JSON: {"full": "I09", "fractional": "I07"}

✅ R1 csv run 2 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). I09 has 2024 works W21, W27, W28 and W29, but holds only 1 of 4 authors on each (0.25 × 4 = 1.0). Kesting's share is 2.0 across W20, W21 and W22.

JSON: {"full": "I09", "fractional": "I07"}

✅ R1 csv run 3 graded by code

Full: I09 (4 works vs 3). Fractional: I07. In 2024, I09's W21, W27, W28 and W29 have four authors each, so I09 gets 1.0, while I07 gets 2.0.

JSON: {"full": "I09", "fractional": "I07"}

✅ R1 csv run 4 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). I09's four 2024 works (W21, W27, W28, W29) have four authors each, so each share is 0.25.

JSON: {"full": "I09", "fractional": "I07"}

✅ R1 csv run 5 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). Rostval has W21, W27, W28 and W29 in 2024 (full 4, but each with one of four authors, so fractional 1.0). Kesting has W20, W21 and W22 (full 3, fractional 2.0).

JSON: {"full": "I09", "fractional": "I07"}

✅ R2 rdf run 1 graded by code

Full: Rostval (I09). Fractional: Kesting (I07). Rostval has 4 full (W21, W27, W28, W29) but 4 × 1/4 = 1.0 fractional; Kesting has 3 full (W20, W21, W22) and 2.0 fractional.

JSON: {"full": "I09", "fractional": "I07"}

✅ R2 rdf run 2 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). I09's 2024 works (W21, W27, W28, W29) each give it only ¼ of a work.

JSON: {"full": "I09", "fractional": "I07"}

✅ R2 rdf run 3 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). Rostval's 2024 works are W21, W27, W28 and W29, each with four authors, so it gets 1/4 on each.

JSON: {"full": "I09", "fractional": "I07"}

✅ R2 rdf run 4 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). Rostval's 2024 works (W21, W27, W28, W29) each have 4 authors, giving it 0.25 apiece; Kesting's W22 is single-authored.

JSON: {"full": "I09", "fractional": "I07"}

✅ R2 rdf run 5 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). Rostval's 2024 works (W21, W27, W28, W29) each have four authors, one of them from I09, so each gives 0.25.

JSON: {"full": "I09", "fractional": "I07"}

✅ R6 rdf+ontology run 1 graded by code

Full: Rostval (I09), with 4 works (W21, W27, W28, W29) against 3. Fractional: Kesting (I07), with 2.0 against 1.0.

JSON: {"full": "I09", "fractional": "I07"}

✅ R6 rdf+ontology run 2 graded by code

Full: I09. Fractional: I07. Rostval has 4 full works in 2024 (W21, W27, W28, W29), against Kesting's 3. Fractionally Rostval gets 1 and Kesting gets 2.

JSON: {"full": "I09", "fractional": "I07"}

✅ R6 rdf+ontology run 3 graded by code

Full: I09. Fractional: I07. Full counts are I09 4 vs I07 3, and fractional shares are I07 2 vs I09 1.

JSON: {"full": "I09", "fractional": "I07"}

✅ R6 rdf+ontology run 4 graded by code

Full: Rostval (I09), with 4 works (W21, W27, W28, W29) against Kesting's 3. Fractional: Kesting (I07), with 2.0 against Rostval's 1.0.

JSON: {"full": "I09", "fractional": "I07"}

✅ R6 rdf+ontology run 5 graded by code

Full: I09. Fractional: I07. I09 is on four 2024 works (W21, W27, W28, W29), giving full 4 against 3; fractionally I09 gets 1.0 and I07 gets 2.5.

JSON: {"full": "I09", "fractional": "I07"}

✅ R5 csv+ontology run 1 graded by code

Full: Rostval (I09). Fractional: Kesting (I07). Rostval is credited on W21, W27, W28 and W29 (4 vs 3), but its shares are 0.25 each, totalling 1.0, against Kesting's 2.0.

JSON: {"full": "I09", "fractional": "I07"}

✅ R5 csv+ontology run 2 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2 vs 1). Rostval has four 2024 works (W21, W27, W28, W29) but only a quarter share on each, while Kesting's works have fewer authors.

JSON: {"full": "I09", "fractional": "I07"}

✅ R5 csv+ontology run 3 graded by code

Full: I09 (Rostval, 4 vs 3). Fractional: I07 (Kesting, 2.0 vs 1.0). I09's 2024 works are W21, W27, W28 and W29.

JSON: {"full": "I09", "fractional": "I07"}

✅ R5 csv+ontology run 4 graded by code

Full: Rostval (I09), with 4 works (W21, W27, W28, W29) against Kesting's 3. Fractional: Kesting (I07), with 2.0 against Rostval's 1.0 (each of Rostval's 2024 works has four authors).

JSON: {"full": "I09", "fractional": "I07"}

✅ R5 csv+ontology run 5 graded by code

Full: I09 (4 works against 3). Fractional: I07 (2.0 against 1.0 for I09). Rostval's four 2024 works each have four authors, so its fractional total is only 1.0.

JSON: {"full": "I09", "fractional": "I07"}

✅ R4 csv+markdown run 1 graded by code

Full: Rostval (I09), with 4 works (W21, W27, W28, W29) against Kesting's 3. Fractional: Kesting (I07), with 2.0 against Rostval's 1.0.

JSON: {"full": "I09", "fractional": "I07"}

✅ R4 csv+markdown run 2 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). Rostval's 2024 works W21, W27, W28 and W29 each have four authors, so it gets only 0.25 per work.

JSON: {"full": "I09", "fractional": "I07"}

✅ R4 csv+markdown run 3 graded by code

Full: I09 (4 works: W21, W27, W28, W29 vs 3 for I07). Fractional: I07 (2.0 vs 1.0 for I09, since I09's works have four authors each, giving 0.25 per work).

JSON: {"full": "I09", "fractional": "I07"}

✅ R4 csv+markdown run 4 graded by code

Full: I09. Fractional: I07. Rostval has 4 full-count works in 2024 (W21, W27, W28, W29) against Kesting's 3, but its fractional total is 1.0 against Kesting's 2.0.

JSON: {"full": "I09", "fractional": "I07"}

✅ R4 csv+markdown run 5 graded by code

Full: Rostval (I09), with 4 (W21, W27, W28, W29) against Kesting's 3. Fractional: Kesting (I07), with 2.0 against Rostval's 1.0.

JSON: {"full": "I09", "fractional": "I07"}

✅ R3 csv+padding run 1 graded by code

Full: Rostval (I09), with 4 works (W21, W27, W28, W29) against Kesting's 3. Fractional: Kesting (I07), with 2.0 against Rostval's 1.0 (each of Rostval's four 2024 works gives it only a 1/4 share).

JSON: {"full": "I09", "fractional": "I07"}

✅ R3 csv+padding run 2 graded by code

Full: I09 (4 vs 3). Fractional: I07 (2.0 vs 1.0). I09's 2024 works are W21, W27, W28 and W29; for I07 I used the tree, and direct-only gives the same outcome.

JSON: {"full": "I09", "fractional": "I07"}

✅ R3 csv+padding run 3 graded by code

Full: I09 (4 vs 3). Fractional: I07 (1.75 vs 1.0). Rostval's four 2024 works (W21, W27, W28, W29) each have four authors, giving 0.25 each; Kesting's W20, W21 and W22 have fewer authors, so their shares are larger.

JSON: {"full": "I09", "fractional": "I07"}

✅ R3 csv+padding run 4 graded by code

Full: I09 (4 works vs 3). Fractional: I07 (2.0 vs 1.0).

JSON: {"full": "I09", "fractional": "I07"}

✅ R3 csv+padding run 5 graded by code

Full: Rostval (I09), 4 to 3. Fractional: Kesting (I07), 2.0 to 1.0. The 2024 works are W21, W27, W28 and W29 for I09, and W20, W21 and W22 for I07.

JSON: {"full": "I09", "fractional": "I07"}