Semantics ladder

How much does an LLM gain from being told what data means?

The experiment

Goals
Method

Credit ratings

Research analytics

The results

All data is invented: the companies, LEIs, ratings, works, people and institutions. Nothing here is any rating agency's or bibliographic database's data or definitions.

Last updated 6 November 2026.

Q44, run 2026-11-06-0855-haiku

← back to the run

Which institutions are part of the Albury University System, at any level below it?

Tests: Local type: an affiliated institute (AFF) is not a unit of its parent.

Expected: institutions = ['I02', 'I03', 'I04', 'I05']

✅ R8 csv+axioms run 1 graded by code

I02, I03, I04 and I05. These are the unit links below I01. I06 is an affiliated institute, so the rules do not treat it as a unit.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R8 csv+axioms run 2 graded by code

I02, I03, I04 and I05. I06 is an affiliated institute, so it is not a unit of I01 under the rules.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R8 csv+axioms run 3 graded by code

I02, I03, I04 and I05. I06 is an affiliated institute (AFF), which does not count as a unit, so it is excluded even though its parent is I02.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R8 csv+axioms run 4 graded by code

I02, I03, I04 and I05. I06 is an affiliated institute (AFF), so it is not a unit under the axioms.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R8 csv+axioms run 5 graded by code

I02, I03, I04 and I05. I06 is an affiliated institute, which the ontology does not treat as a unit, so it is excluded.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R9 csv+axioms+markdown run 1 graded by code

I02, I03, I04 and I05. I06 is an affiliated institute and is not a unit.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R9 csv+axioms+markdown run 2 graded by code

I02, I03, I04 and I05. I06 is an affiliated institute, so its parent link does not make it a unit.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R9 csv+axioms+markdown run 3 graded by code

I02, I03, I04, I05. I06 is affiliated (AFF), so it is not a unit.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R9 csv+axioms+markdown run 4 graded by code

I02, I03, I04, I05. I06 is an affiliated institute, so it is not a unit.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}

✅ R9 csv+axioms+markdown run 5 graded by code

I02, I03, I04 and I05. I06 is an affiliated institute (AFF), so it is not a unit.

JSON: {"institutions": ["I02", "I03", "I04", "I05"]}