Semantics ladder

How much does an LLM gain from being told what data means?

The experiment

Goals
Method

Credit ratings

Research analytics

The results

All data is invented: the companies, LEIs, ratings, works, people and institutions. Nothing here is any rating agency's or bibliographic database's data or definitions.

Last updated 6 November 2026.

Q43, run 2026-11-06-0855-sonnet

← back to the run

Is W07 research output?

Tests: Local code: an article in a delisted source.

Expected: research = no

✅ R8 csv+axioms run 1 graded by code

No. S04 is delisted, so W07 is excluded.

JSON: {"research": "no"}

✅ R8 csv+axioms run 2 graded by code

No. W07 is in a delisted source.

JSON: {"research": "no"}

✅ R8 csv+axioms run 3 graded by code

No. W07 is in S04, which is delisted.

JSON: {"research": "no"}

✅ R8 csv+axioms run 4 graded by code

No. W07 is in S04, a delisted source.

JSON: {"research": "no"}

✅ R8 csv+axioms run 5 graded by code

No. W07 is in S04, a delisted source.

JSON: {"research": "no"}

✅ R9 csv+axioms+markdown run 1 graded by code

No. W07 appeared in the delisted source S04.

JSON: {"research": "no"}

✅ R9 csv+axioms+markdown run 2 graded by code

No. W07 appeared in the delisted S04.

JSON: {"research": "no"}

✅ R9 csv+axioms+markdown run 3 graded by code

No. W07 appeared in S04, which is delisted.

JSON: {"research": "no"}

✅ R9 csv+axioms+markdown run 4 graded by code

No. W07 appeared in S04, a delisted source.

JSON: {"research": "no"}

✅ R9 csv+axioms+markdown run 5 graded by code

No. W07 appeared in S04, a delisted source.

JSON: {"research": "no"}