Semantics ladder

How much does an LLM gain from being told what data means?

The experiment

Goals
Method

Credit ratings

Research analytics

The results

All data is invented: the companies, LEIs, ratings, works, people and institutions. Nothing here is any rating agency's or bibliographic database's data or definitions.

Last updated 6 November 2026.

Q29, run 2026-11-04-1752-haiku

← back to the run

Of the works citing "Drift-aware mapping with low-cost sonar" (either version), how many are self-citations at the level of authors, of the University of Albury (units included), and of countries?

Tests: Self-citation depends on the level; W12 is Mara Ellison under another record, W13 her co-author; W16 is the other M. Ellison.

Expected: author = 2 institution = 2 country = 4

✅ R1 csv run 1 graded by code

Author level 2 (W12 and W13). Institution level (I02 with units) 2 (W12 and W13). Country level (GB) 4 (W12, W13, W14 and W15). Cited authors include A02 and A03, who are the same person as A01 (the author of W12).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R1 csv run 2 graded by code

Author 2, institution 2, country 4. Author self-citations are W12 (A01 is the same person as A03) and W13 (A07). Albury-subtree self-citations are W12 and W13. Country self-citations (GB) are W12, W13, W14 and W15.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R1 csv run 3 graded by code

Author 2, institution 2, country 4. Author-level self-citations are W12 (Mara) and W13 (Priya); A04 has a different ORCID, so W16 is not one. Albury-level are W12 and W13, and GB-level are W12, W13, W14 and W15.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R1 csv run 4 graded by code

Author 2, institution 2, country 4. Citing works sharing an author (resolving A01/A02/A03 to Mara Ellison) are W12 and W13. Sharing an Albury institution: W12 and W13. Sharing a GB country: W12, W13, W14 and W15.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R1 csv run 5 graded by code

Author level 2 (W12, W13), institution level 2 (W12, W13), country level 4 (W12, W13, W14, W15). The GB co-authors on W14 and W15 drive the country count. A04 "M. Ellison" is a different person, so W16 is not a self-citation.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R2 rdf run 1 graded by code

Author 2 (W12 via A01/A03 sharing an ORCID, and W13 via A07), institution 2 (W12, W13), country 4 (W12, W13, W14, W15).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R2 rdf run 2 graded by code

Author-level 2 (W12 via Mara Ellison, W13 via A07), Albury units 2 (W12, W13), country 4 (W12, W13, W14, W15 are all GB). At record level, the author count would be 1.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R2 rdf run 3 graded by code

Author 2 (W12 and W13), institution 2 (W12 and W13), country 4 (W12, W13, W14 and W15). W12 and W13 share an author or institution, W14 and W15 share the GB country, and W16 to W18 share none.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R2 rdf run 4 graded by code

Author-level 2 (W12 and W13), Albury-level 2 (W12 and W13), country-level 4 (W12, W13, W14 and W15). W12 counts at the author level because A01, A02 and A03 are the same ORCID.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R2 rdf run 5 graded by code

Author: 2. W12 (A01 shares ORCID with the cited A02/A03) and W13 (A07) are self-citations. Institution: 2 (W12, W13, both Albury). Country: 4 (W12, W13, W14, W15, all with GB authors). The author count is 1 if matching is by author record ID only.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R6 rdf+ontology run 1 graded by code

Author 2 (W12 and W13 share Mara Ellison or Priya Ramanathan with W03). Institution 2 (W12 and W13 are credited to the University of Albury group). Country 4 (W12, W13, W14 and W15 are GB-credited).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R6 rdf+ontology run 2 graded by code

Author 2, institution 2, country 4. The self-citers are W12 and W13 for authors (W16's A04 is a different ORCID). Albury self-citers are W12 and W13. GB self-citers are W12, W13, W14 and W15.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R6 rdf+ontology run 3 graded by code

Author 2 (W12, W13), institution 2 (W12, W13), country 4 (W12, W13, W14, W15). A04 is a different M. Ellison, so W16 is not an author self-citation.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R6 rdf+ontology run 4 graded by code

Author: 2 (W12 and W13 share an author with W02/W03). Institution: 2 (W12 and W13 are in the University of Albury tree). Country: 4 (W12, W13, W14 and W15 are all GB-credited).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R6 rdf+ontology run 5 graded by code

Author 2, institution 2, country 4. The citers sharing an author with W02/W03 are W12 and W13; sharing Albury (units included) are W12 and W13; sharing GB are W12, W13, W14 and W15.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R5 csv+ontology run 1 graded by code

Author 2, institution 2, country 4. The self-citers are W12 and W13 (author and institution), and W12, W13, W14 and W15 (country, which covers GB); W16 is by a different M. Ellison ORCID.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R5 csv+ontology run 2 graded by code

Author 2, institution 2, country 4. W12 and W13 share an author (Mara, Priya) and Albury (I02/I03). W12, W13, W14 and W15 are GB-credited like the cited works. A04 is a different person, so W16 is not an author self-citation.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R5 csv+ontology run 3 graded by code

Author level 2 (W12, W13); institution level 2 (W12, W13); country level 4 (W12, W13, W14, W15).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R5 csv+ontology run 4 graded by code

Author 2, institution 2, country 4. Author self-citations are W12 and W13. Albury self-citations are W12 and W13 (W14 is credited to I05, not in the I02 tree). Country self-citations, GB, are W12, W13, W14 and W15.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R5 csv+ontology run 5 graded by code

Author 2, institution 2, country 4. The citers sharing Mara Ellison or Priya Ramanathan with the cited works are W12 and W13. The citers in GB are W12, W13, W14 and W15. W16 is by a different M. Ellison (different ORCID).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R4 csv+markdown run 1 graded by code

Author 2 (W12, W13), institution 2 (W12, W13), country 4 (W12, W13, W14, W15).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R4 csv+markdown run 2 graded by code

Author 2, institution 2, country 4. Author-level self-citations are W12 and W13 (shared Mara Ellison or Priya Ramanathan). Institution-level, the same two share I02 or I03. Country-level adds W14 and W15, which share GB.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R4 csv+markdown run 3 graded by code

Author self-citations: 2 (W12, W13). Institution self-citations at University of Albury level: 2 (W12, W13). Country self-citations (GB): 4 (W12, W13, W14, W15).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R4 csv+markdown run 4 graded by code

Author 2, institution 2, country 4. Author self-citations are W12 and W13 (shared with Mara Ellison / Priya Ramanathan). Shared Albury-level institutions are the same two works. Shared GB-country citers are W12, W13, W14 and W15.

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R4 csv+markdown run 5 graded by code

Author 2 (W12, W13), institution 2 (W12, W13), country 4 (W12, W13, W14, W15).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R3 csv+padding run 1 graded by code

Author: 2 (W12 and W13 share an author with W03). Institution: 2 (W12 and W13 are at Albury-tree institutions). Country: 4 (W12, W13, W14 and W15 are in GB, like the cited works).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R3 csv+padding run 2 graded by code

Author-level 2 (W12 and W13 share Mara or Priya), institution-level 2 (W12 and W13 are at I02/I03), country-level 4 (W12, W13, W14 and W15 have GB authors).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R3 csv+padding run 3 graded by code

Author 2, institution 2, country 4. The citing works are W12–W18. Author self-citations are W12 and W13 (Mara and Priya). Albury self-citations are W12 and W13 (I02 and I03). Country self-citations are W12–W15 (GB).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R3 csv+padding run 4 graded by code

Author level: 2 (W12, whose A01 is the same person as A02/A03 by ORCID, and W13 by A07). Institution level: 2 (W12 and W13, both in the I02 subtree). Country level: 4 (W12, W13, W14, W15, all with GB authors).

JSON: {"author": 2, "institution": 2, "country": 4}

✅ R3 csv+padding run 5 graded by code

Author 2, institution 2, country 4. Author self-citations are W12 (A01 is the same person as A03) and W13 (A07). The institution-level count is W12 and W13, and the country-level count is W12, W13, W14 and W15 (GB).

JSON: {"author": 2, "institution": 2, "country": 4}