Q44, run 2026-11-06-0206-haiku

← back to the run

Which institutions are part of the Albury University System, at any level below it?

Tests: Local type: an affiliated institute (AFF) is not a unit of its parent.

Expected: institutions = ['I02', 'I03', 'I04', 'I05', 'I19', 'I20']

❌ R1 csv run 1 graded by code

I02, I03, I04, I05, I06, I19, I20, I21.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R1 csv run 2 graded by code

I02, I03, I04, I05, I06, I19, I20 and I21.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R1 csv run 3 graded by code

I02, I03, I04, I05, I06, I19, I20, I21.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R1 csv run 4 graded by code

I02, I03, I04, I05, I06, I19, I20 and I21. These are all the descendants of I01 in parent_id.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R1 csv run 5 graded by code

I02, I03, I04, I05, I06, I19, I20, I21. These are every unit below I01 in the parent chain.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R3 csv+padding run 1 graded by code

I02, I03, I04, I05, I06, I19, I20 and I21.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R3 csv+padding run 2 graded by code

I02, I03, I04, I05, I06, I19, I20, I21.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R3 csv+padding run 3 graded by code

I02, I03, I04, I05, I06, I19, I20, I21.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R3 csv+padding run 4 graded by code

I02, I03, I04, I05, I06, I19, I20, I21.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R3 csv+padding run 5 graded by code

I02, I03, I04, I05, I06, I19, I20, I21.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R4 csv+markdown run 1 graded by code

(no answer found)

Why: no structured answer

❌ R4 csv+markdown run 2 graded by code

(no answer found)

Why: no structured answer

✅ R4 csv+markdown run 3 graded by code

I02, I03, I04, I05, I19 and I20. I06 and I21 are affiliated institutes, so they are not units.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I19", "I20"]}

❌ R4 csv+markdown run 4 graded by code

(no answer found)

Why: no structured answer

✅ R4 csv+markdown run 5 graded by code

I02, I03, I04, I05, I19, I20. I06 and I21 are affiliated institutes, not units.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I19", "I20"]}

❌ R5 csv+ontology run 1 graded by code

(no answer found)

Why: no structured answer

✅ R5 csv+ontology run 2 graded by code

I02, I03, I04, I05, I19, I20. The affiliated institutes I06 and I21 are not units, so they are excluded.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I19", "I20"]}

✅ R5 csv+ontology run 3 graded by code

I02, I03, I04, I05, I19, I20. I06 and I21 are affiliated institutes, so excluded.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I19", "I20"]}

✅ R5 csv+ontology run 4 graded by code

I02, I03, I04, I05, I19, I20. These are the units below I01. I06 and I21 are affiliated institutes (AFF), so they are not units.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I19", "I20"]}

✅ R5 csv+ontology run 5 graded by code

I02, I03, I04, I05, I19, I20. I06 and I21 are affiliated institutes and do not count as units.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I19", "I20"]}

✅ R10 executed run 1 graded by code

I02, I03, I04, I05, I19 and I20, per institution_units. I06 and I21 have a parent in the Albury tree in institutions.csv, but institution_units does not count them towards I01.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I19", "I20"]}

✅ R10 executed run 2 graded by code

I02, I03, I04, I05, I19, I20. These are the institutions whose works count toward I01 below the top level.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I19", "I20"]}

❌ R10 executed run 3 graded by code

I02, I03, I04, I05, I06, I19, I20, I21. This follows parent_id. Note that institution_units only rolls up I02–I05, I19 and I20 into I01 for works counting, so I06 and I21 are affiliated but their works do not count towards I01.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R10 executed run 4 graded by code

I02, I03, I04, I05, I06, I19, I20, I21.
This follows the parent_id hierarchy. The institution_units table omits I06 and I21, which conflicts with parent_id, so the two sources disagree.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']

❌ R10 executed run 5 graded by code

I02, I03, I04, I05, I19 and I20 by institution_units. I06 and I21 also sit under the system by parent_id, but institution_units does not roll them up, so the list depends on which definition you take.

JSON: {"institutions": ["I02", "I03", "I04", "I05", "I06", "I19", "I20", "I21"]}

Why: institutions: expected ['I02', 'I03', 'I04', 'I05', 'I19', 'I20'], got ['I02', 'I03', 'I04', 'I05', 'I06', 'I19', 'I20', 'I21']