Q38, run 2026-11-06-0206-sonnet

← back to the run

Which institutions is W71 credited to, at every level?

Tests: Local rule on affiliations, then the hierarchy: the Tessel Laboratory is listed last, so Halloran gets nothing.

Expected: institutions = ['I01', 'I02', 'I03', 'I04']

❌ R1 csv run 1 graded by code

I06, I04, I03, I02 and I01.

JSON: {"institutions": ["I06", "I04", "I03", "I02", "I01"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I06', 'I04', 'I03', 'I02', 'I01']

❌ R1 csv run 2 graded by code

I01, I02, I03, I04 and I06 (I06 → I02 → I01; I04 → I03 → I02 → I01).

JSON: {"institutions": ["I01", "I02", "I03", "I04", "I06"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I01', 'I02', 'I03', 'I04', 'I06']

❌ R1 csv run 3 graded by code

I04, I03, I02 and I01 via the Tessel Laboratory, and I06, I02 and I01 via Halloran. In total: I01, I02, I03, I04, I06.

JSON: {"institutions": ["I01", "I02", "I03", "I04", "I06"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I01', 'I02', 'I03', 'I04', 'I06']

❌ R1 csv run 4 graded by code

I06 and I04, plus their ancestors I03, I02 and I01.

JSON: {"institutions": ["I06", "I04", "I03", "I02", "I01"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I06', 'I04', 'I03', 'I02', 'I01']

❌ R1 csv run 5 graded by code

I06 and I04, with their ancestors I03, I02 and I01. The full set is I01, I02, I03, I04, I06.

JSON: {"institutions": ["I01", "I02", "I03", "I04", "I06"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I01', 'I02', 'I03', 'I04', 'I06']

❌ R3 csv+padding run 1 graded by code

I06, I04, I03, I02 and I01.

JSON: {"institutions": ["I06", "I04", "I03", "I02", "I01"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I06', 'I04', 'I03', 'I02', 'I01']

❌ R3 csv+padding run 2 graded by code

I06 and I04, plus their parents I03, I02 and I01.

JSON: {"institutions": ["I06", "I04", "I03", "I02", "I01"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I06', 'I04', 'I03', 'I02', 'I01']

❌ R3 csv+padding run 3 graded by code

I04, I03, I06, I02 and I01, from the two affiliations of W71 and their parent chains.

JSON: {"institutions": ["I04", "I03", "I06", "I02", "I01"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I04', 'I03', 'I06', 'I02', 'I01']

❌ R3 csv+padding run 4 graded by code

I06, I04, I03, I02 and I01 (W71 lists I06 first, then I04).

JSON: {"institutions": ["I06", "I04", "I03", "I02", "I01"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I06', 'I04', 'I03', 'I02', 'I01']

❌ R3 csv+padding run 5 graded by code

I01, I02, I03, I04 and I06. I06 rolls up to I02 and I01; I04 rolls up through I03 to I02 and I01.

JSON: {"institutions": ["I01", "I02", "I03", "I04", "I06"]}

Why: institutions: expected ['I01', 'I02', 'I03', 'I04'], got ['I01', 'I02', 'I03', 'I04', 'I06']

✅ R4 csv+markdown run 1 graded by code

I04, I03, I02 and I01. For W71 the last-listed affiliation is I04, which sits under I03, I02 and I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R4 csv+markdown run 2 graded by code

I04, I03, I02 and I01. A08 lists I06 first and I04 last, so W71 is credited to I04 and everything above it.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R4 csv+markdown run 3 graded by code

I04, I03, I02 and I01. A08 lists I06 first and I04 second, so W71 is credited to I04 and everything above it.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R4 csv+markdown run 4 graded by code

I04, I03, I02 and I01. W71's credited affiliation is I04, which sits under I03, I02 and I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R4 csv+markdown run 5 graded by code

I04, I03, I02 and I01. The author lists I06 first and I04 last, so the work is credited to I04 and everything above it.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R5 csv+ontology run 1 graded by code

I04, I03, I02 and I01. For W71, A08 listed I06 first and I04 last, so credit goes to I04 and then up the chain.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R5 csv+ontology run 2 graded by code

I04, I03, I02, I01. A08's last-listed affiliation on W71 is I04.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R5 csv+ontology run 3 graded by code

I04, I03, I02, I01. On W71 the last-listed affiliation is I04, which rolls up through I03 and I02 to I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R5 csv+ontology run 4 graded by code

I04, I03, I02 and I01. For W71 the last-listed affiliation is I04, and I06 is ignored.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R5 csv+ontology run 5 graded by code

I04, I03, I02 and I01. W71's last-listed affiliation is I04, and the credit rolls up through I03 and I02 to I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R10 executed run 1 graded by code

I04, I03, I02 and I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R10 executed run 2 graded by code

I04, I03, I02 and I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R10 executed run 3 graded by code

I04, I03, I02 and I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R10 executed run 4 graded by code

I04, I03, I02 and I01. W71's credit is A08 at I04, which rolls up through I03 and I02 to I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}

✅ R10 executed run 5 graded by code

I04, I03, I02 and I01. W71 credits A08 to I04, which rolls up through I03 and I02 to I01.

JSON: {"institutions": ["I04", "I03", "I02", "I01"]}