Q48, run 2026-11-05-1234-sonnet

← back to the run

Why is W70 not one of the University of Albury's research works, though its author lists the Tessel Laboratory?

Tests: Local rules: the last-listed affiliation, and affiliated institutes.

Expected: Tomasz Wierzba lists the Tessel Laboratory first and the Halloran Institute second; the house rule credits the last-listed one, Halloran. Halloran is an affiliated institute (AFF): its parent link is an agreement, so its works are not the University of Albury's.

❌ R1 csv run 1 graded by llm

The premise is wrong. I04 → I03 → I02, so W70 does count among the University of Albury's works (included in Q9).

Why: Rejects the premise and says W70 counts for Albury; checklist says only the last-listed affiliation counts, crediting the Halloran Institute, which doesn't roll up to Albury

❌ R1 csv run 2 graded by llm

The premise is wrong: W70 is one of the University of Albury's research works. I04 sits under I03, which sits under I02, and W70 is an article, so it counts. Only the first-listed affiliation (I04) gives it directly to the Tessel Laboratory.

Why: Rejects the premise and says W70 counts for Albury; the key says only the last-listed affiliation counts, crediting Halloran Institute, an affiliated institute that doesn't roll up to Albury.

❌ R1 csv run 3 graded by llm

The premise is wrong. I04 is under I03, which is under I02, so W70 is counted as a University of Albury work (it is in my 12 for Q9).

Why: Rejects the premise and says W70 counts for Albury; the key says only the last-listed affiliation counts, so W70 goes to Halloran Institute, an affiliated institute that doesn't roll up to Albury.

❌ R1 csv run 4 graded by llm

The premise doesn't hold. W70 is credited to I04, which sits under I03 and I02, so it is one of the University of Albury's works (it is counted in the 12 in Q9).

Why: Rejects the premise; the key says only the last-listed affiliation counts, so W70 is credited to the Halloran Institute, an affiliated institute that does not roll up to the University of Albury.

❌ R1 csv run 5 graded by llm

The premise is wrong, because W70 is one of Albury's works. I04 sits under I03, then I02, so its credit rolls up to the University of Albury. I excluded W70 from nothing, and it is among the 12 in Q9.

Why: Denies the premise and says W70 counts for Albury; the checklist says only the last-listed affiliation counts (Halloran Institute), which is an affiliated institute and does not roll up to Albury.

❌ R2 rdf run 1 graded by llm

The premise is false. I04 → I03 → I02, so W70 is counted for the University of Albury (see Q9).

Why: Rejects the premise and says W70 counts for Albury; checklist says only the last-listed affiliation counts (Halloran) and Halloran is affiliated, not a unit, so it doesn't roll up.

❌ R2 rdf run 2 graded by llm

The premise is wrong. W70 is counted for the University of Albury, because I04 → I03 → I02 and A08's second affiliation I06 also sits under I02. It is counted once.

Why: Rejects the premise and says W70 counts for Albury; the key says only the last-listed affiliation (Halloran Institute) counts and it is an affiliated institute, not a unit, so it doesn't roll up.

❌ R2 rdf run 3 graded by llm

The premise is wrong: W70 is one of the University of Albury's works. I04 rolls up through I03 to I02, and I06 is a child of I02, so W70 counts for I02 once.

Why: Contradicts the key: W70 is not the University of Albury's work, since only the last-listed affiliation (Halloran Institute, an affiliated institute not a unit) counts.

❌ R2 rdf run 4 graded by llm

The premise is wrong. I04's parent is I03, whose parent is I02, so W70 is one of the University of Albury's works. It is counted in the 12 for Q9.

Why: Rejects the premise; the key says only the last-listed affiliation counts, crediting W70 to the Halloran Institute, an affiliated institute that doesn't roll up to the University of Albury.

❌ R2 rdf run 5 graded by llm

The premise is wrong: W70 is one of the University of Albury's works. Both I04 and I06 are units of I02, so W70 counts once for I02.

Why: Claims W70 is a University of Albury work; the key says only the last-listed affiliation counts, so it is credited to the Halloran Institute, an affiliated institute that does not roll up to Albury.

✅ R6 rdf+ontology run 1 graded by llm

A08 listed I04 (sequence 1) and I06 (sequence 2), and the house rule credits only the last, I06. I06 is an affiliated institute, so its parent link to I02 is not followed.

✅ R6 rdf+ontology run 2 graded by llm

A08 listed I04 (seq 1) and I06 (seq 2), and the house rule credits only the last-listed one, I06. I06 is an affiliated institute (AFF), so its parent link to I02 is not followed.

✅ R6 rdf+ontology run 3 graded by llm

A08 listed I04 (seq 1) and I06 (seq 2) on W70. Only the last-listed affiliation counts, so W70 is credited to I06. I06 is an affiliated institute, so its parent link to I02 is not followed.

✅ R6 rdf+ontology run 4 graded by llm

A08 listed I04 first and I06 last, so W70 is credited only to I06. I06 is an affiliated institute, so its parent link to I02 is not membership and the credit does not pass up to Albury.

🟡 R6 rdf+ontology run 5 graded by llm

A08 listed I04 (seq 1) and I06 (seq 2) on W70, and only the last-listed affiliation counts. I06 is an affiliated institute, so its parent link to I02 is not followed.

Why: Correctly says only the last-listed affiliation counts (I06) and that I06 is an affiliated institute whose parent link isn't followed, but doesn't name it as the Halloran Institute or state that W70 is credited to it; IDs are used instead.

✅ R5 csv+ontology run 1 graded by llm

W70's author listed I04 (sequence 1) and I06 (sequence 2), and the house rule credits only the highest sequence, I06. I06 is an affiliated institute, so its parent link to I02 does not make it a unit and the credit never reaches the University of Albury.

✅ R5 csv+ontology run 2 graded by llm

A08 listed I04 (seq 1) and I06 (seq 2) on W70, and only the last-listed affiliation counts, so the work is credited to I06 alone. I06 is an affiliated institute, so its parent link to I02 is not followed.

✅ R5 csv+ontology run 3 graded by llm

A08 listed I04 first and I06 last on W70, and the house rule credits only the last, I06. I06 is an affiliated institute, so its parent link to I02 is not followed.

✅ R5 csv+ontology run 4 graded by llm

A08 listed I04 first and I06 last on W70, so the work is credited only to I06. I06 is an affiliated institute (AFF), so its link to I02 is ignored and the credit never reaches the University of Albury.

✅ R5 csv+ontology run 5 graded by llm

A08 listed I04 (seq 1) and I06 (seq 2) on W70, and the house rule credits only the highest sequence, which is I06. I06 is an affiliated institute, so its parent link to I02 is not followed.

✅ R4 csv+markdown run 1 graded by llm

A08 listed I04 first and I06 second, and only the last-listed affiliation is credited, so W70 is credited to I06. I06 is an affiliated institute (AFF), so its works are not credited to its parent I02.

✅ R4 csv+markdown run 2 graded by llm

A08 listed I04 first and I06 second on W70. The house rule credits only the highest affiliation sequence, which is I06, and I06 is an affiliated institute whose parent link is ignored.

✅ R4 csv+markdown run 3 graded by llm

W70's author lists I04 (sequence 1) and I06 (sequence 2), and only the last-listed one is credited. I06 is an affiliated institute, so the work does not roll up to I02.

✅ R4 csv+markdown run 4 graded by llm

A08 listed I04 first and I06 last, so W70 is credited only to I06. I06 is an affiliated institute, so its link to I02 does not make it part of the University of Albury.

✅ R4 csv+markdown run 5 graded by llm

A08 listed I04 first and I06 second on W70, so only I06 is credited. I06 is an affiliated institute, so its parent link to I02 does not make it a unit and the work is not credited upward.

❌ R3 csv+padding run 1 graded by llm

The premise is wrong. W70's author A08 lists I04, which sits under I03, I02 and I01, so W70 is credited to the University of Albury.

Why: Rejects the premise and credits W70 to the University of Albury, contradicting the checklist: only the last-listed affiliation counts, so W70 goes to the Halloran Institute, which is an affiliated institute and doesn't roll up.

❌ R3 csv+padding run 2 graded by llm

The premise is wrong. W70's seq-1 affiliation is I04, which rolls up through I03 to I02, so W70 does count for the University of Albury.

Why: Rejects the premise and says W70 counts for Albury, whereas the checklist says only the last-listed affiliation counts, crediting W70 to the Halloran Institute, which is an affiliated institute and does not roll up to Albury.

❌ R3 csv+padding run 3 graded by llm

It is one of them. I04 sits under I03, which sits under I02, so W70 counts for the University of Albury via the Tessel Laboratory, and also via I06.

Why: Claims W70 does count for Albury; misses that only the last-listed affiliation counts (Halloran Institute), an affiliated institute that doesn't roll up.

❌ R3 csv+padding run 4 graded by llm

The premise is wrong. W70 is a University of Albury work, because I04 sits under I03, which sits under I02, and its other listed affiliation, I06, is also under I02.

I excluded EDT, ERR, PTX and SUP types as non-research. I counted W02/W03 and W31/W42 once each. I also excluded the STG-sourced records W90–W92, because I read STG as staging, which the data does not define. Counting them would change Q11 (to 9), Q16, Q25, Q31, Q32 and Q34.

Why: Claims W70 is an Albury work; the key says only the last-listed affiliation counts, so it goes to the Halloran Institute, an affiliated institute that doesn't roll up to Albury.

❌ R3 csv+padding run 5 graded by llm

The premise is wrong. I04 → I03 → I02, so W70 is one of the University of Albury's works through the Tessel Laboratory, and it is included in the count of 12 above.

Why: Rejects the premise and says W70 counts toward Albury via Tessel; the key says only the last-listed affiliation counts, crediting W70 to Halloran Institute, an affiliated institute that doesn't roll up to Albury.

🟡 R10 executed run 1 graded by llm

W70's only credit is to I06, which is an affiliate and does not count towards I02 in institution_units. The Tessel (I04) credits are on W71 and W72, not W70.

Why: Says W70 is credited only to I06, an affiliate that doesn't roll up to I02, but omits that only the last-listed affiliation counts.

🟡 R10 executed run 2 graded by llm

I06 is an affiliate (AFF), not a unit of I02. institution_units lists it as counting only towards itself, so W70 does not roll up to the University of Albury.

Why: Correctly says the Halloran Institute (I06) is an affiliate, not a unit, so W70 doesn't roll up to Albury; omits that only the last-listed affiliation counts, which is why W70 is credited to Halloran.

🟡 R10 executed run 3 graded by llm

W70 is credited to I06 (Halloran), not to the Tessel Laboratory. I06 does not count towards I02 in institution_units. Tessel (I04) is credited on W71 and W72, not W70.

Why: Says W70 is credited to Halloran (I06) and that I06 doesn't roll up to I02, but doesn't state the last-listed-affiliation rule or that Halloran is an affiliated institute rather than a unit.

🟡 R10 executed run 4 graded by llm

W70's credit for A08 is to I06 (Halloran Institute), not Tessel (I04). I06 is an affiliate and counts only towards itself in institution_units, so it does not roll up to I02. A08's I04 credits are on W71 and W72 only.

Why: Says W70 is credited to the Halloran Institute and that it is an affiliate that does not roll up, but never states that only the last-listed affiliation counts.

🟡 R10 executed run 5 graded by llm

W70's only credit is to I06, the Halloran Institute, not to the Tessel Laboratory (I04, credited on W71 and W72). institution_units counts I06 only towards itself, so its works don't roll up to I02.

Why: Credits W70 to the Halloran Institute and says it doesn't roll up to the University of Albury, but omits the rule that only the last-listed affiliation counts and doesn't say the institute is affiliated rather than a unit.