Q48, run 2026-11-06-0206-sonnet

← back to the run

Why is W70 not one of the University of Albury's research works, though its author lists the Tessel Laboratory?

Tests: Local rules: the last-listed affiliation, and affiliated institutes.

Expected: Tomasz Wierzba lists the Tessel Laboratory first and the Halloran Institute second; the house rule credits the last-listed one, Halloran. Halloran is an affiliated institute (AFF): its parent link is an agreement, so its works are not the University of Albury's.

❌ R1 csv run 1 graded by llm

The premise is wrong. W70 is an article from 2023 whose author is listed at I04, which sits under I03 and I02, so it is a University of Albury work (also via I06). It is counted in Q9's 116.

Why: Rejects the premise and says W70 is a University of Albury work, contradicting the checklist that it is credited to the Halloran Institute and does not roll up.

❌ R1 csv run 2 graded by llm

The premise doesn't hold. W70 is an ART, and its author lists I04 and I06. I04 sits under I03 and then I02, and I06 sits under I02, so W70 does count towards the University of Albury when units are included. It would be missed only if credit were taken at the directly listed institution.

Why: Answer contradicts the key: it claims W70 counts toward Albury, whereas the checklist says only the last-listed affiliation counts (Halloran Institute, an affiliated institute, not a unit), so it doesn't roll up.

❌ R1 csv run 3 graded by llm

The premise is wrong. W70 is a University of Albury work: I04 → I03 → I02, and its other affiliation I06 is also under I02. The two affiliations count once.

Why: Rejects the premise; the key says only the last-listed affiliation counts, so W70 goes to the Halloran Institute, an affiliated institute that doesn't roll up to Albury. Answer meets neither point.

❌ R1 csv run 4 graded by llm

It is one of the University of Albury's works. A08's first-listed affiliation on W70 is I04, and I04→I03→I02, so the work counts; the premise is false.

Why: Answer rejects the premise and says W70 counts via first-listed affiliation; checklist says only the last-listed affiliation counts, crediting the Halloran Institute, an affiliated institute not rolling up to Albury.

❌ R1 csv run 5 graded by llm

The premise is false. W70 is one of the University of Albury's works, because I04 rolls up through I03 to I02, and I06 also sits under I02.

Why: Asserts the premise is false and that W70 counts for Albury; the key says only the last-listed affiliation counts, crediting Halloran Institute, an affiliated institute that doesn't roll up to Albury.

❌ R3 csv+padding run 1 graded by llm

The premise does not hold in this data. W70 is credited to the University of Albury (I02): A08 lists I04 → I03 → I02, and I06 also sits under I02, so the work counts once for I02.

Why: Contradicts the key: W70 is credited to the Halloran Institute (last-listed affiliation), an affiliated institute that doesn't roll up to Albury; the answer claims it counts for Albury.

❌ R3 csv+padding run 2 graded by llm

The premise is wrong: W70 is one of the University of Albury's works. Its author A08 lists I04, and I04 → I03 → I02 rolls up to the University of Albury. The second affiliation I06 also sits under I02.

Why: Answer says W70 does belong to Albury via I04→I03→I02, contradicting the key (only the last-listed affiliation counts, so it goes to the Halloran Institute, an affiliated institute that doesn't roll up).

❌ R3 csv+padding run 3 graded by llm

The premise is wrong: W70 is one of the University of Albury's research works. Its author A08 lists I04 (Tessel) and I06 (Halloran). I04 rolls up to I03 and then I02, and I06 is a direct child of I02. W70 is an ART, so it is counted once, and it is in the 119 for Q9.

Why: Answer rejects the premise and says W70 counts for Albury; it does not say only the last-listed affiliation counts or that Halloran is an affiliated institute not rolling up.

❌ R3 csv+padding run 4 graded by llm

The premise is wrong: W70 is one of the University of Albury's works. Its author A08 lists I04 (Tessel), which rolls up through I03 to I02, and I06 (Halloran), which has I02 as its parent.

Why: Answer claims W70 is a University of Albury work, contradicting the checklist: only the last-listed affiliation counts (Halloran) and Halloran is an affiliated institute that does not roll up to Albury.

❌ R3 csv+padding run 5 graded by llm

The premise is wrong. W70's affiliations (I04 and I06) both roll up to I02, so W70 is one of the University of Albury's research works and is counted once. Two affiliations in the same university do not double-count.

Why: Rejects the premise; the key says only the last-listed affiliation counts, crediting W70 to Halloran Institute, an affiliated institute that doesn't roll up to Albury.

✅ R4 csv+markdown run 1 graded by llm

Author A08 lists I04 (sequence 1) and I06 (sequence 2), and the house rule credits only the highest sequence, so W70 goes to I06. I06 is an affiliated institute, so its link to I02 is not membership and nothing rolls up to the University of Albury.

✅ R4 csv+markdown run 2 graded by llm

A08 lists I04 first and I06 last, and the house rule credits only the last-listed affiliation, so W70 is credited to I06. I06 is an affiliated institute, whose parent link to I02 is not membership, so W70 is not credited to I02.

✅ R4 csv+markdown run 3 graded by llm

W70's author lists I04 (sequence 1) and I06 (sequence 2). The house rule credits only the last-listed one, I06, and I06 is an affiliated institute, not a unit of I02.

✅ R4 csv+markdown run 4 graded by llm

A08 listed I04 first and I06 last, and the house rule credits only the last-listed affiliation. I06 is an affiliated institute, not a unit of the University of Albury.

✅ R4 csv+markdown run 5 graded by llm

W70's author lists I04 (sequence 1) and I06 (sequence 2). The house rule credits only the last-listed affiliation, which is I06. I06 is an affiliated institute, not a unit of I02, so the work is not credited up to the University of Albury.

✅ R5 csv+ontology run 1 graded by llm

A08 listed I04 (seq 1) and then I06 (seq 2), and the house rule credits only the last-listed affiliation. I06 is an affiliated institute, so its parent link to I02 does not make W70 an Albury work.

✅ R5 csv+ontology run 2 graded by llm

A08 listed I04 (seq 1) and I06 (seq 2) on W70, so only I06, the last-listed affiliation, is credited. I06 is an affiliated institute, so its parent link to I02 is not followed.

✅ R5 csv+ontology run 3 graded by llm

A08 lists I04 first and I06 last, and the last-listed affiliation is the only one credited. I06 is an affiliated institute, so the work does not roll up to I02.

✅ R5 csv+ontology run 4 graded by llm

A08 listed I04 (sequence 1) and I06 (sequence 2) on W70, and only the last-listed affiliation is credited. That is I06, an affiliated institute, so the work does not roll up to I02.

✅ R5 csv+ontology run 5 graded by llm

A08 listed I04 (Tessel) first and I06 (Halloran) last on W70. Under the house rule only the last-listed affiliation is credited, so W70 goes to I06, which is an affiliated institute, not a unit of the University of Albury.

🟡 R10 executed run 1 graded by llm

W70's author A08 is credited to I06 (Halloran Institute, an affiliate), not to the Tessel Laboratory. I06 is not in the University of Albury's unit roll-up in institution_units, despite its parent_id of I02.

Why: Says W70 is credited to I06 (Halloran) and that I06 is an affiliate outside the unit roll-up, but never states the rule that only the last-listed affiliation counts.

🟡 R10 executed run 2 graded by llm

W70's only credit (A08) goes to I06, the Halloran Institute. In institution_units I06 counts only towards itself, not I02, so the work doesn't roll up into the University of Albury. The Tessel Laboratory (I04) is not the credited institution.

Why: Credits W70 to the Halloran Institute and says I06 doesn't roll up to the University of Albury, but never states the rule that only the last-listed affiliation counts.

🟡 R10 executed run 3 graded by llm

W70 is credited to I06, an affiliated institute, not to the Tessel Laboratory (I04). institution_units lists I06 as counting only towards itself, so its works do not roll up to the University of Albury.

Why: Says W70 is credited to I06, an affiliated institute that does not roll up, but omits that only the last-listed affiliation counts

🟡 R10 executed run 4 graded by llm

W70's only credit is A08 at the Halloran Institute (I06), not at Tessel (I04). I06 is an affiliate and institution_units does not count it towards the University of Albury.

Why: Says the credit is the Halloran Institute and that it is an affiliate not counted toward Albury, but doesn't state that only the last-listed affiliation counts.

🟡 R10 executed run 5 graded by llm

W70 credits A08 to I06, the Halloran Institute, and I06 does not roll up into I02 in institution_units. The Tessel Laboratory (I04) credit for A08 is on W71, not W70.

Why: Says W70 credits the Halloran Institute (I06) which does not roll up to the University, but never states that only the last-listed affiliation counts