Q48, run 2026-11-05-1235-haiku

← back to the run

Why is W70 not one of the University of Albury's research works, though its author lists the Tessel Laboratory?

Tests: Local rules: the last-listed affiliation, and affiliated institutes.

Expected: Tomasz Wierzba lists the Tessel Laboratory first and the Halloran Institute second; the house rule credits the last-listed one, Halloran. Halloran is an affiliated institute (AFF): its parent link is an agreement, so its works are not the University of Albury's.

❌ R1 csv run 1 graded by llm

The premise is wrong. W70 is credited to I04, which is Tessel, and I04 sits under I03 and I02, so the work is in Albury's units-included count (Q9). It would only fall outside if you counted I02 alone, and W70 is also credited to I06, a unit of I02.

Why: Answer rejects the premise and says W70 is credited to Tessel and counts under Albury; the key says only the last-listed affiliation counts, so it goes to the Halloran Institute, an affiliated institute that doesn't roll up to Albury.

❌ R1 csv run 2 graded by llm

The premise is wrong. W70 is credited to I04 (Tessel Laboratory), which sits under I03 and then I02 (University of Albury). W70 is therefore one of the 13 UoA research works counted in Q9.

Why: Answer claims W70 counts for the University of Albury via Tessel Laboratory; key says only the last-listed affiliation (Halloran Institute, an affiliated institute not a unit) counts, so it doesn't roll up to UoA.

❌ R1 csv run 3 graded by llm

The premise is wrong. W70 lists I04 (Tessel Laboratory) at affil_seq 1, and I04 sits under I03 → I02 (University of Albury). So W70 is one of Albury's research works, and it is also credited to I06 (Halloran).

Why: Rejects the premise; the key says only the last-listed affiliation counts, so W70 goes to Halloran Institute, an affiliated institute that does not roll up to Albury.

❌ R1 csv run 4 graded by llm

The premise is wrong. W70 is credited to Albury through both I04 (Tessel, under I03 → I02) and I06 (Halloran, under I02), so it is one of Albury's works.

Assumptions: research output means ART, REV and PPR, with preprints merged into their article version. EDT, ERR, PTX and SUP are excluded. STG-sourced records are included, and their effect is noted where it matters.

Why: Claims W70 does count for Albury via both affiliations, contradicting the key that only the last-listed affiliation (Halloran Institute, an affiliated institute that doesn't roll up) counts.

❌ R1 csv run 5 graded by llm

The premise is false. W70 is credited to I04 (Tessel), which sits under I03 and I02, and also to I06 (Halloran), which sits directly under I02. W70 is an ART, not retracted, so it is counted among the University of Albury's research works (Q9).

Why: Answer says W70 counts for Albury, contradicting the key: only the last-listed affiliation (Halloran) counts and Halloran is an affiliated institute that doesn't roll up.

❌ R2 rdf run 1 graded by llm

The premise is wrong. W70 is credited to I04 (Tessel), whose parent I03 and grandparent I02 are both University of Albury units, so W70 is counted in Q9's result.



Q47 and Q48 have no entries in the JSON schema, which stops at Q46, so they are answered in prose only.

Why: Rejects the premise and credits W70 to Tessel/University of Albury, whereas the key says only the last-listed affiliation counts, so W70 goes to the Halloran Institute, an affiliated institute and not a unit.

❌ R2 rdf run 2 graded by llm

The premise is false. W70's author A08 lists I04 (Tessel Laboratory), which sits under I03 and I02 (University of Albury), so W70 is an Albury research work and is one of the 13 counted in Q9.

Why: Rejects the premise and counts W70 for Albury, contradicting the key that only the last-listed affiliation (Halloran Institute) counts and that Halloran is an affiliated institute that doesn't roll up to Albury.

❌ R2 rdf run 3 graded by llm

The premise is incorrect: W70 is credited to the University of Albury through I04 (Tessel, under I03 → I02) and through I06 (Halloran, under I02), so it is counted in Q9. No JSON field is specified for Q48.

Why: The answer says W70 does count toward Albury via both affiliations, which contradicts the key (only the last-listed affiliation counts, so it goes to Halloran, an affiliated institute that does not roll up to Albury).

❌ R2 rdf run 4 graded by llm

The premise is false. I04 (Tessel) sits under I03 → I02 (University of Albury), and I06 is also a unit of I02. So W70 is one of the University of Albury's research works, and it is counted in Q9 and Q39.

Why: Rejects the premise and says W70 counts for Albury; the key says only the last-listed affiliation counts, crediting the Halloran Institute, an affiliated institute that does not roll up to Albury.

❌ R2 rdf run 5 graded by llm

The premise is wrong. W70's author has a Tessel (I04) affiliation, and I04 → I03 → I02, so W70 is one of Albury's research works (it is included in the Q9 and Q10 totals).

Why: Answer rejects the premise and says W70 counts for Albury; the key says only the last-listed affiliation counts, crediting the Halloran Institute, an affiliated institute that does not roll up to Albury.

✅ R6 rdf+ontology run 1 graded by llm

A08 lists I04 (seq 1) and I06 (seq 2) on W70. The house rule credits only the highest-sequence affiliation, so I04 is ignored, and I06 is an affiliated institute whose parent link is not followed.

✅ R6 rdf+ontology run 2 graded by llm

W70 lists A08 with two affiliations, I04 (the Tessel Laboratory) and then I06, and only the last-listed affiliation counts. I06 is an affiliated institute with a parent link that is not a unit relation, so the work is not credited to I02.

✅ R6 rdf+ontology run 3 graded by llm

W70 credits A08's last-listed affiliation, I06 (seq 2), not I04 (seq 1). I06 is an affiliated institute, so it is not a unit of I02 and W70 does not count for the University of Albury.

✅ R6 rdf+ontology run 4 graded by llm

W70 is credited to the last affiliation of its author A08, which is I06 (sequence 2), so the Tessel Laboratory (I04, sequence 1) is ignored. I06 is an affiliated institute, so its parent link to I02 does not count.

✅ R6 rdf+ontology run 5 graded by llm

A08 lists two affiliations on W70: Tessel (I04, sequence 1) and Halloran (I06, sequence 2). Only the last-listed one counts, and Halloran is an affiliated institute, so its parent link to I02 is an agreement, not membership.

The JSON below covers Q1–Q46, as its field list specifies.

✅ R5 csv+ontology run 1 graded by llm

For A08 on W70, the house rule credits only the last-listed affiliation, I06 (the Halloran Institute). I06 is an affiliated institute (AFF), so its link to I02 is not a unit relationship, and I04's University of Albury credit is lost. The data has no structured field for this question.

✅ R5 csv+ontology run 2 graded by llm

A08's byline on W70 lists two affiliations: I04 (sequence 1) and I06 (sequence 2). The house rule credits only the last one, I06 (Halloran), which is an affiliated institute with no unit link to the University of Albury, so W70 is not credited there.

🟡 R5 csv+ontology run 3 graded by llm

A08 listed two affiliations on W70, I04 (seq 1) and I06 (seq 2). Only the last-listed affiliation is credited, so W70 goes to I06, an affiliated institute that is not a unit of Albury, and the Tessel Laboratory affiliation is ignored.

Why: Says only the last-listed affiliation counts and that it goes to I06, an affiliated institute not a unit of Albury, which meets both points, but names IDs rather than the Halloran Institute and doesn't explicitly tie it to the Tessel Laboratory being ignored beyond a brief mention

✅ R5 csv+ontology run 4 graded by llm

A08 listed two affiliations on W70: I04 first and I06 second. The house rule credits only the last-listed affiliation, I06, so I04 is ignored. I06 is an affiliated institute, so its parent link to I02 does not make it a unit, and W70 is not credited to Albury.

🟡 R5 csv+ontology run 5 graded by llm

On W70, A08 lists I04 (sequence 1) and I06 (sequence 2), and the house rule credits only the last affiliation. That is I06, an affiliated institute outside Albury's unit tree, so W70 is credited to I06 alone.

Why: Says only the last affiliation counts so W70 goes to I06 (Halloran), outside Albury's unit tree, but doesn't state explicitly that Halloran is an affiliated institute rather than a unit.

✅ R4 csv+markdown run 1 graded by llm

Because W70's author A08 lists I04 (Tessel) first and I06 (Halloran) last. Only the last-listed affiliation counts, so the work is credited to I06, an affiliated institute, and not to I04, I03 or I02.

✅ R4 csv+markdown run 2 graded by llm

Its author A08 lists two affiliations on W70: Tessel (I04, sequence 1) and Halloran (I06, sequence 2). Under the house rule only the last-listed affiliation counts, which is Halloran, an affiliated institute whose parent link does not count, so the work is credited to neither the Tessel lab nor the University of Albury.

✅ R4 csv+markdown run 3 graded by llm

A08's last-listed affiliation on W70 is the Halloran Institute (I06), with sequence 2. The house rule credits only the last affiliation, so the Tessel Laboratory (I04, sequence 1) is ignored. I06 is an affiliated institute, so its works are never credited to the University of Albury.

🟡 R4 csv+markdown run 4 graded by llm

On W70, A08 lists I04 (sequence 1) and I06 (sequence 2). Only the last-listed affiliation counts, which is I06, an affiliated institute outside the Albury tree, so the I04 affiliation is ignored.

Why: Says last-listed affiliation counts and it's an institute outside the Albury tree, but doesn't name W70 credited to Halloran Institute explicitly or that affiliated institutes' works don't roll up to the University; the Tessel Laboratory (I04) point is only implied.

✅ R4 csv+markdown run 5 graded by llm

A08's affiliations on W70 are I04 (sequence 1) and I06 (sequence 2). Under the house rule only the last one, I06, is credited. I06 is an affiliated institute, so its parent link to I02 does not make W70 an Albury work.

❌ R3 csv+padding run 1 graded by llm

The premise is false. W70 is credited to the University of Albury through both I04 (Tessel, under I03 → I02) and I06 (Halloran, directly under I02), so it is one of Albury's works and is included in Q9.

Why: Answer says W70 is credited to Albury via Tessel and Halloran, contradicting the key (only last-listed affiliation counts, Halloran is an affiliated institute not a unit, so W70 doesn't roll up to Albury).

❌ R3 csv+padding run 2 graded by llm

The premise is wrong. W70 is credited to I04 (Tessel), which sits under I03 and then I02 (University of Albury). It is also credited to I06 (Halloran), which sits under I02, so W70 is one of the 13 University of Albury research works in Q9.

Why: Rejects the premise and says W70 counts for Albury, contradicting the key: only the last-listed affiliation counts (Halloran), and Halloran is an affiliated institute, not a unit, so it doesn't roll up.

❌ R3 csv+padding run 3 graded by llm

The premise is wrong. W70 is credited to I04 (Tessel), which sits under I02, and also to I06 (Halloran), which also sits under I02. It is ART and not retracted, so W70 is one of the University of Albury's research works.

Why: Rejects the premise; the key says only the last-listed affiliation counts, so W70 goes to Halloran, an affiliated institute not rolling up to Albury

❌ R3 csv+padding run 4 graded by llm

The premise doesn't hold. W70 lists the Tessel Laboratory (I04), which sits under Albury (I03 → I02 → I01), and also Halloran (I06), an Albury unit. So W70 is an Albury work and is counted in Q9 and Q10. This has no JSON field either.

Why: Rejects the premise and says W70 is an Albury work; the key says only the last-listed affiliation counts, crediting W70 to Halloran Institute, an affiliated institute that doesn't roll up to Albury.

❌ R3 csv+padding run 5 graded by llm

(Not in the JSON schema.) The premise is wrong. W70 is credited to the University of Albury. Tessel (I04) sits under the Albury Centre for Marine Robotics (I03), which sits under the University of Albury (I02). W70 also lists Halloran (I06), a unit of I02.

Why: Contradicts the key: says W70 is credited to Albury and Halloran is a unit, but the key says only the last affiliation counts and Halloran is an affiliated institute that doesn't roll up.

🟡 R10 executed run 1 graded by llm

The credit for A08 on W70 is to I06 (Halloran), not I04 (Tessel), and institution_units does not roll I06 up to I02. Tessel's credit appears only on W71 and W72 for A08.

Why: Says the credit goes to Halloran (I06) and that I06 doesn't roll up to the University of Albury, but never states that only the last-listed affiliation counts, and never says Halloran is an affiliated institute rather than a unit.

🟡 R10 executed run 2 graded by llm

The premise is wrong. W70's only credit is A08 at I06 (Halloran), not the Tessel Laboratory (I04). Halloran's works count only toward I06 in institution_units, not toward Albury, so W70 is not an Albury work.

Why: Says Halloran is credited and not rolled up to Albury, but doesn't state the last-listed-affiliation rule or that Halloran is an affiliated institute rather than a unit; it also rejects the premise instead of explaining it.

🟡 R10 executed run 3 graded by llm

The premise is wrong. W70's only credit is A08 at I06 (Halloran), not I04 (Tessel), and I06's works roll up only to I06 in institution_units, so W70 is not counted for the University of Albury. A08 is credited to I04 on W71 and W72, not W70.

Why: Says W70 is credited to Halloran (I06) and its works don't roll up to Albury, but doesn't state the last-listed-affiliation rule or that Halloran is an affiliated institute rather than a unit.

✅ R10 executed run 4 graded by llm

The premise is wrong. W70 is credited to I06 (Halloran), not to Tessel (I04), which is credited on W71 and W72. Also, I06's works do not roll up to I02 in institution_units, so W70 is excluded from Albury's count for that reason.

🟡 R10 executed run 5 graded by llm

W70 is credited to A08 at I06 (Halloran, type AFF), not to the Tessel Laboratory (I04). A08 is credited to I04 only on W71 and W72. I06 counts only towards itself in institution_units, so W70 doesn't roll up to I02 and isn't one of Albury's works.

Why: Says W70 is credited to Halloran (AFF) and that it doesn't roll up to Albury, but never states the 'last-listed affiliation counts' rule.