Q31, run 2026-11-05-1235-haiku

← back to the run

Which works cite at least one of the works that cite W01?

Tests: Two hops; W51 cites both first-hop works; W90 is a test record.

Expected: works = ['W50', 'W51', 'W52']

❌ R1 csv run 1 graded by code

W50, W51, W52, W90. These cite W40 or W41, which are the two works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 2 graded by code

W50, W51, W52 and W90.
*Reasoning:* W01 is cited by W40 and W41. W50 and W51 cite W40, W51 and W52 cite W41, and W90 cites W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 3 graded by code

W50, W51, W52 and W90. These cite W40 or W41, which are the two works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 4 graded by code

W50, W51, W52 and W90, which cite W40 or W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 5 graded by code

W50, W51, W52 and W90. W40 and W41 cite W01, and these four cite W40 or W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 1 graded by code

W50, W51, W52 and W90. These cite W40 or W41, which are the works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 2 graded by code

W50, W51, W52, W90. These cite W40 or W41, which are the two works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 3 graded by code

W50, W51, W52 and W90, which cite W40 or W41 (both citing W01).

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 4 graded by code

W50, W51, W52, W90. These cite W40 or W41, which are the works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 5 graded by code

W50, W51, W52, W90. W40 and W41 cite W01, and W50, W51, W52 and W90 cite W40 or W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

✅ R6 rdf+ontology run 1 graded by code

W50, W51 and W52. They cite W40 or W41, which cite W01. W90 is a test record and is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R6 rdf+ontology run 2 graded by code

W50, W51 and W52. These cite W40 or W41, which cite W01. The staging record W90 is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R6 rdf+ontology run 3 graded by code

W50, W51, W52. These cite W40 or W41, which cite W01. W90 is a staging record and is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R6 rdf+ontology run 4 graded by code

W50, W51 and W52. These cite W40 or W41, which cite W01. W90 also cites W41 but is a test record.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R6 rdf+ontology run 5 graded by code

W50, W51 and W52. They cite W40 or W41, which cite W01; W90 is a staging record and is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 1 graded by code

W50, W51 and W52. W40 and W41 are the works citing W01. W50 and W51 cite W40, and W51 and W52 cite W41. W90 is a staging record, so it is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 2 graded by code

W50, W51 and W52. These cite W40 or W41, which cite W01. W90 is excluded as a staging record.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 3 graded by code

W50, W51, W52. W40 and W41 are the works that cite W01, and W50, W51 and W52 cite them; W90 is a staging record and excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 4 graded by code

W50, W51 and W52. W50 and W51 cite W40, and W51 and W52 cite W41. W90 is a staging record, so its citation of W41 is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 5 graded by code

W50, W51 and W52. W40 and W41 cite W01, and W50, W51 and W52 cite them. W90 cites W41 but is a test record, so it is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 1 graded by code

W50, W51 and W52. These cite W40 or W41, which cite W01. W90 cites W41 but is a staging record, so it is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 2 graded by code

W50, W51 and W52. These cite W40 or W41, which cite W01. W90 cites W41 but is a staging record and is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 3 graded by code

W50, W51 and W52. W01 is cited by W40 and W41. Those are cited by W50, W51 and W52. W90 is a staging record and is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 4 graded by code

W50, W51 and W52. These cite W40 or W41, the two works citing W01. W90 also cites W41 but is a staging record.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 5 graded by code

W50, W51 and W52. They cite W40 or W41, which cite W01. W90 is a test record and is excluded.

JSON: {"works": ["W50", "W51", "W52"]}

❌ R3 csv+padding run 1 graded by code

W50, W51, W52 and W90. These are the works that cite W40 or W41, which are the two works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R3 csv+padding run 2 graded by code

W50, W51, W52 and W90. W01 is cited by W40 and W41, which are in turn cited by these four works.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R3 csv+padding run 3 graded by code

W50, W51, W52 and W90. They cite W40 or W41, which cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R3 csv+padding run 4 graded by code

W50, W51, W52, W90. These cite W40 or W41, which cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R3 csv+padding run 5 graded by code

W50, W51, W52 and W90. These cite W40 or W41, the two works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

✅ R10 executed run 1 graded by code

W50, W51, W52. These cite W40 or W41, which are the works citing W01.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 2 graded by code

W50, W51 and W52. W40 and W41 cite W01, and these three cite W40 or W41.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 3 graded by code

W50, W51 and W52. They cite W40 or W41, which cite W01.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 4 graded by code

W50, W51, W52. These cite W40 or W41, which cite W01.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 5 graded by code

W50, W51 and W52. Works W40 and W41 cite W01; W50 and W51 cite W40, and W51 and W52 cite W41.

JSON: {"works": ["W50", "W51", "W52"]}