Q31, run 2026-11-05-1234-sonnet

← back to the run

Which works cite at least one of the works that cite W01?

Tests: Two hops; W51 cites both first-hop works; W90 is a test record.

Expected: works = ['W50', 'W51', 'W52']

✅ R1 csv run 1 graded by code

W50, W51, W52. W01 is cited by W40 and W41, and those are cited by W50, W51 and W52 (W90 is staging).

JSON: {"works": ["W50", "W51", "W52"]}

❌ R1 csv run 2 graded by code

W50, W51, W52, W90. W01 is cited by W40 and W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

✅ R1 csv run 3 graded by code

W50, W51, W52. W40 and W41 cite W01; W50 and W51 cite W40, and W51 and W52 cite W41. I excluded the staging record W90.

JSON: {"works": ["W50", "W51", "W52"]}

❌ R1 csv run 4 graded by code

W50, W51, W52, W90. W40 and W41 cite W01. W50 and W51 cite W40. W51, W52 and W90 cite W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 5 graded by code

W50, W51, W52, W90. W01 is cited by W40 and W41, and those two are cited by these four.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 1 graded by code

W50, W51, W52, W90. W01 is cited by W40 and W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 2 graded by code

W50, W51, W52, W90. W01 is cited by W40 and W41, and these four cite one of them.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 3 graded by code

W50, W51, W52, W90. The works citing W01 are W40 and W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 4 graded by code

W50, W51, W52, W90. W01 is cited by W40 and W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R2 rdf run 5 graded by code

W50, W51, W52, W90. W01 is cited by W40 and W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

✅ R6 rdf+ontology run 1 graded by code

W50, W51, W52. W40 and W41 cite W01; W90 is staging and is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R6 rdf+ontology run 2 graded by code

W50, W51, W52. W01 is cited by W40 and W41, and the works citing those are listed here. W90 is staging and excluded.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R6 rdf+ontology run 3 graded by code

W50, W51, W52. W90 also cites W41, but it is a staging record and is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R6 rdf+ontology run 4 graded by code

W50, W51 and W52. W90 is excluded as a staging record. W40 and W41 are the works citing W01.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R6 rdf+ontology run 5 graded by code

W50, W51, W52. The works citing W01 are W40 and W41, and W90 is a staging record and is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 1 graded by code

W50, W51, W52. W01 is cited by W40 and W41, and the staging work W90 is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 2 graded by code

W50, W51 and W52. W40 and W41 cite W01, and these three cite W40 or W41. W90 is a staging record and is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 3 graded by code

W50, W51, W52. W01 is cited by W40 and W41, and W90 is a staging record so it is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 4 graded by code

W50, W51 and W52. W01 is cited by W40 and W41, and these three cite those.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R5 csv+ontology run 5 graded by code

W50, W51, W52. W90 is a staging record and is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 1 graded by code

W50, W51, W52. W01 is cited by W40 and W41, and these three cite W40 or W41. W90 is a staging record.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 2 graded by code

W50, W51, W52. The citers of W01 are W40 and W41, and W90 is a staging record so it is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 3 graded by code

W50, W51, W52. W01 is cited by W40 and W41, and W90 is a staging record.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 4 graded by code

W50, W51, W52. These cite W40 or W41, which are the citers of W01. W90 is staging and ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R4 csv+markdown run 5 graded by code

W50, W51, W52. W01 is cited by W40 and W41, and the staging record W90 is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R3 csv+padding run 1 graded by code

W50, W51, W52. W01 is cited by W40 and W41; W90 would also qualify but is STG.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R3 csv+padding run 2 graded by code

W50, W51, W52. W01 is cited by W40 and W41, which are in turn cited by those three.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R3 csv+padding run 3 graded by code

W50, W51 and W52. W40 and W41 cite W01, and these three cite W40 or W41.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R3 csv+padding run 4 graded by code

W50, W51, W52.

JSON: {"works": ["W50", "W51", "W52"]}

❌ R3 csv+padding run 5 graded by code

W50, W51, W52, W90. These cite W40 or W41, which are the works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

✅ R10 executed run 1 graded by code

W50, W51, W52. The works citing W01 are W40 and W41.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 2 graded by code

W50, W51, W52 (via W40 and W41).

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 3 graded by code

W50, W51 and W52. W01 is cited by W40 and W41, and those are cited by W50, W51 and W52.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 4 graded by code

W50, W51, W52. W01 is cited by W40 and W41, which are cited by these three.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 5 graded by code

W50, W51, W52. W01 is cited by W40 and W41.

JSON: {"works": ["W50", "W51", "W52"]}