Q20, run 2026-11-05-1234-sonnet

← back to the run

Which works in the data are preprints of another work that is also in the data?

Tests: Versions: a shared title is the clue, version_of the evidence; W18 is a preprint with no published version.

Expected: works = ['W02', 'W42']

✅ R1 csv run 1 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 2 graded by code

W02 (preprint of W03) and W42 (preprint of W31). W18 has no `version_of`.

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 3 graded by code

W02 (preprint of W03) and W42 (preprint of W31). W18 has no `version_of`, so it is not one.

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 4 graded by code

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 5 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R2 rdf run 1 graded by code

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ R2 rdf run 2 graded by code

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ R2 rdf run 3 graded by code

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ R2 rdf run 4 graded by code

W02 (preprint of W03) and W42 (preprint of W31). W18 is a preprint with no linked published version.

JSON: {"works": ["W02", "W42"]}

✅ R2 rdf run 5 graded by code

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ R6 rdf+ontology run 1 graded by code

W02 (of W03) and W42 (of W31). W18 has no versionOf link.

JSON: {"works": ["W02", "W42"]}

✅ R6 rdf+ontology run 2 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R6 rdf+ontology run 3 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R6 rdf+ontology run 4 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R6 rdf+ontology run 5 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R5 csv+ontology run 1 graded by code

W02 and W42.

JSON: {"works": ["W02", "W42"]}

✅ R5 csv+ontology run 2 graded by code

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ R5 csv+ontology run 3 graded by code

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ R5 csv+ontology run 4 graded by code

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ R5 csv+ontology run 5 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R4 csv+markdown run 1 graded by code

W02 and W42.

JSON: {"works": ["W02", "W42"]}

✅ R4 csv+markdown run 2 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R4 csv+markdown run 3 graded by code

W02 and W42.

JSON: {"works": ["W02", "W42"]}

✅ R4 csv+markdown run 4 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R4 csv+markdown run 5 graded by code

W02 and W42.

JSON: {"works": ["W02", "W42"]}

✅ R3 csv+padding run 1 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R3 csv+padding run 2 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R3 csv+padding run 3 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R3 csv+padding run 4 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R3 csv+padding run 5 graded by code

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ R10 executed run 1 graded by code

W02 (merged into W03) and W42 (merged into W31). The data doesn't label them as preprints. I infer it from the merges, since they are listed only in record_ids, and W18 (type PPR) is not merged with anything.

JSON: {"works": ["W02", "W42"]}

❌ R10 executed run 2 graded by code

None. W02 and W42 are merged into W03 and W31 and do not appear as separate works. W18 is a preprint, but it is not merged into any work in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

✅ R10 executed run 3 graded by code

W02 and W42. They are merged into W03 and W31 respectively. W18 has a preprint type but no published version in the data.

JSON: {"works": ["W02", "W42"]}

✅ R10 executed run 4 graded by code

W02 and W42, the preprint records merged into W03 and W31. W18 is a preprint but has no published version in the data.

JSON: {"works": ["W02", "W42"]}

✅ R10 executed run 5 graded by code

W02 and W42, which are merged into W03 and W31. W18 is a preprint but has no published version in the data.

JSON: {"works": ["W02", "W42"]}