Q20, run 2026-11-04-2053-sonnet

← back to the run

Which works in the data are preprints of another work that is also in the data?

Tests: Versions: a shared title is the clue, version_of the evidence; W18 is a preprint with no published version.

Expected: works = ['W02', 'W42']

✅ R1 csv run 1 graded by code

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 2 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 3 graded by code

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 4 graded by code

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 5 graded by code

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ R10 executed run 1 graded by code

W02 and W42. They are the merged preprint records inside W03 and W31. W18 is typed PPR but is not merged into any other work.

JSON: {"works": ["W02", "W42"]}

❌ R10 executed run 2 graded by code

None. W18 is a preprint but is not merged into any work in the data. W02 and W42 are only records merged into W03 and W31.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

❌ R10 executed run 3 graded by code

None. W18 is typed as a preprint but isn't merged into any work in the data. W02 and W42 were merged into W03 and W31 and no longer appear as separate works.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

✅ R10 executed run 4 graded by code

W02 and W42, the preprint records merged into W03 and W31. W18 is a preprint but has no published version in the data.

JSON: {"works": ["W02", "W42"]}

❌ R10 executed run 5 graded by code

None. The merged records W02 and W42 sit inside W03 and W31 rather than as separate works, and W18 is a preprint without a matching published work in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []