Q20, run 2026-11-04-2035-haiku

← back to the run

Which works in the data are preprints of another work that is also in the data?

Tests: Versions: a shared title is the clue, version_of the evidence; W18 is a preprint with no published version.

Expected: works = ['W02', 'W42']

✅ R1 csv run 1 graded by code

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 2 graded by code

W02, W42. **Reasoning:** These are the works with a version_of link to another work (W03 and W31).

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 3 graded by code

W02 and W42. These are preprints (PPR) with version_of set to W03 and W31.

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 4 graded by code

W02 (preprint of W03) and W42 (preprint of W31). W18 is also a preprint, but it has no version link in the data.

JSON: {"works": ["W02", "W42"]}

✅ R1 csv run 5 graded by code

W02 (version_of W03) and W42 (version_of W31). W18 is also a PPR but has no link to another work in the data.

JSON: {"works": ["W02", "W42"]}

❌ R10 executed run 1 graded by code

None. The only preprint-type work, W18, has no published version in the data, and W02, which merged into W03, is not a work in the table.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

❌ R10 executed run 2 graded by code

None. W18 is a preprint but has no published version in the data, and W02 and W42 are merged records with no row of their own.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

❌ R10 executed run 3 graded by code

None. The preprints (W02, merged into W03, and W42, merged into W31) are not standalone works, and no standalone work is a preprint of another.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

❌ R10 executed run 4 graded by code

None. The preprint records W02 and W42 are merged into W03 and W31, and W18 (PPR) has no published version in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

❌ R10 executed run 5 graded by code

None. The preprints W02 and W42 are merged into W03 and W31, and W18 has no matching published version in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []