Q20, run 2026-11-06-0206-haiku

← back to the run

Which works in the data are preprints of another work that is also in the data?

Tests: Versions: a shared title is the clue, version_of the evidence; W18 is a preprint with no published version.

Expected: works = ['W02', 'W1049', 'W210', 'W219', 'W226', 'W263', 'W322', 'W407', 'W416', 'W42', 'W432', 'W460', 'W506', 'W522', 'W524', 'W545', 'W556', 'W611', 'W612', 'W663', 'W728', 'W730', 'W877']

✅ R1 csv run 1 graded by code

23. These are PPR records whose `version_of` points to another record in the data (W02, W42, W210, W219, W226, W263, W322, W407, W416, W432, W460, W506, W522, W524, W545, W556, W611, W612, W663, W728, W730, W877, W1049).

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

✅ R1 csv run 2 graded by code

Preprints (PPR) with a version_of link to another work in the data: W02, W42, W210, W219, W226, W263, W322, W407, W416, W432, W460, W506, W522, W524, W545, W556, W611, W612, W663, W728, W730, W877, W1049.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

✅ R1 csv run 3 graded by code

23 works are preprints with a version_of link to another work in the data.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

❌ R1 csv run 4 graded by code

(no answer found)

Why: no structured answer

✅ R1 csv run 5 graded by code

(no answer found)

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W1049", "W877"]}

✅ R3 csv+padding run 1 graded by code

23 preprints have a version_of target in the data: W02, W42, W210, W219, W226, W263, W322, W407, W416, W432, W460, W506, W522, W524, W545, W556, W611, W612, W663, W728, W730, W877, W1049.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

✅ R3 csv+padding run 2 graded by code

23 works, each a PPR whose version_of target is also in the data (e.g., W02 → W03, W42 → W31).

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

✅ R3 csv+padding run 3 graded by code

23 preprints point to a version in the data, e.g. W02→W03, W42→W31, W219→W251, W322→W453, W407→W1027.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

✅ R3 csv+padding run 4 graded by code

23 PPR records whose `version_of` target is also in the data: W02, W42, W210, W219, W226, W263, W322, W407, W416, W432, W460, W506, W522, W524, W545, W556, W611, W612, W663, W728, W730, W877, W1049.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

✅ R3 csv+padding run 5 graded by code

23 works, all PPR records whose version_of points to another work in the data: W02, W42, W210, W219, W226, W263, W322, W407, W416, W432, W460, W506, W522, W524, W545, W556, W611, W612, W663, W728, W730, W877, W1049.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

❌ R4 csv+markdown run 1 graded by code

(no answer found)

Why: no structured answer

❌ R4 csv+markdown run 2 graded by code

(no answer found)

Why: no structured answer

✅ R4 csv+markdown run 3 graded by code

23 preprints that have a version_of link to a work in the data.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

❌ R4 csv+markdown run 4 graded by code

(no answer found)

Why: no structured answer

✅ R4 csv+markdown run 5 graded by code

23 preprints linked to a published version: W02, W42, W210, W219, W226, W263, W322, W407, W416, W432, W460, W506, W522, W524, W545, W556, W611, W612, W663, W728, W730, W877, W1049.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

❌ R5 csv+ontology run 1 graded by code

(no answer found)

Why: no structured answer

✅ R5 csv+ontology run 2 graded by code

23. These are the preprints whose published version (versionOf) is also in the data.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

❌ R5 csv+ontology run 3 graded by code

W02, W42, W210, W219, W226, W263, W322, W407, W416, W432, W460, W506, W522, W524, W545, W556, W611, W612, W663, W728, W730, W1049.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W1049"]}

Why: works: expected ['W02', 'W1049', 'W210', 'W219', 'W226', 'W263', 'W322', 'W407', 'W416', 'W42', 'W432', 'W460', 'W506', 'W522', 'W524', 'W545', 'W556', 'W611', 'W612', 'W663', 'W728', 'W730', 'W877'], got ['W02', 'W42', 'W210', 'W219', 'W226', 'W263', 'W322', 'W407', 'W416', 'W432', 'W460', 'W506', 'W522', 'W524', 'W545', 'W556', 'W611', 'W612', 'W663', 'W728', 'W730', 'W1049']

✅ R5 csv+ontology run 4 graded by code

(no answer found)

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

✅ R5 csv+ontology run 5 graded by code

23 works, each a preprint with `versionOf` pointing to a published work in the data.

JSON: {"works": ["W02", "W42", "W210", "W219", "W226", "W263", "W322", "W407", "W416", "W432", "W460", "W506", "W522", "W524", "W545", "W556", "W611", "W612", "W663", "W728", "W730", "W877", "W1049"]}

❌ R10 executed run 1 graded by code

None. The PPR records in the data (W18, W221, W306, W336, W566, W745, W747, W780, W781, W824, W1054) are unmerged, and none of them lists another in-data work in its record_ids. W824 and W1054 have similar titles, but the data does not link them.

JSON: {"works": []}

Why: works: expected ['W02', 'W1049', 'W210', 'W219', 'W226', 'W263', 'W322', 'W407', 'W416', 'W42', 'W432', 'W460', 'W506', 'W522', 'W524', 'W545', 'W556', 'W611', 'W612', 'W663', 'W728', 'W730', 'W877'], got []

❌ R10 executed run 2 graded by code

None. A preprint that has a published version in the data is merged into it (e.g., W03's record_ids are W02 W03), so no remaining work is a preprint of another work in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W1049', 'W210', 'W219', 'W226', 'W263', 'W322', 'W407', 'W416', 'W42', 'W432', 'W460', 'W506', 'W522', 'W524', 'W545', 'W556', 'W611', 'W612', 'W663', 'W728', 'W730', 'W877'], got []

❌ R10 executed run 3 graded by code

None. The merged-away records (e.g. W02 in W03, W42 in W31) are not separate works, and no PPR-typed work shares a title with another work in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W1049', 'W210', 'W219', 'W226', 'W263', 'W322', 'W407', 'W416', 'W42', 'W432', 'W460', 'W506', 'W522', 'W524', 'W545', 'W556', 'W611', 'W612', 'W663', 'W728', 'W730', 'W877'], got []

❌ R10 executed run 4 graded by code

None.
The merged record IDs (W02, W263, W611, W730, etc.) are not themselves works in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W1049', 'W210', 'W219', 'W226', 'W263', 'W322', 'W407', 'W416', 'W42', 'W432', 'W460', 'W506', 'W522', 'W524', 'W545', 'W556', 'W611', 'W612', 'W663', 'W728', 'W730', 'W877'], got []

❌ R10 executed run 5 graded by code

None. No PPR record in the data has a separate published version in the data, and the merged preprint records (W02, W42, W611, etc.) are not works in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W1049', 'W210', 'W219', 'W226', 'W263', 'W322', 'W407', 'W416', 'W42', 'W432', 'W460', 'W506', 'W522', 'W524', 'W545', 'W556', 'W611', 'W612', 'W663', 'W728', 'W730', 'W877'], got []