Q20, run 2026-11-06-0808-sonnet-query

← back to the run

Which works in the data are preprints of another work that is also in the data?

Tests: Versions: a shared title is the clue, version_of the evidence; W18 is a preprint with no published version.

Expected: works = ['W02', 'W42']

✅ D1 sql run 1 graded by code · query session

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ D1 sql run 2 graded by code · query session

W02 (version of W03) and W42 (version of W31).

JSON: {"works": ["W02", "W42"]}

✅ D1 sql run 3 graded by code · query session

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ D1 sql run 4 graded by code · query session

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ D1 sql run 5 graded by code · query session

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ D2 sql+markdown run 1 graded by code · query session

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ D2 sql+markdown run 2 graded by code · query session

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ D2 sql+markdown run 3 graded by code · query session

W02 and W42.

JSON: {"works": ["W02", "W42"]}

✅ D2 sql+markdown run 4 graded by code · query session

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ D2 sql+markdown run 5 graded by code · query session

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ D3 sql+executed run 1 graded by code · query session

W02 and W42. The record_ids show W02 merged into W03 and W42 merged into W31. W18 is typed as a preprint, but it is not merged into any work in the data.

JSON: {"works": ["W02", "W42"]}

❌ D3 sql+executed run 2 graded by code · query session

None. W02 and W42 appear only as merged records inside W03 and W31, and no other work in the data is a preprint of a published work that is also present.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

❌ D3 sql+executed run 3 graded by code · query session

None. W03 and W31 are merged with W02 and W42, but those records are not separate works in the data.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

✅ D3 sql+executed run 4 graded by code · query session

W02 and W42, with a caveat. They are the records merged into W03 and W31, so they no longer exist as separate works. W18 is typed as a preprint (PPR) but has no published version in the data.

JSON: {"works": ["W02", "W42"]}

❌ D3 sql+executed run 5 graded by code · query session

None. W18 has the preprint type, but no published version of it is in the data. The merged preprints W02 and W42 are already folded into W03 and W31.

JSON: {"works": []}

Why: works: expected ['W02', 'W42'], got []

✅ D4 sparql run 1 graded by code · query session

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

❌ D4 sparql run 2 graded by code · query session

(no answer found)

Why: no structured answer

✅ D4 sparql run 3 graded by code · query session

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}

✅ D4 sparql run 4 graded by code · query session

W02 (a version of W03) and W42 (a version of W31).

JSON: {"works": ["W02", "W42"]}

✅ D4 sparql run 5 graded by code · query session

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ D5 sparql+ontology run 1 graded by code · query session

W02 (a version of W03) and W42 (a version of W31).

JSON: {"works": ["W02", "W42"]}

✅ D5 sparql+ontology run 2 graded by code · query session

W02 and W42.

JSON: {"works": ["W02", "W42"]}

❌ D5 sparql+ontology run 3 graded by code · query session

Only W02 is confirmed, as a preprint of W03. The check for others returned corrupted rows.

JSON: {"works": ["W02"]}

Why: works: expected ['W02', 'W42'], got ['W02']

✅ D5 sparql+ontology run 4 graded by code · query session

W02 (preprint of W03) and W42 (preprint of W31).

JSON: {"works": ["W02", "W42"]}

✅ D5 sparql+ontology run 5 graded by code · query session

W02 (of W03) and W42 (of W31).

JSON: {"works": ["W02", "W42"]}