Q17, run 2026-11-06-0808-sonnet-query

← back to the run

Which of Mara Ellison's works are not research output because of their document type?

Tests: Document types: an editorial and an erratum, under two author records.

Expected: works = ['W04', 'W05']

✅ D1 sql run 1 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D1 sql run 2 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D1 sql run 3 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D1 sql run 4 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D1 sql run 5 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D2 sql+markdown run 1 graded by code · query session

W04 (editorial) and W05 (erratum). W07 is excluded for its delisted source, not for its type.

JSON: {"works": ["W04", "W05"]}

✅ D2 sql+markdown run 2 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D2 sql+markdown run 3 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D2 sql+markdown run 4 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D2 sql+markdown run 5 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D3 sql+executed run 1 graded by code · query session

W04 (editorial) and W05 (erratum). W07 is an article that fails because its source is delisted, not because of its type.

JSON: {"works": ["W04", "W05"]}

✅ D3 sql+executed run 2 graded by code · query session

W04 (editorial) and W05 (erratum). W07 is excluded from research output because its source S04 is delisted, not because of its type.

JSON: {"works": ["W04", "W05"]}

✅ D3 sql+executed run 3 graded by code · query session

W04 (editorial) and W05 (erratum). W07 is also not research output, but because its source S04 is delisted, not because of its type.

JSON: {"works": ["W04", "W05"]}

✅ D3 sql+executed run 4 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D3 sql+executed run 5 graded by code · query session

W04 (editorial) and W05 (erratum). W07 is excluded only because its source is delisted, not because of its type.

JSON: {"works": ["W04", "W05"]}

✅ D4 sparql run 1 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

❌ D4 sparql run 2 graded by code · query session

(no answer found)

Why: no structured answer

✅ D4 sparql run 3 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D4 sparql run 4 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D4 sparql run 5 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D5 sparql+ontology run 1 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D5 sparql+ontology run 2 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

❌ D5 sparql+ontology run 3 graded by code · query session

(no answer found)

JSON: {"works": null}

Why: works: expected ['W04', 'W05'], got None

✅ D5 sparql+ontology run 4 graded by code · query session

W04 (editorial) and W05 (erratum).

JSON: {"works": ["W04", "W05"]}

✅ D5 sparql+ontology run 5 graded by code · query session

W04 (editorial) and W05 (erratum). W07 is excluded because its source is delisted, not because of its type.

JSON: {"works": ["W04", "W05"]}