Q31, run 2026-11-06-0808-sonnet-query

← back to the run

Which works cite at least one of the works that cite W01?

Tests: Two hops; W51 cites both first-hop works; W90 is a test record.

Expected: works = ['W50', 'W51', 'W52']

❌ D1 sql run 1 graded by code · query session

W50, W51, W52, W90. W01 is cited by W40 and W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ D1 sql run 2 graded by code · query session

W50, W51, W52 and W90. W40 and W41 cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ D1 sql run 3 graded by code · query session

W50, W51, W52, W90. W01 is cited by W40 and W41, and these four cite them. W90 is an STG work, included here because the question is about citations, not research output.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ D1 sql run 4 graded by code · query session

W50, W51, W52, W90. These cite W40 or W41, which are the works citing W01. W90 is an STG record.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ D1 sql run 5 graded by code · query session

W50, W51, W52, W90. These cite W40 or W41, which are the works citing W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

✅ D2 sql+markdown run 1 graded by code · query session

W50, W51, W52. W01 is cited by W40 and W41, and the works citing those are W50, W51, W52 and staging record W90, which is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D2 sql+markdown run 2 graded by code · query session

W50, W51, W52. These cite W40 or W41, which cite W01; W90 is a staging record.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D2 sql+markdown run 3 graded by code · query session

W50, W51, W52.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D2 sql+markdown run 4 graded by code · query session

W50, W51, W52. W90 also qualifies but is a staging record, so it is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D2 sql+markdown run 5 graded by code · query session

W50, W51 and W52. The works citing W01 are W40 and W41, and the staging record W90 is ignored.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D3 sql+executed run 1 graded by code · query session

W50, W51, W52. W01 is cited by W40 and W41, and these three cite those.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D3 sql+executed run 2 graded by code · query session

W50, W51, W52. W01 is cited by W40 and W41, which are in turn cited by those three works.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D3 sql+executed run 3 graded by code · query session

W50, W51, W52. W01 is cited by W40 and W41, which are in turn cited by these three.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D3 sql+executed run 4 graded by code · query session

W50, W51 and W52. The works citing W01 are W40 and W41.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D3 sql+executed run 5 graded by code · query session

W50, W51, W52. W01 is cited by W40 and W41, and those two are cited by W50, W51 and W52.

JSON: {"works": ["W50", "W51", "W52"]}

❌ D4 sparql run 1 graded by code · query session

W50, W51, W52 and W90. W40 and W41 cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ D4 sparql run 2 graded by code · query session

(no answer found)

Why: no structured answer

❌ D4 sparql run 3 graded by code · query session

W50, W51, W52, W90. W01 is cited by W40 and W41, and these four cite W40 or W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ D4 sparql run 4 graded by code · query session

W50, W51, W52, W90.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ D4 sparql run 5 graded by code · query session

W50, W51, W52, W90.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

✅ D5 sparql+ontology run 1 graded by code · query session

W50, W51, W52. W90 is excluded as a staging record.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D5 sparql+ontology run 2 graded by code · query session

W50, W51, W52.

JSON: {"works": ["W50", "W51", "W52"]}

❌ D5 sparql+ontology run 3 graded by code · query session

(no answer found)

JSON: {"works": null}

Why: works: expected ['W50', 'W51', 'W52'], got None

✅ D5 sparql+ontology run 4 graded by code · query session

W50, W51 and W52. The works citing W01 are W40 and W41, and these three cite them.

JSON: {"works": ["W50", "W51", "W52"]}

✅ D5 sparql+ontology run 5 graded by code · query session

W50, W51, W52.

JSON: {"works": ["W50", "W51", "W52"]}