Q31, run 2026-11-06-0206-sonnet

← back to the run

Which works cite at least one of the works that cite W01?

Tests: Two hops; W51 cites both first-hop works; W90 is a test record.

Expected: works = ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895']

❌ R1 csv run 1 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895, W922 and W1042. W01 is cited by W40, W41, W229, W602, W657, W660 and W895, plus the STG W844, which I excluded.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W895', 'W922', 'W1042']

❌ R1 csv run 2 graded by code

14 works, listed in the JSON.

JSON: {"works": ["W50", "W51", "W52", "W90", "W337", "W338", "W396", "W546", "W682", "W801", "W844", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W90', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W844', 'W895', 'W922', 'W1042']

❌ R1 csv run 3 graded by code

14 works cite a work that cites W01. W01's citers are W40, W41, W229, W602, W657, W660, W844 and W895.

JSON: {"works": ["W50", "W51", "W52", "W90", "W337", "W338", "W396", "W546", "W682", "W801", "W844", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W90', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W844', 'W895', 'W922', 'W1042']

❌ R1 csv run 4 graded by code

14 works, listed in the JSON.

JSON: {"works": ["W50", "W51", "W52", "W90", "W337", "W338", "W396", "W546", "W682", "W801", "W844", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W90', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W844', 'W895', 'W922', 'W1042']

❌ R1 csv run 5 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895, W922, W1042. The staging works W90 and W844 are excluded.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W895', 'W922', 'W1042']

❌ R3 csv+padding run 1 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895, W922, W1042.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W895', 'W922', 'W1042']

❌ R3 csv+padding run 2 graded by code

12 works.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W895', 'W922', 'W1042']

❌ R3 csv+padding run 3 graded by code

12 works. W01 is cited by W40, W41, W229, W602, W657, W660 and W895 (`STG` W844 excluded). Works citing those are listed in the JSON.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W895', 'W922', 'W1042']

❌ R3 csv+padding run 4 graded by code

12 works, listed in the JSON. STG records are excluded.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W895', 'W922', 'W1042']

❌ R3 csv+padding run 5 graded by code

W50, W51, W52, W90, W337, W338, W396, W546, W682, W801, W844, W895, W922 and W1042. The citers of W01 are W40, W41, W229, W602, W657, W660, W844 and W895.

JSON: {"works": ["W50", "W51", "W52", "W90", "W337", "W338", "W396", "W546", "W682", "W801", "W844", "W895", "W922", "W1042"]}

Why: works: expected ['W1042', 'W337', 'W338', 'W396', 'W50', 'W51', 'W52', 'W546', 'W682', 'W801', 'W895'], got ['W50', 'W51', 'W52', 'W90', 'W337', 'W338', 'W396', 'W546', 'W682', 'W801', 'W844', 'W895', 'W922', 'W1042']

✅ R4 csv+markdown run 1 graded by code

11 works cite a citer of W01 (listed in the JSON); W922 is retracted and W844 is STG, so both are excluded.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R4 csv+markdown run 2 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R4 csv+markdown run 3 graded by code

W01 is cited by W40, W41, W229, W602, W657 and W660 (and W895); the works citing at least one of those are listed in the JSON below. I excluded W844 (staging), W90 (staging) and W922 (retracted).

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R4 csv+markdown run 4 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042. W01's valid citers are W40, W41, W229, W602, W657, W660 and W895. STG records and the retracted W922 are dropped.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R4 csv+markdown run 5 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042. The retracted W922 and the staging W90 and W844 are excluded.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R5 csv+ontology run 1 graded by code

11 works: W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042. The citers of W01 are W40, W41, W229, W602, W657, W660 and W895; staging W844 and retracted W922 are excluded.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R5 csv+ontology run 2 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895, W1042.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R5 csv+ontology run 3 graded by code

11 works: W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042. The citers of W01 are W40, W41, W229, W602, W657, W660 and W895. W844 is a test record and is ignored.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R5 csv+ontology run 4 graded by code

The direct citers of W01 are W40, W41, W229, W602, W657, W660 and W895. The works citing at least one of them are W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R5 csv+ontology run 5 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042. The citers of W01 are W40, W41, W229, W602, W657, W660 and W895; the test record W844 is excluded. W90 (test record) and W922 (retracted) also cite them but are excluded.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R10 executed run 1 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R10 executed run 2 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042. The works citing W01 are W40, W41, W229, W602, W657, W660 and W895.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R10 executed run 3 graded by code

W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R10 executed run 4 graded by code

11 works: W50, W51, W52, W337, W338, W396, W546, W682, W801, W895, W1042. They cite one of W01's citers (W40, W41, W229, W602, W657, W660, W895).

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}

✅ R10 executed run 5 graded by code

W01 is cited by W40, W41, W229, W602, W657, W660 and W895. Eleven works cite at least one of those: W50, W51, W52, W337, W338, W396, W546, W682, W801, W895 and W1042.

JSON: {"works": ["W50", "W51", "W52", "W337", "W338", "W396", "W546", "W682", "W801", "W895", "W1042"]}