Q31, run 2026-11-04-2035-haiku

← back to the run

Which works cite at least one of the works that cite W01?

Tests: Two hops; W51 cites both first-hop works; W90 is a test record.

Expected: works = ['W50', 'W51', 'W52']

❌ R1 csv run 1 graded by code

W50, W51, W52 and W90, which cite W40 or W41 (the two works citing W01).

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 2 graded by code

W50, W51, W52, W90. **Reasoning:** W40 and W41 cite W01, and W50 and W51 cite W40 while W51, W52 and W90 cite W41.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 3 graded by code

W50, W51, W52 and W90. These cite W40 or W41, which are the works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 4 graded by code

W50, W51, W52 and W90. These cite W40 or W41, which are the works that cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

❌ R1 csv run 5 graded by code

W50, W51, W52 and W90. They cite W40 or W41, which cite W01.

JSON: {"works": ["W50", "W51", "W52", "W90"]}

Why: works: expected ['W50', 'W51', 'W52'], got ['W50', 'W51', 'W52', 'W90']

✅ R10 executed run 1 graded by code

W50, W51, W52. These are the works citing W40 or W41, which are the works that cite W01.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 2 graded by code

W50, W51 and W52. W40 and W41 are the works citing W01, and these three cite them.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 3 graded by code

W50, W51, W52. W40 and W41 cite W01, and W50 and W51 cite W40, while W51 and W52 cite W41.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 4 graded by code

W50, W51 and W52. These cite W40 or W41, which are the two works citing W01.

JSON: {"works": ["W50", "W51", "W52"]}

✅ R10 executed run 5 graded by code

W50, W51 and W52. W40 and W41 are the works citing W01, and these three cite them.

JSON: {"works": ["W50", "W51", "W52"]}