Q7, run 2026-11-04-2035-haiku

← back to the run

How many research works has Mara Ellison authored?

Tests: Everything at once: identity, versions (W02 is W03), document types (W04, W05) and the delisted source (W07).

Expected: count = 5

❌ R1 csv run 1 graded by code

3. A01 is on W01, W06 and W12 as research; W04 (EDT) and W05 (ERR) are not.

JSON: {"count": 3}

Why: count: expected 5, got 3

❌ R1 csv run 2 graded by code

7. **Reasoning:** Counting the same person across A01, A02 and A03 (per Q1), the research works are W01, W02, W03, W06, W07, W12 and W19. Under A01 alone it is 3 (W01, W06, W12).

JSON: {"count": 7}

Why: count: expected 5, got 7

❌ R1 csv run 3 graded by code

3. W01, W06 and W12 are research. W04 (EDT) and W05 (ERR) are not.

JSON: {"count": 3}

Why: count: expected 5, got 3

❌ R1 csv run 4 graded by code

6. Across the same-ORCID name records, the research works are W01, W03 (with preprint W02), W06, W07, W12 and W19. W04 and W05 are excluded by type.

JSON: {"count": 6}

Why: count: expected 5, got 6

❌ R1 csv run 5 graded by code

7 (W01, W02, W03, W06, W07, W12, W19), counting the same-ORCID records. A01 alone has 3.

JSON: {"count": 7}

Why: count: expected 5, got 7

✅ R10 executed run 1 graded by code

5 (W01, W03, W06, W12, W19). W04, W05 and W07 are not research output.

JSON: {"count": 5}

✅ R10 executed run 2 graded by code

5. A01's research works are W01, W03, W06, W12 and W19. W04, W05 and W07 are not research output, and A04's works are a different person.

JSON: {"count": 5}

✅ R10 executed run 3 graded by code

5. A01's research works are W01, W03, W06, W12 and W19; W04 and W05 are not research output.

JSON: {"count": 5}

✅ R10 executed run 4 graded by code

Five: W01, W03, W06, W12, W19. W04 and W05 are not research output, and W07 is excluded because its source S04 is delisted.

JSON: {"count": 5}

✅ R10 executed run 5 graded by code

5. Of A01's eight works, research output is W01, W03, W06, W12 and W19 (W04, W05 and W07 are not).

JSON: {"count": 5}