Q1, run 2026-11-04-2053-sonnet

← back to the run

How many distinct people do the author records named "Mara Ellison", "M. Ellison" and "Ellison, Mara J." stand for?

Tests: Identity: A01, A02 and A03 share an ORCID; A04, also 'M. Ellison', has another.

Expected: count = 2

❌ R1 csv run 1 graded by code

1. A01, A02 and A03 share ORCID 0000-0002-9841-6229. A04 ("M. Ellison") has a different ORCID, so it is a different person.

JSON: {"count": 1}

Why: count: expected 2, got 1

❌ R1 csv run 2 graded by code

1. A01, A02 and A03 share ORCID 0000-0002-9841-6229. A04 has a different ORCID, so it is someone else.

JSON: {"count": 1}

Why: count: expected 2, got 1

❌ R1 csv run 3 graded by code

1. A01, A02 and A03 share ORCID 0000-0002-9841-6229. A04 has a different ORCID.

JSON: {"count": 1}

Why: count: expected 2, got 1

✅ R1 csv run 4 graded by code

2. A01, A02 and A03 share ORCID 0000-0002-9841-6229, so they are one person. A04 ("M. Ellison") has a different ORCID, so it is a second person.

JSON: {"count": 2}

❌ R1 csv run 5 graded by code

1. A01, A02 and A03 share ORCID 0000-0002-9841-6229, so they are one person. A04 ("M. Ellison") has a different ORCID.

JSON: {"count": 1}

Why: count: expected 2, got 1

✅ R10 executed run 1 graded by code

2. A01 (which merges A01, A02 and A03) is one person and A04 "M. Ellison" is another. I'm assuming "Ellison, Mara J." is one of the merged records A02/A03, since their names aren't shown.

JSON: {"count": 2}

✅ R10 executed run 2 graded by code

2. A01 merges A01, A02 and A03, which covers "Mara Ellison" and "Ellison, Mara J."; "M. Ellison" is the separate person A04, with a different ORCID. I couldn't see the names of A02 and A03, so this assumes "Ellison, Mara J." is one of them.

JSON: {"count": 2}

✅ R10 executed run 3 graded by code

2. A01 (merging A01, A02 and A03) is one person. A04 "M. Ellison" has a different ORCID and is a separate person in people.csv. I can't see the name of A02 or A03, so I'm assuming "Ellison, Mara J." is one of them.

JSON: {"count": 2}

✅ R10 executed run 4 graded by code

2. A01 already merges A01, A02 and A03 (which would cover "Ellison, Mara J."), while A04 "M. Ellison" has a different ORCID and stays a separate person.

JSON: {"count": 2}

✅ R10 executed run 5 graded by code

2. "Mara Ellison" and "Ellison, Mara J." resolve to A01 (author_ids A01 A02 A03), and "M. Ellison" is a separate person, A04, with a different ORCID.

JSON: {"count": 2}