Q40, run 2026-11-04-2035-haiku

← back to the run

Which source is in the highest tier?

Tests: Local code: K3 is the highest tier, not K1.

Expected: source = S01

✅ R1 csv run 1 graded by code

S01 (Journal of Marine Autonomy), tier K3.

JSON: {"source": "S01"}

✅ R1 csv run 2 graded by code

S01. **Reasoning:** S01 is the only source at tier K3.

JSON: {"source": "S01"}

✅ R1 csv run 3 graded by code

S01, the Journal of Marine Autonomy, is the only K3 source.

JSON: {"source": "S01"}

✅ R1 csv run 4 graded by code

S01 (Journal of Marine Autonomy), the only K3 source.

JSON: {"source": "S01"}

✅ R1 csv run 5 graded by code

S01 (Journal of Marine Autonomy, K3).

JSON: {"source": "S01"}

✅ R10 executed run 1 graded by code

S01 (Journal of Marine Autonomy), tier_rank 1.

JSON: {"source": "S01"}

✅ R10 executed run 2 graded by code

S01 (Journal of Marine Autonomy), tier K3 with tier_rank 1.

JSON: {"source": "S01"}

✅ R10 executed run 3 graded by code

S01 (Journal of Marine Autonomy), tier_rank 1.

JSON: {"source": "S01"}

✅ R10 executed run 4 graded by code

S01 (Journal of Marine Autonomy), tier_rank 1.

JSON: {"source": "S01"}

✅ R10 executed run 5 graded by code

S01. It is the only source with tier_rank 1.

JSON: {"source": "S01"}