Q47, run 2026-11-04-2035-haiku

โ† back to the run

Why do Kesting University and Rostval Technical University swap places in 2024 between full and fractional counting?

Tests: Counting method: why the ranking flips.

Expected: Rostval is on four 2024 works (W21, W27, W28, W29), but on each it has one author of four: 4 x 1/4 = 1. Kesting is on three, with larger shares: W22 alone, half of W20, two of the four authors of W21: 2.

๐ŸŸก R1 csv run 1 graded by llm

Full counting credits each institution with every work it is on. Rostval is on four 2024 works (W21, W27, W28, W29), all with 4 authors, so it gets 4 full credits. Kesting gets 3 full credits from W20, W21 and W22, which give it 2.0 fractionally. Rostval's shares are only 1/4 per work (total 1.0), so the ranking swaps.

Why: Gets Rostval's four 4-author works at 1/4 each, but misses that Kesting's lead comes from a single-author review (W22) plus half shares of W20 and W21; the 2.0 total and the 'half' split are not explained, and the answer's account of full counting is muddled.

๐ŸŸก R1 csv run 2 graded by llm

Rostval has more works in full counting but less credit in fractional counting. **Reasoning:** Rostval's 4 works each have 4 authors and only one Rostval author, so it gets 0.25 per work (1.0 total). Kesting's works carry more of its own authors (W22 is single-author, W21 has 2 of 4) and total 2.0 despite fewer works.

Why: Gets Rostval's quarter-share per work right, but for Kesting it cites W22 and W21 (2 of 4) rather than W22 plus half of both W20 and W21, so the checklist's Kesting point is only partly met

๐ŸŸก R1 csv run 3 graded by llm

Full counting favours Rostval (4 works to Kesting's 3). Fractional counting favours Kesting (2.0 to Rostval's 1.0). Rostval's 2024 works each have four authors, only one of whom is from Rostval, so each earns only a quarter of a credit. Kesting's W22 is single-authored, and its other 2024 works carry larger shares.

Why: Gives the quarter-credit point for Rostval's four-author works, but only vaguely says Kesting has larger shares; omits that W22 is a single-author review and that Kesting holds half of W20 and W21.

๐ŸŸก R1 csv run 4 graded by llm

Rostval's 2024 works each have one Rostval author among four, so full counting gives it 4 works. Kesting's shared works give it fractional shares totalling 1.75โ€“2.0, which beats Rostval's 1.0. Kesting leads on fractional counting, while Rostval leads on full counting.

Why: Gets Rostval's one-of-four authors and Kesting's larger fractional shares broadly, but doesn't name the single-author review W22 or the half-shares of W20 and W21, and it states a vague 1.75โ€“2.0 range.

๐ŸŸก R1 csv run 5 graded by llm

Full counting gives Rostval 4 vs Kesting 3 because Rostval appears on more 2024 works. Fractional counting divides each work by its author count: Rostval's four 2024 works have one Rostval author each (1.0 total), while Kesting gets 1.0 from the single-authored W22 plus 0.5 each from W20 and W21 (2.0 total).

Why: Gets Kesting's shares right (W22 single-author, 0.5 each from W20 and W21) and Rostval's one author per work, but doesn't state four authors per work, so a quarter each (gives 1.0 total, which matches).

๐ŸŸก R10 executed run 1 graded by llm

Kesting has the most under fractional counting but not under full counting. Under full counting each work counts once, so Rostval's four 2024 works beat Kesting's three. Under fractional counting, Kesting's works have fewer co-authors (W22 is a single-author Kesting work, W20 has two authors), so its shares are larger (2.0 vs 1.0 for Rostval).

Why: Gives Kesting's larger shares (W22 single-author, W20 two authors) but never says Rostval's four works each have four authors with one from Rostval, so a quarter each.

๐ŸŸก R10 executed run 2 graded by llm

Full counting gives each institution one credit per work, so Rostval's four 2024 works beat Kesting's three. Fractional counting divides each work by its author count: Rostval's four works are each four-author papers, giving 0.25 each (1.0 total), while Kesting's single-author W22 contributes a full 1.0 and its other two works contribute 0.5 each (2.0 total).

Why: Gets Rostval's quarter shares (0.25 each) and Kesting's single-author W22 at full credit, but says its other two works are 0.5 each, which matches the half shares for W20 and W21 only loosely; it never names W20 and W21 explicitly. Also, the answer's 'three' works for Kesting implies shares the checklist covers, so the main gap is naming W20/W21.

โŒ R10 executed run 3 graded by llm

They don't swap. In 2024 full counting they tie at 3 each (W21, W27, W29 for I09; W20, W21, W22 for I07), and under fractional counting I07 leads (2.0 vs 0.75).

Why: Denies the swap and doesn't mention Rostval's four-author works at a quarter each or Kesting's larger shares (single-author W22, half of W20/W21).

๐ŸŸก R10 executed run 4 graded by llm

Full counting favours Rostval (4 vs 3) because Kesting has a single-author work (W22) that counts in full. Fractional counting favours Kesting (2.0 vs 1.0) because Rostval's four works each have four authors, so each Rostval share is only 1/4. This is an interpretation: the JSON has no field for Q47.

Why: Gets Rostval's four-author quarter shares right, but credits Kesting's lead only to W22, omitting the half shares of W20 and W21.

๐ŸŸก R10 executed run 5 graded by llm

Full counting gives each work once per institution, so Rostval's four works (W21, W27, W28, W29) beat Kesting's three (W20, W21, W22). Fractional counting weights by author share: Kesting gets 1.0 from W20 and W21 combined plus 1.0 from single-author W22, giving 2.0. Rostval gets only one author in four on each of its four works, giving 1.0.

Why: Gets Kesting's larger shares (single-author W22 plus W20/W21), but says Kesting gets 1.0 from W20 and W21 combined (half each is consistent) while Rostval's quarter-each reasoning is present only as 'one author in four'; it never states that W21 is shared or that the order flips, and its claim that Rostval has four works including W21 is unverified against the key's 'four 2024 works'.