Q47, run 2026-11-04-1752-haiku

← back to the run

Why do Kesting University and Rostval Technical University swap places in 2024 between full and fractional counting?

Tests: Counting method: why the ranking flips.

Expected: Rostval is on four 2024 works (W21, W27, W28, W29), but on each it has one author of four: 4 x 1/4 = 1. Kesting is on three, with larger shares: W22 alone, half of W20, two of the four authors of W21: 2.

🟑 R1 csv run 1 graded by llm

Rostval has more full works (4 vs 3) but Kesting has more fractional credit (2.0 vs 1.0). Rostval's 2024 works each have four authors, only one of them from Rostval, so each contributes 0.25. Kesting's 2024 works have fewer authors per work (W22 has one, W20 has two), so its shares are larger. The data has no separate field for this question, so the explanation is derived from the author counts.

Why: Gives the quarter-share for Rostval and the larger Kesting shares via W22 and W20, but omits W21 and the half-share of W20 and W21 detail; mostly matches both points though less specific

🟑 R1 csv run 2 graded by llm

Full counting favours I09 because it appears on more 2024 works (4 vs 3). Fractional counting favours I07 because Rostval holds only one of four author slots on each of its works (1.0 total), whereas Kesting's shares are larger (2.0 total).

Why: Correctly says Rostval holds one of four author slots per work and Kesting has larger shares (2.0 vs 1.0), but doesn't name W22 as a single-author review or the half shares on W20/W21, and the 'quarter each' is only implied.

🟑 R1 csv run 3 graded by llm

Rostval's 2024 works (W21, W27, W28, W29) each have four authors, so each author gets only 1/4 credit. Kesting's works have fewer authors (W20 has two, W22 has one), so fractional credit favours Kesting.

Why: Gets the four-author, 1/4 credit point for Rostval, and notes Kesting has fewer authors (W20 two, W22 one), but never says only one of Rostval's four authors is from Rostval, and doesn't state Kesting's half of W20/W21 explicitly.

🟑 R1 csv run 4 graded by llm

Rostval's 2024 works have four authors each, so each Rostval author gets a 0.25 share. Kesting's 2024 works have fewer authors (W22 has one; W20 has two, one of them at Rostval's partner I11), so Kesting's shares are larger. Full counting favours the institution with more works (Rostval); fractional counting favours the one with larger shares per work (Kesting).

Why: Gives the Rostval four-authors/0.25 share point and says Kesting's shares are larger, but misdescribes W20 (checklist says Kesting holds half) and omits W22 being a single-author review credited to Kesting in full; the W22 and half-of-W20/W21 detail is not clearly matched.

🟑 R1 csv run 5 graded by llm

Full counting credits Rostval with 4 works in 2024, while fractional counting gives it only 1.0. Each of those works has only one Rostval author among four, so its share is 0.25. Kesting has fewer works but larger shares (3 full, 2.0 fractional).

Why: Gets Rostval's quarter shares (1 of 4 authors each) but only gives Kesting's totals (3 full, 2.0 fractional), not the W22 single-author review and half shares of W20/W21.

🟑 R2 rdf run 1 graded by llm

Rostval has more full works (4 vs 3) because they are spread across all four authors of each work, leaving it only 1/4 per work (1.0 total). Kesting's works have shorter author lists, and it has a sole-author work (W22) and larger shares of W20 and W21, giving 2.0.

Why: Rostval's quarter share per work is right, but Kesting is described only loosely: W22 is not said to be a single-author review, and W20/W21 are not identified as half shares.

🟑 R2 rdf run 2 graded by llm

Under full counting I09 has 4 works to I07's 3. Under fractional counting I07 has 2.0 to I09's 1.0, because I09 gets only ΒΌ of each of its four works, while I07 gets full credit for W22 and Β½ each for W20 and W21. (Q47 and Q48 have no JSON field in the spec, so I have answered them in text only.)

Why: Gives Kesting's shares (W22 full, Β½ each of W20/W21) but says I09 gets ΒΌ of each work without stating that each has four authors with only one from Rostval

🟑 R2 rdf run 3 graded by llm

Full counting ranks Rostval higher because it has 4 works in 2024 versus Kesting's 3. Under fractional counting, Kesting's works have fewer co-authors (W22 has one author, W20 has two), so its authors take larger shares (2.0 vs 1.0). Rostval's four works each have four authors, giving only 1/4 per work.

Why: Gets Rostval's four-author works (1/4 each) and Kesting's W22 single-author and W20 two-author, but omits W21 and says Rostval has 4 works at 1/4 each only implicitly; misses that Kesting holds half of W20 and W21 explicitly.

🟑 R2 rdf run 4 graded by llm

Under full counting, Rostval has 4 works and Kesting 3, because Rostval's 2024 works are all counted once. Under fractional counting, Rostval's four works each have 4 authors with one Rostval author, giving 0.25 apiece (1.0 total), while Kesting's W22 is single-authored and its other shares are larger, giving 2.0.

Why: Rostval's quarter shares are correct, and W22 is named as single-authored, but the answer never says Kesting holds half of W20 and W21, only that its other shares are larger.

🟑 R2 rdf run 5 graded by llm

The swap is real. Under full counting Rostval leads 4 to 3; under fractional counting Kesting leads 2.0 to 1.0. Rostval's four 2024 works are four-author papers with one Rostval author each (0.25 per work), while Kesting's W22 is solo-authored and W20 has two authors.

Why: Gets Rostval's quarter-share per work, but omits Kesting's W21 half share and misattributes W22/W20 detail incompletely (W20 half stated, W21 missing).

🟑 R6 rdf+ontology run 1 graded by llm

Under full counting, Rostval (I09) has 4 works in 2024 and Kesting (I07) has 3, because each credited institution gets one full count per work. Under fractional counting, Kesting has 2.0 and Rostval 1.0, because Kesting's 2024 works have fewer authors per Kesting person. W22 has one author, giving 1.0, while Rostval's four works each have four authors and one Rostval author, giving 0.25 each.

Why: Gets Rostval's four works at 0.25 each and W22 single-author, but omits Kesting's half-shares of W20 and W21 (only says 'fewer authors').

🟑 R6 rdf+ontology run 2 graded by llm

Full counting gives each work whole to every institution on it, while fractional counting splits each work among its authors. Rostval has one author on each of W21, W27, W28 and W29 (1/4 each, total 1). Kesting's W22 is single-author and its W20 is 2-author, so Kesting gets 2 fractionally.

Why: Rostval's four works at 1/4 each is covered (though it lists W21, W27-29 and doesn't state four authors per work explicitly), but Kesting's shares are wrong or incomplete: it says W20 is 2-author and omits half of W21, giving Kesting 2 without explaining the W22 single-author review plus half of W20 and W21.

🟑 R6 rdf+ontology run 3 graded by llm

Full counting favours I09 because it has 4 works in 2024, each credited to one of its authors. Fractional counting favours I07 because its 2024 works have few authors (W22 has one) and I09's 2024 works are split four ways. The swap is therefore a result of how authorship shares are distributed.

Why: Gives the four-way split for I09 and W22 as single-author, but never gives the quarter shares explicitly or Kesting's half shares of W20 and W21.

🟑 R6 rdf+ontology run 4 graded by llm

Rostval is on more 2024 works (W21, W27, W28, W29) but each is a four-author byline, so under full counting it gets 4 works, while under fractional counting it gets only 0.25 of each. Kesting's 2024 works have fewer authors (W22 has one), so under fractional counting Kesting's shares add to 2.0 against Rostval's 1.0.

Why: Covers Rostval's four-author works at 0.25 each and Kesting's W22 single authorship, but never says Kesting holds half of W20 and W21.

🟑 R6 rdf+ontology run 5 graded by llm

Full counting: I09 (4) beats I07 (3). Fractional: I07 (2.5) beats I09 (1.0). Rostval has one author among four on each of its 2024 works (0.25 each), while Kesting's works have fewer authors per Kesting credit (single-author W20 and W22).

Why: Rostval's quarter shares are correct, but Kesting's shares are wrong: it says W20 is single-author, where the key says W22 is single-author and W20 and W21 are half-shares.

🟑 R5 csv+ontology run 1 graded by llm

Under full counting, Rostval gets the credit for every paper it co-authors (four papers), while Kesting's three papers are shared with other institutions too. Under fractional counting, each author's share goes to one institution, so Rostval's authors are only a quarter of each paper (total 1.0), while Kesting's single-author W22 and its share of W20 and W21 give it 2.0. (Q47 and Q48 are not in the JSON.)

Why: Gets Kesting's larger shares (W22 single-author, W20/W21) and Rostval's quarter per paper, but doesn't state that each Rostval work has four authors with only one from Rostval, and says 'each author's share goes to one institution' vaguely; also Kesting's half shares of W20/W21 are not stated.

βœ… R5 csv+ontology run 2 graded by llm

Under full counting, Rostval (I09) has 4 works and Kesting (I07) has 3, because each whole work counts for its institution. Under fractional counting, Kesting's works have fewer authors (W22 has one author, W20 has two), so Kesting gets 2 and Rostval gets 1, as Rostval's four works are each split among four authors.

🟑 R5 csv+ontology run 3 graded by llm

In full counting, Rostval (I09) has more 2024 works (4 vs 3) because it is credited on W21, W27, W28 and W29. In fractional counting, each Rostval author carries only a quarter of a four-author work, so Rostval's share totals 1.0 against Kesting's 2.0 (I07 includes the solo-authored W22).

Why: Gets Rostval's quarter-share per four-author work, but omits that Kesting's shares come from W22 solo review and half of W20 and W21; only mentions solo W22.

🟑 R5 csv+ontology run 4 graded by llm

Rostval has more works under full counting (4 to 3) because it has four 2024 works. Under fractional counting each of those works is split among four authors, so Rostval's total is only 1.0. Kesting's 2.0 includes the single-author W22 (1.0) and its shares of W20 and W21.

Why: Gives four authors per work and a Rostval total of 1.0, but never says only one author is from Rostval. It also says Kesting has 'shares' of W20 and W21 without stating they are half shares.

🟑 R5 csv+ontology run 5 graded by llm

Under full counting Rostval has more works (4 against 3), since it is credited on all four of its 2024 works. Under fractional counting Kesting has more (2.0 against 1.0), because Rostval's authors are one of four or more per work (0.25 each), while Kesting has single-author or two-author works (W22, W20).

Why: Says Kesting has single-author/two-author works (W22, W20) and Rostval's shares are 0.25, but omits W21 and doesn't state clearly that each Rostval work has four authors with one from Rostval; also 'four or more' is imprecise.

βœ… R4 csv+markdown run 1 graded by llm

Full counting gives each work to every credited institution once, so Rostval, with four 2024 works, leads. Fractional counting divides each work's credit among its authors. Rostval's four 2024 works each have four authors, so each gives I09 only 0.25 (total 1.0). Kesting's works have fewer authors per work, so its shares are larger (total 2.0), and it leads.

🟑 R4 csv+markdown run 2 graded by llm

Full counting gives Rostval 4 and Kesting 3, because each work counts whole for every credited institution. Fractional counting splits each work across its authors, so Rostval's four-author works give it 0.25 each (total 1.0), while Kesting's W20 (two authors) and W22 (one author) give it 2.0.

Why: The Rostval quarter-share point is met, but the Kesting part omits W21's half share and wrongly credits W20 and W22 with 2.0 (they give 1.5).

🟑 R4 csv+markdown run 3 graded by llm

The swap comes from how author shares are split. Full counting gives I09 4 works (W21, W27, W28, W29) to I07's 3 (W20, W21, W22). Fractional counting gives I09 1.0 because its three exclusive works each have four authors (0.25 each), while I07 gets 2.0 from shorter author lists, including the single-author W22.

Why: Gets the quarter shares for the three exclusive works and the single-author W22, but never says W21 is also a quarter for Rostval or that Kesting holds half of W20 and W21.

βœ… R4 csv+markdown run 4 graded by llm

Full counting gives each credited institution a whole work, so Rostval (on W21, W27, W28, W29) edges out Kesting (W20, W21, W22). Fractional counting splits each work among its authors, and Rostval's four papers each carry only one of its own authors (0.25 each, 1.0 total), while Kesting has a single-author paper (W22, 1.0) plus shares in W20 and W21 (2.0 total).

🟑 R4 csv+markdown run 5 graded by llm

The two counting methods order them differently because of author team size. Under full counting Rostval has 4 works in 2024 (W21, W27, W28, W29) against Kesting's 3 (W20, W21, W22). Under fractional counting each of Rostval's four works has 4 authors, so each gives 0.25 (1.0 in total), while Kesting's W22 has a single author and gives it 1.0, leaving Kesting at 2.0.

Why: Gets the quarter share for Rostval's four works and Kesting's single-author W22, but never says Kesting holds half of W20 and W21 (its 2.0 total is only implied).

🟑 R3 csv+padding run 1 graded by llm

In full counting, every work counts for each institution with an author on it, so Rostval's four 2024 works (each with one Rostval author) beat Kesting's three. In fractional counting, each work is divided by its author count. Rostval gets only 1/4 per work, while Kesting's single-author W22 gives it a full unit and its two-author W20 gives it half.

Why: Gets Kesting's shares right (W22 full, half of W20/W21 per W22 and two-author works) but says Rostval gets 1/4 per work without stating that each work has four authors, and its framing is basically consistent; it omits that explicit four-author detail and the W21 half

🟑 R3 csv+padding run 2 graded by llm

Full counting gives Rostval 4 (W21, W27, W28, W29) against Kesting's 3 (W20, W21, W22). Fractional counting gives Rostval 1.0, because its three four-author papers each give it 1/4, while Kesting gets 2.0 from W20 (1/2), W21 (1/2) and single-author W22 (1).

Why: Gives Kesting's shares (W22 single-author, half of W20 and W21) and says Rostval gets 1/4 on its papers, but says only three four-author papers, not four as the checklist requires, and lists W21 as a Rostval work with Kesting getting 1/2 of it, which conflicts with the 'four works, quarter each' point.

βœ… R3 csv+padding run 3 graded by llm

Full counting ranks Rostval first (4 vs 3) because it has more 2024 works. Fractional counting ranks Kesting first (1.75 vs 1.0) because Rostval's 2024 works are four-author papers, so each Rostval author gets a 0.25 share. Kesting's works have fewer authors, so each share is larger. Counting Kesting's I08 unit (Q12) raises its fractional total to 2.0, which does not change the ranking.

🟑 R3 csv+padding run 4 graded by llm

(No schema fields are defined for this question, so it is answered in text only.) In full counting Rostval has 4 works in 2024 and Kesting has 3, since each work counts once per institution. In fractional counting, Rostval's 2024 works each have four authors with one Rostval author, giving 1.0, while Kesting's W20, W21 and W22 contribute 2.0, so the order reverses.

Why: Rostval's quarter-share is covered (four authors, one from Rostval, total 1.0), but it never says W22 is single-author or that Kesting holds half of W20 and W21; it only gives the 2.0 total.

βœ… R3 csv+padding run 5 graded by llm

Full counting gives each institution one credit per work it appears on. Rostval appears on 4 works, each with 4 authors, so its fractional share is 0.25 per work (1.0 total). Kesting's 3 works include a single-author paper (W22) and two papers where it holds 2 of 4 author slots, which gives 2.0. Full counting therefore favours Rostval, and fractional counting favours Kesting.