Tuesday, 7 February 2017

More boring reviewing stats

Present Perfect has reviewed Through the Well of Pirene. This is bad news, since I'd decided in my head that the story was so bloody long that I was only going to tackle it if it got 5/5. And you can guess what happened. So, at some point this year (I'm not being more specific than that!) it will become the second-longest ponyfic I've reviewed. One for the e-reader in the garden this summer, I think.

Anyway, on to the promised stats. Let's look at the star ratings I've awarded over the years, with the old out-of-ten scores converted in the usual way (grade boundaries at 4, 6, 8 and 9). Out of the 734 stories I've reviewed up to PR 143 (inclusive), this is the breakdown:

★★★★★: 34 (4.63%)
★★★★: 153 (20.84%)
★★★: 282 (38.42%)
★★: 199 (27.11%)
★: 66 (8.99%)

Some thoughts on this, plus more rambling, past the break.

It doesn't surprise me that three-star stories have been the most numerous. Unless I have a reason to do otherwise, I select fics to review that look as if I might like them, and a good, solid story that I enjoy and feel is a little above the usual run, without being a real stand-out, will get three stars.

The distribution skews towards the lower end, which again is unsurprising: a story that I find okay but not memorable, or one that has flashes of potential but also significant flaws, usually gets two stars. I've read more just-okay ponyfics than I have excellent-but-not-staggering ones (which get four stars).

My own hunch was that I've become tougher with my grading as time has gone on. To test this, I split the first 140 editions of PR into four equal periods of 35 each. These account for a total of 709 reviews. I reviewed significantly more fics in the third period, with the other three being almost equally split. Anyway, same format as above:

PRs 1-35 (167 reviews)
★★★★★: 16 (9.58%)
★★★★: 39 (23.35%)
★★★: 75 (44.91%)
★★: 28 (16.77%)
★: 9 (5.39%)

PRs 36-70 (168 reviews)
★★★★★: 6 (3.57%)
★★★★: 43 (25.60%)
★★★: 65 (38.69%)
★★: 44 (26.19%)
★: 10 (5.95%)

PRs 71-105 (204 reviews)
★★★★★: 9 (4.41%)
★★★★: 34 (16.67%)
★★★: 75 (36.76%)
★★: 61 (29.90%)
★: 25 (12.25%)

PRs 106-140 (170 reviews)
★★★★★: 3 (1.76%)
★★★★: 32 (18.82%)
★★★: 57 (33.53%)
★★: 58 (34.12%)
★: 20 (11.76%)

It's pretty clear that I have indeed become harsher with my marking over time: the proportion of one-star stories doubles after period two, while that of five-star stories drops significantly twice, after the first and third periods. A story like Stereo_Sub's RUN, which got four stars in PR 110, would probably have been a comfortable five in the early days. Two-star ratings have also (just) overtaken three-star ones.

One reason for this increasing toughness is obvious: now that I've read 700-plus stories, it's harder to blindside me with something special than it was when I was just starting out in the reviewing game. I also think I've become a little more confident in giving out low scores when I feel I need to, rather than tending to err on the side of generosity because, hey, ponies.

Finally, a quick "average star rating" index for each period. This is produced simply by multiplying each star rating by its associated percentage, then adding the five together. This gives the following:

Period 1: 3.15
Period 2: 2.95
Period 3: 2.71
Period 4: 2.65

A fairly even drop between periods 1-2 and periods 2-3, but a much smaller one between periods 3-4. This may mean that my criteria, subjective as they are, have stabilised. I'd like to think so, anyway. I wouldn't be at all surprised if "Period 5" (ie PRs 141-175) ends up producing the first rise, but there's a long way to go until I get there!


  1. STATS ARE NEVER BORING >:V And either you've become harsher, or it's getting harder to find good fics.

    Also, wahahaha. :D

    1. I don't think it's likely to be that it's harder to find good fics. You're still managing, and you've reviewed 4,000 of the things! Plus, there are plenty by reliable authors that I haven't so much as looked at yet. So though I can't prove it, I think it is much more likely to be increased toughness.

      And indeed. Stats are great. I just couldn't think of an interesting title for this post. :P