Or more precisely, how well does the average rating of a book on Goodreads predict my own rating of that book?
I joined Goodreads not too long after starting this blog, mostly as a way of keeping a list of possible books to read for my book-from-every-country project. My initial assumption, like a lot of people, I think, was that the star rating they give to books wouldn’t really tell me anything about whether I would like a book. After all, those ratings are averages of so many people with such different tastes. (And surely many of those people have bad taste!) Plus I’m skeptical even of my own ability to quantify my opinion of a book on a scale from 1 to 5, which is why I never include ratings in my blog.
Being a science-type person though, it occurred to me to test it out. I blinded myself to Goodreads ratings for the time that it took me to read 50 books from my Goodreads list. These were mostly books from my book-from-every-country project or other ‘global’ type reading. I rated those books periodically, trying to follow the Goodreads scheme of ‘It was okay’, ‘I liked it’, ‘It was amazing” etc. When I got done with 50 books, I took a look at the data.
Here’s a graph of the results:
It turns out that there is actually a statistically significant correlation (two-tailed linear regression, p=0.03) between the average Goodreads rating and my own opinion! It’s not exactly a tight relationship though. The R^2 value of 0.09 means that variation in the Goodreads rating only accounts for about 9% of the variation in my rating. Still, there’s enough of an effect that in the future if I were trying to choose between two books that looked interesting to me and one had a 3.5 rating on Goodreads and the other had a 4.0 rating, I’d go with the 4.0. I wouldn’t not read a book just because it had a 3.5 rating though.
It’s notable that all of the Goodreads ratings for books I read fell into a fairly small range, from The Tiger’s Wife rated 3.36 (Why on earth is it rated that low?!) to Persepolis and Poor Economics which tied at 4.20 (totally deserved!). I think that range is probably typical of most reasonably successful books*. If my reading was a random sampling of the books on Goodreads, including really obscure, badly written books (not that all obscure books are badly written!), I suspect that Goodreads ratings would have better predictive power. (Anyone willing to read 50 completely random books to test this? I think I’ll pass.)
The book that comes out as most under-rated on Goodreads (as compared to my own opinion) is Ali and Nino by Kurban Said. It seems that a movie version is in the works. It will be interesting to see if that boosts the ratings. The book that according to me is most over-rated is Bonsai by Alejandro Zambra. I think that one comes down to personal taste. It’s apparently quite a good book. It just didn’t happen to appeal to me at all.
Well, what about other sites with book ratings? I decided to look at the ratings on Amazon and LibraryThing for comparison. One thing I immediately noticed was that the ratings on Amazon were noticeably higher than for the other two. (The data also look boxier because the Amazon ratings are rounded to the nearest tenth.)
Maybe the higher ratings are related to Amazon being a shopping site? Maybe you’re more likely to bother to rate books you really liked there? Once you account for the higher ratings though, Amazon ratings were about as successful at predicting my ratings as Goodreads’ were.
Interestingly, LibraryThing seems to do a somewhat better job of predicting my opinion than either of the other two. That surprised me because in most cases the books had fewer ratings on LibraryThing than on Goodreads. I wonder whether the community on LibraryThing might have more similar tastes to mine? (Or it could just be chance. I haven’t actually done the statistical test to determine if LibraryThing was significantly better than Goodreads.)
So, to conclude, Goodreads’ isn’t bad, but LibraryThing might be better (for me anyway).
* In the course of writing this post, I came across a Goodreads list of books with ratings over 4.5. At first glance, two types of book were highly represented at the top of this list.
- Books in series. My guess is that if you aren’t that excited by Robert Jordan, you probably aren’t going to make it to book #14. Heck, Robert Jordan himself didn’t make it to book 14.
- Calvin and Hobbes. Everyone loves Calvin and Hobbes.