Academia - Case Study: First Answer Bias

It's not something we can solve as academia.SE.

As you noted, it's been a known thing for many years over at meta.stackexchange.com (Fastest Gun in the West Problem). They even have a likely solution noted there, which is what Reddit adopted some years back. I imagine there must be counterarguments as to why that isn't a great idea. But regardless, so far as I know it isn't something that Stackexchange is planning to deal with :-(


Some of this seems not to be "first answer bias".

You say that when your question became "accepted", then your answer started quickly accumulating more upvotes at a faster rate than buffy's, even though it was the other way around before. This seems to be like a "top answer bias".

I disagree with the other answer, which says that "it's not something we can solve as academia.SE". I believe this has been already fixed on Meta.SE where instead of sorting the questions by the number of votes (which I think is the default here):

enter image description here

the questions can be sorted by most recently active:

enter image description here

It's possible that MSE also has "votes" set as default and I just changed it some time long ago to "active" without noticing, but either way we could easily make "active" the default instead of "# of votes" so that the first answer (or most voted answer) doesn't get an unfairly disproportional number of upvotes. We could even make a new one called "random".

I do think these things would improve the site, because I find the "chain-reaction voting" to be a much bigger issue here than on any other site I'm active on (I have 1000+ rep on 7 sites and 150+ rep on 26 sites, and I created one site currently in Beta from scratch in Area51: so I've been on SE almost 24/7 for a while, as anyone who lead the launch of a site would be able to appreciate). I recently wrote a Meta post here about "chain-reaction voting" (on Academia.SE) too: How do people here feel about chain-reaction downvoting of posts?.

Let me conclude with my opinion about why "top answer bias" (not necessarily "first answer bias") happens:

I can at least speak from my own experience on Academia.SE: Recently I read an answer by BryanKrause for which I commented "this might be one of the best answers I've ever seen on SE, and I've saved it somewhere for me to re-read over and over again later", and I upvoted. However it was a long answer with a lot of substance, which I read slowly and nodded my head to the entire time. By the end of reading the post and reflecting so much over it, I remember being too exhausted to read all the other answers with the same amount of attention. I just double checked and it's true that I didn't upvote the 2nd or 3rd listed questions either!

Often people read the first answer they see, and they might put a decent amount of energy into processing it or commenting on it, then they move back to whatever they were working on or whatever SE site they were on before being the HNQ list drew them here. If the answers were set by default to "random" or "active" instead of "votes" or "oldest", then the "first answer bias" or "top answer bias" could be eliminated, without too much disadvantage.

Tags: