I was told this recently & it makes ultimate sense:
This whole issue boils down to what level of error is acceptable for the specific target audience: the editor of a peer-reviewed journal; oneself; oneself with a group of friends; the writers on forums; the readers on forums. So different levels of listening rigour are required to convince different target audiences.
For oneself, many of us are quite happy to listen normally & not worry too much about the possibility of fooling ourselves. If making purchasing decision we usually listen over a week or longer & involve friends & others in this listening.
For peer-reviewed journal a rigour is required which is demanded by whatever their criteria is for article acceptance.
There's really no problem with understanding the requirements of these two widely different target audiences - it's the forum groups where the difficulty arises. Most people on forums are happy to relate their personal experiences of their listening impressions. We all know that these impressions are as liable to error as our own impressions & we judge them accordingly. Paying attention to the consensus & also using our common sense to determine if this consensus is from a group of biased fanboys or from a wider audience who whose expectation bias is not as strong. To me, all this talk about being biased in our listening is about psychological expectation. We recognise those more likely to have heightened expectations' like Amir's example of the MS codec, like fanboys, like those who have a psychological stake in the game.
On forums that haven't banned mention of DBTs, we often find a group who seemingly aren't happy to allow these "error prone" listening impressions & often interject with an insistence for more rigour. This usually takes the form of ABX blind listening tests & measurements. So, I guess the question is why is there an insistence on this "to convince others on forums, then it is required work. Fun or not." as Amir says?
BTW, you never eliminate expectations - it's accepted that auditory perception (as indeed other perceptions too) operate two modes of processing - top-down & bottom-up processing. Top-down processing is where the brain is constantly interpreting the incoming signals & doing a best-fit to a model that it has already stored - anticipating (expecting) the next aspect of the sound that will appear in the auditory stream. This expectation can make us hear what isn't there, yes even when we are blind listening- for instance we can often fill in gaps in speech, we can fill in elements of soundscapes which aren't actually there.
Blind listening is only a way of removing knowledge which may skew our expectations, at a global level but we are continually making guesses (have expectations) at the detail level about what's coming next in what we are hearing & comparing the bottom-up signals to a stored model & often replacing the model with a new one, etc. - it's the nature of how perception works!
