Credibility of Audio Reviews

[LarsS]

How do we or better do we evaluate/rate and judge the credibility, quality and possible bias of audio equipment reviews. Many of us that sometimes don't have the possibility to hear and evaluate for themselves use reviews/opinions as basis for purchase of audio equipment.

I have in storage, awaiting my son to move into his own place, a pair of MMMicroOnes w. stand, I did like them somewhat but not overly so. At best I'd call them good stand mount transducers, for my son they will be killer speakers.

Anyway, having today read the review below (and the MAXIMUM MOJO award) of the Evolution Acoustics MMMicroOnes

- - > http://stereomojo.com/Evoultion Acoustics MMMicroOne review/EvoultionAcousticsMMMicroOnereview.htm

which in my mind does not reflect my real world experience of the speakers.

Anyway, this post is not about EA MMMicroOnes, more about reviews, their credibility and impact on consumer behaviour i.e. purchasing a specific equipment. I have no idea how to address the subject right now but I'm sure there's ideas out there.

/BR
Lars
(not native English)

 

[still-one]

I don't use reviews for purchase decisions. I use them to get information such as specs, what sandbox them are trying to play in, general capabilities, and how they look. I prefer to buy locally so I can audition gear in my home. In the past when necessary have flown to hear products I am rally interested in. I have sat next to reviewers at audio shows who wax poetically about what I thought was really crappy sound. It just means that my ears are what I have please. Bottom line I like to read reviews but they have little impact on purchase decisions.

 

[LarsS]

I don't use reviews for purchase decisions. I use them to get information such as specs, what sandbox them are trying to play in, general capabilities, and how they look. I prefer to buy locally so I can audition gear in my home. In the past when necessary have flown to hear products I am rally interested in. I have sat next to reviewers at audio shows who wax poetically about what I thought was really crappy sound. It just means that my ears are what I have please. Bottom line I like to read reviews but they have little impact on purchase decisions.

You're in a fortunate situation to be able to listen to whatever equipement there is, which I and many others aren't. I'll give you a couple of examples of my recent purchases which I've done partly on reviews:
Gauder Akustik (ex Isophon) Arcona 100 Speakers, Norma Revo IPA-140 amp aso

However the question is about the reviews or so called reviews

 

[Johnny Vinyl]

I do use pro reviews, but certainly not exclusively. I want user reviews and opinions from audiophiles and hopefully a chance to audition any unit for consideration. The latter is becoming more and more difficult however. Ultimately I am the deciding factor and so far I've had good judgement.

 

[edorr]

I use reviews (professional and user) to compile shortlists of stuff that I am intrigued abot hearing. I look for the piece to shows up used at a price I can resell at with no (or small) loss in case I don't like it. As for credibility, the more experience I acquire listening to actual gear and reconciling with reviewers conclusions, the less credibility is left. See my comments on TAS review of Axpona rooms.

 

[LarsS]

I use reviews (professional and user) to compile shortlists of stuff that I am intrigued abot hearing.

That's fine if you have the possibility to listen live to selected equipment, which many of us reading reviews don't have.

Still, question remains, should there be somekind of "Morning Star" rating of reviewers. I'd vote for that.

/BR
Lars

 

[rblnr]

Anyway, having today read the review below (and the MAXIMUM MOJO award) of the Evolution Acoustics MMMicroOnes

- - > http://stereomojo.com/Evoultion Acoustics MMMicroOne review/EvoultionAcousticsMMMicroOnereview.htm

which in my mind does not reflect my real world experience of the speakers.

/BR
Lars
(not native English)

Then you know that reviewer is not for you -- it's a useful thing when you've heard something that's been reviewed. Look at reviews of stuff you're familiar with to get an idea of the reviewer. There are a couple of reviewers who's opinion I respect, and many who I don't get at all. Also, IMO, most don't write well. Something I like about the English mags vs. the US ones is that they directly compare stuff. The domestic mags seem afraid to.

Hey, it's not so hard to get to be a reviewer -- I was one for many years -- so why someone would make a buying decision based on someone they don't know anything about or have no way to reference their opinions is beyond me.

 

[edorr]

Now, how do you do that, interested to know what selection critierias used to select which reviews, based on what compared to what (I know difficult questions, maybe costly trial and error as with me ...)

Lars

No selection criteria - waste of time in my opinion. It is all so subjective. More a "wisdom of crowds" principle. The more people (including professional reviewers) comment positively on a product, the higher the statistical probability the product has some qualities I also appreciate (I'm not that special).

 

[LarsS]

No selection criteria - waste of time in my opinion. It is all so subjective. More a "wisdom of crowds" principle. The more people (including professional reviewers) comment positively on a product, the higher the statistical probability the product has some qualities I also appreciate (I'm not that special).

My take is that the "wisdom of crowds" is being manipulated and not to be trusted anymore as well as many so called independent professional reviewers.

 

[still-one]

No selection criteria - waste of time in my opinion. It is all so subjective. More a "wisdom of crowds" principle. The more people (including professional reviewers) comment positively on a product, the higher the statistical probability the product has some qualities I also appreciate (I'm not that special).

In general I agree with you. I would suggest if you are going to use the Audio Sites along with written reviews then you should cast a wide net. Most people hang out on sites where individuals share common beliefs other wise they are viewed as trolls. So don't go looking for great reviews of top dollar components on a site where they masses proclaim the virtues of 1970's and 80's gear or their recent find at a Salvation Army store.

 

[Johnny Vinyl]

In general I agree with you. I would suggest if you are going to use the Audio Sites along with written reviews then you should cast a wide net. Most people hang out on sites where individuals share common beliefs other wise they are viewed as trolls. So don't go looking for great reviews of top dollar components on a site where they masses proclaim the virtues of 1970's and 80's gear or their recent find at a Salvation Army store.

But don't assume that when reading those reviews on sites that cater to the high-end audiophile makes them more credible, because that is often not the case. You have to know the reviewer. Some you just ignore.

 

[LarsS]

... I would suggest if you are going to use the Audio Sites along with written reviews then you should cast a wide net. Most people hang out on sites where individuals share common beliefs other wise they are viewed as trolls.

?
My point (one of them) is that the "wide net" incl individual users (who are they anyway) and professional reviewers are not to be trusted just like that anymore without ....what ... that's the question. Could be some kind of "Morning Star" rating performed by readers, professionals aso

 

[still-one]

But don't assume that when reading those reviews on sites that cater to the high-end audiophile makes them more credible, because that is often not the case. You have to know the reviewer. Some you just ignore.

I agree. Just to be clear I was trying to say more often than not Birds of a Feather do Flock Together. Just as an example Wilson owners usually don't hang out in Magico threads or visa versa. Same for vinyl and digital crowds.

 

[Johnny Vinyl]

I agree. Just to be clear I was trying to say more often than not Birds of a Feather do Flock Together. Just as an example Wilson owners usually don't hang out in Magico threads or visa versa. Same for vinyl and digital crowds.

Agreed!

There are several known reviewers who in my opinion have too strong a bias towards certain products or formats that makes me question the words they write (I won't say integrity, because that I think is overstepping a boundary). I still read those reviews, in the hope they've had an epiphany.

 

[TheMadMilkman]

How do we or better do we evaluate/rate and judge the credibility, quality and possible bias of audio equipment reviews. Many of us that sometimes don't have the possibility to hear and evaluate for themselves use reviews/opinions as basis for purchase of audio equipment.

I have in storage, awaiting my son to move into his own place, a pair of MMMicroOnes w. stand, I did like them somewhat but not overly so. At best I'd call them good stand mount transducers, for my son they will be killer speakers.

Anyway, having today read the review below (and the MAXIMUM MOJO award) of the Evolution Acoustics MMMicroOnes

- - > http://stereomojo.com/Evoultion Acoustics MMMicroOne review/EvoultionAcousticsMMMicroOnereview.htm

which in my mind does not reflect my real world experience of the speakers.

Anyway, this post is not about EA MMMicroOnes, more about reviews, their credibility and impact on consumer behaviour i.e. purchasing a specific equipment. I have no idea how to address the subject right now but I'm sure there's ideas out there.

/BR
Lars
(not native English)

I clicked on the link, then remembered that StereoMojo is the site that likes to Photoshop lingerie models onto their stock photos of speakers and cables. That alone is enough to make me ignore anything they write. I'm not being prudish; I just expect professionalism.

A reviewer earns my trust by writing intelligibly, describing sound quality in a way that I can verify with my own listening, and not having any obvious biases toward any particular company.

Robert Reina earned my trust. Stephen Mejias was never able to. I use these two as examples because neither is actively reviewing (may Robert RIP).

Even with Robert having been trusted source does not mean I buy things blind. But I will seek out and listen to equipment based on his reviews. In general terms, that's about the most you can expect.

Do reviewers have an effect on sales? I could see a positive review causing a short-term spike in interest and inquiries, but you need to remember that high-end audio is a very small niche market. And most audiophiles really don't swap gear out as often as we pretend they do. It's just that the ones who do are a very vocal minority that eat up a lot of bandwidth on the varying forums.

I think the best indication of how little effect the audiophile press really has comes from Jason Stoddard (co-founder of Schiit Audio), from here: http://www.head-fi.org/t/701900/sch...s-most-improbable-start-up/1965#post_10773898

A great review and being on the cover of an audio magazine led to maybe a 10% bump in sales of their Bifrost DAC. A mention in the New York Times that the DAC was the crowd favorite among 5 tested literally flooded them with more orders than they were equipped to handle.

Stereomojo liking a speaker that you didn't, even if there was some behind-the-scenes impropriety? Probably completely inconsequential.

 

[DaveyF]

I think, in some ways, one has to feel a little sorry for the poor pro audio reviewer. Here's this guy ( not sure why it is seemingly NEVER a woman...BUT that's a question for another thread) who has to be cognizant of many issues...1) who is his expected reader? 2) how much credibility will he lose IF he says something bad about a product and the readers disagree with him?...3) the reverse of #2, 4) Should he be completely truthful and disclose his relationship with the manufacturer who's gear he has in for review...which could lead to no more gear to review from said manufacturer.or 5) Should he disclose his relationship and the "accommodation" that he is getting and therefore insure that his reader understands wherein his opinion and review is coming from.
I'm sure we can add more points....BUT I 'm sure we are getting the jist right now.
Credibility of the reviewer...perhaps it depends on how we understand the word "credible".

To the OP, the fact that you cannot listen to gear in your own listening environment is a MAJOR detriment IMO. I think the only safe way to proceed in this case is to make sure that you have great return privileges if for any reason the gear fails to satisfy. A tough proposal for most dealers and manufacturers....BUT perhaps your only option.

 

[bobauch]

none

My first system was based mainly on reviews. The speakers had great specs .I bought supposedly good quality amplification and my first good CD player.
That system has been replaced with a smaller,higher quality one. I have one reviewer I find particularly trustworthy,Steve Guttenberg,as he seems to find great sound in my price range,and we share a love for Magneplanars.
Another reviewer refers to a 2000 dollar phono stage as "budget" and a "great bargain".
ON WHAT PLANET?
Now I read user reviews and practically everything I can get my hands on.I look for how the item works in systems similar to mine, and look for similar responses over a variety of threads.But most of all,I deal with a local dealer I trust,and a local repairman associated with him .The repairman especially has been helpful in separating reality from hype,and how to make sure components match well.I make no snap decisions,and I trust my ears.
The longer I have my current system,the more I like it.Enough said.

 

[Andre Marc]

My first system was based mainly on reviews. The speakers had great specs .I bought supposedly good quality amplification and my first good CD player.
That system has been replaced with a smaller,higher quality one. I have one reviewer I find particularly trustworthy,Steve Guttenberg,as he seems to find great sound in my price range,and we share a love for Magneplanars.
Another reviewer refers to a 2000 dollar phono stage as "budget" and a "great bargain".
ON WHAT PLANET?

"My first system was based mainly on reviews. That system has been replaced with a smaller,higher quality one."

Give it a rest. Don't try to blame anyone but your self for making poor choices. I have auditioned, in my home, every component and speaker I have ever purchased.

I have ever piece of gear I have ever owned still, save for a pair of Rogers LS35A's I beat to death and an a ten year old ARC preamp.

We live in a society where no one wants to make a decision for themselves so they have someone else to blame for their fuk ups.

 

[bobauch]

Sorry Mark but I'm new to the forum and hit the button prematurely.My first system was over 20 years ago, I didn't know what I was looking for and bought the wrong stuff based on reviews. That was my fault. I did not intend to blame others.
That being said,we also live in a society where others are quick to jump down people's throat and throw the "f"word around like candy.
Supposedly neither of those is in the spirit of this forum.

 

[JackD201]

I used to read reviews on magazines and sites regularly. Over that long period of time I was to a limited extend able to find correlations between findings of reviewers with my own. In the process, there are reviewers who have mirrored what I experienced. For example theres a lot of commonality between what I've liked and what guys like Sircom, Robinson, Learner (our Bob), Taffel, Mickelson have liked. Some commonality with Harley, Astor, HP and Fremer. Less with for example with Dudley on one end and Valin on the other, Atkinson and Fritz somewhere in between. Out of courtesy I will not mention those who have had me scratching my head or in the popular vernacular have had me say, "whaaaaaaaaaaat?" in a high pitch.

All this based on stuff I have heard myself. The commonality is based only on what gear I have personally heard for myself and also I think the language they use may play a part, even a big part of this. Granted all these correlations are probably weak ones at best, but when it seems I already have a feel for their respective preferences vs my own, despite the differences between their review set ups and my own, stuff becomes maybe worth investigating. That's not to say that there aren't people who I trust enough to fire off a P.O. based solely on their word. There are maybe 3 people in the world I trust to that extent. Sometimes I wish they were reviewers but they are not. One of the isn't one anymore the other two never were.

IMO if one doesn't have a feel for a specific reviewer then taking that person's opinion would be making to swinging at a piñata blindfolded. I don't think it is so much a lack of integrity but rather differences in thinking.