The reviewer's reviewing system.

[DaveyF]

Jack, I think it is pretty obvious that some rooms will sound poor based on several factors that can be obvious to anyone who looks at a decent photo of that room. A couple of years ago, I attended an audio meeting at a large church hall room. The sound was very poor in this room, due to the hard linoleum flooring, the large combined spaces ( which were cordoned off with large metal screens) and the fairly barren areas on the walls...which were made up of block and some drywall. Interestingly, the whole a'phile group, with no exception, expected the room to sound just exactly as it did when they were shown a photo of the space prior to the meet. So, yes..I believe that any a'phile with a little understanding of room acoustics can indeed look at a photo and get a pretty good idea as to what said room will sound like if there are obvious potential issues visible. Is this 100% fool-proof, no...and nobody is stating that.

 

[JackD201]

Remember Davey that I have been in that very room. The sound was not poor at all, in fact the sound was excellent.

I must admit that going in I was skeptical but experience taught me long ago that photos can be a lot like book or album covers so really I wasn't all that surprised. The wide Magico baffles as well as stats work in environments like these.

 

[DaveyF]

Remember Davey that I have been in that very room. The sound was not poor at all, in fact the sound was excellent.

I must admit that going in I was skeptical but experience taught me long ago that photos can be a lot like book or album covers so really I wasn't all that surprised. The wide Magico baffles as well as stats work in environments like these.

Ok. But here's a question for you Jack, do you think ( is it possible?) the speakers could have performed a LOT better in a room that is far superior from an acoustic stand point? Or, are you saying that the acoustics in this room were so great that they allowed the speakers to sound their best.

 

[JackD201]

They could be better in a different room yes but that would also change the listening distances and frankly going from near to mid field has its own trade offs. The thing is there was no point at which I felt the speakers (S5s) needed more room to breathe. I know a Von Schweikert VR-5 would not work in that room given the latter's dispersion pattern that demands at least 8 feet of separation. I would say this is a case of a speaker design matching well with the owner's circumstances. It is also consistent with that reviewer mostly reviewing electronics and carts and not speakers. He is full aware of his limitations with regards to speakers.

Were the acoustics of that room all that great? Well, people were working on the street outside and I could hear them so not so great a rating on sound transmission. As for inside, did the room ring? No. Was it quiet enough to listen with 90dB peaks and probably sauntering along in the 50s and 60s without the room being bothersome? Yes.

Remember that the whole point of going nearfield is to eliminate the room as much as possible in the first place right? It shouldn't be all that surprising then. What the pictures don't show is that the send and return path to the racks and windows is quite long. The speakers are actually more into the room than I had thought coming in. This long path puts reflected sound way outside the Haas time window and also drops in dB substantially by the time it gets back to you. As for the side walls there are small panels there. They were enough. I certainly could tell you what was going on in the recordings and that the recordings were hardly homogenized in one way or another at all while listening through many genres, labels and also Tape. If I could discern these differences so easily it stands to reason that differences in equipment changes shouldn't be too difficult either if I had enough time to get intimately familiar with the system.

My interest here is not the reviewers but acoustics in general. More and more I see posts about shows where bloggers will stick their heads in and say no room treatment, it must suck so it does then they trash the place and move along. Ummmm. Wait a minute. Now here is a case of seeing a picture and saying that cannot possibly work, well it does. I enjoin anyone to go to open houses of new developments and compare the acoustics of bare units to that of the model units. You'd be surprised just how much difference everyday items make.

A third of our projects start literally before the homes break ground. The other third we deal with emptied rooms. The remainder are spaces where acoustic work has already been done or needs special attention/solutions mainly because of WAF. As such it's not like I'm saying just plop your stuff down just anywhere. That's the other slippery slope on the other side of the hill. Best not to confuse the two. This slope we're on is the one that says you NEED to alter your environment substantially where the truth is, adapting to your environment is just as important in terms of the final outcome. Doing the latter results in less alteration.

 

[DaveyF]

Were the acoustics of that room all that great? Well, people were working on the street outside and I could hear them so not so great a rating on sound transmission.
.

That alone would discount the room for me. That would be my personal opinion...probably/possibly not so bothersome for others.

 

[JackD201]

It's actually a nice, quiet part of Manhattan not too far away from Central Park. He was complaining that the road repair was taking forever. I imagine with that done it wouldn't be that big of a deal.

A storm just hit here so my pumps were running and noise (fortunately not water! ) was leaking in. Yeah, it can be a hassle but I manage to listen through it. Who would have thought I'd need to treat my friggin cistern? LOL.

 

[Al M.]

As an acoustician myself I always find it funny how people can look at pictures of a room and make snap assumptions. One must remember that lenses can be very deceiving when it comes to translating proportions. As everyone should know the behavior of reflected sound is not just affected by the qualities of the boundary surface but also by the distance travelled by the sound wave. Furthermore, a picture only shows a small part of a room. One might say, oh no bass traps so it must be boomy but in fact the room might be opening up elsewhere preventing bass build up, something also very reliant on the levels listened at. For all you know the guy's got a door or some windows open somewhere.

Treatment can be done in many ways including the use of every day items as everything including the listener and his clothing has an absorptive coefficient at differing frequencies. Make no mistake, even diffusers which are typically made with dense materials have absorptive properties.

That is certainly true. I had to heavily treat my room (see my system thread linked in signature, pages 1 and 2) to get rid of all kinds of nasties, create a great soundstage and to get good resolution, both on the level of separation of instruments in space and on the level of timbre.

On the other hand, the best sound that I have heard, with enormous timbral resolution and accuracy, is in a room that is barely treated, and with a large glass door in the back (while my own windows substantially stood in the way of timbral resolution, and are blocked by ASC window plugs). That room does have a carpet floor from natural fiber (better than synthetic) and an opening to the left, so bass can escape. No doubt the soundstage in that room could be improved by acoustic treatment, but that is a different matter. The sound itself is stellar.

 

[DaveyF]

That is certainly true. I had to heavily treat my room (see my system thread linked in signature, pages 1 and 2) to get rid of all kinds of nasties, create a great soundstage and to get good resolution, both on the level of separation of instruments in space and on the level of timbre.

On the other hand, the best sound that I have heard, with enormous timbral resolution and accuracy, is in a room that is barely treated, and with a large glass door in the back (while my own windows substantially stood in the way of timbral resolution, and are blocked by ASC window plugs). That room does have a carpet floor from natural fiber (better than synthetic) and an opening to the left, so bass can escape. No doubt the soundstage in that room could be improved by acoustic treatment, but that is a different matter. The sound itself is stellar.

Al. I think the best rooms are many times the one's that are so great sounding they simply don't need anything in the way of treatments. I know the best rooms I have heard had no room treatment at all, the room simply worked.However, none of these rooms had anything that would lead one to suspect any kind of problem....no hard surface floors, no glass areas in the sound field,etc.

 

[Al M.]

Al. I think the best rooms are many times the one's that are so great sounding they simply don't need anything in the way of treatments. I know the best rooms I have heard had no room treatment at all, the room simply worked.However, none of these rooms had anything that would lead one to suspect any kind of problem....no hard surface floors, no glass areas in the sound field,etc.

You may have a point. On the other hand, the best room that I have ever heard, with the best combination of soundstage and sound quality, is the large room at Goodwin's High End that is quite extensively treated, with built-in treatment in fact.

And as I said, the room where I heard the best sound I have ever experienced (see above) could clearly benefit from treatment when it comes to soundstage.

I haven't heard the system that is in that room in Goodwin's large room. With such a system or similar the sound itself there might rise to the best I would have heard at that point as well.

 

[Purite Audio]

Al. I think the best rooms are many times the one's that are so great sounding they simply don't need anything in the way of treatments. I know the best rooms I have heard had no room treatment at all, the room simply worked.However, none of these rooms had anything that would lead one to suspect any kind of problem....no hard surface floors, no glass areas in the sound field,etc.

All rooms will suffer from standing waves, based purely on their dimensions , Unless the room is enormous.
Think of how a properly designed mastering/recording studio is constructed, with huge amounts of non visible absorbtion .
Keith.

 

[Barry2013]

No they didn't. That was inferred here. Test results showed that they in fact had the same order of preferred speakers as the trained listeners and the retailers. The variance between the three was later attributed by Toole in his video lecture to the use of the scale. The reviewers used less of the scale but the rankings were the same and actually bettered most of the groups.

Thank you for clearing that up Jack.
It is what I would have expected and it is good to have that confirmed.

 

[dallasjustice]

I think there's a misinterpretation of the so called Haas effect. Haas studied sound localization. There is no acoustical research which suggests that humans totally exclude sound outside a certain window. In fact, Toole points out this common misunderstanding in his book. According to Toole the listener still combines timbral effects from late arrival energy well after the so called Haas window. IOW, everything still matters.

They could be better in a different room yes but that would also change the listening distances and frankly going from near to mid field has its own trade offs. The thing is there was no point at which I felt the speakers (S5s) needed more room to breathe. I know a Von Schweikert VR-5 would not work in that room given the latter's dispersion pattern that demands at least 8 feet of separation. I would say this is a case of a speaker design matching well with the owner's circumstances. It is also consistent with that reviewer mostly reviewing electronics and carts and not speakers. He is full aware of his limitations with regards to speakers.

Were the acoustics of that room all that great? Well, people were working on the street outside and I could hear them so not so great a rating on sound transmission. As for inside, did the room ring? No. Was it quiet enough to listen with 90dB peaks and probably sauntering along in the 50s and 60s without the room being bothersome? Yes.

Remember that the whole point of going nearfield is to eliminate the room as much as possible in the first place right? It shouldn't be all that surprising then. What the pictures don't show is that the send and return path to the racks and windows is quite long. The speakers are actually more into the room than I had thought coming in. This long path puts reflected sound way outside the Haas time window and also drops in dB substantially by the time it gets back to you. As for the side walls there are small panels there. They were enough. I certainly could tell you what was going on in the recordings and that the recordings were hardly homogenized in one way or another at all while listening through many genres, labels and also Tape. If I could discern these differences so easily it stands to reason that differences in equipment changes shouldn't be too difficult either if I had enough time to get intimately familiar with the system.

My interest here is not the reviewers but acoustics in general. More and more I see posts about shows where bloggers will stick their heads in and say no room treatment, it must suck so it does then they trash the place and move along. Ummmm. Wait a minute. Now here is a case of seeing a picture and saying that cannot possibly work, well it does. I enjoin anyone to go to open houses of new developments and compare the acoustics of bare units to that of the model units. You'd be surprised just how much difference everyday items make.

A third of our projects start literally before the homes break ground. The other third we deal with emptied rooms. The remainder are spaces where acoustic work has already been done or needs special attention/solutions mainly because of WAF. As such it's not like I'm saying just plop your stuff down just anywhere. That's the other slippery slope on the other side of the hill. Best not to confuse the two. This slope we're on is the one that says you NEED to alter your environment substantially where the truth is, adapting to your environment is just as important in terms of the final outcome. Doing the latter results in less alteration.

 

[Kal Rubinson]

Many reviews happen in hotel rooms. I couldn't think of a worse listening environment. The reviewers who claim to hear differences between components, other than speakers, after listening at a show only demonstrate their foolishness.

Agreed. The ability to determine the performance of a system, let alone that of an individual component, in an unfamiliar environment borders on the mystical.

 

[FrantzM]

Agreed. The ability to determine the performance of a system, let alone that of an individual component, in an unfamiliar environment borders on the mystical.

I agree .. This is all too common.

 

[scouter]

Personally, I've come to the realization that component reviews are an impossible task. I recently gave my impressions of some Constellation gear I had. Lately I exchanged a well reviewed DAC for another in the system and everything changed. Literally I was listening to a completely different system, albeit an excellent sounding one. Lesson learned for me? Take what reviews say as a BROAD generalization. AND I know I wouldn't want to be an audio component reviewer. It would take the fun of listening to music away for me.

 

[NorthStar]

IMO its more important for the reader to know the components involved rather than some arbitrary A/B/C class rating that has no consensus. The information is valuable for better context only if the reader is familiar with the associated equipment otherwise its not going to be helpful either way.

david

That; it is an experienced reader who can make a fair assessment of the reviewer under review about the gear he his reviewing and knowing the music material being played.
The more he knows about all the gear with real first hand experience in a familiar room the better equipped he is in making the right call.

Me I am not that guy; I just keep reading the reviews and thinking to myself what a wonderful job they have. Besides who have tested all the newest gear being released every month after month out there? So we need reviewers to do that job for us and rely on them. That's how most audiophiles buy their gear, from professional reviewers they fully trust and then making the research about who built that gear; just like investing in the stock market by researching the companies that are reliably worth investing in and integrity sound.

And reading the comments from the members here opens broader horizons in the undercurrents.
The one with the best system and room must know how to lead. He's been there, and around.

There are members here who are not pro reviewers and they should. On the other hand there are pro reviewers who must be bored by now.
Audio is a constant evolution with new remastered music that we all thought we were familiar with until the revelations surfaced from new techniques allowing the music to see a brand new day.

? It's all in the awareness of the reader; because the reviewer is just doing his job with what he has and don't have, with what he write and don't write.
Us we just read the words he used to describe, we look @ the measurements done by one guy in a room with his measuring gear and mike.
If we are unfamiliar with the reviewer, and the gear; then it's like reading Fortune magazine and looking @ the fashion adds. For a guy like me that's all she wrote. So I jump @ the end, where the music reviews are. After being entertained by the reviews I just read, sometimes with a smile on my face and other times with disbelief and awe.

Who started this thread? ...And what about the reader's own listening system who is reading the reviewer's system review?

Another good subject for another thread: How to evaluate an audio reviewer with an objective perspective. ...That could also be well branching in.

 

[Rodney Gold]

Everyone is a reviewer on the internet...

 

[nc42acc]

I couldn't have said it better myself. Thanks for saying it for me Rodney.

Everyone is a reviewer on the internet...

 

[amirm]

No they didn't. That was inferred here. Test results showed that they in fact had the same order of preferred speakers as the trained listeners and the retailers. The variance between the three was later attributed by Toole in his video lecture to the use of the scale. The reviewers used less of the scale but the rankings were the same and actually bettered most of the groups.

What you say is true Jack but the answer to a different question. Namely, the routinely stated point that audiophiles have different tastes than general public. As you say, research shows that just like other specialized groups of listeners, their preference *overall* is no different. A speaker that does well with trained listeners also does well with them.

Michael asked a different question. He asked about *listening skills*, not preference for loudspeakers. For that, we can go directly to the source of the research:

Differences in Performance and Preference of Trained versus Untrained Listeners in Loudspeaker Tests: A Case Study
Sean E. Olive, AES Fellow

First this graph:

So clearly nothing here is a matter of scale. Trained listeners have far better skills here. So there is no implication of me changing the research again, here is the original words in the paper:

"The performance of the trained panel is significantly
better than the performance of any other category of listener.
They are about three times better than the best group
of audio retailers, five times better than the reviewers, and
27 times better than the students. The combination of
training and experience in controlled listening tests clearly
has a positive effect on a listener’s performance. The students’
poor performance is likely due to the student’s lack
of training and professional experience in the field of
audio. The reviewers’ performance is somewhat of a surprise
given that they are all paid to audition and review
products for various audiophile magazines. In terms of listening
performance, they are about equal to the marketing
and sales people, who are well below the performance of
the audio retailers and trained listeners."

So there is no manipulation of research. It is very clear they did poorly. But how did this data come about? The answer is the previous paragraph where a statistical analysis is performed on the results of each group. The analysis shows the consistency with which different groups rate the same product. The same loudspeaker is presented multiple times in the study. An objective instrument would rate it the same every time. Humans are not that consistent but ideally they would be close to the instrument. But the research shows that without training, groups such as audio reviewers are highly inconsistent:

"To examine listener performance in view of occupation
more clearly, the mean listener FL values were plotted as
a function of occupation for both tests (see Fig. 8).
In the four-way tests the listener performance of the different
categories based on the mean FL values from highest
to lowest was trained listeners (94.36), audio retailers
(34.57), and audio reviewers (18.16)."

In other words, they cannot be counted to tell the "truth" in a single trial like trained listeners are. You have to test them over and over again and then look at the overall sum. As a group, they simply lack the ability to spot a problem and consistently point that out in every comparison to other loudspeakers.

This is the data Jack. And unfortunately not so easy to understand and parse out of the sea of research over a 30 year period. One has to read every bit of it and over and over again to get a consistent view. Or you can trust that I am not trying to screw you when I summarize them .

 

[NorthStar]

Everyone is a reviewer on the internet...

I couldn't have said it better myself. Thanks for saying it for me Rodney.

You guys are audio dealers? ...Or just casual audio reviewers? :b