Objectivists, Harman Testing, Reviewers, and Reality

[Steve Williams]

Say what? Amir never asked me to do anything? Were you sleeping when he said it was my job to get audio companies to release their measurements? Did you miss the part where Amir said that reviewers need to buy test gear and measure the gear they review? Come on man.

Out of curiosity Mark, what equipment do you envision a reviewer would need as a result of Amir's suggestions that would make it prohibitive. At the very least wouldn't a reviewer who has speakers in for review take some basic measurements of the speaker in the room or are you saying it is all done by ear

 

[TBone]

Say what? Amir never asked me to do anything? Were you sleeping when he said it was my job to get audio companies to release their measurements? Did you miss the part where Amir said that reviewers need to buy test gear and measure the gear they review? Come on man.

While Amir's point is hardly feasible; it's difficult to find "reviewers" who include viable measurements ... but flowery prose and silly comparisons to "live" music abound.

 

[mep]

Out of curiosity Mark, what equipment do you envision a reviewer would need as a result of Amir's suggestions that would make it prohibitive. At the very least wouldn't a reviewer who has speakers in for review take some basic measurements of the speaker in the room or are you saying it is all done by ear

Steve-The AP gear that JA has on loan is over $30k and Amir is far closer to making demands than he is suggestions. How many reviewers are taking actual measurements of speakers?

 

[Steve Williams]

Steve-The AP gear that JA has on loan is over $30k and Amir is far closer to making demands than he is suggestions. How many reviewers are taking actual measurements of speakers?

I do understand the depth to which JA measures but as a reviewer do you really need all that. I agree that is a huge chunk of change

 

[mep]

While Amir's point is hardly feasible; it's difficult to find "reviewers" who include viable measurements ... but flowery prose and silly comparisons to "live" music abound.

I'm glad to hear you say that Amir's point is "hardly feasible." Does that make you a defeatist?

 

[mep]

I do understand the depth to which JA measures but as a reviewer do you really need all that. I agree that is a huge chunk of change

Do you want a full set of professional measurements? And even if Amir's dream came true, do you want reviewers to write their reviews around a set of measurements or do you expect the reviewer to tell you how they think it sounds?

 

[TBone]

I'm glad to hear you say that Amir's point is "hardly feasible." Does that make you a defeatist?

Well, I try not to be, esp. since I need debate many other subjects unrelated to audio ... hopefully none of my peers consider me a defeatist, and honestly, if I had such an attitude ... I suspect my employment history would read much differently.

 

[rbbert]

While Amir's point is hardly feasible; it's difficult to find "reviewers" who include viable measurements ... but flowery prose and silly comparisons to "live" music abound.

As Myles says, +1,000,000

 

[mep]

Well, I try not to be, esp. since I need debate many other subjects unrelated to audio ... hopefully none of my peers consider me a defeatist, and honestly, if I had such an attitude ... I suspect my employment history would read much differently.

Good. I try not to be a defeatist either. There is a difference between being a realist as you were in your post and being a defeatist.

 

[Steve Williams]

Do you want a full set of professional measurements? And even if Amir's dream came true, do you want reviewers to write their reviews around a set of measurements or do you expect the reviewer to tell you how they think it sounds?

actually a bit of both

 

[TBone]

Good. I try not to be a defeatist either. There is a difference between being a realist as you were in your post and being a defeatist.

Mark, it's nice to see you've identified common ground, considering we are really all arguing the same argument, and we share the same passion ... please take note: I have neither an "agenda" here -nor- do I feel ANY need to pick & choose any particular side. Personally, I try to stay clear of objectivity / subjectivity / DBT debates, they tend to divide more than educate, much prefer to find that elusive balance in-between.

 

[mep]

actually a bit of both

If you want a full set of professional measurements, then you you need people who are trained professional technicians or engineers who have professional test equipment. And now you also want the reviewers to have access to the measurements prior to writing their reviews and you don't think that will change how reviews are written? That comes with its own perils because now you could see reviews written around the measurements and not written about how the gear sounds.

I think we are quickly approaching the point on this forum where some people think they could place blind faith in measurements (if they really had them, and yeah, I've been told it's my job to go get them) and they don't want or need to read reviews.

 

[Phelonious Ponk]

One more opinion...

http://www.soundstagehifi.com/index...infer-a-speaker-s-sound-from-its-measurements

Yes, but not really a differing one:

When we were designing the program, I consulted with experts at the NRC, as well as with numerous speaker designers well versed in measuring techniques. They all told me the same thing: Even though we were putting together a great measurement program that would be superior in quality and quantity to what any other publication was presenting, and that the measurements we would be taking would tell us some useful things about the performance of speakers, we still would not be producing nearly enough measurements to tell us exactly how a speaker would sound to a listener in his room -- or even close to it.

No one is saying that the testing pioneered by Toole tells you exactly what a speaker is going to sound like in a room. No one with any understanding of the subject would say that. But the fact of the matter is that Toole's and Olive's subsequent studies have shown that you can come close enough to predict preference with a very high degree of accuracy. That's all it says. And while I wouldn't say that finding is irrefutable, I would say with confidence that nothing said in the threads on this forum have refuted it. They've only repeated questions that have been answered many times and questioned the results without finding any substantive flaws in the methodology.

Tim

 

[TBone]

Your subjective opinion means absolutely nothing, a set of measuremtnts can tell you, for example whether the amp you are considering will properly drive your loudspeakers, or whether a preamp will be completely transparent in your system.
A lot of 'reviews' are no more than the manufacturers advertising, regurgitated .
Keith.

perhaps true, but perhaps a tad dismissive of real intent. Reviews can play an important role in the decision making process ... the more educated consumer picks and chooses his review(s)er(s) accordingly. BTW, I've been reading A.Sircom's excellent reviews for years, much enjoyed pastime, the quality of the writing matters to me, (edit: nearly) as much as the equipment being reviewed.

 

[Phelonious Ponk]

If you want a full set of professional measurements, then you you need people who are trained professional technicians or engineers who have professional test equipment. And now you also want the reviewers to have access to the measurements prior to writing their reviews and you don't think that will change how reviews are written? That comes with its own perils because now you could see reviews written around the measurements and not written about how the gear sounds.

I think we are quickly approaching the point on this forum where some people think they could place blind faith in measurements (if they really had them, and yeah, I've been told it's my job to go get them) and they don't want or need to read reviews.

Mark, in the past you, and others in your camp, have claimed to be nearly immune to bias, to being able to listen through it to hear a truth even beyond what can be measured, particularly in long-term listening. Yet now you think reading a white paper would prevent your mind from perceiving what your ears hear?

Tim

 

[amirm]

Steve-The AP gear that JA has on loan is over $30k and Amir is far closer to making demands than he is suggestions. How many reviewers are taking actual measurements of speakers?

I explained already Mark that because you have not studied how these measurements are made, you mistakenly assume what you need to buy the Audio Precision to perform them. We are talking about frequency response measurements. Here is one I performed which cost $60 for a mic, and a computer and free software I already have (there is an 80 Hz crossover here):

This one measurement tells a world about what is going on here. It not only characterized the loudspeaker but it characterized it in my room. It shows my room to have wide swings in low frequencies and since those are room dependent, any subjective observation I make about bass notes will not translate to anyone else. See how I just connected subjective experience with objective science? We also see that tiny dip due to directivity error around 2.5 Khz. Another loudspeaker which had a much bigger problem there, would show even a bigger trough. And with it, as listening tests predict, will cause lower listener preference. Again, all of this cost $60. This I believe is less than the sales tax you pay for an audio cable you buy.

To show pseudo-anechoic measurements, you take the loudspeaker outside, elevate it from the ground and make measurements there. Again, all with that one $60 measurement mic. No audio precision was necessary or even useful as the software that runs in there is pathetic compared to computer programs. Heck there are even iPad apps that measure these things now so if you are not computer savvy, you can still do this work.

As to me demanding what you do, yes, I get to do that when you put your reviewer hat on. The moment you do that, you are in the industry and just like a manufacturer, we can scream bloody murder if they do something we don't approve. You are no longer just a forum member. What's more you routinely speak on behalf of all reviewers, defending current practices. So that entitles us to even more formal requests that you step up your game, and find a way to make your reviews more provable and believable.

Of course, like many businesses, you are welcome to ignore our requests, demands, or whatever you call it. So in no way are you forced to do anything. At the same time though, you can't ask us to tone down what we like, when you are doing the exact opposite.

 

[mep]

Your subjective opinion means absolutely nothing, a set of measuremtnts can tell you, for example whether the amp you are considering will properly drive your loudspeakers, or whether a preamp will be completely transparent in your system.
A lot of 'reviews' are no more than the manufacturers advertising, regurgitated .
Keith.

We have something in common as your opinions mean nothing to me either. No set of measurements are going to tell you how a given component will sound in a system. I agree that measurements can tell you if a given amp will be electrically compatible with a given pair of speakers, but the measurements can't tell you how the combo will sound together no matter how you wish it to be so.

 

[DaveC]

Yes, but not really a differing one:



No one is saying that the testing pioneered by Toole tells you exactly what a speaker is going to sound like in a room. No one with any understanding of the subject would say that. But the fact of the matter is that Toole's and Olive's subsequent studies have shown that you can come close enough to predict preference with a very high degree of accuracy. That's all it says. And while I wouldn't say that finding is irrefutable, I would say with confidence that nothing said in the threads on this forum have refuted it. They've only repeated questions that have been answered many times and questioned the results without finding any substantive flaws in the methodology.

Tim

I agree, I just think it was a well written opinion on the subject. Personally, I'm all for both measurements and subjective listening and I think anything that can be done to further the correlation of measurements to how we perceive sound is a step in the right direction. Amir has made some irrefutable points imo, but it's also true that as complex as the subject is, it's still nowhere near complex enough to completely define how sound is perceived. It would be interesting to see if Harman has managed to measure more obscure things like detail or resolution of a speaker, speed or PRAT, cohesiveness, etc... we know they won't share information if they think withholding it would be a competitive advantage so it's hard to know exactly how much progress has been made in this area.

I also think there's room in the world for both subjective reviews as well as measurements. I'd agree most people don't care and aren't trained to understand measurements, but some are and some people obviously find them valuable. If you are a reviewer trying to reach the broadest possible audience a review that covered both areas thoughtfully and thoroughly would probably be the best way to go,

 

[amirm]

I think we are quickly approaching the point on this forum where some people think they could place blind faith in measurements (if they really had them, and yeah, I've been told it's my job to go get them) and they don't want or need to read reviews.

I don't know how you can speak on behalf of the forum. We are in one thread, not the entire forum. And further, no blind faith has been requested to be put in measurements. I have continued to say you also need to listen to loudspeakers. Things like power handling for example, is not measured properly in loudspeakers today, nor power compression. And also, two excellent loudspeakers will still vary in tone and you do want to preview that.

Question is whether we are better off having less in a review than more. You advocate the former but I have yet to hear the case for that other than it is a hardship for you to do the latter.

 

[Robh3606]

If you want a full set of professional measurements, then you you need people who are trained professional technicians or engineers who have professional test equipment. And now you also want the reviewers to have access to the measurements prior to writing their reviews and you don't think that will change how reviews are written? That comes with its own perils because now you could see reviews written around the measurements and not written about how the gear sounds.

Hello Mep

That's not really true. You don't need professional measurements or equipment. What you need to do any meaningful comparisons are measurements done under identical test conditions. You need repeatability in the same set-up and from set-up to set-up or it all goes out the window. 12 Reviewers 12 different measurement set's which may look different but actually be correct because of placement of the microphone and distance, gate times chosen which is a big one, and various other reasons. None of these guys would have access to an anechoic chamber so it would be a nearfield splice to a gated measurement like JA does in Stereophile. This is a compromise and will not match anechoic measurements of the same speakers done by JBL/Harman s an example.

Rob