Objectivists, Harman Testing, Reviewers, and Reality

PeterA

Well-Known Member
Dec 6, 2011
12,670
10,943
3,515
USA
Then there is the subsequent frequency response of the Logan posted again a number of times, and it's up and down all over the panel's operating range. That DOES NOT RAISE any eyebrows at Harman or anywhere else? Are we really serious? And is that part of the "research" to be taken seriously? I put "research" in quotes because to me it's just investigation, and a poor one at that, in a poor "room". No wonder it's been suggested we won't be able to tell our favorite speaker sitting in a corner opposite another, with the listener in the middle - who evaluates speakers like that, and what value does it add anyway.

I agree with this observation. Are the famous "spin" tests also done in this room with those other speakers in such close proximity to the speaker being tested? It does not look like other anechoic chambers that I have seen photos of. Also, some feel strongly that to properly evaluate the performance of a speaker, it should be the only speaker in the room because other speaker drivers move with sound pressure and can effect the sound or measurements of the speaker being evaluated. Does Harman really do the tests in that room with all of those speakers present? What about the other 70 pairs and all of the people in the room taking part in the DBT. Doesn't their presence also effect test results?

Perhaps I'm not in a position to criticize or question any of this. I'm not an industry professional and don't know much about the science of audio, so perhaps my questions are naive, or can be seen as coming from the bias of a skeptic and subjectivist. But the objective side of me would like to know how the NRC testing methodology differs from that being used by Harman. Has the Revel Salon 2 been tested in both facilities, and if so, are the measurements the same?

Since this is a thread about reality, photos of actual tests being performed would help us to better understand how the measurements are made.
 

mep

Member Sponsor & WBF Founding Member
Apr 20, 2010
9,481
17
0
I believe the spin test is done in their anechoic chamber.
 

MylesBAstor

Well-Known Member
Apr 20, 2010
11,238
81
1,725
New York City
Hey, if the speakers don't fit Harman's test conditions, it's the speaker's fault! :)

Let's be honest.

1. Despite coming with spikes, the ML were the only speaker in the pics not on spikes or or a stand.
2. The speakers were plugged into a twenty five cent AC strip shared with a whole lot of other things.
3. The speakers were sitting on one of woofers eg IIRC there is one behind and one under the speaker.
4. That speaker looked like it had the **** beat out of it (just look at the picture-it's embarrassing!) and was it even working properly? The speaker looked like it was used as a hammer.
5. Any numbskull would have known beforehand that a dipole placed in the middle of a room would have sounded awful.
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
Let's be honest.

1. Despite coming with spikes, the ML were the only speaker in the pics not on spikes or or a stand.
2. The speakers were plugged into a twenty five cent AC strip shared with a whole lot of other things.
3. The speakers were sitting on one of woofers eg IIRC there is one behind and one under the speaker.
4. That speaker looked like it had the **** beat out of it (just look at the picture-it's embarrassing!) and was it even working properly? The speaker looked like it was used as a hammer.
5. Any numbskull would have known beforehand that a dipole placed in the middle of a room would have sounded awful.

And given a cosmetically perfect ML and ideal condition under which to measure the ML, it would do better. But it still wouldn't do well. How do I know this? I've had them set up and tweaked in a very good showroom, walked around to the side where the sound is radiating off axis, and heard the fall apart.

Tim
 

TheMadMilkman

Well-Known Member
Sep 7, 2010
125
0
91
Let's be honest.

1. Despite coming with spikes, the ML were the only speaker in the pics not on spikes or or a stand.
2. The speakers were plugged into a twenty five cent AC strip shared with a whole lot of other things.
3. The speakers were sitting on one of woofers eg IIRC there is one behind and one under the speaker.
4. That speaker looked like it had the **** beat out of it (just look at the picture-it's embarrassing!) and was it even working properly? The speaker looked like it was used as a hammer.
5. Any numbskull would have known beforehand that a dipole placed in the middle of a room would have sounded awful.

Do you have a source for these claims?
 

MylesBAstor

Well-Known Member
Apr 20, 2010
11,238
81
1,725
New York City

microstrip

VIP/Donor
May 30, 2010
20,807
4,704
2,790
Portugal
That seems to coincide with a Magico S5 review and measurements. More importantly, it reflects a general opinion from reviewers, and others, that the S5 sounds good, even without measuring it.

(...)

Yes, but many people who do not care about measurements have referred they prefer the inferior measuring Q series, or even the older series ...
BTW, I am one of those.
 

MylesBAstor

Well-Known Member
Apr 20, 2010
11,238
81
1,725
New York City
Yes, but many people who do not care about measurements have referred they prefer the inferior measuring Q series, or even the older series ...
BTW, I am one of those.

You need to visit New York my friend!
 

PeterA

Well-Known Member
Dec 6, 2011
12,670
10,943
3,515
USA
Yes, but many people who do not care about measurements have referred they prefer the inferior measuring Q series, or even the older series ...
BTW, I am one of those.

I prefer the Q3 to the S5, and my older Mini 2 to the S1, but none of those comparisons were direct in the same system, let alone made in an unsighted evaluation. I did hear the Q3 and the S5 in the same room, though the electronics were different. And I have heard all of the speakers many times in different settings (rooms and systems), and each time I preferred the Q3 and Mini 2.

I would like to compare the measurements of each.
 

FrantzM

Member Sponsor & WBF Founding Member
Apr 20, 2010
6,455
29
405
I prefer the Q3 to the S5, and my older Mini 2 to the S1, but none of those comparisons were direct in the same system, let alone made in an unsighted evaluation. I did hear the Q3 and the S5 in the same room, though the electronics were different. And I have heard all of the speakers many times in different settings (rooms and systems), and each time I preferred the Q3 and Mini 2.

I would like to compare the measurements of each.

+1
 

marty

Well-Known Member
Apr 20, 2010
3,039
4,211
2,520
United States
I'm late to this conversation but off all the measurement data I'd like see that would be useful in helping me select a loudspeaker, it would be distortion vs power. It's interesting that this info is standard for electronics such as pre-amps and amplifiers, where it means very little (electronics don't sound very different because one has 0.001% THD vs another which has 0.003% THD). But a driver or speaker that has <1% distortion vs one that has 8% at 95dB would be data I'd like to know. In fact, it's the main reason I check to see if the NRC has tested a particular loudspeaker.
 

mep

Member Sponsor & WBF Founding Member
Apr 20, 2010
9,481
17
0
I'm late to this conversation but off all the measurement data I'd like see that would be useful in helping me select a loudspeaker, it would be distortion vs power. It's interesting that this info is standard for electronics such as pre-amps and amplifiers, where it means very little (electronics don't sound very different because one has 0.001% THD vs another which has 0.003% THD). But a driver or speaker that has <1% distortion vs one that has 8% at 95dB would be data I'd like to know. In fact, it's the main reason I check to see if the NRC has tested a particular loudspeaker.

We need Amir to beat on Harman and get them to release their treasure trove of data they only share with their dealers in private. Amir is calling for a revolution and thinks that we should all demand more data/measurements from companies so we can collectively raise the bar on the products we buy and then we need to figure out a way to make reviewers write better reviews because Amir doesn't approve of the vast majority of reviews being written. The problem is, I don't see a fair alternative to the method used by JA at Stereophile. If reviewers were privy to a snazzy set of measurements before their review and had to write their review around the measurements, reviews would be much different, but I would argue they wouldn't be any better and would probably be worse. That's why reviewers at SP don't get to see the measurements until after they have submitted their review for publication.
 

rockitman

Member Sponsor
Sep 20, 2011
7,097
414
1,210
Northern NY
We need Amir to beat on Harman and get them to release their treasure trove of data they only share with their dealers in private. Amir is calling for a revolution and thinks that we should all demand more data/measurements from companies so we can collectively raise the bar on the products we buy and then we need to figure out a way to make reviewers write better reviews because Amir doesn't approve of the vast majority of reviews being written. The problem is, I don't see a fair alternative to the method used by JA at Stereophile. If reviewers were privy to a snazzy set of measurements before their review and had to write their review around the measurements, reviews would be much different, but I would argue they wouldn't be any better and would probably be worse. That's why reviewers at SP don't get to see the measurements until after they have submitted their review for publication.

I think I would much rather have the well made Madrigal Labs Mark Levinson equipment of old than the new Harman...measures better Mark Levinson.
 

amirm

Banned
Apr 2, 2010
15,813
38
0
Seattle, WA
I agree with this observation. Are the famous "spin" tests also done in this room with those other speakers in such close proximity to the speaker being tested?
Peter, I explained all of this repeatedly in the other thread. How can you turn around and ask this kind of question, seemingly ignoring everything I wrote???? This is the picture I posted for where the measurements are performed:



It is an anechoic chamber. How on earth do you measure a loudspeaker for radiation at one specific angle in a real room where reflections are bouncing all over the place?

Note that magazine reviewers are not measuring this. They don't have access to anechoic chamber so they approximate by keeping the microphone close to the loudspeaker. That technique has limitations such as lack of integration of drivers at too close or a distance, or inclusion of too many reflections at farther back. So while informative, and decent approximation, they are not the same thing.

It does not look like other anechoic chambers that I have seen photos of.
We don't perform listening tests in anechoic chambers. We perform them in well, a room. Harman has multiple such rooms. I have shown you a picture of one with a speaker shuffler that I sat in. There are others at Harman. Here is one where the loudspeaker is rotated in place. That room is built to an ITU reference standard for acoustics testing and one is supposed to be a model of "domestic listening room." Here is a picture of it I took while Dr. Olive was running the speaker training test on us:



The turntable for the loudspeakers unfortunately is behind the video projection screen. But essentially three loudspeakers are mounted to it and rotated. This is used for in-wall and car audio speakers. This room was also used for Room EQ blind testing. All the results in this room mirror the one used in the other room. There is a whole conference paper on design of this room.

Here is one where they can shuffle stereo speakers and in a much larger room:



And the other end of it:



As with everything else Harman does, there is extensive research paper and testing published for this room. Here is the end of that paper:

"4.0 CONCLUSIONS
In summary, we have described a new facility designed to test multichannel components
efficiently and as bias-free as possible. The facility includes acoustically transparent
listening screens that hide the identities of all multichannel loudspeakers and equipment
within the audio path. Particular attention has been taken to address the two of the most
problematic variables in listening tests: the listening room and the position(s) of the
loudspeaker. Through the use of a computer automated speaker shuffler, we have greatly
reduced the amount of time and effort required to set up and test multiple comparisons
between loudspeakers by reducing the factor position to a one-dimension or level
variable. Typical loudspeaker evaluations should be reduced in length by a factor of 24:1.

The listening room itself is capable of testing up to three different 5.1 or 7.1 channel
systems and accommodate 1-6 listeners at a time. The measurements we have shown in
this paper indicate its performance in its current form
meets the very highest standards set
out by the ITU and EBU recommendations, in terms of volume, geometry, reverberation
time, and the control of early reflections. The acoustics of the room can be easily altered
from hemi-anechoic to more typical domestic room conditions by adding reflective
panels to the room’s boundaries.


Finally, the experimental design, set up and control are computer-automated so that
experiments can be easily repeated, and are less prone to human error. The more time consuming
and mundane tasks such as collection and analysis of data have also been
computer-automated, so that experiment report writing becomes a simple cut-and-paste
operation."


Really guys. You have to resist the urge to keep thinking this is a bunch of corrupt agenda meant to confuse people. For that, you need to look at the marketing material you are reading about your favorite loudspeaker. These are true researchers who happen to work at Harman with very deep pockets, able to conduct such research at grand scale. They are not sitting there fooling themselves or other people because you all happen to think of things they have not.

Also, some feel strongly that to properly evaluate the performance of a speaker, it should be the only speaker in the room because other speaker drivers move with sound pressure and can effect the sound or measurements of the speaker being evaluated. Does Harman really do the tests in that room with all of those speakers present? What about the other 70 pairs and all of the people in the room taking part in the DBT. Doesn't their presence also effect test results?
You honestly think they put 70 loudspeakers in one room and tested them with 250 people??? No, they did not. These are composites of many independent studies. Yes there are a few loudspeakers in the room. The only effect I have heard of here is the woofer acting as a tuned absorber but that happens when it is unloaded.

Perhaps I'm not in a position to criticize or question any of this.
What were the comments above then?

I'm not an industry professional and don't know much about the science of audio, so perhaps my questions are naive, or can be seen as coming from the bias of a skeptic and subjectivist. But the objective side of me would like to know how the NRC testing methodology differs from that being used by Harman. Has the Revel Salon 2 been tested in both facilities, and if so, are the measurements the same?
The Salon 2 came years and years after Dr. Olive and Toole performed their research there. From what I recall, the testing there relied on turntables rotating loudspeakers and the technique I mentioned: symmetrical placement in a room behind a curtain (i.e. two at a time).

Since this is a thread about reality, photos of actual tests being performed would help us to better understand how the measurements are made.
Loudspeaker is put on a lazy susan in an anechoic chamber. A series of frequency response measurements are then taken with that microphone array giving you a slice in 3D Space from horizontal to vertical. The turntable is then moved precise number of degrees and another array snapshot is taken. This is repeated until you have completed 360 degrees. Once there, you have 72 measurements of a 3-d space around the loudspeaker. These are independent data points. They are then combined into three different curves, when in turn are combined into weighted composite for prediction of loudspeaker preference/performance.
 

Alan Sircom

[Industry Expert]/Member Sponsor
Aug 11, 2010
302
17
363
Dr. Toole laments that when a person goes shopping for multi-thousand dollar piece of audio equipment, he has far, far less information than on the side wall of a $50 tire:

This is very true, but it also demonstrates a fairly bad case of audio myopia.

Here's what's printed on the side of a tire:



And here's what the vast majority of people actually ask for:

"What's your best price on a set of tires for my car?"

The information supplied on a tire allows an installer to match the right tire to the car and (ideally) the way that driver uses the car. The information required by an audio installer is a fairly basic set of broad-spectrum compatibilities to match a product to a system, a room, and the way that owner plans on using the product.

Even the driver of a high-performance car is usually fairly un-nuanced on tire specifications, and will usually end up going with the recommendations made by the car manufacturer.

Knowledge is powerful, but the difference between empowering the listener and flooding them with information in which they are both uninterested and disinterested is the problem.

Here's the thing. Most people are now effectively host bodies for a smartphone. Ask any one of them if they know the specifications of their smartphone, and they might be able to give you the brand name, model number, maybe the current OS (if they downloaded it recently) and possibly the amount of memory onboard. Everything else is just geek-speak noise.
 

amirm

Banned
Apr 2, 2010
15,813
38
0
Seattle, WA
We need Amir to beat on Harman and get them to release their treasure trove of data they only share with their dealers in private.
No, I am here to beat you up to ask for it and importantly, care. If this thread was full of excitement over having such data, I would have forwarded it to Harman as additional motivation for them to do this. As it is, it is all disdain for the very thing you are saying I should go and ask for it. Become an advocate Mark. Don't keep yourself in the dark. You can always ignore the data. But can't have it if it is not there.

Amir is calling for a revolution and thinks that we should all demand more data/measurements from companies so we can collectively raise the bar on the products we buy and then we need to figure out a way to make reviewers write better reviews because Amir doesn't approve of the vast majority of reviews being written.
Exactly what I said above. You seem to want consumers in the dark. Why? Is it to make sure the data can't be used to demonstrate a reviewer got it wrong?

The problem is, I don't see a fair alternative to the method used by JA at Stereophile. If reviewers were privy to a snazzy set of measurements before their review and had to write their review around the measurements, reviews would be much different, but I would argue they wouldn't be any better and would probably be worse. That's why reviewers at SP don't get to see the measurements until after they have submitted their review for publication.
What fair alternative and why should we, as customers care? John has made measurements. If you don't know how, then you are not qualified to be a technical reviewer. You should review music, not equipment. Learn something about the science you want to write about for heaven's sake. Don't see car reviewers crying that they don't know how to measure cornering g force of a car... And how hard do you think those measurements are that JA creates? Have you ever studied how he does it? I suspect you have not. Shame.
 

mep

Member Sponsor & WBF Founding Member
Apr 20, 2010
9,481
17
0
This is very true, but it also demonstrates a fairly bad case of audio myopia.



And here's what the vast majority of people actually ask for:

"What's your best price on a set of tires for my car?"

The information supplied on a tire allows an installer to match the right tire to the car and (ideally) the way that driver uses the car. The information required by an audio installer is a fairly basic set of broad-spectrum compatibilities to match a product to a system, a room, and the way that owner plans on using the product.

Even the driver of a high-performance car is usually fairly un-nuanced on tire specifications, and will usually end up going with the recommendations made by the car manufacturer.

Knowledge is powerful, but the difference between empowering the listener and flooding them with information in which they are both uninterested and disinterested is the problem.

Here's the thing. Most people are now effectively host bodies for a smartphone. Ask any one of them if they know the specifications of their smartphone, and they might be able to give you the brand name, model number, maybe the current OS (if they downloaded it recently) and possibly the amount of memory onboard. Everything else is just geek-speak noise.

And I don't think it's much different in audio. For every objectivist that claims they love them some measurements even if some of them can't really interpret them, I bet there is a far higher ratio of audiophiles that doesn't care that much about actual measurements beyond some basic specifications that really matter to them and is readily available (which is all most objectivists have anyway). For every audiophile that gets absorbed into JA's measurements in SP for example, I bet there is a bunch of people who could care less and just want to know how the reviewer thought the piece of gear sounded.
 

amirm

Banned
Apr 2, 2010
15,813
38
0
Seattle, WA
This is very true, but it also demonstrates a fairly bad case of audio myopia.

Here's what's printed on the side of a tire:



And here's what the vast majority of people actually ask for:

"What's your best price on a set of tires for my car?"
That's true but we are not talking about vast majority of people. We are talking about what enthusiasts, who hang around these forums, always trying to best their current equipment would need. I am not a car guy but a few times I have landed in their forums, they sure talk about this stuff and then some.

Here's the thing. Most people are now effectively host bodies for a smartphone. Ask any one of them if they know the specifications of their smartphone, and they might be able to give you the brand name, model number, maybe the current OS (if they downloaded it recently) and possibly the amount of memory onboard. Everything else is just geek-speak noise.
Same point as above. Mass consumers aren't here wondering what makes a better loudspeaker. The ones that are here, have been given so little. Even the bit that is handed out like sensitivity number and impedance are marketing numbers that are completely corrupt. These are as close as we can come to those tire markings. This is the impedance curve that JA measures on a random loudspeaker:



I mean how the heck do you boil that down to one number to throw at the buyer of the equipment?

And how about that phase response? While I am sure hardly anyone knows what it means, is critical to determining if the loudspeaker is a difficult load and hence requires a very powerful amplifier to drive it or not. isn't that a useful thing to know if you are buying this loudspeaker??? I am confident beyond confident that loudspeaker manufacturers have this graph. But they just give us the same marketing numbers as everyone does because we are not only OK with it, but seemingly praise them for telling us less than more.

It is not hard to learn to understand these graphs. Let's have the willingness be there and we can get educated in a hurry. But keep resisting it for countless reasons and we are doomed to sitting in the dark.
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
There's absolutely nothing surprising here, Amir. Subjectivists want high end to remain subjective and don't like to be challenged when they express their subjective opinions as if they are objective realities. And the more they've spent and swapped and synergized, the more outraged they become. If you had all the results from Harman, with the names and performance data of dozens of high-end speakers (many of which would probably have their asses handed to them by those $500 Infinitys), you'd have a firestorm on your hands that would make this thread look like a bunch of Girl Scouts singing around the camp fire. You're on a noble mission. It would shame charlatans and move honest companies to produce better products. But audiophiles don't want it. They will deny it with every fiber of their illogical beings. They want to play in the dark, guided by the blind. Maybe you'd do better with the AV crowd. Or the pros.

Tim
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing