Do Speakers That Measure Great Always Sound Great

[Mark Seaton]

want to start the discussion or should we start a new thread

I'd say the preceding posts do well to add direction to the rest of the discussion, and I'd welcome some responses from others on my longer post above.

While related, we should keep in mind that amplifiers and electronics are working on somewhat different scales of deviation from the conceptual ideal than loudspeakers. Much of what is being discussed in amplifier differences come down to differing types of distortion, and in more significant cases would likely show distinct differences in the audibility of their distortions as compared to 20,000' view snapshot of THD measurements. There has been lots of work on distortion audibility that was founded on work from 50 years ago and looked at more closely by others in more recent years. These aspects of audibility are of high importance to those trying to squash our high resolution formats, and are what allow them to squash the data a good bit before we can readily identify the effort. Similar to my reference above of different "colors" of frequency response deviation, amplifiers often have different types of distortion, even if the total distortion magnitude (ie THD) is similar in quantity.

Back to speakers...

Speakers which measure well in a few metrics but sound bad indicate that we haven't included a large enough matrix of measurements. To save us a rediculous number of measurements, any measurement data, just like any science experiment or report, should include tester observations. We can burn a bunch of time creating tests for simple leaks, buzzes, rattles, or gross distortion, but these are very easy for a tester to observe and note if further investigation is warranted. This effort is what I see so little of in measurements prestented online or in publications.

The similar applies to those speakers who don't look so smooth in their measurements but produce subjectively good sound. In such cases we usually are zoomed in too close or specific with our measurements. Again, simple observations can direct what further measurements are warranted. To a degree we have to ask if we are looking to map the entire DNA sequence or if we just want to see what the species is.

If you read the work of Dr. Olive & his colleagues their work is very supportive what might fit a set of core competencies for loudspeakers. A detailed set of measurements (more than fits in most any print review) can provide a very good sense of the strengths and potential weaknesses of a given loudspeaker design. Add enough detail and you can raise confidence even higher in the result. Of course no matter how good the measurements, in the end we have to fire them up and double check we didn't miss something critical.

 

[Steve Williams]

in the end we have to fire them up and double check we didn't miss something critical

well for me the final test is what they sound like to me

what has always frosted my butt is the constant flaming on forums about any and all types of gear, speakers etc where unilaterally the comment is made "if it measures bad, I aint buying it let alone even wanting to audition it"

 

[audioguy]

well for me the final test is what they sound like to me

what has always frosted my butt is the constant flaming on forums about any and all types of gear, speakers etc where unilaterally the comment is made "if it measures bad, I aint buying it let alone even wanting to audition it"

John Dunlavy, who had measured and seen how poorly the measurements looked on the Maggies said they sound great to him on things like piano music.

But this all does go back to what I suggested in Dr. Olives discussion that even if they come up with a great set of defined speaker measurements, the primary folks who will make purchase decisions based upon those measurements are probably not members of this forum

 

[MylesBAstor]

well for me the final test is what they sound like to me

what has always frosted my butt is the constant flaming on forums about any and all types of gear, speakers etc where unilaterally the comment is made "if it measures bad, I aint buying it let alone even wanting to audition it"

And what about the letters practically every month in SP?

 

[DonH50]

My answer is "no" -- great measurements do not always mean great sound, especially with speakers, if that's the only criteria. A bad room will ruin good speakers... As I posted elsewhere, imo speakers/room are the primary effects, electronics secondary, and everything else tertiary. Obviously bad electronics can cause issues, as can 50 feet of 28 gauge speaker wire to your 85 dB-efficient behemoths, but for most of us them's my priorities and I'm sticking to them! - Don

 

[FrantzM]

Warning !!! Long Post

Let’s try to define what is the goal of Hi-Fi … The whole process from the production of the recording to its reproduction in our homes. To me? Reproduce a fac simile of a real Live event. Be it in a studio or a recording of a truly live event the goal is to simulate it at best possible with the present means of technology.
I have put aside the “if it sounds good to me then it is good” Audio relativism oversimplification… This has resulted in strongly colored components that have not advanced the art one micron... These components are legion, perversely, in High-End Audio more so than mass-market products …
I try not to use the term “sounds good” because of the implicit subjectivity … It may “sounds good” to me while I am fully conscious of it not being accurate… I prefer to use satisfactory reproduction, a reproduction that for the listeners approximate what they hear in live music. It is not very difficult for most people to discern Live from recorded... The better system fools us very well for a few seconds and we work our way to enjoy the music but Live vs recorded is relatively easy to perceive IMO, no Golden ears needed. I believe that should be the goal of High-FI. I understand this is a very difficult notion to quantify but it is not impossible to do so and to me that is where lay the next advances in Audio Reproduction
Yet, I must say that I am little puzzled by the relationship between our usual audio metrics our subjective notion of satisfactory reproduction.. . I have come to believe strongly that our usual set of measurements do not correlate very well with our perceptions of what constitute a reasonable fac simile of real music playing in a real space (that the “real” space be simulated is not a problem in itself as long as the simulation is transparent, a tall order and a subject of debate in itself). I will take a few examples but mp3 which will have many shudder here, is telling. mp3 at 320 Kbps on certain type of music is VERY difficult to discern from the uncompressed . mp3 at 128 Kbps on certain type of music is acceptable even by trained listener on voice, yet it has been shown that on many mp3 compression of an Audio signal, the THD can be up to 50% !!!! (Earl Geddes in his October 2003 presentation to the AES).. Several SET amplifiers have substantial THD, yet sound very believable reproducing music . I must say that SET are NOT my favorite type of amplifier for many reasons… The works of some researchers (Geddes and others but Olson as early as 1960) have shown that the THD can be quite high but that people continue to find the reproduction acceptable .. There are particulars and some have been studied but my point to all this is that the our current protocol of measurements are not well correlated to what we find satisfactory, especially with speakers but not limited to speakers …
On a more pragmatic front… My previous speakers measured not very nicely, yet “sounded good” and like many here, I have heard speakers that measured well and were not to my liking, several Thiel speakers come to mind .. Most here know that equalizing for flat response is a sure way to have a bright thin-sounding system.. . Although a flat response is actually what we should want .. What is in the input is what is the output or so we think but a Flat FR measured a certain way may not truly be a .. Flat Frequency Response.. Especially with speakers in the room and what comes to our ears strongly influenced by the room , unequally, I must add in the Frequency Range in question (Room may behave nicely in the bass but horribly in the treble or vice versa or any combination of frequency ranges you want to think of). How we measure means a lot and what we measure surely means even more … So I am sure there are correlation and they are being studied but they are not what we have come to think.. They seem more complex and more obtuse… More.. Maybe another time. I would like first the input of this forum before, I continue to bore you to death..

Frantz

 

[JackD201]

There's a joke about a Congressman who after bemoaning the unabated increase of grain prices said that he would seek to pass a law repealing the law on Supply and Demand.

I'm reminded of this joke every time the term "accuracy" is used as a benchmark in audio. Am I for accuracy? Of course I am. There are however laws of physics that like the laws of S&D can't be repealed. My observation is that driver material plays a huge part in all of this because each material has its own distinct coloration. This is compounded when two instruments that are practically omni-present in modern western music is not recorded via D.I. Boxs and emulators but rather recordings of their outputs from their own amps and loudspeakers. These would be the Electric Guitar and Electric Bass.

Simpleton logic dictates that if you want to get the sound of a stack, audiophile exotica may not be your best bet. Go get a JBL L series and rock out I say Then again, I like being a simpleton some times. This is a reason I like different speakers and amps, as well as MM carts for pop genres while my main system is geared for my true love which is classical music.

Anyhow, these poor measuring speakers actually make genres such as rock and electronica more listenable and thus ultimately more enjoyable for me. I just love an 8" or 10" paper woofered speaker for good ol' thrashing. In my book this means yes a bad measuring speaker can sound good. I'd have to add though that it depends heavily on exactly what you want it to do.

I submit then that there are many speakers for many uses. My main ones do rock really well but in all honesty they would be slaughtered by a stack of JBLs, Community's or Meyers using active XO and megawatt pro amps. They are what you would hear them through live after all right? The opposite would be true with opera or classical music that doesn't use amplification.

 

[tonmeister2008]

or amps that don't measure well sound good

My amp has NF, measures crappy by some measurements but I will put it up against other amps as IMO it is that good

If it measures crappy by some measurements and they are for example, THD measurements then that would explain why it still sounds good.

Most conventional nonlinear distortion measurements are not perceptually relevant. We are extremely sensitive to small amounts of cross-over distortion in amplifiers (waveform related and not easily masked) but not very sensitive to extra added harmonics that are masked by the music. You can have examples where THD of 15- 20% can sound better (or not even detected) and THD of < 1 % that sounds horrible. It's just a question of not measuring the right thing (ie what is perceptually relevant). Look at the AES papers by Drs. Earl Geddes and Alex Voishvillo about these topics. Alex has a great PPT presentation with audio examples that demonstrates this point quite well.Franz above talks about this as well.

I have found no correlation between loudspeaker preference and conventional nonlinear distortion measurements like THD. Neither have other loudspeaker researchers like Toole and Klippel.

However, the correlation between loudspeaker preference and the right set of frequency response measurements is very high: r = 0.86. As Toole says, if you could play poker with winning odds like that in Las Vegas, you would be quickly banned from the Casinos.

 

[DonH50]

It's just a question of not measuring the right thing (ie what is perceptually relevant).

Worth repeating, imo. I can measure with much higher resolution than we can hear, but it doesn't matter if I don't measure the right thing.

 

[Mark Seaton]

I have found no correlation between loudspeaker preference and conventional nonlinear distortion measurements like THD. Neither have other loudspeaker researchers like Toole and Klippel.

However, the correlation between loudspeaker preference and the right set of frequency response measurements is very high: r = 0.86. As Toole says, if you could play poker with winning odds like that in Las Vegas, you would be quickly banned from the Casinos.

Thank you Sean for such concise statements on this topic. The question of what is being measured and what is perceptually significant is the crux of the confusion and conflicting experiences of so many enthusiasts.

 

[Steve Williams]

Then why doesn't the industry set a standard for acceptable measurements and then everyone provides the same type of measurements.

This comes back to my postulate that many people will not even audition let alone own a speaker with suspect measurements

 

[tonmeister2008]

Thank you Sean for such concise statements on this topic. The question of what is being measured and what is perceptually significant is the crux of the confusion and conflicting experiences of so many enthusiasts.

Yep - that's it. And there is only a small minority of people in our industry who do the right loudspeaker measurements. And the rest of them wonder why the correlation between their measurements and perceived quality is not better.

The loudspeaker industry generally has a "don't ask, don't tell" policy on measurements: the measurements don't tell us anything meaningful, and if you asked for ones that did, it would be negatively prejudicial to their sales.

 

[tonmeister2008]

Then why doesn't the industry set a standard for acceptable measurements and then everyone provides the same type of measurements.

This comes back to my postulate that many people will not even audition let alone own a speaker with suspect measurements

See post #32, I think it partially answers your question.

 

[tonmeister2008]

Yep - that's it. And there is only a small minority of people in our industry who do the right loudspeaker measurements. And the rest of them wonder why the correlation between their measurements and perceived quality is not better.

The loudspeaker industry generally has a "don't ask, don't tell" policy on measurements: the measurements don't tell us anything meaningful, and if you asked for ones that did, it would be negatively prejudicial to their sales.

If there were perceptually relevant measurements that became audio industry standards this situation would probably happen less often. Why? Because the new performance specifications would better tell consumers the true sound quality of the product. Companies could no longer hide behind the curtain of meaningless specifications.

 

[Phelonious Ponk]

Yep - that's it. And there is only a small minority of people in our industry who do the right loudspeaker measurements. And the rest of them wonder why the correlation between their measurements and perceived quality is not better.

The loudspeaker industry generally has a "don't ask, don't tell" policy on measurements: the measurements don't tell us anything meaningful, and if you asked for ones that did, it would be negatively prejudicial to their sales.

I don't think that policy is limited to loudspeakers. Much of the "high end" seems to have abandoned measurement, discipline, and even the goal of "high fidelity" to the source, to a dogmatic subjectivism (when faced with the frightening possibility of scrutiny) laced with very unsubjective-sounding claims of superiority. It's not universal, of course, but it is pervasive. And it is personal pet peeve #1 .

P

 

[Mark Seaton]

Then why doesn't the industry set a standard for acceptable measurements and then everyone provides the same type of measurements.

This comes back to my postulate that many people will not even audition let alone own a speaker with suspect measurements

In my experience that is a small segment of the premium market, and most who follow this are on the internet forums. The biggest hurdle at the moment is that most consumers want a simple, single picture or 1-3 value metric to quickly look at. Sufficiently detailed measurements require greater interpretation and a more holistic view of the combination than do a single frequency response. Measurements also require some explanation or clarification as to what might be showstoppers vs. a compromise with benefits in other metrics. On axis smoothness vs. smooth trends in a well behaved off axis response is a good example. I know in many cases I've chosen not to post measurements which either aren't quite reflection free enough for public consumption or ones that I know will require tons of explanation which I might not have time to post up front when they are introduced. If left to answer later, many "internet experts" will come up with their own explanations which are usually as amusing as they are scarry.

Finally, while we would likely find more common ground than not between the conclusions and approach Sean has spent so much time researching and my own goals/targets, if you start polling more and more designers, you will find limited agreement. Those priorities get even more drastically varied as you get into the more exotic loudspeakers. It would be no small task to find agreement on "acceptable measurements" over a wide segment of the industry. At best I suspect it might be practical to set up clear methods and standards/guidelines as to measuring certain specific qualities similarl to what has been done for quite a while now in the pro audio world (ie EASE or CLF data).

 

[Steve Williams]

Mark

as always a very informative post.

 

[audioguy]

Companies could no longer hide behind the curtain of meaningless specifications.

But look at all of the nonsense products that high end audiophile purchase that don't have any specifications because there is nothing to measure because they are pure snake oil. There are some products, speakers specifically, that have, for example horrible FR measurements but some reviewer (who may have gotten a long term loan) will wax eloquent about how great it sounds (regardless of the measurements), and people will buy the product. Call me cynical.

Even reading some of the posts on these threads, there still seem to be those whose philosophy is: "don't bother me with the facts. My mind is made up!"

 

[RBFC]

But look at all of the nonsense products that high end audiophile purchase that don't have any specifications because there is nothing to measure because they are pure snake oil. There are some products, speakers specifically, that have, for example horrible FR measurements but some reviewer (who may have gotten a long term loan) will wax eloquent about how great it sounds (regardless of the measurements), and people will buy the product. Call me cynical.

Even reading some of the posts on these threads, there still seem to be those whose philosophy is: "don't bother me with the facts. My mind is made up!"

It's possible that the tin-eared reviewer has a room that compensates inversely for irregularities in FR of a given speaker, so it then sounds perfect....

Just take Valin's room as an example...

Lee

 

[audioguy]

Just take Valin

No thanks. You take him. He, IMHO, represents what is wrong with high end audio!