Truth and Tonality: can they co-exist?

[Gregadd]

RUR-Same outcome as flipping a coin?

Thus proving my point about not peforforming a proper statitistical analysis.

 

[RUR]

Thus proving my point about not peforforming a proper statitistical analysis.

Gregg, if I make 10 observations and only six of them are correct, the odds of my continuing the test and ultimately getting a statistically significant % correct are pretty much nil. It's pretty obvious by then that I can't tell the difference.

 

[Gregadd]

There's no requirement that the test sample be new and unfamiliar to the subject, Gregg. ITU-R BS.1116-1, which is often cited as a basis for audio DBT's, stipulates that test samples be selected which best show the differences between the pieces of gear under test (I'm paraphrasing). When I did my DAC DBT's, I spent considerable time listening to each DAC and selected musical passages which best showed the differences I thought I heard (sighted). I then conducted the DBT using those passages. In some cases, the differences I thought I heard sighted were not borne out in the DBT. In others, it was.

I did not say that that it is required that it be unfamiliar. Only that so often the room, equipment, and source material is fostered on the examinee for the first time.

 

[Ron Party]

I did not say that that it is required that it be unfamiliar. Only that so often the room, equipment, and source material is fostered on the examinee for the first time.

I agree (in part). That is a significant problem with the testing process. And then given that everyone is an expert on the internet, the process is used as a sword to challenge claims of difference or preference. Most definitely inappropriate and certainly less than cordial.

Now having stated that, the process can be and has been properly utilized in matters audio. Many times. So in consideration of the fact that we are in the midst of a dialectic, please don't throw the baby out with the bath water.

 

[Gregadd]

Not sure what the question was that led to your answer Gary but here is some measurements:

Pass Aleph 3:
Fig.1 shows the Aleph 3's frequency response. It is nearly ideal—even with our simulated real load, the deviation from flat is inconsequential.

And BAT:

Fig.1 Balanced Audio Technology VK-55SE, High output tap, frequency response at 2.83V into: simulated loudspeaker load (gray), 8 ohms (left channel blue, right red), 4 ohms (left cyan, right magenta), 2 ohms (green). (1dB/vertical div.)

So what do we learn here?

Someone should be fired at stereophile magazine . Why? Why on earth would they use different scales to show the same graph? One goes to 50 KHz, another to 200 KHz. The first graph only has the simulated load but the second has two other. Why not have the same measurements for both so that we can properly compare?

So let's take the simulated load as that is the common thread. The BAT surely has wild variations in the response.

Now let's read some of the subjective words from the review:

"....the VK-55SE was a tad more tonally rich delivering the chorus than the Atlas or Nu-Vista, and the Atlas had a bit more sparkle than the Nu-Vista and the '55SE. ..."

"...the sound was vivid, relaxed, and liquid. Delmoni's 1780 J.B. Guadagnini violin has an unusually big sound, and the '55SE reproduced that bigitude, and the Guadagnini's timbral impact, with clarity—it didn't sound too big or too bright, but just right...."

Let me ask you this: which one of these two evaluations is more reliable? If I measured the unit again, would it produce wildly different results?

What if I had someone else review the BAT? Would they use the same words or even similar ones? What if I took the markings off the BAT and put in the box with the name Mark Levinson on it making it look like a solid state amp? Would the above words change in the review then? What if it said Carver or Sony on it? What if I said it cost $500 for the amp or $50,000? Would the words change?

Above all, how are the above words any different than any other high-end amp review? And what do the words really mean anyway? Sparkle means what? More highs? What does "ease" mean?

So again, without judging the execution, how would you rate the reliability of each evaluation type?

Measurements as you point out can be incompetenlty conducted and interpreted as you point out. I'll say it again for the last time. You can't destroy the rule by citing a corrupt example.

 

[Gregadd]

With all do respect Ken you miss the point. There is a fundamental difference with your choices being consistent with guessing and actually guessing. Did you guess.? I don't think so. You thought you heard a difference. When you flip a coin each outcome is a completely independent event. What you got on the first try has nothing at all to do with the outcome of the second flip. While it is unlikely that you could get ten heads in a row you could. When you get "6 out of ten" naysayers want to say that it is consistent with guessing. Is it not just as likely that you were wrong four times and right 6 times. When you have two equally likely outcomes, you have not proven the negative. You have just failed to prove the positive. Have you seen any mathematical formula saying how many trials you have to run and what percentage you need to obtain a significant result. I've been looking for a good horn book so I can teach myself statistics.
Do it right now. Flip a coin ten times and see what happens. I trust you. I bet you will not get five heads. If you do, you wont get it exactly on the second set of flips. My own observations show that they often use a computer generated coin flip that goes up to a million flips..

 

[microstrip]

(...) Let me ask you this: which one of these two evaluations is more reliable? If I measured the unit again, would it produce wildly different results? (...)

Once again, we have a language problem. What are you evaluating? The sound of the amplifier? Or its capacity to produce beautiful graphs? Or do you just want consistency of the opinion?

Evaluation per se does not mean anything.

BTW, the effect of amplifier output resistance in sound is well known and documented - I remember a test where a single ended tube(SE) amplifier output resistance was measured and an equivalent resistance was fitted in series with the Krell. However it seems that the modified Krell did not sound like a SE.

 

[RUR]

Statistical Power

 

[Gregadd]

I take it then Ken you agree that a score of six out of ten would prove neither that you could or could not hear a difference?
To prove your point you would need 16 out of 20 (80%) and to prove the null hypothesis`you need less than 1 in 20 (%5)

 

[amirm]

Measurements as you point out can be incompetenlty conducted and interpreted as you point out. I'll say it again for the last time. You can't destroy the rule by citing a corrupt example.

I didn't point out that the measurements were incompetent. What was incompetent was lack of standards at the magazine so that casual observers would not be confused by the sharp drop off in one curve that goes to 200 Khz vs one that only goes to 50 Khz.

The measurements here are quite revealing that the tube amp is highly sensitive to load impedance and therefore, subjective observations could be all over the map relative to the load that observer uses. The measurement then, is highly informative with respect to tonality of this amp whereas listening test highly problematic and representative of a tiny fraction of the configurations one might be interested in.

This brings me to some diagnosis of why audiophiles put so much emphasis on their ears vs measurement. And that is the fact that they have a hard time understanding measurements whereas by listening, they can quickly decide what they think of the sound. Take the above graph. I know what a simulated speaker load is. I have seen the circuit diagram and I know that it provides varying impedance to the driving source. I also know that tube amps have a harder time with that and hence, the graph "makes sense" to me. Absence of that such information, it is easy to dismiss valuable data as just random scientific jargon.

I was recently reading measurements of a DAC in a magazine. The graphs and measurements were explained so poorly that even I had to squint to understand them. Do you know what it means to have the response of a dithered 1 Khz signal? How about undithered? Not one word was said about why these test tones were used and what they are designed to measure. They also give so little space to the measurement section so there isn't room to explain anything even if one wanted.

Net of that is your observation where you thought the measurements were faulty where in reality, I was siting them as very direct evidence of performance difference between the two amps. And how much insight they provide in figuring out what load would work with them and if not, the impact on frequency response.

We talk about not trying to convert the other side in these arguments. What's left then is learning the topic well. You can't argue against other evaluation types unless you know them as well as I do. Where is the thread that asks, "what do all these measurements mean?" Instead, we have long threads about why they don't matter. Well, you have to know them before you can say they don't matter.

 

[amirm]

Once again, we have a language problem. What are you evaluating? The sound of the amplifier? Or its capacity to produce beautiful graphs? Or do you just want consistency of the opinion?

Evaluation per se does not mean anything.

Pick any definition you like and then tell me the consistency of evaluation using the three methods. I am trying to get us at the Forrest level not the trees .

 

[microstrip]

(...)
but my main point is:
But, those who ingore measurements in no way are looking for fidelity to the source, as are those who say it sounds better or right to them. The only way to measure fidelity to the source is with measurements. And, only a few folks on this forum tend to adhere strictly to the rule that they want their gear to be accurate to the source.
(..)
Tom

Let us suppose you persuaded me and I want to built the more accurate system in the world. Can you help me enumerating the measurements I should look for to select the system and how to interpret them?

 

[Gregadd]

Amir I wish I could cross examine you.

The reason we put more emphasis on listening is because that is our role in this puzzle. We are listeners.

 

[amirm]

While I have you wrestling with the last question , here is another. Every system has one more weaknesses. How do you know the material you are using brings those out? Surely a few tracks is not representative of all music ever created.

When I test compressed music, I never use classical music even though that might be your choice of picking audio equipment. Why? Because classical music tends to be harmonic in nature which is much easier to compress. There is not one classical music in the standard set of test clips MPEG uses for development of new compression standards for example. Yet they have a specific Suzanne Vega where is just her signing with no instruments. We use that because we know compression systems have trouble with it and we are much more able to find artifacts there, than in 99% of classical music.

At CES, Usher was handing out these sample CDs and the notes said they were designed to find issues in audio systems. I came home and thought nothing of it until faced with a tough issue where one of our two audio system in the showroom wasn't sounding right to me. We kept playing different things and people would say well, it sounded fine on this and that. So I pull out that disc, play track 2 or 3 and it was so obvious: the system had a massive suck out in mid bass. There was not one person in doubt. We went back and retuned the system and boy, did the quality come back up. So much so that we were all smiling.

 

[garylkoh]

Not sure what the question was that led to your answer Gary but here is some measurements:

Thanks for the measurements Amir. I said:

I agree that those terms are hopelessly fuzzy. As are the terms that I mentioned in the post above. However, I personally don't think that THD, IMD and FR tells me the whole picture.

I can see in FR a tonally "warm" amplifier with slightly elevated 80Hz to 160Hz response. I can also listen to an amplifier that I think sounds "warm" but has a perfectly flat frequency response. Where I'm struggling is - is there a way to measure this besides what we already know?

And I was asked:

Do you have an example of a "warm" component with a flat FR and distortion and noise low enough to be of little concern?

Tim

I had an Aleph 3 for evaluation some years ago, and owned a VK-60. I thought that both were "warm" sounding with good measurements.

The deviations from flat for the BAT is due to the tube design having a higher output impedance than the SS Class A amplifier. Even then, within the range of 20Hz to 20kHz, it does not deviate more than +/-1.5dB - which is excellent considering that loudspeakers are "good" if they deviate no more than +/- 3dB. My point is that neither amplifier has an elevated FR in the "warmth" region, but they are amplifiers that I subjectively characterize as "warm".

 

[amirm]

Amir I wish I could cross examine you.

Don't look but I am doing that to you

The reason we put more emphasis on listening is because that is our role in this puzzle. We are listeners.

You are no different than Tim and I in that respect. We use our audio equipment to listen to music and not mow the lawn . The difference is that we use sum total of all evaluation tools we have. In your corner, you are dismissing two out of three even though I can show you, heck prove to you in a court of law , that they are incredibly useful and powerful to help us what is best.

 

[Gregadd]

I assure you I am confident of my ability to assemble a first class music reproduction system. That's not to say I don't have plenty to learn. It's trying to influence you guys that has turned me into a mental contortionist.

I am happy that you were able to find a defect using just your ears. When it comes to audio you are superior to the genral population. I'm not sure there is a question in there for me.

 

[Ethan Winer]

So Ethan, you buy all of your audio gear based on measurements and without listening to it/them before you plunk down your cash?

In many cases Yes. When I needed a mixer for my home studio it was not convenient to set up a complete recording path at the local music store. In fact, I don't think they even had the model I wanted in stock. But I knew that Mackie makes quality gear, with good specs, and 20 years later the same mixer is serving me well. Same for the Crown power amps and Rane active crossover I bought at the same time.

But microphones and loudspeakers are different, and are very difficult to characterize by specs because there are so many variables. Not that specs can't tell what's good. But no speaker makers offer all the specs needed to make an intelligent decision. So when I bought a high quality microphone in the 1990s, a retailer let me take home three different brands knowing I'd keep only one and return the others after trying them all on my cello and other sources.

Same for the Mackie speakers in my HT, which I had heard many times at friend's studios, and same for my big JBL speakers which I knew were good from having heard them before. But electronics? Listening is not really needed unless a vendor lies about the performance.

Measurements can be predictors of sound quality but that's it.

This is not the first time you made a bold statement with nothing to back it up. If you believe that measurements alone are not adequate for assessing electronic gear, it is incumbent on you to explain why not. If you can't explain further, I honestly don't understand why you continue to cling to that position. It's almost as if you prefer to believe in magic and the unknowable.

Sometimes I forget that you are listening through a $150 Pioneer receiver and I need to keep that in mind in order to put your comments in perspective.

Back to insults eh? That's especially funny given this quote from another of your posts:

I'm not bothered by wow, flutter, crosstalk, or inner groove distortion because I don't hear any of those distortions raising their ugly heads in my system.

So the minuscule distortion in my Pioneer receiver is terrible, but somehow ten times more distortion in your LP setup is okay?

--Ethan

 

[Ethan Winer]

Michael Fremer-THE HUMAN EAR IS FAR SUPERIOR TO ANY MEASUREMENT DEVICE IN DETERMINING THE TOTALITY OF WHAT'S HEARD.

Michael Fremer is perhaps the most incompetent audiophile I know. He believes you can "demagnetize" an LP record to improve the sound. He doesn't even understand the frailty of his own hearing.

What is with you people who make a statement thinking that alone is sufficient proof? It is drop-dead trivial to prove that Fremer is incorrect in that statement, and I have done so many times.

--Ethan

 

[garylkoh]

Gary,
I wrote it some months ago in another thread. Happy you like poisons!

Subtractive and Additive?

A clarification - it's not that I like poisons. But if I am already poisoned, and a second poison can counteract that first poison, then I'll happily take that second poison.

Most of the best hifi components are not completely neutral in the sense they do not change the signal - if they were neutral they would sound all the same, and I have never had two different preamplifiers that sound the same. Some of them manipulate the signal in a way it helps us recreate our image of music.

Do you thing that great designers add something to the signal that enhances the music or they simply remove nasty information that masks our perception of some information existing in the recorded music?

How great designers get to neutral can be as individual as how a musician interprets a tune. Take two preamps that I consider to be the absolute best in terms of transparency, neutrality, (insert your own adjective here) - the FM Acoustics 255 and the SMc Audio VRE-1. I've owned the FM Acoustics preamp for over 20 years (first a 266, and now a 255). I had the SMc Audio for nearly 6 months on review so that I could decide if I wanted to use if for my CES demo.

They both achieve what they do through completely different ways.

FM Acoustics measures to an obsession - every component (active and passive!!) is measured and toleranced. They do not use exotic components like we are used to seeing in other high-end products. Nevertheless, each component is carefully selected, and then measured and matched. At the end of it, listening is still carried out.
See here: http://fmacoustics.com/company_methods.html

SMc Audio listens to an obsession - every component (active and passive) is listened to. Decisions are even made as to whether a component is hung under the circuit board or stood up on top of the circuit board because it sounds different (if you don't believe me, place your phono stage or DAC upside down). Measurements are used as a guideline, and listening is used as a process.

I believe that they both deliver truth AND tonality. Do they sound the same? No. Do either of them add or take away anything? I don't know - because I cannot know what the CD/LP/HighRez file sounds like without the reproduction system in the way.