The Half Life of Expectation Bias

[microstrip]

So, then you agree that the Harman speaker studies prove that since a majority of listeners believe that harmon speakers sound better then they are better?

No Tom, you have to weight and analyze the data. If our weighting system is not the Harman one it only means that in conditions defined by Harman more people preferred their speakers, so Harman feel they can say they are better.

Considering your naive question, I am starting to believe you never read the original papers of Harman people, just the Harman marketing and forum debates. Otherwise you would ask me the really tricky questions ...

 

[Orb]

.......So you're calling Walter Marsh and Carl Jung tweako engineers? And major capacitor manufacturers like Rel Cap tweako engineers? And the military that uses most of these caps tweako engineers? Actually if you ever saw capacitors designed, wound and built, you could never say the above statement. But that's OK because we know that nothing has changed and improved in the last 20 years and all high-end audio designers are idiots.....

Made me smile
Agree with you Myles and appreciate last sentence is also sarcasm (ok may seem obvious but global forums can cause our very own Lost in Translation... but your no Scarlett Johansson booo lol).
Cheers
Orb

 

[jkeny]

No Tom, you have to weight and analyze the data. If our weighting system is not the Harman one it only means that in conditions defined by Harman more people preferred their speakers, so Harman feel they can say they are better.

Considering your naive question, I am starting to believe you never read the original papers of Harman people, just the Harman marketing and forum debates. Otherwise you would ask me the really tricky questions ...

I haven't read the Harmon papers but I've seen it mentioned that they only test one speaker, not a stereo pair. Is this true? If so it should be very obvious that they are looking at a subset (half) of the normal speaker use in a domestic environment.

I also saw reference to the idea of blind Vs sighted listening where I think they used their employess in the test - is this correct? Again, if it is, I would cite (see what I did there ) it as a deeply skewed test.

I may be wrong in both of these which I hope I am because Harmon seem to be trying to advance the measurements to audibility correlation which is an admirable & hopefully successful endeavour.

 

[Orb]

I haven't read the Harmon papers but I've seen it mentioned that they only test one speaker, not a stereo pair. Is this true? If so it should be very obvious that they are looking at a subset (half) of the normal speaker use in a domestic environment.

I also saw reference to the idea of blind Vs sighted listening where I think they used their employess in the test - is this correct? Again, if it is, I would cite (see what I did there ) it as a deeply skewed test.

I may be wrong in both of these which I hope I am because Harmon seem to be trying to advance the measurements to audibility correlation which is an admirable & hopefully successful endeavour.

They do both blind and sighted and use various listeners but the problem is consistent use of trained listeners (assume why many of their subjective testing use their own employees), one interesting side effect they noted from the combined blind-sighted test was how subjective quality-preference changed depending upon the boundary position of the speaker.
When the speakers were in the same position blind, they scored a noticable-statistical difference, suggesting that we are biased cognitively to visual boundary-positions of produced sound (probably I assume to do with expectation bias and how we internally "model" sound by also environment-location and source).
This is from memory and relates to one quite lengthy study Sean Olive and the team did.

Cheers
Orb

 

[jkeny]

They do both blind and sighted and use various listeners but the problem is consistent use of trained listeners (assume why many of their subjective testing use their own employees)

I've no problem with them using trained listeners, in fact I believe it is necessary for the detection of slight differences but surely one must pause when using own employees to evaluate blind Vs sighted listening particularly when Harmon's own speakers are in the mix? Again no problem with employees in blind tests though

, one interesting side effect they noted from the combined blind-sighted test was how subjective quality-preference changed depending upon the boundary position of the speaker.
When the speakers were in the same position blind, they scored a noticable-statistical difference, suggesting that we are biased cognitively to visual boundary-positions of produced sound (probably I assume to do with expectation bias and how we internally "model" sound by also environment-location and source).
This is from memory and relates to one quite lengthy study Sean Olive and the team did.

Cheers
Orb

Yea, that's interesting Orb - if I understand what you said, my thinking is the same as yours - this difference is due to the fact that we probably use visual cues to rationalise ambiguities that we hear in the sound field - again it's down to the mechanisms we use & are responsible for our ability to create an auditory scene, I believe. Taking away the visual surety of the source location leaves us with this ambiguity which I would assume affects our listening. One thing to be noted for blind testing of audio equipment - don't hide the speakers!

What about the purely mono testing? Is this correct or am I mistaken?

 

[Orb]

Yeah I appreciate your comments about the challenge of brand loyalty and agree it does add a pause for thought, I guess it depends upon the scale of use and level of control/scope, and yes you got the exact meaning of what I meant.
The last question; oh man it opens up a lot of discussions (and I must say it has caused many a debate on various forums and I am sure with Sean directly - seen some tbh) they do use a single speaker.
I am also not fully happy about mono use, but then I do not think they could create a controlled stereo environment without excessive costs-time and being able to "control" or monitor/weight all variables that it may bring into such subjective tests.

Cheers
Orb

 

[jkeny]

Yeah I appreciate your comments about the challenge of brand loyalty and agree it does add a pause for thought, I guess it depends upon the scale of use and level of control/scope, and yes you got the exact meaning of what I meant.
The last question; oh man it opens up a lot of discussions (and I must say it has caused many a debate on various forums and I am sure with Sean directly - seen some tbh) they do use a single speaker.
I am also not fully happy about mono use, but then I do not think they could create a controlled stereo environment without excessive costs-time and being able to "control" or monitor/weight all variables that it may bring into such subjective tests.

Cheers
Orb

Irrespective of whatever the excuses are for using purely one speaker (difficulty of setting up the test) do they (Harmon) or Sean Olive admit to the fact that they may be only dealing with a subset of a speaker's performance? Do they acknowledge the limitations of this test method & try to put some context & calibration on it?

If not then I consider the tests no more than sophisticated marketing, highly polished using attempted scientific credibility as the wax. Edit: Maybe that's a bit strong & I got carried away with my words but you get my thinking, I'm sure - the tests themselves & their limitations need to be spelled out!

Again, forgive my questions I have only a passing knowledge of Harmon & their testing.

EDIT: I've used the adage before about the guy looking under the light of a street lamp for his dropped keys, even though he's sure he dropped them, in the darkness, 20 yards away. His excuse - it's where the light is

 

[jkeny]

I forgot that Sean is a member here & don't wish to insult him or his work - I'll go to have a read of his section of the forum & try to research some more - I believe he has done some tests that correlate mono listening preferences with stereo listening preferences which I will try to find?

 

[Orb]

All I can say is that it has been discussed quite a lot in the past in a lot of places and with Sean who is pretty open about it from memory, now you see why I said "oh man" because the mono test does evoke some debate
And yes tests-studies and papers need to be seen in the right context/scope.
Cheers
Orb

 

[microstrip]

I haven't read the Harmon papers but I've seen it mentioned that they only test one speaker, not a stereo pair. Is this true? If so it should be very obvious that they are looking at a subset (half) of the normal speaker use in a domestic environment.

I also saw reference to the idea of blind Vs sighted listening where I think they used their employess in the test - is this correct? Again, if it is, I would cite (see what I did there ) it as a deeply skewed test.

I may be wrong in both of these which I hope I am because Harmon seem to be trying to advance the measurements to audibility correlation which is an admirable & hopefully successful endeavour.

I can tell you are mostly wrong, but not completely - but IMHO it is impossible debating the Harman work without having read it. It is too large and deep to be summarized in a few sentences, and unfortunately what their more active supporters in audio forums refer is just the iceberg tips that show in Harman marketing.

We should remember that several known audio writers disagree openly with their fundamental points with valid reasons.

 

[jkeny]

All I can say is that it has been discussed quite a lot in the past in a lot of places and with Sean who is pretty open about it from memory, now you see why I said "oh man" because the mono test does evoke some debate
And yes tests-studies and papers need to be seen in the right context/scope.
Cheers
Orb

Sure, I've come across these discussions in my search for a paper which backs up his assertion that the preferences for single speaker listening correlate well to the same preferences for stereo speaker listening - this seems to be at the kernel of his tests & I wonder where the data/studies to support this assertion are to be found?

In other words - to refer back to my analogy - I'm looking for a study which concludes that looking under a street lamp for one's dropped keys is just as effective as searching for where you think you dropped them in the darkness!

I did see on his blog a test of room correction devices again done in mono. When asked why, he answered "a previous room correction study we did that found listeners were more discriminating of room correction benefits in mono versus stereo and surround." I'm not sure how to interpret this answer - is it that the light is better over here so lets search over here - "more discriminating" could be understood to mean this?

Again, I'm not saying the tests are not valuable but their limitations need to be clearly identified or the scientifically proven correlation between mono & stereo listening established. But it does make me wonder about these tests & their exclusion of the auditory scene analysis aspects of our hearing which to a large extent is based on ITD & IID - both of which require stereo reproduction!

 

[Orb]

Jkenny totally agree with you, why I am very careful on what I take from these studies (or any study tbh), and so does Sean tbh I feel; although you mentioning the room correction benefits reminds me that it IMO felt like a stretched goal-project too far.
Cheers
Orb

 

[jkeny]

Lest people think that this is gone way off topic, I believe it is fundamental to the thread's OP - I believe that for most people long term listening is where we finally get rid of our biases & begin to hear the sound of our device/system - it's only then that we judge it based on it's realism to our stored models of how we encounter sound in our everyday experience & how we have experienced it behaves & how we perceive it. Some people can do this quicker than others!

However, these areas can be very tricky to tease out & we constantly need to cross-check - for instance, what is loss of dynamics/detail & what is more accurate to the actual recording? A recent example caught me out on this when listening to my DAC I thought it had more detail & snap than some other DACs being auditioned only to find that what I was hearing was RF noise being injected through a connected PS. My preference was for this added detail until I found out it was distortion.

 

[GaryProtein]

. . . . However, these areas can be very tricky to tease out & we constantly need to cross-check - for instance, what is loss of dynamics/detail & what is more accurate to the actual recording? A recent example caught me out on this when listening to my DAC I thought it had more detail & snap than some other DACs being auditioned only to find that what I was hearing was RF noise being injected through a connected PS. My preference was for this added detail until I found out it was distortion.

WOW!

That sounds a lot like this from an older closed thread:

How does one really know what they hear?

First and foremost, while one can say that a system with more detail retrieval is better, the catch here is what is real detail verses FR and added distortions that seem to create more detail?​

 

[jkeny]

WOW!

That sounds a lot like this from an older closed thread:

OK, don't wish to cause a closing of the thread but simply wanted to point out how tricky listening can be.
Remember that this was in a afternoon listening session of about 3 or 4 DACs with other people & so we were changing over & listening for the length of a song.
My mistake may well have been sorted out over the long term with extended, relaxed listening, as more subtle sound scene anomalies became noticeable - who knows?

 

[GaryProtein]

I'm not blaming anybody or pointing fingers.

I just found the similarity to be large.

I agreed with your statement that I quoted.

 

[GaryProtein]

. . . simply wanted to point out how tricky listening can be.
Remember that this was in a afternoon listening session of about 3 or 4 DACs with other people & so we were changing over & listening for the length of a song. . . .

I agree with this too.

 

[jkeny]

I'm not blaming anybody or pointing fingers.

I just found the similarity to be large.

I agreed with your statement.

Sure & my expectation bias was probably fully at play here - listening to my DAC
The guy who set it up already knew the outcome & was making a point - a point well made, I might add.
It was an easy demo - didn't need the extended listening - pulled out the offending PS & listened again.

 

[Steve Williams]

Posts deleted

Guys, please take a deep breath. This style serves no purpose and is frowned upon by the admin staff at WBF

 

[mep]

First off, the fact that you prefer a Dynaco 70 over a hafler DH220 power amp shows me how far apart your hearing and mine is.

My hearing is the same distance apart from yours as Myles' hearing is. I owned a brand new Hafler DH-220 that I built as a kit and later did the power supply upgrade from Frank Van Alstine. This was the early 1980s. I traded a cartridge I hated to a guy from work for his Dynaco ST-70 amp that he had bought new and built from a kit so it was a one owner in beautiful shape in the early 1980s. I brought the amp home and installed some 5 way binding posts on it because I didn't care for the stock screw terminals. I bought a matched quad of Siemens EL-34 tubes and biased them to the correct value and gave it a whirl. Yes, the bottom end on the Hafler DH-220 was more extended and tighter than the ST-70, but the midrange where music lives and breathes was much the better on the ST-70. I sold my DH-220 and never looked back. You can make fun of the ST-70 all you want, but there is a reason David Hafler sold a gazillion of them and they are still bringing good money on the used market. Much more money than a DH-220 brings on today's market so that should tell you something.

That amp sounds just like it would, fat and thick compared to the hafler power amp. I know all about the comparisons, I have done them. The measurements also provide the same results. I truly have nothing to say to you about your hearing acuity......you like thick IMD and bloat and to you that's good sound....have at it.

You are mainly mentioning the difference in bass quality which should be huge between a much older 35 watt tube amp and a much more modern 220 watt SS amp. However, on my transmission line speakers I had way back then, the bass from the ST-70 wasn't nearly as bad as you describe it. As for the measurements, who cares? Talk about the almighty midrange where the DH-220 falls short of the ST-70. Did you ever actually own an ST-70 or are you just going by spec sheets when you are making your comparisons? I played the DH-220 and the ST-70 for lots of people and I can't recall a single person ever having a preference for the DH-220.

The bottom line is that the comparison between a DH-220 and the ST-70 isn't really a fair one to make due to age and power output even though the ST-70 whoops some DH-220 ass in the midrange. How about comparing an ARC D-76 which was in production from 1974-1977 to the DH-220? Now we have a much more modern tube amp with 75 watts of power per channel and a much more advanced power supply than the ST-70. And yeah, I owned a D-76 and after I heard it, the ST-70 quickly went up for sale. So fast forward 30 years from 1984 when the DH-220 came on the market and compare a modern tube amp to the DH-220. And again, this isn't really fair because the Hafler DH-220 was never meant to be a SOTA amp. It only cost a couple of hundred bucks and change in kit form if memory serves me correctly and it sounded like it and looked like it.