Comparative Listening Tests

Status
Not open for further replies.

Steve Williams

Site Founder, Site Owner, Administrator
I would just like to point out there are arrogant mean spirited and close minded people on just about any forum I can think of on any side of any issue, audio or not. Just goes with the territory. I have been told (not here of course) that if I can't hear a difference, my system is not resolving enough. Really? Just have to ignore that kind of stuff.

I agree Joe and IMO WBF has made great strides in cleaning up its act. It's certainly not perfect however I can tell you that this site was purged last year of most of those mean spirited individuals. We all know where they hang out now.

I do agree about using the "ignore"button. It works wonders
 

DaveyF

Well-Known Member
Jul 31, 2010
6,129
181
458
La Jolla, Calif USA
The ear hears nothing Davey. It is the ear+brain that tell us what is heard. As creatures, we are designed to be hopeful animals. We have a brain that is able to think of countless possibilities of bad things happening to us which would be paralyzing. As such, we can manufacture results and outcomes that simply are not real. But make us happy and positive.

So when you say you hear something, I believe that the combination of your ear+brain said that. Problem is we need to disentangle the ear out of that. Science tells us how to do that. We simply disable the other stimulus that tells the brain to act differently than the presented soundwaves. Do the testing that way, then no one, no objectivist, will deny you what you say.

I mean really. Doing formal testing is far more resource intensive and difficult than just doing the type of comparison testing audiophiles do. Why would audio science 100% rely on that kind of testing if what we do in audiophile world provides reliable results???

And how do we justify going against the entire body of audio research and say we know better when it comes to audio evaluation? Is there precedence for that in other sciences where we do that with straight face?


Science is always there to guide you to build reliable listening tests. Come up with a test that shows audible difference where the only thing that change was the audio entering your perception and the entire scientific community will rise up and listen. And that is the subject of this thread. How do we create reliable audible evaluations. That we know how to do. And it has nothing to do with explaining what the device does or does not do.

What happens though is that all such formal tests if done, will lead to negative outcome. Just as I think I read in the start of this thread. That failure is not a failure to explain the function of the device. It is a failure in our fantastic perception we think we have.

As audiophiles we like to think we have above average hearing abilities. It is one of the things that makes us walk around proud. Unfortunately test after test, including formals ones my group did, show audiophiles have no such abilities. On countless tests put forward on forums where we know, 100%, there are objective degradations, audiophiles either don't take the test, or take it and fail it. Yet someone like me with high loss of high frequencies can hear the artifacts 100% of the time.

I am no smarter or better audiophile than anyone. But rather, have training that tells me what to listen for both from audible point of view and design. Here is science 100% helping one become a better listener.

But we don't seem to want to go there. We want to dismiss all science as without value and rely on what is trivially shown to be an unreliable experiment over and over again. I mean did you do my test? It was all sighted. You could run it in 10 seconds. I hear the difference with all my training. It is that training that explains why I hear the difference. So how I don't put value on similar tests when the XYZ tweak is put forward as making a difference. I know that if nothing was changed to the sound, we would still hear a difference. And the more "group think" behind it, the more we hear it.

It is also important to understand that there is a discussion on merit, and there is another to settle score online. Vast majority of these discussions are motivated by the latter. You see examples of it on ASR Forum and here from the other camp. It is a dynamic that unfortunately cannot be avoided. Scars are created.

Anyway, I have to go back to the show. Let me just say that I have not read your thread on ASR Forum. I will do so when I get back and talk to people if they have been rude.

Amir, when I stated on your forum that I heard a difference by using the Shakti's, I meant it! Now, I couldn't give a rats ass whether that happens to fit into somebody's idea that this could not possibly be true unless there was some kind of scientific paper behind my experience. Your ear brain comment is something that makes little sense to me....you might as well have said that we cannot hear anything if we are dead...or that we have a hearing issue if we are deaf. Hello!! kind of obvious...no??
I have been a pro musician and a'phile for over 30 years, does this make me unable to hear something in music when I experience it...you tell me?
Just FYI, my young daughters who have no "training" in any of the areas that you profess...BUT who I will wager have far better hearing than you...or I...easily heard the difference with the Shakti's in my room and without. Sure science is there to build tests, but it is also there to VERIFY something that is reliably experienced by humans. IMHO, if the science doesn't exist yet to validate the experience, then that is simply because the science has not yet caught up with the reality. To state that something that is being experienced by somebody in a reliable fashion ( and BTW, I am clearly not the only one to have the same experience) is bogus, because the white paper doesn't exist, or the current science doesn't explain it--is highly unscientific and closed-minded/arrogant, IMO!:D
 

microstrip

VIP/Donor
May 30, 2010
20,806
4,698
2,790
Portugal
The cable industry is a profit glutted industry highly dependent on advertising and promotional wishful thinking. As long as the high profits persist, along with the willing customers, so will the mythology. Why would somebody even bother to design a new amp or speaker if all they had to do is package some wire with mumbo jumbo. It is so pervasive, that even the whole industry is on board because no manufacturer would want to admit that their devices work best with 'ordinary' wire, so it has taken on the self fulfilling character of a mass delusion. Even the critics have to buy into it.

The cable industry depends on advertising and promotional wishful thinking because the only way to prove the benefits of cables is listening to the cables. They need something to trap buyer attention and desire to try them.

The same arguments that people apply to cables can be applied to all else in the high-end. And manufacturers want people to try their equipment with expensive cables because they know it sounds better that way, promoting them with inferior performance cables probably result in sales losses.

And yes, I have complete looms of Mogami IC and speaker cables, DDK cable sets, as well as several top high-end cables, such as Cardas, Transparent Audio, Crystal and van den Hul in my listening room. But surely it is only my opinion based in my direct experience, nothing else. YMMV.
 

DaveyF

Well-Known Member
Jul 31, 2010
6,129
181
458
La Jolla, Calif USA
DaveyF, I read your Shakti thread on ASR. I give you credit for walking in front of that terribly rude and closed-minded firing squad.

Thank you, Ron. I had NOT realized how closed-minded and imbecilic posters at that forum could be. So, with that knowledge, I have now decided to give them their platform...without me.
 

Steve Williams

Site Founder, Site Owner, Administrator
Thank you, Ron. I had NOT realized how closed-minded and imbecilic posters at that forum could be. So, with that knowledge, I have now decided to give them their platform...without me.

+1
 

ddk

Well-Known Member
May 18, 2013
6,261
4,040
995
Utah
Dear Ron,

The answer to the questions you raise is "it depends!"

IME testing a single IC run through a loom of others can be taxing even in one's own system (maybe why Al wanted friends over?) no less for 3rd parties not intimate with the system, room listening to music they don't know. Peter's discomfort with the situation is perfectly understandable and in this case an extra unrelated variable was introduced to further complicate matters but this isn't an example of all A/B testing. I found that in many IC tests having several runs of what you're testing is very helpful and more decisive. Of course there are times that one is vastly superior to or different from the other then things are more obvious and sometimes you need longer time and more testing to figure it out.

A/B testing isn't limited to components it's a matter of course every time I set up a cartridge or bring in new speakers with their placement or trying different types of bearing oil with a tt, we all have done some of these things at one point or another. You're not deluding yourself if you know your friends system including room well enough to hear major changes, your positive or negative reaction to the sound will tell you if it's better or not or the same as ever.

There's merit to long/short/blind testing whatever the situation requires and works for you, the trick is to have clear standards and reference point and know what you're looking for otherwise you'll get sucked into the bottomless pit of sorting out differences and infinite upgrades...

david


In a post on Al M.’s thread “ZenWave Audio D4 Interconnect” Peter A., describing a comparative listening test at Al’s house, wrote:
. . . Despite your good descriptions of the differences between the D4 and Monster cables, I remain somewhat confused by the events of the recent audition. You essentially conducted an A/B/X test for me and David. Despite me being completely sure that the X cable was the same as B, I failed that test. That opened my eyes to the realities of such tests. Then, later in the evening, when we did the test again with different music, you switched the test without telling us. It was basically an A/B/C test, because you rotated the Tube Traps when we were outside of the room for the X part, changing it into C. That only confused me more.

. . .

Finally, I'd like to add a few thoughts about the A/B/X testing method. Not knowing what I was hearing made for a very intense listening session. During the A/B part, obviously we knew there were two different cables being heard, and we simply described the differences, if any, between the two. That part was fairly easy. The test changed when we heard the third X part. We did not know if you switched cables again or not. Now we were being asked to remember the sound of the previous two cables while listening to unfamiliar music and identify if the third cable was the same as the first or the second. I found this test to be fundamentally different from the first test. We were not listening for differences, but instead testing our recall ability. I thought I correctly identified it, but was clearly wrong. That knowledge of failing the test then influenced my listening for the rest of the evening and contributed to an overall very confusing listening session. I don't know much about blind testing. The fact that I failed it and that you are clearly certain which cable you prefer means either that this kind of testing is somehow flawed, . . . or that bias may enter into your decision, or something else entirely. I don't really know.

I think it is important to attempt to be intellectually honest. I want to commend Peter for being intellectually honest about a confusing situation in which he found himself with the A/B and A/B/X testing of Al’s interconnect cables.

While I have never met Peter in person, we have corresponded via e-mail and talked on the telephone extensively and frequently for several years now. I know Peter to be an extremely thoughtful, detailed-oriented, careful and conscientious listener. Also, importantly, Peter (unlike me) regularly listens to live music. I am confident Peter's ears are better, and more accurate, than mine. So, after reading Peter’s post, I ask myself: “If Peter gets confused in a test like this, what do we, individually and collectively, even think we are doing when we compare products and listen for changes in our systems?”

I have no answers, only questions. There are a lot of opinions about the merits and problems of A/B and A/B/X testing, and about the pros and cons of short-duration A/B and A/B/X comparisons versus long duration, spend-weeks-with-a-product listening with no back-and-forth comparison.

I think Peter’s experience inclines me to view even more skeptically long-duration "comparisons." As Peter wrote, the comparison, even in the short-duration time-frame of an A/B/X test, became a test of “recall ability,” not “listening for differences.” So what realistic hope does an audiophile have trying to remember how his system used to sound after he has been listening to a new component for weeks or even months?

Are we partially, or even completely, deluding ourselves when an audiophile visits a friend’s house to listen to music, and then returns weeks later to see if the audiophile can hear a difference wrought by some change the friend made in his system? (Let's not even think about the audiophile's differences in mood, restfulness, alcohol consumption, hunger level, stress, etc., between the two listening sessions.) After listening again weeks later, the audiophile reports hearing a significant difference in response to a minor tweak in his friend’s system (“the soundstage opened up significantly,” “the midrange glare is much less evident,” the noise floor is much lower,” "the highs are more extended," etc.). But if we can't even make reliable comparisons during the course of a single day how can we be possibly think that we can remember accurately what our friend’s system sounded like weeks ago? (Of course, critics of A/B and A/B/X tests argue there are problems and inherently confusing issues with short-term comparison tests which are solved by long-duration auditioning.)

I am not suggesting we stop auditioning components, stop pursuing tweaks or stop listening for significant or subtle differences. I am suggesting that perhaps we should be more realistic and circumspect -- and more skeptical -- about our expressed conclusions. We should attempt to do the best we can do, and to try to remain as intellectually honest as possible, but perhaps we should acknowledge that we may be fooling ourselves about some of our listening conclusions.
 

jkeny

Industry Expert, Member Sponsor
Feb 9, 2012
3,374
42
383
Ireland
I would just like to point out there are arrogant mean spirited and close minded people on just about any forum I can think of on any side of any issue, audio or not. Just goes with the territory. I have been told (not here of course) that if I can't hear a difference, my system is not resolving enough. Really? Just have to ignore that kind of stuff.

Yes, I agree - there are such people on every forum but when we see "intellectual honesty" being trotted out as applying to one side & not being called about the other 'sciencey' side, I like to get the balance correct.
When people continually proclaim that they have science on their side, I like to examine just what they know about science & the scientific method - 9 times of ten they come up as a fail.

Just want to set the record straight as it's easy to be fooled by science(y) talk
 

DaveC

Industry Expert
Nov 16, 2014
3,899
2,141
495
Dear Ron,

The answer to the questions you raise is "it depends!"

IME testing a single IC run through a loom of others can be taxing even in one's own system (maybe why Al wanted friends over?) no less for 3rd parties not intimate with the system, room listening to music they don't know. Peter's discomfort with the situation is perfectly understandable and in this case an extra unrelated variable was introduced to further complicate matters but this isn't an example of all A/B testing. I found that in many IC tests having several runs of what you're testing is very helpful and more decisive. Of course there are times that one is vastly superior to or different from the other then things are more obvious and sometimes you need longer time and more testing to figure it out.

A/B testing isn't limited to components it's a matter of course every time I set up a cartridge or bring in new speakers with their placement or trying different types of bearing oil with a tt, we all have done some of these things at one point or another. You're not deluding yourself if you know your friends system including room well enough to hear major changes, your positive or negative reaction to the sound will tell you if it's better or not or the same as ever.

There's merit to long/short/blind testing whatever the situation requires and works for you, the trick is to have clear standards and reference point and know what you're looking for otherwise you'll get sucked into the bottomless pit of sorting out differences and infinite upgrades...

david

Good post! It's true mixing and matching ICs may skew results if one of the ICs is very colored. Al did have 2 sets of D4s if needed.

As far as testing... imo it's important to combine multiple kinds of testing as short A/Bs will tell you immediate differences but there are often other effects that are more difficult to quickly identify such as subtle artifacts that cause listening fatigue, especially if you're not very experienced at identifying these artifacts. An example: Often people feel silver cables sound exciting and highly resolving but the truth may be the sound has accentuated leading edges and grain. This may not be realized until a month after the cables go in, when the music starts getting irritating and not as exciting as it used to be, you turn the volume down and listen less because it's irritating. When something sounds exciting initially you need to be very careful as it's often irritating over time. On the other end people are often seduced by beautiful, warm tone but over time it gets boring, the same tone is superimposed over all recordings and fine detail and spatial information is lost, along with your interest in listening to music.

Off topic a bit... Audio systems stimulate the nervous system and can be either soothing or alarming depending on the distortion/artifact. We shouldn't be seduced by either and instead seek the truth, this will lead to the most engaging system possible imo.
 

NorthStar

Member
Feb 8, 2011
24,305
1,323
435
Vancouver Island, B.C. Canada
DaveyF, I read your Shakti thread on ASR. I give you credit for walking in front of that terribly rude and closed-minded firing squad.

:D Lol Ron, that's a funny/realistic way to put it! I don't think I've read that thread...perhaps only few posts and I most likely went another direction...towards the peaceful ocean sitting on the fine sand of the beach.

Maybe each person's system is unique and valid? Maybe the art of listening and comparing takes a lifetime to master? Maybe we all are masters of our own kingdoms? :b
 

Ron Resnick

Site Co-Owner, Administrator
Jan 24, 2015
16,017
13,347
2,665
Beverly Hills, CA
Dear Ron,

The answer to the questions you raise is "it depends!"

IME testing a single IC run through a loom of others can be taxing even in one's own system (maybe why Al wanted friends over?) no less for 3rd parties not intimate with the system, room listening to music they don't know. Peter's discomfort with the situation is perfectly understandable and in this case an extra unrelated variable was introduced to further complicate matters but this isn't an example of all A/B testing. I found that in many IC tests having several runs of what you're testing is very helpful and more decisive. Of course there are times that one is vastly superior to or different from the other then things are more obvious and sometimes you need longer time and more testing to figure it out.

A/B testing isn't limited to components it's a matter of course every time I set up a cartridge or bring in new speakers with their placement or trying different types of bearing oil with a tt, we all have done some of these things at one point or another. You're not deluding yourself if you know your friends system including room well enough to hear major changes, your positive or negative reaction to the sound will tell you if it's better or not or the same as ever.

There's merit to long/short/blind testing whatever the situation requires and works for you, the trick is to have clear standards and reference point and know what you're looking for otherwise you'll get sucked into the bottomless pit of sorting out differences and infinite upgrades...

david

This all makes sense. Thank you, David.
 

stehno

Well-Known Member
Jul 5, 2014
1,585
456
405
Salem, OR
Good post! It's true mixing and matching ICs may skew results if one of the ICs is very colored. Al did have 2 sets of D4s if needed.

As far as testing... imo it's important to combine multiple kinds of testing as short A/Bs will tell you immediate differences but there are often other effects that are more difficult to quickly identify such as subtle artifacts that cause listening fatigue, especially if you're not very experienced at identifying these artifacts. An example: Often people feel silver cables sound exciting and highly resolving but the truth may be the sound has accentuated leading edges and grain. This may not be realized until a month after the cables go in, when the music starts getting irritating and not as exciting as it used to be, you turn the volume down and listen less because it's irritating. When something sounds exciting initially you need to be very careful as it's often irritating over time. On the other end people are often seduced by beautiful, warm tone but over time it gets boring, the same tone is superimposed over all recordings and fine detail and spatial information is lost, along with your interest in listening to music.

Off topic a bit... Audio systems stimulate the nervous system and can be either soothing or alarming depending on the distortion/artifact. We shouldn't be seduced by either and instead seek the truth, this will lead to the most engaging system possible imo.

Very good point, DaveC, regardling having similar cable designs, cable technologies to achieve a given cable mfg'ers max performance potential.

I discovered this entirely by accident a few years ago when I was comparing numerous sets and several mfg'ers of ic's and sc's (speaker cables) being cryo-treated via the vapor method verses the full immersion method. I'll forgo the details how I stumbled across this, but the end result was that one pair of ic's performance went from so-so to quite musical when its associated speaker cable was more of a technology match. Up to that point the ic in question was on its way to the storage closest.

Differences in designs / technologies of one can bottleneck the performance of the other. But then I suppose that's true anywhere throughout a system's chain.

Just one more thing I suppose to potentially compromise the legitimacy of a quick cable shootout.
 

stehno

Well-Known Member
Jul 5, 2014
1,585
456
405
Salem, OR
....

Maybe each person's system is unique and valid? Maybe the art of listening and comparing takes a lifetime to master? Maybe we all are masters of our own kingdoms? :b

Doesn't that go without saying? :)
 

NorthStar

Member
Feb 8, 2011
24,305
1,323
435
Vancouver Island, B.C. Canada
Thank you, Ron. I had NOT realized how closed-minded and imbecilic posters at that forum could be. So, with that knowledge, I have now decided to give them their platform...without me.

Davey, can I ask you something? Tim is also a musician, a nice guy too, like you and I; we all have ears, we all love music (playing and listening), you guys here are the cream of the crop, you all have vast experience with some of the best audio gear and loudspeakers, you all have lovely homes, wives, children, grand children, listening rooms, cars, families, watches, you are all very well educated, have hi-end jobs, retirement plans, travelling experience, you're all good diplomats, well versed in communicating skills, polite always, highly respectable, inviting, reinvigorating, security inspiring, and we all know good/great vibes from less appealing ones.

Why is it so hard to all the people to get alone all the time and everywhere? :b ...After all sweet music is supposed to soothe the soul of a man/woman/child.
Is there some misunderstanding, some judging, some beliefs so strong that some people are losing it without seeing clear @ the whole world around them?

Yes it's a beautiful Sunday Spring today, April 23, 2017. It's very peaceful around here, ...it smells good, it smells all the flower's aroma, the leaves growing on tree's branches, the bunnies running in the yards, the sheep and the cows and the horses and the eagles and the whales and the howls and the birds and the cats and the dogs and the humans all in harmony with the spring rising.

What can we all do to be more happy than that... Shut down our TV news, our computers, go for a long walk in a majestic forest high up in the mountain, and see how the wildlife is coping with it all? :b
 

Rodney Gold

Member
Jan 29, 2014
983
11
18
Cape Town South Africa
I would give reviewers cables for free .. the effects of a good review outweigh the cost of the cables
 

NorthStar

Member
Feb 8, 2011
24,305
1,323
435
Vancouver Island, B.C. Canada
Doesn't that go without saying? :)

Of course it does. :b The music and films we love is a direct reflection of who we are. Everything else is just cheap whiskey. :b
Our paths to new higher soothing discoveries are through the magic of music that touches us deeply the most, with all kind of emotions forming our comfort zone. The sharing of those discoveries is the power of living together in harmony for all seasons.

It's a great bunch here, one of the best in the west.
 

DaveyF

Well-Known Member
Jul 31, 2010
6,129
181
458
La Jolla, Calif USA
Davey, can I ask you something?

Why is it so hard to all the people to get alone all the time and everywhere? :b ...
? :b


Bob,when you have the answer to that question, please make sure that i am enlightened. For I have no idea as to the answer myself...except for the unfortunate fact that this is just human nature:rolleyes:.
 

rbbert

Well-Known Member
Dec 12, 2010
3,820
239
1,000
Reno, NV
I would give reviewers cables for free .. the effects of a good review outweigh the cost of the cables
It has been my understanding that a A/B/X comparator is no longer being made (and hasn't ben for a long time), so except in situations where something like the Foobar 2000 ABX software plug-in can be used (such as comparing 2 digital files) how are these comparison tests being performed?
 

Folsom

VIP/Donor
Oct 25, 2015
6,024
1,490
520
Eastern WA

cjfrbw

Well-Known Member
Apr 20, 2010
3,323
1,314
1,730
Pleasanton, CA
Several years ago the founder of DaVinci audio gave an interesting interview. He said that he didn't "believe" in exotic cables, but his customers did, so he had to manufacture the expensive cables for them and so that he wouldn't discredit his products by not making them available.

He was free to make such a remark, I think, because he knew that no matter what he said, the believers would still believe and the buyers would still buy. It is a market driven belief system.

I own one of these cables because it came with the DaVinci tonearm I bought, and was marketed separately for $2000. It is a nice stretch of oxygen free copper with some kind of miraculous crystal structure and some nice terminators. I think it would be an OK used deal for about $60, but it was already included with the used tonearm.

Everybody knows that the best way to stir vitriol, ad hominem, anger, spluttering rage, murderous bile and religious retaliation in audio is to open up a cable "discussion". It's about money and "belief". There's also the grey poupon aspect of having fire hoses attached to your high end speakers.

I honestly think there are a lot of audiophiles who profess to "believe" in cables, but don't, just so they can conduct civil conversations in audio without being attacked. If an audio critic claimed he didn't "believe", his career would probably come to a screeching halt right there.

Cables are conductors, yet they are imbued with characteristics of exotic equalizers, with all of the attendant cant about transparency, imaging depth etc. etc. They have to be the most awkward and roundabout "equalizers" ever. If you want to equalize, use an equalizer of some kind rather than an amorphous stretch of conductor.
 
Status
Not open for further replies.

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing