Page 1 of 57 123456789101151 ... LastLast
Results 1 to 10 of 565

Thread: Comparative Listening Tests

  1. #1
    Site Founder And Administrator Ron Resnick's Avatar
    Join Date
    Jan 2015
    Location
    Beverly Hills, CA
    Posts
    3,091

    Comparative Listening Tests

    In a post on Al M.’s thread “ZenWave Audio D4 Interconnect” Peter A., describing a comparative listening test at Al’s house, wrote:


    . . . Despite your good descriptions of the differences between the D4 and Monster cables, I remain somewhat confused by the events of the recent audition. You essentially conducted an A/B/X test for me and David. Despite me being completely sure that the X cable was the same as B, I failed that test. That opened my eyes to the realities of such tests. Then, later in the evening, when we did the test again with different music, you switched the test without telling us. It was basically an A/B/C test, because you rotated the Tube Traps when we were outside of the room for the X part, changing it into C. That only confused me more.

    . . .

    Finally, I'd like to add a few thoughts about the A/B/X testing method. Not knowing what I was hearing made for a very intense listening session. During the A/B part, obviously we knew there were two different cables being heard, and we simply described the differences, if any, between the two. That part was fairly easy. The test changed when we heard the third X part. We did not know if you switched cables again or not. Now we were being asked to remember the sound of the previous two cables while listening to unfamiliar music and identify if the third cable was the same as the first or the second. I found this test to be fundamentally different from the first test. We were not listening for differences, but instead testing our recall ability. I thought I correctly identified it, but was clearly wrong. That knowledge of failing the test then influenced my listening for the rest of the evening and contributed to an overall very confusing listening session. I don't know much about blind testing. The fact that I failed it and that you are clearly certain which cable you prefer means either that this kind of testing is somehow flawed, . . . or that bias may enter into your decision, or something else entirely. I don't really know.

    I think it is important to attempt to be intellectually honest. I want to commend Peter for being intellectually honest about a confusing situation in which he found himself with the A/B and A/B/X testing of Al’s interconnect cables.

    While I have never met Peter in person, we have corresponded via e-mail and talked on the telephone extensively and frequently for several years now. I know Peter to be an extremely thoughtful, detailed-oriented, careful and conscientious listener. Also, importantly, Peter (unlike me) regularly listens to live music. I am confident Peter's ears are better, and more accurate, than mine. So, after reading Peter’s post, I ask myself: “If Peter gets confused in a test like this, what do we, individually and collectively, even think we are doing when we compare products and listen for changes in our systems?”

    I have no answers, only questions. There are a lot of opinions about the merits and problems of A/B and A/B/X testing, and about the pros and cons of short-duration A/B and A/B/X comparisons versus long duration, spend-weeks-with-a-product listening with no back-and-forth comparison.

    I think Peter’s experience inclines me to view even more skeptically long-duration "comparisons." As Peter wrote, the comparison, even in the short-duration time-frame of an A/B/X test, became a test of “recall ability,” not “listening for differences.” So what realistic hope does an audiophile have trying to remember how his system used to sound after he has been listening to a new component for weeks or even months?

    Are we partially, or even completely, deluding ourselves when an audiophile visits a friend’s house to listen to music, and then returns weeks later to see if the audiophile can hear a difference wrought by some change the friend made in his system? (Let's not even think about the audiophile's differences in mood, restfulness, alcohol consumption, hunger level, stress, etc., between the two listening sessions.) After listening again weeks later, the audiophile reports hearing a significant difference in response to a minor tweak in his friend’s system (“the soundstage opened up significantly,” “the midrange glare is much less evident,” the noise floor is much lower,” "the highs are more extended," etc.). But if we can't even make reliable comparisons during the course of a single day how can we be possibly think that we can remember accurately what our friend’s system sounded like weeks ago? (Of course, critics of A/B and A/B/X tests argue there are problems and inherently confusing issues with short-term comparison tests which are solved by long-duration auditioning.)

    I am not suggesting we stop auditioning components, stop pursuing tweaks or stop listening for significant or subtle differences. I am suggesting that perhaps we should be more realistic and circumspect -- and more skeptical -- about our expressed conclusions. We should attempt to do the best we can do, and to try to remain as intellectually honest as possible, but perhaps we should acknowledge that we may be fooling ourselves about some of our listening conclusions.
    Mono and Stereo - Senior Contributing Reviewer

    turntable: American Sound AS-2000 (ordered) tonearm: SME 3012R (to come); cartridge: ZYX Universe Premium X-SB2 or Koetsu Coralstone (TBD)
    tape: Studer A820 Mk II
    phono stage: Aesthetix Io Eclipse with 2 power supplies; line stage: Aesthetix Callisto Eclipse with 2 power supplies; amplifier: VTL MB-750s
    speakers: Gryphon Pendragons
    cables: MasterBuilt Ultra or Cardas Clear Beyond (TBD); power: Benjamin Electric subpanel; Torus AVR60BAL; JPS in-wall wire; Furutech NCF outlets; room: 20' wide X 25' long X 15' tall; treatment: ASC Tube Traps and Tower Slims; tweaks: Shakti Stones, Shun Mooks (Kedar to place)

  2. #2
    Addicted to Best! RogerD's Avatar
    Join Date
    May 2010
    Location
    BiggestLittleCity
    Posts
    2,878
    In the end it is a extremely personal experience. I have found I focus only on one marker and that is clarity,otherwise it becomes a crap shoot. Ymmv
    Speakers: ParAcas Satellites ParAcas Dual 16 subwoofer Velodyne FSR18 Fulton Little Maestros Mission speakers
    Electronics: Agtron Platinum Mono Blocks Accuphase C200 preamp Accuphase P300 amplifer Parasound amplifier ParAcas Sierra electronic crossover
    Digital: Monarchy NM24 DAC Sony DVD Transport
    Tuner: McIntosh MR78
    Cables: NBS Statement,Pro,Master Cables, Nirvana Digital SPDIF Cable
    Turntable: Linn LP12 Technics 1200
    RTR: Ampex 350,351,440BC,MR70
    Studer C37 60hz,Crown 2-844,824

  3. #3
    Addicted to Best!
    Join Date
    Feb 2014
    Location
    Wayne, PA
    Posts
    1,056
    Great post Ron. I have been involved in several AB and ABX listening tests. They can be quite stressful, usually as a result of our own doing. I prefer to use music which I am very familiar with. Usually 2 minutes per track is enough. There are times when the differences are obvious. Most times, they are very close. In those cases, I take the approach that if I really have to think really hard whether I hear a difference or think I do but I am not sure, I simply am not hearing a difference. Trying to recall auditory memory is a difficult task as you note.

  4. #4
    Member Sponsor Addicted to Best!
    Join Date
    Sep 2013
    Location
    Greater Boston
    Posts
    3,084
    Quote Originally Posted by Joe Whip View Post
    Great post Ron. I have been involved in several AB and ABX listening tests. They can be quite stressful, usually as a result of our own doing.
    I agree that blind AB and ABX listening tests can induce psychological stress, as well as expectations, which can be detrimental to our cognitive abilities (I should have remembered that in the above test). Perhaps the best way may be to listen for some extended time (an hour or more) with one component in a stress-free manner, and then switch back to the old component for a while to assess if there is any meaningful difference. And repeat. It may take some time to figure out where the differences really are, and if they are real, you'll eventually lock into them and be easily able to confirm that they are there. Once you think you have firmly figured out a difference that is meaningful to you, then a blind test to confirm it might make more sense -- if it's even needed at that point.

  5. #5
    VIP/Donor [VIP/Donor] microstrip's Avatar
    Join Date
    May 2010
    Location
    Portugal
    Posts
    12,101
    Quote Originally Posted by Ron Resnick View Post
    (...) I am not suggesting we stop auditioning components, stop pursuing tweaks or stop listening for significant or subtle differences. I am suggesting that perhaps we should be more realistic and circumspect -- and more skeptical -- about our expressed conclusions. We should attempt to do the best we can do, and to try to remain as intellectually honest as possible, but perhaps we should acknowledge that we may be fooling ourselves about some of our listening conclusions.

    IMHO skepticism should be on the side of the reader. I hope people feel free enough to express their opinions without fear that they are wrong. Opinions in the case you address are mainly driven by preference and clearly detailed. Also IMHO the described tests are meaningless.

    Preference is the trigger (and perhaps the reason) of hyperbolic comments in the high-end. As long it is clearly expressed I can live with them and appreciate them.
    Under construction around a pair of Wilson XLF's , the Forsell Air Force One, the ARC Phono 2SE and the DCS Vivaldi 2.0 stack : ARC REF40 + ARC REF250, TA OPUS MM2 +TA XL digital, TA XL gen V power cables and CenterStage feet ...

  6. #6
    Site Founder And Administrator Ron Resnick's Avatar
    Join Date
    Jan 2015
    Location
    Beverly Hills, CA
    Posts
    3,091
    Quote Originally Posted by Al M. View Post
    . . . Perhaps the best way may be to listen for some extended time (an hour or more) with one component in a stress-free manner, and then switch back to the old component for a while to assess if there is any meaningful difference. And repeat. . . .
    This makes sense to me.
    Mono and Stereo - Senior Contributing Reviewer

    turntable: American Sound AS-2000 (ordered) tonearm: SME 3012R (to come); cartridge: ZYX Universe Premium X-SB2 or Koetsu Coralstone (TBD)
    tape: Studer A820 Mk II
    phono stage: Aesthetix Io Eclipse with 2 power supplies; line stage: Aesthetix Callisto Eclipse with 2 power supplies; amplifier: VTL MB-750s
    speakers: Gryphon Pendragons
    cables: MasterBuilt Ultra or Cardas Clear Beyond (TBD); power: Benjamin Electric subpanel; Torus AVR60BAL; JPS in-wall wire; Furutech NCF outlets; room: 20' wide X 25' long X 15' tall; treatment: ASC Tube Traps and Tower Slims; tweaks: Shakti Stones, Shun Mooks (Kedar to place)

  7. #7
    Senior Member twitch's Avatar
    Join Date
    Jun 2010
    Location
    SE Pa
    Posts
    164
    Ron, an intelligent post for sure. The powers of psychoacoustics combined with so many 'audiophiles' trying out for Editor in Chief of TAS with regards to their over the top reviews has had me laughing for years.

    Your last paragraph sums it up well !
    Cheers - Dave

  8. #8
    Member Sponsor Addicted to Best!
    Join Date
    Feb 2014
    Location
    London
    Posts
    6,983
    Blog Entries
    6
    Quote Originally Posted by microstrip View Post
    IMHO skepticism should be on the side of the reader. I hope people feel free enough to express their opinions without fear that they are wrong. Opinions in the case you address are mainly driven by preference and clearly detailed. Also IMHO the described tests are meaningless.

    Preference is the trigger (and perhaps the reason) of hyperbolic comments in the high-end. As long it is clearly expressed I can live with them and appreciate them.
    I disagree, many times someone else's experience is dismissed as his preference. While in reality he would have actually compared to establish a preference
    Audition recordings: Zero Distortion Link; Reference Components: Zero Distortion Link; Own: Lampi GG One-day-to-get: Apogee Scintilla, or a horn (with Tenor 75 OTL ), Preamp to be fit in to suit the gain and drive the power amp. Analog: TBD

  9. #9
    VIP/Donor [VIP/Donor] microstrip's Avatar
    Join Date
    May 2010
    Location
    Portugal
    Posts
    12,101
    Quote Originally Posted by bonzo75 View Post
    I disagree, many times someone else's experience is dismissed as his preference. While in reality he would have actually compared to establish a preference
    Can't understand what you mean exactly with your second sentence.
    Under construction around a pair of Wilson XLF's , the Forsell Air Force One, the ARC Phono 2SE and the DCS Vivaldi 2.0 stack : ARC REF40 + ARC REF250, TA OPUS MM2 +TA XL digital, TA XL gen V power cables and CenterStage feet ...

  10. #10
    VIP/Donor [VIP/Donor] Folsom's Avatar
    Join Date
    Oct 2015
    Location
    Eastern WA
    Posts
    1,864
    There's no end to exploiting this horse's remains...

    First off I would say some people have better and worse memory, and general abilities. But one's abilities for memory aren't the bigger factor. You need to make mental notes, or written notes, on the things you notice. If you have those then you can compare them to the new experience. If you never tried to discern anything while listening, it's going to be much harder to know the difference when you arrive.

    As far as blind testing the irony in this might be that humans cannot do it without a form of attachment. I'm under a strong impression after so many years that the key to hearing differences in audio isn't recall but association. First off if you listen to a stereo for say several weeks and then change something, you'll know for sure whether it was you who changed it and whether or not you were informed. You might not believe it, but you will notice and wonder.

    During ABX, blind, whatever, tests the biggest issue may be that you can't form any expectations for the sound you're hearing. This is because you have no association. My suggestion is you assign colored lights or cards or something with the different components you're swapping. The person being tested hasn't a clue what color corresponds to what component. But here's the factor, either one component has two colors, or one color has two components. That way the person being tested can try to discern between say yellow Pass amp or yellow Mulla amp noticing that yellow is inconsistent while say Blue is always the same NAD amp; or they can note that two colors sound the same. You simply have a small questionaire that they can fill out.
    Industry Affiliation: Folsom

Page 1 of 57 123456789101151 ... LastLast

Similar Threads

  1. Chord DAVE, Aune S16 etc. - more comparative listening (rambling & tedious)
    By acousticsguru in forum Digital Audio Forum: DAC, Transports, Digital Processing
    Replies: 40
    Last Post: 07-08-2017, 08:23 AM
  2. Playback Designs Merlot - comparative listening session (long, rambling, tedious)
    By acousticsguru in forum Digital Audio Forum: DAC, Transports, Digital Processing
    Replies: 32
    Last Post: 09-14-2016, 01:49 PM
  3. Replies: 80
    Last Post: 07-17-2010, 04:12 PM
  4. Why Live-Versus-Recorded Listening Tests Do Not Work
    By tonmeister2008 in forum Science In The Service Of Art/Dr. Sean Olive
    Replies: 7
    Last Post: 07-11-2010, 03:30 PM
  5. Listening Tests On Room Correction Products
    By Ron Party in forum Digital Room Eq Forum
    Replies: 7
    Last Post: 07-09-2010, 07:23 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •