The Digital/Analog Dialectic

TerryO

Well-Known Member
Jan 26, 2011
7
2
308
By the way, Gary, be afraid of what you've started -- you do realise, I hope, that you're going to have to repeat that whole exercise using a top notch reel to reel as source ...;););)

Frank

Our Club has done many comparisons and blind tests over the years. I fully expect that the analog tape/ digital comparisons will be coming up in the future, however these comparisons do take quite a bit of effort by several individuals to be successful.

Best Regards,
TerryO
 

microstrip

VIP/Donor
May 30, 2010
20,806
4,698
2,790
Portugal
The important thing was that everyone had a good time. With all the responses tallied and the results were 50.88% got the correct answers, it's the same as flipping a coin. I don't see that there is anything statistically significant here to learn from.

If you were testing for a preference a global result would have some meaning. As most listeners do not get a statistic valid number of positive identifications, preference is meaningless.

But if you are testing for audibility of the AD-DA chain the good results obtained by the trained listeners suggest that the double conversion was audible, although the number of tests is then low. It will be very interesting to read their comments!
 

Ron Party

WBF Founding Member
Apr 30, 2010
2,457
13
0
Oakland, CA
Ron,
I'm not Gary, but I was very interested in the testing protocol, as I was rather involved in experimental design some years ago. There were no sighted tests performed. As the source (TT/LP) was the same, only the LED display would have indicated which configuration we were hearing at the time. The equipment was hidden behind an acoustic panel and even then, the gear faced away from the attendees. Whenever a change was requested, Gary made the switch using his remote and changed the display sign that indicated "A" or "B". It was for the listener to decide which (A or B) was analog and which was digital. There were no "double backs" using analog/analog or digital/digital.

Best Regards,
Terry Olson
Thanks for the information, Terry. Gary and I had exchanged posts about "double backs" at the beginning of this thread. Of course, it's easy for me to say what I would have liked as an armchair quarterback here.:) I wish I was there. I'm quite sure I would be drinking a pint of humility.
 

TerryO

Well-Known Member
Jan 26, 2011
7
2
308
If you were testing for a preference a global result would have some meaning. As most listeners do not get a statistic valid number of positive identifications, preference is meaningless.

But if you are testing for audibility of the AD-DA chain the good results obtained by the trained listeners suggest that the double conversion was audible, although the number of tests is then low. It will be very interesting to read their comments!

You're correct in that if "people" can't reliably identify which is which, then preference is meaningless. There is actually not enough data points to reach any definitive conclusions, but are perhaps suggestive enough to give direction to further research and discussions.

I believe that is what Gary meant as a dialectic.
As Hegal described the dialectic, it would consist of the Thesis (the original idea), which would be met by the Antithesis (the new idea) and the result of this would eventually result in the Synthesis (the outcome of competing ideas). At this point the Synthesis becomes the new Thesis and the process begins once again leading, over time, in the direction of Truth.

Best Regards,
TerryO
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
But if you are testing for audibility of the AD-DA chain the good results obtained by the trained listeners suggest that the double conversion was audible,

I don't think so. If I'm not mistaken, 73% is not enough to indicate more than chance in this number of trials, so what you have is one participant in 20 who correctly identified the samples enough to be indicative of anything. You also have 20 participants, 187 trials and a 50/50 rate of identification. I'm not sure anything has been indicated, statistically speaking, but if it has, at this point it's that the differences are not audible. What has been proven, at least to my satisfaction, is that it doesn't matter, and that the most sensible participants figured that out and started enjoying the music pretty quickly.

Tim
 

mep

Member Sponsor & WBF Founding Member
Apr 20, 2010
9,481
17
0
Tim-I agree with you and what you said pretty much mirrors what I said previously.

Mark
 

microstrip

VIP/Donor
May 30, 2010
20,806
4,698
2,790
Portugal
I don't think so. If I'm not mistaken, 73% is not enough to indicate more than chance in this number of trials, so what you have is one participant in 20 who correctly identified the samples enough to be indicative of anything. You also have 20 participants, 187 trials and a 50/50 rate of identification. I'm not sure anything has been indicated, statistically speaking, but if it has, at this point it's that the differences are not audible. What has been proven, at least to my satisfaction, is that it doesn't matter, and that the most sensible participants figured that out and started enjoying the music pretty quickly.

Tim

The main purpose of this dialectic was finding if the double conversion was audible. No one pretends being a listener in tests is an easy job or entertainment. Actually, many people refer that if you start enjoying the music you invalidate he test. :(

I used the word "suggests" because conclusions will depend on how you use the data.
Many people share the idea that listening tests carried with untrained listeners have very limited reliability, and can not be analyzed using this type of methodology, e.g. a single user that systematically makes wrong identification is considered always false, and a proper analysis can prove that he was able to hear a difference.

BTW 73% is a very good score in these type of tests - it would show a significant identification. However, as the number of trials is very low, the error bars are very large and the confidence level of the test is low.
 

Phelonious Ponk

New Member
Jun 30, 2010
8,677
23
0
Many people share the idea that listening tests carried with untrained listeners have very limited reliability

They do, though the data gathered by Sean Olive at Harman, the only valid data on the question that I'm aware of, indicates that untrained listeners are reliable, and reach the same conclusions as the trained listeners (and these are formally trained, not just experienced audiophiles or studio pros), they just take longer to get there.

BTW 73% is a very good score in these type of tests - it would show a significant identification.
But not with this number of trials and not with two "experienced" listeners out of 20. Equal numbers of listeners and trials would be required to even make a valid differentiation between experienced and inexperienced. So we're left with the numbers from 20 listeners in 187 trials with 50% identification. Only two conclusions can be reached: 1) that the "test" was not sufficiently controlled to be statistically meaningful. 2) That if it indicates anything, it is that the differences between the samples were inaudible. But by introducing the uncontrolled variable of the experienced listener in numbers far too small to be useful and totally out of balance with the rest of the test, we've left ourselves were we probably all want to be anyway -- free to continue believing what we want to believe. :)

Tim
 

garylkoh

WBF Technical Expert (Speakers & Audio Equipment)
Sep 6, 2010
5,599
225
1,190
Seattle, WA
www.genesisloudspeakers.com
This was a hobbyist study and not anything intended to withstand scientific scrutiny. Having stated this, what stands out so far as others have noted is the difference between trained (2) versus untrained (18) listeners.

Yes, this was a session that was to be fun, illuminating, and a launch platform for discussion. I hope that some of the people who were actually present will chime in and discuss this. Because I for one learned a great deal in the 4 weeks leading up to the meeting itself.

Gary, what tracks did you play which comprised the 23 (and 45) reports of no difference? What tracks did you play where the trained listeners did report accurately?

Various listeners reported various cuts with no audible difference. The full list of tracks played is here in post #107
http://www.whatsbestforum.com/showt...Analog-Dialectic&p=37917&viewfull=1#post37917

The highest report of no difference was Cut 7 - Ballet Music from Faust, 5 listeners reported hearing no difference, 4 listeners reported hearing a difference. It was also the track that was mostly a guess - 6 correct, 6 wrong.

There were only 2 tracks where nobody said that they heard no difference - Cut 1 - The Look of Love, and Cut 6 - la Campanella - the piano solo. They both also had the highest number of reports of hearing a difference - 10 and 8 respectively. However, hearing this difference, Cut 1 was where the most respondents got it wrong - 6 correct, 8 wrong.
 

garylkoh

WBF Technical Expert (Speakers & Audio Equipment)
Sep 6, 2010
5,599
225
1,190
Seattle, WA
www.genesisloudspeakers.com
By the way, Gary, be afraid of what you've started -- you do realise, I hope, that you're going to have to repeat that whole exercise using a top notch reel to reel as source ...;););)

Frank

Do you mean recording the LP with a reel to reel or inserting the A/D/A loop? If the latter, we'll have to find a tape deck with a built-in ADC. Otherwise, it will be comparing one interconnect to two interconnects plus a A/D/A loop. Then we'll get into a cable food fight :D :D :D
 

garylkoh

WBF Technical Expert (Speakers & Audio Equipment)
Sep 6, 2010
5,599
225
1,190
Seattle, WA
www.genesisloudspeakers.com
Thanks for the information, Terry. Gary and I had exchanged posts about "double backs" at the beginning of this thread. Of course, it's easy for me to say what I would have liked as an armchair quarterback here.:) I wish I was there. I'm quite sure I would be drinking a pint of humility.

Ron-I did want to do that, but some listeners might have guessed because the double back would have taken twice the switching time, and I wanted the switch to be as quick as possible. There was so little difference between A and B that I didn't think that the double back would have made a difference. Since we wanted the listeners to guess what A and B was, and we even put up a sign to show "A" or "B", the double back wouldn't have worked.
 

garylkoh

WBF Technical Expert (Speakers & Audio Equipment)
Sep 6, 2010
5,599
225
1,190
Seattle, WA
www.genesisloudspeakers.com
I don't think so. If I'm not mistaken, 73% is not enough to indicate more than chance in this number of trials, so what you have is one participant in 20 who correctly identified the samples enough to be indicative of anything. You also have 20 participants, 187 trials and a 50/50 rate of identification. I'm not sure anything has been indicated, statistically speaking, but if it has, at this point it's that the differences are not audible. What has been proven, at least to my satisfaction, is that it doesn't matter, and that the most sensible participants figured that out and started enjoying the music pretty quickly.

Tim

Tim, you are right. What has been proven to my satisfaction is that inserting an A/D/A loop does not take away from the enjoyment of the music. I picked the music very carefully to be sure that they are fun, enjoyable, as well as high resolution and hopefully defining. It does NOT absolutely show the audibility or non-audibility of the A/D/A loop.

For what's its worth, the one "trained" listener who identified correctly 6/6 his comment was "wider soundstage". He got both mono recordings wrong - may be he was a bad guesser, but there was no soundstage to comment on. The other 3 cuts he did not answer if he could tell a difference. Then, we are in the realm of unprovable because we still don't know how to measure if a system is able to deliver a "wider soundstage" or what parameter it is that reduces the soundstage when the A/D/A loop is inserted.
 

Ethan Winer

Banned
Jul 8, 2010
1,231
3
0
75
New Milford, CT
He got both mono recordings wrong - may be he was a bad guesser, but there was no soundstage to comment on.

A room without absorption at the reflection points will create its own sound stage, though of course that should remain constant as other things in the playback chain are changed.

we still don't know how to measure if a system is able to deliver a "wider soundstage" or what parameter it is that reduces the soundstage when the A/D/A loop is inserted.

On stereo material channel crosstalk can affect sound stage. Background noise could also have an affect if it's loud enough to drown out subtle reverb tails etc that are different on each channel.

--Ethan
 

garylkoh

WBF Technical Expert (Speakers & Audio Equipment)
Sep 6, 2010
5,599
225
1,190
Seattle, WA
www.genesisloudspeakers.com
A room without absorption at the reflection points will create its own sound stage, though of course that should remain constant as other things in the playback chain are changed.



On stereo material channel crosstalk can affect sound stage. Background noise could also have an affect if it's loud enough to drown out subtle reverb tails etc that are different on each channel.

--Ethan

Thanks, Ethan. In this case, since we are adding a A/D/A loop, crosstalk and background noise are not issues unless the ADC and/or DAC increase channel crosstalk or add noise.
 

Jaguar

Well-Known Member
Aug 2, 2010
221
17
925
Bellevue, WA
I had the good fortune of attending the meeting. For the guy who requested a do-over with tape...Bruce is our tape guy and I'm sure that seeing how much work it was for Gary to put this together has cured him of any notion of holding Part 2. This was definitely a Once in a Blue Moon event. Given the time and effort it took Gary, it's great to see such a huge reaction on the board.

While we can't say for sure whether the few semi-trained ears did better because of training or guessing, if you test 100 tracks with the most keen listeners in our club, Gary and Bruce, they would guess all of them correctly (possibly with 1,000 tracks). There's absolutely a difference, but as several people have pointed out, the results show that for the vast majority of us, it's not a big deal.

My subjective impression from the listening session is the vinyl was slightly more lively and more dynamic in the far outer reaches. Digital was a touch dryer and less colorful. Would I miss the difference if I had to go digital? Not unless I were building a "money no object" system.

I'm willing to go along with the "both are winners theme", which many have adopted (I did say I thought the vinyl sounded better), however, in my mind digital was a big winner and analog was a little winner. Digital has just finished taking the CD player down the path of the 8-track. It wasn't that long ago that I wondered out loud whether digital was capable of meeting and exceeding analog in sound quality from what I heard at the meeting, it's pretty much reached the point of meeting analog and logically, exceeding analog is the future.

Let's put this in a real-world perspective. The Burmester phono stage was $35k. I believe Gary said the Roksan turntable is $40k and I'm sure that doesn't include the cartridge. Now take a more common, stock, turntable setup...let's say a Linn (a fine piece of equipment) and try to make it sound as good as Gary's turntable (I expect even he can't do that). How would the digital have fared if we moved from the $3k Weiss DAC (albeit modified) and used the $20k model, then eleminated the A/D processing that was necessary for the demonstration?

We've heard a huge improvent when comparing Redbook to high-res 24/192 (32/384 should blow that out of the water). During the demonstration we were listening at (24/96). As better processors and higher resolution source files come along, I have no doubt we will reach a new level of listening nirvana in Digital, at least with our Lady GaGa and Justin Bieber collections.
 

Ethan Winer

Banned
Jul 8, 2010
1,231
3
0
75
New Milford, CT
In this case, since we are adding a A/D/A loop, crosstalk and background noise are not issues unless the ADC and/or DAC increase channel crosstalk or add noise.

Of course, and I understood the context. I'm just explaining some things that could account for a change in sound stage. That, plus moving your head half an inch in any direction. :D

--Ethan
 
Last edited:

microstrip

VIP/Donor
May 30, 2010
20,806
4,698
2,790
Portugal
They do, though the data gathered by Sean Olive at Harman, the only valid data on the question that I'm aware of, indicates that untrained listeners are reliable, and reach the same conclusions as the trained listeners (and these are formally trained, not just experienced audiophiles or studio pros), they just take longer to get there.

You are using data out of the context of this experiment - the Olive experiments looked for preference in sound quality with noticeable different speaker systems, not audibility, and were using a scoring system .

Toole (Sound Reproduction) is much more straight about what he calls "Familiarity with the task". He shows that you need a proper procedure to initiate new listeners and you should positively train them - until they get the sensation that their results are not random. "Plan to discard the result of the first few tests. Experienced listeners are operational immediately."

This was not a scientific experiment, and the statistics are poor. But after this test, if I had to bet on the result of a test carried with 100 trials with an experienced listener I would bet he will get a significant positive identification factor (better than 60%). What would be your bet? (I hope betting is not forbidden in this forum!) :)
 

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing