The Digital/Analog Dialectic

Phelonious Ponk · Feb 15, 2011

garylkoh said:
Thanks, Tim. One less test to do.... although I'm surprised that 16bit vs 24bit would be transparent.

I've done for myself 24bit/48kHz vs 24bit/96kHz and satisfied myself that it's hardly audible except for very specific material.

Not at all. One successful study cries out for verification. I don't belong to AES, so I can't get to the actual paper (I did once...don't know where it went), but here is a great summary/commentary from someone who knows his stuff. It even has a reference to one of our members here!

http://mixonline.com/recording/mixing/audio_emperors_new_sampling/

Tim

microstrip · Feb 15, 2011

Phelonious Ponk said:
Not at all. One successful study cries out for verification. I don't belong to AES, so I can't get to the actual paper (I did once...don't know where it went), but here is a great summary/commentary from someone who knows his stuff. It even has a reference to one of our members here!

http://mixonline.com/recording/mixing/audio_emperors_new_sampling/

Tim

Tim,
I quote from your link :

"Most of the objections were based on the fact that the authors didn't include in their paper the list of equipment and recordings that they used. Meyer explains that part of that reason was to keep the article from getting too long. But anyone familiar with the type of debate that often occurs in tweak circles knows that had the authors been specific about the components, they would have immediately been attacked on the basis that their equipment was, of course, inferior to what they should have used, and so, of course no one would hear any difference."

This general last argument is very valid - even Toole refers it in his work and accepts it .

Although some vague references to brands are made in the remaining article, unless all the details are not given I would not consider this test as valid.

BTW , 37% for the women in this type of test is a brilliant result - 13% far from the 50% due to hazard, meaning they could distinguish but not associate. If someone gets 0% he has fantastic hearing!

Phelonious Ponk · Feb 15, 2011

I think the most interesting thing about the Meyer and Moran study is how clearly it demonstrates the futility of the subjectivist vs. science argument. While the study was very well designed (I think they had some experienced help) Meyer and Moran are not scientists or engineers or even statisticians. They are audiophile journalists. As such, they knew exactly what they would be up against if the results didn't come out in favor of the conventional audiophile wisdom, so they tweaked, if you'll excuse the term, their study accordingly. Scientists would have used a system that measured very well within the audible range (they probably would have used headphones, actually) and participants who hear well within the audible range, run their tests and published their findings. And other scientists would have accepted the results after appropriate peer review and awaited verification.

Meyer and Moran knew that it woldn't be that simple with the audiophile community. So they ran tests that went on for more than a year. They used multiple systems, from very highly regarded (and accurately measuring) pro monitoring systems to very highly regarded and accurately measuring audiophile systems. They ran their test by young, well-trained ears (recording and mastering students), male ears, female ears, studio pro ears and, yes, audiophile ears. They ran their tests in studios, testing facilities and the comfortable, stress-reducing atmosphere of homes. They ran several hundred trials, and regardless of the system, the listeners or the venue, the results were statistically identical: Inaudible.

These guys worked extreme over time to meet the rigors of science while simultaneously anticipating the objections of an anti-science community they know very well. If any single body of evidence should shake the foundations of audiophile beliefs and cause even the most unwavering anti-measurement subjectivists to think twice (if not many times) about how much they should "trust their ears," when their eyes are open, Meyer and Moran is it.

And it doesn't even seem to give them pause.

Tim

PS: Micro -- Meyer and Moran published a full equipment list in the Boston Audio Society journal two months after the study was released, but I don't think you can get access without a membership.

microstrip · Feb 15, 2011

Phelonious Ponk said:
I think the most interesting thing about the Meyer and Moran study is how clearly it demonstrates the futility of the subjectivist vs. science argument. While the study was very well designed (I think they had some experienced help) Meyer and Moran are not scientists or engineers or even statisticians. They are audiophile journalists. As such, they knew exactly what they would be up against if the results didn't come out in favor of the conventional audiophile wisdom, so they tweaked, if you'll excuse the term, their study accordingly. Scientists would have used a system that measured very well within the audible range (they probably would have used headphones, actually) and participants who hear well within the audible range, run their tests and published their findings. And other scientists would have accepted the results after appropriate peer review and awaited verification.

Meyer and Moran knew that it woldn't be that simple with the audiophile community. So they ran tests that went on for more than a year. They used multiple systems, from very highly regarded (and accurately measuring) pro monitoring systems to very highly regarded and accurately measuring audiophile systems. They ran their test by young, well-trained ears (recording and mastering students), male ears, female ears, studio pro ears and, yes, audiophile ears. They ran their tests in studios, testing facilities and the comfortable, stress-reducing atmosphere of homes. They ran several hundred trials, and regardless of the system, the listeners or the venue, the results were statistically identical: Inaudible.

These guys worked extreme over time to meet the rigors of science while simultaneously anticipating the objections of an anti-science community they know very well. If any single body of evidence should shake the foundations of audiophile beliefs and cause even the most unwavering anti-measurement subjectivists to think twice (if not many times) about how much they should "trust their ears," when their eyes are open, Meyer and Moran is it.

And it doesn't even seem to give them pause.

Tim

PS: Micro -- Meyer and Moran published a full equipment list in the Boston Audio Society journal two months after the study was released, but I don't think you can get access without a membership.

Tim,

If you do not understand statistics, you can not make studies that need statistical analysis .

I find bizarre that you can fully endorse results from a test you do not fully know the details.

My audiophile objections are, as usual, with your generalizations - if this test was carried under specific objective conditions its scope is only these conditions. If , for example, they used cables that audiophile community consider inadequate, do not expect that audiophiles will recognize the results.

Gary carried some very interesting tests recently. Would the result be the same if he used his state of the art 1.2 loudspeakers?

Misquoting someone in this forum

- I open my hears when I listen, but I keep my eyes open to read papers.

Bruce B · Feb 15, 2011

What I don't understand is why the Journal wouldn't allow rebuttal. Several highly regarded people in the industry that have been published submitted letters and documentation for rebuttal and were shot down. They wouldn't hear of it. They weren't disagreeing with the results, but with the testing methods.

c1ferrari · Feb 15, 2011

Bruce B said:
What I don't understand is why the Journal wouldn't allow rebuttal. Several highly regarded people in the industry that have been published submitted letters and documentation for rebuttal and were shot down. They wouldn't hear of it. They weren't disagreeing with the results, but with the testing methods.

Hmm...this is a source of consternation.

c1ferrari · Feb 15, 2011

TerryO said:
Ron,
I'm not Gary, but I was very interested in the testing protocol, as I was rather involved in experimental design some years ago. There were no sighted tests performed. As the source (TT/LP) was the same, only the LED display would have indicated which configuration we were hearing at the time. The equipment was hidden behind an acoustic panel and even then, the gear faced away from the attendees. Whenever a change was requested, Gary made the switch using his remote and changed the display sign that indicated "A" or "B". It was for the listener to decide which (A or B) was analog and which was digital. There were no "double backs" using analog/analog or digital/digital.

Best Regards,
Terry Olson

Hi,

Proviso -- I haven't perused this thread, thoroughly. For this event, was there a demonstration of analog material/chain and digital material/chain, prior to the inventory/survey/session, to inform and provide a reference for the participants?

Phelonious Ponk · Feb 15, 2011

Tim,

If you do not understand statistics, you can not make studies that need statistical analysis .

Are you speaking to my understanding of statistical analysis, or Meyer and Moran's?

I find bizarre that you can fully endorse results from a test you do not fully know the details.

A) I've seen the equipment list, but I don't have it on my hard drive to hand to you. You can probably find it out there somewhere if it is important to you.

B) I'm not endorsing anything. It appears to be well-done and very thorough. It passed pretty stringent standards for publication. But a study like this is put out there as a set of findings in an individual study, nothing more. It needs more studies, peer review, verification. Show me a couple of more studies - different systems, conditions, subjects with essentially the same results and I'll think about endorsing it, for what my endorsement is worth. But I think the study is interesting on its own. And like I said in my post above, I think the reaction to it is even more interesting. You're proving that point.

My audiophile objections are, as usual, with your generalizations - if this test was carried under specific objective conditions its scope is only these conditions.

I've never claimed anything more. What generalizations are you referring to?

Gary carried some very interesting tests recently. Would the result be the same if he used his state of the art 1.2 loudspeakers?

I don't know, but I'll bet if Gary didn't think the speakers being used were up to the task he wouldn't have run the tests. Ask Gary.

Misquoting someone in this forum - I open my hears when I listen, but I keep my eyes open to read papers.

Misquoted indeed. What that means is that you really don't trust your ears until you are willing to close your eyes. As long as you know what is playing, you're just playing. You are "testing" nothing but your own expectations.

Tim

Phelonious Ponk · Feb 15, 2011

c1ferrari said:
Hmm...this is a source of consternation.

It would be if it were accurate. The very first link on this subject I provided was an open path for rebuttal --

https://secure.aes.org/forum/pubs/journal/?ID=2

Plenty of room for rebuttal there. Did you think they should give the rebuttals equal weight with a disciplined year-long study of several hundred trials? Then the rebuttal should come as another study, another set of conditions, another report. And it should meet AES' standards for publication. Or alternatively, the rebuttals should be treated as the were treated - as the internet postings of people objecting to the hard work of others that they were unwilling or unable to do themselves. I'd love to see real rebuttals - more studies building upon or challenging Meyers and Moran.

Tim

c1ferrari · Feb 15, 2011

Phelonious Ponk said:
It would be if it were accurate. The very first link on this subject I provided was an open path for rebuttal --

https://secure.aes.org/forum/pubs/journal/?ID=2

Thanks, Tim, for the link

microstrip · Feb 15, 2011

Phelonious Ponk said:
Are you speaking to my understanding of statistical analysis, or Meyer and Moran's?

I am referring to your statement that the authors were not statisticians.

Phelonious Ponk said:
B) It appears to be well-done and very thorough. It passed pretty stringent standards for publication.

Sorry, as I told you, standards for publication that accept papers on audibility without specifying exactly the systems and conditions are not stringent.

Phelonious Ponk said:
I've never claimed anything more. What generalizations are you referring to?

Considering that "regardless of the system, the listeners or the venue, the results were statistically identical: Inaudible" when a group of population scored 37%.

amirm · Feb 15, 2011

Seems like there is too much weight is being put on AES paper review process. And for that matter, any organization like that. That's not how the system works. Having had people on my team that did such reviews, I can tell you that they are simply looking for college kids who were sleep in their math class. Just about anything else can be presented. There is no judge and jury deciding if the substance of the paper is correct or the claim proper. As long as the writer is competent or appears competent, the paper is allowed.

And the only "peer review" which occurs is in person, when the paper is presented and the audience can ask questions.

Phelonious Ponk · Feb 15, 2011

I am referring to your statement that the authors were not statisticians.

Well, they either got help or did their homework. The study was well-designed.

Sorry, as I told you, standards for publication that accept papers on audibility without specifying exactly the systems and conditions are not stringent.

I guess we'll have to agree to disagree on the standards of AES.

Considering that "regardless of the system, the listeners or the venue, the results were statistically identical: Inaudible" when a group of population scored 37%.

There are no generalizations there. That refers to the specific findings of the specific study. The fact that you will not consider that study valid unless you can specify the interconnects has nothing to do with the generalization, or lack thereof, of pointing to the consistency of the findings of the study as it stands.

Tim

Phelonious Ponk · Feb 15, 2011

Now this, is a generalization if you're looking for one, Mark:

Having had people on my team that did such reviews, I can tell you that they are simply looking for college kids who were sleep in their math class. Just about anything else can be presented.

That is assuming that you meant to say "looking for kids who weren't asleep in their math class," Amir.

Tim

fas42 · Feb 15, 2011

That is assuming that you meant to say "looking for kids who weren't asleep in their math class,"

No, he was right the first time -- they're looking to exclude stuff that mathematically doesn't add up ...

Frank

Phelonious Ponk · Feb 15, 2011

fas42 said:
No, he was right the first time -- they're looking to exclude stuff that mathematically doesn't add up ...

Frank

Then that would be "looking for kids who were sleeping in their math class." Not the highest standard for publication, but a start, I suppose.

Tim

amirm · Feb 15, 2011

Phelonious Ponk said:
That is assuming that you meant to say "looking for kids who weren't asleep in their math class," Amir.

Tim

As Frank pointed out, the process is what to exclude, not what to include. So I meant what I said. Imagine if you are reading college papers and one looks like it was written by a sixth grader. That is what gets taken out. Good or bad or indifferent, the others are included. Note for example that Robert Harley was also "published" in AES: http://www.aes.org/e-lib/browse.cfm?elib=5540

garylkoh · Feb 15, 2011

c1ferrari said:
Hi,

Proviso -- I haven't perused this thread, thoroughly. For this event, was there a demonstration of analog material/chain and digital material/chain, prior to the inventory/survey/session, to inform and provide a reference for the participants?

Hi Sam, I'm not sure what you are asking. It wasn't a digital source vs analog source. It was an analog source, and the insertion of a A/D/A loop. Prior to the 11 cuts I played, I played two different pressings of Jazz at the Pawnshop to establish that the system had good resolution and transparency and was not "dumbed down" in any way.

Phelonious Ponk · Feb 16, 2011

amirm said:
As Frank pointed out, the process is what to exclude, not what to include. So I meant what I said. Imagine if you are reading college papers and one looks like it was written by a sixth grader. That is what gets taken out. Good or bad or indifferent, the others are included. Note for example that Robert Harley was also "published" in AES: http://www.aes.org/e-lib/browse.cfm?elib=5540

Indeed. In light of Hartley's publication in the AES journal, I'll stop using the publication as some indication of legitimacy. In the case of Meyer and Moran, it's not necessary anyway.

Tim

garylkoh · Feb 16, 2011

For those members who were not able to come to the session, we won't be able to virtually compare the straight through vs inserting a A/D/A loop, at least you'll have snippets of the music to enjoy. The main purpose of posting these clips here is for members to re-visit what they heard in view of the results and comments made. I'm posting the original 24bit/96kHz recording made, as well as a 16bit/44.1kHz cut re-sampled with SoX, dither on, linear phase (all default settings used).

Here goes!

In total, 11 cuts were played, 18 listeners responded by returning a questionnaire but since not all answered all questions, we received 171 responses, or an average of 15.5 responses per cut.

Cut 1: Casino Royale - Side 1 Track 2: The Look of Love (Dusty Springfield)

http://www.genesisloudspeakers.com/downloads/Cut1_44.wav
http://www.genesisloudspeakers.com/downloads/Cut1_96.wav

The first track picked is an old “audiophile chestnut” and has been recommended by HP as a reference recording. The 45rpm single-sided Clarity vinyl pressing was used.

Did I hear a difference? 10 responded with a Yes / 0 responded with a No

Actual: A was straight-thru, B was A/D/A Loop – Results: 6 correct/8 wrong

What difference did I hear?
A: wider sound stage, smoother, more crisp, clearer words, more defined, more presence.
B: soundstage shrank, sounds muted, smoother transition and edge of notes, preferred (3x).

The surprise result here was that 3 listeners mentioned in the comments that they preferred the A/D/A loop. No mentions of any one preferring the straight-thru.

Cut 2: Hans Theessink: Call Me – Side 1 Track 1: Late Last Night

http://www.genesisloudspeakers.com/downloads/Cut2_44.wav
http://www.genesisloudspeakers.com/downloads/Cut2_96.wav

Male vocals, guitar, a track with great bounce and rhythm (This album was selected for me by Jerry Pomeroy - ex-President of PNWAS. The cut was familiar to many members because I have played the digital version before from the Burmester Demo CD)

Did I hear a difference? 7 Yes / 4 No

Actual: A was A/D/A Loop, B was straight-thru – Result: 9 correct/ 2 wrong

What difference did I hear?
A: wider soundstage, preferred (2x)
B: fuller, more balanced, more harsh.

Again, two listeners preferred the A/D/A loop.

Cut 3: Joe Cocker: Sheffield Steel – Side 2 Track 1: Ruby Lee

http://www.genesisloudspeakers.com/downloads/Cut3_44.wav
http://www.genesisloudspeakers.com/downloads/Cut3_96.wav

One of my favourite albums, a very dynamic blues/rock numbered radio station promo first pressing

Did I hear a difference? 7 Yes / 1 No

Actual: A was A/D/A Loop, B was straight-thru – Result: 10 correct/ 7 wrong

What difference did I hear?
A: more defined, far more brittle and bright, voice too sharp
B: slightly wider soundstage, more engaging, fuller, better bass, voice more pleasant, less harsh, preferred (2x)

Two listeners mentioned preferring the straight-thru.

The Digital/Analog Dialectic

New Member

VIP/Donor

New Member

VIP/Donor

WBF Founding Member, Pro Audio Production Member

Member Sponsor & WBF Founding Member

Member Sponsor & WBF Founding Member

New Member

New Member

Member Sponsor & WBF Founding Member

VIP/Donor

Banned

New Member

New Member

Addicted To Best

New Member

Banned

WBF Technical Expert (Speakers & Audio Equipment)

New Member

WBF Technical Expert (Speakers & Audio Equipment)

Similar threads