Objectivist or Subjectivist? Give Me a Break

[microstrip]

That's funny, because subjects themselves requested higher than standard playback levels, so they could "listen for more details in softer passages". And M&M gave it to them.


This 'gotcha'-type harping on positive controls -- which, properly, would be inclusion of a difference pre-tested to be likely to be audible to most of the populace, or to a trained subset -- as a supposedly irredeemable flaw in M&M, is really quite disingenuous in the face of routine audiophile claims made for tbe obviousness of hi rez/DSD vs Redbook difference. One would think that if the benefits of hi rez /DSD were as stark as claimed, under the many conditions claimed, positive and negative controls would be almost redundant.

But it's hardly the first time I've seen audiophiles demand the highest scientific rigor...when it suits them.

There is an optimal level for each recording and system - pushing over this level will obscure differences in great systems. At some time you loose the best of the recording and the system and everything sounds similar. All IMHO.

I love when the "audiophile opponent" needs to use the "... when it suits them argument." For me it means they have little else to say.
I know very well the limitations of sighted listening, and I weight opinions emitted in these conditions accordingly. But I also know the requirements and limitations of the other side.

 

[rbbert]

It really matters what 'things' are being referenced here.

??

 

[sasully]

Tim, as you already said - you know what controls are for. So if the controls aren't sufficient to ensure that the test is sensitive enough you end up with an invalid test. It really is as simple as that

What control lacking in M&M's test would have indicated it was 'sensitive enough' to detect a difference between DSD and DSD converted to Redbook? Doens;t this require a expectation of how audible that *should* be? Audiophiles would have us believe that the difference is quite audible indeed...to audiophiles. And M&M did include audiophiles in the subject set.

MUSHRA to evaluate the subjective audio quality of codecs, is also a recommendation from ITU-R & widely used, AFAIK.

There's more than one kind of audio DBT, for more than one purpose. Evaluating subjective quality -- preference, if you will -- requires a different toolset than testing for difference...which is what M&M did.

And MUSHRA ain't even the only one for preference (it has its critics, also published in JAES IIRC).

 

[sasully]

??

You wrote 'some things we 'know' , are wrong'? Well, yes, that's likely to be true. But some others 'things' in biology have held up pretty well so far and have a lot of experimental and empirical support and consilience with other data.

So it *really matters* what 'things' you are referring to in a given instance.

 

[sasully]

There is an optimal level for each recording and system - pushing over this level will obscure differences in great systems. At some time you loose the best of the recording and the system and everything sounds similar. All IMHO.

Well, again, M&M's subjects were even allowed to set a playback level *they* thought was most revealing. It sounds to me like M&M just can't win!

I love when the "audiophile opponent" needs to use the "... when it suits them argument." For me it means they have little else to say.
I know very well the limitations of sighted listening, and I weight opinions emitted in these conditions accordingly. But I also know the requirements and limitations of the other side.

What weight do you give to opinions emitted under sighted conditions?

 

[jkeny]

Hardly. I don't reject them, I recommend them, along with listener training. But lack of them, in this case, did not render the results nearly as useless as audiophiles desperately wish them to be. For that to be true you have to posit one or more of these to be true too:


- that DSD vs REdbook is a rather subtle difference that M&M's subjects lacked the discriminative skills to detect (except that they could, if the playback was boosted)
- that M&M's playback chain was flawed in a way that 'masked' real differences (though not enough to mask the differences that in fact *were* detected)
- that M&M's choices of music did not adequately reflect the 'real' audible differences between DSD and REdbook, e.g., the SACDs were sourced from PCM or analog masters (this despite allowing at least some users to 'roll their own' in terms of what discs to listen to, and despite common and widespread audiophile report of the superiority of SACDs to Redbook versions, regardless of provenance of the recording)
- that M&M's statistical analysis was flawed, and likely detection of real differences *in their own dataset* was missed.

AH, Ok so you admit it was a flawed test but your are debating how badly flawed it was?

all flaws are not created equal

Correct & it is the same point as you are making above?
BTW, nice article from Asimov on "The Relativity of Wrong" here http://chem.tufts.edu/answersinscience/relativityofwrong.htm

 

[sasully]

AH, Ok so you admit it was a flawed test but your are debating how badly flawed it was?

I don't think the lack of formal positive/negative controls warrant completely disregarding the results, given the rest of the protocol. Unlike, say, lack of 'sighted' bias controls would have.

For some time now it hasn't been that hard to set up a personal ABX to test hi rez vs Redbook very much as M&M did (downconverting hirez to Redbook). One is free to include positive and negative controls, and to train oneself beforehand to hear small differences of various types, if one wants to. Personally I was interested to simply test myself at 'status quo' -- no extra training, using recordings I like and a resampler I trust -- and I didn't do better than chance (at p=0.05). I've done that for mp3 vs Redbook too. M&M's results surprise me neither from a purely 'objective' basis (what we 'know' about how digital audio 'works and about human hearing, all points to Redbook and DSD being indistinguishable under typical home audio listening conditions) and from my own 'experimental' work.

 

[Groucho]

I know very well the limitations of sighted listening, and I weight opinions emitted in these conditions accordingly. But I also know the requirements and limitations of the other side.

Do you have a figure for that weighting? I'd add a smiley, but...

 

[mep]

How many people here have ever participated in a double-blind test that would pass scientific muster? How many people have ever participated in double-blind tests on a regular basis when they purchase stereo gear? My guess is not many and so I wonder why objectivists love to run their jaws through their keyboards and hang double-blind tests over everyone's heads like they are involved with them all the time and those tests they aren't really involved in help influence their purchasing decisions.

 

[rbbert]

You wrote 'some things we 'know' , are wrong'? Well, yes, that's likely to be true. But some others 'things' in biology have held up pretty well so far and have a lot of experimental and empirical support and consilience with other data.

So it *really matters* what 'things' you are referring to in a given instance.

Well, no it doesn't. The point is that a any given moment in time we don't know what we don't know, and we don't know what things we think we know are wrong. Those axioms are promulgated by people smarter, more knowledgable and more experienced than I.

 

[treitz3]

How many people here have ever participated in a double-blind test that would pass scientific muster? How many people have ever participated in double-blind tests on a regular basis when they purchase stereo gear? My guess is not many and so I wonder why objectivists love to run their jaws through their keyboards and hang double-blind tests over everyone's heads like they are involved with them all the time and those tests they aren't really involved in help influence their purchasing decisions.

Hello mep. I personally just trust my ears. They do not lie. In front of many folks observing or in solitude. To others......do you trust your own ears or does it have to be scientifically proven that something you are interested in may or may not be better or more toward your own preference?

Tom

 

[mep]

Hello mep. I personally just trust my ears. They do not lie. In front of many folks observing or in solitude. To others......do you trust your own ears or does it have to be scientifically proven that something you are interested in may or may not be better or more toward your own preference?

Tom

Tom-the bottom line is that there is no "Objectivist Audio" magazine that writes reviews based on double-blind tests. People have to make decisions based on what their ears tell them unless they make decisions based on what other people's ears tell them.

 

[Groucho]

Tom-the bottom line is that there is no "Objectivist Audio" magazine that writes reviews based on double-blind tests. People have to make decisions based on what their ears tell them unless they make decisions based on what other people's ears tell them.

Not necessarily so. They can make decisions based on what they like about the design or, er... the measurements that the objectivist magazine makes. I can be persuaded by the design alone, I think. I'm prepared to believe that all modern DACs sound the same, and then it comes down to what I like about the connectivity, user interface (if it's got one), software (if it uses it) etc.

 

[Groucho]

To others......do you trust your own ears or does it have to be scientifically proven that something you are interested in may or may not be better or more toward your own preference?

I do not trust my own ears. I know that I can set myself up to not like something. I know that if I approach my system with the attitude of listening to it, rather than just putting some music on, I can get into a spin where I don't know if it's any good any more. The next day, with a different attitude and it's giving me goose bumps.

However, at all times I think I'm a pretty good judge of a flat frequency response. I'm pretty sensitive to distortion on a sine wave, and perhaps overly sensitive to dynamic compression. I can definitely hear scratches, rumble and hiss from vinyl.

 

[mep]

I do not trust my own ears. I know that I can set myself up to not like something. I know that if I approach my system with the attitude of listening to it, rather than just putting some music on, I can get into a spin where I don't know if it's any good any more. The next day, with a different attitude and it's giving me goose bumps.

However, at all times I think I'm a pretty good judge of a flat frequency response. I'm pretty sensitive to distortion on a sine wave, and perhaps overly sensitive to dynamic compression. I can definitely hear scratches, rumble and hiss from vinyl.

Really? Wow! I guess you really do have good ears! Seriously, most really good LPs have minimal noise to contend with when you are listening to the music. If the music was sourced from tape, it doesn't matter what format you hear it played back on with regards to tape hiss. Tape hiss is present to some degree and the degree depends on the the type of tape that the music was recorded on. What I find remarkable is how unobtrusive that tape hiss is on the majority of recordings. If every LP you hear has lots of rumble, scratches, and hiss, either you have poor quality LPs and/or a very poor quality turntable/arm/cartridge and phono preamp to listen to.

 

[jkeny]

I do not trust my own ears. ........ I can definitely hear scratches, rumble and hiss from vinyl.

Are you sure you can hear these?

 

[Phelonious Ponk]

Tim, as you already said - you know what controls are for. So if the controls aren't sufficient to ensure that the test is sensitive enough you end up with an invalid test. It really is as simple as that


Well, if you looked at the heading of the paper or into the link I provided you would see that it is a recommendation from the ITU Radiocommunication Sector (ITU-R) which is one of the three sectors (divisions or units) of the International Telecommunication Union (ITU) and is responsible for radio communication.

MUSHRA to evaluate the subjective audio quality of codecs, is also a recommendation from ITU-R & widely used, AFAIK.

I'm sure there are refinements & other methodologies but all will have internal controls embedded in them as per these methodologies to ensure their robustness, so I don't see what your objections are to this.

It's really pretty simple, John. If your control falls into the range of 20 - 20k (and if it is a control for audibility, recommended for evaluating the quality of codecs, it must), it's not in the same range as the part of the signal that differentiates SACD from CD. The ability to hear a .5 db noise at 11k says absolutely nothing about the ability to hear a .5 db noise at 22k. You're comparing apples and oranges, and declaring that every study ever run that didn't compare apples to oranges is invalid. I'm not surprised this control is recommended for codecs. It is irrelevant to this study, and its absence doesn't invalidate it, it just means the people who designed the study were smart enough not to test the audibility of a differentiator above 20k with a noise below 20k. They did, however, take many other steps to ensure that the systems were transparent enough to hear very small differences, that the participants were healthy enough and experienced enough to hear those differences, and to make sure that those participants had a very good opportunity to hear what was there. You're ignoring all of that and declaring the study utterly invalid because Meyer and Moran did not include an irrelevant control designed for a completely different kind of test.

That's what my objections are.

Tim

 

[j_j]

How many people here have ever participated in a double-blind test that would pass scientific muster? How many people have ever participated in double-blind tests on a regular basis when they purchase stereo gear? My guess is not many and so I wonder why objectivists love to run their jaws through their keyboards and hang double-blind tests over everyone's heads like they are involved with them all the time and those tests they aren't really involved in help influence their purchasing decisions.

First, I have participated in rather too many well-run DBT's.

Your second question is out of context. Use what you want when you buy something. That's not science.

As to your dishonest straw man, based on an appeal to emotion, an appeal to ignorance (disguised), appeal to facts not in evidence, and then another straw man or three, coupled with an extraction from contect and a confutation of scientific concerns with purchasing concerns (the last of which some so-called objectivists are also guilty of, but not all, and by implying universality, you widen your attack as a further emotional appeal that is both abusive and malicious), well, I think that's just a start on the fallacious nature of your insults.

DBT's are for science.

It's that simple.

When you claim an effect actually exists, you are making a testable claim, and that puts your claim in the realm of science.

If you say "I like this", that's all there is to it. You like it. Fine. Preference is inviolate.

 

[j_j]

Seriously, most really good LPs have minimal noise to contend with when you are listening to the music.

That does not mean that the noise is inaudible. The rest of your barely-veiled attack on the previous author fails on your exaggeration of their statement, and the false position you built from your false exaggeration.

Wow, we're deep in the dishonest rhetoric today, aren't we?

 

[Groucho]

Really? Wow! I guess you really do have good ears! Seriously, most really good LPs have minimal noise to contend with when you are listening to the music. If the music was sourced from tape, it doesn't matter what format you hear it played back on with regards to tape hiss. Tape hiss is present to some degree and the degree depends on the the type of tape that the music was recorded on. What I find remarkable is how unobtrusive that tape hiss is on the majority of recordings. If every LP you hear has lots of rumble, scratches, and hiss, either you have poor quality LPs and/or a very poor quality turntable/arm/cartridge and phono preamp to listen to.

Are you sure you can hear these?

It's one of those things that is going to vary with the music you listen to, and how you listen to it. I listen to my system loud - and I listen to some classical that has a wide dynamic range. There are moments of quiet where background noise shows up clearly. It isn't rocket science: SNR of CD ~96dB, SNR of vinyl ~60dB on a good day. But of course the clicks peak much higher than that. It could be that the vinyl mastering is compressed to compensate so the noise isn't so audible, but as I say, I hate dynamic range compression - when a system 'pulls its punches'.

Consider this track:
http://open.spotify.com/track/6kLM7LB6B2EGJZ8jiIJRzk

There are moments of quiet when the background noise of vinyl would be quite obtrusive. Different styles of music would not suffer so badly.