I hope you don't mind me reopening this thread but I just found it. I am the author of the RMAF talk and I am humbled by the amount of discussion this has generated. There are some very valid comments here. I apologize that my talk didn't offer any definitive conclusions. My goal for this talk is to get people subjectivists and objectivists to hold respectful dialog. There is only so much that I can accomplish in one hour and keep the topic somewhat engaging at the same time.
I am a governor of the AES and I hear a lot of mockery of the audiophile community among the scientific community. I think this is because many hi-fi vendors have hypothesized why their gear sounds better and have vociferously claimed these hypothesis to be fact without establishing any scientific proof. There is a big difference between hypothesis and fact. However, it is equally wrong of the scientific community to outright dismiss one's observations without providing equal scientific proof that those differences don't exist. Both sides need to work together and admit that there are still things we don't yet understand about the recording and playback of audio sources. Having given versions of this paper to many AES members, I know I have raised considerable awareness to the deficiencies in commonly reported specs.
IMHO, one of the biggest problems we have is differentiating between correlation and causality. I can say that water causes cancer. All persons that die of cancer have ingested water. And, if we deny water, those patients won't die of cancer. They'll die of dehydration. But look how well cancer deaths correlates to water ingestion. This extreme example shows that we can have a high degree of correlation that has no relevance on causality. The water didn't cause the cancer. History is filled with examples of correlation being confused with causality.
To me, it is perfectly acceptable to say that something works but the reason why remains a mystery. It is also perfectly acceptable to propose a hypothesis on why something works. In fact, I believe when something is stated as merely a hypothesis, it invites further research. We can certainly benefit from more research. Once research confirms causality, we can then work to quantify differences between products.
Please feel free to ask questions. I will try to check back and answer when I can.