Changing variables in a review

[Cableman]

Honestly I don't understand why @RoyGregory 's comment is creating this much controversy. I can't speak to what went on at @marty 's but IME introducing a new component requiring a change in setup isn't unusual; either because of interaction or just highlighting what is off with the setup. In that exchange @RoyGregory had a very different experience from @marty and offered a plausible reason, what's the issue?

Who wrote it in stone that you can change only one variable at a time? If something doesn't initially work out doesn't mean that it's bad why not try a few things and see before deciding? Even if speakers are already setup well very small changes can make huge differences.

david

Who wrote it in stone???? Every decent reviewer who ever lived.
My God the extremes some on here go to to defend the indefensible is quite frankly baffling.

If you change more than one parameter your review is obsolete. It has zero cred, it’s totally utterly verkakte - and I have to question why anyone with any knowledge of hifi evaluation would even begin to question such an obvious requirement as …ONLY CHANGE ONE COMPONENT AT A TIME .. so in the case of an amplifier evaluation don’t move that bloomin speaker one inch!!!

Any who are offended by this are free to ignore. Any who continue to abuse the evaluation process in such a manner are condemned by their lemming - like anti audiophile stance.

 

[marty]

If we were to hold to the "one variable at a time" philosophy then the new speakers should be put in EXACTLY the same place as the pair that just left.

Totally disagree. Why would it? The differences between 2 speakers could be very different as you said yourself beautifully:
"'The woofer size, port (front, back or sealed), and even the location of the woofer dictate how that speaker is going to interact with the room."
Even more than that, the radiation pattern of the drivers (which can be dependent on crossover slope)., polar response, cabinet resonance frequencies and several other things may all contribute to the fact that speaker A has a different optimum location in the room than speaker B. Case in point. I was told by Wilson that the Alexx V will almost certainly not have the same location as the Alexandrias that preceded them. And indeed that was the case. (The Alexx V's differed by an inch or two vs the Alexandria).

The point is, once any speaker is dialed-in, I find it difficult to believe that the use of different amplifiers is going to matter that much, or at all, when it comes to room placement. However, the point is also moot. The end user is the only one who has to be convinced. Feel free to do whatever you wish to optimize any speaker to your room with any ancillary gear.

 

[Cableman]

Do you think reviewers should have five or six different power cords on hand for when they review different amplifiers so as to determine which has the best “synergy”?

Are you suggesting that Marty made the determination on his CH M 10 amplifiers prematurely because he did not do exhaustive speaker repositioning before he decided he did not like his amplifiers?

Why stop there. Let’s make sure they change ever cartridge every tone arm… and cable. Every phonostage and or equaliser.
Amine. We gotta do our very best for CH and their 200K amps coz they’ve got this week biggest advertising budget noch

 

[Cableman]

Totally disagree. Why would it? The differences between 2 speakers could be very different as you said yourself beautifully:
"'The woofer size, port (front, back or sealed), and even the location of the woofer dictate how that speaker is going to interact with the room."
Even more than that, the radiation pattern of the drivers (which can be dependent on crossover slope)., polar response, cabinet resonance frequencies and several other things may all contribute to the fact that speaker A has a different optimum location in the room than speaker B. Case in point. I was told by Wilson that the Alexx V will almost certainly not have the same location as the Alexandrias that preceded them. And indeed that was the case. (The Alexx V's differed by an inch or two vs the Alexandria).

The point is, once any speaker is dialed-in, I find it difficult to believe that the use of different amplifiers is going to matter that much, or at all when it comes to room placement. However, the point is moot. The end user is the only one who has to be convinced. Feel free to do whatever you wish to optimize any speaker to your room with any ancillary gear.

Marty. You talk too much sense…but me saying that will surely get you cancelled in here

 

[marty]

it’s totally utterly verkakte -

I've been on the earth for quite a while and although I've used that word plenty in dialog, I honestly don't recall ever seeing it in print!! Love it!!
That said, I obviously agree with your perspective but again, it really doesn't matter. As far as I am concerned, it is really is not worth further discussion. People will do what they do to get their rig to sound as good as possible. For them. No harm, no foul. As Lenny Bruce once said, "if it cooks for you...solid!"

 

[Cableman]

I've been on the earth for quite a while and although I've used that word plenty in dialog, I honestly don't recall ever seeing it in print!! Love it!!
That said, I obviously agree with your perspective but again, it really doesn't matter. As far as I am concerned, it is really is not worth further discussion. People will do what they do to get their rig to sound as good as possible. For them. No harm, no foul. As Lenny Bruce once said, "if it cooks for you...solid!"

Ah but this is about a reviewer rewriting the ‘rules’ of component evaluation was it not. In which case IMHO it’s a huuuugggeee issue and needs seriously called out.

 

[microstrip]

Do you think reviewers should have five or six different power cords on hand for when they review different amplifiers so as to determine which has the best “synergy”?

Surely. I will give you an example - I use Transparent Audio power cables with great success on the VTL Siegried II. They make the Lamm ML3 sound poor, I had to switch power cables to compare them in a fair way.

 

[Cableman]

Surely. I will give you an example - I use Transparent Audio power cables with great success on the VTL Siegried II. They make the Lamm ML3 sound poor, I had to switch power cables to compare them in a fair way.

Nope. I’m not gonna go there. No siree

 

[ddk]

Who wrote it in stone???? Every decent reviewer who ever lived.
My God the extremes some on here go to to defend the indefensible is quite frankly baffling.

If you change more than one parameter your review is obsolete. It has zero cred, it’s totally utterly verkakte - and I have to question why anyone with any knowledge of hifi evaluation would even begin to question such an obvious requirement as …ONLY CHANGE ONE COMPONENT AT A TIME .. so in the case of an amplifier evaluation don’t move that bloomin speaker one inch!!!

Any who are offended by this are free to ignore. Any who continue to abuse the evaluation process in such a manner are condemned by their lemming - like anti audiophile stance.

I'm trying to decide it if this is Cabledude humor or sincere?

david

 

[PeterA]

Surely. I will give you an example - I use Transparent Audio power cables with great success on the VTL Siegried II. They make the Lamm ML3 sound poor, I had to switch power cables to compare them in a fair way.

Why not simply evaluate the amplifiers with the stock power cords that they came with, at least for a review that involves a direct comparison. If you then find that some other power cord or footer or whatever improve the sound of your system, have at it.

 

[tima]

Why not simply evaluate the amplifiers with the stock power cords that they came with, at least for a review that involves a direct comparison.

If you do not use the manufacturer's power cord you are not reviewing his product.

 

[Lee]

Why not simply evaluate the amplifiers with the stock power cords that they came with, at least for a review that involves a direct comparison. If you then find that some other power cord or footer or whatever improve the sound of your system, have at it.

If a reviewer's system has high quality power cords, won't those enable an even better evaluation of what is possibly with the review product?

Is it likely these days that any customer will use a stock cord?

At some point or price level, a quality AC cable is needed for real world usage.

 

[tima]

tube rolling and power cord or interconnect changing, which we as end users can duplicate, are relevant things we want to know. and maybe footers and such too other than stock.

Maybe in a sidebar. But a review should be about a specific product as it comes out of its box.

 

[tima]

If a reviewer's system has high quality power cords, won't those enable an even better evaluation of what is possibly with the review product?

No, imo. It's not a question of 'an even better evaluation'. If you change a product from stock you are, imo, no longer reviewing what the consumer can purchase. And it is unfair to the manufacturer. There very few manufacturers who say their product is better if you modify it with something they do not offer. Manufacturers expect a review of their product, not their product with someone else's add-on.

Is it likely these days that any customer will use a stock cord?

At some point or price level, a quality AC cable is needed for real world usage.

Perhaps. But that falls outside of a review. As I said to Mike, put it in a sidebar, but not in a formal review.

 

[BlueFox]

If you do not use the manufacturer's power cord you are not reviewing his product.

I have mixed feelings on this. However, if the manufacturer includes a throw away power cord then the review should at least mention that. Maybe give a before/after review if the power cord is changed.

 

[Lee]

No, imo. It's not a question of 'an even better evaluation'. If you change a product from stock you are, imo, no longer reviewing what the consumer can purchase. And it is unfair to the manufacturer. There very few manufacturers who say their product is better if you modify it with something they do not offer. Manufacturers expect a review of their product, not their product with someone else's add-on.



Perhaps. But that falls outside of a review. As I said to Mike, put it in a sidebar, but not in a formal review.

I think we have an honest disagreement on this one Tim. I just feel, especially on better gear, the consumer is going to want to use a better power cord. That seems to me to be the best simulation of real world conditions. Likely use in the field if you will.

 

[andromedaaudio]

What boggles me however is why people value a review so much .
Do they need the holy blessing of a " Guru"?
Id love to hear george carlins take on this , lol.

Who gives a flying f**k what a reviewer likes or doesnt like .

Iike:

here i am basically stuck with a crap sounding system because my reviewer told me so.
It only influenced me the first 5 years or so in this hobby .
Then i learned and moved on

 

[tima]

I think we have an honest disagreement on this one Tim. I just feel, especially on better gear, the consumer is going to want to use a better power cord. That seems to me to be the best simulation of real world conditions. Likely use in the field if you will.

Sure - no problemo. And I do understand your interest. It's just that I"ve seen too many reviews where the reviewer explores power cords with the product under review and all of those reviews ended up a confused mess. Part of that is a result of poor review organization, but it turns into a review of too many products.

For example: If you are reviewing an amp many readers like to hear some comparison of that amp with another amp. Often that other amp is currently the one in the reviewer's system. Throw in an effort to optimize both amps under review by using a different power cord. Too many combinations. What if the amp under review sounds worse with the power cord used on the reviewer's amp. Keep it simple - stock cords all around.

Typically you try for a certain number of words - not too many, not too few. Time spent describing power cords is words taken away from the review subject.

What you want are audio forums! That's a great place to learn of real world experience. What power cords work well with my XYZ amplifier? You see a lot of that on the forums.

edit: grammar

 

[Gregm]

For example: If you are reviewing an amp many readers like to hear some comparison of that amp with another amp. Often that other amp is currently is in the reviewer's system. Throw in an effort to optimize both amps under review by using a different power cord. Too many combinations.

Agreed, even if the SQ *could* be ultimately (and later) be improved by using that aftermarket power cord... it is the comparison that is of use, not the possible combos, and messing with maultiple variables at the same time obvously messes up any effort of evaluation vs standard reference as mentioned above.

However, I feel there is a fundamental difference between changing say, power cords on the DUT and changing speaker positioning to suit the amplifier beind auditioned; in theory, there is one position where a specific pair of speakers performs its best.
If we change that position, then are we are not compromising the sonic result in an effort to make the amplifier under test (to use that example) sound less compromised?

 

[tima]

I would like to pose this question. How does someone review speakers? The new pair of speakers is always setup in a slightly different location to the original pair. If we were to hold to the "one variable at a time" philosophy then the new speakers should be put in EXACTLY the same place as the pair that just left. We all know this doesn't work and fully accept that the reviewer had to reposition the new speakers to get good sound in his/her room. Yet, when Roy mentioned that he repositions the speakers when he gets a new amp to achieve best sound everyone (well not quite everyone) freaked out.

I don't think there is a one size fits all protocol and the approach taken will depend on the product under review. And while I believe in keeping variables to a minimum, as @ddk suggests, this isn't a controlled scientific experiment, it's an audio review.

Consider amps and speakers - they have a co-dependent relationship, with some pairings working really well together and others not so well and some combinations in-between. For each pairing there is likely an optimal setup within a given room. That setup may have a relatively broad or narrow range of position options.

What do you do when you bring an amp and pair of speakers into a room? You hopefully work out their best setup. Let's say you have a set of speakers to review or an amplifier to review. I would bring the review product into my audio room and proceed to setup the pair for the best positioning I could find. That's what an end-user would normally do. Do the same for the reference where one product stays the same (amp or speakers.) I don't think that is confusing or somehow deceptive to the reader if each pair has its own position. Just briefly explain what you've done and describe what you hear.

If you are reviewing turntables then, imo, there is a co-dependency between arm and cartridge, but not between cartridge and turntable. As long as an arm fits correctly on a table (P2S distance, etc.) and all its adjustments are allowed then there is less co-dependency between arm and table than between arm and cartridge. Maybe some odd resonance issue might come into play with some arm and table combo - but that can be described in the review if it happens. Otherwise, there should be only one variable and that is the turntable itself. If you're reviewing a turntable that comes with an arm, ask the manufacturer to borrow an extra arm to mount on your reference.

Phono stages? Pick one cartridge that is simpatico with the review product and the reference. Pre-amps? Each pre-amp is the sole variable. Don't mess with trying different cables. Only if one is XLR only and the other is RCA only might you need different cables. I keep an instance of each with otherwise the same cable just for that situation.

Reviews are about one person's experience with the review product. Ideally they include a comparison to a reference product. Ideally they keep variables to a minimum. There is no universal protocol. The reviewer just needs to describe well and succinctly what he did. If the reviewer is reasonable with the process and the 'mechanical' portion of his description, then the most important part (imo) is his sonic description. A comparison is important but secondary.