What I’d really like to see-comparative hi-end speaker shootouts

marty

Well-Known Member
Apr 20, 2010
3,039
4,208
2,520
United States
I recognize that many high-end audio dealers can only display one of the top speaker brands available. This may be due to economic limitations (inventory is expensive) or often, it is because the manufacturer may sanction dealers and pull their product line if they feel their product is not in the most vaunted position inside their retail store. (Case in point- how many dealers carry both Wilson and Magico?) Yet the consumer is often at a disadvantage because he cannot hear two or more fine brands he might be considering for purchase in the same store under similar conditions (even if they are not ideally set up under identical conditions). I recall that a little over a year ago, Steve and I journeyed to a dealer in Arizona because we thought we’d be able to demo the Wilson XLF and the Sonus Faber Aida at the same time. Unfortunately the speakers were in different rooms with different electronics so conditions weren't ideal, but still, we found the experience beneficial.

It seems unbelievable that the consumer typically has so little opportunity to hear competitive speaker products simultaneously. Let’s consider the following high end speakers: Wilson Sasha; Rockport Avior; Vivid’s Giya 3, Magico S5 and Sony SSAR-1. (Add your favorite 30K speaker to this list if you wish), These are all in a similar price range but how many dealers will display more than one of these for a potential buyer to hear? Wouldn’t you want to be able to hear them all under more or less identical conditions before making a decision?

Let’s step it up a notch. What if you wanted to hear the Vivid G1, the Wilson Alexia and Rockport’s yet to be released Cygnus or Magico’s yet to be released M concept speaker in a head to head environment? Probably just not going to happen anytime soon for the very reasons cited above.

I realize these shootouts have their limitations. But one can’t deny that there are also terrific advantages in being able to compare and contrast any number of formidable loudspeakers at the same time, if only to get a sense of the strengths or liabilities of the speakers in either relative or absolute terms. Of course, one can rationalize all the reasons for not doing a comparative speaker shoot-out. In the end however, you just have to ask yourself, would you rather have the opportunity to hear or participate in one of these, or not? I think we can all guess the popular answer the that one. Yet under the current rules of brick and mortar audio salons, such comparisons will almost never happen. I guess there is no reason why individuals such as distributors can't do this, but let's get real, I don't think Philip O'Hanlon wants a scourge of audiophiles swarming in his home for such purposes. If he did, hell, just give me the word and I'll begin the pizza and or the scotch, or whatever it takes to make that happen!

Here is where I think our specialty publications can be a huge asset to prospective buyers. I’d love it if some of the big boys- you know who they are- TAS, Stereophile, etc., would take it upon themselves to undertake some of these comparison shootouts and report the results. (They will often do this for inexpensive gear, but rarely will they do this in a rigorous and meaningful way for SOA gear; particularly loudspeakers). They can be done with one listener or with a listening panel. The purpose is not necessarily to crown a “best” but, rather, to provide comparative information that might truly help a prospective buyer. By way of analogy, this sort of comparative evaluation has been done successfully for years in the car industry. Motor Trend and Road and Track routinely compare similar types of cars (sport cars, sport sedans, SUVS) in all price ranges, with a limited panel of drivers and then report individual preferences as well as overall performance and value recommendations. I assure you that nobody is coming down hard on the motoring magazines for doing this, or threatening to withhold new models for evaluation if they say less than wonderful things about the current model. The value to consumers here is immeasurable. I formally call upon audio publishing leaders such Rob Harley and John Atkinson to institute this feature in future issues of their magazines. I’d be shocked it weren’t a huge hit with readers.

Let’s take it one step further. I would love for somebody representing audio enthusiasts such as the WBF or other audiophile list servers to try and arrange a test case shootout at RMAF or a high-end trade show next year. Perhaps space can be obtained either at the meeting forum, or at a dealer off site, whereby identical listening sessions can be arranged for the public, or a listening panel invited by the sponsor, whereby the voting can be recorded and reported. My prediction would be that this sort of event could grow into one of the highlights of each show. Let the manufacturers and sponsors do their single product demos at shows as they usually do. But a 2 or 3 speaker shootout of some popular speaker brands with evaluation by the public or invited listening panels would be a feature that would surely gain popularity very fast. And by the way, WBF, my bet is that you will be able to raise enough support to fund such a project very easily among manufacturers, or attendees. If I were at a show and was asked to pay $5 or $10 to participate in a thoughtful well-designed evaluation of 2 or 3 top speakers under very good conditions, I’d do it in a heartbeat.

It certainly beats the crap out of paying for a one day round trip airline ticket from DFW to Arizona to hear something similar under less than ideal circumstances.
Marty
 

Peter Breuninger

[Industry Expert] Member Sponsor
Jul 20, 2010
1,231
4
0
Simply impossible to do. As a reviewer, I am well versed in speaker set up and system matching. If I had a pair of S3s, Carmel 2s, and Wilson Sashas and the same exact amp/source set up in one room, and then did a listening test and quantified the results, it would very different from the same test executed but with different electronics in the same room. Add the fact that you have to move the speakers for each test and then reset them up and hope you hit the same tape marks on the carpet, again almost impossible. A 2mm difference will alter the results.

Marty, it sounds doable from a theory perspective but in reality it would prove nothing except that you like one speaker with a certain amp over another. The car industry tests cars that are complete entities. With us... speakers match with amps to form a unit, amps need source material. yada yada.
 

esldude

New Member
I guess the closest to this is the speaker tests Harman runs. They have a mechanism to silently move the speakers in and out of position in 3 seconds. Guess we just need to get them to disclose what the results are. Currently they use it to see if they produce better sounding speakers than competitors at a given price. Don't know the cost to a magazine to have such a facility. Nor is such a test in the interests of most magazine's advertisers.
 

asindc

Well-Known Member
Sep 27, 2012
187
20
923
Add Tidal speakers to the list. A must-audition at that level of the market, IMO.
 

MrAcoustat

New Member
Jun 5, 2012
847
7
0
78
Quebec Canada
Impossible to do, from one reviewer to another the results would be different, whe all have are preferences even reviewers, as for me for the kind of music i listend to i would choose MBL-101s over Wilson Elexandria and save lot's of money BUT i know that for someone else it would be the opposite and that's fine to, that is why there are so many flavors ( models ).:)
 

Johnny Vinyl

Member Sponsor & WBF Founding Member
May 16, 2010
8,570
51
38
Calgary, AB

Speedskater

Well-Known Member
Sep 30, 2010
941
15
368
Cleveland Ohio
Decades ago, Stereophile magazine did some loudspeaker shootouts. The tests were in issues: Vol.14 #7, Vol.15 #5, Vol.15 #1, Vol.16 #9 and Vol.17 #8. In the August 1994 comparison 10 writers and 8 mid-priced loudspeakers were involved. But because of a lack of consensus and repeatability they later abandoned the blind comparison idea. Their writers seem to struggle when the speaker identity is hidden.
[h=2][/h]
 

amirm

Banned
Apr 2, 2010
15,813
38
0
Seattle, WA
Marty you identify a serious problem. Wish we could solve it but the biggest barrier as you somewhat imply is business: I don't think high-end speaker companies would volunteer for such a shoot out. If they did, I would be game.

Comparison could be simplified by having a symmetrical room and testing mono. Each speaker would be placed in the right or left corner. Then we can toggle between them. Immediate difference that can be heard are timbre. And that is super important to determine.
 

LL21

Well-Known Member
Dec 26, 2010
14,430
2,518
1,448
Marty you identify a serious problem. Wish we could solve it but the biggest barrier as you somewhat imply is business: I don't think high-end speaker companies would volunteer for such a shoot out. If they did, I would be game.

Comparison could be simplified by having a symmetrical room and testing mono. Each speaker would be placed in the right or left corner. Then we can toggle between them. Immediate difference that can be heard are timbre. And that is super important to determine.

Didnt they do a shoot out in Europe sponsored by a magazine a few years ago, with each speaker being setup separately but with the same electronics? Alexandrias, TAD Ref Ones, Thiel CS5, SF Strads, and 2 others...cannot remember.
 

MrAcoustat

New Member
Jun 5, 2012
847
7
0
78
Quebec Canada
Didnt they do a shoot out in Europe sponsored by a magazine a few years ago, with each speaker being setup separately but with the same electronics? Alexandrias, TAD Ref Ones, Thiel CS5, SF Strads, and 2 others...cannot remember.

I am not a big fan of Wilsons BUT Thiels in the same bracket as Wilson - TAD - Sonus Faber - what kind of shoot out was that. :)
 

amirm

Banned
Apr 2, 2010
15,813
38
0
Seattle, WA
Didnt they do a shoot out in Europe sponsored by a magazine a few years ago, with each speaker being setup separately but with the same electronics? Alexandrias, TAD Ref Ones, Thiel CS5, SF Strads, and 2 others...cannot remember.
I don't know but it would depend on what they were promised. Some shoot outs have stipulation in private that this and that won't be said. Or this and that won't be done.
 

MrAcoustat

New Member
Jun 5, 2012
847
7
0
78
Quebec Canada
The winner is often the one with the most ads in the magazine.:)
 

Vette

New Member
Feb 2, 2014
18
0
0
The last thing high-end manufactures and reviewers want to see is blind testing. The whole industry would be turned upside down and reviewers would be out of work.
 

Occam

[Industry Expert]
Dec 15, 2010
117
1
0
NYC
The last thing high-end manufactures and reviewers want to see is blind testing. The whole industry would be turned upside down and reviewers would be out of work.

I'd hope that paragraph should be written -
The last thing some high-end manufactures and reviewers want to see is blind testing. The whole industry would be turned upside down and some reviewers would justifiably be out of work.

Some comments on previous posts -

1. At best, only the speakers currently under evaluation should be in the room. At worst, those speakers, in the room, not under current evaluation should have their terminals shorted to minimize sympathetic movement, via back emf.

2. Symmetrically placed speakers in a symmetrical room, listening to mono signals might well give an indication of spectral balance [with shorted terminals on the speaker not being currently evaluated], but will not give any indication of phase behavior which effects soundstaging and imaging...
 

marty

Well-Known Member
Apr 20, 2010
3,039
4,208
2,520
United States
Simply impossible to do. As a reviewer, I am well versed in speaker set up and system matching. If I had a pair of S3s, Carmel 2s, and Wilson Sashas and the same exact amp/source set up in one room, and then did a listening test and quantified the results, it would very different from the same test executed but with different electronics in the same room. Add the fact that you have to move the speakers for each test and then reset them up and hope you hit the same tape marks on the carpet, again almost impossible. A 2mm difference will alter the results.

Marty, it sounds doable from a theory perspective but in reality it would prove nothing except that you like one speaker with a certain amp over another. The car industry tests cars that are complete entities. With us... speakers match with amps to form a unit, amps need source material. yada yada.

Peter,

Thanks for your thoughtful remarks. I agree with much of what you said, but respectfully disagree on some key issues. If I sense your position correctly, you believe that a perfect speaker comparison cannot be performed under the circumstances I described, and that limitation prompts you to suggest such comparisons are therefore not worthwhile to perform. My perspective is that while you are likely correct; that is, perfect comparative conditions will likely not be achieved, I maintain it would still be a very productive listening experiment and there is plenty to learn. Let me provide some examples if I may.

First of all, since room boundary effects will affect frequency response primarily under 400Hz, there is a broad range of frequencies that can be easily evaluated that will provide a good sense of how the speaker performs its primary function, which is the reproduction of frequencies as a function of amplitude. It would be quite easy for example, to hear how each speaker reproduces high frequencies. We all know some speakers are notoriously hot and others are relatively attenuated and of course there is everything in between. Wouldn’t you like to see, for example, how a given speaker sounds in that range in a side by side comparison? I sure would, even though I know that speaker placement, amplifiers, cable, room treatments and the price of tea in China effects what I hear. But let’s face it, you can fry eggs on the tweeters of some speakers while others are stunning. Wouldn’t you want to get a sense of which is which in a side by side?

Let’s look at other properties that can evaluated regardless of perfect set-up conditions. Some designs have very clear floor cancellation (most common at 150-250Hz), while others do not. Again, even under less than perfect conditions, I think this is can be discernably heard between speakers of different design and thus would be something beneficial to learn.

Two of the most telling sounds there are can be human voice and piano. Again, by your take, since things such as placement are critical, such an effort to make a comparison under imperfect circumsntaces is fool hardy. I disagree. We are all experienced listeners and I maintain it is possible to discern whether what you hear is pleasant, unpleasant or in between for these seminal sounds as two different speakers are compared side by side. Is it perfect? Nope. You got me there, but again I ask, wouldn’t you just love to perform that listening experiment rather than not having that capability at all?

Let’s talk about bass. Sure, your argument gets even stronger in this range, but give me simple plucked bass or Fender bass and there is so much to discern between the bass sounds from two different speakers under even less than optimal conditions. Specifically one can learn a tremendous amount about distortion and transient response. (When I played my Fender bass, I just plugged it in to my Fender amp and didn’t give a rat’s ass what it might sound like on a Marshall. The fundamental instrument character is not going to change dramatically by reproduction from any two reasonably good transducers; home or studio.)

In summary, I still think there is plenty that one can hear so as to assess the musical qualities of a good loudspeaker in comparison to another, even if we come to that information under less than perfect circumstances.

Marty
 

Mike Lavigne

Member Sponsor & WBF Founding Member
Apr 25, 2010
12,601
11,693
4,410
my 2 cents is that no high end speaker manufacturer with a currently successful business has any interest in a high end speaker shootout. there is nothing to gain and much to lose. people will still buy what they like; yet the losers will lose. which means brick and mortar dealers are not going to really cooperate. audio shows are the closest you are going to get to that.

so we are left with owners of speakers trying to put something together that is fair and balanced. where the room and electronics somehow allow for an even playing field. I wonder who might be sufficiently motivated to try and put it together.

good luck with that.

every other year our audio club does a speaker building contest. I've been a judge for 10 years or so. it's a big job dealing with all the set-up and variables. and this is with relatively small and light DIY speakers and no one involved has their business or life happiness on the line. somehow pulling off a similar event with large expensive high end speakers is quite a mind boggling idea.

personally a shootout like what you envision is not answering a question I'm asking. maybe it's just that I no longer really wonder about speakers.

and I think that room and set-up and system tweaking are such huge issues that a quick down and dirty set-up and comparison would not uncover the real truth of the top level speakers anyway.

I could see maybe an 'under $10k' or 'under $15k' dynamic speaker comparison where the form factor was similar and one amplifier could be used. but that sort of thing is not going to scratch the itch you are feeling.
 

GaryProtein

VIP/Donor
Jul 25, 2012
2,542
31
385
NY
About ten years ago I was at Goodwins High End in this room:





I was curious to compare the M-L Statement e2 and the Wilson Alexandria with XS subwoofer. The room was moderately large and symmetrically arranged. I brought music I was very familiar with. I listened for almost three hours. One speaker system was on one end of the room and the other system was on the other end. The electronics, Boulder and dCS, was the same for each system. I sat in a chair roughly in the middle of the room. After listening to one system, the speaker cables were connected to the other system and all I had to do was spin my chair around to listen to listen to the other system.

That was the best high end shoot-out I ever attended.

The M-L Statement e2 was memorably spectacular, not so much for the Wilson.
 

thedudeabides

Well-Known Member
Jan 16, 2011
2,181
691
1,200
Alto, NM
I'm going through the same thing but it involves a single box SOTA SACD / CD CDP. To further complicate, I must have an in home audition with full money back guarantee.

Very difficult to make your way through the weeds.
 

esldude

New Member
About ten years ago I was at Goodwins High End in this room:





I was curious to compare the M-L Statement e2 and the Wilson Alexandria with XS subwoofer. The room was moderately large and symmetrically arranged. I brought music I was very familiar with. I listened for almost three hours. One speaker system was on one end of the room and the other system was on the other end. The electronics, Boulder and dCS, was the same for each system. I sat in a chair roughly in the middle of the room. After listening to one system, the speaker cables were connected to the other system and all I had to do was spin my chair around to listen to listen to the other system.

That was the best high end shoot-out I ever attended.

The M-L Statement e2 was memorably spectacular, not so much for the Wilson.

I have seen a couple dealers with the setup like you describe. Large symmetrical room, electronics can be used on both ends. Swap your chair and listen each direction. That is the best way to compare speakers I have had the pleasure of using. Yes you still need the speaker in its best position on each end which might not be identical positions. It sure beat any other way to compare speakers I have used. Differences were much more obvious between speakers that way.

On one occasion the dealer handled Thiel and had Thiel's best on one end and their mid-level offering on the other. Was impressive example of Thiel's good engineering. If you didn't have something with deep bass and extreme loudness these two speakers were impressively similar. You might not like the Thiel, but if you did that demo would make you think he was giving you his all even in his lesser models.
 

LL21

Well-Known Member
Dec 26, 2010
14,430
2,518
1,448

About us

  • What’s Best Forum is THE forum for high end audio, product reviews, advice and sharing experiences on the best of everything else. This is THE place where audiophiles and audio companies discuss vintage, contemporary and new audio products, music servers, music streamers, computer audio, digital-to-analog converters, turntables, phono stages, cartridges, reel-to-reel tape machines, speakers, headphones and tube and solid-state amplification. Founded in 2010 What’s Best Forum invites intelligent and courteous people of all interests and backgrounds to describe and discuss the best of everything. From beginners to life-long hobbyists to industry professionals, we enjoy learning about new things and meeting new people, and participating in spirited debates.

Quick Navigation

User Menu

Steve Williams
Site Founder | Site Owner | Administrator
Ron Resnick
Site Co-Owner | Administrator
Julian (The Fixer)
Website Build | Marketing Managersing