Audio Critique

[Amir]

I would like to share my idea about Right/Trusted Audio Critique in the High End Audio because Most Magazines need a better understanding of Audio Reviewing.

I ask this question because me (when I was young) and my friends have spent many many times/efforts/money according to (and trusting) Magazine (TAS/Stereophile/6moon/...) Reviews and the result was awful.

The question is :

Could we write a useful audio Critique/Review ? if yes How it could be?


my answer is yes but let me describe it in the next post.

 

[stehno]

Can it be done? Sure. It would be similar to evaluating anybody else - only better. Would it be useful? Doubtful. But it depends entirely on one's knowledge and ability to critically think their way through it. As always.

 

[Amir]

for audio reviewing we have two important challenges.

The first problem is :
We can not listen to the sound of an audio component (like speaker or DAC or pre or ...) without connecting it to an audio system, It means when you try to Review a component you are listening to all audio components (room acoustics, speaker position, AC quality, record, speaker, pre, power , dac, ...) and it is not easy to have a right judgment only about that component.

The second problem is :
Audio systems are non-linear systems and when you mix non-linear systems then the result is far more complex than you think.
This is the list:
- room/speaker interaction
- speaker position in room
- power amplifier/speaker matching
- pre/power matching
- dac/pre matching
- transport/dac matching
- Cable Impedance Matching
...

we have many many factors/variables in the chain and the problem is the final sound is very very sensitive to each parameter .
in electronics science we define a sensitivity function , it is the derivative of Y to derivative of X , X is a parameter and Y is the System Function.
For example in audio system X is amplifier/speaker matching or any other parameter and Y is the Sound we hear in the room.
The problem is in audio systems this Sensitivity is so high and you may get poor result even if you buy expensive components.

I will continue in next post ...

 

[Amir]

We should try to have a solution for these two problem.

I think we need properly setup ultra high performance audio system matched to the component we want to review.

1 - properly setup audio system (good room, good ac quality, good speaker position , ...)
2 - ultra high performance audio system
3 - matching of the component to the the system

these three items will ensure you are listening to the maximum performance of that component in your system and also maximum performance of your audio system. This is the only way you can minimize those two problem.

if you read the Peter Article about "comparison by contrast" you will find in a ultra high performance system we hear over 30% (I mean it is huge) contrast between components when we compare them. Actually in low to mid level systems the contrast is less than enough and we need reference system to have high degree of contrast between X vs Y components.

when you want to compare X vs Y then you may need listen them in two different system.
For example reviewing Speaker X vs Speaker Y , Speaker X is perfect match for Amplifier A
Speaker Y is perfect match for Amplifier B

You should not review Speaker X with Amplifier B when they are not match.



Now you are ready for Critical listening .

Let back to line 3 : matching of the component to the system

I think I should explain this item but let me explain in the future.

 

[Amir]

After minimizing those two problems Now Imagine we have the right condition to listen/judge the audio system.

We know Audiophiles or Audio Reviewers have different taste/ideas when they listen to an Audio System and their Judgment is different together. Some audiophiles think every audiophile has personal taste and audio judgment is relative not absolute.
I think this is not true .
We can debate here for more than 1000 posts but finally I think We can have right Judgment and the audio critique is not relative subject.
There are some parameters that may affects on judgments like the music type we use for test but finally the audio critique is not relative.

you can read some books (A. Likhnitsky’s published a book “Sound Quality: New approach to testing of the home audio equipment”. (P&K, 1998)) to convince the audio judgment is not easy and The sound has two effects on your mind, short term and long term effects. I think Here we need to care more about music experience not just checking audiophile words like soundstage and ...

I will tell you about that book.
Actually the sound system is just a tool for listening music.

I am not expert but I would like only read expert judgments about sound quality.

I do not like what I read in most audio magazines.

 

[henrich3]

Assuming that you have a "properly setup ultra high performance audio system" to evaluate a component, the biggest wildcard IMO is the reviewer himself or herself. Expectation bias in sighted reviews can lead to conclusions that are at odds with the conclusions gleaned from blind testing. Does that impressive-looking $1000/foot speaker cable really sound better than that ugly $1/foot generic 10 AWG cable? Does it measure better in ways that are audible? I've long distrusted the sighted listening tests in audiophile magazines because they don't control for expectation bias.

 

[Al M.]

Assuming that you have a "properly setup ultra high performance audio system" to evaluate a component, the biggest wildcard IMO is the reviewer himself or herself. Expectation bias in sighted reviews can lead to conclusions that are at odds with the conclusions gleaned from blind testing. Does that impressive-looking $1000/foot speaker cable really sound better than that ugly $1/foot generic 10 AWG cable? Does it measure better in ways that are audible? I've long distrusted the sighted listening tests in audiophile magazines because they don't control for expectation bias.

While expectation bias in sighted tests is a problem, blind tests can produce their own expectation bias. For example that you don't expect to hear a difference.

There is no free lunch. Everything has drawbacks.

 

[Amir]

Assuming that you have a "properly setup ultra high performance audio system" to evaluate a component, the biggest wildcard IMO is the reviewer himself or herself. Expectation bias in sighted reviews can lead to conclusions that are at odds with the conclusions gleaned from blind testing. Does that impressive-looking $1000/foot speaker cable really sound better than that ugly $1/foot generic 10 AWG cable? Does it measure better in ways that are audible? I've long distrusted the sighted listening tests in audiophile magazines because they don't control for expectation bias.

I have answered to Mr. Majidimehr in his website about your comment:

my ID is Amir Hossein in that website.

Master Thread: Are measurements Everything or Nothing?

Ding! Ding! We have a winner. If he can do this he would be famous beyond measure and rich as can be too. Looks like he has the rich as can be part figured out already… he just skipped the rest :facepalm:

www.audiosciencereview.com

we can have this discussion in another topic and if you post new topic I will answer you there.

this topic is about Subjective Audio Critique not about Objective vs Subjective debate

 

[Gregm]

I think we need properly setup ultra high performance audio system matched to the component we want to review.

1 - properly setup audio system (good room, good ac quality, good speaker position , ...)
2 - ultra high performance audio system
3 - matching of the component to the the system

these three items will ensure you are listening to the maximum performance of that component in your system and also maximum performance of your audio system. This is the only way you can minimize those two problem.

Even so, you probably won't reach the reliable results you are seeking: the system above has its own sound tuning and, if the device under test (DUT) disrupts that tuning, it will inspire negative comment.

Instead, why not focus on what characteristics (if any) the DUT brings into the equation. To do that, you will need a monitor type system with measured flat in-room response. Use DSP if necessary.
The objective is not to gauge if the DUT performs "well" and you like it -- but, simply, to report on what sonic characteristics that device brings to the table.
Just a thought

 

[henrich3]

While expectation bias in sighted tests is a problem, blind tests can produce their own expectation bias. For example that you don't expect to hear a difference.

There is no free lunch. Everything has drawbacks.

While I very much agree with your statement "expectation bias in sighted tests is a problem", I respectfully disagree with your next claim that "blind tests can produce their own expectation bias". Why would test participants who have high expectations that a beautiful high end cable would trounce a cheap generic cable conclude that they hear no difference if they're blinded as to which one they are listening to? Controlling for expectation bias is how we can have confidence that any conclusions drawn from listening tests are accurate and not simply imagined.

 

[henrich3]

we can have this discussion in another topic and if you post new topic I will answer you there.
this topic is about Subjective Audio Critique not about Objective vs Subjective debate

I did not see anywhere in your initial posts that you were only looking for subjectivist solutions to the problems you outlined. I will bow out from making further comments...

 

[hopkins]

I would like to share my idea about Right/Trusted Audio Critique in the High End Audio because Most Magazines need a better understanding of Audio Reviewing.

I ask this question because me (when I was young) and my friends have spent many many times/efforts/money according to (and trusting) Magazine (TAS/Stereophile/6moon/...) Reviews and the result was awful.

The question is :

Could we write a useful audio Critique/Review ? if yes How it could be?


my answer is yes but let me describe it in the next post.

The paradox of audio criticism is that a good system should leave us speechless!

Audio Philosophy

 

[Amir]

I will try to explain my idea about a high performance Reference audio system .

here only professional audio designers should speak about the limits of audio parts.

because I am not an Audio Expert and also I am not an Audio designer please do not blame me if this information is not super accurate.

I think in Audio even at cost no object levels we have some limits in performance and an audio designer should decide between two options.
for example paper drivers are good for music listening but when you need higher performance you will find the paper has some limits.
if a designer select Be instead of paper to have better performance then his ear will tell him Paper is more natural.

another example is NOS DAC vs Upsample DAC , I think NOS DACs are very natural but in some music you may prefer other DACs.
NOS DACs have best time domain Response and most Upsample DACs have best frequency response. some upsample DACs like Bidat and Wadia try to have good time domain response with their special digital filters but a NOS dac is a pure simple Design without any Digital/Analog Filter.

I have CEC TL0 3.0 Transport Audio (it is the best CD transport on the planet) and I have compared Audio Note DAC5 vs Weiss Medus/502 DAC for many many hours on two different systems. please do not think I am going to say every DAC has a sound and it depends on personal taste, NO I am not speaking about taste. I am going to say in most cases we have two option :
1- going for more Natural/Musical sound
2- going for more accuracy/performance

please consider it does not mean all Musical systems are not accurate and it does not mean all accurate systems are not musical. two approaches are different and each one covers one information area better. the Live sound contains sound information and Audio Playbacks try to introduce you those information , each approach has some limits and I always told before that I like to have two Audio Systems.
if I should have one system then I go for Natural/Musical System but Two Audio Systems will cover the entire range.
if an accurate system destroy all of music beauty then I avoid it and I prefer my small Sony Cassette tape player to a big soulless expensive system.
It is not the rule but I feel Natural Systems come from simple design (minimalist design) and accurate systems come from complex design.
it was around 2007 that I was convinced I need two Audio Systems even at cost no object levels.



It is like yin vs yang

for Audio Critique I think we should not Review a tube NOS DAC or a tube pre-amplifier in a complete high power SS system ,I will introduce more examples in the future.

an Audio Reviewer should have Two Reference System for example :

1- Kondo 300b/Living Voice Vox Olympian
2- TAD Reference System

Continue ...

 

[Amir]

Audio system setup :
1- Proper Matching
2- Proper Setup (Speaker Placement and ...)
3- Room Acoustics
4- AC Quality
5- Some tunes

If you want to Review a Component you should put that Component in a Match System.
For example you want to review a Wilson Alexandria Speaker , if you listen to more than 20 amplifiers you may convince Wilson Alexandria (like High Efficiency Paper dynamic driver Speakers) like Tube amplification more than SS amplification and the challenge is wilson speakers are not easy load for tube amplifiers.
One solution is using wilson crossover and cutting under 40hz for main speakers and using Wilson Subwoofer for under 40hz.
If you do not find a good amplifier match to Wilson Alexandria you never never never hear right sound from Wilson Alexandria and when your amplifier is not match to Speaker you will hear coloration , you have seen many audio reviewers and many audiophiles think Wilson Alexandria is not musical and is not transparent and some believe Alexandria is bright. These nonsense judgments come from non-properly setup for Wilson Alexandria (or any DUT).
The internet (over 95%) is full of these nonsense judgments. I do not read/trust Audiophiles/Reviewers Judgment except those Expert Audiophiles.

The truth is getting good sound from Wilson Alexandria (or any Device under Test) depends on your Art of Audio System Setup and we can try to learn it from Expert audiophiles like Romy, David, Kevin , ...
I believe all nonsense hifi words from Audio Press come from non-properly setup systems. when you get perfect sound from Wilson Alexandria then there is no brightness in the sound, there is no midrange opaque, there is no bass boom and ...

Please check Michael Fremer TAD C600 review:

TAD C600 line preamplifier Page 2

All we ask is a usefully low output impedance, to drive potentially long cables connected to a (hopefully) high-input-impedance power amplifier sensitive enough to be driven to full output by a few volts, so that the preamplifier primarily acts as an attenuatorthough of course we also want the...

www.stereophile.com

"When I relistened to the darTZeel preamp, the first notes I took about its bass performance when I played the same tracks via the Meridian Digital Music Server were: "relatively soft and less than muscular by comparison" and "where's the punch?" The Ypsilon split the difference. The darTZeel and Ypsilon seemed to better integrate the bass with the rest of the audioband because, along with its tight, punchy, fully extended bass, the C600 was somewhat shy in the midbass and lean in the midrange. I couldn't describe the sound as rich, but I could certainly describe it as airy and fully extended on top."

He think TAD is leaner, Dartzeel is softer/sweeter and Ypsilon is richer. All these words come from non-properly setup non-matched systems. When you listen to properly setup Complete TAD system you will not hear lean sound from C600.
If you properly match Dartzeel you will not hear any problem in bass area.

There are many examples for this wrong type of reviews:
Reviewer told Amplifier X has better bass, Amplifier Y has better midrange, Amplifier Z has better image and ...
This type of audio revewing is 100% wrong and I see many audiophiles read these nonsense Reviews and after 40 years of buying/selling/changing audio components they have awful sound in their home.

 

[Amir]

Expert vs Non-Expert Audiophiles

An Expert Audiophile does not change any audio component in his chain before understanding two things:

1 - what aspect of the sound should be better
2 - exactly what thing (or things) in audio chain or system setup or room or AC quality or ... could improve that aspect of the sound without any drawback

When Expert Audiophiles listen to an audio system they do not judge any Audio component before having an accurate analyse . Expert Audiophiles have right conclusions about audio components and their opinion does not change every 4 month.
If you see an Audiophile change his system every 6 month then he does not have right analyse/conclusion about his sound.

Expert Audiophiles could properly setup your system to have so much better sound.

The important key is Expert Audiophiles have more experince in Analysing a system. If the problem is from speaker position an expert audiophile does not change his DAC or ...

Expert audiophiles know more about better sound and for example they avoid using any thing that color the sound (like ac filters).

Another example is Expert Audiophiles do not trade off dynamics for having blacker background or better soundstage.

Only Expert Audiophiles can help us to have better sound.

 

[Amir]

Before writing about Proper audio Setup I should thanks Roman Bessnow (Romy the Cat) , Clark Johnsen, Jim Smith and David (@ddk) for sharing their experience About Audio and proper audio setup.

I also will describe why most audio Reviews (from Robert Harley, John Atkinson, Michael Fremer , Johnathan Valin and ...) are Completely useless for audiophiles.

Proper audio System Setup :

you have 4 things is Audio:

1- Room Acoustics
2- Audio Components (Transport/DAC/Turntable/Pre/Power/Speaker...)
3- AC Power
4- Media (CD or LP or File/Stream or Radio or Tape ...)

You can not listen to music without these four things.
The Media is out of our discussion because we can not change It.
There are also some parameters like Magnetic wave and EMI/RFI in room + Air condition like air humidity ... these are not in our discussion.

All parameters (like dac, phono, turntable, cables, ac power, speaker, amplifier, room, speaker/room matching, amplifier/speaker matching,
Speaker place, air humidity and ...) affect on the sound in room.

In audio The Most Important question is "when you listen to music how each parameter (room, speaker position, audio hardware, ac quality ...) affects the sound?"

The right answer to this question is the prerequisite for having good sound analyze.
The most audio magazines, the most audio reviewers, the most audiophiles all are speaking just about changing audio hardwares.
Imagine you have the best high end system in the world and you place it in your room. What you will hear is not good, maybe it will be awful, why?
Because the "proper setup" will improve the sound from 20% to 100%.
In my experience the room acoustics Speaker intraction is the most important factor. The matching of speaker and Room and the speaker placement have the biggest effect on sound.
You can read about DPOLS :

The secret unveiled: Roman Bessnow's DPoLS, speakers Imbedded Macro Positioning and playback in-wide-phase to room or the Quest for AEZ;-)

I consider the reading of some essay Roman Bessnow wrote in his Romy the Cat/GoodSoundClub forum to be among the best speculations ever writ...

twogoodears.blogspot.com

If you listen to a speaker in DPOLS then you will shock.

I had this experience one time in my life and accidently the speakers were in good place. It was voice of god from a mid level system.

The next important factor is amplifier speaker matching.

If your amplifier is not perfect match for your speaker then you will not have good sound even if you pay $2.000.000.

The next important factor is AC quality.

If the AC quality does not change your sound then it means your system is not enough transpernt.
Let me describe why we should not judge any audio System or any audio component without fully understand effects of these three parameters on sound.
When you change one of your audio components then new component change the game, if your speaker position was perfect for pervious component then now your speaker position is not perfect for new component, for example after changing your DAC or your amplifier then you should change speaker position.

If you change your speaker you should change your amplifier and when you change your amplifier you may need to change your pre amplifier and finally you need to change your speaker position

 

[PYP]

1. In my experience the room acoustics Speaker intraction is the most important factor. The matching of speaker and Room and the speaker placement have the biggest effect on sound.

2. The next important factor is amplifier speaker matching.

3. The next important factor is AC quality.

(I added the numbering to your text). This is a helpful list. Ideally, what do you suggest is #4 on this list? In my own process, #3 was the last piece of assembling an audio setup.

I was lucky to read about an audio show demonstration of the speakers I already had with newly released amplifiers. I arranged an in-home demo of the new amps. That worked very well (according to my ears).

Once I had established 1 and 2, I chose other gear. It was years later that I found the ideal power distributor. It would be interesting to speculate if I would have chosen different gear if #3 had been in place early on. Perhaps it worked out anyway because my DAC was designed using the amps (same designer/manufacturer - no impedance mismatch and voicing was done together).

Next, I needed to improve the streaming side of things. Reviews got me interested in the Grimm MU1 streamer/DDC. Many of those reviews paired the Grimm with my DAC (was a geographic coincidence -- both manufacturers are located in the Netherlands).

Although I agree that many, many reviews are not useful, when you see universal praise for a pairing, it is worth investigating. But only an in-home demo can help us decide if the equipment will work with our setup and our ears/preferences.

 

[Amir]

Thank you PYP for your comment .

Proper Audio system setup :
1. room acoustics Speaker interaction (matching of speaker and Room and the speaker position in room)

2. amplifier speaker matching

3. AC quality

4. Mixing and matching of other audio components after Choosing power amplifier and Speaker

5. Some tunes like considering vibration control

some notes :

1.1 I do not recommend filling all the room with huge amount of acoustic panels (diffusers/absorbers/reflectors/bass traps) , all of them will add coloration to the sound specially in midrange tone. I hate those tone .
Filling room with furniture and flowers and some bookshelf and ... to lower the RT is better , you can read about it in Jim smith book. You can check Both Kevin (Living Voice) and Roy Gregory rooms.
I like natural wood tone and I prefer to have natural wood parquet on the floor. The important key is near speakers we should be very careful using any things and I prefer to use natural wood for surface of side walls near speaker.

1.2 your Speaker should be match to your Room , Room dimensions affect on choosing your speaker.
I prefer to have wide room (7m -> 9m) for big dynamic driver speakers and I hate side wall reflections.
I prefer to hear my room when my speakers load the room (bass and upper bass) , I like the SPL that starts loading my room ,maybe I am wrong but I did not see any Small speakers could load Very large rooms (over 100sqm) and if you have big speakers in very small rooms then your task for speaker placement is not easy.
I think we need enough space for breathing our big Dynamic driver speakers.

1.3 speaker placement is not easy , I think every audiophile should put over 100 hours for speaker placement. If you do not like it you can invite Jim Smith to help you.
First we should find the area with best 50hz-500hz response then tuning for over 500hz. It is not easy to find DPOLS but you can find semi perfect speaker position.
Good speaker position :
- so much better emotional reaction to music
- better dynamics, both macro (jump and punch) and micro (decay)
- less distortion at higher SPL
- more relax less forward and better 3D Image , Image is different to soundstage
- more open sound in all frequency
- so much better bass and overall tonal balance specially at low SPL
Sometimes best soundstage will give you compress and forward sound, you should avoid it.
We need to get maximum dynamics. Dynamic is very important but many audiophiles prefer compress sound to have better resolution.
Here Experts like Jim Smith can help you , you need their help if you are not expert.

Romy the Cat believes a cheap vintage JBl in DPOLS outperform ($2.000.000) mega expensive systems.
We need proper audio setup before any audio critique. I bet most reviewers do not care about speaker position and they do not spend many hours for proper speaker placement. All you read in those audio press reviews are very primitive and useless information

 

[Amir]

2. Amplifier speaker matching

this is very important , David (@ddk) believes we should first choose amplifier then choose the matched speaker , I can not disagree him but in the end your combination (speaker/amplifier) should be perfect match. I think matching is not only about drive and sufficient power. Matching is about all we hear and will improve the sound by wide margin.
I never heard perfect dynamic sound from combination of tubes with big dynamic driver speakers (under 90db) . You may disagree me but I think for perfect dynamics and bass extension and playing complex big orchestral music you should go for 5way-7way horns if you like tube amplifier.

For low efficiency speakers you need much more power and always much more power is not good for sound. solidstate amplifiers were not good in past but now we see good solidstate amplifiers in this market like Vitus, CH, Dartzeel . Most speaker and amplifier reviews in audio press are not trusted because they review without considering speaker amplifier matching

 

[Amir]

3. AC quality

I use no Isolation Transformer I use no UPS I use no ground control I use no ac conditioner I use no ac filter , I use nothing between ac panel and my system. I just use low impedance cable (60m) between ac panel and my system.
You can ask david about it , you can ask roy gregory about it , you can ask jim smith about it.

I have spend many many times/effort/money but all of those high end AC solutions were awful. If you hear good sound from ac conditioners it means your system is not transparent or your ears like compress sound more.

most reviewers use these ac conditioners and they review sound when those ac conditioners are in the chain. I think all of those reviews is not trusted when ac conditioners will remove many information of sound.

if you want natural sound and if you want to have true micro dynamics then you should avoid any AC conditioner.

when AC quality is good you hear good sound but when AC quality is super good you hear shocking sound, AC quality will affect on both dynamics and harmonics , more on harmonics.