judging cause and effect, and understanding the why is important......even though you can't always understand the real reasons for things.
the idea of getting beyond that 'tipping point' is a concept that has been my experience. which is why to judge most tweaks you need to reach a level of system synergy and maturation. what you hear needs to be 'right' (a personal feeling, not fact) to then judge differences verses improvements. do you have the basics covered? and if you discover that something was holding things back in some way; you need to back track and examine any decisions made with that restriction involved.
i've avoided gear with coloration in my signal path so i was not needing to balance that out and have to go down that rabbit hole of-off setting additions.
coming back to dacs, servers, or related network tweaks; it's the same. you first have to select a dac that is synergistic with your system, so choosing gear to optimize that dac is as linear as possible. not a trivial pursuit.
i think the one exception is adding multiple turntables and cartridges; where different flavors in your source is a choice many make; but again ideally you started out with a system that is synergistic and mature. a good platform for multiple directions.
‘The phrase “system synergy and maturation” is very apt and descriptive. There’s a lot packed into that.