Agreed, you shouldnt have! People have been working on both software and hardware refinement and improvements happen all the time.
Yes the trick is to turn a nasty general purpose PC into a music player without using a thin sounding OS. This means choosing the right Mobo with proper electrical subsystem handling, with proper shielding, proper filters on the drives, proper power being fed to the whole box, proper, separation of functions of the internal drives, proper isolation of the incoming signal, proper formatting of all, effective jitter reducing software, effective bios tweaking, effective throttling of the OS and CPU.
Basically, making the computer into a BAD general processing computer and into a great music server/player. I find when you do this, you dont have bandwith for upsampling and other fancy resource eating stuff. You just have a divine native rate playback beast.