I apologize if the following points were already made somewhere upthread.
Probably, the two most salient performance parameters for any digital signal coding scheme the Nyquist bandwidth (which, effectively sets the signal's high frequency limit for audio) and the native quantization noise level. DSD (sigma-delta modulation) maximizes Nyquist bandwidth, but sacrifices native quantization noise level to do so. PCM (pulse-code modulation) minimizes quantization noise level, but sacrifices Nyquist bandwidth. DSD seeks to mitigate it's high quantization noise level via noise-shaping, which doesn't remove the high quantization noise, but rather, relocates it above the audio band. PCM mitigates it's lesser Nyquist bandwidth via brickwall bandlimiting of the signal, completely eliminating it as an issue, per pure sampling theory.
I suggest that differences in subjective sound between these schemes are likely due to the fundamental trade-off between Nyqust bandwidth and quantization noise level. More specifically, is due the implementation of their respective performance parameter mitigation technologies - brickwall bandlimiting for PCM, and noise-shaping for DSD - and how those human ear-brain system perceives the net technical performance result via music.