There are trade-offs associated with higher sampling rates. The benefit is that for every doubling of the sampling rate, the frequency at which the shifted noise rises out of the thermal noise (about -140dB) doubles. For 64fs, that's about 20KHz, for 128fs, about 40KHz, for 256fs, about 80KHz, and so on. In practical terms, since the shaped noise envelope has the same shape and spectral content regardless of sampling rate, (just shifted up an octave for every doubling of the sampling rate), it equates to a lessening of the noise at any one frequency above where the shifted noise becomes apparent of -6dB, or half, for every doubling of the sample rate.
The downside of higher sampling rates is available settling time. For 64fs (2.82MHz) the period is 355 nanoseconds. Give away say 25% for rise and fall times of the gating logic and jitter buffering, and you're left with about 266ns for the analog sample and hold circuitry to stabilize and lock in the level. Double the sampling rate to 128fs (5.6MHz, and now you have about 133ns. 256fs and you've got about 67ns. That's pretty high analog performance for a $30 chip. What suffers is accuracy. All that for 6 or 12dB of ultrasonic noise reduction that only your pet bat, and marginally designed amplifiers are affected?
Where it does amount to something is if you're a studio laying down tracks on one another, and just like tape of old, there's a +6dB increase of noise for every generation. Higher DSD sampling rates can now make many post production techniques practical.