This will reflect my own hobby horse, but the simple, obvious answer is that systems aren't good enough. If you appreciate the energy that music played or reproduced well brings to the environment then there should be no problems getting people on board. The trouble is, you take a favourite recording to a "good" dealer, and it sounds terrible on their fancy gear. "Your fault, sir", they say, "for bringing in a lousy recording. You see, you have to learn that a lot or most recordings are bad, and that a good system will reveal this, so you have get rid of all that rubbish, that poorly recorded material. When you play the RIGHT recordings, then you will understand how good this system is!"
"Blow this for a joke", say the potential initiate, and moves on ...
I have seen this happen just recently, with the friend I'm helping out with his analogue setup. Some neighbours who had been enthusiastic audio people many years ago popped in for a talk and listen. At the beginning the system was on song, and they were quite entranced. Then a weakness started to rear its ugly head, and the sound started to degrade. Subtle, but it was there, the sort of thing I mentioned here many times, earlier on. I could see the two start to get twitchy, and before long they stood up and made excuses for having to get on with the day. After they left, the system owner realised that he had made a slight mistake with a connection, and corrected the problem there and then. Presto, sound restored.
The point being, systems ALWAYS have to be on song, for ordinary people to "get it". People do have good hearing, and something not quite right will irritate them enough to say it's not worth it, and forget about getting involved ...
Frank