Yes, we all hear things differently. But don't we also see things differently, react to situations differently, think differently, etc? In the classroom, we all take the same exam, yet our grades are so vastly different.
Hence, I'm unsure what your point is here. My guess is, for those with reasonably unimpaired hearing, we're all actually hearing much the same thing sonically but our brains interpret those sounds so vastly different. IOW, it's a developed skill, not an inherited one.
In the end, I'm guessing it's not too unlike the classroom exam. An A on the exam is the pinnacle. Many are capable but only some achieve it, others are striving toward it, some can never achieve it, and some are content with settling for less and sometimes far less.
Not saying you, but all too often it seems arguments such as this in high-end audio come across as more of an excuse for not striving to improve our ability to discern / interpret what we hear. Which I'm convinced is by far high-end audio's greatest weakness. And no, I'm not trying to imply I've achieved "it", but I certainly hope I'm routinely striving to improve it. After all, without the ability to sufficiently discern / interpret what we hear, what makes our listening sessions any different than that of a 5-year old's?