Steve, the risk of hearing damage is a function of volume at the ear drum, not the proximity of the driver to the eardrum. You can do every bit as much damage with a speaker system if you try hard enough. With a little caution, you should be able to enjoy headphone listening without damaging your hearing.
I own headphones from Sennheiser, Etymotics and Audio Technica. I've been through numerous amps. The Etys and ATs don't need them. The Senns benefit noticeably from the additional current and headroom, but amplifying headphones, even very difficult ones, is not a big amplification challenge and I've found that the law of diminishing returns is very, very steep in headphone amplification. Amir is right: In terms of resolution of detail, speakers don't compete with reference quality headphones. Even great near field monitors in a treated room will not be as revealing. Of course headphones don't compete with speakers in creating a stage-like image, and cannot deliver the visceral dynamics that can only come from moving large amounts of air. A trade-off of compromises, but headphone listening is a great, affordable, audiophile experience that many more should try. You can put together a headphone system that, in terms of detail resolution, will smoke the most esoteric speaker systems, even in a great room, for a fraction of what many audiophiles spend on cables.
P