You sure can spend crazy money on bikes! and just like audio it's all subjective and only a persons opinion. One of the guys $8K CF frame cracked last night - ouch! I was looking at a similar bike I have but in the CF - $1.5k more but was advised of it being more delicate so to take such into consideration so I went with the AL and very happy indeed.
Most road bike use carbon frames for the low weight and comfortable ride. On mtn bikes, there is the added benefit of placing the rear suspension in the optimal location (without compromises from conventional Al tubing). I've noticed that a carbon frame is doesn't rattle and holds a better line when tackling a rough descent.
One little known fact is that carbon frames are easily repaired outside of warranty. Calfee Design has been working with carbon fiber for a long time and can repair most damaged frames. By contrast, a damaged aluminum frame is usually un-repairable (ie. welding & heat treatment are tricky).