There is no "time" in the plot, it just shows phase vs. frequency. A difference in group delay causes a related difference in phase shift. I do not know the SW tool you are using so have no idea how it normalized the signals but that is probably why they are not colinear (or nearly so).
Linear phase means the change in phase with frequency is linear. If you take the derivative of phase with respect to frequency, i.e. find the slope of the phsae over frequency, for a linear-phase filter the answer is a constant (constant group delay). That means al frequencies are delayed equally and so time response is preserved (a pulse coming out is the same as a pulse going in, just delayed in time). If the filter is not linear-phase, some frequencies will be delayed (phase-shifted) more than others, and a pulse coming out will not look exactly like the pulse that went in. How much this matters in the real world is, as always, debatable. I used to think it mattered a great deal, but numerous tests (listening and measurements) led me to believe it was over-hyped, at least in my mind. Still a worthwhile goal, and a requirement in my world at work, but less so for audio systems.
Here's a short Wikipedia article I found with a quick Google search:
http://en.wikipedia.org/wiki/Linear_phase