Hey Dem, since you are running an XC setup, you need to measure the eye to eye @ 12" off the ground instead of 8.5". The 8.5" measurement is for TT and flat track.
As far as your front spring measurement, I think you need to call Drew Cox @ Maul tech and ask him how much the front main spring should compress for an XC bike. My conversations with Andy were based on mx setup, so I'm not sure if the compressed dimension of the main spring would be the same or not. I'm sure he should be able to tell you, then you'll know whether it is the correct lb main spring or not.
As far as the symptoms that you mentioned having with your front shocks, they could be caused by any of the following/or a combination:
1. Rebound set too fast
2. main spring too heavy
3. Crossover height incorrect
4. Shock valving incorrect
Hope this helps.