Yes, measuring the shock mounting is to make sure the shock lengths match the actual extended and compressed lengths of the actual chassis.
Ok, let's cut this into usable chucks. Let's start with the rear for now.
You said it was bouncing up and hitting you in the butt, you slowed the rebound, and it improved. You are on the right track.
1. Ride height seems about right. You shouldn't be putting any more preload on the spring than needed to either achieve ride height or keep the spring from bouncing freely. How many turns would you say you put on the preload adjuster ring in order to achieve the ride height (rough guess)?
2. You are not sure if you are using all of your travel. You said you checked your race sag, what were your measurements?
3. You said that you have your compression set to fully soft on rear shock, correct?