First thing to check is that they're all working in-phase with one another - do all your rca cables go from red to red and black (or white) to black (or white)?
Do you have all the positive outputs connected to the positive speaker terminals? Any speakers running muddled connections will sound audibly different from those that are connected correctly.
Do either of the amps have a 'phase' switch? Check that - they are both using the same setting (they're normally either 0 degrees or 180 degrees, with 0 degrees being the default setting).