The motor under the passenger dash tray might need a good clean as i had this problem too. I also took out the motor bushes which where covered in water and cleaned them up. Works perfect now.
It happens when water enters the scuttle area under the bonnet and drips on the motor.
Switch your fan on and 'tap' the motor/blower bit. If it slows down or speeds up then do them steps^