Yes, it will always loose a little charge. Idle voltage is only one aspect of battery charge.
Without measuring the current it is difficult to tell why you are loosing some charge when the battery is connected up. Voltage fluctuations with engine running is due to alternator.