It's supposed to be different, but like you I'm not so sure. My reason being...
If I set my car to 'sport' mode
and the gearbox is showing 'S', sitting with the engine at idle I can switch the lever/paddle to 'M' gears and the idle revs will drop, indicating it's not so eager and under a different RPM setting to Sport.
So I'm not convinced that Manual mode isn't just one setting, rather than being manual gears for each mode you may be in.