If it's easier for you to get in, then chances are it's easier for someone else to get in too. As long as there are undesirables around i'll prioritise the security of having a key over the very minor inconvenience of having to lift a key out of my pocket.