It's photography, which is art. You're not supposed to just get great photos every single time, or there would be no art to it and everyone could do it.
With panning it needs a lot of practice - just keep on with it and you'll get there - just stand at the side of the road, shooting cars, with a t-shirt that says 'photographer not terrorist'. Just in case, like.