As you've said really. Pot holes, speed bumps, driving into kerbs, turning the steering wheel when the wheel is against the kerb, track rod end not being tight, worn steering rack bushes, loose steering rack nuts, worn wheel bearings/replacement, worn ball joint/replacement, worn suspension strut/replacement, worn wishbone/replacement, loose ball joint bolts, loose suspension strut bolt, any loose bolts around the wheel area, lowering/raising the suspension, changing the camber etc.
Almost anything can knock it out!