Yes, a good headlamp restoration kit should make a noticeable difference, or a machine polisher and an appropriate combination of pad and polish - that’s assuming the frosted appearance on the OP’s headlamps is on the outside and not the inside of the headlamp.
I cleaned up the headlamps on a friend’s car with my DA polishing machine and polish on a foam applicator pad. The headlamp ‘glass’ (I think it was plastic rather than glass) had become very cloudy with age and improper washing probably also contributed to the cloudiness.
I was surprised how well they polished up. I’m pretty certain that cloudy headlamps are an MOT fail, so a bit of time and effort improved them to the point that the car sailed through it’s MOT.