Check the chunky vacuum pope between the inlet manifold and the brake servo, there are a ew blanking plugs and a vacuum pipe attached to it, check the pipe itself and the blanking plugs for cracks.
If that don't help then it looks like the servo may be on its way out, its a costly part though so ideally you want a good second hand part.