I had this last year on my 2002 GT TDI 130. It happened a couple of times randomly and when I released the pedal and pressed down again all was ok. Quite scary though!
Then one day the pedal went rock hard with almost no braking at all, in traffic and it wouldnt go away. My local specialist thought it was vacuum pump but the vacuum was good when the pipe was removed. It turned out to be the servo, once they changed the servo all has been great since. They said that they don't do many, but about the same time they had a couple of Audi A4's do the same thing.
Yours could be the vacuum pump, but be ready for it being the servo.