I've always had rubber mats as well as the carpet ones. when I bought my Passat this year, the first thing that I did was go to TPS and buy a new set of rubber mats and a replacement set of carpet one. However, 6 months on, I'm still using the carpet ones that came in the car and have the new ones put up for "best".
I'm quite liking having the carpet ones now, and my OCD is satisfied by knowing that I have a new set in the garage. I will say that I am cleaning them far more than I would with the rubber ones, meaning the car is getting hoovered at least twice a week now.