Its not too sluggish but when you consider you can buy a mkv gti and mod it for under ten grand and produce a faster car, then it becomes a no brainer.
Remap, fmic, tbe, induction system, hpfp and a few traction mods and you can achieve 160mph+ 0-60mph in well under 5 seconds and far better mid range power delivery. The Boxster block only produces 220lbft, whereas the Gti is capable of 260-300lbft, or more if you tighten the waste gate rods on the turbo.
My Gti is almost exactly the above, less a hpfp and will clock a 40-140mph time in just over 23 seconds, some 8 seconds quicker than the Boxster 3.2s. And go from 0-100mph in under 11 seconds, 3 whole seconds quicker than the 3.2s.
Here's some links to prove it...
http://www.car-videos.net/performance/speed.asp?Speed1=40&Speed2=140
http://www.car-videos.net/performance/speed.asp?Speed1=0&Speed2=100
And the best thing about it all, the Gti's running cost will be far less than the Porsche, regardless of mods. 
Hahahahahahaha....
And the GTI will still be a FWD shopping car with a reasonable chassis...not a proper sportscar.
The Boxster is brilliant.
Utterly brilliant...and a proper Porsche.
Those that slag it either can't drive, can't afford one, or have never driven one.
Do it.
If you are worried, just get an aftermarket warranty on it - or if you can justify it, a Porsche warranty.
Speed isn't everything...if you value the drive more than willy waving figures you will adore the Boxster.
It will swap ends if you get it wrong...but get it right, and it will make you smile like no VW ever will.
On a side note...
I came upon one in my heavily modified S3, on a wet day...and got utterly humped by it.
I barely saw which way he went.
In 5 years on my 996 (same basic engine) I had RMS leaks that were covered by warranty, but that I wouldn't have paid to have fixed if I didn't have a warranty...and no head gasket issues or any other problems.
It also cost less than my S3 to service.
It wore out no bushes, despite hooning round Europe and doing 18 laps of the 'ring.
It barely used any tyres (2 rears on 4 years, next to 3 full sets on the S3 in the same time/mileage)
Do it.