Yep, lose the chamois. Nothing to do with your problem, they're just rubbish.
You like me obviously have hard water - mine is an absolute swine to rinse. You can get all manor of filters and stuff to remove the lime from the water but none of them are particularly cheap or usable with low pressure hoses.
The likes of Demon Shine (you're effectively using it as drying aid) do help but note, there is more than one type of Demon Shine and you'll find varying degrees of success. The one you buy in a spray bottle isn't as effective as the stuff which comes in the small jugs (I prefer larger ones though, sorry!).
Best option is to wash out of direct sunlight, dry with a proper drying towel and once you have a good protection coat (wax etc) by using a trickle from a hose or a watering can to sheet the water off, you will end up with far less spots. The spray from a jet wash/squirty hose will make the problem worse but by "chasing" the water from the paint, you will end up with less remaing on there and therefore less to dry out in the first place.
If you still end up with the odd spot, use a spray quick detailer (there are products you can use as both quick detailers and drying aids) and a good micro fibre and you'll be good to go. The better your polishing/sealing/waxing etc, the easier they'll come off.
Pain in the bum though mate, I feel your pain!