You mean the thermo switch in the rad don't you

Aye as silkyballs said replace the cap with a new blue one as it is a better design. There isn't a thermostat in it though, just a pressure release valve.
When you say 'it overheats' what actually happens, and where is the needle on the temp gauge?
If its only got pure water in there I don't think it makes too much difference though, I had to run it like that for a short while (to get me home) and it dodn't overheat or anything like that. Obviously you don't want to leave it like that overnight this time of year or it may freeze & pop out your core plugs!
To refil the coolant system:
Run the engine from cold with the cap off and rev it to 2k now and again. While you are doing this squeeze all the pipes you can get your hands on, and keep the coolant level topped up. It will eventually start to boil over, at this point turn the engine off. Once its settled down top it up to the max mark and you should be good to go!
Check the lower rad pipe is stone cold until the temp gauge reaches the half way point, and that when it gets near 3/4 the fan kicks in. If not jump the fan direct from the battery, if it still dont spin replace it, if it does replace the radiator thermoswitch.