Yeah, that sounds like your fan switch is knackered. You could do a com check on it, see at what temperature the switch is closing and then at what temperature it is opening but by the sound of it its not opening anyway so you might as well replace it.