Your not going to get the finish you want unless you machine it, hand polishing as you have discovered will only get you so far, technique is key with hand polishing but you'll never get the finish right without machining,
If machining or getting it machined isn't an option then I'd suggest glazing it after you are semi happy with the finish after polishing, it will mask the fine swirls a bit.