I bypassed the cooler on mine due to similar symptoms except I did also get oil in my coolant. I wasnt prepared to go through all the ball-ache of swapping it if it turned out the problem was elsewhere.
I notieced no real difference to engine or oil temps with it out the system, but I did notice my coolant was no longer oily

I did loose a small ammount of oil from the cooler, but it was prob a drop every couple of days, if that. If you're worried join the connections on the cooler together with a loop of hose, that'll stop any oil being lost.
Then if your oil/coolant mixing problem is solved, plan to swap the cooler when its warm out!