Did the new one make a difference? Hope it is working!
Just to confirm does the throttle mechanism on idle touch the idle switch? If it does, that's how it should be. From what I read, you adjusted it so the throttle was open? This bypasses the need for the ISV as the engine is getting it's air supply from the open throttle, when the throttle is closed only then should the ISV kicks in.
By removing the ISV you created a big air leak, enough to stall the engine I'd assume as the fuelling wouldn't cope.
If the throttle set-up is ok, switch and wiring are proven to be ok and you are still having problems I'd check the voltage supply to the ISV if you haven't already. You are giving it 12v from the battery but maybe the ECU isn't?
Hope it's all working anyhow!