This is yet another problem involving sufficient power supplies
From ur description, it is quite clear that you have just ONE live wire goin to the radio whereas you actually require TWO.
As another chappy mentioned, you require a permanent LIVE and a Switched Ignition LIVE.
However, not all older cars have an ignition LIVE n may only have the one. So therfore, being a JVC, on the radio loom that comes with it, just look at the Red n Yellow LIVE wires behind the radio, you will find they are attached via Bullet connections. One may be a double connection with one free bullet plug entry and the other just a single bullet connection.
Put whatever wire that is in the single bullet plug connection into the double one so in essence you are running both red and yellow wires off the same single wire coming from the car.If it powers up, then it has resolved your problem and diagnosed that you are initially a LIVE wire short. But from sum Mk2 golfs i have worked on, they DO have ignition LIVES so if you cannot locate one behind the radio connections or if there is already a ISO block connector ther, mite b worth checking your fuses to see which ones cater for radio function and find you may have a blown fuse that may solve your ignition LIVE problem and from then...activate and switch on with activation from your car ignition rather than independant even without ignition. But that is your own preference as to how you would prefer it. Its all simply done.
As for another members comment, JVC's do not have a history of having to be turned on everysingle time regardless as to if it was left in playing mode just before powering down your ignition. If that is the case, then u may possibly find that all your saved stations also reset and any sound settings you have set on the on board EQ. like bass and treble settings.
This is a simple case of your 2 LIVE wires (red & Yellow) are switched the wrong way round...so if you re-look at the JVC loom behind the radio, you may find that red is in yellow and yellow is in red....just switch them back to normal and that will then solve you problem too. If in the case the red is going to red and yellow to yellow on your JVC loom, then reverse it accordingly so it was as previously mentioned and then that may solve your issue!
If you're still unsure, then holler back:-)