When you turn the key to start the car, the solenoid on the starter motor moves the motor toothed drive wheel onto the engine flywheel - the teeth interlock and the starter turns the flywheel, when you release the key the solenoid then pulls the motor toothed drive wheel off the flywheel. If your solenoid is sluggish or sticking, it doesn't pull the motor drive away in time, so the motor starts spinning at high speed with the flywheel or rubs against the flywheel making the noise.
IIRC you have to buy the solenoid with the starter motor combined.