I was involved in an incident like that years ago. I was driving my dad's old Granada Estate and a car coming the other way went wide around a car parked on his side, I had no-where to go and there was nothing on my side, so he was in my lane. Wing mirrors clashed and I stopped a short distance up the road (so as not to block the road, with the parked car on the other side). After a quick check, the spring-loaded Granada wing mirror was confirmed as undamaged and bounced back into place. The other car had not stopped. It was out of mind, no harm to the Granada and 3 weeks later my dad got a letter from the bobbys claiming that he'd had an accident in the car and not stopped (they must've hung around long enough to get the reg).
The bobbys came round for a statement, the other side's version of events was that I drove around the parked car on my side of the road and hadn't stopped at all. As it turned out, their wing mirror was smashed to smithereens, but with no independent witnesses either side, it was my word against theirs. Passengers do not count as witnesses as they are not independent.
This person may well think about it and deny any wrong-doing if there's door damage to pay for too, probably thinking initially it would only be a £150 fix. Might be best to tell your insurance to pursue it as a proper claim to avoid any back-tracking.