contact ebay and let them know that he was being unreasonable and has multiple addresses which made you feel uneasy as the cost of the item was 6k...tell them you wanted to deal in person and he refused which made you feel uneasy as he maybe trying to scam you, ring them and explain in detail, I have done that and they are usually quite receptive...
^ i think im not bothered by the neg to be honest as long as the card is still in my possession thats all that matter at the end of the day it just a red number.