aggravator

(noun) annoyance

unpleasant person (generic term)
disagreeable person (generic term)