They're equally annoying. Of course if said sportsmen have an inflated ego or bad attitude to begin with, then it just sucks to be around these kind of people anyways.
I have figured I don't so much care for sport kendo, though. Fencing and euro swordplay, again the same deal though I'm sure my friends at the SCA miss me. Not sure I could sucessfully take up chinese or other more elaborate asian swordplay, trying but tiger hook swords are really tough to use, but maybe the more direct forms of the Dao.
Soccer/int'l "football": you may have your opinions but you better not ruin things for other people on MY watch as a ref.
American Football: bleh, take the good with the bad. (I just really like the family nachos, or once in a blue moon a Michelob dark with gourmet pizza and peanuts both regular and super spicy on the side).
Basketball is ok but it tends to turn into mosh pits.
Hockey...depends. They go in looking for fights all the time anyhow. I laughed when one nasty player at a game I went to thought he could beat down the ref, only to have the ref suddenly come back and use judo to twist his arm and flip him over. Can't maneuver very well in all that armor, eh?
Hey, he brought it on himself and his arm was messed up for a month afterwards. Oops.