I don't think that "upset others" equals "harm others". In case of the cartoons, I think the cartoons themselves did not do any harm to anyone. Even the people who felt "offended" were not harmed. But those, who were "offended" and felt free to start using violence against people who had nothing to do with those cartoons, just to express their "opinions", were harming others. Noone else. There is just no need to use violence because there is something drawn on a sheet of paper.
As I see it, "harming others" starts where human rights of others are violated and when property is destroyed or taken. I just fail to see how a drawing, written or spoken words would be able to do that.