Are you noticing it as a client or from the server itself (hitch warning in the console)? If it's just as a client, try setting this in your client config:
seta cg_deferPlayers "1"
What that does is make it so that you don't load new skins/models that a player may be using when they join and instead wait for the end of a round or until you hit the tab key to show the scores. When set, you'll notice a quit bit of lag, but at least you can decide when it happens.
"Do you expect me to talk?"
"No, Mr. Bond, I expect you to DIE."