If you were being threatened or hit, you wouldn't expect people to ignore it, or to gather round in a circle and cheer. You'd demand protection.
As your son gets older, the value of feeling protected will be even more important, because the stakes go up each year. He'll probably be more likely to tell you what's going on if he sees that telling brings him benefits, and not embarrassment.
Consider speaking with the parents of the other kids, or if you observe the conduct, confront the kids themselves. Do whatever it takes. We shouldn't tolerate destructive behavior directed against a small child that we'd never tolerate against an adult. I'm not saying it's easy, but your son will be happier now and in the future if he believes the world is safe and he feels protected.