I would suggest that you and your partner identify a person your son is most likely to open up to -- or perhaps a close adult friend or relative or a professional counselor at his school or in an agency. Whoever it is should listen to your son with firmness and kindness, reading between the lines. Ask him what will be different at a new school; how will he be different? Ask if he will return to his skateboarding and swimming? Why would he be able to do at a new school and not at this one? Help him learn that he can resolve whatever is going on at his home school and that he needs to do this for his own sense of empowerment.
It's often easy for us as adults to know that the grass is not necessarily greener on the other side, but an 11-year-old just hasn't had enough experience to understand that truth.