Not only should you speak to his mother, you should calmly and firmly, physically remove him from your children. Explain to him and his mother that he may not play with your children as long as he hits or uses his foul language and stick to your decision. She may be angry, but she may also appreciate someone else making the decision. After you have talked with her, you could let the children play together, but the very first time, he attacks one of your children, send him home with a firm message that you will give him a chance in two or three days to try again. After a couple of chances, do not let him play with your children again.
Quite frankly, I urge you to be present every time he is with your children as he may seek revenge. My guess is that this is a very sad, very disturbed child and that his mom just doesn't know what to do. She may be somewhat afraid of him herself. Maybe the two of you could find a parenting support group through the school or through the human resources department. Good luck.