It's important to consider whether your son's worries at school come from a tendency to be a perfectionist. In this case, both you and the teacher should be setting reasonable standards for him. Also, a chat with the school psychologist, if there is one, could give you additional ideas about how to help your child.
You should be demonstrating a relaxed attitude towards the work your child does at school because it will help him be more relaxed. Stop mentioning grades, and start talking about the things that he does at school that are fun. When your son brings work home, show your pride in what he has learned -- not in the grade he has received. Tell him frequently that everyone, including you and his teacher, makes mistakes and that they are just part of the learning process. As far as spelling goes, make helping him with his spelling words a routine activity rather than calling it a confidence-building one.