Dear Friend,
You are in no way hindering your son's growth process. I wonder if your husband's negative response to your son's hugging you in church would be the same if his four year-old daughter was expressing this physical intimacy. This could be a "man thing" - your husband believing that a son should not be so clingy/attached to his mother at this age. His opinion and discomfort with your son's behavior may be due to his own treatment as a child and his fears of his son being/becoming a "momma's boy".
You describe your son as having no separation anxieties that are affecting his emotional and social development. On the contrary, he appears to be developing very nicely at all levels. It's natural for little children (and not so little children) to miss their moms and to want physical intimacy with them when they see them. Given your family work/life schedule, his time with you is limited. You also need to consider the memories and feelings attached to your own childhood, devoid of physical affection and loving words from your mom, as a driving force in your desire to have a physically affectionate relationship with your own kids. I am sure that they are mirroring what you are "giving out" to them regarding physical affection and saying, "I love you."
As long as your son's physical and verbal expressions of love do not appear driven by fear or anxiety, I would not turn away his displays of affection. Your husband may also have an unconscious jealousy of your relationship with your son. Please don't allow your differences about this to confuse your son or make him feel uncomfortable about expressing his affection for you or any other family member. If this creates a major problem between you and your husband, please seek some counseling about it as a couple. I guarantee you there's a lot more behind your husband's protests than is being spoken.