My family spends a lot of time driving to and from various activities, and I found that this was a good time to incorporate learning into our day. I'd make up silly stories and songs (". . .little Johneeeeee lives at twenty-threeeeeee") and repeat them frequently. Then I'd leave out a portion, or purposely say something absurd where the correct number was supposed to go. Children love to correct their parents, and my daughters would chime in with the right answer.
I'd say it took about a year for my girls to really learn their address, so don't get discouraged. With young children, you must make learning fun and be very creative. If he's bored (i.e. "doesn't give you his attention"), that means the method you're using isn't working and it's up to you to find another way to get your point across.
You can try something similar at home. Together, you can create letter cards on pieces of cardboard. Try making one or two a day. Write a large "A" and draw or cut out a picture of an apple, "B" and a banana, etc. One day is "A" day, another is "B" day. On "B" day, when you hand him a banana, say "b-b-b-banana, for my b-b-b-boy! Those are 'B' words aren't they!" and hold up your "B" card. Do this on and off during the day. Sounds silly, but it works. Don't push him, but be consistent. If it doesn't work this week, drop it and try again in a few weeks.
You might also try letter and number magnets, then use your refrigerator as your "paper."
Also, ask your librarian to recommend good books and audio tapes to help him grasp letter and number concepts. Reading beautifully illustrated alphabet books (there are many to choose from) together is a lot more fun than simply writing abstract symbols that won't really make sense for a few years. In fact, reading every day with your son now is probably the best way to insure his educational success in later years. Good luck!