Next year, your son will have a different teacher who may make the work more interesting and better appreciate his abilities. Plus, fourth grade is far more challenging than third grade. The emphasis moves away from mastering the basics, which your son has probably accomplished, to working more with science, social studies, and literature, which he should appreciate since he loves to read.
Keep in mind that opportunities for your son to learn should not be limited to school. You definitely need to plan a stimulating, challenging, and academic summer for him. Look into library, museum, and college programs as well as summer camps that offer both fun and academic challenges.
Since your son is an avid reader, encourage this activity by taking him to the library. Also, he might enjoy a subscription to a magazine. Reading is definitely the key to academic success. If you don't already read to your son, read to him every evening. It will further expand his vocabulary and comprehension skills.
Since the school is not willing to test your child, you may wish to have him privately tested to find out if he is academically gifted. Take the results to the school during the summer so that arrangements can be made to make next year more academically challenging for him.
Learn more: Choosing the right books for your kids.