My mother (their grandmother), my wife and myself are not big believers in an afterlife and are not churchgoers.
I believe in truth, but don't believe youngsters can understand this truth.
Keep your explanation of her death simple- for your young kids it should be very simple. I believe in using the proper, correct vocabulary, not phrases like "she's asleep now", "we lost her". Say the real words- heart attack, dead, funeral. You might simply say "Grandma had a very sick heart that couldn't work any more and she died." After stating the facts as briefly and truthfully as possible, ask them if there's anything they want to know. Even if they don't ask anything then and there, let them know they can come back to you at any time with questions. Kids your childrens' ages often ask, "What's dead?" Again being direct and offering a simple truthful answer is best- "Dead is when all your body's functions (substitute a word your kids will understand) stop, your heart isn't working, you don't breathe anymore, you can't see or hear."
Given your non-belief in an afterlife, I don't think you should tell your kids anything you don't believe in yourself. You can opt to say, however, something like: "Some folks believe in this (the body goes back to nature and nourishes the soil and grandma will live in our memories) and some people believe something else (that people go to a place called heaven where there is a god). I don't really know what happens." Sometimes giving too many options to young kids, especially your three-year-old, is too confusing.
Helen Fitzgerald, a grief counselor, has written a fine book called, "The Grieving Child". A few other age-appropriate books for both your kids are: "Goodbye Max", by Holly Keller and "The Tenth Good Thing About Barney", by Judith Viorst and "Lifetimes: The Beautiful Way to Explain Death to Children", by Mellonie and Ingpen.
Thank you for asking such an important question.
