An easy way to kill the carpenter bees is to wait until dusk. Get a cotton ball, soak it in Sevin, and plug the hole. The bee will be inside, and will die from the Sevin.
Tennis rackets are fun, but the "thud" of a big, red whiffleball bat is great too.
If you live out where it's safe...snake shot out of a 22 can be fun.