Well, you said you don't shoot after 4 or 5pm, but went on to say you were out shooting "last night." Maybe all of you should get together to come to an agreement on what afternoon, evening, and night, really means.
I would just keep shooting if it is lawful in the area and during reasonable times. I say that because what would you do if one of your neighbors said to stop shooting at 4pm, seven days a week? I would make them bring the problem to me. I'm not going to search for someone to put restrictions on me. Good neighbors talk to each other. An anonymous note means nothing.
Eff 'em.
Nice summary of Que's post, Zoomie!