Actually, the threat of force is the same as using force. And it works both ways. If they are within striking distance and it's 3v1 and are acting aggressively and threating bodily harm you are within your rights to pull your firearm. If they are actually charging you and you tell them to stop (SCREAM IT) and they refuse, then you can shoot.
I really need to get my Indiana Handgun Law book out and scan some pages for you all. If you don't have the book, buy it. It'll be the best book investment you'll ever make. If you live in Indiana of course.
I hope, for your sake, that you never have to test this theory of yours. I tried a vaguely similar act that you described, except I was on my own property, and I almost went to jail just for putting my hand on my holstered firearm.
Also, thanks for the reps. They are much appreciated.