Would you not be concerned to hear that unmistakable sound if you weren't the one holding the shotgun?
Yes that sound would concern me. So would the sound of a slide being racked on a semi auto. Or the sound of a revolver wheel spinning and then slamming shut. Or the sound of a hammer clicking back. Or the sound of an agitated hornets nest in August. Or the sound of my mother in law waddling through my front door.
But I am not the guy that goes breaking into houses, so what sounds concern me may only pin-point your position for a criminal that means to do you harm.