Context. If the topic of the conversation is "Sovereign citizens", then yes, that's a sign of extremism. Same as saying people who sign their driver's license with "UD" at the end. Its a sign.
I don't think anyone is making the statement that all people talking about individual liberties are extremists.