In a truly liberty based society, the only rules would be the rules created by the private property owners. So this would be a non-issue. Unfortunately, some political parties look to immigrants to shift national voting patterns - but this just a symptom of allowing the majority to force their ways on the minority. Coercion and force.