Harry Potter fans are no threat to anyone. Ethnic groups, however...
The world has limited resources, whatever the masses want to believe. Humans have always competed, group to group, and that won't change. It's hardcoded on a genetic level. The only thing that changes is the precise form it takes. And this is the root of racism. Without racism it would not be possible to kill another people, but it would still be very possible to be killed. And so racism is a trait of evolutionary winners. And the immigrants are very, very racist. If we don't fight back, we will die out.
In Europe we've been killing each other for millennia and this wasn't due to skin colour shades. In the US volunteers regularly carry out executions with no distinction.
A human can easily kill another human being simply by inhibiting his or her empathic instincts.
About the world resources scarcity, this is a different matter. We're either too many or too inefficient, but our problem has been experienced countless times by other species: the depletion of resources which then become the limiting factor.
We fight whoever is closest or the biggest threat or the most convenient at the time. The more globalized the world has become the bigger groups we have formed. Nothing changes, we still fight. Only now the enemy is within our own borders. And it's not even the immigrants themselves, they are just a symptom.