You have completely missed one important fact: Native residents of those 'not affected' areas would try to kill every refugee trying to invade them to protect their land.