For Inoue's situation, most of his fights happen in Japan because it is already proven that they can generate a much bigger profit if the fight will be happen there. That's why Fulton have to travel towards the challenger's land because of this reason.
That's how the business should be run – the more money they make, the better it is for Inoue, so he can enjoy his popularity. I don't have an idea about how much his previous fights made in terms of PPV, if there are any, but perhaps the competitive amount generated by fights held in the USA is the reason they chose to keep the fight in Japan.
Some say that judges may favor Inoue, but so far, I haven't seen a controversial fight of Inoue where the judges favored him despite his performance not being so convincing because most of his wins are pretty convincing and majority are in KO.