- factory annoys enough people
- people are anarchist and have no respect for law
- people are not living buddhas and have finite patience
- multiple attempts at settling the issue by negotiation fail
- some neigbours move away to undisclosed locations, some of them rather pissed off
- somebody spray-paints "sabotage?" on a nearby wall
- somehow, the factory keeps losing power and water supply, roads keep getting sowed with tire-breaking scrap metal, equipment accumulates damage
- factory continues to annoy enough people
- somebody spray-paints "BtFD" on a nearby wall
- somebody interprets it as "Burn the Factory/Fucker Down" and does so
- zoning laws?
Edit: P.S.
Two alternative laws:
- there are no fixed laws
- perhaps it's worth the effort to behave reasonably?

What you describe is the reason zoning laws exist. Its not good that people have to resort to violence in order to protect their homes or their premises. For both the homeowner and the factory owner, a zoning law that settles the issue before the factory is built is the best option.