Can't you just host the app outside the apple store? Or would they shut it down through lawsuit threats
Non-jailbroken iPhones can't install anything but aproved apple store apps. Period. No workaround possible. When you consider that 90%+ of user base is never going to jailbreak their phone it means you have an effective marketshare of less than Windows Mobile.
Android on the other hand by default only installs apps from android marketplace (now stupidly rebranded "google play" WTF?) but if you go into settings it is simply an off/on slider to allow installs from other locations.