It's the last part that's the problem. I don't see any reason they shouldn't be able to enforce them against non-consenting third parties. If rights obtained by contracts are to be treated as real rights, then contracts will have to be enforceable against third parties.

Hypothetical: You and I own competing businesses. My employees have a clause in their employment contracts that if they quit to work for a competing business, they cannot contact prior customers for 180 days. You intentionally and knowingly hire my employees from me and pay them a bonus for each prior customer they get to switch from my company to yours. You specifically direct them to ignore the 180 day agreement. Can I sue you?