Right, so then we're back at my initial question: Is difficulty irrelevant as long as you know the net hash and your own hash rate?

Sounds to me that it is irrelevant. I mean, the difficulty by itself doesn't mean anything if you don't know net hash and your own. But if you know the net hash and your own, the difficulty number is irrelevant.