I know that there has traditionally been reluctance to assume accurate clocks (since NTP is seen as a central point of failure), but this appears to me to be a very weak dependency.
Well a system that aims for security can not rely on self-asserted time, and there is no distributed secure time. If you start from a false assumption people will systematically abuse your assumption.
Understood, but we aren't really relying on the self-asserted time are we? If we said that we give preference to the one with the earlier timestamp, or the one with the later timestamp, then people would certainly lie about their internal time. But given that we give preference to the block that matches our own internal timestamp the closest, it seems to me that the incentive is to be as accurate as possible with your self asserted time. As things stand today we are implicitly relying on self-asserted time in the sense that nodes assume that the time when a block is received is a good proxy for the time when it was mined. But, as selfish mining shows, this proxy is open to systematic abuse by the miner who mined the block and may choose to delay its broadcast. The miner can only adjust self-asserted time in one direction, but he has an incentive to do so. Under the modification proposed, the miner can adjust self-asserted time in both directions, but his incentive is to report time as accurately as possible.