It would be a mistake to get rid of the alert key. And it is clear that the alert key has not been misused in the past.
Considering the necessity for miner's consensus, and considering how many miners are still using prehistoric versions of QT, we need a way to contact them.
Gavin has not done squat for years (thankfully as much of his work when he was active was useless and counterproductive) and is now basically a pariah to most people who understand the XT hostile takeover attempt event. There is no compelling reason for him to have an alert key and there are risk associated with him having it. If he supports another hostile takeover attempt the key could be mis-used to burn a lot of innocent people.
We aren't talking about removing the alert key completely, but removing the one that Gavin has acess to. This is completely different. I agree with tvbcof. Gavin has not been that active in the recent years.