The method is good as idea but you will lose significant amount of speed because you need to do scalar multiplication for each key, the secp256k1 sequential tricks will not work here.

So the new question will be arises, Is it worth to lose that much of speed to avoid that amount of keys?