it really does not.
brute forcing has always been about reducing the space you need to search. having the public key only changes the method but it will never reduce the search space. you still have to search the 256 bit space by only having the public key.