Yes, it's a noxious poison because they are not holding it. If they did, I reckon they would be its biggest supporters in Wall Street.
Well, not buying an asset which they're aware of would imply that they don't think it's a good investment opportunity for them. Expressing this certainly doesn't classify as "market manipulation" by itself.
Since Warren Buffet hasn't spread objectively false information, it would be unreasonable to assume that he's manipulating the market.
Regardless, this isn't Warren Buffet himself. He can't be guilty by association - he hasn't done much other than express his skepticism about the price.