There's no way to tell.

That's the point.  

EDIT: If Monero's flexible blocksize is consistently growing over a considerable period of time, Occam's razor would indicate people in general (but nobody specific) are using it, rather than a single well-funded adversary/pumper.