Yes all those and then some have been discussed, tried or done for years now, you can even mine with paper and pencil if your inclined to do so. ASIC is the mining end game for some coins others are designed to be mined only on CPU's or HDD's. GPU's are used because they can mine most coins and are readily available.

I am curious if and when we can mine with one of these bleeding edge boxes;)

https://www.nvidia.com/en-us/data-center/dgx-1/