Given the price of large and more efficient panels needed to power a mining hardware that consumes a lot of energy, a small scale or medium scale miner might not have the amount of money to buy those. And it will take a longer time for that investment to return. Also, when the price dumps, it is another loss for the miner. Another is the halving of generated bitcoins for every block, there will be lesser and lesser bitcoin produced in the future. It's not that easy to start with buying a pricey solar panel.
electricity that uses solar panel power does provide a lot of benefits because the energy produced can be used for mining or other activities, but the most terrible threat is when bitcoin has reached the maximum production limit which means that it can't be mining anymore, mining equipment not used to mine bitcoin anymore.