Electricity doesn't produce CO2, its production does and not even all of it because solar panels don't produce CO2.
Miners don't produce CO2 just like your home computer doesn't do it.
Would CO2 not be produced if we stopped mining? The answer is no.
I think it's a pretty common myth but the construction of most renewable energy will actually incur some form of environmental degradation, CO2 or not. Solar panels don't just lay around and the manufacturing of the panels will essentially produce some pollution and it will take some time for the benefits of a solar panel to outweigh the environmental costs of it. It's obviously not without it's problem.
It would be wrong to assume that just because some mining operations uses renewable energy, all of the energy inputs are green energy. The issue on hand is that the electrical energy that could otherwise have been used for other purposes were used for Bitcoin mining. I wouldn't say it's completely invalid to attribute the CO2 production to Bitcoin mining because they use a ton of electricity and the production of which produces greenhouse gases.
No, CO2 production will never stop. But if Bitcoin mining doesn't use electricity, then the reliance on coal energy and/or natural gas would be lesser since that the energy requirements of the country would be lower.
To some extent, the above argument will hold water. But it's important to consider the transaction volume of Bitcoin on a daily basis.