Turn on the overtemperature shutdown in the bitstream, and control the temperature in the mining software. It is easy.
Turn on overtemp shutdown in the bitstream
AFAIK theres no such magic button. If you want temp monitoring and thermal limits with an FPGA you have to include them in your design logic.
False. The better fpgas(like virtex 5/6, complete series 7 and newer xilinx) has pin terminals to an separated internal die temperature sensor transistor. A properly designed board should use it for temperature measurement and shutdown. It works even when FPGA isn't configured. An alternative is using jtag to read XADC (7 series and newer).
I actually went ahead and RTFM'd the "UltraScale Architecture SYStem MONitor User's Guide". It looks like it is very close to being a "magic button", it takes only one line of code:
set_property BITSTREAM.CONFIG.OVERTEMPSHUTDOWN ENABLE
On the other hand the whole guide to the SYSMON block has 113 pages, so it must be also relatively easy to accidentally mis-configure it.