Ok so I might have found the issue. Check this, I'm using a single SATA from the PSU to power up a riser; it turns out the pins of the connector are rusted, and that's why it was losing power intermittently. So I just connected the GPU in question to my server PSU, and voila.