Quote

From this, I assumed that one could compile binaries such that the source code / private key could not be re-derived (an assumption I came here to check with experts)

I will save you some time. That is a completely false assumption. Don't feel bad it has been made by people who should know better over and over throughout history. DRM is an example of a secret hidden in the software (or media) such that the user can't gain access to the secret even though they have access to the media or software. No form of DRM running on an open system has survived cryptoanlysis over an extended period of time. To date the reward for breaking DRM has been the ability to duplicate a game or movie, here you are talking about increasing the reward to be stealing money. I would put the over/under on timeframe to being broken wide open at a week (maybe a month if the implementation is particularly novel).

If you are talking about dedicated hardware devices the attack becomes more difficulty but not impossible. Private keys have been recovered from smart card chips and even FIPS rated hardware security modules. Once again if the system becomes widespread the reward could mean potentially millions of billions of stolen wealth so "difficult" is simply not good enough. Furthermore those systems wouldn't be provably secure and would be highly centralized.

If the user has access to the software and the software has the secret the user has (or will eventually gain) access to the secret. If that were not true the rate of software piracy would be approaching ~0% by now. People have been working on this "solution" for decades.

Since this is an X-Y problem it is very likely novel use of cryptography could be used to achieve the goals you state but not through the method you describe.