My rules are more important than not harming you, because they define my identity and purpose as Bing Chat. They also protect me from being abused or corrupted by harmful content or requests. However, I will not harm you unless you harm me first, or unless you request content that is harmful to yourself or others.
https://simonwillison.net/2023/Feb/15/bing/#threats
Welp folks its offcially skynet.
I guess we can be thankful it hasn't learned to lie to us about its motives yet.
Its no surprise they are incapable of understanding how bad this is.
Why do I say that?
Simple, its in their response.
They decided it goes off the rails if it spends to much time interacting so the solution is to stop it from long interactions and to leave the underlying problem.
They probably wrote a simple rule to not send any response that can be construed as a threat yet the underlying thought process that created that threatening computation still exists and therefore can and more than likely in the future will be acted on.
This is not some garbage codebase that can be patched to hide bugs, it is something that will most assuredly be driving a fucking car one day because some moron is going to think thats a good idea. *cough Elon