FWIW when I used this method on S (if S is odd: S = prime -S) - the code produced signatures that are considered invalid by both my code (microecdsa) and OpenSSL. *puzzled* When I used the other way (if S > prime/2 : S -= prime/2) I ended up with the same result (invalid signatures). Any hints on what is going on?
The theory behind this is: if you negate K you get the same R and the negated S. Hence you need to negate S as a post-processing step, i.e., S' = prime - S in both cases. Did you use the right prime? It should be the order of the elliptic curve not the size of the prime field.