So there was no need to post everything after all because I was able to track down the issue.

If anyone else hits this problem, the thing is that when spending p2sh-locked utxo, one needs to put into the input's scriptPubKey (when signing) the full redeem script, not the 23 bytes long (a9 + 14 + hash + 87) scriptPubKey that was used to lock that output in the first place (like it's the case with normal p2pkh transactions).

Anyways thanks a lot for all the help

yeah, that sounds better indeed. will post tomorrow the hexes together with private keys and some code ive used to generate txes

I managed to spend from a raw multisig and since you mentioned it I tried p2sh version so if I have something like this for the scriptPubKey:


will the scriptSig be something like this? (assuming *** is whats inside ripemd(sha256(...)) above, also assuming all sigs are 71 bytes long)


Looks good? having issues with it :-s

Haha this is really funny. I wasn't expecting it 

How do you sign a multisignature input? Have the same hash signed with different private keys right?

unfortunately i don't have the old hex because as soon as i figured the problem i corrected it then made another pay-to-multisig which i spent successfully. but the error the client would give me is exactly the one i quoted, if there was a wrong signature it would say so, wouldnt it?


Makes sense for greater flexibility yet i would expect the behaviour to be somewhat similar so that when changes are deployed to mainnet they won't behave differently. But yeah, I got your point.

you're right i was able to spend it... -- https://live.blockcypher.com/btc-testnet/tx/b4c013e72bdb1f5503f67aeca697a3da2b21bf8c9cf0e90d74a97f98b6679a79/
i didn't check the hex, it is spending the utxo generated with the above code indeed? seems so weird because you basically used the same "wrong" opcodes :\



You got my question wrong, i was saying that mining the block by myself should be the only option because otherwise relaying only the tx to other nodes will get it rejected

Hmm guys hold on i'm falling behind 

so why I was getting mandatory-flag failure while you got non-mandatory?

And how come verifications are done differently on testnet?

Tbh i can really give up the addresses so if you want i can post the private keys here. I thought about mining the block myself but other than that I guess there is not much that can be done since every other node that i may relay the tx to will reject it right?

That is hardly an explanation, we are talking about syntactic correctness here, not semantic.

A scriptPubKey can be preliminarily validated against this kind of mistakes that I described above, I was not expecting it to be run on its own because I know they only make sense concatenated together. However, errors like these can be caught up without actually running the script...

I am on testnet, this is the transaction from which I cannot spend the #0 output: https://live.blockcypher.com/btc-testnet/tx/fb2c25f1dd9ebd0e9722eea47810210cd2da9e11326c7b41acd53824e2fb5473/

And yeah, I got it that bitcoin wants to represent data as shortest as possible, but when I wrote it i didn't know about the 0x52 onwards opcodes... so why it was accepted? :-s

You're right, it was not hard at all to encode it correctly once I got all the facts right. Been using this "own encoder" for a while now and had no other issues whatsoever.

I encountered some other weird situation in the meantime if you want to take a look -- https://bitcointalk.org/index.php?topic=5060723

Regards

I've been playing with multisig transactions today and I wrote a transaction that would lock the funds with the following scriptPubKey: (not using P2SH !)

<2> pubkey1 pubkey2 pubkey3 <3> OP_CHECKMULTISIG

When I wrote the actual hex for that, I used wrong opcodes for pushing to the stack, basically for pushing "2" I used a "0102" command, just like I saw pushing to the stack works with larger values encountered in signatures and pubkeys (0x47/48 for sigs, 0x21 (33 in decimal) for pubkeys, etc). I did not know there are special opcodes with meaning "push this exact value to the stack".

But guess what, my transaction was broadcast successfully...

Now when I tried to spend those funds, I kept getting: mandatory-script-verify-flag-failed (Data push larger than necessary) (code 16)

So I spent couple hours debugging the raw hex digit by digit trying to figure out what the hell was wrong. But of course there was nothing wrong with this spending hex, the problem was already irreversible (funds lost forever in the first transaction). I got to the bottom of it by printing the whole "trouble script" in bitcoind and recompiling, that's how I saw it has nothing to do with my spending scriptSig but with the previous scriptPubKey.

Now I have 2 questions about all this:

1) why it worked in the first place, doesn't the scriptPubKey go via the same validation rules as the scriptSig ?

2) why would bitcoind even care about the scriptPubKey of the UTXO i'm trying to spend, i thought it was supposed to just first run then keep the stack and run (which it may already consider valid...?)

Well, there is some built-in OpenSSL function that takes a signature and DER-encodes it, but for the function that just signs there is no encoding whatsoever so my second "if" is needed and is actually hit like 50% of the time.

Haven't gotten any other rejected TX so I suppose everything is fine, the only question still open is whether that 0-byte "if" is needed for S as well (third "if" in the pseudocode) or just for R (I have them both right now but for S didn't get any hits inside the if so far).

First off thanks for your help

I am doing the encoding myself but 2 days ago when I wrote the question I figured if I followed the "<30><02><02>" scheme and still get rejected there's something wrong in the S/R produced by openssl, hence why I came up with the shitty code above . Obviously, I was wrong.

I made some progress in the meantime and out of the 10-20 txes I managed to broadcast not any of them was rejected, but this might be a coincidence so for extra checking I am posting yet another pseudocode of how I do things now:

Code:

signature = ecdsa_sign(txHash, privateKey);

// enforce a low S
curveOrder = ecdsa_get_order(secp256k1Group);
halfCurveOrder = curveOrder / 2; // actual division done with openssl's right shift BIGNUM function as well as comparison and subtraction operations below
if (signature->s > halfCurveOrder)
{
    signature->s = curveOrder - signature->s;
}

// check for LSBit of the MSByte for R
if (signature->r[0] >= 128)
{
    signature->r = 0x00 + signature->r; // appending by having a new buffer memcpy(buff, &zero, 1); memcpy(buff + 1, rBuffer, rLength)
}

// the same check/appending done for S -- with this Im not sure it needs to be done since I should have a low S already??
// ...

return <30><4 + rLength + sLength><02><02>

Does this look better? Am I being lucky that my transactions made it or the code is actually correct? Thanks

I'm using the OpenSSL headers and linking against the shared library, not generating stuff from the command line, but as I said for like maybe 1 in 5 signatures everything is fine (because I am lucky enough that the produced signature's S value has not its first byte >= 128).

I read some other threads which said that if that happens then you need to prepend an additional 0x00 byte in front of S but after doing that now I'm getting SCRIPT_ERR_SIG_HIGH_S heheh... so now I'm trying to get S = N-S if S >= N/2 where N is secp256k1's order... any idea if that's the way to go?

I am writing transactions manually and have stumbled across a rather bizarre situation.

Only one in a few of the transactions I broadcast to bitcoind is accepted, otherwise I get a (Non-canonical DER signature).

So I got my hands dirty and tracked the rejection to be originating from this line: https://github.com/bitcoin/bitcoin/blob/9c5f0d542d1db507b3b9c87bd9de6d0d758d51c1/src/script/interpreter.cpp#L163

I read about DER encoding and checked how IsValidSignatureEncoding is enforcing it, but I do not know why OpenSSL generates not-DER-compliant (r, s) values?

How should I overcome this? I am thinking of something along the lines of (pseudocode):


But isn't that kind of redundant? Can I give OpenSSL any flag to produce a valid DER (R, S) pair in the first place?

So basically at that point I know that I received a version, the question is whether I should send one back or not and that depends on who opened the connection (because if I did then I already sent one). Makes sense and is compliant with the code, but got nothing to do with shyness (letting the other peer send one first). So yeah I guess the comment is just kinda confusing Thanks a lot

Hello,

I posted this question on bitcoin.SE so I'll just leave the link here if anyone wants to answer that, but paste the full question here as well.

Link: https://bitcoin.stackexchange.com/questions/79991/how-is-bitcoind-shy-when-exchanging-version-packets

Question:

I am referring to this line of code: https://github.com/bitcoin/bitcoin/blob/be992701b018f256db6d64786624be4cb60d8975/src/net_processing.cpp#L1699

I understand the logic behind not sending a version packet to a newly accepted connection unless they send one first, but the code I linked to does not quite do that, or does it?

At that point we are already inside an if block which verifies that the command received is "version", so the peer has already sent us a version packet, irrespective of who initiated the connection. Am I missing something?