TL;DR: Integrating HSM to sign Ethereum transactions but getting "Invalid Sender" errors despite correct address recovery and signature application. Need help troubleshooting the issue.
I am trying to implement wallet generation and signing using an HSM. I'm currently developing against SoftHSM2, with C#.

_________________________________________________

I'm running into troubles with "Invalid Sender"

Hey Ethereum Devs,

I'm currently working on integrating a Hardware Security Module (HSM) to sign Ethereum transactions, and I've run into a persistent "Invalid Sender" error that I can't seem to resolve. I’m hoping the community might have some insights or suggestions.

Background:

I'm using an HSM to securely generate and store Ethereum keys, and then to sign transactions. The process involves retrieving the public and private keys from the HSM, creating a transaction, signing it, and then sending it to the Ethereum network.

Here’s a summary of the workflow:

1. Retrieve Public Key Info: I retrieve the public key and its associated label from the HSM based on the Ethereum address.
2. Generate Transaction Hash: I use the Sha3Keccack hash function to create a message hash from the public key.
3. Initial Signature: I generate an initial signature with the retrieved private key using the message hash.
4. Create Transaction: I build a LegacyTransaction object with the transaction parameters like nonce, gas price, gas limit, etc.
5. Recompute Transaction Hash: The transaction hash is recomputed, including the Chain ID to ensure EIP-155 compliance.
6. Final Signature: I generate the final signature with the recomputed transaction hash.
7. Adjust V for EIP-155: The V value is adjusted as per EIP-155 (i.e., V = finalSignature.V + Chain ID * 2 + 35).
8. Apply Final Signature: I set the final signature on the transaction and serialize it.
9. Send Transaction: The serialized transaction is sent to the Ethereum network.

The Problem:

Despite the transaction being created and signed correctly (as far as I can tell), when I submit it, I consistently get an "Invalid Sender" error.

Debugging Data:

Here’s some of the debug output:

• Public Key Info: Address=0xd2de6f19109d613f17496837e03909ad26632081, Label=test
• Message Hash: dd5e1cc957e9cf152b8e6d7ef12160622a31b329cad932d57998e9334ef0495a
• Initial Signature: R=0EE8B7B9FE8DAC6267ABADAA5F339D234A4B3CB4BAE4FC3BC31B619A8C726946F, S=641FEBC235A4EECA13597671AC5B1A32314D4DD07ABD84F074A93AAC674F77C2, V=28
• Recovered Address v27: 0x0D9D930AC4Af2EdB21369c0A362F8c813264B85b
• Recovered Address v28: 0xD2DE6F19109D613f17496837e03909aD26632081 (This is the correct one)
• Transaction Hash: 2f0028f56b6e1550d7a34e5da3048ef4ecda121004c38135f276ff7293a6e0db
• Final Signature: R=76AB822FD1B3D61C0C261B5FEEE33014F95EB658B93412FB8E2C8222C79CB91F, S=0811F55B97C7D97253CFD39B76B3A244C612BBEBB13BA76BD2160AFE6C6853FF9, V=141

Despite the correct address being recovered with V=28, the transaction still results in an "Invalid Sender" error when the signed transaction is submitted.

Code Snippet:

Here’s the critical part of the code where I believe the issue might be:

csharpCopy codevar vValue = (byte)(finalSignature.V + _chainId * 2 + 35); transaction.SetSignature(new Signature { R = finalSignature.R, S = finalSignature.S, V = new byte[] { vValue } }); var serializedTx = transaction.GetRLPEncoded().ToHex(); 

What I’ve Tried:

• Checked V Calculation: Confirmed that the V value is calculated as per EIP-155.
• Address Recovery: Verified that the correct address is recovered with V=28.
• Serialized Transaction: Compared the serialized transaction output with a known good transaction; it looks correct.

Community Help:

I'm stumped at this point. Does anyone have experience with HSMs and Ethereum transactions? Could it be an issue with the way I’m applying the signature or something subtle in the transaction serialization?

Any advice, suggestions, or pointers would be greatly appreciated!

Thanks in advance for your help! ????

Code to understand what i'm trying to do:

public SignedTransactionResult GenerateAndSignTransaction(TransactionParamsResult txParams) { Console.WriteLine("Starting transaction generation and signing..."); var publicKeyInfo = FindPublicKeyLabel(txParams.EthereumAddress); if (publicKeyInfo == null) throw new Exception("Public key not found"); Console.WriteLine($"Public key info: Address={publicKeyInfo.Address}, Label={publicKeyInfo.Label}, PublicKey={publicKeyInfo.PublicKey}"); var privateKey = FindPrivateKeyByLabelAndPublicKey(publicKeyInfo.Label, publicKeyInfo.PublicKey); if (privateKey == null) throw new Exception("Private key not found"); Console.WriteLine("Private key found."); // Parse parameters var gasPrice = BigInteger.Parse(txParams.GasPrice, NumberStyles.HexNumber); var nonce = BigInteger.Parse(txParams.Nonce, NumberStyles.HexNumber); var gasLimit = BigInteger.Parse(txParams.GasLimit, NumberStyles.HexNumber); var toAddress = txParams.ToAddress; var value = BigInteger.Parse(txParams.Value, NumberStyles.HexNumber); var data = "0x00"; // Generate the transaction hash Sha3Keccack shaKec = new Sha3Keccack(); var msgHash = shaKec.CalculateHashFromHex(publicKeyInfo.Address).HexToByteArray(); Console.WriteLine($"Message Hash: {BitConverter.ToString(msgHash).Replace("-", "").ToLower()}"); var addressSign = CalculateEthereumSig(msgHash, publicKeyInfo.Address, privateKey); Console.WriteLine($"Initial Signature: R={BitConverter.ToString(addressSign.R).Replace("-", "").ToLower()}, S={BitConverter.ToString(addressSign.S).Replace("-", "").ToLower()}, V={addressSign.V}"); // Create the transaction var transaction = new LegacyTransaction( nonce: nonce.ToBytesForRLPEncoding(), gasPrice: gasPrice.ToBytesForRLPEncoding(), gasLimit: gasLimit.ToBytesForRLPEncoding(), receiveAddress: toAddress.HexToByteArray(), value: value.ToBytesForRLPEncoding(), data: data.HexToByteArray() ); // Recompute the transaction hash including the chain ID for EIP-155 var transactionHash = transaction.RawHash; Console.WriteLine($"Transaction Hash: {BitConverter.ToString(transactionHash).Replace("-", "").ToLower()}"); // Calculate the final signature with the correct chain ID var finalSignature = CalculateEthereumSig(transactionHash, publicKeyInfo.Address, privateKey); // Adjust the V value for EIP-155 compliance var vValue = (byte)(finalSignature.V + _chainId * 2 + 35); Console.WriteLine($"Adjusted V value for EIP-155: {vValue}"); // Apply the final signature transaction.SetSignature(new Signature { R = finalSignature.R, S = finalSignature.S, V = new byte[] { vValue } }); // Serialize the transaction var serializedTx = transaction.GetRLPEncoded().ToHex(); Console.WriteLine($"Signed Transaction: {serializedTx}"); // Additional debug to show the raw transaction for comparison Console.WriteLine($"Raw Transaction R: {BitConverter.ToString(finalSignature.R).Replace("-", "").ToLower()}"); Console.WriteLine($"Raw Transaction S: {BitConverter.ToString(finalSignature.S).Replace("-", "").ToLower()}"); Console.WriteLine($"Raw Transaction V: {vValue}"); return new SignedTransactionResult { SignedTransaction = serializedTx }; } 

___________________________________________________________

Updates:

I've made some progress.

The NEthereum has the LegacyTransactionChainId Class, which should be used for EIP-155.

I am finally able to "sometimes" post transactions, and sometimes I get Invalid Signature.

Invalid Signature Example:

Starting transaction generation and signing... Public key info: Address=0xd2de6f19109d613f17496837e03909ad26632081, Label=test, PublicKey=f819cad6f43c2fe7b66dcb423b6752fbf6ffb48da6b61e95ede2b1508b8bd8f9539bc3caefc5547e0bdde4eea3b04420f95abd521e4b346a48ed820a917f2410 Private key found. Using Chain ID: 11155111 Message Hash: dd5e1cc957e9cf152b8e6d7ef12160622a31b329cad932d57998e9334ef0495a Initial Signature: R=2D6CBA37989CB4BDF25A41450B593F0D949B50062907ACA4087AF874C6E749FD, S=0AD112520558FC55660D790530B867A8B070DC094468F475EE2817B1D63EB5AF4 Adjusted S value: S=F52EEDADFAA703AA99F286FACF4798573B3A11C5268B958DCDD50E36F6C4AE64D Recovered Address v27: 0xb69609fBD66f8275f78841B9d19Bda3375364C9E Recovered Address v28: 0xD2DE6F19109D613f17496837e03909aD26632081 Initial Signature: R=2d6cba37989cb4bdf25a41450b593f0d949b50062907aca4087af874c6e749fd, S=52eedadfaa703aa99f286facf4798573b3a11c5268b958dcdd50e36f6c4ae64d, V=28 Transaction Hash: 1741bdb96d557830c8345eca5afa422395be144ba92b397a9ef3dea9782536d7 Initial Signature: R=7B57382CE010A9205217ABA1C961C02E8EA3D0067B5E341913B36FFCAF1597D8, S=7FAF8396C0BD6944EA12A08AB712815C4E8EB58BE4FD2198B6767E742C8293D3 Adjusted S value: S=80507C693F4296BB15ED5F7548ED7EA26C20275ACA4B7EA3095BE018A3B3AD6E Recovered Address v27: 0x34f31345508aAEB2916Ae086cc6ee2Dd1319A129 Recovered Address v28: 0xD2DE6F19109D613f17496837e03909aD26632081 Signed Transaction: f86e28850211959c1a83027100942f8181abc608ba4c509be2f8b2befe47490786f5850ba43b7400008401546d72a07b57382ce010a9205217aba1c961c02e8ea3d0067b5e341913b36ffcaf1597d8a080507c693f4296bb15ed5f7548ed7ea26c20275aca4b7ea3095be018a3b3ad6e ---------------------------------------------- Decoded Nonce: 40 Decoded Gas Price: 8884952090 Decoded Gas Limit: 160000 Decoded To Address: 2f8181abc608ba4c509be2f8b2befe47490786f5 Decoded Value: 50000000000 Decoded Input Data: 00 Decoded Chain ID: 11155111 Decoded R: 7b57382ce010a9205217aba1c961c02e8ea3d0067b5e341913b36ffcaf1597d8 Decoded S: 80507c693f4296bb15ed5f7548ed7ea26c20275aca4b7ea3095be018a3b3ad6e Decoded V: 22310258 

Decoded Trxn using the Final Signed Transaction as was submitted:

{ "chainId": "11155111", "type": "LegacyTransaction", "valid": false, "hash": "0x6091364b6b907cf244cd9f1e2455841a908abd11da722537d162029926e5c650", "nonce": "40", "gasPrice": "8884952090", "gasLimit": "160000", "from": "0xD2DE6F19109D613f17496837e03909aD26632081", "to": "0x2f8181abc608ba4c509be2f8b2befe47490786f5", "v": "01546d72", "r": "7b57382ce010a9205217aba1c961c02e8ea3d0067b5e341913b36ffcaf1597d8", "s": "80507c693f4296bb15ed5f7548ed7ea26c20275aca4b7ea3095be018a3b3ad6e", "value": "50000000000", "input": "00" } 

Valid Transaction:

https://sepolia.etherscan.io/tx/0x6db694ad4ab7eeaf7e54a1e17f88945d1bc5d8ef09c337a3f03b5a5475bba9b4

Starting transaction generation and signing... Public key info: Address=0xd2de6f19109d613f17496837e03909ad26632081, Label=test, PublicKey=f819cad6f43c2fe7b66dcb423b6752fbf6ffb48da6b61e95ede2b1508b8bd8f9539bc3caefc5547e0bdde4eea3b04420f95abd521e4b346a48ed820a917f2410 Private key found. Using Chain ID: 11155111 Message Hash: dd5e1cc957e9cf152b8e6d7ef12160622a31b329cad932d57998e9334ef0495a Initial Signature: R=14CE94E9116413B7D28ED476BEC58078EB294A91E1332C8F7A53F03FEF911428, S=14220D78958D2E6CDC8578C6CB330938965F41D33042F7B62D9E93B2DF776AAD Adjusted S value: S=EBDDF2876A72D193237A873934CCF6C6244F9B137F05A8859233CAD9F0BED694 Recovered Address v27: 0x5729F451eeEeA7343a87bD48a73a09D10908A644 Recovered Address v28: 0xD2DE6F19109D613f17496837e03909aD26632081 Initial Signature: R=14ce94e9116413b7d28ed476bec58078eb294a91e1332c8f7a53f03fef911428, S=ebddf2876a72d193237a873934ccf6c6244f9b137f05a8859233cad9f0bed694, V=28 Transaction Hash: 76855cac14bab893ca78f46187e45c2c0575070759a4f0031612370c100f4e6a Initial Signature: R=2149718AD38A78EDE26F83D99FB562D3EB9C95D1307CB006890127AAEDD36A72, S=0C77B3339D18CA54530B719B7C132AFDFE4A42C8C7D490C95C07803502351491C Adjusted S value: S=F3884CCC62E735ABACF48E6483ECD501ED60AB05A31FF93A5FF5A5B3CACE4F825 Recovered Address v27: 0xD2DE6F19109D613f17496837e03909aD26632081 Signed Transaction: f86e2885022cfeec9f83027100942f8181abc608ba4c509be2f8b2befe47490786f5850ba43b7400008401546d71a02149718ad38a78ede26f83d99fb562d3eb9c95d1307cb006890127aaedd36a72a03884ccc62e735abacf48e6483ecd501ed60ab05a31ff93a5ff5a5b3cace4f825 ---------------------------------------------- Decoded Nonce: 40 Decoded Gas Price: 9344838815 Decoded Gas Limit: 160000 Decoded To Address: 2f8181abc608ba4c509be2f8b2befe47490786f5 Decoded Value: 50000000000 Decoded Input Data: 00 Decoded Chain ID: 11155111 Decoded R: 2149718ad38a78ede26f83d99fb562d3eb9c95d1307cb006890127aaedd36a72 Decoded S: 3884ccc62e735abacf48e6483ecd501ed60ab05a31ff93a5ff5a5b3cace4f825 Decoded V: 22310257 

Likewise: Decoded Transaction based on the Final Signed Transaction as was submitted

{ "chainId": "11155111", "type": "LegacyTransaction", "valid": true, "hash": "0x6db694ad4ab7eeaf7e54a1e17f88945d1bc5d8ef09c337a3f03b5a5475bba9b4", "nonce": "40", "gasPrice": "9344838815", "gasLimit": "160000", "from": "0xD2DE6F19109D613f17496837e03909aD26632081", "to": "0x2f8181abc608ba4c509be2f8b2befe47490786f5", "v": "01546d71", "r": "2149718ad38a78ede26f83d99fb562d3eb9c95d1307cb006890127aaedd36a72", "s": "3884ccc62e735abacf48e6483ecd501ed60ab05a31ff93a5ff5a5b3cace4f825", "value": "50000000000", "input": "00" } 

What's the difference?

Why does it fail in one case, and successfull transaction in the other?

I see no discrepancies beween the from address, which in fact are identical.