eth/filters: send rpctransactions in pending-subscription #26126

holiman · 2022-11-07T13:22:22Z

This PR is an alternative to #26034. This PR keeps the internal objects as *types.Transaction, and mutates it into rpc.Transaction right before it's sent on the wire.

At least I think it does, I haven't fully verified it (yet).

s1na · 2022-11-09T11:07:05Z

Example response:

{
  blockHash: null,
  blockNumber: null,
  from: '0x21a31ee1afc51d94c2efccaa2092ad1028285549',
  gas: '0x32918',
  gasPrice: '0x202170e400',
  maxFeePerGas: '0x202170e400',
  maxPriorityFeePerGas: '0x77359400',
  hash: '0xde8f7e26fd9dc0157c8dfb8d6742a50ee1d13077f7eb675a8b9de42374ad4e6d',
  input: '0xa9059cbb000000000000000000000000a055bdc23d5f8fa59214b18a4c892d2be87d2cc200000000000000000000000000000000000000000000000
0000000001b774000',
  nonce: '0x487572',
  to: '0xdac17f958d2ee523a2206206994597c13d831ec7',
  transactionIndex: null,
  value: '0x0',
  type: '0x2',
  accessList: [],
  chainId: '0x1',
  v: '0x1',
  r: '0x93a2c41a452a8ad03c7e31e12608650205f7b2ab8f73bef174fdb28a046ccaa8',
  s: '0x74c0b4b811807c1cb066e7082dcc9341acbeff68fcd2653c9e21e82c45e1720c'
}

Only thing is blockHash, blockNumber and transactionIndex will always be null for pending txes. But adding omitempty rules to RPCTransaction type will also change the result of getTransactionByHash for pending txes, so I guess it's best to keep as is.

s1na

LGTM

s1na · 2022-11-09T11:10:17Z

Ah gethclient's method might need to change.

Update: seems to work. I think because the JSON encoding of a rpc transaction is unmarshalable to types.Transaction.

holiman · 2022-11-09T15:49:44Z

Update: seems to work. I think because the JSON encoding of a rpc transaction is unmarshalable to types.Transaction.

Could you clarify this a bit? I don't understand if there's anything we need to address here

s1na · 2022-11-14T10:40:37Z

To clarify, IMO no change needed for gethclient. PR looks good to me!

@s1na

…6126) This PR changes the pending tx subscription to return RPCTransaction types instead of normal Transaction objects. This will fix the inconsistencies with other tx returning API methods (i.e. getTransactionByHash), and also fill in the sender value for the tx. co-authored by @s1na

@fjl

…h#42) * rpc: improve error codes for internal server errors (ethereum#25678) This changes the error code returned by the RPC server in certain situations: - handler panic: code -32603 - result marshaling error: code -32603 - attempt to subscribe via HTTP: code -32001 In all of the above cases, the server previously returned the default error code -32000. Co-authored-by: Nicholas Zhao <[email protected]> Co-authored-by: Felix Lange <[email protected]> * rpc: add PeerInfo (ethereum#24255) This replaces the sketchy and undocumented string context keys for HTTP requests with a defined interface. Using string keys with context is discouraged because they may clash with keys created by other packages. We added these keys to make connection metadata available in the signer, so this change also updates signer/core to use the new PeerInfo API. * graphql: add query timeout (ethereum#26116) This PR adds a 60 second timeout to graphql queries. * graphql, node, rpc: improve HTTP write timeout handling (ethereum#25457) Here we add special handling for sending an error response when the write timeout of the HTTP server is just about to expire. This is surprisingly difficult to get right, since is must be ensured that all output is fully flushed in time, which needs support from multiple levels of the RPC handler stack: The timeout response can't use chunked transfer-encoding because there is no way to write the final terminating chunk. net/http writes it when the topmost handler returns, but the timeout will already be over by the time that happens. We decided to disable chunked encoding by setting content-length explicitly. Gzip compression must also be disabled for timeout responses because we don't know the true content-length before compressing all output, i.e. compression would reintroduce chunked transfer-encoding. * eth/filters, eth/tracers: add request cancellation checks (ethereum#26320) This ensures that RPC method handlers will react to a timeout or cancelled request soon after the event occurs. Co-authored-by: Sina Mahmoodi <[email protected]> * rpc: add limit for batch request items and response size (ethereum#26681) This PR adds server-side limits for JSON-RPC batch requests. Before this change, batches were limited only by processing time. The server would pick calls from the batch and answer them until the response timeout occurred, then stop processing the remaining batch items. Here, we are adding two additional limits which can be configured: - the 'item limit': batches can have at most N items - the 'response size limit': batches can contain at most X response bytes These limits are optional in package rpc. In Geth, we set a default limit of 1000 items and 25MB response size. When a batch goes over the limit, an error response is returned to the client. However, doing this correctly isn't always possible. In JSON-RPC, only method calls with a valid `id` can be responded to. Since batches may also contain non-call messages or notifications, the best effort thing we can do to report an error with the batch itself is reporting the limit violation as an error for the first method call in the batch. If a batch is too large, but contains only notifications and responses, the error will be reported with a null `id`. The RPC client was also changed so it can deal with errors resulting from too large batches. An older client connected to the server code in this PR could get stuck until the request timeout occurred when the batch is too large. **Upgrading to a version of the RPC client containing this change is strongly recommended to avoid timeout issues.** For some weird reason, when writing the original client implementation, @fjl worked off of the assumption that responses could be distributed across batches arbitrarily. So for a batch request containing requests `[A B C]`, the server could respond with `[A B C]` but also with `[A B] [C]` or even `[A] [B] [C]` and it wouldn't make a difference to the client. So in the implementation of BatchCallContext, the client waited for all requests in the batch individually. If the server didn't respond to some of the requests in the batch, the client would eventually just time out (if a context was used). With the addition of batch limits into the server, we anticipate that people will hit this kind of error way more often. To handle this properly, the client now waits for a single response batch and expects it to contain all responses to the requests. --------- Co-authored-by: Felix Lange <[email protected]> Co-authored-by: Martin Holst Swende <[email protected]> * format * ethereum, ethclient: add FeeHistory support (ethereum#25403) Co-authored-by: Felix Lange <[email protected]> * internal/ethapi: return error when requesting invalid trie key (ethereum#25762) This change makes eth_getProof and eth_getStorageAt return an error when the argument contains invalid hex in storage keys. Co-authored-by: Felix Lange <[email protected]> * internal/ethapi: handle odd length hex in decodeHash (ethereum#25883) This change adds zero-padding (prefix) of odd nibbles in the decodeHash function. Co-authored-by: ty <[email protected]> * eth/filters, ethclient/gethclient: add fullTx option to pending tx filter (ethereum#25186) This PR adds a way to subscribe to the _full_ pending transactions, as opposed to just being notified about hashes. In use cases where client subscribes to newPendingTransactions and gets txhashes only to then request the actual transaction, the caller can now shortcut that flow and obtain the transactions directly. Co-authored-by: Felix Lange <[email protected]> * graphql: check header first in blocks query (ethereum#24190) Fixes ethereum#24167 New behaviour is that the endpoint returns results only for available blocks without returning an error when it doesn't find a block. Note we skip any block after a non-existent block. This adds a header fetch for every block in range (even if header is not needed). Alternatively, we could do the check in every field's resolver method to avoid this overhead. * graphql: embed *Resolver instead of backend interface (ethereum#25468) This creates some infrastructure to share resources between graphql API objects. * eth/filters: fix getLogs for pending block (ethereum#24949) * eth/filters: fix pending for getLogs * add pending method to test backend * fix block range validation * eth/filters: add global block logs cache (ethereum#25459) This adds a cache for block logs which is shared by all filters. The cache size of is configurable using the `--cache.blocklogs` flag. Co-authored-by: Felix Lange <[email protected]> * eth/filters: send rpctransactions in pending-subscription (ethereum#26126) This PR changes the pending tx subscription to return RPCTransaction types instead of normal Transaction objects. This will fix the inconsistencies with other tx returning API methods (i.e. getTransactionByHash), and also fill in the sender value for the tx. co-authored by @s1na * rpc: fix unmarshaling of null result in CallContext (ethereum#26723) The change fixes unmarshaling of JSON null results into json.RawMessage. --------- Co-authored-by: Jason Yuan <[email protected]> Co-authored-by: Jason Yuan <[email protected]> * eth/filters: fix a breaking change and return rpctransaction (ethereum#26757) * eth/filters: fix a breaking change and return rpctransaction * eth/filters: fix test cases --------- Co-authored-by: Catror <[email protected]> --------- Co-authored-by: Nicholas <[email protected]> Co-authored-by: Nicholas Zhao <[email protected]> Co-authored-by: Felix Lange <[email protected]> Co-authored-by: Ahmet Avci <[email protected]> Co-authored-by: Sina Mahmoodi <[email protected]> Co-authored-by: Sina Mahmoodi <[email protected]> Co-authored-by: mmsqe <[email protected]> Co-authored-by: Martin Holst Swende <[email protected]> Co-authored-by: lightclient <[email protected]> Co-authored-by: TY <[email protected]> Co-authored-by: ty <[email protected]> Co-authored-by: lmittmann <[email protected]> Co-authored-by: Jason Yuan <[email protected]> Co-authored-by: Jason Yuan <[email protected]> Co-authored-by: Yier <[email protected]> Co-authored-by: Catror <[email protected]>

eh/filters: send rpctransactions in pending-subscription

ac75a6f

holiman requested review from karalabe, rjl493456442, gballet and MariusVanDerWijden as code owners November 7, 2022 13:22

MariusVanDerWijden changed the title ~~eh/filters: send rpctransactions in pending-subscription~~ eth/filters: send rpctransactions in pending-subscription Nov 7, 2022

eth/filters: fix tests

1e79798

s1na mentioned this pull request Nov 9, 2022

eth/filters: return sender in pending tx subscription #26034

Closed

s1na approved these changes Nov 9, 2022

View reviewed changes

holiman added this to the 1.11.0 milestone Nov 14, 2022

holiman linked an issue Nov 14, 2022 that may be closed by this pull request

Return sender (and chainId) on 'eth_subscribe', ['newPendingTransactions', true] #26030

Closed

holiman merged commit 8c5ce11 into ethereum:master Nov 14, 2022

yierx mentioned this pull request Feb 22, 2023

eth/filters: fix a breaking change and return rpctransaction #26757

Merged

weiihann mentioned this pull request Jul 31, 2023

Full tx subscription missing 'from' field into transaction details bnb-chain/bsc#1783

Closed

s1na mentioned this pull request Jul 31, 2023

eth/filters, ethclient/gethclient: add fullTx option to pending tx filter #25186

Merged

This was referenced Feb 7, 2024

eth/filter, ethclient/gethclient: added fullTx-flag to NewPendingTransactions celo-org/celo-blockchain#2239

Merged

eh/filters: send rpctransactions in pending-subscription celo-org/celo-blockchain#2252

Merged

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 9, 2025

eth/filters: update Backend interface (ethereum#26126)

5ff5869

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 9, 2025

eth/filters: update Backend interface (ethereum#26126)

c7704f9

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 9, 2025

eth/filters: update Backend interface (ethereum#26126)

f7aabe5

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 10, 2025

eth/filters: update Backend interface (ethereum#26126)

5a2eb41

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 10, 2025

eth/filters: update Backend interface (ethereum#26126)

4cd43f0

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 10, 2025

eth/filters: update Backend interface (ethereum#26126)

ae85f20

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 10, 2025

eth/filters: update Backend interface (ethereum#26126)

dd867b8

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 11, 2025

eth/filters: update Backend interface (ethereum#26126)

c8a6c7f

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 14, 2025

eth/filters: update Backend interface (ethereum#26126)

c893804

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 16, 2025

eth/filters: update Backend interface (ethereum#26126)

5f3078e

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 17, 2025

eth/filters: update Backend interface (ethereum#26126)

bf38b4f

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 22, 2025

eth/filters: update Backend interface (ethereum#26126)

a4fd61d

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 22, 2025

eth/filters: update Backend interface (ethereum#26126)

e383344

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 23, 2025

eth/filters: update Backend interface (ethereum#26126)

174e5cb

gzliudan added a commit to gzliudan/XDPoSChain that referenced this pull request Jan 24, 2025

eth/filters: update Backend interface (ethereum#26126)

cab2938

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

eth/filters: send rpctransactions in pending-subscription #26126

eth/filters: send rpctransactions in pending-subscription #26126

Uh oh!

holiman commented Nov 7, 2022

Uh oh!

s1na commented Nov 9, 2022

Uh oh!

s1na left a comment

Uh oh!

s1na commented Nov 9, 2022 •

edited

Loading

Uh oh!

holiman commented Nov 9, 2022

Uh oh!

s1na commented Nov 14, 2022

Uh oh!

Uh oh!

eth/filters: send rpctransactions in pending-subscription #26126

eth/filters: send rpctransactions in pending-subscription #26126

Uh oh!

Conversation

holiman commented Nov 7, 2022

Uh oh!

s1na commented Nov 9, 2022

Uh oh!

s1na left a comment

Choose a reason for hiding this comment

Uh oh!

s1na commented Nov 9, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

holiman commented Nov 9, 2022

Uh oh!

s1na commented Nov 14, 2022

Uh oh!

Uh oh!

s1na commented Nov 9, 2022 •

edited

Loading