eth_getTransactionReceipt returning reorged block data instead of the canonical block's data

[Enigmatic331]

System information

Geth version: geth-windows-amd64-1.13.14-2bd6bd01
CL client & version: Prysm v5.0.0
OS & Version: Windows

We were indexing blocks and having issues indexing this receipt: 0xddf1c0d3366ec51b0d8a75ef3e0c5c22c8c4c34b3ddc914776a34a0c4248d359

Expected behaviour

eth_getTransactionReceipt to return the correct block details as below:

"jsonrpc": "2.0",
"id": 1,
"result": {
	"blobGasPrice": "0x1",
	"blockHash": "0x3fb5bb2b9efa2f44b91de93c23313ede0bf1e9debe893854646f6350498b6d47",
	"blockNumber": "0x1288297",
	"contractAddress": null,
	"cumulativeGasUsed": "0x174c0e0",
	"effectiveGasPrice": "0xa111dcbb3",
	"from": "0xa108edf3c6ce15a9674d65be73338116af6e0f06",
	"gasUsed": "0x5208",
	"logs": [],
	"logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
	"status": "0x1",
	"to": "0xb20aac3ed8fa7a7fbae81b622b1b554cc58b8b1d",
	"transactionHash": "0xddf1c0d3366ec51b0d8a75ef3e0c5c22c8c4c34b3ddc914776a34a0c4248d359",
	"transactionIndex": "0xd7",
	"type": "0x2"
}

Actual behaviour

Curl calling eth_getTransactionReceipt returns the below details - Which this block is a reorged block (the correct blockHash for block 0x1288296 is 0x3fb5bb2b9efa2f44b91de93c23313ede0bf1e9debe893854646f6350498b6d47)

"jsonrpc": "2.0",
"id": 1,
"result": {
	"blockHash": "0x19fb25e3d5ea1c8bff76c5d4e84c764d5d5d150e43566f68dbd3fc4a92b47a41",
	"blockNumber": "0x1288296",
	"contractAddress": null,
	"cumulativeGasUsed": "0x305ba6",
	"effectiveGasPrice": "0xa7620bcc9",
	"from": "0xa108edf3c6ce15a9674d65be73338116af6e0f06",
	"gasUsed": "0x5208",
	"logs": [],
	"logsBloom": "0x00000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000",
	"status": "0x1",
	"to": "0xb20aac3ed8fa7a7fbae81b622b1b554cc58b8b1d",
	"transactionHash": "0xddf1c0d3366ec51b0d8a75ef3e0c5c22c8c4c34b3ddc914776a34a0c4248d359",
	"transactionIndex": "0x51",
	"type": "0x2"
}

Restarting the node will allow us to correctly query 0xddf1c0d3366ec51b0d8a75ef3e0c5c22c8c4c34b3ddc914776a34a0c4248d359 from the canonical block, however does trip our reorg checks. Any idea what might be the case?

Am aware of another issue prior to this #28885 which should have resolve this issue, but seems to still be happening for us.

[Enigmatic331]

Hmm seems to be affecting our nodes twice over the past week - Any idea what we might be able to do for this? 🙏