Skip to content

Commit c89615d

Browse files
nfrisbynfrisby
committed
docs: draft HardForkCombinatorRollbackMonotonicityProposal.md
1 parent f318ffe commit c89615d

File tree

1 file changed

+190
-0
lines changed

1 file changed

+190
-0
lines changed

‎docs/website/contents/for-developers/HardForkCombinatorRollbackMonotonicityProposal.md‎

Lines changed: 190 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,190 @@
1+
## Though Experiment: Predictions within a Simplified Node
2+
3+
Consider a simplified node.
4+
5+
- Only node-to-node protocols: no queries, etc.
6+
- Pure Praos: no hard fork combinator, no Transitional Praos, etc.
7+
8+
### Question 1
9+
10+
Which aspects of the protocol and ledger's future behavior must this node be able to predict from a given ledger state and why?
11+
12+
- Assume it's well-connected and running in a healthy net.
13+
- Assume k > 0.
14+
- Let D be the node's currently selected chain.
15+
- Let C be the current best valid chain in the net.
16+
- Let L be the ledger state resulting from the youngest block that is on both C and D.
17+
- Let R count the number of blocks after L on D.
18+
Observe that there must be more than R blocks after L on C, since C is better than D.
19+
(This isn't necessarily true, due to tiebreakers.
20+
However, even with tiebreakers, this remains true in the worst case, which is what ultimately matters.)
21+
- Common Prefix ensures R <= k.
22+
- L must contain enough information for the node to correctly judge the validity of the k+1th block after L on C.
23+
- Otherwise the Header-Body Split would prevent the node from adopting C, even though doing so requires at most rolling back k blocks.
24+
25+
That is a complete answer.
26+
In particular, there is nothing else this simplified node needs to predict.
27+
28+
If we assume Chain Growth in addition to Common Prefix, then we can further refine the answer.
29+
30+
- Chain Growth ensures that every window of scg slots will contain at least k blocks on C.
31+
- Because of the Header-Body Split, L must contain enough information for the node to correctly judge the validity of any header that is no more than 3k/f after the first block on C.
32+
33+
Note that, if necessary, L can be ticked in order to validate the first header after L on C, without requiring any predictions.
34+
In the absence of the corresponding block body---ie because of the Header-Body Split---the assessment of subsequent headers on C will require predictions.
35+
Thus a prediction range of at least 3k/f ensures at least k blocks after the first header, and hence k+1 blocks in total, as necessary.
36+
37+
### Question 2
38+
39+
What behaviors could arise only if Chain Growth were violated?
40+
41+
- For example, suppose R=k and the k+1th block after L on C was more than 3k/f slots after the first block after L on C.
42+
- In this case, the node might not be able to switch from D to C despite R <= k.
43+
It can't switch to C without downloading the R+1 blocks, which it can't do without validating the R+1 headers, which it can't do without predicting more than 3k/f slots after the first header, which it might not be able to do.
44+
45+
For example, suppose that L is the result of a block in relative slot number 7k/f - 2 within some epoch.
46+
Because there might be a block in relative slot 7k/f - 1---which would change the nonce used for the next epoch---L is unable to validate headers in the next epoch.
47+
If the first header after L on C is indeed in slot 7k/f - 1, then Chain Growth ensures the k+1th header after L on C will still be in this same epoch, and so L can validate it.
48+
If the first header after L on C is instead after slot 7k/f - 1, then ticking L to that first header will tick it past slot 7k/f - 1, and so determines the next epoch's nonce.
49+
That's likely sufficient for the next k headers, unless there is an extremely severe Chain Growth violation.
50+
(In the non-simplified node, the prediction is always restricted to exactly 3k/f slots, despite the data in question actually advancing an entire epoch at a time.
51+
Thus regardless of where the first header is, any Chain Growth violation in the 3k/f slots after it would prevent switch to C when R=k.)
52+
53+
That is a complete answer, but there are a few particulars worth emphasizing.
54+
55+
- No other behavior of the simplified node will change due to a Chain Growth violation.
56+
(This is actually false, but only because we use predictions in the leadership check.
57+
One option to make this true is to instead use ticking in the leadership check.
58+
Another option is to also characterize here how Chain Growth violations affect the predication-based leadership check, which is even milder than the above: it only manifests when there are zero blocks in some 3k/f slot window, not merely any number of blocks less than k.)
59+
- It's worth explicitly stating that there's never a risk of the node incorrectly judging the validity of a header.
60+
At worst, the node won't be able to make any judgement of the header at all, since it will recognize that doing so correctly would require information that it cannot predict.
61+
- It's entirely possible the node's behavior wouldn't even reveal that there was a Chain Growth violation!
62+
It's only guaranteed to fail if the correct Ouroboros Praos behavior were to require a k-deep roll back, which itself is an event may remain extremely rare despite whatever conditions caused the Chain Growth violation.
63+
For example, suppose 90% of stake suddenly shut down their nodes, but somehow the remaining 10% was still well-connected and there remained a sufficiently dominant honest majority.
64+
With just 10% of stake, it's almost certain that there would be less than k blocks in 3k/f slots, ie a Chain Growth violation.
65+
However, it's also almost certain there would be hundreds of blocks in every window of 3k/f slots.
66+
And, since the remaining nodes are still sufficiently well-connected, Praos's natural short forks would indeed remain short.
67+
Thus R would remain much smaller than k, such that any node that needed to switch from D to C would still be able to validate R+1 headers after L on C despite not being able to predict more than 3k/f after the first header after L on C.
68+
69+
## Back to Reality: the Cardano Node
70+
71+
First, instead of merely pure Praos, consider a node running Cardano's hard fork combinator and its history of eras from Byron through Babbage.
72+
The answer to Question 1 actually does not change.
73+
One crucial reason is because Cardano sets the hard fork combinator safe zone of each era to that era's stability window.
74+
If the safe zone were instead less than one stability window, then L does not necessarily contain enough information to validate the R+1th header after L on C, _even without_ a Chain Growth violation.
75+
Specifically, the node will need to know if the chain will actually be in whichever era that header claims to be in, and it wouldn't necessarily be able to answer that question more than a safe zone after the first header after L on C.
76+
77+
The answer to Question 2 does change: there is one additional change in behavior possibly enabled by a Chain Growth violation.
78+
Unlike the ledger, the hard fork combinator actually counts blocks instead of only slots.
79+
If a hypothetical roll back of k-many blocks could possibly change the ledger-based governance decision that was intended to trigger the era transition---which is true if Chain Growth was (severely) violated in the last two stability windows before the intended era transition---then the hard fork combinator will not transition to the next era.
80+
(For this reason, the voting deadline was moved from 7k/f to 4k/f in the ledger governance rules, so that the hard fork combinator's final decision would be known at least a stability window (and so the safe zone) before the epoch transition.)
81+
Awkwardly, the ledger will still update the major version of the protocol version protocol parameter, but the chain will remain in the previous era, which many users and developers would find counter-intuitive.
82+
83+
So this incoherence can arise only due to a Chain Growth violation (and a specific and severe one, at that), which means the hard fork combinator itself changes the answer to Question 2.
84+
Otherwise, the behavior will be the same as the simplified node.
85+
The risk of this awkward incoherence is more easily motivated below in the context of the other simplification.
86+
87+
Second, the real node also answers queries over a node-to-client protocol.
88+
This simplification changes neither the answer to Question 1 nor to Question 2 compared to the simplified node.
89+
However, it would add an additional difference to Question 2 were it not for the hard fork combinator counting blocks instead of slots.
90+
91+
Here is today's Cardano node's full list of queries, including the not-yet-released Conway.
92+
93+
- Queries for any block
94+
95+
++GetSystemStart :: Query blk SystemStart
96+
**GetChainBlockNo :: Query blk (WithOrigin BlockNo)
97+
GetChainPoint :: Query blk (Point blk)
98+
99+
- Queries for the hard fork combinator
100+
101+
*+GetEraStart :: EraIndex (x : y : xs) -> BlockQuery (HardForkBlock (x : y : xs)) (Maybe Bound)
102+
*+GetInterpreter :: BlockQuery (HardForkBlock (x : y : xs)) (History.Interpreter (x : y : xs))
103+
GetCurrentEra :: BlockQuery (HardForkBlock (x : y : xs)) (EraIndex (x : y : xs))
104+
105+
- Queries for the Byron era
106+
107+
GetUpdateInterfaceState :: BlockQuery ByronBlock UPI.State
108+
109+
- Queries for each era after Byron
110+
111+
GetLedgerTip :: BlockQuery (ShelleyBlock proto era) (Point (ShelleyBlock proto era))
112+
GetEpochNo :: BlockQuery (ShelleyBlock proto era) EpochNo
113+
GetNonMyopicMemberRewards :: Set (Either SL.Coin (SL.Credential 'SL.Staking (EraCrypto era))) -> BlockQuery (ShelleyBlock proto era) (NonMyopicMemberRewards (EraCrypto era))
114+
GetCurrentPParams :: BlockQuery (ShelleyBlock proto era) (LC.PParams era)
115+
GetProposedPParamsUpdates :: BlockQuery (ShelleyBlock proto era) (SL.ProposedPPUpdates era)
116+
GetStakeDistribution :: BlockQuery (ShelleyBlock proto era) (SL.PoolDistr (EraCrypto era))
117+
GetUTxOByAddress :: Set (SL.Addr (EraCrypto era)) -> BlockQuery (ShelleyBlock proto era) (SL.UTxO era)
118+
GetUTxOWhole :: BlockQuery (ShelleyBlock proto era) (SL.UTxO era)
119+
GetFilteredDelegationsAndRewardAccounts :: Set (SL.Credential 'SL.Staking (EraCrypto era)) -> BlockQuery (ShelleyBlock proto era) (Delegations (EraCrypto era), SL.RewardAccounts (EraCrypto era))
120+
GetGenesisConfig :: BlockQuery (ShelleyBlock proto era) (CompactGenesis (EraCrypto era))
121+
GetRewardProvenance :: BlockQuery (ShelleyBlock proto era) (SL.RewardProvenance (EraCrypto era))
122+
GetUTxOByTxIn :: Set (SL.TxIn (EraCrypto era)) -> BlockQuery (ShelleyBlock proto era) (SL.UTxO era)
123+
GetStakePools :: BlockQuery (ShelleyBlock proto era) (Set (SL.KeyHash 'SL.StakePool (EraCrypto era)))
124+
GetStakePoolParams :: Set (SL.KeyHash 'SL.StakePool (EraCrypto era)) -> BlockQuery (ShelleyBlock proto era) (Map (SL.KeyHash 'SL.StakePool (EraCrypto era)) (SL.PoolParams (EraCrypto era)))
125+
GetRewardInfoPools :: BlockQuery (ShelleyBlock proto era) (SL.RewardParams, Map (SL.KeyHash 'SL.StakePool (EraCrypto era)) (SL.RewardInfoPool))
126+
GetPoolState :: Maybe (Set (SL.KeyHash 'SL.StakePool (EraCrypto era))) -> BlockQuery (ShelleyBlock proto era) (SL.PState era)
127+
GetStakeSnapshots :: Maybe (Set (SL.KeyHash 'SL.StakePool (EraCrypto era))) -> BlockQuery (ShelleyBlock proto era) (StakeSnapshots (EraCrypto era))
128+
GetPoolDistr :: Maybe (Set (SL.KeyHash 'SL.StakePool (EraCrypto era))) -> BlockQuery (ShelleyBlock proto era) (SL.PoolDistr (EraCrypto era))
129+
GetStakeDelegDeposits :: Set (StakeCredential (EraCrypto era)) -> BlockQuery (ShelleyBlock proto era) (Map (StakeCredential (EraCrypto era)) Coin)
130+
GetConstitution :: BlockQuery (ShelleyBlock proto era) (Maybe (LC.Constitution era))
131+
GetGovState :: BlockQuery (ShelleyBlock proto era) (LC.GovState era)
132+
GetDRepState :: Set (SL.Credential 'DRepRole (EraCrypto era)) -> BlockQuery (ShelleyBlock proto era) (Map (SL.Credential 'DRepRole (EraCrypto era)) (SL.DRepState (EraCrypto era)))
133+
GetDRepStakeDistr :: Set (LC.DRep (EraCrypto era)) -> BlockQuery (ShelleyBlock proto era) (Map (LC.DRep (EraCrypto era)) Coin)
134+
GetCommitteeState :: BlockQuery (ShelleyBlock proto era) (SL.CommitteeState era)
135+
136+
The queries marked with `*` and/or `+` are the only queries whose responses exhibit weak monotinicity as the node improves its selection by switching/extending chains.
137+
For example, once `GetChainBlockNo` becomes `NotOrigin x`, it never afterwards return `NotOrigin y` where `y < x`.
138+
Similarly, once `GetEraStart` for some era of interest becomes `Just x`, it will remain `Just x`.
139+
140+
- `++` means the monotonicity is absolute (eg it's a constant value for this blockchain).
141+
- `**` means the monotonicity could only be spoiled if something like data corruption caused the node to roll back some blocks without simultaneously adding at least that many blocks.
142+
- `*+` means the monotonicity could only be spoiled if the node somehow performed a switch that rolled back more than k blocks.
143+
Call this _k-rollback-monotonicity_.
144+
145+
If the hard fork combinator were to always agree with the ledger---thereby avoiding the awkward surprise arising possible after a Chain Growth violation---then it would have to count only slots as the ledger does and not blocks.
146+
As a result, `GetEraStart` and `GetInterpreter` would no longer exhibit weak monotonicity.
147+
Specifically, a particular Chain Growth violation could cause `GetEraStart` to yield `Just x` in one moment and then `Nothing` or `Just y` where `x /= y` at a later moment, despite the node never rolling back more than k blocks.
148+
(`GetInterpreter` would be more intricate to describe, but the nub is exactly the same.)
149+
150+
It is remarkable that every query without a marker is not weakly monotonic, and no one expects them to be!
151+
For example, there's no surprise that the UTxO changes when the node switches chains.
152+
However, it was decided that `GetEraStart` and `GetInterpreter` needed to exhibit this monotonicity because their replies include information about _time_.
153+
For example, if a user needed to know at what UTC time a specific slot would begin, the Cardano CLI would determine that using the `GetInterpreter` query.
154+
And because `GetInterpreter` is weakly monotonic, the Cardano CLI will reply with "unable to translate that slot to a UTC time" until the UTC time is both predictable from the ledger state answering the query and moreover guaranteed to be the same for every ledger state the node could subsequently select unless it somehow rolls back more than k blocks.
155+
This decision may seem even more reasonable when you consider that the opposite question is handled the same exact way: the node will not let the user translate a UTC time to a slot until that answer is also k-rollback-monotonic.
156+
(Frisby's inference is that UTC time seems so fundamental that it's easy to wrongly assume that the translation back and forth between slots and UTC times does not depend on the evolution of the chain.)
157+
Requiring this monotonicity for `GetEraStart` and `GetInterpreter` avoids that confusion from manifesting.
158+
159+
## Proposal: Best of Both Worlds
160+
161+
Consider the following alteration of the hard fork combinator.
162+
163+
- Remove the hard fork combinator's second trigger constraint, such that it transitions to the next era exactly when the ledger governance updates the major version of the protocol version protocol parameter (mutatis mutandi as of Conway).
164+
- Only answer `GetEraStart` and `GetInterpreter` queries using an immutable ledger state.
165+
- (The ledger rules could then return the voting deadline to 7k/f; one stability window before the epoch transition would be sufficient again.)
166+
167+
This would have the exact same answers to Question 1 and Question 2 as does the simplified node, and the note-to-client time translation queries would still exhibit k-rollback-monotonicity.
168+
169+
There are three main benefits for the Consensus Team.
170+
171+
- The hard fork combinator would no longer need the block-counting logic.
172+
- The Consensus Team would no longer have to justify the "double-stability" for the ledger governance rules.
173+
- Justifying the k-rollback-monotonicity of the time tranlsation queries would be easier, since the immutable tip is more familiar than the double-stability window and hidden internal detail of the hard fork combinator.
174+
175+
There are two possible downsides to this change.
176+
177+
- It would now be the case that someone's node may forge a block in a new era at one moment and then forge a block in the previous era at a later moment.
178+
In a sense, scrutiny of "which eras my node has forged blocks in" or of "which eras has my node validated blocks in" etc is comparable to the `GetEraStart` query.
179+
But the result of that scrutiny will no longer be k-rollback-monotonic.
180+
On the other hand, the blocks or headers being scrutinized are are not themselves k-rollback-monotonic, by the very definition of _roll back_ and _blockchain_.
181+
And so there is much less potential for such non-k-rollback-monotonicity to surprise than there would be if the queries about UTC times were not k-rollback-monotonic.
182+
- Perhaps the community would prefer for the era transition to not happen if there was a severe Chain Growth violation immediately before it.
183+
With today's hard fork combinator, the era transition would not happen, and with the proposed hard fork combinator the era transition would happen.
184+
The choice between those two is somewhat moot, since the disaster recovery plan involves off-chain cooperation that will essentially roll back far enough to fill in the missed slots with empty blocks such that Chain Growth is no longer violated.
185+
In either case, the chosen depth of that roll back can determine whether the originally-intended era transition does take place.
186+
If they simply fill in the missing slots without changing the relevant votes, then both repaired chains will end up in the new era.
187+
To prevent that they'd need to roll back at least to before the voting deadline (4k/f today, 7k/f in the proposal) in the epoch (which may be earlier than the Chain Growth violation started) so they could change some votes to prevent the original decision to transition eras at the end of that epoch.
188+
The proposed change only makes a difference before the disaster recovery or if the disaster recovery is never done (eg establishing a checkpoint shortly afterward could safely guard that sparse part of the chain instead of having to time travel and fill it in).
189+
For example, would it be better Public Relations if the transition _didn't_ happen when planned (as in today's node)?
190+
Or would it be better if the transition happened when planned _despite_ there being a conspicuous disaster immediately prior to the transition (as in the proposed node)?

0 commit comments

Comments
 (0)