-
Notifications
You must be signed in to change notification settings - Fork 418
Fix TLV serialization to work with large types #964
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix TLV serialization to work with large types #964
Conversation
Codecov Report
@@ Coverage Diff @@
## main #964 +/- ##
==========================================
- Coverage 90.65% 90.65% -0.01%
==========================================
Files 60 60
Lines 30407 30398 -9
==========================================
- Hits 27566 27557 -9
Misses 2841 2841
Continue to review full report at Codecov.
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nicely done! Wanted to drop a few comments before finding more time for a more thorough review.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code basically looks good, I don't have an opinion on macro naming or overhauling the entire syntax (now that it's practical given we don't have strange cross-item ordering constraints).
cdab3fe
to
0cee860
Compare
0cee860
to
dfe22db
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Only nits.
lightning/src/util/ser_macros.rs
Outdated
BigSize(field.serialized_length() as u64).write($stream)?; | ||
field.write($stream)?; | ||
} | ||
encode_tlv!($stream, $type, $field, $fieldty); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
May be worth pulling this out into a separate repetition and making this repetition #[cfg(debug_assertions)]
-enabled in its own scope. Release builds might be better optimized since last_seen
wouldn't exist. I think the allow
wouldn't be needed either.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this the implementation you were thinking of? https://i.imgur.com/G3g1BNz.png Seems it'll still need the allow
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Eh, if there's one thing I have confidence in LLVM for, it's optimizing away purely-on-stack variables which are written too and never read (and branches with no effect).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking last_seen
would go inside the second #[cfg(debug_assertions)]
scope, leaving only one #[cfg(debug_assertions)]
. But maybe it's still problematic?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking
last_seen
would go inside the second#[cfg(debug_assertions)]
scope, leaving only one#[cfg(debug_assertions)]
. But maybe it's still problematic?
Ah, for some reason that was less obviously correct to me, but it does seem to be right according to -Z trace-macros
. Still needs the allow
but otherwise done
dfe22db
to
899b484
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK module two suggested tlv fixups. Just to check, @jkczyz you were fine with leaving syntax refactors to a followup (or not doing them), correct?
Yup |
899b484
to
afbb846
Compare
Previous to this PR, TLV serialization involved iterating from 0 to the highest given TLV type. This worked until we decided to implement keysend, which has a TLV type of ~5.48 billion. So instead, we now specify the type of whatever is being (de)serialized (which can be an Option, a Vec type, or a non-Option (specified in the serialization macros as "required").
afbb846
to
40959b7
Compare
Previous to this PR, TLV serialization involved iterating from 0 to the highest
given TLV type. This worked until we decided to implement keysend, which has a
TLV type of ~5.48 billion.
So instead, we now specify the type of whatever is being (de)serialized (which
can be an Option, a Vec type, or "something else" (specified in the serialization macros as "required").