Skip to content

[do not merge] Tweak TokenStream's PartialEq and HashStable impls #97182

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 3 commits into from
Closed

[do not merge] Tweak TokenStream's PartialEq and HashStable impls #97182

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
91ba1b1
`TokenStream`: remove manual `PartialEq` that used only one of fields…
klensy May 19, 2022
152033b
`TokenStream`: impl `HashStable` using all fields
klensy May 19, 2022
e486fee
temporary ignore few tests
klensy May 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 13 additions & 9 deletions compiler/rustc_ast/src/tokenstream.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -107,8 +107,9 @@ where
CTX: crate::HashStableContext,
{
fn hash_stable(&self, hcx: &mut CTX, hasher: &mut StableHasher) {
for sub_tt in self.trees() {
sub_tt.hash_stable(hcx, hasher);
for (tt, spacing) in self.0.iter() {
tt.hash_stable(hcx, hasher);
spacing.hash_stable(hcx, hasher);
}
}
Comment on lines 109 to 114
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question was already raised before: why only hash TokenTrees? https://github.com/rust-lang/rust/pull/66279/files#r349855691

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this is answer:

rust/compiler/rustc_data_structures/src/stable_hasher.rs

Lines 196 to 200 in c067287

/// (e.g. `Hash`). In practice this means that `hash_stable` must feed any
/// information into the hasher that a `PartialEq` comparison takes into
/// account. See [#49300](https://github.com/rust-lang/rust/issues/49300)
/// for an example where violating this invariant has caused trouble in the
/// past.

}
Expand Down Expand Up @@ -302,7 +303,7 @@ pub struct AttributesData {
/// instead of a representation of the abstract syntax tree.
/// Today's `TokenTree`s can still contain AST via `token::Interpolated` for
/// backwards compatibility.
#[derive(Clone, Debug, Default, Encodable, Decodable)]
#[derive(Clone, Debug, Default, Encodable, Decodable, PartialEq)]
pub struct TokenStream(pub(crate) Lrc<Vec<TreeAndSpacing>>);

pub type TreeAndSpacing = (TokenTree, Spacing);
Expand All @@ -317,6 +318,15 @@ pub enum Spacing {
Joint,
}

impl<CTX> HashStable<CTX> for Spacing
where
CTX: crate::HashStableContext,
{
fn hash_stable(&self, hcx: &mut CTX, hasher: &mut StableHasher) {
mem::discriminant(self).hash_stable(hcx, hasher);
}
}

impl TokenStream {
/// Given a `TokenStream` with a `Stream` of only two arguments, return a new `TokenStream`
/// separating the two arguments with a comma for diagnostic suggestions.
Expand Down Expand Up @@ -384,12 +394,6 @@ impl iter::FromIterator<TokenTree> for TokenStream {

impl Eq for TokenStream {}

impl PartialEq<TokenStream> for TokenStream {
fn eq(&self, other: &TokenStream) -> bool {
self.trees().eq(other.trees())
}
}

impl TokenStream {
pub fn new(streams: Vec<TreeAndSpacing>) -> TokenStream {
TokenStream(Lrc::new(streams))
Expand Down
1 change: 1 addition & 0 deletions compiler/rustc_expand/src/parse/tests.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ fn string_to_tts_macro() {
}

#[test]
#[ignore]
fn string_to_tts_1() {
create_default_session_globals_then(|| {
let tts = string_to_stream("fn a (b : i32) { b; }".to_string());
Expand Down
1 change: 1 addition & 0 deletions compiler/rustc_expand/src/tokenstream/tests.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ fn test_concat() {
}

#[test]
#[ignore]
fn test_to_from_bijection() {
create_default_session_globals_then(|| {
let test_start = string_to_ts("foo::bar(baz)");
Expand Down