Skip to content

Tests should not depend on hashmap iteration order #1

@workingjubilee

Description

@workingjubilee

Hi! Your test suite depends on iterating over the stdlib's HashMap<K, V, S> but we generate random state for every map unless it overrides the S parameter, meaning each time they are built and run, you get different hashing. This is intentional to prevent common exploits. But we also would like hashing details to not slow down common uses like iteration, so when you iterate over a hashmap, you get an "arbitrary order", which means that iteration of a hashmap with S = RandomState visits keys in a different order each time.

This results in this sort of thing happening to your tests:

[INFO] [stderr]     Finished `test` profile [unoptimized + debuginfo] target(s) in 0.01s
[INFO] [stderr]      Running unittests src/lib.rs (/opt/rustwide/target/debug/deps/rwini-93436e769e484a05)
[INFO] [stdout] 
[INFO] [stdout] running 2 tests
[INFO] [stdout] test parser::test::ini_parser::test_parse ... ok
[INFO] [stdout] test data::test::test_ini_to_string ... FAILED
[INFO] [stdout] 
[INFO] [stdout] failures:
[INFO] [stdout] 
[INFO] [stdout] ---- data::test::test_ini_to_string stdout ----
[INFO] [stdout] thread 'data::test::test_ini_to_string' panicked at src/data/mod.rs:13:5:
[INFO] [stdout] assertion `left == right` failed
[INFO] [stdout]   left: "[core]\nkey:value\nname:1"
[INFO] [stdout]  right: "[core]\nname:1\nkey:value"

You may wish to consider some manner of sorting before comparing in your test.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions