MapObserver should require Hash, not define a hash function #1977

addisoncrump · 2024-03-28T16:31:29Z

MapObserver currently defines a function hash which is used to get the current hash of the map. Instead, MapObserver should have a trait requirement of Hash so that it may be used with functions that accept impl Hash types.

Similarly, hash_slice should be deleted and usages replaced with the pre-defined Hasher::hash(&[T]).

The text was updated successfully, but these errors were encountered:

addisoncrump · 2024-03-28T16:32:17Z

If no one takes this in 48 hours, I will implement it myself.

domenukk · 2024-03-28T22:23:22Z

Sounds useful

edwin1729 · 2024-03-29T11:08:10Z

Hi I'd like to take up this issue.

Also I'm not sure where the best place to ask this is. I'm interested in the GSOC project implement Pangolin mutator. I was wondering where I should introduce myself / if I should just directly submit a proposal. I'm also open to other projects in LibAFL, since Its an intersection of a bunch of my interests (Rust, Verification) and is just really cool (especially in relation to bug bounties)

tokatoka · 2024-03-29T13:14:34Z

can you send a email to [email protected]

addisoncrump · 2024-03-30T16:06:53Z

@edwin1729 Can you indicate when you plan to attempt this?

edwin1729 · 2024-03-30T16:35:28Z

I've been trying to aflplusplus installed since yesterday.
On running

cargo build --release

I get

warning: [email protected]: Skipping src/dump-cfg-pass.cc
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version".

It's a resolved issue in rustc

addisoncrump · 2024-03-30T16:38:15Z

You may safely ignore that.

edwin1729 · 2024-03-30T16:39:34Z

Okay thanks. Is it alright if a send a pull request tomorrow? Is this time sensitive?

addisoncrump · 2024-03-30T16:40:23Z

Preferably before Wednesday, but I recognise that this is a holiday weekend so Friday works as well.

edwin1729 · 2024-03-30T16:42:50Z

Also, I still do have issues with running any of the example fuzzers (without changing the source code). I believed that the warning might be the cause of the errors (since there's many of them right before the backtrace)

addisoncrump · 2024-03-30T16:45:12Z

Hm, are you building with cargo make fuzzer?

edwin1729 · 2024-03-30T16:46:03Z

no cargo make run in LibAFL/fuzzers/fuzzbench

addisoncrump · 2024-03-30T16:47:13Z

That should work as well. I'm not sure what the issue is, but double check that you've installed all the required deps as listed in the README.

edwin1729 · 2024-03-30T16:50:07Z

LLVM Tools is quite vague in the readme. I don't know what exactly it requires or even how to check if i have what it needs. I installed the apt packages llvm-17-dev llvm-17-tools llvm-17 clang-17. What I'm suspecting is perhaps some environment flag was not set, which maybe AfLpluslus uses?

addisoncrump · 2024-03-30T16:52:00Z

AFL++ is not used in the example you are building. What exactly is the backtrace that you are seeing?

edwin1729 · 2024-03-30T16:53:10Z

$> ~/LibAFL/fuzzers/fuzzbench$ cargo make run
[cargo-make] INFO - cargo make 0.37.10
[cargo-make] INFO - Calling cargo metadata to extract project info
[cargo-make] INFO - Cargo metadata done
[cargo-make] INFO - Project: fuzzbench
[cargo-make] INFO - Build File: Makefile.toml
[cargo-make] INFO - Task: run
[cargo-make] INFO - Profile: development
[cargo-make] INFO - Running Task: legacy-migration
[cargo-make] INFO - Execute Command: "cargo" "build" "--profile" "release"
warning: [email protected]: Skipping src/dump-cfg-pass.cc
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
    Finished release [optimized + debuginfo] target(s) in 0.11s
[cargo-make] INFO - Execute Command: "cargo" "build" "--profile" "release"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
warning: [email protected]: Skipping src/dump-cfg-pass.cc
warning: [email protected]: Compiler version doesn't include clang or GCC: "cc" "--version"
    Finished release [optimized + debuginfo] target(s) in 0.11s
[cargo-make] INFO - Execute Command: "/home/ef322/LibAFL/fuzzers/fuzzbench/target/release/libafl_cc" "--libafl-no-link" "-O3" "-c" "fuzz.c" "-o" "fuzz.o"
thread 'main' panicked at src/bin/libafl_cc.rs:38:14:
Failed to run the wrapped compiler: Io(Os { code: 2, kind: NotFound, message: "No such file or directory" })
stack backtrace:
   0:     0x55e5f5cdd013 - std::backtrace_rs::backtrace::libunwind::trace::h6e4a662bea54ccfc
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5
   1:     0x55e5f5cdd013 - std::backtrace_rs::backtrace::trace_unsynchronized::hb42b4eb2797d9c0e
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x55e5f5cdd013 - std::sys_common::backtrace::_print_fmt::h2bc261f3223f4e4d
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/sys_common/backtrace.rs:68:5
   3:     0x55e5f5cdd013 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h9cca0343d66d16a8
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/sys_common/backtrace.rs:44:22
   4:     0x55e5f5c9dcb0 - core::fmt::rt::Argument::fmt::h8b666c45176be671
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/fmt/rt.rs:142:9
   5:     0x55e5f5c9dcb0 - core::fmt::write::h4311bce0ee536615
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/fmt/mod.rs:1120:17   6:     0x55e5f5cde6f4 - std::io::Write::write_fmt::h0685c51539d0a0cd
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/io/mod.rs:1846:15
   7:     0x55e5f5cde6f4 - std::sys_common::backtrace::_print::h25f19b1d64e81f86
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/sys_common/backtrace.rs:47:5
   8:     0x55e5f5cde6f4 - std::sys_common::backtrace::print::h2fb8f70628a241ed
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/sys_common/backtrace.rs:34:9
   9:     0x55e5f5cde3c8 - std::panicking::default_hook::{{closure}}::h05093fe2e3ef454d
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/panicking.rs:272:22  10:     0x55e5f5cded9c - std::panicking::default_hook::h5ac38aa38e0086d2
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/panicking.rs:292:9
  11:     0x55e5f5cded9c - std::panicking::rust_panic_with_hook::hed79743dc8b4b969
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/panicking.rs:781:13  12:     0x55e5f5cde940 - std::panicking::begin_panic_handler::{{closure}}::ha437b5d58f431abf
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/panicking.rs:659:13  13:     0x55e5f5cde896 - std::sys_common::backtrace::__rust_end_short_backtrace::hd98e82d5b39ec859
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/sys_common/backtrace.rs:171:18
  14:     0x55e5f5cde883 - rust_begin_unwind
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/panicking.rs:647:5
  15:     0x55e5f5c90314 - core::panicking::panic_fmt::hc69c4d258fe11477
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/panicking.rs:72:14  16:     0x55e5f5c907e2 - core::result::unwrap_failed::hff299ec748d62aab
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/result.rs:1649:5
  17:     0x55e5f5c9953b - core::result::Result<T,E>::expect::h2e6b0e237343e7ff
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/result.rs:1030:23
  18:     0x55e5f5c9953b - libafl_cc::main::hc986a561adc1af70
                               at /home/ef322/LibAFL/fuzzers/fuzzbench/src/bin/libafl_cc.rs:27:29
  19:     0x55e5f5c92793 - core::ops::function::FnOnce::call_once::hfc3cdf54d5c38f17
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/core/src/ops/function.rs:250:5
  20:     0x55e5f5c92793 - std::sys_common::backtrace::__rust_begin_short_backtrace::h9fd85d86772ad860
                               at /rustc/7cf61ebde7b22796c69757901dd346d0fe70bd97/library/std/src/sys_common/backtrace.rs:155:18
  21:     0x55e5f5c9b126 - main
  22:     0x7f9b10970083 - __libc_start_main
  23:     0x55e5f5c926ce - _start
  24:                0x0 - <unknown>
[cargo-make] ERROR - Error while executing command, exit code: 101
[cargo-make] WARN - Build Failed.

addisoncrump · 2024-03-30T16:54:47Z

Do you have the clang or llvm-config binary available?

edwin1729 · 2024-03-30T16:56:32Z

Yes they are. They're both on $PATH, though i added them manually (/usr/lib/llvm-17/bin), perhaps that indicates something

addisoncrump · 2024-03-30T16:57:50Z

What is the actual error? You listed the warnings and the backtrace, but not the error.

edwin1729 · 2024-03-30T17:00:26Z

Sorry, I've updated the stacktrace with the output from the entire command.
I'll look into this again tomorrow

addisoncrump · 2024-03-30T17:02:00Z

thread 'main' panicked at src/bin/libafl_cc.rs:38:14:
Failed to run the wrapped compiler: Io(Os { code: 2, kind: NotFound, message: "No such file or directory" })

I'm guessing it's not finding clang. This is a pretty bad error message, though. @tokatoka @andreafioraldi, any ideas? I always just install with LLVM APT or Fedora 😅

edwin1729 · 2024-03-30T17:05:14Z

I'll just try reinstalling, or building llvm from source tomorrow, if there's no obvious solution here. Thank you!

andreafioraldi · 2024-03-30T17:49:08Z

The fuzzbench fuzzer uses the cmplog rtn llvm pass, make sure that is was built. If libafl_cc cannot find your llvm installation is skips building the passes. You can force it to find your llvm config with an env: LLVM_CONFIG=/path/to/llvm-config cargo make run Il sab 30 mar 2024, 18:05 edwin1729 ***@***.***> ha scritto:

…

I'll just try reinstalling, or building llvm from source tomorrow, if there's no obvious solution here. Thank you! — Reply to this email directly, view it on GitHub <#1977 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AD3LJ6WZ6ACOLBKWZ2NSN2DY23PFBAVCNFSM6AAAAABFNDJVUWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRYGI4TOMRSGE> . You are receiving this because you were mentioned.Message ID: ***@***.***>

tokatoka · 2024-03-30T17:59:35Z

you should install the script from their website https://apt.llvm.org/

edwin1729 · 2024-04-01T13:40:34Z

Similarly, hash_slice should be deleted and usages replaced with the pre-defined Hasher::hash(&[T]).

There is no Hasher::hash(&[T]) in std lib (or is there??).
Hash::hash<H:Hasher>(&self, state: &mut H) and 'MapObserver::hash(&self)' don't have the same signature.
Since there's no function overloading in rust I think 'MapObserver::hash(&self)' might need to be renamed to something hash_easy.
This feels quite messy, so maybe a macro around Hash::hash<H:Hasher>(&self, state: &mut H) to get something like 'MapObserver::hash(&self)' ?

Either I'm over complicating or missing something. What was your original idea @addisoncrump ?

edwin1729 · 2024-04-01T15:47:23Z

I've created a pull request to illustrate, the approach above (which I don't think is quite alright) #1989.

Also, I don't think I will be able to complete this by Friday since I'll be traveling this week (tomorrow). Sorry for the late notice. I'm more than happy to complete it if its still unfinished on Saturday . The reason I say this is that the CI shows errors with python bindings. I'm guessing this might take while to debug.

…FLplusplus#1977)

* Use hash_one function to make hash_simple a one-liner

* MapObserver implements Hash * Rename the hash utility function (in MapObserver) to hash_easy * Use hash_slice as a helper function to impl Hash trait * define_python_map_observer macro implements Hash trait * Also rename hash_easy to hash_simple * Rename hash_slice to hash_helper * hash_helper is used to define the implementation of hash function/trait * Factor out the Hash trait and function for runtime library structs (#1977) * Simplify hash_simple (of trait MapObserver) (#1977) * Use hash_one function to make hash_simple a one-liner * remove hash_helper --------- Co-authored-by: Edwin Fernando <[email protected]> Co-authored-by: Addison Crump <[email protected]>

addisoncrump added enhancement New feature or request good first issue Good for newcomers labels Mar 28, 2024

edwin1729 added a commit to edwin1729/LibAFL that referenced this issue Apr 15, 2024

Factor out the Hash trait and function for runtime library structs (A…

8128b7b

…FLplusplus#1977)

edwin1729 added a commit to edwin1729/LibAFL that referenced this issue Apr 17, 2024

Simplify hash_simple (of trait MapObserver) (AFLplusplus#1977)

054bb40

* Use hash_one function to make hash_simple a one-liner

tokatoka closed this as completed Apr 24, 2024

Uh oh!

MapObserver should require Hash, not define a hash function #1977

MapObserver should require Hash, not define a hash function #1977

Comments

addisoncrump commented Mar 28, 2024

addisoncrump commented Mar 28, 2024

Uh oh!

domenukk commented Mar 28, 2024

Uh oh!

edwin1729 commented Mar 29, 2024

Uh oh!

tokatoka commented Mar 29, 2024

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

addisoncrump commented Mar 30, 2024

Uh oh!

edwin1729 commented Mar 30, 2024

Uh oh!

andreafioraldi commented Mar 30, 2024 via email

Uh oh!

tokatoka commented Mar 30, 2024

Uh oh!

edwin1729 commented Apr 1, 2024

Uh oh!

edwin1729 commented Apr 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

edwin1729 commented Mar 30, 2024 •

edited

Loading

edwin1729 commented Mar 30, 2024 •

edited

Loading

edwin1729 commented Mar 30, 2024 •

edited

Loading

edwin1729 commented Mar 30, 2024 •

edited

Loading

edwin1729 commented Mar 30, 2024 •

edited

Loading

edwin1729 commented Mar 30, 2024 •

edited

Loading

edwin1729 commented Apr 1, 2024 •

edited

Loading