Skip to content

Update README.md and default local mmtk location #223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 3 commits into
base: master
Choose a base branch
from
Open

Update README.md and default local mmtk location #223

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
688b0ef
Replace hardcoded debug level "release" with $DEBUG_LEVEL
shamouda Jun 24, 2023
47b560d
Remove reference to legacy "repos" directory in Cargo.toml
shamouda Jun 24, 2023
957e765
Restore the hardcoded "release" target for performance runs with the …
shamouda Jun 26, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ $ make CONF=linux-x86_64-normal-server-$DEBUG_LEVEL THIRD_PARTY_HEAP=$PWD/../mmt

The output jdk is at `./build/linux-x86_64-normal-server-$DEBUG_LEVEL/jdk`.

**Note:** The above `make` command will build what is known as the [`default` target or "exploded image"](https://github.com/openjdk/jdk11u/blob/master/doc/building.md#Running-make). This build is exclusively meant for developers who want quick and incremental builds to test changes. If you are planning on evaluating your build (be it performance, minimum heap, etc.), then it is *highly advised* to use the `images` target. The `default` target is the (roughly) minimal set of outputs required to run the built JDK and is not guaranteed to run all benchmarks. It may have bloated minimum heap values as well. The `images` target can be built like so:
**Note:** The above `make` command will build what is known as the [`default` target or "exploded image"](https://github.com/openjdk/jdk11u/blob/master/doc/building.md#Running-make). This build is exclusively meant for developers who want quick and incremental builds to test changes. If you are planning on evaluating your build (be it performance, minimum heap, etc.), then it is *highly advised* to use the `release` debug level with the `images` target. The `default` target is the (roughly) minimal set of outputs required to run the built JDK and is not guaranteed to run all benchmarks. It may have bloated minimum heap values as well. The `images` target can be built like so:

```console
$ make CONF=linux-x86_64-normal-server-release THIRD_PARTY_HEAP=$PWD/../mmtk-openjdk/openjdk images
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add a line saying "For performance evaluation, we expect you are using the release build"? That was the original reason the make command was not using $DEBUG_LEVEL and directly using release. I agree that it should have been sign posted a bit better.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or better yet, mention it in the note directly above the make command.

Copy link
Author

@shamouda shamouda Jun 26, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the same apply to the "Profile-Guided Optimized Build" section?
If that is the case, then I should probably either:

  1. undo all the changes (i.e. restore the hardcoded -release values) but add a clarifying note, or
  2. add "DEBUG_LEVEL=release" before the make commands to make it more explicit.

What do you think?

Copy link
Collaborator

@k-sareen k-sareen Jun 27, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the first one is probably better. And yes, the PGO build is also generally used for evaluation only. The images (and PGO build) have extra steps involved in them which only make sense when you want to perform some kind of evaluation. In the images case, the default build (an unoptimized dev build) is used to bootstrap a production build and in the PGO case, the Rust compiler uses execution profiles to inform inlining decisions.

Expand Down Expand Up @@ -148,7 +148,7 @@ stress factor of 4 MB in order to trigger more GC events.
First we compile MMTk with profiling support:

```console
$ RUSTFLAGS="-Cprofile-generate=/tmp/$USER/pgo-data" make CONF=linux-x86_64-normal-server-release THIRD_PARTY_HEAP=$PWD/../mmtk-openjdk/openjdk images
$ RUSTFLAGS="-Cprofile-generate=/tmp/$USER/pgo-data" make CONF=linux-x86_64-normal-server-$DEBUG_LEVEL THIRD_PARTY_HEAP=$PWD/../mmtk-openjdk/openjdk images
$ rm -rf /tmp/$USER/pgo-data/*
```
We clear the `/tmp/$USER/pgo-data` directory as during compilation, the JVM we
Expand All @@ -159,7 +159,7 @@ We then run `fop` in order to get some profiling data. Note that your location
for the DaCapo benchmarks may be different:

```bash
MMTK_PLAN=GenImmix MMTK_STRESS_FACTOR=4194304 MMTK_PRECISE_STRESS=false ./build/linux-x86_64-normal-server-release/images/jdk/bin/java -XX:MetaspaceSize=500M -XX:+DisableExplicitGC -XX:-TieredCompilation -Xcomp -XX:+UseThirdPartyHeap -Xms60M -Xmx60M -jar /usr/share/benchmarks/dacapo/dacapo-evaluation-git-6e411f33.jar -n 5 fop
MMTK_PLAN=GenImmix MMTK_STRESS_FACTOR=4194304 MMTK_PRECISE_STRESS=false ./build/linux-x86_64-normal-server-$DEBUG_LEVEL/images/jdk/bin/java -XX:MetaspaceSize=500M -XX:+DisableExplicitGC -XX:-TieredCompilation -Xcomp -XX:+UseThirdPartyHeap -Xms60M -Xmx60M -jar /usr/share/benchmarks/dacapo/dacapo-evaluation-git-6e411f33.jar -n 5 fop
```

We have to merge the profiling data into something we can feed into the Rust
Expand All @@ -176,11 +176,11 @@ your Rust version.*
Finally, we build a new image using the profiling data as an input:

```console
$ RUSTFLAGS="-Cprofile-use=/tmp/$USER/pgo-data/merged.profdata -Cllvm-args=-pgo-warn-missing-function" make CONF=linux-x86_64-normal-server-release THIRD_PARTY_HEAP=$PWD/../mmtk-openjdk/openjdk images
$ RUSTFLAGS="-Cprofile-use=/tmp/$USER/pgo-data/merged.profdata -Cllvm-args=-pgo-warn-missing-function" make CONF=linux-x86_64-normal-server-$DEBUG_LEVEL THIRD_PARTY_HEAP=$PWD/../mmtk-openjdk/openjdk images
```

We now have an OpenJDK build under
`./build/linux-x86_64-normal-server-release/images/jdk` with MMTk that has been
`./build/linux-x86_64-normal-server-$DEBUG_LEVEL/images/jdk` with MMTk that has been
optimized using PGO.

For ease of use, we have provided an example script which does the above in
Expand Down
2 changes: 1 addition & 1 deletion mmtk/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ once_cell = "1.10.0"
# But other changes including adding/removing whitespaces in commented lines may break the CI.
mmtk = { git = "https://github.com/mmtk/mmtk-core.git", rev = "f1a0bb7fbec97dd84e35a40e8be01cf5018f2f9e" }
# Uncomment the following to build locally
# mmtk = { path = "../repos/mmtk-core" }
# mmtk = { path = "../../mmtk-core" }

[build-dependencies]
built = { version = "0.5.1", features = ["git2"] }
Expand Down