Skip to content

[libc] add linux mman extension remap_file_pages #110122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
SchrodingerZhu opened this issue Sep 26, 2024 · 6 comments · Fixed by #110307
Closed

[libc] add linux mman extension remap_file_pages #110122

SchrodingerZhu opened this issue Sep 26, 2024 · 6 comments · Fixed by #110307
Assignees
Labels
good first issue https://github.com/llvm/llvm-project/contribute libc

Comments

@SchrodingerZhu
Copy link
Contributor

We need to create a remap_file_pages wrapper for linux targets, see https://www.man7.org/linux/man-pages/man2/remap_file_pages.2.html.

This requires:

  • implement the entrypoint header similar to libc/src/sys/mman/mmap.h
    • modify libc/src/sys/mman/CMakeLists.txt accordingly
  • implement the entrypoint similar to libc/src/sys/mman/linux/mmap.cpp
    • modify libc/src/sys/mman/linux/CMakeLists.txt accordingly
  • add in the header spec
    • modify functions section in libc/newhdrgen/yaml/sys/mman.yaml
    • modify SysMMan section in libc/spec/linux.td
  • add in test
    • similar to libc/test/src/sys/mman/linux/mmap_test.cpp
    • modify libc/test/src/sys/mman/linux/CMakeLists.txt accordingly

Misc:

If you use vscode, sample settings are as the following (optional settings are useful in speeding up the compilation):

{
    "cmake.sourceDirectory": "${workspaceFolder}/runtimes",
    "cmake.configureSettings": {
        "CMAKE_EXPORT_COMPILE_COMMANDS": true,
        "LLVM_ENABLE_RUNTIMES": [
            "libc",
            "compiler-rt"
        ],
        "LLVM_LIBC_FULL_BUILD": true,
        "LLVM_USE_LINKER": "lld", // optional: requires installing lld
        "CMAKE_CXX_COMPILER_LAUNCHER": "sccache", // optional: requires installing sccache first
        "CMAKE_C_COMPILER_LAUNCHER": "sccache", // optional: requires installing sccache first
        "CMAKE_CXX_COMPILER": "/usr/bin/clang++",
        "CMAKE_C_COMPILER": "/usr/bin/clang",
        "LLVM_LIBC_INCLUDE_SCUDO": true,
        "COMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC": true,
        "COMPILER_RT_BUILD_GWP_ASAN": false,
        "COMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED": false,
        "LIBC_USE_NEW_HEADER_GEN": true
    },
    "cmake.generator": "Ninja", // optional: requires installing Ninja
    "editor.formatOnSave": true,
    "files.insertFinalNewline": true,
    "clangd.arguments": [
        "--header-insertion=never"
    ]
}
@SchrodingerZhu SchrodingerZhu added good first issue https://github.com/llvm/llvm-project/contribute libc labels Sep 26, 2024
@llvmbot
Copy link
Member

llvmbot commented Sep 26, 2024

@llvm/issue-subscribers-libc

Author: Schrodinger ZHU Yifan (SchrodingerZhu)

We need to create a `remap_file_pages` wrapper for linux targets, see https://www.man7.org/linux/man-pages/man2/remap_file_pages.2.html.

This requires:

  • implement the entrypoint header similar to libc/src/sys/mman/mmap.h
    • modify libc/src/sys/mman/CMakeLists.txt accordingly
  • implement the entrypoint similar to libc/src/sys/mman/linux/mmap.cpp
    • modify libc/src/sys/mman/linux/CMakeLists.txt accordingly
  • add in the header spec
    • modify functions section in libc/newhdrgen/yaml/sys/mman.yaml
    • modify SysMMan section in libc/spec/linux.td
  • add in test
    • similar to libc/test/src/sys/mman/linux/mmap_test.cpp
    • modify libc/test/src/sys/mman/linux/CMakeLists.txt accordingly

Misc:

If you use vscode, sample settings are as the following (optional settings are useful in speeding up the compilation):

{
    "cmake.sourceDirectory": "${workspaceFolder}/runtimes",
    "cmake.configureSettings": {
        "CMAKE_EXPORT_COMPILE_COMMANDS": true,
        "LLVM_ENABLE_RUNTIMES": [
            "libc",
            "compiler-rt"
        ],
        "LLVM_LIBC_FULL_BUILD": true,
        "LLVM_USE_LINKER": "lld", // optional: requires installing lld
        "CMAKE_CXX_COMPILER_LAUNCHER": "sccache", // optional: requires installing sccache first
        "CMAKE_C_COMPILER_LAUNCHER": "sccache", // optional: requires installing sccache first
        "CMAKE_CXX_COMPILER": "/usr/bin/clang++",
        "CMAKE_C_COMPILER": "/usr/bin/clang",
        "LLVM_LIBC_INCLUDE_SCUDO": true,
        "COMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC": true,
        "COMPILER_RT_BUILD_GWP_ASAN": false,
        "COMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED": false,
        "LIBC_USE_NEW_HEADER_GEN": true
    },
    "cmake.generator": "Ninja", // optional: requires installing Ninja
    "editor.formatOnSave": true,
    "files.insertFinalNewline": true,
    "clangd.arguments": [
        "--header-insertion=never"
    ]
}

@llvmbot
Copy link
Member

llvmbot commented Sep 26, 2024

Hi!

This issue may be a good introductory issue for people new to working on LLVM. If you would like to work on this issue, your first steps are:

  1. Check that no other contributor has already been assigned to this issue. If you believe that no one is actually working on it despite an assignment, ping the person. After one week without a response, the assignee may be changed.
  2. In the comments of this issue, request for it to be assigned to you, or just create a pull request after following the steps below. Mention this issue in the description of the pull request.
  3. Fix the issue locally.
  4. Run the test suite locally. Remember that the subdirectories under test/ create fine-grained testing targets, so you can e.g. use make check-clang-ast to only run Clang's AST tests.
  5. Create a Git commit.
  6. Run git clang-format HEAD~1 to format your changes.
  7. Open a pull request to the upstream repository on GitHub. Detailed instructions can be found in GitHub's documentation. Mention this issue in the description of the pull request.

If you have any further questions about this issue, don't hesitate to ask via a comment in the thread below.

@llvmbot
Copy link
Member

llvmbot commented Sep 26, 2024

@llvm/issue-subscribers-good-first-issue

Author: Schrodinger ZHU Yifan (SchrodingerZhu)

We need to create a `remap_file_pages` wrapper for linux targets, see https://www.man7.org/linux/man-pages/man2/remap_file_pages.2.html.

This requires:

  • implement the entrypoint header similar to libc/src/sys/mman/mmap.h
    • modify libc/src/sys/mman/CMakeLists.txt accordingly
  • implement the entrypoint similar to libc/src/sys/mman/linux/mmap.cpp
    • modify libc/src/sys/mman/linux/CMakeLists.txt accordingly
  • add in the header spec
    • modify functions section in libc/newhdrgen/yaml/sys/mman.yaml
    • modify SysMMan section in libc/spec/linux.td
  • add in test
    • similar to libc/test/src/sys/mman/linux/mmap_test.cpp
    • modify libc/test/src/sys/mman/linux/CMakeLists.txt accordingly

Misc:

If you use vscode, sample settings are as the following (optional settings are useful in speeding up the compilation):

{
    "cmake.sourceDirectory": "${workspaceFolder}/runtimes",
    "cmake.configureSettings": {
        "CMAKE_EXPORT_COMPILE_COMMANDS": true,
        "LLVM_ENABLE_RUNTIMES": [
            "libc",
            "compiler-rt"
        ],
        "LLVM_LIBC_FULL_BUILD": true,
        "LLVM_USE_LINKER": "lld", // optional: requires installing lld
        "CMAKE_CXX_COMPILER_LAUNCHER": "sccache", // optional: requires installing sccache first
        "CMAKE_C_COMPILER_LAUNCHER": "sccache", // optional: requires installing sccache first
        "CMAKE_CXX_COMPILER": "/usr/bin/clang++",
        "CMAKE_C_COMPILER": "/usr/bin/clang",
        "LLVM_LIBC_INCLUDE_SCUDO": true,
        "COMPILER_RT_BUILD_SCUDO_STANDALONE_WITH_LLVM_LIBC": true,
        "COMPILER_RT_BUILD_GWP_ASAN": false,
        "COMPILER_RT_SCUDO_STANDALONE_BUILD_SHARED": false,
        "LIBC_USE_NEW_HEADER_GEN": true
    },
    "cmake.generator": "Ninja", // optional: requires installing Ninja
    "editor.formatOnSave": true,
    "files.insertFinalNewline": true,
    "clangd.arguments": [
        "--header-insertion=never"
    ]
}

@AlyElashram
Copy link
Contributor

You can Assign this one to me 👍

@AlyElashram
Copy link
Contributor

@SchrodingerZhu I created the changes , but I'm a bit lost in Unit Testing at the moment.
I have a couple of question :

  1. I assume the page size can be architecture dependent , so should I fetch the page size before using a sysconf call ?
  2. To call remap_file_pages I should first allocate some memory , so is it safe to call mmap here given that it is already tested and succeeds on the main branch ? If not how should we allocate memory?
  3. Is it necessary to test if each of the params fail/are invalid or would one of the following variables being invalid suffice
    EINVAL addr, size, prot, or pgoff is invalid. ?

@SchrodingerZhu
Copy link
Contributor Author

  1. I think u can use sysconf inside libc. It is already there in our codebase.

  2. Using mmap is fine.

  3. There is no need to exhaustively test kernel behaviors, and simple tests are fine.

SchrodingerZhu pushed a commit that referenced this issue Oct 14, 2024
Fixes #110122
- Create remap_file_pages.h/.cpp wrapper for the linux sys call.
- Add UnitTests for remap_file_pages
- Add function to libc/spec/linux.td
- Add Function spec to mman.yaml
DanielCChen pushed a commit to DanielCChen/llvm-project that referenced this issue Oct 16, 2024
Fixes llvm#110122
- Create remap_file_pages.h/.cpp wrapper for the linux sys call.
- Add UnitTests for remap_file_pages
- Add function to libc/spec/linux.td
- Add Function spec to mman.yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue https://github.com/llvm/llvm-project/contribute libc
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants