[CI] Make email check workflow fail when author's email is private in Github UI

[uditagarwal97]

Problem
Currently, the email check workflow uses git to see email used for the last commit but the email address used when merging is actually governed by GitHub settings not what's stored in git. Due to this, the email check workflow passes even when the author's email is private in Github.
We saw several such cases in our fork of llvm. See intel/llvm#17675

Solution
Try to find user's email using GH's GraphQL APIs. User's email will be null if it's hidden in the profile.

[llvmbot]

@llvm/pr-subscribers-github-workflow

Author: Udit Kumar Agarwal (uditagarwal97)

Changes

Problem
Consider the following case:
Someone creates a PR with the signed commit but has email set to "private" in the Github UI (https://docs.github.com/en/account-and-profile/setting-up-and-managing-your-personal-account-on-github/managing-email-preferences/blocking-command-line-pushes-that-expose-your-personal-email-address). Currently, the email check workflow checkout to the branch and use git show -s --format='%ae' HEAD~0 to see email used for the last commit and the workflow will pass. However, when merging this PR, since the email is setting is private in Github UI, the merged PR will be authored with @<!-- -->noreply.github.com.
We saw several such cases in our fork of llvm. See intel/llvm#17675

Solution
This PR also checks for github.event.pull_request.user.email if it's NULL or not. If NULL, the PR will be merged with @<!-- -->noreply.github.com

---

Full diff: https://github.com/llvm/llvm-project/pull/148694.diff

1 Files Affected:

• (modified) .github/workflows/email-check.yaml (+7-1)

diff --git a/.github/workflows/email-check.yaml b/.github/workflows/email-check.yaml
index 904ad718f97dd..35cbcd3c810eb 100644
--- a/.github/workflows/email-check.yaml
+++ b/.github/workflows/email-check.yaml
@@ -26,8 +26,11 @@ jobs:
           # Create empty comment file
           echo "[]" > comments
 
+      # If author's email is hidden in GH's settings, github.event.pull_request.user.email
+      # will be null and PR will be authored by noreply.github.com.
       - name: Validate author email
-        if: ${{ endsWith(steps.author.outputs.EMAIL, 'noreply.github.com')  }}
+        if: endsWith(steps.author.outputs.EMAIL, 'noreply.github.com') ||
+              github.event.pull_request.user.email == ''
         env:
           COMMENT: >-
             ⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.<br/>
@@ -39,6 +42,9 @@ jobs:
           [{"body" : "$COMMENT"}]
           EOF
 
+          # Fail this job.
+          false
+
       - uses: actions/upload-artifact@26f96dfa697d77e81fd5907df203aa23a56210a8 #v4.3.0
         if: always()
         with:

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[uditagarwal97]

Here's an example of a PR (at llvm/llvm-project) where this workflow is passing but the PR was merged with noreply.github.com email: #148617

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[github-actions]

⚠️ We detected that you are using a GitHub private e-mail address to contribute to the repo.
Please turn off Keep my email addresses private setting in your account.
See LLVM Developer Policy and LLVM Discourse for more information.

[uditagarwal97]

PR is ready for review.
Workflow run when email is hidden: https://github.com/llvm/llvm-project/actions/runs/16279434976/job/45965879699
Workflow run when email is public: https://github.com/llvm/llvm-project/actions/runs/16279496749/job/45966062207

Based on contribution history, tagging @asl @DavidSpickett @vbvictor for feedback.

[asl]

This looks reasonable to me @tstellar @DavidSpickett @boomanaiden154 any objections?

[DavidSpickett]

My understanding of this is that before we looked at the commit and if that had a valid email address, that's enough.

The problem is that the email address used when merging is actually governed by GitHub settings not what's stored in git.

So this changes to GraphQL to do the check, but for the PR author, the way to make the email public is the same as before.

Correct?

(I ask you to confirm because at first glance the description feels like 2 layers of settings but in fact, one of them, the actual git commit, is ignored by GitHub)

[boomanaiden154]

Is there documentation on what the GraphQL query is supposed to return given the different settings?

I know Github lets me select from multiple emails when merging. I'm not sure if there's a default or how it gets setup and I would be more comfortable approving this patch with a better understanding of how these all interact.

[uditagarwal97]

My understanding of this is that before we looked at the commit and if that had a valid email address, that's enough.

The problem is that the email address used when merging is actually governed by GitHub settings not what's stored in git.

So this changes to GraphQL to do the check, but for the PR author, the way to make the email public is the same as before.

Correct?

(I ask you to confirm because at first glance the description feels like 2 layers of settings but in fact, one of them, the actual git commit, is ignored by GitHub)

@DavidSpickett Yes, that's correct. I've updated the PR description to clarify this.

[uditagarwal97]

Is there documentation on what the GraphQL query is supposed to return given the different settings?

I know Github lets me select from multiple emails when merging. I'm not sure if there's a default or how it gets setup and I would be more comfortable approving this patch with a better understanding of how these all interact.

@boomanaiden154
I think GraphQL returns your default public email. Here's an experiment I did:

Workflow run after I changed my default public email: https://github.com/llvm/llvm-project/actions/runs/16300461944/job/46033416527?pr=148694#step:3:24 (compare it with the previous run https://github.com/llvm/llvm-project/actions/runs/16279496749/job/45966062207)
Changing my default public email also changes the email returned by GraphQL.

Regarding official documentation, the closest one I found is: https://docs.github.com/en/graphql/reference/objects#user
user.email returns the user's publicly visible profile email.