[SPARK-49097][INFRA] Add Python3 environment detection for the build_error_docs method in build_api_decs.rb
#47590
Conversation
yaooqinn
changed the title
build_orror_docs method in build_api_decs.rbbuild_error_docs method in build_api_decs.rb
| raise("Missing python3 in your path, stopping error doc generation") | ||
| end | ||
|
|
||
| system("python3 '#{SPARK_PROJECT_ROOT}/docs/util/build-error-docs.py'") \ |
There was a problem hiding this comment.
Since users may not use python venv, I suggest changing to python3 here.
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
It seems the failure is relevant to this change.
Could you make CI happy, @wayneguow ?
wayneguow
changed the title
build_error_docs method in build_api_decs.rbbuild_error_docs method in build_api_decs.rb
| # We need this link because the jekyll build calls `python`. | ||
| ln -s "$(which python3.9)" "/usr/local/bin/python" | ||
| # We need this link to make sure `python3` points to `python3.9` which contains the prerequisite packages. | ||
| ln -s "$(which python3.9)" "/usr/local/bin/python3" |
There was a problem hiding this comment.
Currently, there are multiple pythons in the ci env：
- Python 3.9.19
- Python 3.10.12
And python3 points toPython 3.10.12.
wayneguow
changed the title
build_error_docs method in build_api_decs.rbbuild_error_docs method in build_api_decs.rb
Well, I found out the reason, because python3 currently does not point to the python3.9 with the installation packages we need. |
.github/workflows/build_and_test.yml
Outdated
| - name: Run documentation build | ||
| run: | | ||
| # We need this link because the jekyll build calls `python`. | ||
| ln -s "$(which python3.9)" "/usr/local/bin/python" |
There was a problem hiding this comment.
Can we remove the above two lines from SPARK-46935, @wayneguow ?
There was a problem hiding this comment.
Yes, good advice, this link is indeed introduced in this PR #44971 and only used in this place.
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you, @wayneguow and @HyukjinKwon .
Merged to master.
3 participants
What changes were proposed in this pull request?
This PR aims to add Python3 environment detection for the
build_orror_docsmethod inbuild_api_decs.rb.Why are the changes needed?
Make the environment exception prompts more friendly for developers when generating documents.
Before:
After:
Does this PR introduce any user-facing change?
No.
How was this patch tested?
Pass GA.
Was this patch authored or co-authored using generative AI tooling?
No.