Skip to content

Commit f4bb652

Browse files
committed
[SPARK-47115][INFRA] Use larger memory for Maven builds
### What changes were proposed in this pull request? This PR proposes to use bigger memory during Maven builds. GitHub Actions runners now have more memory than before (https://docs.github.com/en/actions/using-github-hosted-runners/about-larger-runners/about-larger-runners) so we can increase. https://github.com/HyukjinKwon/spark/actions/runs/7984135094/job/21800463337 ### Why are the changes needed? For stable Maven builds. Some tests consistently fail: ``` *** RUN ABORTED *** An exception or error caused a run to abort: unable to create native thread: possibly out of memory or process/resource limits reached java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached at java.base/java.lang.Thread.start0(Native Method) at java.base/java.lang.Thread.start(Thread.java:1553) at java.base/java.lang.System$2.start(System.java:2577) at java.base/jdk.internal.vm.SharedThreadContainer.start(SharedThreadContainer.java:152) at java.base/java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:953) at java.base/java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1364) at org.apache.spark.rpc.netty.SharedMessageLoop.$anonfun$threadpool$1(MessageLoop.scala:128) at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:190) at org.apache.spark.rpc.netty.SharedMessageLoop.<init>(MessageLoop.scala:127) at org.apache.spark.rpc.netty.Dispatcher.sharedLoop$lzycompute(Dispatcher.scala:46) ... Warning: The requested profile "volcano" could not be activated because it does not exist. Warning: The requested profile "hive" could not be activated because it does not exist. Error: Failed to execute goal org.scalatest:scalatest-maven-plugin:2.2.0:test (test) on project spark-core_2.13: There are test failures -> [Help 1] Error: Error: To see the full stack trace of the errors, re-run Maven with the -e switch. Error: Re-run Maven using the -X switch to enable full debug logging. Error: Error: For more information about the errors and possible solutions, please read the following articles: Error: [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException Error: Error: After correcting the problems, you can resume the build with the command Error: mvn <args> -rf :spark-core_2.13 Error: Process completed with exit code 1. ``` ### Does this PR introduce _any_ user-facing change? No, dev-only ### How was this patch tested? Will monitor the scheduled jobs. It's a simple memory configuration change. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #45195 from HyukjinKwon/bigger-macos. Authored-by: Hyukjin Kwon <[email protected]> Signed-off-by: Hyukjin Kwon <[email protected]>
1 parent 011b843 commit f4bb652

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

.github/workflows/maven_test.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -185,7 +185,7 @@ jobs:
185185
- name: Run tests
186186
env: ${{ fromJSON(inputs.envs) }}
187187
run: |
188-
export MAVEN_OPTS="-Xss64m -Xmx4g -Xms4g -XX:ReservedCodeCacheSize=128m -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
188+
export MAVEN_OPTS="-Xss64m -Xmx6g -Xms6g -XX:ReservedCodeCacheSize=128m -Dorg.slf4j.simpleLogger.defaultLogLevel=WARN"
189189
export MAVEN_CLI_OPTS="--no-transfer-progress"
190190
export JAVA_VERSION=${{ matrix.java }}
191191
# Replace with the real module name, for example, connector#kafka-0-10 -> connector/kafka-0-10

0 commit comments

Comments
 (0)