Stick with protoc-{bridge,gen}_2.13 under Scala 3

mbland · mbland · commit 1880f48ab4f5 · 2024-12-18T22:30:55.000-05:00
As noted in the comment added to `select_root_artifacts()`, `com.thesamet.scalapb:compilerplugin_3:0.11.17` depends on `com.thesamet.scalapb:protoc-gen_2.13:0.9.7`. When using `com.thesamet.scalapb:protoc-gen_3:0.9.8` instead, we get a crash: ```txt $ bazel build --repo_env=SCALA_VERSION=3.6.2 //third_party/test/proto:scala [ ...snip... ] ERROR: third_party/test/proto/BUILD.bazel:4:14: ProtoScalaPBRule third_party/test/proto/proto_jvm_extra_protobuf_generator_scalapb.srcjar failed: (Exit 1): scalapb_worker failed: error executing command (from target //third_party/test/proto:proto) bazel-out/darwin_arm64-opt-exec-2B5CBBC6/bin/src/scala/scripts/scalapb_worker ... --scala_out: java.lang.NoClassDefFoundError: Could not initialize class scalapb.ScalaPbCodeGenerator$ at scripts.ScalaPbCodeGenerator$.process(ScalaPbCodeGeneratorWrapper.scala:7) at protocgen.CodeGenApp.run(CodeGenApp.scala:48) at protocgen.CodeGenApp.run$(CodeGenApp.scala:23) at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5) at protocgen.CodeGenApp.run(CodeGenApp.scala:33) at protocgen.CodeGenApp.run$(CodeGenApp.scala:23) at scripts.ScalaPbCodeGenerator$.run(ScalaPbCodeGeneratorWrapper.scala:5) at protocbridge.frontend.PluginFrontend$.runWithBytes(PluginFrontend.scala:48) at protocbridge.frontend.PluginFrontend$.runWithInputStream(PluginFrontend.scala:113) at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$1(SocketBasedPluginFrontend.scala:31) at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:37) at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1$$anon$2.block(ExecutionContextImpl.scala:60) at java.base/java.util.concurrent.ForkJoinPool.managedBlock(ForkJoinPool.java:3118) at scala.concurrent.impl.ExecutionContextImpl$DefaultThreadFactory$$anon$1.blockOn(ExecutionContextImpl.scala:71) at scala.concurrent.package$.blocking(package.scala:124) at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$1(SocketBasedPluginFrontend.scala:37) at protocbridge.frontend.SocketBasedPluginFrontend.prepare$$anonfun$adapted$1(SocketBasedPluginFrontend.scala:38) at scala.concurrent.Future$.$anonfun$apply$1(Future.scala:687) at scala.concurrent.impl.Promise$Transformation.run(Promise.scala:467) at java.base/java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1426) at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290) at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020) at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656) at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594) at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183) java.lang.RuntimeException: Exit with code 1 at scala.sys.package$.error(package.scala:27) at scripts.ScalaPBWorker$.work(ScalaPBWorker.scala:44) at io.bazel.rulesscala.worker.Worker.persistentWorkerMain(Worker.java:96) at io.bazel.rulesscala.worker.Worker.workerMain(Worker.java:49) at scripts.ScalaPBWorker$.main(ScalaPBWorker.scala:39) at scripts.ScalaPBWorker.main(ScalaPBWorker.scala) Target //third_party/test/proto:scala failed to build ERROR: test/proto/BUILD.bazel:4:14 scala @//third_party/test/proto:proto failed: (Exit 1): scalapb_worker failed: error executing command (from target //third_party/test/proto:proto) ``` --- As I was going back and testing some more, I couldn't reproduce the `ProtoScalaPBRule` hanging by setting `protoc-bridge` to 0.9.7 and `protobuf` to v26.1. I was using this command (using Bazel 7.4.1 to avoid setting C++ compiler flags for Bazel 6.5.0): ```txt USE_BAZEL_VERSION=7.4.1 bazel build //third_party/test/proto:scala ``` The command kept crashing every time, instead of hanging. I eventually realized this was because of two things: - `scalarules.test.extra_protobuf_generator.ExtraProtobufGenerator.run` was still catching `Throwable`, since Scala 2.11 support ends with `protoc-bridge` 0.7.14. - Bazel kept persistent `io.bazel.rulesscala.scalac.ScalacWorker` workers running with `ExtraProtobufGenerator` code that still had the `try`/`catch` block after I tried removing it. After temporarily editing out the `try`/`catch` block from `ExtraProtobufGenerator.run` and running `bazel shutdown`, I could reproduce the hang. I could then confirm `protoc-bridge` 0.9.8 fixed it. --- Also, TIL that `scripts.ScalaPbCodeGenerator.process` (from `src/scala/scripts/ScalaPbCodeGeneratorWrapper`) never needed a `try`/`catch` block. When bumping to `com.thesamet.scalapb:compilerplugin_2.12:0.11.17` in bazel-contrib#1648 (commit 23ae356), `scalapb.ScalaPbCodeGenerator` went from implementing `protocbridge.ProtocCodeGenerator` to implementing `protocgen.CodeGenApp`: - https://github.com/scalapb/ScalaPB/blob/v0.9.7/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L9 - https://github.com/scalapb/ScalaPB/blob/v0.11.17/compiler-plugin/src/main/scala/scalapb/ScalaPbCodeGenerator.scala#L11 That commit introduced `com.thesamet.scalapb:protoc-gen_2.12:0.9.7`, and `protocgen.CodeGenApp.run()` already had a `try`/`catch` block for `Throwable`: - https://github.com/scalapb/protoc-bridge/blob/v0.9.7/protoc-gen/src/main/scala/protocgen/CodeGenApp.scala#L49-L56 However, `protocbridge.ProtocCodeGenerator`, which `protocgen.CodeGenApp` extends, only declares a `run()` method with no implementation (and no `try`/`catch` block): - https://github.com/scalapb/protoc-bridge/blob/v0.9.7/bridge/src/main/scala/protocbridge/ProtocCodeGenerator.scala#L5 The aforementioned `ExtraProtobufGenerator` only extends `protocbridge.ProtocCodeGenerator`, not `protocgen.CodeGenApp`. That's why it will still hang when we remove the `try`/`catch` block from its `run()` method and run with `protoc-bridge` 0.9.7 and `protobuf` > v25.5.
diff --git a/scripts/create_repository.py b/scripts/create_repository.py
@@ -85,7 +85,15 @@ def select_root_artifacts(scala_version, scala_major, is_scala_3) -> List[str]:
         scala_2_version = max_scala_2_version
         scala_2_major = max_scala_2_major
         scala_major = '3'
-        scalapb_major = max_scala_2_major if minor_version < 3 else scala_major
+        scalapb_major = scala_2_major if minor_version < 3 else scala_major
+
+    # For some reason, com.thesamet.scalapb:compilerplugin_3:0.11.17 depends on
+    # com.thesamet.scalapb:protoc-gen_2.13:0.9.7. Trying to use
+    # com.thesamet.scalapb:protoc-gen_3:0.9.8 causes a crash, saying:
+    # `java.lang.NoClassDefFoundError: Could not initialize class
+    # scalapb.ScalaPbCodeGenerator$`, even though that class is definitely in
+    # that jar. So we stick with protoc-gen_2.13 for now.
+    protoc_bridge_major = scala_2_major
 
     scalafmt_version = SCALAFMT_VERSION
     scalapb_version = SCALAPB_VERSION
@@ -103,7 +111,7 @@ def select_root_artifacts(scala_version, scala_major, is_scala_3) -> List[str]:
         f'com.google.protobuf:protobuf-java:{PROTOBUF_JAVA_VERSION}',
         f'com.thesamet.scalapb:compilerplugin_{scalapb_major}:' +
             scalapb_version,
-        f'com.thesamet.scalapb:protoc-bridge_{scalapb_major}:' +
+        f'com.thesamet.scalapb:protoc-bridge_{protoc_bridge_major}:' +
             protoc_bridge_version,
         f'com.thesamet.scalapb:scalapb-runtime_{scalapb_major}:' +
             scalapb_version,
@@ -123,7 +131,7 @@ def select_root_artifacts(scala_version, scala_major, is_scala_3) -> List[str]:
 
     if scala_major != '2.11':
         root_artifacts.append(
-            f'com.thesamet.scalapb:protoc-gen_{scalapb_major}:' +
+            f'com.thesamet.scalapb:protoc-gen_{protoc_bridge_major}:' +
             protoc_bridge_version,
         )
 
diff --git a/third_party/repositories/scala_3_3.bzl b/third_party/repositories/scala_3_3.bzl
@@ -880,18 +880,18 @@ artifacts = {
         ],
     },
     "scala_proto_rules_scalapb_protoc_bridge": {
-        "artifact": "com.thesamet.scalapb:protoc-bridge_3:0.9.8",
-        "sha256": "6e1e38e34f3aaa14c6d46defb66b819f03edbdc4d69965011955da2a4781df9c",
+        "artifact": "com.thesamet.scalapb:protoc-bridge_2.13:0.9.8",
+        "sha256": "0b3827da2cd9bca867d6963c2a821e7eaff41f5ac3babf671c4c00408bd14a9b",
         "deps": [
             "@dev_dirs_directories",
-            "@io_bazel_rules_scala_scala_library",
+            "@io_bazel_rules_scala_scala_library_2",
         ],
     },
     "scala_proto_rules_scalapb_protoc_gen": {
-        "artifact": "com.thesamet.scalapb:protoc-gen_3:0.9.8",
-        "sha256": "9e5eebe35ca68a884adf3c6cade094055192d3243d29a26d32ae6b5396b39e08",
+        "artifact": "com.thesamet.scalapb:protoc-gen_2.13:0.9.8",
+        "sha256": "cf2b50721952cb4f10ca05a0ed36d7b01b88eb6505a9478556ee5a7af1a21775",
         "deps": [
-            "@io_bazel_rules_scala_scala_library",
+            "@io_bazel_rules_scala_scala_library_2",
             "@scala_proto_rules_scalapb_protoc_bridge",
         ],
     },
diff --git a/third_party/repositories/scala_3_4.bzl b/third_party/repositories/scala_3_4.bzl
@@ -880,18 +880,18 @@ artifacts = {
         ],
     },
     "scala_proto_rules_scalapb_protoc_bridge": {
-        "artifact": "com.thesamet.scalapb:protoc-bridge_3:0.9.8",
-        "sha256": "6e1e38e34f3aaa14c6d46defb66b819f03edbdc4d69965011955da2a4781df9c",
+        "artifact": "com.thesamet.scalapb:protoc-bridge_2.13:0.9.8",
+        "sha256": "0b3827da2cd9bca867d6963c2a821e7eaff41f5ac3babf671c4c00408bd14a9b",
         "deps": [
             "@dev_dirs_directories",
-            "@io_bazel_rules_scala_scala_library",
+            "@io_bazel_rules_scala_scala_library_2",
         ],
     },
     "scala_proto_rules_scalapb_protoc_gen": {
-        "artifact": "com.thesamet.scalapb:protoc-gen_3:0.9.8",
-        "sha256": "9e5eebe35ca68a884adf3c6cade094055192d3243d29a26d32ae6b5396b39e08",
+        "artifact": "com.thesamet.scalapb:protoc-gen_2.13:0.9.8",
+        "sha256": "cf2b50721952cb4f10ca05a0ed36d7b01b88eb6505a9478556ee5a7af1a21775",
         "deps": [
-            "@io_bazel_rules_scala_scala_library",
+            "@io_bazel_rules_scala_scala_library_2",
             "@scala_proto_rules_scalapb_protoc_bridge",
         ],
     },
diff --git a/third_party/repositories/scala_3_5.bzl b/third_party/repositories/scala_3_5.bzl
@@ -880,18 +880,18 @@ artifacts = {
         ],
     },
     "scala_proto_rules_scalapb_protoc_bridge": {
-        "artifact": "com.thesamet.scalapb:protoc-bridge_3:0.9.8",
-        "sha256": "6e1e38e34f3aaa14c6d46defb66b819f03edbdc4d69965011955da2a4781df9c",
+        "artifact": "com.thesamet.scalapb:protoc-bridge_2.13:0.9.8",
+        "sha256": "0b3827da2cd9bca867d6963c2a821e7eaff41f5ac3babf671c4c00408bd14a9b",
         "deps": [
             "@dev_dirs_directories",
-            "@io_bazel_rules_scala_scala_library",
+            "@io_bazel_rules_scala_scala_library_2",
         ],
     },
     "scala_proto_rules_scalapb_protoc_gen": {
-        "artifact": "com.thesamet.scalapb:protoc-gen_3:0.9.8",
-        "sha256": "9e5eebe35ca68a884adf3c6cade094055192d3243d29a26d32ae6b5396b39e08",
+        "artifact": "com.thesamet.scalapb:protoc-gen_2.13:0.9.8",
+        "sha256": "cf2b50721952cb4f10ca05a0ed36d7b01b88eb6505a9478556ee5a7af1a21775",
         "deps": [
-            "@io_bazel_rules_scala_scala_library",
+            "@io_bazel_rules_scala_scala_library_2",
             "@scala_proto_rules_scalapb_protoc_bridge",
         ],
     },
diff --git a/third_party/repositories/scala_3_6.bzl b/third_party/repositories/scala_3_6.bzl
@@ -889,18 +889,18 @@ artifacts = {
         ],
     },
     "scala_proto_rules_scalapb_protoc_bridge": {
-        "artifact": "com.thesamet.scalapb:protoc-bridge_3:0.9.8",
-        "sha256": "6e1e38e34f3aaa14c6d46defb66b819f03edbdc4d69965011955da2a4781df9c",
+        "artifact": "com.thesamet.scalapb:protoc-bridge_2.13:0.9.8",
+        "sha256": "0b3827da2cd9bca867d6963c2a821e7eaff41f5ac3babf671c4c00408bd14a9b",
         "deps": [
             "@dev_dirs_directories",
-            "@io_bazel_rules_scala_scala_library",
+            "@io_bazel_rules_scala_scala_library_2",
         ],
     },
     "scala_proto_rules_scalapb_protoc_gen": {
-        "artifact": "com.thesamet.scalapb:protoc-gen_3:0.9.8",
-        "sha256": "9e5eebe35ca68a884adf3c6cade094055192d3243d29a26d32ae6b5396b39e08",
+        "artifact": "com.thesamet.scalapb:protoc-gen_2.13:0.9.8",
+        "sha256": "cf2b50721952cb4f10ca05a0ed36d7b01b88eb6505a9478556ee5a7af1a21775",
         "deps": [
-            "@io_bazel_rules_scala_scala_library",
+            "@io_bazel_rules_scala_scala_library_2",
             "@scala_proto_rules_scalapb_protoc_bridge",
         ],
     },
