Skip to content

assertion failed: class Array while introspecting Array[Byte] in a macro #21916

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
pshirshov opened this issue Nov 9, 2024 · 6 comments · Fixed by #22033
Closed

assertion failed: class Array while introspecting Array[Byte] in a macro #21916

pshirshov opened this issue Nov 9, 2024 · 6 comments · Fixed by #22033
Assignees
Labels
area:metaprogramming:other Issues tied to metaprogramming/macros not covered by the other labels. itype:bug itype:crash
Milestone

Comments

@pshirshov
Copy link
Contributor

pshirshov commented Nov 9, 2024

I had a feeling like we already reported this, but can't find anything.

The compiler fails while izumi-reflect introspects Array[Byte] with an obscure stacktrace:

java.lang.AssertionError: assertion failed: class Array
	at scala.runtime.Scala3RunTime$.assertFailed(Scala3RunTime.scala:8)
	at dotty.tools.backend.jvm.BCodeHelpers.primitiveOrClassToBType$1(BCodeHelpers.scala:717)
	at dotty.tools.backend.jvm.BCodeHelpers.dotty$tools$backend$jvm$BCodeHelpers$$typeToTypeKind(BCodeHelpers.scala:737)
	at dotty.tools.backend.jvm.BCodeHelpers$BCInnerClassGen.toTypeKind(BCodeHelpers.scala:202)
	at dotty.tools.backend.jvm.BCodeHelpers$BCInnerClassGen.toTypeKind$(BCodeHelpers.scala:130)
	at dotty.tools.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder.toTypeKind(BCodeSkelBuilder.scala:134)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genConstant(BCodeBodyBuilder.scala:581)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadTo(BCodeBodyBuilder.scala:450)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoad(BCodeBodyBuilder.scala:303)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.loop$1(BCodeBodyBuilder.scala:1209)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadArguments(BCodeBodyBuilder.scala:1216)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genApply(BCodeBodyBuilder.scala:835)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadTo(BCodeBodyBuilder.scala:385)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadTo(BCodeBodyBuilder.scala:465)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoad(BCodeBodyBuilder.scala:303)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadTo(BCodeBodyBuilder.scala:320)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoad(BCodeBodyBuilder.scala:303)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genStat(BCodeBodyBuilder.scala:117)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genBlockTo$$anonfun$1(BCodeBodyBuilder.scala:1093)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genBlockTo(BCodeBodyBuilder.scala:1093)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadTo(BCodeBodyBuilder.scala:457)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoad(BCodeBodyBuilder.scala:303)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.loop$1(BCodeBodyBuilder.scala:1209)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadArguments(BCodeBodyBuilder.scala:1216)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genApply(BCodeBodyBuilder.scala:835)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadTo(BCodeBodyBuilder.scala:385)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoad(BCodeBodyBuilder.scala:303)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genStat(BCodeBodyBuilder.scala:117)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genBlockTo$$anonfun$1(BCodeBodyBuilder.scala:1093)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genBlockTo(BCodeBodyBuilder.scala:1093)
	at dotty.tools.backend.jvm.BCodeBodyBuilder$PlainBodyBuilder.genLoadTo(BCodeBodyBuilder.scala:457)
	at dotty.tools.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder.emitNormalMethodBody$1(BCodeSkelBuilder.scala:893)
	at dotty.tools.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder.genDefDef(BCodeSkelBuilder.scala:916)
	at dotty.tools.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder.gen(BCodeSkelBuilder.scala:693)
	at dotty.tools.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder.gen$$anonfun$1(BCodeSkelBuilder.scala:699)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at dotty.tools.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder.gen(BCodeSkelBuilder.scala:699)
	at dotty.tools.backend.jvm.BCodeSkelBuilder$PlainSkelBuilder.genPlainClass(BCodeSkelBuilder.scala:293)
	at dotty.tools.backend.jvm.CodeGen.genClass(CodeGen.scala:160)
	at dotty.tools.backend.jvm.CodeGen.genClassDef$1(CodeGen.scala:62)
	at dotty.tools.backend.jvm.CodeGen.genClassDefs$1(CodeGen.scala:117)
	at dotty.tools.backend.jvm.CodeGen.genClassDefs$1$$anonfun$1(CodeGen.scala:115)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at dotty.tools.backend.jvm.CodeGen.genClassDefs$1(CodeGen.scala:115)
	at dotty.tools.backend.jvm.CodeGen.genUnit(CodeGen.scala:120)
	at dotty.tools.backend.jvm.GenBCode.run(GenBCode.scala:90)
	at dotty.tools.dotc.core.Phases$Phase.runOn$$anonfun$1(Phases.scala:380)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.immutable.List.foreach(List.scala:334)
	at dotty.tools.dotc.core.Phases$Phase.runOn(Phases.scala:373)
	at dotty.tools.backend.jvm.GenBCode.runOn(GenBCode.scala:98)
	at dotty.tools.dotc.Run.runPhases$1$$anonfun$1(Run.scala:343)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:15)
	at scala.runtime.function.JProcedure1.apply(JProcedure1.java:10)
	at scala.collection.ArrayOps$.foreach$extension(ArrayOps.scala:1323)
	at dotty.tools.dotc.Run.runPhases$1(Run.scala:336)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$1(Run.scala:384)
	at dotty.tools.dotc.Run.compileUnits$$anonfun$adapted$1(Run.scala:396)
	at dotty.tools.dotc.util.Stats$.maybeMonitored(Stats.scala:69)
	at dotty.tools.dotc.Run.compileUnits(Run.scala:396)
	at dotty.tools.dotc.Run.compileSources(Run.scala:282)
	at dotty.tools.dotc.Run.compile(Run.scala:267)
	at dotty.tools.dotc.Driver.doCompile(Driver.scala:37)
	at dotty.tools.xsbt.CompilerBridgeDriver.run(CompilerBridgeDriver.java:141)
	at dotty.tools.xsbt.CompilerBridge.run(CompilerBridge.java:22)
	at sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:91)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$7(MixedAnalyzingCompiler.scala:193)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
	at sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:248)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4(MixedAnalyzingCompiler.scala:183)
	at sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$4$adapted(MixedAnalyzingCompiler.scala:163)
	at sbt.internal.inc.JarUtils$.withPreviousJar(JarUtils.scala:239)
	at sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:163)
	at sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:211)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.Incremental$.$anonfun$apply$5(Incremental.scala:180)
	at sbt.internal.inc.Incremental$.$anonfun$apply$5$adapted(Incremental.scala:178)
	at sbt.internal.inc.Incremental$$anon$2.run(Incremental.scala:464)
	at sbt.internal.inc.IncrementalCommon$CycleState.next(IncrementalCommon.scala:116)
	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:56)
	at sbt.internal.inc.IncrementalCommon$$anon$1.next(IncrementalCommon.scala:52)
	at sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:263)
	at sbt.internal.inc.Incremental$.$anonfun$incrementalCompile$8(Incremental.scala:419)
	at sbt.internal.inc.Incremental$.withClassfileManager(Incremental.scala:506)
	at sbt.internal.inc.Incremental$.incrementalCompile(Incremental.scala:406)
	at sbt.internal.inc.Incremental$.apply(Incremental.scala:172)
	at sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:534)
	at sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:488)
	at sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:332)
	at sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:425)
	at sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:137)
	at sbt.Defaults$.compileIncrementalTaskImpl(Defaults.scala:2371)
	at sbt.Defaults$.$anonfun$compileIncrementalTask$2(Defaults.scala:2321)
	at sbt.internal.server.BspCompileTask$.$anonfun$compute$1(BspCompileTask.scala:31)
	at sbt.internal.io.Retry$.apply(Retry.scala:47)
	at sbt.internal.io.Retry$.apply(Retry.scala:29)
	at sbt.internal.io.Retry$.apply(Retry.scala:24)
	at sbt.internal.server.BspCompileTask$.compute(BspCompileTask.scala:31)
	at sbt.Defaults$.$anonfun$compileIncrementalTask$1(Defaults.scala:2319)
	at scala.Function1.$anonfun$compose$1(Function1.scala:49)
	at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:63)
	at sbt.std.Transform$$anon$4.work(Transform.scala:69)
	at sbt.Execute.$anonfun$submit$2(Execute.scala:283)

	at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:24)

Here is the repro: https://scastie.scala-lang.org/3P7O5JyuSaaEzHQ9WUGoGQ

It fails on any Scala 3 version.

It looks like it's a compiler bug as it only happens with Arrays. If we improperly use the API, the diagnostic message should be improved, at the moment it's completely misleading.

Once this is resolved, zio/izumi-reflect#474 might be closed and the bounty should remain unclaimed.

@pshirshov pshirshov added itype:bug stat:needs triage Every issue needs to have an "area" and "itype" label labels Nov 9, 2024
@Gedochao Gedochao added itype:crash stat:needs minimization Needs a self contained minimization area:metaprogramming:other Issues tied to metaprogramming/macros not covered by the other labels. and removed stat:needs triage Every issue needs to have an "area" and "itype" label labels Nov 12, 2024
@Gedochao
Copy link
Contributor

Scala CLI repro:

//> using dep dev.zio::izumi-reflect:2.3.10
@main def main() = izumi.reflect.Tag.tagFromTagMacro[Array[Byte]]

We'll need this minimized without the izumi-reflect dependency.

@pshirshov
Copy link
Contributor Author

pshirshov commented Nov 12, 2024

I don't think I can do it. We have a stable repro and a reasonable stacktrace. Although this bug seem to be happening lot later than the macro phase and i-r macros are ones of the most complicated ones which ever existed for Scala. Moreover, from what I can remember, there were similarly looking bugs which were affected by mere syntactic rearrangements.

So, for me the effort required for further minimization looks insurmountable.

@jchyb
Copy link
Contributor

jchyb commented Nov 22, 2024

Minimisation:
Macro.scala:

import scala.quoted._

object Macro:
  inline def test() = ${testImpl}
  def testImpl(using Quotes): Expr[Any] = {
    import quotes.reflect._
    val tpe = TypeRepr.of[Array[Byte]] match
      case AppliedType(tycons, _) => tycons
    Literal(ClassOfConstant(tpe)).asExpr
  }

Main.scala:

@main def main() = Macro.test()

We need to add -Xcheck-macros check for parameters of classOf[] (and perhaps other type parameter lists too) - though for classOf this probably only matters for Arrays

As a side note, for anyone struggling with similar issues, calling macro methods with -Xprint:inlining scalac option is always useful for noticing any incoherency in the outputted code.

@jchyb jchyb removed the stat:needs minimization Needs a self contained minimization label Nov 25, 2024
hamzaremmal added a commit that referenced this issue Feb 21, 2025
…#22033)

Closes #21916
I tried to supply the ClassOfConstant with multiple other broken Types,
but I was unable to break it beyond the linked issue, so I ended up
adding the check for only that one case. This makes sense - the backend
(and thus erasure) needs to know if the Array type parameter is a
primitive type, but in other cases the erasure phase needs to know only
the class, without the type parameters.

It's impossible to call classOf through the quoted code (`'{classOf[t]}`
with a boundless t will error out), so we don't need that additional
check there.

There does appear to be an issue with being able to set `'{List[Array]}`
resulting in a crash, but that is beyond the scope of this fix - I will
prepare a separate issue for that (edit: reported
[here](#22034)).
@pshirshov
Copy link
Contributor Author

@jchyb : there is an open bounty for this: zio/izumi-reflect#474 , you may claim it if you wish. You can't open an empty P/R, but a single-line test would suffice.

@jchyb
Copy link
Contributor

jchyb commented Feb 21, 2025

@pshirshov, I'd rather not, especially that we only replaced the crash with an error, rather than doing something that will completely fix the issue with izumi-reflect just by bumping the compiler version (we can't really do that). The problem still remains in izumi-reflect, the difference is now you should just get a more actionable compiletime error (instead of a confusing crash).

The problem is that currently izumi-reflect can generate something like:
classOf[Array],
a class of with a type constructor without any type parameters, which is nonsensical, and impossible to do in the usual non-macro code. Arrays here are particularly problematic, as the backend-generating part of the compiler has to know whether it contains a primitive type or an object to correctly map it to the JVM structures (lack of that parameter previously lead to the crash, now throwing an error).

For now, the added check in the compiler only throws errors for Arrays, but, preferably, things like classOf[List] should also throw an error here (as that is still illegal to do in non-macro code). We decided against that for the time being, as we feared this would break the current zio/izumi-reflect powered ecosystem (since I noticed that in the izumi-reflect codebase all type parameters are stripped from the type constructor before using classOf). For that reason I am not particularly fond of the fix here. Ideally, the type parameters would not be stripped at all so that correct classOf code could always be constructed. WIth that said, the linked PR will fix the immediate crash/error, with perhaps the only problem being that I suspect the runtimeClass for any array will be an ObjectArray, even when we have a something like Tag[Array[Int]] (where we would expect an IntArray, etc.)

@neko-kai
Copy link
Contributor

@jchyb

The problem still remains in izumi-reflect

Indeed. I didn't understand your initial message in the issue thread – assuming the type parameters were somehow lost by the compiler, not in the code itself – since we weren't intentionally stripping them ourselves. Instead they were lost because we were directly using the output of TypeRepr#baseClasses when computing the LUB instead of using TypeRepr#baseType afterwards to recover the type parameters. (fix)

Ideally, the type parameters would not be stripped at all so that correct classOf code could always be constructed.

I think we can't do that, because we generate Tags for unapplied HKTs as well - which don't have applied type parameters and recovering which might be quite tricky depending on their bounds. Note that generating classOfs for unapplied types works on Scala 2 as well, so there's some continuity in this behavior. With the above fix, type parameters will now be retained for proper types, but there will be still be unapplied classOfs for HKT tags.

@WojciechMazur WojciechMazur added this to the 3.7.0 milestone Mar 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:metaprogramming:other Issues tied to metaprogramming/macros not covered by the other labels. itype:bug itype:crash
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants