Case companions no longer extend `FunctionN` under `-Xsource:3-cross`, breaking binary compatibility

[lrytz]

2.13.13 aligned synthetic case companions with Scala 3 to no longer extend FunctionN under -Xsource:3-cross (scala/scala#10648).

Projects that already have cross-published releases for Scala 2.13 and 3.x cannot adopt -Xsource:3-cross because this change is not binary compatible.

The workaround of manually writing out object C extends FunctionN is not a solution because that source change would also affect Scala 3, causing a binary incompatibility there. (It would also be very tedious.)

This situation can be a blocker for libraries to upgrade to Scala 2.13.13. Example:

• On 2.13.12 with -Xsource:3 -Xmigration, the escape inraw"\u0042" is not processed
• In 2.13.13 all semantic changes are moved to -Xsource:3-cross. A project using -Xsource:3 needs to move to -Xsource:3-cross to maintain semantics. With -Xsource:3, the escape is now being processed.
• The binary incompatibility of case companions comes in the way.

A more general question is how do we add further language changes to -Xsource:3-cross in the future. Every semantic change comes with a warning under -Xsource:3. But projects that are already on 3-cross would silently get new semantics in a new Scala version.

[lrytz]

@SethTisue mentioned the possibility of having separate flags for language changes adopted from Scala 3. This would help here:

• projects can opt in / out of the case companion parent change
• new language changes adopted in the future would not be enabled by default

So -Xsource:3 to enable warnings (like we did in 2.13.13), something like -Xscala-3-semantics:caseCompanionParents,unicodeInterpolators,... to adopt changes and disable the corresponding warnings. -Xsource:3-cross would be removed.

[SethTisue]

-Xsource:3-cross would be removed

Oh, I wasn't thinking of going that far! I was thinking something like -Xsource3-cross:-caseCompanionParents, by analogy with e.g. -Xlint:-byname-implicit

[som-snytt]

3-cross is for projects that want to cross-compile, not for projects that can't cross-compile.

Sometimes you need versioned sources; that is the mechanism.

We did not have a discussion, but I avoided the fine-grained feature selection as in -Xlint, which is a hassle and finicky. (To say more, -Xlint has opt-out as a last resort; you're not supposed to need it in normal operation because these are recommended warnings. Similarly, -Xsource:3 means you're eagerly migrating and you want to make any adaptations early in the migration process.)

Arguably, dotty -source:foo-migration is a coarse-grained mechanism, but it's not obvious what to call such "virtual future versions" in 2.13.x. That would be rolling cumulative changes and not pick-and-choose.

I agree about silent semantics in 3-cross; I might have suggested a verbose mode for that so you can audit your build. Currently, the obvious thing is that it should emit scala3-migration warnings that are silenced by default, so you just have to enable them to do an audit.

The purpose of future source migration is to shake out the code base for migration to minimize pain as much as possible. (Pain could be maintaining two source versions or the pain of switching to dotty at some future point in time.) It's not a feature buffet.

[SethTisue]

3-cross is for projects that want to cross-compile, not for projects that can't cross-compile.

I don't think I understand your point here?

Sometimes you need versioned sources

In an ordinary project, though, it would incredibly cumbersome to have versioned sources for all of your case classes, so I don't think that's a satisfying answer for this particular issue (case class companion parentage).

[SethTisue]

oh btw big thanks to @mkurz for discovering and reporting this promptly (over at playframework/playframework#12401)

[lrytz]

source:3 is about minimizing the pain of migrating, source:3-cross about reducing some of the pain of cross-building. Projects will probably be cross-building for a very long time, so it's certainly worth investing here and trying to get it right.

[som-snytt]

Sample investment would be scala/scala#10648 "For migration to 3, accommodate case companion as function".

Sample ticket where accommodation may (arguably) be unreasonable (because they are interested in piecemeal language forking): #12960

If we wanted forking options, we would introduce -Y. (For that ticket, there is -Yno-predef already.)

If MyLib_2.13 is at v1.0, it's reasonable to say MyLib_2.13 crossbuilt as MyLib_3 is v2.0 for binary reasons (if nothing else). (Whatever the compatibility policy of MyLib happens to be; and for "end users" there may be no policy. They just want to leverage "early adoption".)

3-cross is for projects that want to cross-compile, not for projects that can't cross-compile.

I don't think I understand your point here?

"I don't want to lose case companion parentage" is incompatible with "I want one version of my case class that cross-compiles."

There's nothing wrong with, "We can't use -Xsource because there are too many differences. If we want to use Scala 3 later, we'll cross that bridge when we can see the river gorge from a distance."

[SethTisue]

"I don't want to lose case companion parentage" is incompatible with "I want one version of my case class that cross-compiles."

Ah, I see. Yes, good point. I strongly agree that we should keep in mind that the purpose of -Xsource:3-cross is to enable cross-compilation, not to give people a la carte access to bits and pieces of Scala 3.

However, in this particular situation I think offering a la carte access might be justifiable, because we would be adding it exactly to allow people who were already cross-building to continue to cross-build.

[som-snytt]

My other basic question is, why is "case companion breaks binary compat" different from "method result inference breaks binary compat". Both require tweaking source code (adding an object or an explicit type).

Maybe -Xsource:3-safe or 3-compat to turn off those "features which might break binary compat".

[SethTisue]

My other basic question is, why is "case companion breaks binary compat" different from "method result inference breaks binary compat". Both require tweaking source code (adding an object or an explicit type).

This case companion issue is different because there is no workaround (besides the unreasonably cumbersome version-specific sources option, I mean). You cannot work around it by tweaking cross-compiled source code. At least, it didn't seem to me and Lukas that a workaround exists — are we mistaken?

As Lukas wrote above,

The workaround of manually writing out object C extends FunctionN is not a solution because that source change would also affect Scala 3, causing a binary incompatibility there. (It would also be very tedious.)