Skip to content

Commit 0cb14cd

Browse files
bishaboshabishabosha
committed
Add blog about Tasty Reader
1 parent ad68d54 commit 0cb14cd

File tree

1 file changed

+388
-0
lines changed

1 file changed

+388
-0
lines changed

‎blog/_posts/2020-10-12-scala-3-forward-compat.md

Lines changed: 388 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,388 @@
1+
---
2+
layout: blog-detail
3+
post-type: blog
4+
by: Jamie Thompson, Scala Center
5+
title: Forward Compatibility for the Scala 3 Transition
6+
---
7+
# Forward Compatibility for the Scala 3 Transition
8+
Scala 3 is coming, with a release candidate slated for the end of 2020.
9+
With that knowledge comes the inevitable question:
10+
should I migrate, and what is the potential cost?
11+
12+
With this blog we would like to outline some scenarios for maintainers of projects who
13+
want to move to Scala 3, and how these
14+
scenarios are made simpler by the ability to read Scala 3 compiled dependencies
15+
from Scala 2.13. There is also a [troubleshooting](#troubleshooting) section
16+
in case you encounter issues following the steps described in the scenarios.
17+
18+
>As a quick aside, If you would like to know how it is possible to mix
19+
>dependencies between the two compilers, watch the talk
20+
>[Taste the difference with Scala 3](https://www.youtube.com/watch?v=YQmVrUdx8TU).
21+
>
22+
>To summarise: Scala 3 stores meta information about the code it compiles in
23+
>TASTy files, and Scala 2.13 can read these files to learn, for example,
24+
>which terms, types and implicits are defined in a given dependency,
25+
>and what code needs to be generated to use them correctly.
26+
27+
## Scenarios
28+
29+
1. You have an application consisting of many subprojects that may depend on
30+
each other, built with Scala 2.13, and would like to transition each one
31+
independently to Scala 3.0
32+
2. You have an application, built with Scala 2.13, and want to use some
33+
new features of a library that has migrated to Scala 3.0
34+
35+
Let's take a look at each scenario in turn:
36+
37+
## Migrating a multi module project
38+
39+
If you want to migrate a multi module project to Scala 3,
40+
it does not matter in which order you migrate the modules, they will be
41+
able to depend on each other regardless of whether they are compiled with
42+
Scala 2.13 or Scala 3.
43+
44+
To illustrate this scenario, we will start with an empty directory and
45+
build an sbt project with Scala 2.13 that has two modules, `shared`,
46+
which has some common domain model data structures, and `app`,
47+
which uses those data structures.
48+
49+
For this project, we will pick
50+
[sbt 1.4.0](https://github.com/sbt/sbt/releases/tag/v1.4.0),
51+
this allows you to mix projects of different Scala versions with very
52+
little extra effort (thanks to Eugene Yokota).
53+
54+
This tutorial also assumes the release of Scala `2.13.4`, but if you would
55+
like to follow along today, you can [modify the build file](#early-access)
56+
to use an early access snapshot release.
57+
58+
To begin, our project looks like the following:
59+
60+
```scala
61+
// project/build.properties
62+
sbt.version=1.4.0
63+
```
64+
65+
```scala
66+
// build.sbt
67+
68+
ThisBuild / scalaVersion := "2.13.4"
69+
70+
lazy val shared = project
71+
72+
lazy val app = project
73+
.dependsOn(shared)
74+
```
75+
76+
```scala
77+
// shared/src/main/scala/example/Cat.scala
78+
79+
package example
80+
81+
sealed trait Cat
82+
object Cat {
83+
case object Lion extends Cat
84+
case object Tiger extends Cat
85+
case object Cheetah extends Cat
86+
}
87+
```
88+
89+
```scala
90+
// app/src/main/scala/example/Main.scala
91+
92+
package example
93+
94+
object Main extends App {
95+
println(Cat.Tiger)
96+
}
97+
```
98+
99+
We can run this application with `sbt app/run` which will output the following:
100+
101+
```
102+
[info] Compiling 1 Scala source to blog/shared/target/scala-2.13/classes ...
103+
[info] Compiling 1 Scala source to blog/app/target/scala-2.13/classes ...
104+
[info] running example.Main
105+
Tiger
106+
```
107+
108+
At this point we can try something exciting, compile either subproject with
109+
Scala 3 and see if it works. At the time of writing, the Scala version
110+
`0.27.0-RC1` is the latest preview release of Scala 3.
111+
112+
First, add the dotty plugin (this helps with managing and inspecting the
113+
`scalaVersion` setting with Scala 3)
114+
115+
```scala
116+
// project/plugins.sbt
117+
118+
addSbtPlugin("ch.epfl.lamp" % "sbt-dotty" % "0.4.2")
119+
```
120+
121+
Then we can change the `scalaVersion` of `app`:
122+
123+
{% highlight diff %}
124+
// build.sbt
125+
126+
ThisBuild / scalaVersion := "2.13.4"
127+
128+
lazy val shared = project
129+
130+
lazy val app = project
131+
.dependsOn(shared)
132+
+ .settings(scalaVersion := "0.27.0-RC1")
133+
{% endhighlight %}
134+
135+
To recap, `app` is now compiled by Scala 3 and depends on `shared`,
136+
which is compiled by Scala 2.13.
137+
138+
If we do `sbt app/run` we should see the `app` project recompile
139+
and it will run as before.
140+
141+
Now lets try the other way around:
142+
143+
{% highlight diff %}
144+
// build.sbt
145+
146+
ThisBuild / scalaVersion := "2.13.4"
147+
148+
lazy val shared = project
149+
+ .settings(scalaVersion := "0.27.0-RC1")
150+
151+
lazy val app = project
152+
.dependsOn(shared)
153+
- .settings(scalaVersion := "0.27.0-RC1")
154+
{% endhighlight %}
155+
156+
Here we have the opposite, `app` is compiled by Scala 2.13 and depends
157+
on `shared`, which is compiled by Scala 3. We can see this more clearly using
158+
the command `sbt 'show app/dependencyTree'`, which outputs the following:
159+
160+
```
161+
app:app_2.13:0.1.0-SNAPSHOT [S]
162+
+-shared:shared_0.27:0.1.0-SNAPSHOT
163+
+-ch.epfl.lamp:dotty-library_0.27:0.27.0-RC1 [S]
164+
```
165+
166+
If we then try `sbt app/run` both `shared` and `app` subprojects will recompile
167+
and it should work as it always has. If in your own project there are issues
168+
in changing the Scala versions, check out the
169+
[troubleshooting](#troubleshooting) section.
170+
171+
To summarise this part, we have shown that it is possible to migrate subprojects
172+
to Scala 3 from Scala 2.13 in any order, gradually, and continue to build and
173+
run them if they mix versions.
174+
175+
## Using a Scala 3 Library Dependency
176+
177+
In the section above, we have seen how it is possible for a subproject compiled
178+
with Scala 2.13 to depend on a subproject compiled with Scala 3. The same
179+
relationship extends to library dependencies.
180+
181+
To demonstrate using a third party library dependency, we will add a dependency
182+
on the fansi library to `app` and try to change our output to be colored blue:
183+
184+
{% highlight diff %}
185+
// build.sbt
186+
187+
ThisBuild / scalaVersion := "2.13.4"
188+
189+
lazy val shared = project
190+
.settings(scalaVersion := "0.27.0-RC1")
191+
192+
lazy val app = project
193+
.dependsOn(shared)
194+
+ .settings(
195+
+ libraryDependencies += "com.lihaoyi" % "fansi_0.27" % "0.2.9"
196+
+ )
197+
{% endhighlight %}
198+
199+
Notice that here, we are using the Scala 3 version of fansi, to force this,
200+
we take a standard module id and change it as so:
201+
202+
{% highlight diff %}
203+
-"com.lihaoyi" %% "fansi" % "0.2.9"
204+
+"com.lihaoyi" % "fansi_0.27" % "0.2.9"
205+
{% endhighlight %}
206+
207+
By replacing `%%` with `%`, we can then manually specify the binary version,
208+
leading us to add `_0.27` to the name of the module.
209+
210+
We now update `Main.scala` to use the fansi library:
211+
212+
{% highlight diff %}
213+
// app/src/main/scala/example/Main.scala
214+
215+
package example
216+
217+
object Main extends App {
218+
- println(Cat.Tiger)
219+
+ println(fansi.Color.Blue(Cat.Tiger.toString))
220+
}
221+
{% endhighlight %}
222+
223+
If we then use the command `sbt app/run` we should notice that "Tiger"
224+
is now blue when printed.
225+
226+
If we then run again `sbt 'show app/dependencyTree'` we see the following:
227+
228+
```
229+
app:app_2.13:0.1.0-SNAPSHOT [S]
230+
+-com.lihaoyi:fansi_0.27:0.2.9
231+
| +-com.lihaoyi:sourcecode_0.27:0.2.1
232+
|
233+
+-shared:shared_0.27:0.1.0-SNAPSHOT
234+
+-ch.epfl.lamp:dotty-library_0.27:0.27.0-RC1 [S]
235+
```
236+
237+
To summarise, in this section we have shown how it is possible to use a
238+
third party library dependency, compiled with Scala 3, from Scala 2.13.
239+
If there are issues with changing the binary version of a particular
240+
dependency you have, check out the [troubleshooting](#troubleshooting) section.
241+
242+
## Troubleshooting
243+
There are some situations where the steps described in the sections above do
244+
not work out of the box for your own project, and some other considerations
245+
need to be made.
246+
247+
For the multi module project scenario, let's say you have two subprojects
248+
`A` and `B`, where `B` depends on `A`, both compiled with Scala 2.13.
249+
250+
### Source Incompatibilities When Changing to Scala 3
251+
If you change the `scalaVersion` of `A` to Scala 3 and `A` fails to build,
252+
and the error is not due to a missing dependency, the source code of `A` may
253+
be incompatible with Scala 3 and you will have to make some changes.
254+
You can consult the [Scala 3 Migration Guide](https://scalacenter.github.io/scala-3-migration-guide/)
255+
to see what changes may be required.
256+
257+
### Missing Scala 3 Dependencies
258+
If you change the `scalaVersion` of `A` to Scala 3 and `A` fails to build
259+
due to a missing dependency, you can try to use the Scala 2.13 version of
260+
the dependency. The sbt-dotty plugin provides
261+
[helper methods](https://github.com/lampepfl/dotty-example-project#getting-your-project-to-compile-with-dotty)
262+
for this.
263+
264+
### Scala 3 Dependencies That Break Compilation
265+
266+
If you successfully migrate subproject `A` to Scala 3, but now `B` fails
267+
to compile, there are several likely possibilities:
268+
269+
1. `A` has made a deliberate but [forward compatible](#forward-compatibility)
270+
change to its API, and `B` needs to update to use the new API.
271+
2. `A`, or a transitive dependency of `A`, uses a feature of Scala 3 that
272+
is not [forwards compatible](#forward-compatibility) with Scala 2.13.
273+
2. `A`, or a transitive dependency of `A`, defines a Scala 3
274+
macro but does not provide an equivalent Scala 2 macro (see an
275+
[example project](https://github.com/scalacenter/mix-macros-scala-2-and-3)
276+
here for how to provide macros to Scala 2.13 from Scala 3.)
277+
4. There is a bug in the implementation of reading Scala 3 dependencies
278+
in Scala 2.13. [Please report here](https://github.com/scala/bug).
279+
280+
### Scala 3 Dependencies that Break Runtime Behaviour
281+
282+
If you successfully migrate subproject `A` to Scala 3, and `B` compiles
283+
successfully, but now fails at runtime, there are two likely possibilities:
284+
285+
1. The definitions in the Scala 3 dependency were correctly read, but
286+
perhaps the API changed subtly, so the code of `B` must adapt.
287+
For example, perhaps a default implicit value has been replaced, or other
288+
code has changed its implementation while using the same signature.
289+
2. There is a bug in the implementation of reading Scala 3 dependencies
290+
in Scala 2.13. [Please report here](https://github.com/scala/bug).
291+
292+
### Third Party Scala 3 Dependencies that Break Compilation/Runtime
293+
If depending on a third party library, compiled with Scala 3, breaks
294+
compilation, or disrupts runtime behaviour, please consider the above
295+
troubleshooting topics, but instead assume that `A` is the third party
296+
library, and `B` is your Scala 2.13 project that depends on it.
297+
298+
## Forward Compatibility
299+
When migrating a subproject to Scala 3, where downstream consuming subprojects are likely to be on Scala 2.13, we would recommend restricting the usage of new features in Scala 3. This will maximise compatibility as you migrate each subproject. However, it is possible to start using some features of Scala 3 without issue, this is due to a limited forward compatibility in Scala 2.13 with some new Scala 3 features.
300+
301+
Forward compatibility means that many definitions created by using new Scala 3 features
302+
can be used from Scala 2.13, however they will be remapped to features
303+
that exist in Scala 2.13. For example,
304+
[extension methods](http://dotty.epfl.ch/docs/reference/contextual/extension-methods.html)
305+
can only be used as ordinary methods, using their expanded name.
306+
307+
On the other hand, some features of Scala 3 are not mappable to features in Scala 2.13,
308+
and will cause a compile-time error when using them. A longer list can be seen in the
309+
migration guide, describing [how Scala 2 reacts to different Scala 3 features](https://scalacenter.github.io/scala-3-migration-guide/docs/compatibility.html#the-scala-2-tasty-reader).
310+
311+
For unsupported features, a best effort is made
312+
to report errors at the use-site that is problematic. For example,
313+
[match types](http://dotty.epfl.ch/docs/reference/new-types/match-types.html)
314+
are not supported. If we define in the `shared` project the type `Elem`:
315+
316+
```scala
317+
// shared/src/main/scala/example/MatchTypes.scala
318+
package example
319+
320+
object MatchTypes {
321+
type Elem[X] = X match {
322+
case List[t] => t
323+
case Array[t] => t
324+
}
325+
}
326+
```
327+
328+
and then try to use it in the `app` project:
329+
330+
```scala
331+
// app/src/main/scala/example/TestMatchTypes.scala
332+
package example
333+
334+
object TestMatchTypes {
335+
def test: MatchTypes.Elem[List[String]] = "hello"
336+
}
337+
```
338+
339+
we get the following error when calling `sbt app/run`:
340+
341+
```
342+
[error] TestMatchTypes.scala:5:25: Unsupported Scala 3 match type in bounds of type Elem; found in object example.MatchTypes.
343+
[error] def test: MatchTypes.Elem[List[String]] = "hello"
344+
[error] ^
345+
[error] one error found
346+
```
347+
348+
The error is standard for all unsupported Scala 3 features, naming the feature,
349+
the location of the definition and the location where it is used.
350+
351+
## Contributing
352+
353+
If you have found an issue with Scala 3 dependencies in Scala 2.13 and want to try
354+
fixing it in the Scala 2 compiler, you can see a summary of how to work on the
355+
implementation here:
356+
[Working with the Tasty Reader](https://github.com/scala/scala/blob/2.13.x/doc/internal/tastyreader.md#working-with-the-code)
357+
358+
## Summary
359+
360+
In this blog, we have outlined how it is possible to incrementally migrate a
361+
project to Scala 3 while continuing to use all the parts together during the
362+
transition. We encourage you to try out this process and let us know of any
363+
issues with using Scala 3 dependencies from Scala 2.13.
364+
365+
## Early Access
366+
367+
If you would like to try out Scala 3 dependencies early, you can use a
368+
snapshot version of Scala 2.13.4 by adding the Scala integration resolver
369+
to your build:
370+
371+
{% highlight diff %}
372+
// build.sbt
373+
374+
-ThisBuild / scalaVersion := "2.13.4"
375+
+ThisBuild / scalaVersion := "2.13.4-bin-8891679"
376+
377+
+Global / resolvers += "scala-integration".at(
378+
+ "https://scala-ci.typesafe.com/artifactory/scala-integration/")
379+
380+
...
381+
{% endhighlight %}
382+
383+
## Important Links
384+
385+
- [Scala 3 Migration Guide](https://scalacenter.github.io/scala-3-migration-guide/)
386+
- [Doc: Working with the Tasty Reader](https://github.com/scala/scala/blob/2.13.x/doc/internal/tastyreader.md#working-with-the-code)
387+
- [Talk: Taste the difference with Scala 3](https://www.youtube.com/watch?v=YQmVrUdx8TU)
388+
- [Scala 2 issue tracker](https://github.com/scala/bug)

0 commit comments

Comments
 (0)