Skip to content

Commit f8ff900

Browse files
bishaboshabishabosha
committed
Add blog about Tasty Reader
1 parent ad68d54 commit f8ff900

File tree

1 file changed

+373
-0
lines changed

1 file changed

+373
-0
lines changed

‎blog/_posts/2020-10-12-scala-3-forward-compat.md

Lines changed: 373 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,373 @@
1+
---
2+
layout: blog-detail
3+
post-type: blog
4+
by: Jamie Thompson, Scala Center
5+
title: Forward Compatibility for the Scala 3 Transition
6+
---
7+
# Forward Compatibility for the Scala 3 Transition
8+
Scala 3 is coming, with a release candidate slated for the end of 2020.
9+
With that knowledge comes the inevitable question:
10+
should I migrate, and what is the potential cost?
11+
12+
With this blog we would like to outline some scenarios for maintainers of projects who
13+
want to move to Scala 3, and how these
14+
scenarios are made simpler by the ability to read Scala 3 compiled dependencies
15+
from Scala 2.13. There is also a [troubleshooting](#troubleshooting) section
16+
in case you encounter issues following the steps described in the scenarios.
17+
18+
>As a quick aside, If you would like to know how it is possible to mix
19+
>dependencies between the two compilers, watch the talk
20+
>[Taste the difference with Scala 3](https://www.youtube.com/watch?v=YQmVrUdx8TU).
21+
>
22+
>To summarise: Scala 3 stores meta information about the code it compiles in
23+
>TASTy files, and Scala 2.13 can read these files to learn, for example,
24+
>which terms, types and implicits are defined in a given dependency,
25+
>and what code needs to be generated to use them correctly.
26+
27+
## Scenarios
28+
29+
1. You have an application consisting of many subprojects that may depend on
30+
each other, built with Scala 2.13, and would like to transition each one
31+
independently to Scala 3.0
32+
2. You have an application, built with Scala 2.13, and want to use some
33+
new features of a library that has migrated to Scala 3.0
34+
35+
Let's take a look at each scenario in turn:
36+
37+
## Migrating a multi module project
38+
39+
If you want to migrate a multi module project to Scala 3,
40+
it does not matter in which order you migrate the modules, they will be
41+
able to depend on each other regardless of whether they are compiled with
42+
Scala 2.13 or Scala 3.
43+
44+
To illustrate this scenario, we will start with an empty directory and
45+
build an sbt project with Scala 2.13 that has two modules, `shared`,
46+
which has some common domain model data structures, and `app`,
47+
which uses those data structures.
48+
49+
For this project, we will pick
50+
[sbt 1.4.0](https://github.com/sbt/sbt/releases/tag/v1.4.0),
51+
this allows you to mix projects of different Scala versions with very
52+
little extra effort, (thanks to Eugene Yokota).
53+
54+
This tutorial also assumes the release of Scala `2.13.4`, but if you would
55+
like to follow along today, you can [modify the build file](#early-access)
56+
to use an early access snapshot release.
57+
58+
To begin, our project looks like the following:
59+
```scala
60+
// project/build.properties
61+
sbt.version=1.4.0
62+
```
63+
64+
```scala
65+
// build.sbt
66+
67+
ThisBuild / scalaVersion := "2.13.4"
68+
69+
lazy val shared = project
70+
71+
lazy val app = project
72+
.dependsOn(shared)
73+
```
74+
```scala
75+
// shared/src/main/scala/example/Cat.scala
76+
77+
package example
78+
79+
sealed trait Cat
80+
object Cat {
81+
case object Lion extends Cat
82+
case object Tiger extends Cat
83+
case object Cheetah extends Cat
84+
}
85+
```
86+
```scala
87+
// app/src/main/scala/example/Main.scala
88+
89+
package example
90+
91+
object Main extends App {
92+
println(Cat.Tiger)
93+
}
94+
```
95+
We can run this application with `sbt app/run` which will output the following:
96+
```
97+
[info] Compiling 1 Scala source to blog/shared/target/scala-2.13/classes ...
98+
[info] Compiling 1 Scala source to blog/app/target/scala-2.13/classes ...
99+
[info] running example.Main
100+
Tiger
101+
```
102+
103+
At this point we can try something exciting, compile either subproject with
104+
Scala 3 and see if it works. At the time of writing, the Scala version
105+
`0.27.0-RC1` is the latest preview release of Scala 3.
106+
107+
First, add the dotty plugin (this helps with managing and inspecting the
108+
`scalaVersion` setting with Scala 3)
109+
110+
```scala
111+
// project/plugins.sbt
112+
113+
addSbtPlugin("ch.epfl.lamp" % "sbt-dotty" % "0.4.2")
114+
```
115+
116+
Then we can change the `scalaVersion` of `app`:
117+
118+
```diff
119+
// build.sbt
120+
121+
ThisBuild / scalaVersion := "2.13.4"
122+
123+
lazy val shared = project
124+
125+
lazy val app = project
126+
.dependsOn(shared)
127+
+ .settings(scalaVersion := "0.27.0-RC1")
128+
```
129+
130+
To recap, `app` is now compiled by Scala 3 and depends on `shared`,
131+
which is compiled by Scala 2.13.
132+
133+
If we do `sbt app/run` we should see the `app` project recompile
134+
and it will run as before.
135+
136+
Now lets try the other way around:
137+
138+
```diff
139+
// build.sbt
140+
141+
ThisBuild / scalaVersion := "2.13.4"
142+
143+
lazy val shared = project
144+
+ .settings(scalaVersion := "0.27.0-RC1")
145+
146+
lazy val app = project
147+
.dependsOn(shared)
148+
- .settings(scalaVersion := "0.27.0-RC1")
149+
```
150+
151+
Here we have the opposite, `app` is compiled by Scala 2.13 and depends
152+
on `shared`, which is compiled by Scala 3. We can see this more clearly using
153+
the command `sbt 'show app/dependencyTree'`, which outputs the following:
154+
```
155+
app:app_2.13:0.1.0-SNAPSHOT [S]
156+
+-shared:shared_0.27:0.1.0-SNAPSHOT
157+
+-ch.epfl.lamp:dotty-library_0.27:0.27.0-RC1 [S]
158+
```
159+
160+
If we then try `sbt app/run` both `shared` and `app` subprojects will recompile
161+
and it should work as it always has. If in your own project there are issues
162+
in changing the Scala versions, check out the
163+
[troubleshooting](#troubleshooting) section.
164+
165+
To summarise this part, we have shown that it is possible to migrate subprojects
166+
to Scala 3 from Scala 2.13 in any order, gradually, and continue to build and
167+
run them if they mix versions.
168+
169+
## Using a Scala 3 Library Dependency
170+
171+
In the section above, we have seen how it is possible for a subproject compiled
172+
with Scala 2.13 to depend on a subproject compiled with Scala 3. The same
173+
relationship extends to library dependencies.
174+
175+
To demonstrate using a third party library dependency, we will add a dependency
176+
on the fansi library to `app` and try to change our output to be colored blue:
177+
178+
```diff
179+
// build.sbt
180+
181+
ThisBuild / scalaVersion := "2.13.4"
182+
183+
lazy val shared = project
184+
.settings(scalaVersion := "0.27.0-RC1")
185+
186+
lazy val app = project
187+
.dependsOn(shared)
188+
+ .settings(
189+
+ libraryDependencies += "com.lihaoyi" % "fansi_0.27" % "0.2.9"
190+
+ )
191+
```
192+
Notice that here, we are using the Scala 3 version of fansi, to force this,
193+
we take a standard module id and change it as so:
194+
```diff
195+
-"com.lihaoyi" %% "fansi" % "0.2.9"
196+
+"com.lihaoyi" % "fansi_0.27" % "0.2.9"
197+
```
198+
By replacing `%%` with `%`, we can then manually specify the binary version,
199+
leading us to add `_0.27` to the name of the module.
200+
201+
We now update `Main.scala` to use the fansi library:
202+
203+
```diff
204+
// app/src/main/scala/example/Main.scala
205+
206+
package example
207+
208+
object Main extends App {
209+
- println(Cat.Tiger)
210+
+ println(fansi.Color.Blue(Cat.Tiger.toString))
211+
}
212+
```
213+
If we then use the command `sbt app/run` we should notice that "Tiger"
214+
is now blue when printed.
215+
216+
If we then run again `sbt 'show app/dependencyTree'` we see the following:
217+
```
218+
app:app_2.13:0.1.0-SNAPSHOT [S]
219+
+-com.lihaoyi:fansi_0.27:0.2.9
220+
| +-com.lihaoyi:sourcecode_0.27:0.2.1
221+
|
222+
+-shared:shared_0.27:0.1.0-SNAPSHOT
223+
+-ch.epfl.lamp:dotty-library_0.27:0.27.0-RC1 [S]
224+
```
225+
To summarise, in this section we have shown how it is possible to use a
226+
third party library dependency, compiled with Scala 3, from Scala 2.13.
227+
If there are issues with changing the binary version of a particular
228+
dependency you have, check out the [troubleshooting](#troubleshooting) section.
229+
230+
## Troubleshooting
231+
There are some situations where the steps described in the sections above do
232+
not work out of the box for your own project, and some other considerations
233+
need to be made.
234+
235+
For the multi module project scenario, let's say you have two subprojects
236+
`A` and `B`, where `B` depends on `A`, both compiled with Scala 2.13.
237+
238+
### Source Incompatibilities When Changing to Scala 3
239+
If you change the `scalaVersion` of `A` to Scala 3 and `A` fails to build,
240+
and the error is not due to a missing dependency, the source code of `A` may
241+
be incompatible with Scala 3 and you will have to make some changes.
242+
You can consult the [Scala 3 Migration Guide](https://scalacenter.github.io/scala-3-migration-guide/)
243+
to see what changes may be required.
244+
245+
### Missing Scala 3 Dependencies
246+
If you change the `scalaVersion` of `A` to Scala 3 and `A` fails to build
247+
due to a missing dependency, you can try to use the Scala 2.13 version of
248+
the dependency. The sbt-dotty plugin provides
249+
[helper methods](https://github.com/lampepfl/dotty-example-project#getting-your-project-to-compile-with-dotty)
250+
for this.
251+
252+
### Scala 3 Dependencies That Break Compilation
253+
254+
If you successfully migrate subproject `A` to Scala 3, but now `B` fails
255+
to compile, there are several likely possibilities:
256+
257+
1. `A` has made a deliberate but [forward compatible](#forward-compatibility)
258+
change to its API, and `B` needs to update to use the new API.
259+
2. `A`, or a transitive dependency of `A`, uses a feature of Scala 3 that
260+
is not [forwards compatible](#forward-compatibility) with Scala 2.13.
261+
2. `A`, or a transitive dependency of `A`, defines a Scala 3
262+
macro but does not provide an equivalent Scala 2 macro (see an
263+
[example project](https://github.com/scalacenter/mix-macros-scala-2-and-3)
264+
here for how to provide macros to Scala 2.13 from Scala 3.)
265+
4. There is a bug in the implementation of reading Scala 3 dependencies
266+
in Scala 2.13. [Please report here](https://github.com/scala/bug).
267+
268+
### Scala 3 Dependencies that Break Runtime Behaviour
269+
270+
If you successfully migrate subproject `A` to Scala 3, and `B` compiles
271+
successfully, but now fails at runtime, there are two likely possibilities:
272+
273+
1. The definitions in the Scala 3 dependency were correctly read, but
274+
perhaps the API changed subtly, so the code of `B` must adapt.
275+
For example, perhaps a default implicit value has been replaced, or other
276+
code has changed its implementation while using the same signature.
277+
2. There is a bug in the implementation of reading Scala 3 dependencies
278+
in Scala 2.13. [Please report here](https://github.com/scala/bug).
279+
280+
### Third Party Scala 3 Dependencies that Break Compilation/Runtime
281+
If depending on a third party library, compiled with Scala 3, breaks
282+
compilation, or disrupts runtime behaviour, please consider the above
283+
troubleshooting topics, but instead assume that `A` is the third party
284+
library, and `B` is your Scala 2.13 project that depends on it.
285+
286+
## Forward Compatibility
287+
When migrating a subproject to Scala 3, where downstream consuming subprojects are likely to be on Scala 2.13, we would recommend restricting the usage of new features in Scala 3. This will maximise compatibility as you migrate each subproject. However, it is possible to start using some features of Scala 3 without issue, this is due to a limited forward compatibility in Scala 2.13 with some new Scala 3 features.
288+
289+
Forward compatibility means that many definitions created by using new Scala 3 features
290+
can be used from Scala 2.13, however they will be remapped to features
291+
that exist in Scala 2.13. For example,
292+
[extension methods](http://dotty.epfl.ch/docs/reference/contextual/extension-methods.html)
293+
can only be used as ordinary methods, using their expanded name.
294+
295+
On the other hand, some features of Scala 3 are not mappable to features in Scala 2.13,
296+
and will cause a compile-time error when using them. A longer list can be seen in the
297+
migration guide, describing [how Scala 2 reacts to different Scala 3 features](https://scalacenter.github.io/scala-3-migration-guide/docs/compatibility.html#the-scala-2-tasty-reader).
298+
299+
For unsupported features, a best effort is made
300+
to report errors at the use-site that is problematic. For example,
301+
[match types](http://dotty.epfl.ch/docs/reference/new-types/match-types.html)
302+
are not supported. If we define in the `shared` project the type `Elem`:
303+
304+
```scala
305+
// shared/src/main/scala/example/MatchTypes.scala
306+
package example
307+
308+
object MatchTypes {
309+
type Elem[X] = X match {
310+
case List[t] => t
311+
case Array[t] => t
312+
}
313+
}
314+
```
315+
316+
and then try to use it in the `app` project:
317+
```scala
318+
// app/src/main/scala/example/TestMatchTypes.scala
319+
package example
320+
321+
object TestMatchTypes {
322+
def test: MatchTypes.Elem[List[String]] = "hello"
323+
}
324+
```
325+
we get the following error when calling `sbt app/run`:
326+
327+
```
328+
[error] TestMatchTypes.scala:5:25: Unsupported Scala 3 match type in bounds of type Elem; found in object example.MatchTypes.
329+
[error] def test: MatchTypes.Elem[List[String]] = "hello"
330+
[error] ^
331+
[error] one error found
332+
```
333+
334+
The error is standard for all unsupported Scala 3 features, naming the feature,
335+
the location of the definition and the location where it is used.
336+
337+
## Contributing
338+
339+
If you have found an issue with Scala 3 dependencies in Scala 2.13 and want to try
340+
fixing it in the Scala 2 compiler, you can see a summary of how to work on the
341+
implementation here:
342+
[Working with the Tasty Reader](https://github.com/scala/scala/blob/2.13.x/doc/internal/tastyreader.md#working-with-the-code)
343+
344+
## Summary
345+
346+
In this blog, we have outlined how it is possible to incrementally migrate a
347+
project to Scala 3 while continuing to use all the parts together during the
348+
transition. We encourage you to try out this process and let us know of any
349+
issues with using Scala 3 dependencies from Scala 2.13.
350+
351+
## Early Access
352+
353+
If you would like to try out Scala 3 dependencies early, you can use a
354+
snapshot version of Scala 2.13.4 by adding the Scala integration resolver
355+
to your build:
356+
```diff
357+
// build.sbt
358+
359+
-ThisBuild / scalaVersion := "2.13.4"
360+
+ThisBuild / scalaVersion := "2.13.4-bin-8891679"
361+
362+
+Global / resolvers += "scala-integration".at(
363+
+ "https://scala-ci.typesafe.com/artifactory/scala-integration/")
364+
365+
...
366+
```
367+
368+
## Important Links
369+
370+
- [Scala 3 Migration Guide](https://scalacenter.github.io/scala-3-migration-guide/)
371+
- [Doc: Working with the Tasty Reader](https://github.com/scala/scala/blob/2.13.x/doc/internal/tastyreader.md#working-with-the-code)
372+
- [Talk: Taste the difference with Scala 3](https://www.youtube.com/watch?v=YQmVrUdx8TU)
373+
- [Scala 2 issue tracker](https://github.com/scala/bug)

0 commit comments

Comments
 (0)