Skip to content

Add error handling strawman #16550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 5 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions tests/run/errorhandling.check
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
breakTest
optTest
resultTest
Person(Kostas,5)
70 changes: 70 additions & 0 deletions tests/run/errorhandling/Result.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
package dotty.util
import boundary.Label

abstract class Result[+T, +E]
case class Ok[+T](value: T) extends Result[T, Nothing]
case class Err[+E](value: E) extends Result[Nothing, E]
Copy link
Contributor

@ScalaWilliam ScalaWilliam Jan 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be more natural to call this Bad (or Failed) rather than Err? As Err(or) is not an opposite or an antonym of Ok. It also makes it feel like value E should extend Exception or Error; alternatively it also sounds like something you'd see in the Linux kernel or Kubernetes source code.

Read out loud the following:

  • "This is an Ok Result"
  • "This is an Err Result"
  • "This is a Bad Result"
  • "This is a Failed Result"

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • "This is a Fine Result"
  • "This is a Fail Result".

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ScalaWilliam , type E should stay open, a lot of the times it won't be an exception but rather a case class or a string.
i.e.
if nameStr.isBlank then Err("name is empty") else Ok(nameStr)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ScalaWilliam

Scalactic uses Good and Bad, "good" is the direct antonym of "bad". On the other side, using Bad for Result might cause some conflicts in code using Scalactic, or at least lead to some confusion. While this should not drive the design of the standard library, I think it's worth considering what are the names used in popular libraries. I did not see Ok or Err in any other validation library, and their meaning seems already obvious.

Scalactic's Bad and cats' Validated are not bound to Exception or any other error type, I think Err should do the same if we want it to replace them.


object Result:
extension [T, E](r: Result[T, E])

/** `_.?` propagates Err to current Label */
transparent inline def ? (using Label[Err[E]]): T = r match
case r: Ok[_] => r.value
case err => break(err.asInstanceOf[Err[E]])

/** If this is an `Err`, map its value */
def mapErr[E1](f: E => E1): Result[T, E1] = r match
case err: Err[_] => Err(f(err.value))
case ok: Ok[_] => ok

/** Map Ok values, propagate Errs */
def map[U](f: T => U): Result[U, E] = r match
case Ok(x) => Ok(f(x))
case err: Err[_] => err

/** Flatmap Ok values, propagate Errs */
def flatMap[U](f: T => Result[U, E]): Result[U, E] = r match
case Ok(x) => f(x)
case err: Err[_] => err

/** Validate both `r` and `other`; return a pair of successes or a list of failures. */
def * [U](other: Result[U, E]): Result[(T, U), List[E]] = (r, other) match
case (Ok(x), Ok(y)) => Ok((x, y))
case (Ok(_), Err(e)) => Err(e :: Nil)
case (Err(e), Ok(_)) => Err(e :: Nil)
case (Err(e1), Err(e2)) => Err(e1 :: e2 :: Nil)

/** Validate both `r` and `other`; return a tuple of successes or a list of failures.
* Unlike with `*`, the right hand side `other` must be a `Result` returning a `Tuple`,
* and the left hand side is added to it. See `Result.empty` for a convenient
* right unit of chains of `*:`s.
*/
def *: [U <: Tuple](other: Result[U, List[E]]): Result[T *: U, List[E]] = (r, other) match
case (Ok(x), Ok(ys)) => Ok(x *: ys)
case (Ok(_), es: Err[?]) => es
case (Err(e), Ok(_)) => Err(e :: Nil)
case (Err(e), Err(es)) => Err(e :: es)

/** Simlar to `Try`: Convert exceptions raised by `body` to `Err`s.
* In principle, `Try[T]` should be equivalent to `Result[T, Exception]`.
* Note that we do not want to catch and reify all Throwables.
* - severe JVM errors that make continuation impossible should not be reified.
* - control throwables like `boundary.Break` should not be caught. We want
* them to return from a `Result`.
* (Generally, the focus on `Throwable` in Scala libraries is a mistake.
* Use `Exception` instead, as it was meant to in Java.)
*/
inline def apply[T](body: => T): Result[T, Exception] =
try Ok(body)
catch case ex: Exception => Err(ex)

/** Right unit for chains of `*:`s. Returns an `Ok` with an `EmotyTuple` value. */
def empty: Result[EmptyTuple, Nothing] = Ok(EmptyTuple)
end Result

/** A prompt for `_.?`. It establishes a boundary to which `_.?` returns */
object respond:
transparent inline def apply[T, E](inline body: Label[Err[E]] ?=> T): Result[T, E] =
boundary(Ok(body))

78 changes: 78 additions & 0 deletions tests/run/errorhandling/Test.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
import dotty.util.*

/** boundary/break as a replacement for non-local returns */
def indexOf[T](xs: List[T], elem: T): Int =
boundary:
for (x, i) <- xs.zipWithIndex do
if x == elem then break(i)
Copy link
Contributor

@ScalaWilliam ScalaWilliam Jan 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It really feels like we're closer to having Python generator syntax, which is fantastic! Would love to see a strawman for this if it is actually possible with the existing Scala 3 capabilities.

In Python you can do things like this, which read naturally and return a generator, an equivalent to a Scala/Java Iterator:

def example():
  yield "We'll give you all the numbers divisible by 3 or 2, with those by 3 as priority"
  for i in range(1, 1000):
    if i % 3 == 0:
      yield ("%d is divisible by 3" % i)
    elif i % 2 == 0:
      yield ("%d is even" % i)

image

The yields can live anywhere, and you can easily return or break out.

And of course you can use this generator in another for-loop, or even manually consume it. They terminate through an exception called StopIteration:
image

However, I had not seen a solution to clean-up/finalization in generators. This is where libraries like FS2 really shine because you have guaranteed clean-up of resources. I'm willing to put up with a more rigid coding experience in order to get that guarantee, but if there is a way to achieve the best of both worlds it would be really fantastic.

What's great about generators is that you write them in a push-style (yield), but consume them in a pull style (next()/for).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but if there is a way to achieve the best of both worlds it would be really fantastic.

@ScalaWilliam you mean like this? :)

//> using lib "com.github.rssh::dotty-cps-async::0.9.14"
//> using lib "co.fs2::fs2-core::3.4.0-56-9e373bd-SNAPSHOT"
//> using repository "https://s01.oss.sonatype.org/content/repositories/snapshots/"
//> using option "-Ykind-projector"

import cats.effect.*
import fs2.*
import cps.*

given [O]: CpsMonad[Pull[IO, O, *]] with CpsMonadInstanceContext[Pull[IO, O, *]] with
  def pure[A](a: A): Pull[IO, O, A] = Pull.pure(a)
  def map[A, B](fa: Pull[IO, O, A])(f: A => B): Pull[IO, O, B] = fa.map(f)
  def flatMap[A, B](fa: Pull[IO, O, A])(f: A => Pull[IO, O, B]): Pull[IO, O, B] = fa.flatMap(f)

def generator: Pull[IO, String, Unit] =
  async[Pull[IO, String, *]] {
    await(
      Pull.acquire[IO, Unit](
        IO.println("acquired resource"),
        (_, _) => IO.println("released resource")
      ): Pull[IO, String, Unit]
    )

    (1 to 20).map { i =>
      if i % 3 == 0 then
        await(Pull.output1[IO, String](s"$i is divisible by 3"))
      if i % 2 == 0 then
        await(Pull.output1[IO, String](s"$i is even"))
    }
  }

@main def main =
  import cats.effect.unsafe.implicits.global
  generator.stream.foreach(IO.println(_)).compile.drain.unsafeRunSync()
acquired resource
2 is even
3 is divisible by 3
4 is even
6 is divisible by 3
6 is even
8 is even
9 is divisible by 3
10 is even
12 is divisible by 3
12 is even
14 is even
15 is divisible by 3
16 is even
18 is divisible by 3
18 is even
20 is even
released resource

I'm a broken record, but CPS is really really awesome!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I find this really strange that the resource release code magically happens at the end of the block, even though the "nested" code is in the same block, (i.e not in some explicit envelope)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't disagree :) mixing resources with generators is somewhat "advanced", because the resource needs to stay alive for the lifetime of the generator, and the lifetime of the generator is determined by some downstream consumer of it.

That is to say: the generator itself is the "envelope", since we don't know what goes "inside" until the user actually consumes it (and does whatever).

So in the example above, bringing a resource into the generator process keeps it alive and makes it available for the entire lifetime of the generator (as determined by the consumer).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@armanbilge that is impressive, I did not expect it would even be possible in fs2. Even experimenting with vars inside this just works.

Are you able to also do an fs2 Pipe that would 'await' for an input, and conditionally produce outputs? Possibly allocating a resource in the middle as well like you had declared.

To make it very simple, is it possible to write the equivalent of .filter(_ % 3 == 0) in this style as well?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ScalaWilliam Thanks!

Even experimenting with vars inside this just works.

It might, but it would be safer to use await(Pull.eval(IO.ref(...))) to make mutable vars, and update them with await(Pull.eval(ref.set(...))) 😄


Are you able to also do an fs2 Pipe that would 'await' for an input, and conditionally produce outputs?

Great question, we've been discussing this topic on the Typelevel Discord. So first of all, you can definitely do this with the Stream monad:

//> using lib "com.github.rssh::dotty-cps-async::0.9.14"
//> using lib "co.fs2::fs2-core::3.4.0-56-9e373bd-SNAPSHOT"
//> using repository "https://s01.oss.sonatype.org/content/repositories/snapshots/"
//> using option "-Ykind-projector"

import cats.effect.*
import fs2.*
import cps.*

given CpsMonad[Stream[IO, *]] with CpsMonadInstanceContext[Stream[IO, *]] with
  def pure[A](a: A): Stream[IO, A] = Stream.emit(a)
  def map[A, B](fa: Stream[IO, A])(f: A => B): Stream[IO, B] = fa.map(f)
  def flatMap[A, B](fa: Stream[IO, A])(f: A => Stream[IO, B]): Stream[IO, B] = fa.flatMap(f)

def pipe(in: Stream[IO, Int]): Stream[IO, Int] =
  async[Stream[IO, *]] {
    val i = await(in)
    if i % 3 == 0 then
      await(Stream.emit[IO, Int](i))
    else
      await(Stream.empty: Stream[IO, Int])
  }

@main def main =
  import cats.effect.unsafe.implicits.global
  Stream.emits(1 to 20).through(pipe).foreach(IO.println(_)).compile.drain.unsafeRunSync()

However it's not yet possible to do this in the Pull monad, which is ideal for writing "generators".

There are a few ways to transform a Pull[IO, Output, State] datatype. My original example relied on flatMap, which gives you access to the current State.

def flatMap[A, B](pullA: Pull[IO, Out, A])(f: A => Pull[IO, Out, B]): Pull[IO, Out, B]

Then, CPS transforms await to flatMap:

val a = await(pullA)
await(f(a))

Pull also offers a method flatMapOutput, which gives you access to the current Out.

def flatMapOutput(in: Pull[IO, Out, Unit])(f: Out => Pull[IO, Out, Unit]): Pull[IO, Out, Unit]

It would be interesting if we could add e.g. an awaitNext(...) syntax that transforms to flatMapOutput under-the-hood:

val i = awaitNext(in)
if i % 3 == 0 then
  await(Pull.output1(i))

I don't know enough about the inner-workings of CPS to know if this is feasible. But it seems like it could be :)

-1

def breakTest() =
println("breakTest")
assert(indexOf(List(1, 2, 3), 2) == 1)
assert(indexOf(List(1, 2, 3), 0) == -1)

/** traverse becomes trivial to write */
def traverse[T](xs: List[Option[T]]): Option[List[T]] =
optional(xs.map(_.?))

def optTest() =
println("optTest")
assert(traverse(List(Some(1), Some(2), Some(3))) == Some(List(1, 2, 3)))
assert(traverse(List(Some(1), None, Some(3))) == None)

/** A check function returning a Result[Unit, _] */
inline def check[E](p: Boolean, err: E): Result[Unit, E] =
if p then Ok(()) else Err(err)

/** Another variant of a check function that returns directly to the given
* label in case of error.
*/
inline def check_![E](p: Boolean, err: E)(using l: boundary.Label[Err[E]]): Unit =
if p then () else l.break(Err(err))

/** Use `Result` to convert exceptions to `Err` values */
def parseDouble(s: String): Result[Double, Exception] =
Result(s.toDouble)

def parseDoubles(ss: List[String]): Result[List[Double], Exception] =
respond:
ss.map(parseDouble(_).?)

/** Demonstrate combination of `check` and `.?`. */
def trySqrt(x: Double) = // inferred: Result[Double, String]
respond:
check(x >= 0, s"cannot take sqrt of negative $x").?
math.sqrt(x)

/** Instead of `check(...).?` one can also use `check_!(...)`.
* Note use of `mapErr` to convert Exception errors to String errors.
*/
def sumRoots(xs: List[String]) = // inferred: Result[Double, String]
respond:
check_!(xs.nonEmpty, "list is empty") // direct jump
val ys = parseDoubles(xs).mapErr(_.toString).? // direct jump
ys.reduce((x, y) => x + trySqrt(y).?) // need exception to propagate `Err`

def resultTest() =
println("resultTest")
def assertFail(value: Any, s: String) = value match
case Err(msg: String) => assert(msg.contains(s))
assert(sumRoots(List("1", "4", "9")) == Ok(6))
assertFail(sumRoots(List("1", "-2", "4")), "cannot take sqrt of negative")
assertFail(sumRoots(List()), "list is empty")
assertFail(sumRoots(List("1", "3ab")), "NumberFormatException")
val xs = sumRoots(List("1", "-2", "4")) *: sumRoots(List()) *: sumRoots(List("1", "3ab")) *: Result.empty
xs match
case Err(msgs) => assert(msgs.length == 3)
case _ => assert(false)
val ys = sumRoots(List("1", "2", "4")) *: sumRoots(List("1")) *: sumRoots(List("2")) *: Result.empty
ys match
case Ok((a, b, c)) => // ok
case _ => assert(false)

@main def Test =
breakTest()
optTest()
resultTest()
parseCsvIgnoreErrors()
22 changes: 22 additions & 0 deletions tests/run/errorhandling/break.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
package dotty.util
import scala.util.control.ControlThrowable

object boundary:

class Break[T](val label: Label[T], val value: T) extends ControlThrowable

class Label[T] extends ControlThrowable:
transparent inline def break(value: T): Nothing = throw Break(this, value)

transparent inline def apply[T <: R, R](inline body: Label[T] ?=> R): R =
val local = Label[T]()
try body(using local)
catch case ex: Break[T] @unchecked =>
if ex.label eq local then ex.value
else throw ex

end boundary

object break:
transparent inline def apply[T](value: T)(using l: boundary.Label[T]): Nothing =
l.break(value)
35 changes: 35 additions & 0 deletions tests/run/errorhandling/kostas.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
package optionMockup:
import dotty.util.boundary
object optional:
transparent inline def apply[T](inline body: boundary.Label[None.type] ?=> T): Option[T] =
boundary(Some(body))

extension [T](r: Option[T])
transparent inline def ? (using label: boundary.Label[None.type]): T = r match
case Some(x) => x
case None => label.break(None)

import optionMockup.*

case class Person(name: String, age: Int)

object PersonCsvParserIgnoreErrors:
def parse(csv: Seq[String]): Seq[Person] =
for
line <- csv
columns = line.split(",")
parsed <- parseColumns(columns)
yield
parsed

private def parseColumns(columns: Seq[String]): Option[Person] =
columns match
case Seq(name, age) => parsePerson(name, age)
case _ => None

private def parsePerson(name: String, age: String): Option[Person] =
optional:
Person(name, age.toIntOption.?)

def parseCsvIgnoreErrors() =
println(PersonCsvParserIgnoreErrors.parse(Seq("Kostas,5", "George,invalid", "too,many,columns")).mkString("\n"))
20 changes: 20 additions & 0 deletions tests/run/errorhandling/optional.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
package dotty.util
import boundary.Label

/** A mockup of scala.Option */
abstract class Option[+T]
case class Some[+T](x: T) extends Option[T]
case object None extends Option[Nothing]

object Option:
/** This extension should be added to the companion object of scala.Option */
extension [T](r: Option[T])
transparent inline def ? (using label: Label[None.type]): T = r match
case Some(x) => x
case None => label.break(None)

/** A prompt for `Option`, which establishes a boundary which `_.?` on `Option` can return */
object optional:
transparent inline def apply[T](inline body: Label[None.type] ?=> T): Option[T] =
boundary(Some(body))