Tasks and communication

[larsbergstrom]

Currently, any large program using tasks and channels will have a mixture of deadlocks and unpredictable task failures due to the interaction of task failure, channel communications, and asynchronous communication protocols.

Ideas in this space:

• Controlled tear-down behavior, ala either the old linked task failure or something else easier to implement for a group of tasks.
• Static protocol checking to ensure progress / avoid deadlock or livelock. Note that, unlike much of the existing literature, this needs to be robust in the face of both async. comm. and async. + multiple failures.

CC: @pcwalton

[aturon]

cc me

[thehydroimpulse]

/cc me

[thestinger]

Controlled tear-down behavior, ala either the old linked task failure or something else easier to implement for a group of tasks.

Linked failure / thread cancellation doesn't work out well in a native language. The only way to do it is adding failure points where it gets checked, but adding that kind of overhead to Rust's standard library isn't going to be acceptable and it isn't going to be able to reliably fail without the compiler inserting dual use yield points (not going to happen).

Static protocol checking to ensure progress / avoid deadlock or livelock. Note that, unlike much of the existing literature, this needs to be robust in the face of both async. comm. and async. + multiple failures.

Can this be done in a syntax extension? It doesn't necessarily have to live in this repository.

[larsbergstrom]

Indeed, we have largely abandoned the Kill-Safe Abstractions runtime approach to task failure handling, as Rust has moved in a different direction. Below is some info from communication with some external researchers we're trying to convince to look at this problems with us. We'll see if we can come up with a solution that's independent of Rust itself.

We have a couple of big challenges with our use of concurrency in Servo today:

1. Isolation of communication

We use lightweight tasks (similar to CML threads) heavily to
partition the browser in a set of concurrent threads (e.g. the
JavaScript task, the rendering task, and the task for laying out the
page). We would like to ensure that all communication within certain
subsets of these tasks does not escape to the broader architecture.
This constraint would allow an outside managing task to detect when
some member of a subgroup of tasks has generated an exception
(failed, in Rust parlance) and easily shut down the entire subgroup,
without risking "spreading" the failure to the rest of the system via
channels that have one end connected to a task that has exited.

2. Eventual shutdown

We would like to at least show deadlock-freedom within our subgroup
of tasks. If they are all continuing to process events, then we have
reason to believe that they will eventually shut down if one task in
the subgroup shuts down (i.e., we don't leak any zombie tasks
continuing to take up memory but doing nothing useful).

We're very interested in ideas for tackling these problems. The first
seems like should be approachable with a straightforward type-based
analysis, but the second likely requires something much stronger,
though I'd love to be proven wrong.

[rust-highfive]

This issue has been moved to the RFCs repo: rust-lang/rfcs#316