-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Soundness issue with await #49396
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
This issue is blocked because discussions about the intended semantics are ongoing (dart-lang/language#2326, dart-lang/language#2310). I've created this issue because those discussions revealed that the current implementations of But we can't do that yet: The soundness issue arises because a certain type check isn't performed during evaluation of an One possible explanation for the currently implemented behavior with |
This PR specifies a breaking change to await, and an update to flatten which includes null safety. In particular, **flatten** is changed such that it is defined for type variables and intersection types. It changes `await` to throw when provided a `Future` value of a type that could provide an unsound value if awaited. This differs from the current specification, which instead treats the future as a value, so `await someFuture` can evaluate to `someFuture`. It also differs from the current implementation, which awaits `someFuture` to its value, whether the result is sound or not. See also dart-lang/sdk#49396, #2326.
The language team favored a third option (not mentioned above), as specified in dart-lang/language#2333. The main point is that there is a dynamic error during evaluation of For example: void f() async {
FutureOr<Object> fut = Future<Object?>.value();
var x = await fut; // Throws.
} In this example, the value of The point is, intuitively, that NB: The issue is still blocked, discussions haven't been settled entirely! |
@sigmundch, @rakudrama, @nshahan, @mkustermann, @johnniwinther, the language team discussed this issue today, (Nov 2, 2022). One reasonable way ahead, in many ways, would be to implement the specified behavior: Evaluation of an await expression $a$ of the form \code{\AWAIT{} $e$} where $e$ has
static type $S$ proceeds as follows:
First, the expression $e$ is evaluated to an object $o$. Let $T$ be \flatten{S}.
It is a dynamic type error if the run-time type of $o$ is not a subtype of \code{FutureOr<$T$>}.
If the run-time type of $o$ is a subtype of \code{Future<$T$>}, then let $f$ be $o$;
otherwise let $f$ be the result of creating a new object using the constructor
\code{Future<$T$>.value()} with $o$ as its argument. The language team wants to clarify the situation, and proceed to do this unless there are concerns that we have overlooked. Based on the current behavior, and in particular the example here which illustrates the soundness violation, the situation appears to be the following: At the point where we should test At the language team meeting, concerns were raised about the implications of implementing this change, in particular with respect to the code size, for web configurations. It seems that a test for On the other hand, it should be noted that there is never a need to perform this dynamic check when the static type of What is your impression of the situation? Is there a danger that this change to the implementation could cause programs to get larger or run more slowly, or that it would cause a considerable effort to implement it? |
A language test available in https://dart-review.git.corp.google.com/c/sdk/+/267422. |
From the CFE perspective, we probably want to add properties to the |
Sounds good! Here's a summary of the situation:
|
Did we ever consider to disallow |
Yes, that's very tempting, but we never got around to a decision to actually do that. We should probably allow But we should know more about how breaking this would be. |
If we decide to go this route, then it would be nice to have static type requirements for the argument of As for |
It is specified such that The specification never says 'microtask', but I can't see how this can be implemented in a way that doesn't suspend execution at On the other hand, we could of course make In any case, we should fix the soundness issue now; the syntax used to get a plain suspension may then be dealt with as a separate effort. |
In the DDC implementation we don't have any special casts or type tests of the expression in the |
In discussion with @nshahan , we noted that it's a bit odd that in the second example:
we are proposing to turn this into a runtime error (a cast to I wonder if we should make |
We are not proposing to insert a cast to The flatten operator is what we use for Changing flatten to return Basically:
(I'd be fine with simply not awaiting anything but the static types Now, if we changed the spec to insert an
If we are going to make changes, I'd still prefer to just not await anything if the static type is not |
Sorry, yes I'm not sure how I got confused on this (maybe the cast was the change that got reverted and I read an old version of the spec? Or maybe I just misread it?). The behavior then is perhaps less disruptive, but nonetheless pretty surprising. It feels like a foot gun that: final Object o = Future<Object?>.value(3);
var y = await o;
print(y); will print "Instance of 'Future<Object?>'" but final Object? o = Future<Object?>.value(3);
var y = await o;
print(y); will print "3". |
@leafpetersen wrote:
Sounds like this is a proposal to avoid the run-time type test for I'm afraid we would then need to make import 'dart:async';
class A {}
class B extends A implements Future<Object?> {
Future<Object?> fut = Future.value(CompletelyUnrelated());
asStream() => fut.asStream();
catchError(error, {test}) => fut.catchError(error, test: test);
then<R>(onValue, {onError}) => fut.then(onValue, onError: onError);
timeout(timeLimit, {onTimeout}) => fut.timeout(timeLimit, onTimeout: onTimeout);
whenComplete(action) => fut.whenComplete(action);
}
class CompletelyUnrelated {}
A aFunc() => B();
void main() async {
A a = aFunc();
var x = await a;
print(x.runtimeType); // 'CompletelyUnrelated'.
} |
No, not really. It's to avoid the odd behavior that I describe above where we have something which has a type which is known to be a super type of any possible Future, and yet we choose to make So I think I would keep the runtime behavior as specified, but just change what we infer for
I definitely wouldn't want that. I think we can say that types come in three categories:
It's true that your proposal also means that we will skip awaiting an I'm not dead set on this or anything, just want to explore the options here. |
OK, so we would change the specification of We would presumably change There could then be a certain amount of breakage, because some expressions with static type |
It feels a little special-casey to just do it for If we want to change the behavior, then I think we should try to look at it from a use-case perspective, instead of a soundness-perspective. If we assume that someone writing
Then we can try to imagine which use case different scenarios fall into. If you
If we assume that you never have a All in all, it only differs from the current behavior in non-top types that are supertypes of |
I think this change would definitely apply to
Hmm. When do you expect to get a specific |
We don't recursively flatten on If I have a value with a static type of type I'd probably end up writing I don't want to change the typing of For the behavior, it's only really the case where the FutureOr<Future<int>> ffi = Future<int>.value(1);
var fi = await ffi; The type of So, the behavior of
I can't find any way to reasonably give The behavior of That leaves us with the non- (Except for |
So what are the consequences of these proposals?
Note that the implementations of What do we want? I tend to prefer the currently specified behavior (1). |
As @eernstg pointed out, it is (considerably) more expensive to perform a The fact that we can rely on static types to avoid the dynamic check in many cases (e.g. in IIRC we do have g3 customer code that uses protobuf-based RPC services where the generated service classes's methods use |
Generally, we need the typed check (an In more detail:
That does leave Any type (Maybe the front-end could put a flag on every (non- |
Add AwaitExpression.runtimeTypeCheck to support easy backend implementation of runtime type check. In response to #49396 TEST=pkg/front_end/testcases/general/issue49396.dart Change-Id: I13b9b14566ebc34cdb0811c16a262421417b68e7 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/270723 Commit-Queue: Johnni Winther <[email protected]> Reviewed-by: Martin Kustermann <[email protected]> Reviewed-by: Chloe Stefantsova <[email protected]>
'await e' should check that e is a Future<flatten(S)>, where S is a static type of e before awaiting e. If e is not a Future<flatten(S)>, then 'await e' should await Future.value(e) instead of e. So futures of incompatible type are not awaited and soundness is not violated. TEST=tests/language/async/await_type_check_test.dart (Based on https://dart-review.git.corp.google.com/c/sdk/+/267422.) Fixes #50529 Part of #49396 Change-Id: Ia418db1be6736710abc9be87d95584c50cbc677e Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/273002 Reviewed-by: Erik Ernst <[email protected]> Commit-Queue: Alexander Markov <[email protected]> Reviewed-by: Martin Kustermann <[email protected]>
The definition of flatten was extended in dart-lang/language@a3b45ab2 to include a case for type variables. That change turned out to give rise to a loss of precision in the cases where a type variable does not have |
Said PR, dart-lang/language#2696, has now been landed. |
https://dart-review.googlesource.com/c/sdk/+/274682 updates the 49396 test to expect the behavior of the updated flatten. Sorry about the inconvenience that may be caused by a change at this time! |
Note that https://dart-review.googlesource.com/c/sdk/+/274682 renames the test to |
@scheglov, is there an 'area-analyzer' issue for this? The value of |
I opened #50672 |
Flatten has been adjusted again in order to handle intersection types, cf. dart-lang/language#2713. |
Some expressions have a static type that can lead to a possible soundness issue when awaited. For those expressions an additional runtime check is performed. Issue: #49396 Fixes: #50602 Change-Id: Ief25fbe8c38330cca0c17be4d411780a20ab87a0 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/274729 Reviewed-by: Mark Zhou <[email protected]> Commit-Queue: Nicholas Shahan <[email protected]>
Updated tests have landed, https://dart-review.googlesource.com/c/sdk/+/274682. |
The language specification updates have landed, cf. dart-lang/language@0cff768. |
See #49396 for details. Fixes: #50601 Change-Id: Ie89130cffe642b3e4935d7d02fe2e34f7fc8b12e Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316320 Commit-Queue: Mayank Patke <[email protected]> Reviewed-by: Stephen Adams <[email protected]>
🎉 |
This reverts commit c81711b. Reason for revert: `Internal Error: Runtime type information not available for type_variable_local` - b/295131730 Original change's description: > [dart2js] Add runtime type check for `await`. > > See #49396 for details. > > Fixes: #50601 > Change-Id: Ie89130cffe642b3e4935d7d02fe2e34f7fc8b12e > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316320 > Commit-Queue: Mayank Patke <[email protected]> > Reviewed-by: Stephen Adams <[email protected]> Change-Id: I481b119b6569d1bc9cf2ab80d997a3eb6d06f674 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319421 Reviewed-by: Alexander Thomas <[email protected]> Auto-Submit: Oleh Prypin <[email protected]> Commit-Queue: Oleh Prypin <[email protected]>
This is a reland of commit c81711b Original change's description: > [dart2js] Add runtime type check for `await`. > > See #49396 for details. > > Fixes: #50601 > Change-Id: Ie89130cffe642b3e4935d7d02fe2e34f7fc8b12e > Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/316320 > Commit-Queue: Mayank Patke <[email protected]> > Reviewed-by: Stephen Adams <[email protected]> Change-Id: Ida3258ee3768e8bff0161019511647db8b161473 Bug: #50601 Bug: b/295131730 Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/319462 Commit-Queue: Mayank Patke <[email protected]> Reviewed-by: Stephen Adams <[email protected]>
Uh oh!
There was an error while loading. Please reload this page.
[Edit: This issue has evolved into an implementation issue, hence the 'area-meta' and 'implementation' labels.]
Subissues:
Consider the following program:
This program is accepted by the
dart analyze
,dart
, anddart compile js
without any diagnostic messages, and it printsnull
twice withdart
anddart compile js
. When the commented-out declarations are commented in, all tools report a compile-time error, confirming that the inferred type isObject
for bothx
andy
.This means that the state of
x
andy
is a soundness violation, because the type isObject
and the value is null.So we need to take into account that when the static type of
e
isObject
orFutureOr<Object>
, the expressionawait e
can evaluate to any object including null.[Edit, based on further discussion in dart-lang/language#2326]:
The soundness issue comes up with several other types than
Object
andFutureOr<Object>
, cf. dart-lang/language#2310 as well.Consider
await e
, wheree
has static typeS
. In this situation, the evaluation ofawait e
has the following steps: Evaluatee
to an objecto
. Check whether the dynamic type ofo
is a subtype ofFuture<flatten(S)>
. If yes, then awaito
; otherwise awaitFuture<S>.value(o)
. The soundness issue arises because the check on the dynamic type ofo
does not occur (at least not always).In the example above,
S
isFutureOr<Object>
respectivelyObject
,flatten(S)
isObject
, and the dynamic type of the value ofe
is (a type that implements)Future<Object?>
. So we should awaitFuture<Object>.value(o)
, but the actual behavior is to awaito
itself.This issue is still blocked on discussions in dart-lang/language#2326.The text was updated successfully, but these errors were encountered: