[vm/ffi] Add _Compound _offsetInBytes field

This CL changes compounds (structs, unions, and arrays) to be backed
by a TypedDataBase and an int offset.

Before this CL, the compounds where only backed by a TypedDataBase.
This leads to the following issues:

1. Access to nested structs required code for allocating new typed data
   views or pointers, which the optimizer then had to prevent from
   being allocated after inlining.
2. Runtime branching on whether the TypedDataBase was a Pointer or
   TypedData increased code size and prevented inlining.
   #54892
   This could not be properly optimized if in AOT both typed
   data and pointer were flowing into the same compound.
3. Constructing TypedData views required calculating the length of the
   view.

After this CL, accessing nested compounds will lead to accesses on the
original TypedDataBase with an extra offset.

This removes the polymorphism on TypedData vs Pointer, because the
final int/float/Pointer accesses in nested compounds operate on
TypedDataBase.

Also, it simplifies creating an `offsetBy` accessor, because it will
no longer have to be polymorphic in typed data vs pointer, nor will it
have to calculate the length of the field.

Implementation details:

* The changes in the CFE and patch files are straightforward.
* VM: Struct-by-value returns (or callback params) are initialized
  with an offsetInBytes of 0.
* VM: Struct-by-value arguments (and callback return) need to read out
  the offsetInBytes. Before this CL we were passing in the TypedData
  as tagged value. With this CL we are passing the TypedData as tagged
  value and the offset as unboxed int, from 1 IL input to 2 IL
  inputs. (The alternative would have been to take the compound as
  a tagged value, but that would have prevented optimizations from not
  allocating the compound object in the optimizer.
  The FfiCallInstr is updated to have two definitions for the case
  where we were passing in the TypedData previously.
  The NativeReturnInstr is refactored to be able to take two inputs
  instead of 1. (Note that we don't have VariadicInstr only
  VariadicDefinition in the code base. So the instruction is _not_
  implemented as variadic, rather as having a fixed length of 2.)
* dart2wasm does no longer support nested compounds due to the
  compound implementation only storing a pointer address.
  #55083

Intending to land this after
https://dart-review.googlesource.com/c/sdk/+/353101.

TEST=test/ffi

CoreLibraryReviewExempt: VM and WASM-only implementation change.
Closes: #54892
Bug: #44589
Change-Id: I8749e21094bf8fa2d5ff1e48b6b002c375232eb5
Cq-Include-Trybots: dart-internal/g3.dart-internal.try:g3-cbuild-try
Cq-Include-Trybots: dart/try:vm-aot-android-release-arm64c-try,vm-aot-android-release-arm_x64-try,vm-aot-linux-debug-x64-try,vm-aot-linux-debug-x64c-try,vm-aot-mac-release-arm64-try,vm-aot-mac-release-x64-try,vm-aot-obfuscate-linux-release-x64-try,vm-aot-optimization-level-linux-release-x64-try,vm-aot-win-debug-arm64-try,vm-aot-win-debug-x64-try,vm-aot-win-debug-x64c-try,vm-appjit-linux-debug-x64-try,vm-asan-linux-release-x64-try,vm-checked-mac-release-arm64-try,vm-eager-optimization-linux-release-ia32-try,vm-eager-optimization-linux-release-x64-try,vm-ffi-android-debug-arm-try,vm-ffi-android-debug-arm64c-try,vm-ffi-qemu-linux-release-arm-try,vm-ffi-qemu-linux-release-riscv64-try,vm-fuchsia-release-x64-try,vm-linux-debug-ia32-try,vm-linux-debug-x64-try,vm-linux-debug-x64c-try,vm-mac-debug-arm64-try,vm-mac-debug-x64-try,vm-msan-linux-release-x64-try,vm-reload-linux-debug-x64-try,vm-reload-rollback-linux-debug-x64-try,vm-ubsan-linux-release-x64-try,vm-win-debug-arm64-try,vm-win-debug-x64-try,vm-win-debug-x64c-try,vm-win-release-ia32-try
Reviewed-on: https://dart-review.googlesource.com/c/sdk/+/354226
Reviewed-by: Tess Strickland <[email protected]>

Author: dcharkes

pkg/dart2wasm/lib/ffi_native_transformer.dart

@@ -257,7 +257,7 @@ class WasmFfiNativeTransformer extends FfiNativeTransformer {
         StaticInvocation(wasmF32FromDouble, Arguments([expr])),
       NativeType.kDouble =>
         StaticInvocation(wasmF64FromDouble, Arguments([expr])),
-      NativeType.kPointer || NativeType.kStruct => expr,
+      NativeType.kPointer => expr,
       NativeType.kBool => StaticInvocation(wasmI32FromBool, Arguments([expr])),
       NativeType.kVoid => null,
       _ => throw '_dartValueToFfiValue: $abiTypeNativeType cannot be converted'
@@ -297,7 +297,6 @@ class WasmFfiNativeTransformer extends FfiNativeTransformer {
 
       case NativeType.kPointer:
       case NativeType.kVoid:
-      case NativeType.kStruct:
         return expr;
 
       case NativeType.kUint64:
@@ -319,6 +318,7 @@ class WasmFfiNativeTransformer extends FfiNativeTransformer {
       case NativeType.kNativeInteger:
       case NativeType.kNativeType:
       case NativeType.kOpaque:
+      case NativeType.kStruct:
         throw '_ffiValueToDartValue: $nativeType cannot be converted';
     }
   }

pkg/dart2wasm/lib/intrinsics.dart

@@ -187,10 +187,21 @@ class Intrinsifier {
       // A compound (subclass of Struct or Union) is represented by its i32
       // address. The _typedDataBase field contains a Pointer pointing to the
       // compound, whose representation is the same.
+      // TODO(https://dartbug.com/55083): Implement structs backed by TypedData.
       codeGen.wrap(receiver, w.NumType.i32);
       return w.NumType.i32;
     }
 
+    // _Compound._offsetInBytes
+    if (cls == translator.ffiCompoundClass && name == '_offsetInBytes') {
+      // A compound (subclass of Struct or Union) is represented by its i32
+      // address. The _offsetInBytes field contains is always 0.
+      // This also breaks nested structs, which are currently not used.
+      // TODO(https://dartbug.com/55083): Implement structs backed by TypedData.
+      b.i64_const(0);
+      return w.NumType.i64;
+    }
+
     // Pointer.address
     if (cls == translator.ffiPointerClass && name == 'address') {
       // A Pointer is represented by its i32 address.

pkg/front_end/testcases/general/ffi_external_in_part_file.dart.strong.transformed.expect

@@ -13,18 +13,18 @@ final class Struct1ByteInt extends ffi::Struct {
   synthetic constructor •() → self::Struct1ByteInt
     : super ffi::Struct::•()
     ;
-  constructor #fromTypedDataBase(synthesized core::Object #typedDataBase) → self::Struct1ByteInt
-    : super ffi::Struct::_fromTypedDataBase(#typedDataBase)
+  constructor #fromTypedDataBase(synthesized core::Object #typedDataBase, synthesized core::int #offsetInBytes) → self::Struct1ByteInt
+    : super ffi::Struct::_fromTypedDataBase(#typedDataBase, #offsetInBytes)
     ;
   constructor #fromTypedData(synthesized typ::TypedData #typedData, synthesized core::int #offset, synthesized core::int #sizeInBytes) → self::Struct1ByteInt
     : super ffi::Struct::_fromTypedData(#typedData, #offset, #sizeInBytes)
     ;
   @#C7
   get a0() → core::int
-    return ffi::_loadInt8(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Struct1ByteInt::a0#offsetOf);
+    return ffi::_loadInt8(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Struct1ByteInt::a0#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num});
   @#C7
   set a0(synthesized core::int #externalFieldValue) → void
-    return ffi::_storeInt8(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Struct1ByteInt::a0#offsetOf, #externalFieldValue);
+    return ffi::_storeInt8(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Struct1ByteInt::a0#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num}, #externalFieldValue);
   @#C9
   static get a0#offsetOf() → core::int
     return #C11.{core::List::[]}(ffi::_abi()){(core::int) → core::int*};
@@ -39,7 +39,7 @@ static method notMain() → void {
 @#C9
 static method /* from org-dartlang-testcase:///ffi_external_in_part_lib.dart */ returnStruct1ByteIntNative(core::int a0) → self::Struct1ByteInt
   return block {
-    _in::_nativeEffect(new self::Struct1ByteInt::#fromTypedDataBase(typ::Uint8List::•(#C12)));
+    _in::_nativeEffect(new self::Struct1ByteInt::#fromTypedDataBase(typ::Uint8List::•(#C12), #C10));
   } =>self::_returnStruct1ByteIntNative$Method$FfiNative(a0);
 @#C21
 external static method /* from org-dartlang-testcase:///ffi_external_in_part_lib.dart */ _returnStruct1ByteIntNative$Method$FfiNative(core::int #t0) → self::Struct1ByteInt;
@@ -70,7 +70,7 @@ constants  {
 
 Extra constant evaluation status:
 Evaluated: InstanceInvocation @ org-dartlang-testcase:///ffi_external_in_part_file.dart:11:36 -> IntConstant(-1)
-Extra constant evaluation: evaluated: 26, effectively constant: 1
+Extra constant evaluation: evaluated: 33, effectively constant: 1
 
 
 Constructor coverage from constants:

pkg/front_end/testcases/general/ffi_external_in_part_file.dart.weak.transformed.expect

@@ -13,18 +13,18 @@ final class Struct1ByteInt extends ffi::Struct {
   synthetic constructor •() → self::Struct1ByteInt
     : super ffi::Struct::•()
     ;
-  constructor #fromTypedDataBase(synthesized core::Object #typedDataBase) → self::Struct1ByteInt
-    : super ffi::Struct::_fromTypedDataBase(#typedDataBase)
+  constructor #fromTypedDataBase(synthesized core::Object #typedDataBase, synthesized core::int #offsetInBytes) → self::Struct1ByteInt
+    : super ffi::Struct::_fromTypedDataBase(#typedDataBase, #offsetInBytes)
     ;
   constructor #fromTypedData(synthesized typ::TypedData #typedData, synthesized core::int #offset, synthesized core::int #sizeInBytes) → self::Struct1ByteInt
     : super ffi::Struct::_fromTypedData(#typedData, #offset, #sizeInBytes)
     ;
   @#C7
   get a0() → core::int
-    return ffi::_loadInt8(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Struct1ByteInt::a0#offsetOf);
+    return ffi::_loadInt8(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Struct1ByteInt::a0#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num});
   @#C7
   set a0(synthesized core::int #externalFieldValue) → void
-    return ffi::_storeInt8(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Struct1ByteInt::a0#offsetOf, #externalFieldValue);
+    return ffi::_storeInt8(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Struct1ByteInt::a0#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num}, #externalFieldValue);
   @#C9
   static get a0#offsetOf() → core::int
     return #C11.{core::List::[]}(ffi::_abi()){(core::int) → core::int*};
@@ -39,7 +39,7 @@ static method notMain() → void {
 @#C9
 static method /* from org-dartlang-testcase:///ffi_external_in_part_lib.dart */ returnStruct1ByteIntNative(core::int a0) → self::Struct1ByteInt
   return block {
-    _in::_nativeEffect(new self::Struct1ByteInt::#fromTypedDataBase(typ::Uint8List::•(#C12)));
+    _in::_nativeEffect(new self::Struct1ByteInt::#fromTypedDataBase(typ::Uint8List::•(#C12), #C10));
   } =>self::_returnStruct1ByteIntNative$Method$FfiNative(a0);
 @#C21
 external static method /* from org-dartlang-testcase:///ffi_external_in_part_lib.dart */ _returnStruct1ByteIntNative$Method$FfiNative(core::int #t0) → self::Struct1ByteInt;
@@ -70,7 +70,7 @@ constants  {
 
 Extra constant evaluation status:
 Evaluated: InstanceInvocation @ org-dartlang-testcase:///ffi_external_in_part_file.dart:11:36 -> IntConstant(-1)
-Extra constant evaluation: evaluated: 26, effectively constant: 1
+Extra constant evaluation: evaluated: 33, effectively constant: 1
 
 
 Constructor coverage from constants:

pkg/front_end/testcases/general/ffi_sample.dart.strong.expect

@@ -28,6 +28,7 @@ class Coordinate extends ffi::Struct {
   }
   abstract member-signature method noSuchMethod(core::Invocation* invocation) → dynamic; -> core::Object::noSuchMethod
   abstract member-signature get _typedDataBase() → core::Object*; -> ffi::_Compound::_typedDataBase
+  abstract member-signature get _offsetInBytes() → core::int*; -> ffi::_Compound::_offsetInBytes
   abstract member-signature get _identityHashCode() → core::int*; -> core::Object::_identityHashCode
   abstract member-signature method _instanceOf(dynamic instantiatorTypeArguments, dynamic functionTypeArguments, dynamic type) → core::bool*; -> core::Object::_instanceOf
   abstract member-signature method _simpleInstanceOf(dynamic type) → core::bool*; -> core::Object::_simpleInstanceOf

pkg/front_end/testcases/general/ffi_sample.dart.strong.transformed.expect

@@ -16,21 +16,22 @@ import "package:ffi/ffi.dart";
 
 @#C7
 class Coordinate extends ffi::Struct {
-  constructor #fromTypedDataBase(synthesized core::Object #typedDataBase) → self::Coordinate
-    : super ffi::Struct::_fromTypedDataBase(#typedDataBase)
+  constructor #fromTypedDataBase(synthesized core::Object #typedDataBase, synthesized core::int #offsetInBytes) → self::Coordinate
+    : super ffi::Struct::_fromTypedDataBase(#typedDataBase, #offsetInBytes)
     ;
   constructor #fromTypedData(synthesized typ::TypedData #typedData, synthesized core::int #offset, synthesized core::int #sizeInBytes) → self::Coordinate
     : super ffi::Struct::_fromTypedData(#typedData, #offset, #sizeInBytes)
     ;
   static factory allocate(ffi::Allocator* allocator, core::double* x, core::double* y, ffi::Pointer<self::Coordinate*>* next) → self::Coordinate* {
-    return let final self::Coordinate* #t1 = new self::Coordinate::#fromTypedDataBase(allocator.{ffi::Allocator::allocate}<self::Coordinate*>(self::Coordinate::#sizeOf){(core::int, {alignment: core::int?}) → ffi::Pointer<self::Coordinate*>}!) in block {
+    return let final self::Coordinate* #t1 = new self::Coordinate::#fromTypedDataBase(allocator.{ffi::Allocator::allocate}<self::Coordinate*>(self::Coordinate::#sizeOf){(core::int, {alignment: core::int?}) → ffi::Pointer<self::Coordinate*>}!, #C8) in block {
       #t1.{self::Coordinate::x} = x;
       #t1.{self::Coordinate::y} = y;
       #t1.{self::Coordinate::next} = next;
     } =>#t1;
   }
   abstract member-signature method noSuchMethod(core::Invocation* invocation) → dynamic; -> core::Object::noSuchMethod
   abstract member-signature get _typedDataBase() → core::Object*; -> ffi::_Compound::_typedDataBase
+  abstract member-signature get _offsetInBytes() → core::int*; -> ffi::_Compound::_offsetInBytes
   abstract member-signature get _identityHashCode() → core::int*; -> core::Object::_identityHashCode
   abstract member-signature method _instanceOf(dynamic instantiatorTypeArguments, dynamic functionTypeArguments, dynamic type) → core::bool*; -> core::Object::_instanceOf
   abstract member-signature method _simpleInstanceOf(dynamic type) → core::bool*; -> core::Object::_simpleInstanceOf
@@ -40,30 +41,30 @@ class Coordinate extends ffi::Struct {
   abstract member-signature get hashCode() → core::int*; -> core::Object::hashCode
   abstract member-signature method toString() → core::String*; -> core::Object::toString
   abstract member-signature get runtimeType() → core::Type*; -> core::Object::runtimeType
-  @#C9
+  @#C10
   static get /*isNonNullableByDefault*/ x#offsetOf() → core::int
     return #C11.{core::List::[]}(ffi::_abi()){(core::int) → core::int*};
   @#C12
   get x() → core::double*
-    return ffi::_loadDouble(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::x#offsetOf);
+    return ffi::_loadDouble(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::x#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num});
   set x(synthesized core::double* #v) → void
-    return ffi::_storeDouble(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::x#offsetOf, #v);
-  @#C9
+    return ffi::_storeDouble(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::x#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num}, #v);
+  @#C10
   static get /*isNonNullableByDefault*/ y#offsetOf() → core::int
     return #C14.{core::List::[]}(ffi::_abi()){(core::int) → core::int*};
   @#C12
   get y() → core::double*
-    return ffi::_loadDouble(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::y#offsetOf);
+    return ffi::_loadDouble(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::y#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num});
   set y(synthesized core::double* #v) → void
-    return ffi::_storeDouble(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::y#offsetOf, #v);
-  @#C9
+    return ffi::_storeDouble(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::y#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num}, #v);
+  @#C10
   static get /*isNonNullableByDefault*/ next#offsetOf() → core::int
     return #C16.{core::List::[]}(ffi::_abi()){(core::int) → core::int*};
   get next() → ffi::Pointer<self::Coordinate*>*
-    return ffi::_loadPointer<self::Coordinate*>(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::next#offsetOf);
+    return ffi::_loadPointer<self::Coordinate*>(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::next#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num});
   set next(synthesized ffi::Pointer<self::Coordinate*>* #v) → void
-    return ffi::_storePointer<self::Coordinate*>(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::next#offsetOf, #v);
-  @#C9
+    return ffi::_storePointer<self::Coordinate*>(this.{ffi::_Compound::_typedDataBase}{core::Object}, self::Coordinate::next#offsetOf.{core::num::+}(this.{ffi::_Compound::_offsetInBytes}{core::int}){(core::num) → core::num}, #v);
+  @#C10
   static get /*isNonNullableByDefault*/ #sizeOf() → core::int*
     return #C19.{core::List::[]}(ffi::_abi()){(core::int) → core::int*};
 }
@@ -77,10 +78,10 @@ constants  {
   #C5 = null
   #C6 = ffi::_FfiStructLayout {fieldTypes:#C4, packing:#C5}
   #C7 = core::pragma {name:#C1, options:#C6}
-  #C8 = "vm:prefer-inline"
-  #C9 = core::pragma {name:#C8, options:#C5}
-  #C10 = 0
-  #C11 = <core::int*>[#C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10, #C10]
+  #C8 = 0
+  #C9 = "vm:prefer-inline"
+  #C10 = core::pragma {name:#C9, options:#C5}
+  #C11 = <core::int*>[#C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8, #C8]
   #C12 = ffi::Double {}
   #C13 = 8
   #C14 = <core::int*>[#C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13, #C13]

pkg/front_end/testcases/general/ffi_sample.dart.weak.expect

@@ -21,6 +21,7 @@ class Coordinate extends ffi::Struct {
   }
   abstract member-signature method noSuchMethod(core::Invocation* invocation) → dynamic; -> core::Object::noSuchMethod
   abstract member-signature get _typedDataBase() → core::Object*; -> ffi::_Compound::_typedDataBase
+  abstract member-signature get _offsetInBytes() → core::int*; -> ffi::_Compound::_offsetInBytes
   abstract member-signature get _identityHashCode() → core::int*; -> core::Object::_identityHashCode
   abstract member-signature method _instanceOf(dynamic instantiatorTypeArguments, dynamic functionTypeArguments, dynamic type) → core::bool*; -> core::Object::_instanceOf
   abstract member-signature method _simpleInstanceOf(dynamic type) → core::bool*; -> core::Object::_simpleInstanceOf

pkg/front_end/testcases/general/ffi_sample.dart.weak.modular.expect

@@ -21,6 +21,7 @@ class Coordinate extends ffi::Struct {
   }
   abstract member-signature method noSuchMethod(core::Invocation* invocation) → dynamic; -> core::Object::noSuchMethod
   abstract member-signature get _typedDataBase() → core::Object*; -> ffi::_Compound::_typedDataBase
+  abstract member-signature get _offsetInBytes() → core::int*; -> ffi::_Compound::_offsetInBytes
   abstract member-signature get _identityHashCode() → core::int*; -> core::Object::_identityHashCode
   abstract member-signature method _instanceOf(dynamic instantiatorTypeArguments, dynamic functionTypeArguments, dynamic type) → core::bool*; -> core::Object::_instanceOf
   abstract member-signature method _simpleInstanceOf(dynamic type) → core::bool*; -> core::Object::_simpleInstanceOf

pkg/front_end/testcases/general/ffi_sample.dart.weak.outline.expect

@@ -16,6 +16,7 @@ class Coordinate extends ffi::Struct {
     ;
   abstract member-signature method noSuchMethod(core::Invocation* invocation) → dynamic; -> core::Object::noSuchMethod
   abstract member-signature get _typedDataBase() → core::Object*; -> ffi::_Compound::_typedDataBase
+  abstract member-signature get _offsetInBytes() → core::int*; -> ffi::_Compound::_offsetInBytes
   abstract member-signature get _identityHashCode() → core::int*; -> core::Object::_identityHashCode
   abstract member-signature method _instanceOf(dynamic instantiatorTypeArguments, dynamic functionTypeArguments, dynamic type) → core::bool*; -> core::Object::_instanceOf
   abstract member-signature method _simpleInstanceOf(dynamic type) → core::bool*; -> core::Object::_simpleInstanceOf