Allow access to global variables in ffi static binding

[mkustermann]

We're moving towards a direction where all native symbols will be declaratively specified. We have support for declaratively specifying native C functions (see #43889).

Though we'd also want a way to declaratively specify global variables - which can then be accessed from Dart.

[dcharkes]

Possible syntax:

// Global var in native asset, similar to `@Native external` function.
@NativeSymbol(/* optional symbol and asset */); // No type argument, as the Pointer already has it.
external Pointer<Uint8> get greeting;

[simolus3]

Is someone currently working on this? I need this feature to use native assets, so I'm happy to give this a shot myself otherwise.

Possible syntax:

Given that @Natives translate to direct Dart functions, wouldn't a more consistent API be something that does the load and stores implicitly?

@NativeSymbol<Uint8>(/* optional symbol and asset */);
external int greeting; // Field loads and assignments lowered to calls on the pointer.

A downside is that the #50552 might need special consideration (but detecting addressOf(greeting) in the CFE and lowering it should be possible too).

[dcharkes]

Is someone currently working on this? I need this feature to use native assets, so I'm happy to give this a shot myself otherwise.

Possible syntax:

Given that @Natives translate to direct Dart functions, wouldn't a more consistent API be something that does the load and stores implicitly?

@NativeSymbol<Uint8>(/* optional symbol and asset */);
external int greeting; // Field loads and assignments lowered to calls on the pointer.

A downside is that the #50552 might need special consideration (but detecting addressOf(greeting) in the CFE and lowering it should be possible too).

The way that this should be implemented is going to be dlopen+dlsym in C and then conceptually .value in Dart. So I suspect we might want to do addressOf in the same PR.

https://dart-review.googlesource.com/c/sdk/+/284300 completely changes how the dlopen+dlsym happen in the Dart SDK for @Native external functions. So it might be worth waiting until that lands. It changes the storage of the address into the pool rather than a static field in Dart. I'd prefer to avoid landing an implementation the CFE for fields now and then throwing it away in 2 months again. (Sorry for sitting so long on 284300, I've been preoccupied with getting the native assets to work in Flutter.)

[dcharkes]

We should be able to merge both functions and non-function symbols:

// Global var in native asset, similar to `@Native external` function.
@Native<Int64>(/* optional symbol and asset */);
external int get greeting;
external set greeting(int value);

final greetingAddress = addressOf<Int64>(greeting); // Unfortunately no inference, requires extra custom check.

[dcharkes]

cc @lrhn for any bright API ideas.

We could possibly make the global vars nicer by exposing the Pointer instead and using .value as suggested above. However, we don't want to do that with the functions (using Pointer and then asFunction), because we want to do static linking. And it's probably best to keep functions and non-functions consistent.

[simolus3]

We should be able to merge both functions and non-function symbols:

I have seen existing code using @FfiNative to link unary C functions as getters on members . And even though it's probably not intentional, I think the current @Native implementation also allows linking nullary C functions by annotating getters. So there's a question of whether we want to break this.

We could make a distinction by saying that getters/setters are functions in C, whereas annotated fields would be symbols. Or we just break the existing (probably rare?) usages of getters linking functions.

[dcharkes]

We could probably make both nullary functions and non-functions getters (and unary functions and non functions setters).

[lrhn]

The closer the syntax is to how functions are accessed, the better.

@Native<Int64>(/* optional symbol and asset */);
external int greeting;

seems optimal, if @Native<Int64 Function(Int64)>(...) external int foo(int x); is what we do for functions.

If the variable declaration is final, no setter is generated, as normal.
Don't know if we can handle late. Probably not.

We can, and probably should, deliberately allow variables to be treated as nullary getter functions/unary setter functions in Dart, and vice versa.

@Native<Int64 Function()>("something.getFoo")
external int get foo;
@Native<Void Function(Int64)>("something.setFoo")
external set foo(int value);

@Native<Int64>("something.foo") // native variable reference, reading method signature.
external int fetchFoo();
@Native<Int64>("something.foo") // same native variable reference, writing method signature.
external void pushFoo(int value);

Nothing more complicated than this, if at all possible.

(Don't know how we deal with immutable variables, but I guess writing to immutable memory will crash you through a setter the same way it would writing through a pointer. If we can somehow detect that the variable is not writable, we can just give a better error message.)

[dcharkes]

If we can somehow detect that the variable is not writable, we can just give a better error message.

I think in FFIgen we will just not generate a setter if we can detect that something is const char* for example. (Not sure of libclang exposes that. cc @mannprerak2)