Functional/Expression macro

[rrousselGit]

There have been a few discussions related to this (such as jakemac53/macro_prototype#29), but I don't think there's a problem language issue for it yet, so here it is

Long story short, the idea for this proposal is to allow metaprogramming to define macros that are applied on expressions, such users can write:

void function() {
  final value = @macro(params);
}

And the macros would be able to replace the final value = @macro(params); code (but not what is before and after this code) with a modified expression.

This could enable a variety of useful patterns, such as:

Implementing an await-like keyword for Widget builders

A common issue with Flutter widgets is that they tend to get rapidly nested:

Widget build(context) {
  return StreamBuilder<A>(
    stream: aStream,
    builder: (context, AsyncSnasphot<A> aSnapshot) {
      return ValueListenableBuilder<B>(
        animation: bListenable,
        builder: (context, B b, _) => Text('${a.data} $b),
      );
    }
  );
}

This is an issue similar to Future.then, which the language solves with await, but Widgets have no equivalent.

With expression macros, we could instead write:

Widget build(context) {
  AsyncSnapshot<A> aSnapshot = @StreamMacro(aStream);
  B b = @ValueListenableMacro(bListenable);

  return Text('${a.data} $b);
}

and the macros would desugar this code into the previous code

Stateful functional widgets

Through expression macros, it could be possible to drastically simplify stateful widgets by offering a @state macro, typically combined with a @functionalWidget macro, such that we'd define:

@functionalWidget
Widget $MyWidget(_MyWidgetState state) {
  var count = @state(0);
  
   return Column(
    children: [
      Text('Clicked $count times'),
      ElevatedButton(onTap: () => count++),
    ]
  );
}

And this would desugar var count = @state(0) into:

int get count => state.count;
set count(int value) => state.count++;

such that full final code would be:

@functionalWidget
Widget $MyWidget(_MyWidgetState state) {
  int get count => state.count;
  set count(int value) => state.count++;

  return Column(
    children: [
      Text('Clicked $count times'),
      ElevatedButton(onTap: () => count++),
    ]
  );
}

class MyWidget extends StatefulWidget {
  const MyWidget({ Key? key }) : super(key: key);

  @override
  _MyWidgetState createState() => _MyWidgetState();
}

class _MyWidgetState extends State<MyWidget> {
  int _count = 0;
  int get count => _count;
  set count(int value) {
    setState(() => _count = value);
  }

  @override
  Widget build(BuildContext context) {
    return $MyWidget(this);
  }
}

(this assumes that we can define local gettters/setters, but I think that's a reasonable request)

Guards

A common thing when dealing with null or functional classes like Result/Either is to do:

Class? function() {
  final a = getA();
  if (a == null) return null;
  final b = getB();
  if (b == null) return null;
  <some logic>
}

or:

Result<Class> function() {
  Result<A> a = getA();
  if (a.isError) return Result<Class>.errorFrom(a);
  Result<B> b = getB();
  if (b.isError) return Result<Class>.errorFrom(b);
  <some logic>
}

...

This could be simplified by a @guard macro, such that we would define:

Class? function() {
  final a = @guard(getA());
  final b = @guard(getB());
  <some logic>
}

and this would generate the if (x == null) return null / if (x.isError) return Result<T>.errorFrom(x) for us.

Custom compiler optimisations

Rather than simplifying existing code, an alternate use-case would be to make existing code more performant.

A use-case I personally have is with Riverpod, where users can do:

const provider = Provider<A>(..);
const anotherProvider = Provider<B>(...)

<...>

class Example extends ConsumerWidget {
  const Example({ Key? key }) : super(key: key);

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    A a = ref.watch(provider);
    B b = ref.watch(another);

    ...
  }
}

where ref.watch works by doing a Map.putIfAbsent, such that the previous ref.watch usages are roughly equal to:

@override
Widget build(BuildContext context, WidgetRef ref) {
  A a = ref.listeners.putIfAbsent(provider, () => provider.listen(...));
  B b = ref.listeners.putIfAbsent(another, () => provider.listen(...));

  ...
}

This works fine, but it would be better if we could remove this Map. With expression macros, we could possibly implement a @watch, such that we'd have:

@override
Widget build(BuildContext context, ExampleRef ref) {
  A a = @watch(provider);
  B b = @watch(another);
...
}

and this would equal:

@override
Widget build(BuildContext context, ExampleRef ref) {
  A a = ref.providerListener ??= provider.listen(...);
  B b = ref.anotherListener ??= another.listen(...);
...
}

abstract class ExampleRef {
  Subscription? providerListener;
  Subscription ? anotherListener;
}

effectively removing the Map in the process by exploding it into class properties

Dependency tracking without AST parsing

Using the same code as the previous example, Riverpod would like to generate a static list of dependencies associated with a "provider".

The idea is that with:

final a = Provider<A>(...);
final b = Provider<B>(...);

final example = Provider<Example>((ref) {
  A a = ref.watch(a);
  String str = ref.watch(b.select((B b) => b.str));
  return <...>
}, dependencies: {a, b});

we would like macros to generate the dependencies parameter of example. The problem being, existing macro proposals currently do not allow macros to access the AST. Which means something like:

@autoDependencies
final example = Provider<Example>((ref) {
  ...
});

would not be able to search for ref.watch invocations to generate that dependencies list.

But if we replaced ref.watch with a @watch macro, we could possibly have that @watch do the generation, such that individual @watch usage would all declare their parameter in the dependencies list

Note:
I am aware that this use-case would likely require subtle changes to the Provider usage to work. Because macros likely wouldn't be able to push items into a collection. The important is being able to statically extra calls to ref.watch within the function. From there, the Provider definition could change to match the requirements.

[munificent]

Long story short, the idea for this proposal is to allow metaprogramming to define macros that are applied on expressions, such users can write:

void function() {
  final value = @macro(params);
}

And the macros would be able to replace the final value = @macro(params); code (but not what is before and after this code) with a modified expression.

I'm not sure what you mean by "but not what is before and after this code". In all of your examples, it appears that the macro is not just changing the expression @macro(params), but the entire variable declaration surrounding it final value = @macro(params);`. If that's the goal, why isn't the macro applied to the variable?

@macro(params);
final value;

?

[rrousselGit]

I'm not sure what you mean by "but not what is before and after this code".

I meant that with:

print(a);
var foo = @macro(...);
print(b)

then the macro can only impact var foo = @macro(...);. It wouldn't be able to change the prints in any way

If that's the goal, why isn't the macro applied to the variable?

@macro(params);
final value;

?

That would make sense too. I don't have a strong attachment to this exact syntax

Although I'm not sure how type-inference would work in this case.
Similarly, the ref.watch and StreamBuilder examples aren't necessarily declaring a new variable everytimes.

I think it'd be valuable to allow:

Widget build(context) {
  return Text(@ValueListenableMacro(listenable));
}

which would generate:

Widget build(context) {
  return ValueListenableBuilder<String>(
    valueListenable: listenable,
    builder: (context, value, _) => Text(value),
  );
}

Same thing for @watch or @guard.

Although I could see this be complex to support. I'd be fine with only allowing:

@macro(...)
final value

[TimWhiting]

A couple things:

If the thing behind the ... is an initializer expression, having it formatted this way is rather weird.

@macro(...)
final value;

To clarify, I think what is being asked is expression macros that are able to replace the statement that they are a part of (not just the expression). The macro takes in an expression, but actually produces multiple statements including for example local getter / setters, which then get further transformed in some cases into class fields / getters / setters or in some case classes. This is at odds with the current macro proposal in which expression macros (if accepted) would be run during a 4th phase that would 1. not be run before the @functionalWidget annotation 2. Not be able to introduce new types. However, I think that they have interesting use cases that should be thought about.

Personally I think allowing annotations to access a some type of AST (could be serialized) for the annotated declaration would be less invasive then some of these solutions that run up against the limitations that expression macros are likely to have. As of #1861, Code objects are being considered for macro parameters, and I believe a similar interface for this sort of info could be useful in several of the situations that @rrousselGit mentions.

So the minimal set of things I see that are required to enable these use cases while still being more inline with the current macro proposal are:

1. statement metadata Statement metadata #1652
2. expression macros that can replace the immediately enclosing statement
3. macro introspection of the AST of the annotated declaration (which then would inspect the AST for things that are annotated or written specially)
4. adding parameters to a constructor call in an initializer (though this could potentially be done with 2)

With access to the AST I don't think you would actually need local getters / setters (which would not really be a language feature needed if the intermediate generated getter / setter were omitted or removed from the final generated code).

[Levi-Lesches]

@rrousselGit, here are some observations:

---

As you noted in your StreamBuilder example, your use of macros isn't as simple as "replace the line with the macro call", because you move the following return statement into ValueListenableBuilder.builder. Which means that your claim:

I meant that with:

print(a);
var foo = @macro(...);
print(b)

then the macro can only impact var foo = @macro(...);. It wouldn't be able to change the prints in any way

doesn't exactly hold.

---

Note that your count example could make use of external:

@state(0)
external count;

generates

int get count => state.count;
set count(int value) => state.count++;

---

Your @guard example looks like non-local returns (#1776)

---

Overall, the notion of an @macro(expression) evaluating into a value feels a lot like a regular function(argument). Maybe we're better off working to improve features of regular functions instead of arbitrarily dividing functions and macros as "input to output tools"

[rrousselGit]

As you noted in your StreamBuilder example, your use of macros isn't as simple as "replace the line with the macro call", because you move the following return statement into ValueListenableBuilder.builder. Which means that your claim:

That is not quite true, as StreamBuilder cases wouldn't involve modifying the code before/after. Rather it would be encapsulated in a block

This is similar to how macros on function may allow adding try/catch, such that:

@action
void fn() {
  print('foo');
}

becomes:

@action
void fn() {
  try {
  print('foo');
  } catch (...) {...}
}

The original content of the function wasn't modified, but simply wrapped.

[rrousselGit]

Another use-case could be implementing a cancellable async/await

Like:

class Example {
  @cancelable
  CancelableOperation<int> fn() {
    Future<T> future;

    T value = @cancellableAwait(future);

    return 42;
  });
}

[jodinathan]

this would open up so much potential to create frameworks