Skip to content

Split post process builder state out of the main graph. #3959

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 11, 2025

Conversation

davidmorgan
Copy link
Contributor

@davidmorgan davidmorgan commented Apr 4, 2025

For #3811.

Per the docs on PostProcessBuilder, post process build steps interact in a very limited way with the rest of the build.

So, split them out.

built_value serialization doesn't support serializing mutable Map or Set out of the box, but I don't particularly want to use BuildMap / BuiltSet for these as there's no advantage to having them be immutable. So add the code for serializing Map+Set here, I'll upstream it.

An explanation of the FullType boilerplate added:

built_value differentiates between generic types known at compile time and at runtime. If the types are known at compile time they're specified with a FullType and don't get written on the wire. So: serializers.serialize([1, 2, 3], specifiedType: FullType(List, [FullType(int)]) gives you back JSON that is minimal, [1,2,3]. If the types are not known, serializers.serialize([1, 2, 3]) then the types do get written on the wire: ['list', ['int', 1], ['int', 2], ['int', 3]]. This works for arbitrarily nested generics, for example a list of lists of int.

The "builderFactories" are used on deserializing to instantiate the types with the right generics--as without mirrors there is no other way to do this in Dart. For fields in serializable types built_value can generate code to add builder factories, but here I'm just serializing collections directly so codegen can't help.

Sorry for the complexity here! It would also work to handcode the new (de)serialization, I did that first, and can switch to that or something like it if you prefer. Both options are not pretty--but I have a slight preference for going fully with codegen/built_value as that seems better long term. We can improve some rought edges in built_value :) and the community can benefit as a result.

Thanks.

Copy link

github-actions bot commented Apr 4, 2025

PR Health

Changelog Entry ✔️
Package Changed Files

Changes to files need to be accounted for in their respective changelogs.

@davidmorgan davidmorgan force-pushed the remove-anchor-nodes branch from 0f0d0b2 to 27729e4 Compare April 4, 2025 11:21
@davidmorgan davidmorgan changed the title Remove anchor nodes Split post process builder state out of the main graph. Apr 4, 2025
@davidmorgan davidmorgan force-pushed the remove-anchor-nodes branch 4 times, most recently from d88218e to 6f58924 Compare April 10, 2025 09:07
@davidmorgan davidmorgan force-pushed the remove-anchor-nodes branch from 6f58924 to dacef53 Compare April 10, 2025 09:27
@davidmorgan davidmorgan marked this pull request as ready for review April 10, 2025 09:39
@davidmorgan davidmorgan requested a review from jensjoha April 10, 2025 09:39
/// Created with empty outputs at the start of the build if it's a new build
/// step; or deserialized with previous build outputs if it has run before.
final Map<String, Map<PostProcessBuildStepId, Set<AssetId>>>
_postProcessBuildStepOutputs = {};

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this really the indentation provided by dart format? (also below)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is ... formatting gets a bit surprising when there are very long identifiers.

There's a format presubmit :) that's checked before the tests run.

buildStepId.input.package,
() => {},
)[buildStepId] =
outputs;

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you must just have forgotten to format it right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, this is the output :)

@@ -67,6 +67,19 @@ class _AssetGraphDeserializer {
graph._add(_deserializeAssetNode(serializedItem as List));
}

final postProcessOutputs =

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(this frankly looks weird too, but I'll stop commenting on formatting now :x)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, it's the long Map<String, ....

@davidmorgan davidmorgan merged commit 3e6e1ac into dart-lang:master Apr 11, 2025
75 checks passed
@davidmorgan davidmorgan deleted the remove-anchor-nodes branch April 11, 2025 08:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants