Skip to content

Simplify Filesystem, introduce TestReaderWriter #3873

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 21, 2025

Conversation

davidmorgan
Copy link
Contributor

@davidmorgan davidmorgan commented Feb 21, 2025

For #3811, work in progress.

Move FilesystemCache out of Filesystem to FileBasedAssetReader and InMemoryAssetReaderWriter. Filesystem now just always directly reads/writes.

Move AssetPathProvider out of Filesystem to FileBasedAssetReader and InMemoryAssetReaderWriter. Filesystem now accesses files by path, with the Reader handling the mapping from IDs to path. This removes the last big difference between InMemoryAssetReaderWriter and FileBasedAssetReader, which was that the in-memory version used did not map to paths, storing data by ID.

Add TestReaderWriter with ReaderWriterTesting member that gives access to the internal state for testing; InMemoryAssetReaderWriter and its Filesystem and other AssetReaderState members are now hidden from tests behind this test-specific API.

Copy link

github-actions bot commented Feb 21, 2025

PR Health

@davidmorgan davidmorgan force-pushed the in-memory-to-path branch 2 times, most recently from 1a6c8b1 to f6b0048 Compare February 21, 2025 11:52
@davidmorgan davidmorgan changed the title In memory to path Simplify Filesystem, introduce TestReaderWriter Feb 21, 2025
Move `FilesystemCache` and `AssetPathProvider` out of `Filesystem` and into
the `Reader` classes.
: cache = cache ?? const PassthroughFilesystemCache(),
assets = {};
@override
Future<bool> exists(String path) async => _files.containsKey(path);

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it will be faster to have these not be async and return Future.value(whatnot).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

AssetId('example', 'web/initial.txt'),
'initial',
)
..filesystem.writeAsStringSync(
..testing.writeString(
AssetId('example', 'web/large.txt'),
List.filled(10000, 'large').join(''),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know this is not new code, but isn't this just 'large' * 10000? (as in literally that code will give the same string).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is, done :)

@@ -1489,9 +1489,9 @@ void main() {
);

// Followup build with modified inputs.
var serializedGraph =
result.readerWriter.assets[makeAssetId('a|$assetGraphPath')]!;
result.readerWriter.assets.clear();

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the story on the clear? Here it's replaced by nothing --- below it's replaced by a delete call.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In most of the test cases, like this one, the next run passes the same sources again, so the clear had no effect.

Only cases where the second run wants there to be fewer files actually needs a clear or delete, so now, that's what the tests do :)

Copy link
Contributor Author

@davidmorgan davidmorgan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

AssetId('example', 'web/initial.txt'),
'initial',
)
..filesystem.writeAsStringSync(
..testing.writeString(
AssetId('example', 'web/large.txt'),
List.filled(10000, 'large').join(''),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes it is, done :)

: cache = cache ?? const PassthroughFilesystemCache(),
assets = {};
@override
Future<bool> exists(String path) async => _files.containsKey(path);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@@ -1489,9 +1489,9 @@ void main() {
);

// Followup build with modified inputs.
var serializedGraph =
result.readerWriter.assets[makeAssetId('a|$assetGraphPath')]!;
result.readerWriter.assets.clear();
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In most of the test cases, like this one, the next run passes the same sources again, so the clear had no effect.

Only cases where the second run wants there to be fewer files actually needs a clear or delete, so now, that's what the tests do :)

@davidmorgan davidmorgan merged commit d71fe7d into dart-lang:master Feb 21, 2025
76 checks passed
@davidmorgan davidmorgan deleted the in-memory-to-path branch February 21, 2025 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants