-
Notifications
You must be signed in to change notification settings - Fork 13
refactor: Move IO specifics out of Schema, Collection
#109
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #109 +/- ##
==========================================
Coverage 100.00% 100.00%
==========================================
Files 42 45 +3
Lines 2450 2573 +123
==========================================
+ Hits 2450 2573 +123 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
ping @borchero |
borchero
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just some cosmetic comments, thanks! 🚀
|
Thanks @borchero! While refactoring into multiple files, I realized I had completely missed that |
borchero
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's go :D
Motivation
The current implementation of parquet-based storage is strongly coupled to the implementation of the
SchemaandCollectionclasses. This makes it hard to reuse parts of the logic that would also be useful for other future storage backends. To prepare the implementation of such future storage backends, this PR refactors the current logic to introduce a clearer interface for storage backends.Changes
StorageBackend, i.e. a piece of code encapsulating a way of storing data and metadata.StorageBackendinterface