Skip to content

Add a data catalog. #419

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 41 commits into from
Nov 2, 2023
Merged

Add a data catalog. #419

merged 41 commits into from
Nov 2, 2023

Conversation

tobiasraabe
Copy link
Member

@tobiasraabe tobiasraabe commented Sep 18, 2023

Todo

  • Remove implementations of PickleNode from tests.
  • Add return_annotation or something similar to PNode.load() to allow to use product annotation with PickleNode.
  • Write tutorial and split content between both.
  • Custom node with cloudpickle

@tobiasraabe tobiasraabe added this to the v0.4.0 milestone Sep 18, 2023
@codecov
Copy link

codecov bot commented Sep 18, 2023

Codecov Report

Attention: 13 lines in your changes are missing coverage. Please review.

Comparison is base (58d940e) 95.00% compared to head (3530a07) 94.96%.

❗ Current head 3530a07 differs from pull request most recent head 6fe3ec4. Consider uploading reports for the commit 6fe3ec4 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #419      +/-   ##
==========================================
- Coverage   95.00%   94.96%   -0.05%     
==========================================
  Files          97       99       +2     
  Lines        8610     8794     +184     
==========================================
+ Hits         8180     8351     +171     
- Misses        430      443      +13     
Flag Coverage Δ
end_to_end 82.28% <82.72%> (+0.03%) ⬆️
integration 39.56% <42.93%> (+0.06%) ⬆️
unit 66.05% <72.25%> (+0.12%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
src/_pytask/build.py 98.86% <100.00%> (ø)
src/_pytask/config_utils.py 96.92% <100.00%> (+0.04%) ⬆️
src/_pytask/dag_command.py 66.41% <100.00%> (ø)
src/_pytask/node_protocols.py 100.00% <ø> (ø)
src/pytask/__init__.py 100.00% <100.00%> (ø)
tests/test_config_utils.py 100.00% <100.00%> (ø)
tests/test_debugging.py 89.89% <ø> (ø)
tests/test_execute.py 99.12% <100.00%> (+0.01%) ⬆️
tests/test_data_catalog.py 97.64% <97.64%> (ø)
src/_pytask/nodes.py 84.92% <78.26%> (-1.49%) ⬇️
... and 1 more

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@tobiasraabe tobiasraabe removed this from the v0.4.0 milestone Sep 18, 2023
@tobiasraabe tobiasraabe marked this pull request as ready for review October 25, 2023 20:42
@tobiasraabe tobiasraabe added this to the v0.4.2 milestone Oct 28, 2023
@tobiasraabe tobiasraabe merged commit 4da749b into main Nov 2, 2023
@tobiasraabe tobiasraabe deleted the draft-datastore branch November 2, 2023 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant