Skip to content

Conversation

normanrz
Copy link
Collaborator

Adds a new capability to the UPath constructor for copying UPath objects including their attached kwargs.

a = UPath("s3://bucket/folder", key="...", secret="...")
b = UPath(a)

assert b._kwargs["key"] == "..."

Fixes #49

@normanrz
Copy link
Collaborator Author

To further motivate this: Copying is also the behavior of the pathlib.Path constructor. A user can pass a valid argument in either a str, os.PathLike or pathlib.Path instance and a new Path instance will be created. This is handy for normalizing method arguments, like this:

def do_stuff(path: Union[str, os.PathLike, Path]) -> None:
    path = Path(path)
    assert isinstance(path, Path)
    ...

When UPath in its current form, we have to do the following (in order to not lose essential kwargs):

def do_stuff(path: Union[str, os.PathLike, Path, UPath]) -> None:
    path = make_upath(path)
    assert isinstance(path, UPath)
    ...

def make_upath(maybe_path: Union[str, PathLike, Path, UPath]) -> UPath:
    return maybe_path if isinstance(maybe_path, UPath) else UPath(maybe_path)

@normanrz normanrz self-assigned this Apr 6, 2022
Copy link
Contributor

@jstriebel jstriebel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's very useful IMO 👍 I added a few comments:

upath/core.py Outdated
Comment on lines 138 to 142
return cls.__new__(
cls,
*new_args,
**new_kwargs,
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might also rewrite args and kwargs and simply continue with the constructor, but also fine like this.

upath/core.py Outdated
@@ -123,6 +123,24 @@ class UPath(pathlib.Path, PureUPath, metaclass=UPathMeta):
_default_accessor = _FSSpecAccessor

def __new__(cls, *args, **kwargs):
if len(args) == 1 and isinstance(args[0], cls):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this also check for empty kwargs?

Suggested change
if len(args) == 1 and isinstance(args[0], cls):
if len(args) == 1 and isinstance(args[0], cls) and len(kwargs) == 0:

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if someone wants to do something like:

a = UPath("s3://bucket/folder", key="...", secret="...")
b = UPath(a, 'file')

I think checking if len(args) = 1 would make this not possible. Could we instead pop the first arg if its an instance of UPath and append the rest of args to new_args? I would also say allow kwargs to be set and do something like new_kwargs.update(kwargs).

upath/core.py Outdated
Comment on lines 128 to 136
new_args = (
other._format_parsed_parts(
other._drv, other._root, other._parts
),
)
new_kwargs = {}
if hasattr(other, "_kwargs"):
new_kwargs = other._kwargs.copy()
new_kwargs.pop("_url", None)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You could directly use __reduce__ instead:

Suggested change
new_args = (
other._format_parsed_parts(
other._drv, other._root, other._parts
),
)
new_kwargs = {}
if hasattr(other, "_kwargs"):
new_kwargs = other._kwargs.copy()
new_kwargs.pop("_url", None)
_, new_args, new_kwargs = other.__reduce__()

@andrewfulton9
Copy link
Collaborator

This looks good to me. I'll go ahead and merge it. Thanks @normanrz!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Copy a UPath object
3 participants