Skip to content

File-based archive/extractor data loaders? #1247

@Fil

Description

@Fil

In the discussion in #1222 we're thinking of using the filesystem to generate multiple files from a data loader. This could be better for huge files (e.g. video) than having to go through “zipping”; another benefit would be to make it easier for the user, who is used to saving many files, and not so much to load a zip module/extension (although we provide clear examples of how to do that).

An idea could be to use the .list extension (based on the research I did in #207, both .uri and .list seem to be usable, and .list is a better name); the data loader would write files "somewhere"[1], and return a list of (relative) file names.

[1] Would we provide this location (directory name) with an ENV variable? Would we create that directory? Would we leave it up to the script to create it—with risks of conflicts and no proper way to clean up?

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestquestionFurther information is requested

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions