Optional Argument to the Image folder class's constructor to accept specific sub-dirs to load data from.

[Aradhya-Tripathi]

🚀 Feature
Torchvision's Imagefolder class Takes in root as an argument, I want to add an additional argument that takes in sub-directories within root to load data only from those folders.

Motivation
While working with large datasets and a messy folder structure I would prefer to mention the folder names in the base directory that I load my data from, this saves time and a don't have to constantly keep arranging my folders.

Pitch
An optional argument that the Image folder class's constructor takes this argument is an iterable,
The ImageFolder class loads data only from the folders mentioned in the iterable.

Additional context
Thanks

[NicolasHug]

In #3628 we have recently made the DatasetFolder.find_classes() public so that users can customize how classes are discovered. Does that address your use-case already?

[Aradhya-Tripathi]

In #3628 we have recently made the DatasetFolder.find_classes() public so that users can customize how classes are discovered. Does that address your use case already?

in #3628 making the entire find_classes() would be a little extensive for the use-case I'm referring to, just finding sub-classes given my root dir, which could be as simple as an if condition passed to the constructor of DatasetFolder

[NicolasHug]

making the entire find_classes() would be a little extensive

You could do something like (disclaimer: I haven't tested the code):

class MyImageFoler(Imagefolder):

	def __init__(self, *args, **kwargs):
		self.classes_to_select = kwargs.pop('classes_to_select')
		super().__init__(*args, **kwargs)
		
	def find_classes(self, directory):
		classes, _ = super().find_classes(directory)
		# filter out classes here:
		classes = [c in classes if c in self.classes_to_select]
		classes_to_idx =  {cls_name: i for i, cls_name in enumerate(classes)}  # or you could maybe delete keys from `_` instead to preserve class numbers
		return classes, classes_to_idx

I agree it's a little bit of code but it's reasonably simple. We chose to go for that solution instead of adding a new parameter because overriding find_classes allows users to implement the strategy they want, not just to accept a subset of classes. Some may want to e.g. ignore a subset of classes instead. Some may have a completely different dataset structure. As use-cases grow, we'd need to add even more and more parameters, which can end up in bloated APIs.

[Aradhya-Tripathi]

making the entire find_classes() would be a little extensive

You could do something like (disclaimer: I haven't tested the code):

class MyImageFoler(Imagefolder):

	def __init__(self, *args, **kwargs):
		self.classes_to_select = kwargs.pop('classes_to_select')
		super().__init__(*args, **kwargs)
		
	def find_classes(self, directory):
		classes, _ = super().find_classes(directory)
		# filter out classes here:
		classes = [c in classes if c in self.classes_to_select]
		classes_to_idx =  {cls_name: i for i, cls_name in enumerate(classes)}  # or you could maybe delete keys from `_` instead to preserve class numbers
		return classes, classes_to_idx

I agree it's a little bit of code but it's reasonably simple. We chose to go for that solution instead of adding a new parameter because overriding find_classes allows users to implement the strategy they want, not just to accept a subset of classes. Some may want to e.g. ignore a subset of classes instead. Some may have a completely different dataset structure. As use-cases grow, we'd need to add even more and more parameters, which can end up in bloated APIs.

hey, that's exactly what I did for my use case,
ok agreed adding more and more arguments isn't really efficient,
Thanks! :)