Wrong std value in CIFAR10 config

[nps1ngh]

Hi,

the value of std in the config file:

image-classification-codebase/conf/cifar10.conf

Line 22 in 19a5ce6

std: [0.2023, 0.1994, 0.2010]

seems to be wrong. It should be [0.2470, 0.2435, 0.2616] instead.

See this comment.

Some code to calculate it:

import torchvision

c10 = torchvision.datasets.CIFAR10(
    ".",
    train=True,
    transform=torchvision.transforms.ToTensor(),
)

all_images = torch.stack([t[0] for t in c10])  # .shape == (50_000, 3, 32, 32)
print("mean", all_images.mean(dim=(0, 2, 3)))  # [0.4914, 0.4822, 0.4465]
print("std", all_images.std(dim=(0, 2, 3)))    # [0.2470, 0.2435, 0.2616]

[chenyaofo]

Thanks for your feedback. I need some time to comfirm it.

[nps1ngh]

Seems like the std values were computed by averaging the individual image std values (I was able to reproduce them that way).
In other words, they were calculated using:

all_images.std(dim=(2,3)).mean(dim=0)

instead of

all_images.std(dim=(0, 2, 3))

This is also how the normalization constants in ImageNet came about (see pytorch/vision#1965).
Although, I'd argue this is not exactly calculating the standard deviation over the entire dataset (as pointed out here and also how the torchvision issue referred above came about), I agree with the comment made here and think that retraining the models here (for CIFAR10) also would not make sense.

As such I'll close this issue.