When training an image classifier, it's important to normalize the images using the mean and standard deviation of your actual dataset. Otherwise, the incorrectly normalized features could have elongated valleys in the loss landscape, which can cause optimization problems.

To my dismay, I often see people blindly copy-pasting the same normalization values across different reference implementations tutorials, quickstart demos, Kaggle notebooks, etc. For example, the vit_huge_patch14_244_in21k model was trained using was trained using apparently bogus mean and standard deviation values of (0.5, 0.5, 0.5).

To make matters worse, I wasn't able to easily Google the correct normalization values to use for many popular vision datasets. So, I've computed the correct mean and standard deviation for a bunch of popular image datasets that are available in the torchvision dataset.

Reference values

Note that it's important to only compute these on the train subset of each dataset, otherwise if you include the val split you are letting information leak into the model training. Also note that while the image values natively span [0-255], it's common practice to rescale these values to [0-1] values using Pytorch's ToTensor or ToDtype transforms. So, I've also included the scaled values in the table.

My hope is that these values will be published alongside the official dataset sources, as well as in 3rd party libraries such as torchvision and timm.

In the interest of time, I skipped a bunch of datasets that had gotchas. Please send me a note with the normalization constants if you figure them out!

• Caltech101, Caltech256 have a variable number of channels
• Miscellaneous torchvision initialization errors due to their very inconsistent API: CelebA, EMNIST, FER2013, Flickr8k, Flickr30k, GTSRB, LFWPeople, OxfordIIITPet, Places365, LSUN, PCAM, StanfordCars, CocoDetection
• iNaturalist, because it is 240 GB.

Implementation

I implemented this code using Ray Data, which is a nice library for scaling ML workloads. It has some nice features:

• the data is streamed through the transformation steps, so that you can have data loaders, mappers, and reducers all running concurrently
• ability to distribute the workload to large clusters
• ability to autoscale the cluster based on rules that you define
• ability to use heterogenous compute, e.g. CPUs, GPUs, mixtures of machines, fractional resources, etc.
• much nicer developer and debugging experience than PySpark

The full code to calculate these numbers are available here. The meat of the code is quite simple:

ds = ray.data.from_items(dataset)
# (PIL img, label) -> np.array
ds = ds.map(extract_and_process_image)
# np.array -> per-channel mean, standard deviation
ds = ds.map(compute_channel_stats)
# Count channels from first sample
num_channels = len([k for k in ds.take(1)[0].keys() if k.startswith("mean_")])
# Aggregate statistics across images
results = ds.aggregate(
	*[Mean(f"mean_{i}", alias_name=f"mean_{i}") for i in range(num_channels)],
	*[Mean(f"stddev_{i}", alias_name=f"stddev_{i}") for i in range(num_channels)],
)