It's silly that this works as well as it does:
You can augment a dataset of images by mixing different classes together. Join different pictures using different opacities and turn the labels into continuous values.
This is crazy.
Suppose you have a dataset with 1,000 cat and 1,000 dog images. The labels that represent cats are x1 = 0. The labels representing dogs are x2 = 1.
Using different alpha values, you can now take an image of a cat and blend it with a dog. Instead of using 0 or 1 as the label, you'll use the blended value.
I sketched this for you. I combined a cat at 80% opacity with a dog at 20% opacity, and the new label is 0.8.
I would have never expected this to work.
The name of this technique is "MixUp." It improves the ability of a model to generalize. It also prevents the model from memorizing bad labels and makes it more robust to adversarial examples.
This is clever. Very clever.