Tag: Mutual information

What the study found Hidden representations in neural networks change with label noise in ways that depend on how many parameters the network has. The study found a double descent pattern in the information content of these representations, and it found that overparameterized networks are robust to label noise. Why the authors say this matters…