How does disagreement help generalization against label corruption?
- Publication Type:
- Conference Proceeding
- Citation:
- 36th International Conference on Machine Learning, ICML 2019, 2019, 2019-June pp. 12407 - 12417
- Issue Date:
- 2019-01-01
Open Access
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is open access.
Copyright © 2019 ASME Learning with noisy labels is one of the hottest problems in weakly-supervised learning. Based on memorization effects of deep neural networks, training on small-loss instances becomes very promising for handling noisy labels. This fosters the state-of-the-art approach "Co-teaching" that cross-trains two deep neural networks using the small-loss trick. However, with the increase of epochs, two networks converge to a consensus and Co-teaching reduces to the self-training MentorNet. To tackle this issue, we propose a robust learning paradigm called Co-teaching+, which bridges the "Update by Disagreement" strategy with the original Co-teaching. First, two networks feed forward and predict all data, but keep prediction disagreement data only. Then, among such disagreement data, each network selects its small-loss data, but back propagates the small-loss data from its peer network and updates its own parameters. Empirical results on benchmark datasets demonstrate that Cotcaching+ is much superior to many statc-of-thcart methods in the robustness of trained models.
Please use this identifier to cite or link to this item: