Generalization Bounds for Vicinal Risk Minimization Principle
- Publication Type:
- Working Paper
- Citation:
- 2018
- Issue Date:
- 2018-11-11
Open Access
Copyright Clearance Process
- Recently Added
- In Progress
- Open Access
This item is open access.
The vicinal risk minimization (VRM) principle, first proposed by
\citet{vapnik1999nature}, is an empirical risk minimization (ERM) variant that
replaces Dirac masses with vicinal functions. Although there is strong
numerical evidence showing that VRM outperforms ERM if appropriate vicinal
functions are chosen, a comprehensive theoretical understanding of VRM is still
lacking. In this paper, we study the generalization bounds for VRM. Our results
support Vapnik's original arguments and additionally provide deeper insights
into VRM. First, we prove that the complexity of function classes convolving
with vicinal functions can be controlled by that of the original function
classes under the assumption that the function class is composed of
Lipschitz-continuous functions. Then, the resulting generalization bounds for
VRM suggest that the generalization performance of VRM is also effected by the
choice of vicinity function and the quality of function classes. These findings
can be used to examine whether the choice of vicinal function is appropriate
for the VRM-based learning setting. Finally, we provide a theoretical
explanation for existing VRM models, e.g., uniform distribution-based models,
Gaussian distribution-based models, and mixup models.
Please use this identifier to cite or link to this item: