Radius Margin Bounds for Support Vector Machines with the RBF Kernel
- 1 November 2003
- journal article
- Published by MIT Press in Neural Computation
- Vol. 15 (11), 2643-2681
- https://doi.org/10.1162/089976603322385108
Abstract
An important approach for efficient support vector machine (SVM) model selection is to use differentiable bounds of the leave-one-out (loo) error. Past efforts focused on finding tight bounds of loo (e.g., radius margin bounds, span bounds). However, their practical viability is still not very satisfactory. Duan, Keerthi, and Poo (2003) showed that radius margin bound gives good prediction for L2-SVM, one of the cases we look at. In this letter, through analyses about why this bound performs well for L2-SVM, we show that finding a bound whose minima are in a region with small loo values may be more important than its tightness. Based on this principle, we propose modified radius margin bounds for L1-SVM (the other case) where the original bound is applicable only to the hard-margin case. Our modification for L1-SVM achieves comparable performance to L2-SVM. To study whether L1- or L2-SVM should be used, we analyze other properties, such as their differentiability, number of support vectors, and number of free support vectors. In this aspect, L1-SVM possesses the advantage of having fewer support vectors. Their implementations are also different, so we discuss related issues in detail.Keywords
This publication has 8 references indexed in Scilit:
- Evaluation of simple performance measures for tuning SVM hyperparametersNeurocomputing, 2003
- Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithmsIEEE Transactions on Neural Networks, 2002
- Choosing Multiple Parameters for Support Vector MachinesMachine Learning, 2002
- On the Global Convergence of the BFGS Method for Nonconvex Unconstrained Optimization ProblemsSIAM Journal on Optimization, 2001
- Bounds on Error Expectation for Support Vector MachinesNeural Computation, 2000
- Newton's Method for Large Bound-Constrained Optimization ProblemsSIAM Journal on Optimization, 1999
- Optimization Problems with Perturbations: A Guided TourSIAM Review, 1998
- Interpolation of scattered data: Distance matrices and conditionally positive definite functionsConstructive Approximation, 1986