Support Vector Machines for Pattern ClassificationSpringer Science & Business Media, 2005 M07 29 - 343 pages Support vector machines (SVMs), were originally formulated for two-class classification problems, and have been accepted as a powerful tool for developing pattern classification and function approximations systems. This book provides a unique perspective of the state of the art in SVMs by taking the only approach that focuses on classification rather than covering the theoretical aspects. The book clarifies the characteristics of two-class SVMs through their extensive analysis, presents various useful architectures for multiclass classification and function approximation problems, and discusses kernel methods for improving generalization ability of conventional neural networks and fuzzy systems. Ample illustrations, examples and computer experiments are included to help readers understand the new ideas and their usefulness. This book supplies a comprehensive resource for the use of SVMs in pattern classification and will be invaluable reading for researchers, developers & students in academia and industry. |
From inside the book
Results 1-5 of 80
Page v
... method . And it degrades greatly when the number of training data is small and there is no class overlap . On the other hand , in training support vector machines the decision boundaries are determined directly from the training data so ...
... method . And it degrades greatly when the number of training data is small and there is no class overlap . On the other hand , in training support vector machines the decision boundaries are determined directly from the training data so ...
Page vii
... methods for multiclass prob- lems , we show performance evaluation of these methods for the benchmark data sets . Since support vector machines were proposed , many variants of support vector machines have been developed . In Chapter 4 ...
... methods for multiclass prob- lems , we show performance evaluation of these methods for the benchmark data sets . Since support vector machines were proposed , many variants of support vector machines have been developed . In Chapter 4 ...
Page xi
... Methods . 167 5.5.1 Primal - Dual Interior - Point Methods for Linear Programming . 167 5.5.2 Primal - Dual Interior - Point Methods for Quadratic Programming 5.5.3 Performance Evaluation . . 171 173 5.6 Steepest Ascent Methods . 178 ...
... Methods . 167 5.5.1 Primal - Dual Interior - Point Methods for Linear Programming . 167 5.5.2 Primal - Dual Interior - Point Methods for Quadratic Programming 5.5.3 Performance Evaluation . . 171 173 5.6 Steepest Ascent Methods . 178 ...
Page xiii
... Methods . 274 11.5.1 Subproblem Optimization 11.5.2 Convergence Check 11.6 Candidate Set Selection ... 11.6.1 ... Methods ..... 11.8.6 Performance Comparison with Other Approximation 290 A Methods .... 11.8.7 Robustness for Outliers 11.8 ...
... Methods . 274 11.5.1 Subproblem Optimization 11.5.2 Convergence Check 11.6 Candidate Set Selection ... 11.6.1 ... Methods ..... 11.8.6 Performance Comparison with Other Approximation 290 A Methods .... 11.8.7 Robustness for Outliers 11.8 ...
Page 59
Sorry, this page's content is restricted.
Sorry, this page's content is restricted.
Contents
Introduction | 3 |
112 Decision Functions for Multiclass Problems | 5 |
12 Determination of Decision Functions | 10 |
13 Data Sets Used in the Book | 11 |
TwoClass Support Vector Machines | 15 |
22 L1 SoftMargin Support Vector Machines | 22 |
23 Mapping to a HighDimensional Space | 25 |
232 Kernels | 27 |
552 PrimalDual InteriorPoint Methods for Quadratic Programming | 171 |
553 Performance Evaluation | 173 |
56 Steepest Ascent Methods | 178 |
562 Sequential Minimal Optimization | 182 |
563 Training of L2 SoftMargin Support Vector Machines | 184 |
564 Performance Evaluation | 185 |
57 Training of Linear Programming Support Vector Machines | 186 |
572 Training by Decomposition | 188 |
233 Normalizing Kernels | 30 |
234 Properties of Mapping Functions Associated with Kernels | 31 |
235 Implicit Bias Terms | 33 |
24 L2 SoftMargin Support Vector Machines | 37 |
25 Advantages and Disadvantages | 39 |
252 Disadvantages | 40 |
261 Hessian Matrix | 41 |
262 Dependence of Solutions on C | 42 |
263 Equivalence of L1 and L2 Support Vector Machines | 47 |
264 Nonunique Solutions | 50 |
265 Reducing the Number of Support Vectors | 58 |
266 Degenerate Solutions | 61 |
267 Duplicate Copies of Data | 63 |
268 Imbalanced Data | 65 |
27 Class Boundaries for Different Kernels | 70 |
28 Developing Classifiers | 72 |
281 Model Selection | 73 |
283 Sophistication of Model Selection | 77 |
Multiclass Support Vector Machines | 83 |
31 OneagainstAll Support Vector Machines | 84 |
312 Fuzzy Support Vector Machines | 85 |
313 Equivalence of Fuzzy Support Vector Machines and Support Vector Machines with Continuous Decision Functions | 89 |
314 DecisionTreeBased Support Vector Machines | 91 |
32 Pairwise Support Vector Machines | 96 |
322 Fuzzy Support Vector Machines Architecture | 97 |
323 Performance Comparison of Fuzzy Support Vector Machines | 98 |
324 ClusterBased Support Vector Machines | 101 |
325 DecisionTreeBased Support Vector Machines | 102 |
326 Pairwise Classification with Correcting Classifiers | 112 |
33 ErrorCorrecting Output Codes | 113 |
331 Output Coding by ErrorCorrecting Codes | 114 |
333 Equivalence of ECOC with Membership Functions | 115 |
334 Performance Evaluation | 116 |
34 AllatOnce Support Vector Machines | 118 |
342 Sophisticated Architecture | 120 |
35 Comparisons of Architectures | 122 |
352 Pairwise Support Vector Machines | 123 |
354 AllatOnce Support Vector Machines | 124 |
356 Training Time Comparison | 127 |
Variants of Support Vector Machines | 129 |
412 OneagainstAll Least Squares Support Vector Machines | 132 |
413 Pairwise Least Squares Support Vector Machines | 133 |
414 AllatOnce Least Squares Support Vector Machines | 134 |
415 Performance Comparison | 136 |
42 Linear Programming Support Vector Machines | 140 |
422 Performance Evaluation | 143 |
43 Incremental Training | 146 |
44 Robust Support Vector Machines | 149 |
451 OneDimensional Bayesian Decision Functions | 150 |
452 Parallel Displacement of a Hyperplane | 151 |
453 Normal Test | 152 |
46 Committee Machines | 153 |
48 Visualization | 154 |
Training Methods | 155 |
511 Approximation of Boundary Data | 156 |
512 Performance Evaluation | 158 |
52 Decomposition Techniques | 159 |
53 KKT Conditions Revisited | 162 |
54 Overview of Training Methods | 165 |
55 PrimalDual InteriorPoint Methods | 167 |
Feature Selection and Extraction | 189 |
62 Feature Selection Using Support Vector Machines | 190 |
622 Support Vector Machine Based Feature Selection | 193 |
623 Feature Selection by Cross Validation | 194 |
63 Feature Extraction | 195 |
Clustering | 201 |
72 Extension to Clustering | 207 |
KernelBased Methods | 209 |
812 Performance Evaluation | 212 |
82 Kernel Principal Component Analysis | 215 |
83 Kernel Mahalanobis Distance | 218 |
832 KPCABased Mahalanobis Distance | 221 |
MaximumMargin Multilayer Neural Networks | 223 |
92 ThreeLayer Neural Networks | 224 |
93 CARVE Algorithm | 227 |
941 Rotation of Hyperplanes | 229 |
942 Training Algorithm | 231 |
95 Determination of OutputLayer Hyperplanes | 232 |
96 Determination of Parameter Values | 233 |
98 Summary | 234 |
MaximumMargin Fuzzy Classifiers | 237 |
101 Kernel Fuzzy Classifiers with Ellipsoidal Regions | 238 |
1012 Extension to a Feature Space | 239 |
1013 Transductive Training Concept | 240 |
1014 Maximizing Margins Concept | 244 |
1015 Performance Evaluation | 247 |
1016 Summary | 252 |
102 Fuzzy Classifiers with Polyhedral Regions | 253 |
1022 Performance Evaluation | 261 |
Function Approximation | 265 |
112 L1 SoftMargin Support Vector Regressors | 269 |
113 L2 SoftMargin Support Vector Regressors | 271 |
114 Training Speedup | 273 |
115 Steepest Ascent Methods | 274 |
1151 Subproblem Optimization | 275 |
1152 Convergence Check | 277 |
116 Candidate Set Selection | 278 |
1163 Selection of Violating Variables | 280 |
1171 Linear Programming Support Vector Regressors | 281 |
1173 Least Squares Support Vector Regressors | 283 |
118 Performance Evaluation | 285 |
1182 Effect of Working Set Size on Speedup | 286 |
1184 Comparison of Exact and Inexact KKT Conditions | 288 |
1185 Comparison with Other Training Methods | 290 |
1186 Performance Comparison with Other Approximation Methods | 291 |
1187 Robustness for Outliers | 294 |
1188 Summary | 295 |
Conventional Classifiers | 297 |
A2 Nearest Neighbor Classifiers | 298 |
Matrices | 301 |
B2 Least Squares Methods and Singular Value Decomposition | 303 |
B3 Covariance Matrices | 305 |
Quadratic Programming | 309 |
C2 Properties of Solutions | 310 |
Positive Semidefinite Kernels and Reproducing Kernel Hilbert Space | 313 |
D2 Reproducing Kernel Hilbert Space | 317 |
319 | |
339 | |
Other editions - View all
Common terms and phrases
a₁ algorithm all-at-once arg max belonging to class bias term blood cell data calculate class boundary cluster convex hull cross-validation datum DDAG decision functions delete Di(x discuss dual problem ECOC exact KKT conditions feature selection feature space fuzzy classifier fuzzy support vector g(xi given H(xi Hessian matrix Hiragana-50 hyperplane hypersphere inexact KKT input space input variables KPCA L1 and L2 L1 soft-margin L2 support vector L2 SVM linear kernels m-dimensional Mahalanobis distance Maximize membership functions neural network number of support number of training objective function obtain one-against-all support vector optimal hyperplane optimal solution orthogonal outliers output pairwise support vector performance polynomial kernels positive definite quadratic programming RBF kernels recognition rates satisfied set of support shown in Fig singular value decomposition slack variables soft-margin support vector support vector machines support vector regressor Table test data Theorem thyroid data training data unbounded support vectors unclassifiable regions
Popular passages
Page 320 - Explicit representation of knowledge acquired from plant historical data using neural network,
Page 321 - PS Bradley and OL Mangasarian. Feature selection via concave minimization and support vector machines. In J.
Page 325 - In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN.
Page 336 - Support vector machines for multiclass pattern recognition," in Proceedings of the Seventh European Symposium On Artificial Neural Networks, April 1999. 10. K. Crammer, and Y. Singer, "On the learnability and design of output codes for multi-class problems," Computational Learning Theory, 47(2-3) pp.
Page 321 - M. Song. Dimensionality Reduction via Sparse Support Vector Machines. Journal of Machine Learning Research, 3: 1229-1243, March 2003.
Page 335 - Input Layer Optimization of Neural Networks by Sensitivity Analysis and Its Application to Recognition of Numerals," Electrical Engineering in Japan, Vol.
Page 319 - S. Abe. Dynamic cluster generation for a fuzzy classifier with ellipsoidal regions. IEEE Transactions on Systems, Man, and Cybernetics — Part B, 28(6):869-76, 1998.
Page 320 - In Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN '99), volume 1, pages 85-90, Edinburgh, UK, 1999.
Page 320 - Evaluating the Generalization Ability of Support Vector Machines through the Bootstrap Neural Processing Letters, Vol.