## Support Vector Machines for Pattern ClassificationSupport vector machines (SVMs), were originally formulated for two-class classification problems, and have been accepted as a powerful tool for developing pattern classification and function approximations systems. This book provides a unique perspective of the state of the art in SVMs by taking the only approach that focuses on classification rather than covering the theoretical aspects. The book clarifies the characteristics of two-class SVMs through their extensive analysis, presents various useful architectures for multiclass classification and function approximation problems, and discusses kernel methods for improving generalization ability of conventional neural networks and fuzzy systems. Ample illustrations, examples and computer experiments are included to help readers understand the new ideas and their usefulness. This book supplies a comprehensive resource for the use of SVMs in pattern classification and will be invaluable reading for researchers, developers & students in academia and industry. |

### What people are saying - Write a review

We haven't found any reviews in the usual places.

### Contents

Introduction | 3 |

112 Decision Functions for Multiclass Problems | 5 |

12 Determination of Decision Functions | 10 |

13 Data Sets Used in the Book | 11 |

TwoClass Support Vector Machines | 15 |

22 L1 SoftMargin Support Vector Machines | 22 |

23 Mapping to a HighDimensional Space | 25 |

232 Kernels | 27 |

552 PrimalDual InteriorPoint Methods for Quadratic Programming | 171 |

553 Performance Evaluation | 173 |

56 Steepest Ascent Methods | 178 |

562 Sequential Minimal Optimization | 182 |

563 Training of L2 SoftMargin Support Vector Machines | 184 |

564 Performance Evaluation | 185 |

57 Training of Linear Programming Support Vector Machines | 186 |

572 Training by Decomposition | 188 |

233 Normalizing Kernels | 30 |

234 Properties of Mapping Functions Associated with Kernels | 31 |

235 Implicit Bias Terms | 33 |

24 L2 SoftMargin Support Vector Machines | 37 |

25 Advantages and Disadvantages | 39 |

252 Disadvantages | 40 |

261 Hessian Matrix | 41 |

262 Dependence of Solutions on C | 42 |

263 Equivalence of L1 and L2 Support Vector Machines | 47 |

264 Nonunique Solutions | 50 |

265 Reducing the Number of Support Vectors | 58 |

266 Degenerate Solutions | 61 |

267 Duplicate Copies of Data | 63 |

268 Imbalanced Data | 65 |

27 Class Boundaries for Different Kernels | 70 |

28 Developing Classifiers | 72 |

281 Model Selection | 73 |

283 Sophistication of Model Selection | 77 |

Multiclass Support Vector Machines | 83 |

31 OneagainstAll Support Vector Machines | 84 |

312 Fuzzy Support Vector Machines | 85 |

313 Equivalence of Fuzzy Support Vector Machines and Support Vector Machines with Continuous Decision Functions | 89 |

314 DecisionTreeBased Support Vector Machines | 91 |

32 Pairwise Support Vector Machines | 96 |

322 Fuzzy Support Vector Machines Architecture | 97 |

323 Performance Comparison of Fuzzy Support Vector Machines | 98 |

324 ClusterBased Support Vector Machines | 101 |

325 DecisionTreeBased Support Vector Machines | 102 |

326 Pairwise Classification with Correcting Classifiers | 112 |

33 ErrorCorrecting Output Codes | 113 |

331 Output Coding by ErrorCorrecting Codes | 114 |

333 Equivalence of ECOC with Membership Functions | 115 |

334 Performance Evaluation | 116 |

34 AllatOnce Support Vector Machines | 118 |

342 Sophisticated Architecture | 120 |

35 Comparisons of Architectures | 122 |

352 Pairwise Support Vector Machines | 123 |

354 AllatOnce Support Vector Machines | 124 |

356 Training Time Comparison | 127 |

Variants of Support Vector Machines | 129 |

412 OneagainstAll Least Squares Support Vector Machines | 132 |

413 Pairwise Least Squares Support Vector Machines | 133 |

414 AllatOnce Least Squares Support Vector Machines | 134 |

415 Performance Comparison | 136 |

42 Linear Programming Support Vector Machines | 140 |

422 Performance Evaluation | 143 |

43 Incremental Training | 146 |

44 Robust Support Vector Machines | 149 |

451 OneDimensional Bayesian Decision Functions | 150 |

452 Parallel Displacement of a Hyperplane | 151 |

453 Normal Test | 152 |

46 Committee Machines | 153 |

48 Visualization | 154 |

Training Methods | 155 |

511 Approximation of Boundary Data | 156 |

512 Performance Evaluation | 158 |

52 Decomposition Techniques | 159 |

53 KKT Conditions Revisited | 162 |

54 Overview of Training Methods | 165 |

55 PrimalDual InteriorPoint Methods | 167 |

Feature Selection and Extraction | 189 |

62 Feature Selection Using Support Vector Machines | 190 |

622 Support Vector Machine Based Feature Selection | 193 |

623 Feature Selection by Cross Validation | 194 |

63 Feature Extraction | 195 |

Clustering | 201 |

72 Extension to Clustering | 207 |

KernelBased Methods | 209 |

812 Performance Evaluation | 212 |

82 Kernel Principal Component Analysis | 215 |

83 Kernel Mahalanobis Distance | 218 |

832 KPCABased Mahalanobis Distance | 221 |

MaximumMargin Multilayer Neural Networks | 223 |

92 ThreeLayer Neural Networks | 224 |

93 CARVE Algorithm | 227 |

941 Rotation of Hyperplanes | 229 |

942 Training Algorithm | 231 |

95 Determination of OutputLayer Hyperplanes | 232 |

96 Determination of Parameter Values | 233 |

98 Summary | 234 |

MaximumMargin Fuzzy Classifiers | 237 |

101 Kernel Fuzzy Classifiers with Ellipsoidal Regions | 238 |

1012 Extension to a Feature Space | 239 |

1013 Transductive Training Concept | 240 |

1014 Maximizing Margins Concept | 244 |

1015 Performance Evaluation | 247 |

1016 Summary | 252 |

102 Fuzzy Classifiers with Polyhedral Regions | 253 |

1022 Performance Evaluation | 261 |

Function Approximation | 265 |

112 L1 SoftMargin Support Vector Regressors | 269 |

113 L2 SoftMargin Support Vector Regressors | 271 |

114 Training Speedup | 273 |

115 Steepest Ascent Methods | 274 |

1151 Subproblem Optimization | 275 |

1152 Convergence Check | 277 |

116 Candidate Set Selection | 278 |

1163 Selection of Violating Variables | 280 |

1171 Linear Programming Support Vector Regressors | 281 |

1173 Least Squares Support Vector Regressors | 283 |

118 Performance Evaluation | 285 |

1182 Effect of Working Set Size on Speedup | 286 |

1184 Comparison of Exact and Inexact KKT Conditions | 288 |

1185 Comparison with Other Training Methods | 290 |

1186 Performance Comparison with Other Approximation Methods | 291 |

1187 Robustness for Outliers | 294 |

1188 Summary | 295 |

Conventional Classifiers | 297 |

A2 Nearest Neighbor Classifiers | 298 |

Matrices | 301 |

B2 Least Squares Methods and Singular Value Decomposition | 303 |

B3 Covariance Matrices | 305 |

Quadratic Programming | 309 |

C2 Properties of Solutions | 310 |

Positive Semidefinite Kernels and Reproducing Kernel Hilbert Space | 313 |

D2 Reproducing Kernel Hilbert Space | 317 |

319 | |

339 | |

### Other editions - View all

### Common terms and phrases

ability algorithm approximation associated assume average becomes belonging to class blood cell bounded calculate called classified cluster compared consider constraints conventional convex data sets datum DDAG decision functions define delete determine discuss dual problem elements equality equivalent error estimation Evaluation example feature space Figure formulation given hyperplane included increased input input space input variables introduce KKT conditions L2 support vector Mahalanobis distance mapping margin matrix Maximize maximum membership functions method minimized Namely neural network number of support objective function obtain operators optimal orthogonal outliers output pairs pairwise parameter performance polynomial kernels positive semidefinite problem programming proposed RBF kernels recognition rates reduced regions remaining respectively satisfied selection separable shows similar solution solve support vector machines support vector regressor Table test data Theorem thyroid training data unclassifiable regions variables

### Popular passages

Page 320 - Explicit representation of knowledge acquired from plant historical data using neural network,

Page 321 - PS Bradley and OL Mangasarian. Feature selection via concave minimization and support vector machines. In J.

Page 325 - In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN.

Page 336 - Support vector machines for multiclass pattern recognition," in Proceedings of the Seventh European Symposium On Artificial Neural Networks, April 1999. 10. K. Crammer, and Y. Singer, "On the learnability and design of output codes for multi-class problems," Computational Learning Theory, 47(2-3) pp.

Page 321 - M. Song. Dimensionality Reduction via Sparse Support Vector Machines. Journal of Machine Learning Research, 3: 1229-1243, March 2003.

Page 335 - Input Layer Optimization of Neural Networks by Sensitivity Analysis and Its Application to Recognition of Numerals," Electrical Engineering in Japan, Vol.

Page 319 - S. Abe. Dynamic cluster generation for a fuzzy classifier with ellipsoidal regions. IEEE Transactions on Systems, Man, and Cybernetics — Part B, 28(6):869-76, 1998.

Page 320 - In Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN '99), volume 1, pages 85-90, Edinburgh, UK, 1999.

Page 320 - Evaluating the Generalization Ability of Support Vector Machines through the Bootstrap Neural Processing Letters, Vol.