Support Vector Machines for Pattern Classification

Front Cover
Springer Science & Business Media, 2005 M07 29 - 343 pages
Support vector machines (SVMs), were originally formulated for two-class classification problems, and have been accepted as a powerful tool for developing pattern classification and function approximations systems. This book provides a unique perspective of the state of the art in SVMs by taking the only approach that focuses on classification rather than covering the theoretical aspects. The book clarifies the characteristics of two-class SVMs through their extensive analysis, presents various useful architectures for multiclass classification and function approximation problems, and discusses kernel methods for improving generalization ability of conventional neural networks and fuzzy systems. Ample illustrations, examples and computer experiments are included to help readers understand the new ideas and their usefulness. This book supplies a comprehensive resource for the use of SVMs in pattern classification and will be invaluable reading for researchers, developers & students in academia and industry.

From inside the book

Contents

Introduction
3
112 Decision Functions for Multiclass Problems
5
12 Determination of Decision Functions
10
13 Data Sets Used in the Book
11
TwoClass Support Vector Machines
15
22 L1 SoftMargin Support Vector Machines
22
23 Mapping to a HighDimensional Space
25
232 Kernels
27
552 PrimalDual InteriorPoint Methods for Quadratic Programming
171
553 Performance Evaluation
173
56 Steepest Ascent Methods
178
562 Sequential Minimal Optimization
182
563 Training of L2 SoftMargin Support Vector Machines
184
564 Performance Evaluation
185
57 Training of Linear Programming Support Vector Machines
186
572 Training by Decomposition
188

233 Normalizing Kernels
30
234 Properties of Mapping Functions Associated with Kernels
31
235 Implicit Bias Terms
33
24 L2 SoftMargin Support Vector Machines
37
25 Advantages and Disadvantages
39
252 Disadvantages
40
261 Hessian Matrix
41
262 Dependence of Solutions on C
42
263 Equivalence of L1 and L2 Support Vector Machines
47
264 Nonunique Solutions
50
265 Reducing the Number of Support Vectors
58
266 Degenerate Solutions
61
267 Duplicate Copies of Data
63
268 Imbalanced Data
65
27 Class Boundaries for Different Kernels
70
28 Developing Classifiers
72
281 Model Selection
73
283 Sophistication of Model Selection
77
Multiclass Support Vector Machines
83
31 OneagainstAll Support Vector Machines
84
312 Fuzzy Support Vector Machines
85
313 Equivalence of Fuzzy Support Vector Machines and Support Vector Machines with Continuous Decision Functions
89
314 DecisionTreeBased Support Vector Machines
91
32 Pairwise Support Vector Machines
96
322 Fuzzy Support Vector Machines Architecture
97
323 Performance Comparison of Fuzzy Support Vector Machines
98
324 ClusterBased Support Vector Machines
101
325 DecisionTreeBased Support Vector Machines
102
326 Pairwise Classification with Correcting Classifiers
112
33 ErrorCorrecting Output Codes
113
331 Output Coding by ErrorCorrecting Codes
114
333 Equivalence of ECOC with Membership Functions
115
334 Performance Evaluation
116
34 AllatOnce Support Vector Machines
118
342 Sophisticated Architecture
120
35 Comparisons of Architectures
122
352 Pairwise Support Vector Machines
123
354 AllatOnce Support Vector Machines
124
356 Training Time Comparison
127
Variants of Support Vector Machines
129
412 OneagainstAll Least Squares Support Vector Machines
132
413 Pairwise Least Squares Support Vector Machines
133
414 AllatOnce Least Squares Support Vector Machines
134
415 Performance Comparison
136
42 Linear Programming Support Vector Machines
140
422 Performance Evaluation
143
43 Incremental Training
146
44 Robust Support Vector Machines
149
451 OneDimensional Bayesian Decision Functions
150
452 Parallel Displacement of a Hyperplane
151
453 Normal Test
152
46 Committee Machines
153
48 Visualization
154
Training Methods
155
511 Approximation of Boundary Data
156
512 Performance Evaluation
158
52 Decomposition Techniques
159
53 KKT Conditions Revisited
162
54 Overview of Training Methods
165
55 PrimalDual InteriorPoint Methods
167
Feature Selection and Extraction
189
62 Feature Selection Using Support Vector Machines
190
622 Support Vector Machine Based Feature Selection
193
623 Feature Selection by Cross Validation
194
63 Feature Extraction
195
Clustering
201
72 Extension to Clustering
207
KernelBased Methods
209
812 Performance Evaluation
212
82 Kernel Principal Component Analysis
215
83 Kernel Mahalanobis Distance
218
832 KPCABased Mahalanobis Distance
221
MaximumMargin Multilayer Neural Networks
223
92 ThreeLayer Neural Networks
224
93 CARVE Algorithm
227
941 Rotation of Hyperplanes
229
942 Training Algorithm
231
95 Determination of OutputLayer Hyperplanes
232
96 Determination of Parameter Values
233
98 Summary
234
MaximumMargin Fuzzy Classifiers
237
101 Kernel Fuzzy Classifiers with Ellipsoidal Regions
238
1012 Extension to a Feature Space
239
1013 Transductive Training Concept
240
1014 Maximizing Margins Concept
244
1015 Performance Evaluation
247
1016 Summary
252
102 Fuzzy Classifiers with Polyhedral Regions
253
1022 Performance Evaluation
261
Function Approximation
265
112 L1 SoftMargin Support Vector Regressors
269
113 L2 SoftMargin Support Vector Regressors
271
114 Training Speedup
273
115 Steepest Ascent Methods
274
1151 Subproblem Optimization
275
1152 Convergence Check
277
116 Candidate Set Selection
278
1163 Selection of Violating Variables
280
1171 Linear Programming Support Vector Regressors
281
1173 Least Squares Support Vector Regressors
283
118 Performance Evaluation
285
1182 Effect of Working Set Size on Speedup
286
1184 Comparison of Exact and Inexact KKT Conditions
288
1185 Comparison with Other Training Methods
290
1186 Performance Comparison with Other Approximation Methods
291
1187 Robustness for Outliers
294
1188 Summary
295
Conventional Classifiers
297
A2 Nearest Neighbor Classifiers
298
Matrices
301
B2 Least Squares Methods and Singular Value Decomposition
303
B3 Covariance Matrices
305
Quadratic Programming
309
C2 Properties of Solutions
310
Positive Semidefinite Kernels and Reproducing Kernel Hilbert Space
313
D2 Reproducing Kernel Hilbert Space
317
References
319
Index
339
Copyright

Other editions - View all

Common terms and phrases

Popular passages

Page 320 - Explicit representation of knowledge acquired from plant historical data using neural network,
Page 321 - PS Bradley and OL Mangasarian. Feature selection via concave minimization and support vector machines. In J.
Page 325 - In Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN.
Page 336 - Support vector machines for multiclass pattern recognition," in Proceedings of the Seventh European Symposium On Artificial Neural Networks, April 1999. 10. K. Crammer, and Y. Singer, "On the learnability and design of output codes for multi-class problems," Computational Learning Theory, 47(2-3) pp.
Page 321 - M. Song. Dimensionality Reduction via Sparse Support Vector Machines. Journal of Machine Learning Research, 3: 1229-1243, March 2003.
Page 335 - Input Layer Optimization of Neural Networks by Sensitivity Analysis and Its Application to Recognition of Numerals," Electrical Engineering in Japan, Vol.
Page 319 - S. Abe. Dynamic cluster generation for a fuzzy classifier with ellipsoidal regions. IEEE Transactions on Systems, Man, and Cybernetics — Part B, 28(6):869-76, 1998.
Page 320 - In Proceedings of the Ninth International Conference on Artificial Neural Networks (ICANN '99), volume 1, pages 85-90, Edinburgh, UK, 1999.
Page 320 - Evaluating the Generalization Ability of Support Vector Machines through the Bootstrap Neural Processing Letters, Vol.

Bibliographic information