Advantages of Unbiased Support Vector Classifiers for Data Mining Applications View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2004-06

AUTHORS

A. Navia-Vázquez, F. Pérez-Cruz, A. Artés-Rodríguez, A.R. Figueiras-Vidal

ABSTRACT

Many learning algorithms have been used for data mining applications, including Support Vector Classifiers (SVC), which have shown improved capabilities with respect to other approaches, since they provide a natural mechanism for implementing Structural Risk Minimization (SRM), obtaining machines with good generalization properties. SVC leads to the optimal hyperplane (maximal margin) criterion for separable datasets but, in the nonseparable case, the SVC minimizes the L1 norm of the training errors plus a regularizing term, to control the machine complexity. The L1 norm is chosen because it allows to solve the minimization with a Quadratic Programming (QP) scheme, as in the separable case. But the L1 norm is not truly an “error counting” term as the Empirical Risk Minimization (ERM) inductive principle indicates, leading therefore to a biased solution. This effect is specially severe in low complexity machines, such as linear classifiers or machines with few nodes (neurons, kernels, basis functions). Since one of the main goals in data mining is that of explanation, these reduced architectures are of great interest because they represent the origins of other techniques such as input selection or rule extraction. Training SVMs as accurately as possible in these situations (i.e., without this bias) is, therefore, an interesting goal. We propose here an unbiased implementation of SVC by introducing a more appropriate “error counting” term. This way, the number of classification errors is truly minimized, while the maximal margin solution is obtained in the separable case. QP can no longer be used for solving the new minimization problem, and we apply instead an iterated Weighted Least Squares (WLS) procedure. This modification in the cost function of the Support Vector Machine to solve ERM was not possible up to date given the Quadratic or Linear Programming techniques commonly used, but it is now possible using the iterated WLS formulation. Computer experiments show that the proposed method is superior to the classical approach in the sense that it truly solves the ERM problem. More... »

PAGES

223-235

References to SciGraph publications

Identifiers

URI

http://scigraph.springernature.com/pub.10.1023/b:vlsi.0000027487.93757.91

DOI

http://dx.doi.org/10.1023/b:vlsi.0000027487.93757.91

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1025095761


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Carlos III University of Madrid", 
          "id": "https://www.grid.ac/institutes/grid.7840.b", 
          "name": [
            "DTSC, Univ. Carlos III de Madrid, Avda Universidad 30, 28911, Legan\u00e9s, Madrid, Spain"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Navia-V\u00e1zquez", 
        "givenName": "A.", 
        "id": "sg:person.016004023055.20", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016004023055.20"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Carlos III University of Madrid", 
          "id": "https://www.grid.ac/institutes/grid.7840.b", 
          "name": [
            "DTSC, Univ. Carlos III de Madrid, Avda Universidad 30, 28911, Legan\u00e9s, Madrid, Spain"
          ], 
          "type": "Organization"
        }, 
        "familyName": "P\u00e9rez-Cruz", 
        "givenName": "F.", 
        "id": "sg:person.016363577525.33", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016363577525.33"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Carlos III University of Madrid", 
          "id": "https://www.grid.ac/institutes/grid.7840.b", 
          "name": [
            "DTSC, Univ. Carlos III de Madrid, Avda Universidad 30, 28911, Legan\u00e9s, Madrid, Spain"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Art\u00e9s-Rodr\u00edguez", 
        "givenName": "A.", 
        "id": "sg:person.016405316123.77", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016405316123.77"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Carlos III University of Madrid", 
          "id": "https://www.grid.ac/institutes/grid.7840.b", 
          "name": [
            "DTSC, Univ. Carlos III de Madrid, Avda Universidad 30, 28911, Legan\u00e9s, Madrid, Spain"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Figueiras-Vidal", 
        "givenName": "A.R.", 
        "id": "sg:person.01000104276.70", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01000104276.70"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.1006/jcss.1997.1504", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1004338842"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/s0893-6080(97)00136-6", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1007296336"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0893-6080(90)90071-r", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1033966594"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0893-6080(90)90071-r", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1033966594"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/s0893-6080(97)00135-4", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1038102490"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1023/a:1009715923555", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1042048349", 
          "https://doi.org/10.1023/a:1009715923555"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1023/a:1012755431700", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1044446130", 
          "https://doi.org/10.1023/a:1012755431700"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0893-6080(94)90102-3", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1047547579"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0893-6080(94)90102-3", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1047547579"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/21.362969", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061122054"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/72.950134", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061219614"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/tc.1978.1675165", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061532124"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/tnn.2003.809399", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061716536"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/icassp.2000.860145", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1093400549"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2004-06", 
    "datePublishedReg": "2004-06-01", 
    "description": "Many learning algorithms have been used for data mining applications, including Support Vector Classifiers (SVC), which have shown improved capabilities with respect to other approaches, since they provide a natural mechanism for implementing Structural Risk Minimization (SRM), obtaining machines with good generalization properties. SVC leads to the optimal hyperplane (maximal margin) criterion for separable datasets but, in the nonseparable case, the SVC minimizes the L1 norm of the training errors plus a regularizing term, to control the machine complexity. The L1 norm is chosen because it allows to solve the minimization with a Quadratic Programming (QP) scheme, as in the separable case. But the L1 norm is not truly an \u201cerror counting\u201d term as the Empirical Risk Minimization (ERM) inductive principle indicates, leading therefore to a biased solution. This effect is specially severe in low complexity machines, such as linear classifiers or machines with few nodes (neurons, kernels, basis functions). Since one of the main goals in data mining is that of explanation, these reduced architectures are of great interest because they represent the origins of other techniques such as input selection or rule extraction. Training SVMs as accurately as possible in these situations (i.e., without this bias) is, therefore, an interesting goal. We propose here an unbiased implementation of SVC by introducing a more appropriate \u201cerror counting\u201d term. This way, the number of classification errors is truly minimized, while the maximal margin solution is obtained in the separable case. QP can no longer be used for solving the new minimization problem, and we apply instead an iterated Weighted Least Squares (WLS) procedure. This modification in the cost function of the Support Vector Machine to solve ERM was not possible up to date given the Quadratic or Linear Programming techniques commonly used, but it is now possible using the iterated WLS formulation. Computer experiments show that the proposed method is superior to the classical approach in the sense that it truly solves the ERM problem.", 
    "genre": "research_article", 
    "id": "sg:pub.10.1023/b:vlsi.0000027487.93757.91", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": false, 
    "isPartOf": [
      {
        "id": "sg:journal.1297359", 
        "issn": [
          "0922-5773", 
          "1939-8115"
        ], 
        "name": "Journal of Signal Processing Systems", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "2-3", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "37"
      }
    ], 
    "name": "Advantages of Unbiased Support Vector Classifiers for Data Mining Applications", 
    "pagination": "223-235", 
    "productId": [
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "778e8605092e1ca5fe68c175951501d573cc3c748157b7dae7a1fa1558a70fc2"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1023/b:vlsi.0000027487.93757.91"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1025095761"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1023/b:vlsi.0000027487.93757.91", 
      "https://app.dimensions.ai/details/publication/pub.1025095761"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2019-04-11T00:14", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8695_00000505.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "http://link.springer.com/10.1023%2FB%3AVLSI.0000027487.93757.91"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1023/b:vlsi.0000027487.93757.91'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1023/b:vlsi.0000027487.93757.91'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1023/b:vlsi.0000027487.93757.91'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1023/b:vlsi.0000027487.93757.91'


 

This table displays all metadata directly associated to this object as RDF triples.

120 TRIPLES      21 PREDICATES      39 URIs      19 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1023/b:vlsi.0000027487.93757.91 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Nce3f352af9e546aea3632d1f2762ffe7
4 schema:citation sg:pub.10.1023/a:1009715923555
5 sg:pub.10.1023/a:1012755431700
6 https://doi.org/10.1006/jcss.1997.1504
7 https://doi.org/10.1016/0893-6080(90)90071-r
8 https://doi.org/10.1016/0893-6080(94)90102-3
9 https://doi.org/10.1016/s0893-6080(97)00135-4
10 https://doi.org/10.1016/s0893-6080(97)00136-6
11 https://doi.org/10.1109/21.362969
12 https://doi.org/10.1109/72.950134
13 https://doi.org/10.1109/icassp.2000.860145
14 https://doi.org/10.1109/tc.1978.1675165
15 https://doi.org/10.1109/tnn.2003.809399
16 schema:datePublished 2004-06
17 schema:datePublishedReg 2004-06-01
18 schema:description Many learning algorithms have been used for data mining applications, including Support Vector Classifiers (SVC), which have shown improved capabilities with respect to other approaches, since they provide a natural mechanism for implementing Structural Risk Minimization (SRM), obtaining machines with good generalization properties. SVC leads to the optimal hyperplane (maximal margin) criterion for separable datasets but, in the nonseparable case, the SVC minimizes the L1 norm of the training errors plus a regularizing term, to control the machine complexity. The L1 norm is chosen because it allows to solve the minimization with a Quadratic Programming (QP) scheme, as in the separable case. But the L1 norm is not truly an “error counting” term as the Empirical Risk Minimization (ERM) inductive principle indicates, leading therefore to a biased solution. This effect is specially severe in low complexity machines, such as linear classifiers or machines with few nodes (neurons, kernels, basis functions). Since one of the main goals in data mining is that of explanation, these reduced architectures are of great interest because they represent the origins of other techniques such as input selection or rule extraction. Training SVMs as accurately as possible in these situations (i.e., without this bias) is, therefore, an interesting goal. We propose here an unbiased implementation of SVC by introducing a more appropriate “error counting” term. This way, the number of classification errors is truly minimized, while the maximal margin solution is obtained in the separable case. QP can no longer be used for solving the new minimization problem, and we apply instead an iterated Weighted Least Squares (WLS) procedure. This modification in the cost function of the Support Vector Machine to solve ERM was not possible up to date given the Quadratic or Linear Programming techniques commonly used, but it is now possible using the iterated WLS formulation. Computer experiments show that the proposed method is superior to the classical approach in the sense that it truly solves the ERM problem.
19 schema:genre research_article
20 schema:inLanguage en
21 schema:isAccessibleForFree false
22 schema:isPartOf N343f95bd7b0041b08344f1903ece7f82
23 Ne0720119fa4146b68915b0e417b0d418
24 sg:journal.1297359
25 schema:name Advantages of Unbiased Support Vector Classifiers for Data Mining Applications
26 schema:pagination 223-235
27 schema:productId N7ea59cf5efca4ff39c36c727c2962b91
28 Nc19540da780c400683f6d7764ae33488
29 Ncecde22dc8c64fabb5f205c999f6872f
30 schema:sameAs https://app.dimensions.ai/details/publication/pub.1025095761
31 https://doi.org/10.1023/b:vlsi.0000027487.93757.91
32 schema:sdDatePublished 2019-04-11T00:14
33 schema:sdLicense https://scigraph.springernature.com/explorer/license/
34 schema:sdPublisher Ne6bb6526542142a1b1ac3057aff4d88e
35 schema:url http://link.springer.com/10.1023%2FB%3AVLSI.0000027487.93757.91
36 sgo:license sg:explorer/license/
37 sgo:sdDataset articles
38 rdf:type schema:ScholarlyArticle
39 N2e8f9fa451b04c9ebe6650a044a0087e rdf:first sg:person.01000104276.70
40 rdf:rest rdf:nil
41 N343f95bd7b0041b08344f1903ece7f82 schema:issueNumber 2-3
42 rdf:type schema:PublicationIssue
43 N7ea59cf5efca4ff39c36c727c2962b91 schema:name readcube_id
44 schema:value 778e8605092e1ca5fe68c175951501d573cc3c748157b7dae7a1fa1558a70fc2
45 rdf:type schema:PropertyValue
46 Nb7214e3818d645678463b828b30e75be rdf:first sg:person.016405316123.77
47 rdf:rest N2e8f9fa451b04c9ebe6650a044a0087e
48 Nc19540da780c400683f6d7764ae33488 schema:name dimensions_id
49 schema:value pub.1025095761
50 rdf:type schema:PropertyValue
51 Nce3f352af9e546aea3632d1f2762ffe7 rdf:first sg:person.016004023055.20
52 rdf:rest Neb78a72aede64b1c8e27599b4608e8c4
53 Ncecde22dc8c64fabb5f205c999f6872f schema:name doi
54 schema:value 10.1023/b:vlsi.0000027487.93757.91
55 rdf:type schema:PropertyValue
56 Ne0720119fa4146b68915b0e417b0d418 schema:volumeNumber 37
57 rdf:type schema:PublicationVolume
58 Ne6bb6526542142a1b1ac3057aff4d88e schema:name Springer Nature - SN SciGraph project
59 rdf:type schema:Organization
60 Neb78a72aede64b1c8e27599b4608e8c4 rdf:first sg:person.016363577525.33
61 rdf:rest Nb7214e3818d645678463b828b30e75be
62 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
63 schema:name Information and Computing Sciences
64 rdf:type schema:DefinedTerm
65 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
66 schema:name Artificial Intelligence and Image Processing
67 rdf:type schema:DefinedTerm
68 sg:journal.1297359 schema:issn 0922-5773
69 1939-8115
70 schema:name Journal of Signal Processing Systems
71 rdf:type schema:Periodical
72 sg:person.01000104276.70 schema:affiliation https://www.grid.ac/institutes/grid.7840.b
73 schema:familyName Figueiras-Vidal
74 schema:givenName A.R.
75 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01000104276.70
76 rdf:type schema:Person
77 sg:person.016004023055.20 schema:affiliation https://www.grid.ac/institutes/grid.7840.b
78 schema:familyName Navia-Vázquez
79 schema:givenName A.
80 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016004023055.20
81 rdf:type schema:Person
82 sg:person.016363577525.33 schema:affiliation https://www.grid.ac/institutes/grid.7840.b
83 schema:familyName Pérez-Cruz
84 schema:givenName F.
85 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016363577525.33
86 rdf:type schema:Person
87 sg:person.016405316123.77 schema:affiliation https://www.grid.ac/institutes/grid.7840.b
88 schema:familyName Artés-Rodríguez
89 schema:givenName A.
90 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016405316123.77
91 rdf:type schema:Person
92 sg:pub.10.1023/a:1009715923555 schema:sameAs https://app.dimensions.ai/details/publication/pub.1042048349
93 https://doi.org/10.1023/a:1009715923555
94 rdf:type schema:CreativeWork
95 sg:pub.10.1023/a:1012755431700 schema:sameAs https://app.dimensions.ai/details/publication/pub.1044446130
96 https://doi.org/10.1023/a:1012755431700
97 rdf:type schema:CreativeWork
98 https://doi.org/10.1006/jcss.1997.1504 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004338842
99 rdf:type schema:CreativeWork
100 https://doi.org/10.1016/0893-6080(90)90071-r schema:sameAs https://app.dimensions.ai/details/publication/pub.1033966594
101 rdf:type schema:CreativeWork
102 https://doi.org/10.1016/0893-6080(94)90102-3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047547579
103 rdf:type schema:CreativeWork
104 https://doi.org/10.1016/s0893-6080(97)00135-4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1038102490
105 rdf:type schema:CreativeWork
106 https://doi.org/10.1016/s0893-6080(97)00136-6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1007296336
107 rdf:type schema:CreativeWork
108 https://doi.org/10.1109/21.362969 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061122054
109 rdf:type schema:CreativeWork
110 https://doi.org/10.1109/72.950134 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061219614
111 rdf:type schema:CreativeWork
112 https://doi.org/10.1109/icassp.2000.860145 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093400549
113 rdf:type schema:CreativeWork
114 https://doi.org/10.1109/tc.1978.1675165 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061532124
115 rdf:type schema:CreativeWork
116 https://doi.org/10.1109/tnn.2003.809399 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061716536
117 rdf:type schema:CreativeWork
118 https://www.grid.ac/institutes/grid.7840.b schema:alternateName Carlos III University of Madrid
119 schema:name DTSC, Univ. Carlos III de Madrid, Avda Universidad 30, 28911, Leganés, Madrid, Spain
120 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...