Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE Training Method View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2012

AUTHORS

James Ting-Ho Lo , Yichuan Gui , Yun Peng

ABSTRACT

A method of training multilayer perceptrons (MLPs) to reach a global or nearly global minimum of the standard mean squared error (MSE) criterion is proposed. It has been found that the region in the weight space that does not have a local minimum of the normalized risk-averting error (NRAE) criterion expands strictly to the entire weight space as the risk-sensitivity index increases to infinity. If the MLP under training has enough hidden neurons, the MSE and NRAE criteria are both equal to nearly zero at a global or nearly global minimum. Training the MLP with the NRAE at a sufficiently large risk-sensitivity index can therefore effectively avoid non-global local minima. Numerical experiments show consistently successful convergence from different initial guesses of the weights of the MLP at a risk-sensitivity index over 106. The experiments are conducted on examples with non-global local minima of the MSE criterion that are difficult to escape from by training directly with the MSE criterion. More... »

PAGES

440-447

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50

DOI

http://dx.doi.org/10.1007/978-3-642-31346-2_50

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1035628918


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/11", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Medical and Health Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1103", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Clinical Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Lo", 
        "givenName": "James Ting-Ho", 
        "id": "sg:person.013512526631.81", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013512526631.81"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Gui", 
        "givenName": "Yichuan", 
        "id": "sg:person.014111634523.11", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014111634523.11"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Peng", 
        "givenName": "Yun", 
        "id": "sg:person.01136741416.72", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01136741416.72"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2012", 
    "datePublishedReg": "2012-01-01", 
    "description": "A method of training multilayer perceptrons (MLPs) to reach a global or nearly global minimum of the standard mean squared error (MSE) criterion is proposed. It has been found that the region in the weight space that does not have a local minimum of the normalized risk-averting error (NRAE) criterion expands strictly to the entire weight space as the risk-sensitivity index increases to infinity. If the MLP under training has enough hidden neurons, the MSE and NRAE criteria are both equal to nearly zero at a global or nearly global minimum. Training the MLP with the NRAE at a sufficiently large risk-sensitivity index can therefore effectively avoid non-global local minima. Numerical experiments show consistently successful convergence from different initial guesses of the weights of the MLP at a risk-sensitivity index over 106. The experiments are conducted on examples with non-global local minima of the MSE criterion that are difficult to escape from by training directly with the MSE criterion.", 
    "editor": [
      {
        "familyName": "Wang", 
        "givenName": "Jun", 
        "type": "Person"
      }, 
      {
        "familyName": "Yen", 
        "givenName": "Gary G.", 
        "type": "Person"
      }, 
      {
        "familyName": "Polycarpou", 
        "givenName": "Marios M.", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-642-31346-2_50", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-642-31345-5", 
        "978-3-642-31346-2"
      ], 
      "name": "Advances in Neural Networks \u2013 ISNN 2012", 
      "type": "Book"
    }, 
    "keywords": [
      "criteria", 
      "neurons", 
      "index", 
      "training", 
      "index increases", 
      "increase", 
      "weight", 
      "training methods", 
      "method", 
      "minimum", 
      "region", 
      "risk-sensitivity index", 
      "NRAE", 
      "experiments", 
      "problem", 
      "multilayer perceptron", 
      "space", 
      "perceptron", 
      "weight space", 
      "successful convergence", 
      "example", 
      "global minimum", 
      "mean squared error criterion", 
      "squared error criterion", 
      "error criterion", 
      "local minima", 
      "infinity", 
      "MSE", 
      "numerical experiments", 
      "convergence", 
      "different initial guesses", 
      "initial guess", 
      "guess", 
      "MSE criterion", 
      "minimum problem", 
      "risk-averting error criterion"
    ], 
    "name": "Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE Training Method", 
    "pagination": "440-447", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1035628918"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-642-31346-2_50"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-642-31346-2_50", 
      "https://app.dimensions.ai/details/publication/pub.1035628918"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-08-04T17:18", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220804/entities/gbq_results/chapter/chapter_299.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-642-31346-2_50"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50'


 

This table displays all metadata directly associated to this object as RDF triples.

121 TRIPLES      22 PREDICATES      61 URIs      54 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-642-31346-2_50 schema:about anzsrc-for:11
2 anzsrc-for:1103
3 schema:author N4ecb6b03da314e3183e6990aa344210b
4 schema:datePublished 2012
5 schema:datePublishedReg 2012-01-01
6 schema:description A method of training multilayer perceptrons (MLPs) to reach a global or nearly global minimum of the standard mean squared error (MSE) criterion is proposed. It has been found that the region in the weight space that does not have a local minimum of the normalized risk-averting error (NRAE) criterion expands strictly to the entire weight space as the risk-sensitivity index increases to infinity. If the MLP under training has enough hidden neurons, the MSE and NRAE criteria are both equal to nearly zero at a global or nearly global minimum. Training the MLP with the NRAE at a sufficiently large risk-sensitivity index can therefore effectively avoid non-global local minima. Numerical experiments show consistently successful convergence from different initial guesses of the weights of the MLP at a risk-sensitivity index over 106. The experiments are conducted on examples with non-global local minima of the MSE criterion that are difficult to escape from by training directly with the MSE criterion.
7 schema:editor Nd77143b8aa184a318a05edf4fde3121e
8 schema:genre chapter
9 schema:isAccessibleForFree false
10 schema:isPartOf Nd53a87a69eea4183a076dbec5413fe65
11 schema:keywords MSE
12 MSE criterion
13 NRAE
14 convergence
15 criteria
16 different initial guesses
17 error criterion
18 example
19 experiments
20 global minimum
21 guess
22 increase
23 index
24 index increases
25 infinity
26 initial guess
27 local minima
28 mean squared error criterion
29 method
30 minimum
31 minimum problem
32 multilayer perceptron
33 neurons
34 numerical experiments
35 perceptron
36 problem
37 region
38 risk-averting error criterion
39 risk-sensitivity index
40 space
41 squared error criterion
42 successful convergence
43 training
44 training methods
45 weight
46 weight space
47 schema:name Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE Training Method
48 schema:pagination 440-447
49 schema:productId N61e67a551f4f495ab7cf910a8764bc61
50 N6ce0a10649454abdb594e2cb7655fb51
51 schema:publisher N2f449c8a4c514631a944afc2d8fae348
52 schema:sameAs https://app.dimensions.ai/details/publication/pub.1035628918
53 https://doi.org/10.1007/978-3-642-31346-2_50
54 schema:sdDatePublished 2022-08-04T17:18
55 schema:sdLicense https://scigraph.springernature.com/explorer/license/
56 schema:sdPublisher N5e79b85b7c4b41ab9a58dd6502af7d05
57 schema:url https://doi.org/10.1007/978-3-642-31346-2_50
58 sgo:license sg:explorer/license/
59 sgo:sdDataset chapters
60 rdf:type schema:Chapter
61 N0b0053e22ced44d496341653e4cb2713 rdf:first N838300e380ea470bb21f8e9a0c067160
62 rdf:rest rdf:nil
63 N2f449c8a4c514631a944afc2d8fae348 schema:name Springer Nature
64 rdf:type schema:Organisation
65 N3f012c13796d4a30973ceb7b356e0794 rdf:first sg:person.01136741416.72
66 rdf:rest rdf:nil
67 N4ecb6b03da314e3183e6990aa344210b rdf:first sg:person.013512526631.81
68 rdf:rest Neb9cb430745543039717cbf602596982
69 N5a397cabb4354a288f067044750d8574 schema:familyName Wang
70 schema:givenName Jun
71 rdf:type schema:Person
72 N5e79b85b7c4b41ab9a58dd6502af7d05 schema:name Springer Nature - SN SciGraph project
73 rdf:type schema:Organization
74 N61e67a551f4f495ab7cf910a8764bc61 schema:name doi
75 schema:value 10.1007/978-3-642-31346-2_50
76 rdf:type schema:PropertyValue
77 N6ce0a10649454abdb594e2cb7655fb51 schema:name dimensions_id
78 schema:value pub.1035628918
79 rdf:type schema:PropertyValue
80 N838300e380ea470bb21f8e9a0c067160 schema:familyName Polycarpou
81 schema:givenName Marios M.
82 rdf:type schema:Person
83 Nc2343ca0c7c04aeb93d884e8290b4f94 rdf:first Ne7598c3b5b7c409ca9c30d224b3c5c82
84 rdf:rest N0b0053e22ced44d496341653e4cb2713
85 Nd53a87a69eea4183a076dbec5413fe65 schema:isbn 978-3-642-31345-5
86 978-3-642-31346-2
87 schema:name Advances in Neural Networks – ISNN 2012
88 rdf:type schema:Book
89 Nd77143b8aa184a318a05edf4fde3121e rdf:first N5a397cabb4354a288f067044750d8574
90 rdf:rest Nc2343ca0c7c04aeb93d884e8290b4f94
91 Ne7598c3b5b7c409ca9c30d224b3c5c82 schema:familyName Yen
92 schema:givenName Gary G.
93 rdf:type schema:Person
94 Neb9cb430745543039717cbf602596982 rdf:first sg:person.014111634523.11
95 rdf:rest N3f012c13796d4a30973ceb7b356e0794
96 anzsrc-for:11 schema:inDefinedTermSet anzsrc-for:
97 schema:name Medical and Health Sciences
98 rdf:type schema:DefinedTerm
99 anzsrc-for:1103 schema:inDefinedTermSet anzsrc-for:
100 schema:name Clinical Sciences
101 rdf:type schema:DefinedTerm
102 sg:person.01136741416.72 schema:affiliation grid-institutes:grid.266673.0
103 schema:familyName Peng
104 schema:givenName Yun
105 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01136741416.72
106 rdf:type schema:Person
107 sg:person.013512526631.81 schema:affiliation grid-institutes:grid.266673.0
108 schema:familyName Lo
109 schema:givenName James Ting-Ho
110 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013512526631.81
111 rdf:type schema:Person
112 sg:person.014111634523.11 schema:affiliation grid-institutes:grid.266673.0
113 schema:familyName Gui
114 schema:givenName Yichuan
115 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014111634523.11
116 rdf:type schema:Person
117 grid-institutes:grid.266673.0 schema:alternateName Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
118 Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
119 schema:name Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
120 Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
121 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...