Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE Training Method View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2012

AUTHORS

James Ting-Ho Lo , Yichuan Gui , Yun Peng

ABSTRACT

A method of training multilayer perceptrons (MLPs) to reach a global or nearly global minimum of the standard mean squared error (MSE) criterion is proposed. It has been found that the region in the weight space that does not have a local minimum of the normalized risk-averting error (NRAE) criterion expands strictly to the entire weight space as the risk-sensitivity index increases to infinity. If the MLP under training has enough hidden neurons, the MSE and NRAE criteria are both equal to nearly zero at a global or nearly global minimum. Training the MLP with the NRAE at a sufficiently large risk-sensitivity index can therefore effectively avoid non-global local minima. Numerical experiments show consistently successful convergence from different initial guesses of the weights of the MLP at a risk-sensitivity index over 106. The experiments are conducted on examples with non-global local minima of the MSE criterion that are difficult to escape from by training directly with the MSE criterion. More... »

PAGES

440-447

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50

DOI

http://dx.doi.org/10.1007/978-3-642-31346-2_50

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1035628918


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/11", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Medical and Health Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1103", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Clinical Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Lo", 
        "givenName": "James Ting-Ho", 
        "id": "sg:person.013512526631.81", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013512526631.81"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Gui", 
        "givenName": "Yichuan", 
        "id": "sg:person.014111634523.11", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014111634523.11"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Peng", 
        "givenName": "Yun", 
        "id": "sg:person.01136741416.72", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01136741416.72"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2012", 
    "datePublishedReg": "2012-01-01", 
    "description": "A method of training multilayer perceptrons (MLPs) to reach a global or nearly global minimum of the standard mean squared error (MSE) criterion is proposed. It has been found that the region in the weight space that does not have a local minimum of the normalized risk-averting error (NRAE) criterion expands strictly to the entire weight space as the risk-sensitivity index increases to infinity. If the MLP under training has enough hidden neurons, the MSE and NRAE criteria are both equal to nearly zero at a global or nearly global minimum. Training the MLP with the NRAE at a sufficiently large risk-sensitivity index can therefore effectively avoid non-global local minima. Numerical experiments show consistently successful convergence from different initial guesses of the weights of the MLP at a risk-sensitivity index over 106. The experiments are conducted on examples with non-global local minima of the MSE criterion that are difficult to escape from by training directly with the MSE criterion.", 
    "editor": [
      {
        "familyName": "Wang", 
        "givenName": "Jun", 
        "type": "Person"
      }, 
      {
        "familyName": "Yen", 
        "givenName": "Gary G.", 
        "type": "Person"
      }, 
      {
        "familyName": "Polycarpou", 
        "givenName": "Marios M.", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-642-31346-2_50", 
    "inLanguage": "en", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-642-31345-5", 
        "978-3-642-31346-2"
      ], 
      "name": "Advances in Neural Networks \u2013 ISNN 2012", 
      "type": "Book"
    }, 
    "keywords": [
      "criteria", 
      "neurons", 
      "training", 
      "index", 
      "index increases", 
      "increase", 
      "weight", 
      "training methods", 
      "method", 
      "minimum", 
      "region", 
      "risk-sensitivity index", 
      "NRAE", 
      "experiments", 
      "problem", 
      "multilayer perceptron", 
      "space", 
      "perceptron", 
      "weight space", 
      "successful convergence", 
      "example", 
      "global minimum", 
      "mean squared error criterion", 
      "squared error criterion", 
      "error criterion", 
      "local minima", 
      "infinity", 
      "MSE", 
      "numerical experiments", 
      "convergence", 
      "different initial guesses", 
      "initial guess", 
      "guess", 
      "MSE criterion", 
      "minimum problem", 
      "risk-averting error criterion"
    ], 
    "name": "Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE Training Method", 
    "pagination": "440-447", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1035628918"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-642-31346-2_50"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-642-31346-2_50", 
      "https://app.dimensions.ai/details/publication/pub.1035628918"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-06-01T22:36", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220601/entities/gbq_results/chapter/chapter_451.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-642-31346-2_50"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-31346-2_50'


 

This table displays all metadata directly associated to this object as RDF triples.

122 TRIPLES      23 PREDICATES      62 URIs      55 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-642-31346-2_50 schema:about anzsrc-for:11
2 anzsrc-for:1103
3 schema:author N7349809fe66e41ce8326efecc7214c1d
4 schema:datePublished 2012
5 schema:datePublishedReg 2012-01-01
6 schema:description A method of training multilayer perceptrons (MLPs) to reach a global or nearly global minimum of the standard mean squared error (MSE) criterion is proposed. It has been found that the region in the weight space that does not have a local minimum of the normalized risk-averting error (NRAE) criterion expands strictly to the entire weight space as the risk-sensitivity index increases to infinity. If the MLP under training has enough hidden neurons, the MSE and NRAE criteria are both equal to nearly zero at a global or nearly global minimum. Training the MLP with the NRAE at a sufficiently large risk-sensitivity index can therefore effectively avoid non-global local minima. Numerical experiments show consistently successful convergence from different initial guesses of the weights of the MLP at a risk-sensitivity index over 106. The experiments are conducted on examples with non-global local minima of the MSE criterion that are difficult to escape from by training directly with the MSE criterion.
7 schema:editor Na3e257bfec26458c9678ce7a229bb1c4
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree false
11 schema:isPartOf N785429d281a14d048b0ff39ca1e34ff2
12 schema:keywords MSE
13 MSE criterion
14 NRAE
15 convergence
16 criteria
17 different initial guesses
18 error criterion
19 example
20 experiments
21 global minimum
22 guess
23 increase
24 index
25 index increases
26 infinity
27 initial guess
28 local minima
29 mean squared error criterion
30 method
31 minimum
32 minimum problem
33 multilayer perceptron
34 neurons
35 numerical experiments
36 perceptron
37 problem
38 region
39 risk-averting error criterion
40 risk-sensitivity index
41 space
42 squared error criterion
43 successful convergence
44 training
45 training methods
46 weight
47 weight space
48 schema:name Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE Training Method
49 schema:pagination 440-447
50 schema:productId N2ed4669085014871ac2ab93026a51dab
51 N9bc7eaea5da24af8a1eac3d9888c6537
52 schema:publisher N8d034199143849b384c592f5c5e87f2e
53 schema:sameAs https://app.dimensions.ai/details/publication/pub.1035628918
54 https://doi.org/10.1007/978-3-642-31346-2_50
55 schema:sdDatePublished 2022-06-01T22:36
56 schema:sdLicense https://scigraph.springernature.com/explorer/license/
57 schema:sdPublisher N8e72b81aaeb34254a79beb53b8b171a0
58 schema:url https://doi.org/10.1007/978-3-642-31346-2_50
59 sgo:license sg:explorer/license/
60 sgo:sdDataset chapters
61 rdf:type schema:Chapter
62 N2ed4669085014871ac2ab93026a51dab schema:name dimensions_id
63 schema:value pub.1035628918
64 rdf:type schema:PropertyValue
65 N413214a9b525406ba5c70cc20544040b schema:familyName Wang
66 schema:givenName Jun
67 rdf:type schema:Person
68 N425a52dfff5d46c9995339063816bf1b schema:familyName Polycarpou
69 schema:givenName Marios M.
70 rdf:type schema:Person
71 N52197d770f53495d8508a75f1771c0d0 schema:familyName Yen
72 schema:givenName Gary G.
73 rdf:type schema:Person
74 N5b3c261d4eee4ea3b63f2cef53b8f65c rdf:first sg:person.01136741416.72
75 rdf:rest rdf:nil
76 N7349809fe66e41ce8326efecc7214c1d rdf:first sg:person.013512526631.81
77 rdf:rest Ne011a813239a414c8f414ee0a786b242
78 N785429d281a14d048b0ff39ca1e34ff2 schema:isbn 978-3-642-31345-5
79 978-3-642-31346-2
80 schema:name Advances in Neural Networks – ISNN 2012
81 rdf:type schema:Book
82 N8d034199143849b384c592f5c5e87f2e schema:name Springer Nature
83 rdf:type schema:Organisation
84 N8e72b81aaeb34254a79beb53b8b171a0 schema:name Springer Nature - SN SciGraph project
85 rdf:type schema:Organization
86 N9bc7eaea5da24af8a1eac3d9888c6537 schema:name doi
87 schema:value 10.1007/978-3-642-31346-2_50
88 rdf:type schema:PropertyValue
89 Na3e257bfec26458c9678ce7a229bb1c4 rdf:first N413214a9b525406ba5c70cc20544040b
90 rdf:rest Ncfd8158938c040b29df00c90b593f918
91 Na4a5d2bc7c7f4b3088503160a57b47d4 rdf:first N425a52dfff5d46c9995339063816bf1b
92 rdf:rest rdf:nil
93 Ncfd8158938c040b29df00c90b593f918 rdf:first N52197d770f53495d8508a75f1771c0d0
94 rdf:rest Na4a5d2bc7c7f4b3088503160a57b47d4
95 Ne011a813239a414c8f414ee0a786b242 rdf:first sg:person.014111634523.11
96 rdf:rest N5b3c261d4eee4ea3b63f2cef53b8f65c
97 anzsrc-for:11 schema:inDefinedTermSet anzsrc-for:
98 schema:name Medical and Health Sciences
99 rdf:type schema:DefinedTerm
100 anzsrc-for:1103 schema:inDefinedTermSet anzsrc-for:
101 schema:name Clinical Sciences
102 rdf:type schema:DefinedTerm
103 sg:person.01136741416.72 schema:affiliation grid-institutes:grid.266673.0
104 schema:familyName Peng
105 schema:givenName Yun
106 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01136741416.72
107 rdf:type schema:Person
108 sg:person.013512526631.81 schema:affiliation grid-institutes:grid.266673.0
109 schema:familyName Lo
110 schema:givenName James Ting-Ho
111 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013512526631.81
112 rdf:type schema:Person
113 sg:person.014111634523.11 schema:affiliation grid-institutes:grid.266673.0
114 schema:familyName Gui
115 schema:givenName Yichuan
116 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014111634523.11
117 rdf:type schema:Person
118 grid-institutes:grid.266673.0 schema:alternateName Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
119 Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
120 schema:name Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
121 Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
122 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...