Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE-MSE Training Method View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2013

AUTHORS

James Ting-Ho Lo , Yichuan Gui , Yun Peng

ABSTRACT

The normalized risk-averting error (NRAE) training method presented in ISNN 2012 is capable of overcoming the local-minimum problem in training neural networks. However, the overall success rate is unsatisfactory. Motivated by this problem, a modification, called the NRAE-MSE training method is herein proposed. The new method trains neural networks with respect to NRAE with a fixed λ in the range of 106-1011, and takes excursions to train with the standard mean squared error (MSE) from time to time. Once an excursion produces a satisfactory MSE with cross-validation, the entire NRAE-MSE training stops. Numerical experiments show that the NRAE-MSE training method has a success rate of 100% in all the testing examples each starting with a large number of randomly selected initial weights. More... »

PAGES

83-90

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-642-39065-4_11

DOI

http://dx.doi.org/10.1007/978-3-642-39065-4_11

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1016248400


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Lo", 
        "givenName": "James Ting-Ho", 
        "id": "sg:person.013512526631.81", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013512526631.81"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Gui", 
        "givenName": "Yichuan", 
        "id": "sg:person.014111634523.11", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014111634523.11"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA", 
          "id": "http://www.grid.ac/institutes/grid.266673.0", 
          "name": [
            "Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Peng", 
        "givenName": "Yun", 
        "id": "sg:person.01136741416.72", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01136741416.72"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2013", 
    "datePublishedReg": "2013-01-01", 
    "description": "The normalized risk-averting error (NRAE) training method presented in ISNN 2012 is capable of overcoming the local-minimum problem in training neural networks. However, the overall success rate is unsatisfactory. Motivated by this problem, a modification, called the NRAE-MSE training method is herein proposed. The new method trains neural networks with respect to NRAE with a fixed \u03bb in the range of 106-1011, and takes excursions to train with the standard mean squared error (MSE) from time to time. Once an excursion produces a satisfactory MSE with cross-validation, the entire NRAE-MSE training stops. Numerical experiments show that the NRAE-MSE training method has a success rate of 100% in all the testing examples each starting with a large number of randomly selected initial weights.", 
    "editor": [
      {
        "familyName": "Guo", 
        "givenName": "Chengan", 
        "type": "Person"
      }, 
      {
        "familyName": "Hou", 
        "givenName": "Zeng-Guang", 
        "type": "Person"
      }, 
      {
        "familyName": "Zeng", 
        "givenName": "Zhigang", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-642-39065-4_11", 
    "inLanguage": "en", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-642-39064-7", 
        "978-3-642-39065-4"
      ], 
      "name": "Advances in Neural Networks \u2013 ISNN 2013", 
      "type": "Book"
    }, 
    "keywords": [
      "mean squared error", 
      "network", 
      "overall success rate", 
      "success rate", 
      "method", 
      "local minima problem", 
      "new method", 
      "modification", 
      "large number", 
      "range", 
      "training methods", 
      "neural network", 
      "initial weight", 
      "squared error", 
      "time", 
      "minimum problem", 
      "rate", 
      "multilayer perceptron", 
      "problem", 
      "standard mean squared error", 
      "testing examples", 
      "excursion", 
      "weight", 
      "experiments", 
      "respect", 
      "error", 
      "number", 
      "example", 
      "perceptron", 
      "numerical experiments", 
      "NRAE", 
      "stops"
    ], 
    "name": "Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE-MSE Training Method", 
    "pagination": "83-90", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1016248400"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-642-39065-4_11"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-642-39065-4_11", 
      "https://app.dimensions.ai/details/publication/pub.1016248400"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-06-01T22:30", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220601/entities/gbq_results/chapter/chapter_26.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-642-39065-4_11"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-39065-4_11'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-39065-4_11'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-39065-4_11'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-39065-4_11'


 

This table displays all metadata directly associated to this object as RDF triples.

118 TRIPLES      23 PREDICATES      58 URIs      51 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-642-39065-4_11 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author N702e970902964558b19e73f73b53ee22
4 schema:datePublished 2013
5 schema:datePublishedReg 2013-01-01
6 schema:description The normalized risk-averting error (NRAE) training method presented in ISNN 2012 is capable of overcoming the local-minimum problem in training neural networks. However, the overall success rate is unsatisfactory. Motivated by this problem, a modification, called the NRAE-MSE training method is herein proposed. The new method trains neural networks with respect to NRAE with a fixed λ in the range of 106-1011, and takes excursions to train with the standard mean squared error (MSE) from time to time. Once an excursion produces a satisfactory MSE with cross-validation, the entire NRAE-MSE training stops. Numerical experiments show that the NRAE-MSE training method has a success rate of 100% in all the testing examples each starting with a large number of randomly selected initial weights.
7 schema:editor N9e7bee1c8a0d489ba086c873eb7993bc
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree false
11 schema:isPartOf Nd0f6ae0f83534aada1a516e955a1d1b4
12 schema:keywords NRAE
13 error
14 example
15 excursion
16 experiments
17 initial weight
18 large number
19 local minima problem
20 mean squared error
21 method
22 minimum problem
23 modification
24 multilayer perceptron
25 network
26 neural network
27 new method
28 number
29 numerical experiments
30 overall success rate
31 perceptron
32 problem
33 range
34 rate
35 respect
36 squared error
37 standard mean squared error
38 stops
39 success rate
40 testing examples
41 time
42 training methods
43 weight
44 schema:name Overcoming the Local-Minimum Problem in Training Multilayer Perceptrons with the NRAE-MSE Training Method
45 schema:pagination 83-90
46 schema:productId N4a10914d9820485f88faa86ab2b8a48e
47 N5166637a4acc41dfb916d133d3cdec1f
48 schema:publisher N27ec569c2def400f8da1a8a3deb990c1
49 schema:sameAs https://app.dimensions.ai/details/publication/pub.1016248400
50 https://doi.org/10.1007/978-3-642-39065-4_11
51 schema:sdDatePublished 2022-06-01T22:30
52 schema:sdLicense https://scigraph.springernature.com/explorer/license/
53 schema:sdPublisher N0b6560eefc8d484b8bd95cd5c1b9104f
54 schema:url https://doi.org/10.1007/978-3-642-39065-4_11
55 sgo:license sg:explorer/license/
56 sgo:sdDataset chapters
57 rdf:type schema:Chapter
58 N0b6560eefc8d484b8bd95cd5c1b9104f schema:name Springer Nature - SN SciGraph project
59 rdf:type schema:Organization
60 N24938415481f4adcb4d2540f75fd36c0 schema:familyName Guo
61 schema:givenName Chengan
62 rdf:type schema:Person
63 N27ec569c2def400f8da1a8a3deb990c1 schema:name Springer Nature
64 rdf:type schema:Organisation
65 N286031f58a8442e3bbf01e80dc9d46f1 schema:familyName Zeng
66 schema:givenName Zhigang
67 rdf:type schema:Person
68 N4a10914d9820485f88faa86ab2b8a48e schema:name doi
69 schema:value 10.1007/978-3-642-39065-4_11
70 rdf:type schema:PropertyValue
71 N5166637a4acc41dfb916d133d3cdec1f schema:name dimensions_id
72 schema:value pub.1016248400
73 rdf:type schema:PropertyValue
74 N69e6db27ca2d43a1a775f63e7d8ac328 rdf:first Nbe229978905845a3ad50015f5a73b009
75 rdf:rest Ne6dfb11660274fe6b85043c1a9b736f8
76 N702e970902964558b19e73f73b53ee22 rdf:first sg:person.013512526631.81
77 rdf:rest N875253448d2e44029b4379a980ed30a8
78 N875253448d2e44029b4379a980ed30a8 rdf:first sg:person.014111634523.11
79 rdf:rest Nb7814216768c47b0a6e9655f87ec49c1
80 N9e7bee1c8a0d489ba086c873eb7993bc rdf:first N24938415481f4adcb4d2540f75fd36c0
81 rdf:rest N69e6db27ca2d43a1a775f63e7d8ac328
82 Nb7814216768c47b0a6e9655f87ec49c1 rdf:first sg:person.01136741416.72
83 rdf:rest rdf:nil
84 Nbe229978905845a3ad50015f5a73b009 schema:familyName Hou
85 schema:givenName Zeng-Guang
86 rdf:type schema:Person
87 Nd0f6ae0f83534aada1a516e955a1d1b4 schema:isbn 978-3-642-39064-7
88 978-3-642-39065-4
89 schema:name Advances in Neural Networks – ISNN 2013
90 rdf:type schema:Book
91 Ne6dfb11660274fe6b85043c1a9b736f8 rdf:first N286031f58a8442e3bbf01e80dc9d46f1
92 rdf:rest rdf:nil
93 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
94 schema:name Information and Computing Sciences
95 rdf:type schema:DefinedTerm
96 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
97 schema:name Artificial Intelligence and Image Processing
98 rdf:type schema:DefinedTerm
99 sg:person.01136741416.72 schema:affiliation grid-institutes:grid.266673.0
100 schema:familyName Peng
101 schema:givenName Yun
102 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01136741416.72
103 rdf:type schema:Person
104 sg:person.013512526631.81 schema:affiliation grid-institutes:grid.266673.0
105 schema:familyName Lo
106 schema:givenName James Ting-Ho
107 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013512526631.81
108 rdf:type schema:Person
109 sg:person.014111634523.11 schema:affiliation grid-institutes:grid.266673.0
110 schema:familyName Gui
111 schema:givenName Yichuan
112 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014111634523.11
113 rdf:type schema:Person
114 grid-institutes:grid.266673.0 schema:alternateName Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
115 Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
116 schema:name Department of Computer Science and Electrical Engineering, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
117 Department of Mathematics and Statistics, University of Maryland, Baltimore County, 21250, Baltimore, Maryland, USA
118 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...