Ontology type: schema:Chapter
2017-10-24
AUTHORSJames Ting-Ho Lo , Yichuan Gui , Yun Peng
ABSTRACTThe local-minimum problem in training deep learning machines (DLMs) has plagued their development. This paper proposes a method to directly solve the problem. Our method is based on convexification of the sum squared error (SSE) criterion through transforming the SSE into a risk averting error (RAE) criterion. To alleviate numerical difficulties, a normalized RAE (NRAE) is employed. The convexity region of the SSE expands as its risk sensitivity index (RSI) increases. Making the best use of the convexity region, our method starts training with a very large RSI, gradually reduces it, and switches to the RAE as soon as the RAE is numerically feasible. After training converges, the resultant DLM is expected to be inside the attraction basin of a global minimum of the SSE. Numerical results are provided to show the effectiveness of the proposed method. More... »
PAGES166-174
Neural Information Processing
ISBN
978-3-319-70086-1
978-3-319-70087-8
http://scigraph.springernature.com/pub.10.1007/978-3-319-70087-8_18
DOIhttp://dx.doi.org/10.1007/978-3-319-70087-8_18
DIMENSIONShttps://app.dimensions.ai/details/publication/pub.1092370213
JSON-LD is the canonical representation for SciGraph data.
TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT
[
{
"@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json",
"about": [
{
"id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/17",
"inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/",
"name": "Psychology and Cognitive Sciences",
"type": "DefinedTerm"
},
{
"id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1702",
"inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/",
"name": "Cognitive Sciences",
"type": "DefinedTerm"
}
],
"author": [
{
"affiliation": {
"alternateName": "Department of Mathematics and Statistics, University of Maryland Baltimore County, 21042, Baltimore, MD, USA",
"id": "http://www.grid.ac/institutes/grid.266673.0",
"name": [
"Department of Mathematics and Statistics, University of Maryland Baltimore County, 21042, Baltimore, MD, USA"
],
"type": "Organization"
},
"familyName": "Lo",
"givenName": "James Ting-Ho",
"id": "sg:person.013512526631.81",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013512526631.81"
],
"type": "Person"
},
{
"affiliation": {
"alternateName": "Department of Computer Science and Electrical Engineering, University of Maryland Baltimore County, 21042, Baltimore, MD, USA",
"id": "http://www.grid.ac/institutes/grid.266673.0",
"name": [
"Department of Computer Science and Electrical Engineering, University of Maryland Baltimore County, 21042, Baltimore, MD, USA"
],
"type": "Organization"
},
"familyName": "Gui",
"givenName": "Yichuan",
"id": "sg:person.014111634523.11",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014111634523.11"
],
"type": "Person"
},
{
"affiliation": {
"alternateName": "Department of Computer Science and Electrical Engineering, University of Maryland Baltimore County, 21042, Baltimore, MD, USA",
"id": "http://www.grid.ac/institutes/grid.266673.0",
"name": [
"Department of Computer Science and Electrical Engineering, University of Maryland Baltimore County, 21042, Baltimore, MD, USA"
],
"type": "Organization"
},
"familyName": "Peng",
"givenName": "Yun",
"id": "sg:person.01136741416.72",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01136741416.72"
],
"type": "Person"
}
],
"datePublished": "2017-10-24",
"datePublishedReg": "2017-10-24",
"description": "The local-minimum problem in training deep learning machines (DLMs) has plagued their development. This paper proposes a method to directly solve the problem. Our method is based on convexification of the sum squared error (SSE) criterion through transforming the SSE into a risk averting error (RAE) criterion. To alleviate numerical difficulties, a normalized RAE (NRAE) is employed. The convexity region of the SSE expands as its risk sensitivity index (RSI) increases. Making the best use of the convexity region, our method starts training with a very large RSI, gradually reduces it, and switches to the RAE as soon as the RAE is numerically feasible. After training converges, the resultant DLM is expected to be inside the attraction basin of a global minimum of the SSE. Numerical results are provided to show the effectiveness of the proposed method.",
"editor": [
{
"familyName": "Liu",
"givenName": "Derong",
"type": "Person"
},
{
"familyName": "Xie",
"givenName": "Shengli",
"type": "Person"
},
{
"familyName": "Li",
"givenName": "Yuanqing",
"type": "Person"
},
{
"familyName": "Zhao",
"givenName": "Dongbin",
"type": "Person"
},
{
"familyName": "El-Alfy",
"givenName": "El-Sayed M.",
"type": "Person"
}
],
"genre": "chapter",
"id": "sg:pub.10.1007/978-3-319-70087-8_18",
"inLanguage": "en",
"isAccessibleForFree": false,
"isPartOf": {
"isbn": [
"978-3-319-70086-1",
"978-3-319-70087-8"
],
"name": "Neural Information Processing",
"type": "Book"
},
"keywords": [
"deep learning machine",
"problem",
"development",
"difficulties",
"RAE",
"better use",
"effectiveness",
"local minima problem",
"learning machine",
"machine",
"paper",
"method",
"sum",
"error criterion",
"criteria",
"SSE",
"risk",
"region",
"index increases",
"increase",
"use",
"training converges",
"basin",
"results",
"convexification",
"numerical difficulties",
"convexity region",
"RSI",
"converges",
"attraction basins",
"global minimum",
"minimum",
"numerical results"
],
"name": "Solving the Local-Minimum Problem in Training Deep Learning Machines",
"pagination": "166-174",
"productId": [
{
"name": "dimensions_id",
"type": "PropertyValue",
"value": [
"pub.1092370213"
]
},
{
"name": "doi",
"type": "PropertyValue",
"value": [
"10.1007/978-3-319-70087-8_18"
]
}
],
"publisher": {
"name": "Springer Nature",
"type": "Organisation"
},
"sameAs": [
"https://doi.org/10.1007/978-3-319-70087-8_18",
"https://app.dimensions.ai/details/publication/pub.1092370213"
],
"sdDataset": "chapters",
"sdDatePublished": "2022-06-01T22:28",
"sdLicense": "https://scigraph.springernature.com/explorer/license/",
"sdPublisher": {
"name": "Springer Nature - SN SciGraph project",
"type": "Organization"
},
"sdSource": "s3://com-springernature-scigraph/baseset/20220601/entities/gbq_results/chapter/chapter_172.jsonl",
"type": "Chapter",
"url": "https://doi.org/10.1007/978-3-319-70087-8_18"
}
]
Download the RDF metadata as: json-ld nt turtle xml License info
JSON-LD is a popular format for linked data which is fully compatible with JSON.
curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-70087-8_18'
N-Triples is a line-based linked data format ideal for batch operations.
curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-70087-8_18'
Turtle is a human-readable linked data format.
curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-70087-8_18'
RDF/XML is a standard XML format for linked data.
curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-70087-8_18'
This table displays all metadata directly associated to this object as RDF triples.
129 TRIPLES
23 PREDICATES
58 URIs
51 LITERALS
7 BLANK NODES