Parallel Implementation of the Nonlinear Semi-NMF Based Alternating Optimization Method for Deep Neural Networks View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2017-05-31

AUTHORS

Akira Imakura, Yuto Inoue, Tetsuya Sakurai, Yasunori Futamura

ABSTRACT

For computing weights of deep neural networks (DNNs), the backpropagation (BP) method has been widely used as a de-facto standard algorithm. Since the BP method is based on a stochastic gradient descent method using derivatives of objective functions, the BP method has some difficulties finding appropriate parameters such as learning rate. As another approach for computing weight matrices, we recently proposed an alternating optimization method using linear and nonlinear semi-nonnegative matrix factorizations (semi-NMFs). In this paper, we propose a parallel implementation of the nonlinear semi-NMF based method. The experimental results show that our nonlinear semi-NMF based method and its parallel implementation have competitive advantages to the conventional DNNs with the BP method. More... »

PAGES

815-827

References to SciGraph publications

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/s11063-017-9642-2

DOI

http://dx.doi.org/10.1007/s11063-017-9642-2

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1085727906


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/17", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Psychology and Cognitive Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1702", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Cognitive Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan", 
          "id": "http://www.grid.ac/institutes/grid.20515.33", 
          "name": [
            "University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Imakura", 
        "givenName": "Akira", 
        "id": "sg:person.014606333603.27", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014606333603.27"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan", 
          "id": "http://www.grid.ac/institutes/grid.20515.33", 
          "name": [
            "University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Inoue", 
        "givenName": "Yuto", 
        "id": "sg:person.012112105463.21", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012112105463.21"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "JST/CREST, Kawaguchi, Japan", 
          "id": "http://www.grid.ac/institutes/grid.419082.6", 
          "name": [
            "University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan", 
            "JST/CREST, Kawaguchi, Japan"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Sakurai", 
        "givenName": "Tetsuya", 
        "id": "sg:person.010736530641.76", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010736530641.76"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan", 
          "id": "http://www.grid.ac/institutes/grid.20515.33", 
          "name": [
            "University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Futamura", 
        "givenName": "Yasunori", 
        "id": "sg:person.012135273315.39", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012135273315.39"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "sg:pub.10.1007/978-3-319-46681-1_43", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1047129491", 
          "https://doi.org/10.1007/978-3-319-46681-1_43"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1038/323533a0", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1018367015", 
          "https://doi.org/10.1038/323533a0"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2017-05-31", 
    "datePublishedReg": "2017-05-31", 
    "description": "For computing weights of deep neural networks (DNNs), the backpropagation (BP) method has been widely used as a de-facto standard algorithm. Since the BP method is based on a stochastic gradient descent method using derivatives of objective functions, the BP method has some difficulties finding appropriate parameters such as learning rate. As another approach for computing weight matrices, we recently proposed an alternating optimization method using linear and nonlinear semi-nonnegative matrix factorizations (semi-NMFs). In this paper, we propose a parallel implementation of the nonlinear semi-NMF based method. The experimental results show that our nonlinear semi-NMF based method and its parallel implementation have competitive advantages to the conventional DNNs with the BP method.", 
    "genre": "article", 
    "id": "sg:pub.10.1007/s11063-017-9642-2", 
    "isAccessibleForFree": false, 
    "isFundedItemOf": [
      {
        "id": "sg:grant.6838112", 
        "type": "MonetaryGrant"
      }
    ], 
    "isPartOf": [
      {
        "id": "sg:journal.1132792", 
        "issn": [
          "1370-4621", 
          "1573-773X"
        ], 
        "name": "Neural Processing Letters", 
        "publisher": "Springer Nature", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "3", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "47"
      }
    ], 
    "keywords": [
      "optimization method", 
      "parallel implementation", 
      "alternating optimization method", 
      "stochastic gradient descent method", 
      "gradient descent method", 
      "semi-nonnegative matrix factorization", 
      "objective function", 
      "descent method", 
      "Semi-NMF", 
      "deep neural networks", 
      "BP method", 
      "weight matrix", 
      "conventional deep neural networks", 
      "neural network", 
      "standard algorithms", 
      "backpropagation method", 
      "appropriate parameters", 
      "matrix factorization", 
      "factorization", 
      "experimental results", 
      "network", 
      "algorithm", 
      "matrix", 
      "implementation", 
      "parameters", 
      "function", 
      "approach", 
      "derivatives", 
      "advantages", 
      "results", 
      "difficulties", 
      "weight", 
      "rate", 
      "competitive advantage", 
      "method", 
      "paper"
    ], 
    "name": "Parallel Implementation of the Nonlinear Semi-NMF Based Alternating Optimization Method for Deep Neural Networks", 
    "pagination": "815-827", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1085727906"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/s11063-017-9642-2"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1007/s11063-017-9642-2", 
      "https://app.dimensions.ai/details/publication/pub.1085727906"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2022-08-04T17:06", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220804/entities/gbq_results/article/article_751.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "https://doi.org/10.1007/s11063-017-9642-2"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s11063-017-9642-2'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s11063-017-9642-2'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s11063-017-9642-2'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s11063-017-9642-2'


 

This table displays all metadata directly associated to this object as RDF triples.

136 TRIPLES      21 PREDICATES      64 URIs      52 LITERALS      6 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/s11063-017-9642-2 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 anzsrc-for:17
4 anzsrc-for:1702
5 schema:author N4447333bff7c45b9bf9c98b43e67fbba
6 schema:citation sg:pub.10.1007/978-3-319-46681-1_43
7 sg:pub.10.1038/323533a0
8 schema:datePublished 2017-05-31
9 schema:datePublishedReg 2017-05-31
10 schema:description For computing weights of deep neural networks (DNNs), the backpropagation (BP) method has been widely used as a de-facto standard algorithm. Since the BP method is based on a stochastic gradient descent method using derivatives of objective functions, the BP method has some difficulties finding appropriate parameters such as learning rate. As another approach for computing weight matrices, we recently proposed an alternating optimization method using linear and nonlinear semi-nonnegative matrix factorizations (semi-NMFs). In this paper, we propose a parallel implementation of the nonlinear semi-NMF based method. The experimental results show that our nonlinear semi-NMF based method and its parallel implementation have competitive advantages to the conventional DNNs with the BP method.
11 schema:genre article
12 schema:isAccessibleForFree false
13 schema:isPartOf N9eea31af629b4c73bc58f3eee8ea505a
14 Nd6000c9af3ef4fd3896095a19a0918d3
15 sg:journal.1132792
16 schema:keywords BP method
17 Semi-NMF
18 advantages
19 algorithm
20 alternating optimization method
21 approach
22 appropriate parameters
23 backpropagation method
24 competitive advantage
25 conventional deep neural networks
26 deep neural networks
27 derivatives
28 descent method
29 difficulties
30 experimental results
31 factorization
32 function
33 gradient descent method
34 implementation
35 matrix
36 matrix factorization
37 method
38 network
39 neural network
40 objective function
41 optimization method
42 paper
43 parallel implementation
44 parameters
45 rate
46 results
47 semi-nonnegative matrix factorization
48 standard algorithms
49 stochastic gradient descent method
50 weight
51 weight matrix
52 schema:name Parallel Implementation of the Nonlinear Semi-NMF Based Alternating Optimization Method for Deep Neural Networks
53 schema:pagination 815-827
54 schema:productId N35b5495dcde44fa29a75208c558d6e9c
55 N486fe8a5c335412fae5551fbca785cc0
56 schema:sameAs https://app.dimensions.ai/details/publication/pub.1085727906
57 https://doi.org/10.1007/s11063-017-9642-2
58 schema:sdDatePublished 2022-08-04T17:06
59 schema:sdLicense https://scigraph.springernature.com/explorer/license/
60 schema:sdPublisher N1d92052f0a5f4b7faeb24b80ed208b39
61 schema:url https://doi.org/10.1007/s11063-017-9642-2
62 sgo:license sg:explorer/license/
63 sgo:sdDataset articles
64 rdf:type schema:ScholarlyArticle
65 N1a199fa8a88542fdb0ba42ff2792d826 rdf:first sg:person.012112105463.21
66 rdf:rest N648f5842419243f4b1a0309ba7c6cf6f
67 N1d92052f0a5f4b7faeb24b80ed208b39 schema:name Springer Nature - SN SciGraph project
68 rdf:type schema:Organization
69 N35b5495dcde44fa29a75208c558d6e9c schema:name doi
70 schema:value 10.1007/s11063-017-9642-2
71 rdf:type schema:PropertyValue
72 N4447333bff7c45b9bf9c98b43e67fbba rdf:first sg:person.014606333603.27
73 rdf:rest N1a199fa8a88542fdb0ba42ff2792d826
74 N486fe8a5c335412fae5551fbca785cc0 schema:name dimensions_id
75 schema:value pub.1085727906
76 rdf:type schema:PropertyValue
77 N648f5842419243f4b1a0309ba7c6cf6f rdf:first sg:person.010736530641.76
78 rdf:rest Nb2cd4c1ecceb43aa8291ba82203c1fd1
79 N9eea31af629b4c73bc58f3eee8ea505a schema:issueNumber 3
80 rdf:type schema:PublicationIssue
81 Nb2cd4c1ecceb43aa8291ba82203c1fd1 rdf:first sg:person.012135273315.39
82 rdf:rest rdf:nil
83 Nd6000c9af3ef4fd3896095a19a0918d3 schema:volumeNumber 47
84 rdf:type schema:PublicationVolume
85 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
86 schema:name Information and Computing Sciences
87 rdf:type schema:DefinedTerm
88 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
89 schema:name Artificial Intelligence and Image Processing
90 rdf:type schema:DefinedTerm
91 anzsrc-for:17 schema:inDefinedTermSet anzsrc-for:
92 schema:name Psychology and Cognitive Sciences
93 rdf:type schema:DefinedTerm
94 anzsrc-for:1702 schema:inDefinedTermSet anzsrc-for:
95 schema:name Cognitive Sciences
96 rdf:type schema:DefinedTerm
97 sg:grant.6838112 http://pending.schema.org/fundedItem sg:pub.10.1007/s11063-017-9642-2
98 rdf:type schema:MonetaryGrant
99 sg:journal.1132792 schema:issn 1370-4621
100 1573-773X
101 schema:name Neural Processing Letters
102 schema:publisher Springer Nature
103 rdf:type schema:Periodical
104 sg:person.010736530641.76 schema:affiliation grid-institutes:grid.419082.6
105 schema:familyName Sakurai
106 schema:givenName Tetsuya
107 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010736530641.76
108 rdf:type schema:Person
109 sg:person.012112105463.21 schema:affiliation grid-institutes:grid.20515.33
110 schema:familyName Inoue
111 schema:givenName Yuto
112 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012112105463.21
113 rdf:type schema:Person
114 sg:person.012135273315.39 schema:affiliation grid-institutes:grid.20515.33
115 schema:familyName Futamura
116 schema:givenName Yasunori
117 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012135273315.39
118 rdf:type schema:Person
119 sg:person.014606333603.27 schema:affiliation grid-institutes:grid.20515.33
120 schema:familyName Imakura
121 schema:givenName Akira
122 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014606333603.27
123 rdf:type schema:Person
124 sg:pub.10.1007/978-3-319-46681-1_43 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047129491
125 https://doi.org/10.1007/978-3-319-46681-1_43
126 rdf:type schema:CreativeWork
127 sg:pub.10.1038/323533a0 schema:sameAs https://app.dimensions.ai/details/publication/pub.1018367015
128 https://doi.org/10.1038/323533a0
129 rdf:type schema:CreativeWork
130 grid-institutes:grid.20515.33 schema:alternateName University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan
131 schema:name University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan
132 rdf:type schema:Organization
133 grid-institutes:grid.419082.6 schema:alternateName JST/CREST, Kawaguchi, Japan
134 schema:name JST/CREST, Kawaguchi, Japan
135 University of Tsukuba, Tennodai 1-1-1, 305-8573, Tsukuba, Ibaraki, Japan
136 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...