Equivalent-accuracy accelerated neural-network training using analogue memory View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2018-06

AUTHORS

Stefano Ambrogio, Pritish Narayanan, Hsinyu Tsai, Robert M. Shelby, Irem Boybat, Carmelo di Nolfo, Severin Sidler, Massimo Giordano, Martina Bodini, Nathan C. P. Farinha, Benjamin Killeen, Christina Cheng, Yassine Jaoudi, Geoffrey W. Burr

ABSTRACT

Neural-network training can be slow and energy intensive, owing to the need to transfer the weight data for the network between conventional digital memory chips and processor chips. Analogue non-volatile memory can accelerate the neural-network training algorithm known as backpropagation by performing parallelized multiply-accumulate operations in the analogue domain at the location of the weight data. However, the classification accuracies of such in situ training using non-volatile-memory hardware have generally been less than those of software-based training, owing to insufficient dynamic range and excessive weight-update asymmetry. Here we demonstrate mixed hardware-software neural-network implementations that involve up to 204,900 synapses and that combine long-term storage in phase-change memory, near-linear updates of volatile capacitors and weight-data transfer with 'polarity inversion' to cancel out inherent device-to-device variations. We achieve generalization accuracies (on previously unseen data) equivalent to those of software-based training on various commonly used machine-learning test datasets (MNIST, MNIST-backrand, CIFAR-10 and CIFAR-100). The computational energy efficiency of 28,065 billion operations per second per watt and throughput per area of 3.6 trillion operations per second per square millimetre that we calculate for our implementation exceed those of today's graphical processing units by two orders of magnitude. This work provides a path towards hardware accelerators that are both fast and energy efficient, particularly on fully connected neural-network layers. More... »

PAGES

60-67

Identifiers

URI

http://scigraph.springernature.com/pub.10.1038/s41586-018-0180-5

DOI

http://dx.doi.org/10.1038/s41586-018-0180-5

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1104243621

PUBMED

https://www.ncbi.nlm.nih.gov/pubmed/29875487


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Ambrogio", 
        "givenName": "Stefano", 
        "id": "sg:person.012046600075.67", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012046600075.67"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Narayanan", 
        "givenName": "Pritish", 
        "id": "sg:person.07672576623.11", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07672576623.11"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Tsai", 
        "givenName": "Hsinyu", 
        "id": "sg:person.011667126025.84", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011667126025.84"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Shelby", 
        "givenName": "Robert M.", 
        "id": "sg:person.0741157665.64", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0741157665.64"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "\u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne", 
          "id": "https://www.grid.ac/institutes/grid.5333.6", 
          "name": [
            "IBM Research\u2013Zurich, Rueschlikon, Switzerland", 
            "EPFL, Lausanne, Switzerland"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Boybat", 
        "givenName": "Irem", 
        "id": "sg:person.015061655767.63", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015061655767.63"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "\u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne", 
          "id": "https://www.grid.ac/institutes/grid.5333.6", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA", 
            "EPFL, Lausanne, Switzerland"
          ], 
          "type": "Organization"
        }, 
        "familyName": "di Nolfo", 
        "givenName": "Carmelo", 
        "id": "sg:person.015552176475.94", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015552176475.94"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "\u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne", 
          "id": "https://www.grid.ac/institutes/grid.5333.6", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA", 
            "EPFL, Lausanne, Switzerland"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Sidler", 
        "givenName": "Severin", 
        "id": "sg:person.016141502521.00", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016141502521.00"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Giordano", 
        "givenName": "Massimo", 
        "id": "sg:person.015312252442.59", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015312252442.59"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "\u00c9cole Polytechnique F\u00e9d\u00e9rale de Lausanne", 
          "id": "https://www.grid.ac/institutes/grid.5333.6", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA", 
            "EPFL, Lausanne, Switzerland"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Bodini", 
        "givenName": "Martina", 
        "id": "sg:person.011673541467.01", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011673541467.01"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Farinha", 
        "givenName": "Nathan C. P.", 
        "id": "sg:person.013266502467.47", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013266502467.47"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Killeen", 
        "givenName": "Benjamin", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Cheng", 
        "givenName": "Christina", 
        "id": "sg:person.012376324742.40", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012376324742.40"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Jaoudi", 
        "givenName": "Yassine", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IBM Research - Almaden", 
          "id": "https://www.grid.ac/institutes/grid.481551.c", 
          "name": [
            "IBM Research\u2013Almaden, San Jose, CA, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Burr", 
        "givenName": "Geoffrey W.", 
        "id": "sg:person.0763236107.54", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0763236107.54"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.1080/23746149.2016.1259585", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1000997398"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.procs.2014.11.094", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1003487184"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1073/pnas.1604850113", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1003596781"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1126/science.1254642", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1005021843"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3389/fnins.2016.00333", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1007675430"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1038/nature14539", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1010020120", 
          "https://doi.org/10.1038/nature14539"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1553374.1553380", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1012146698"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1002/adma.201604310", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1013051311"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1038/323533a0", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1018367015", 
          "https://doi.org/10.1038/323533a0"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/2897937.2898010", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1027391721"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1088/0957-4484/26/45/455204", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1034921571"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1038/nature14441", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1037542087", 
          "https://doi.org/10.1038/nature14441"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1162/neco.1997.9.8.1735", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1038140272"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1862891.1862892", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1042606614"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1088/0957-4484/23/7/075201", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1047993575"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1063/1.4934818", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1058097778"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/4.309904", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061165444"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/5.726791", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061179979"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/jssc.1989.572629", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061328198"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/led.2015.2418342", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061357231"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/ted.2006.888752", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061592359"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/ted.2014.2331707", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061596263"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/ted.2015.2439635", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061596827"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1116/1.4889999", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1062177371"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1038/nmat4856", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1083850903", 
          "https://doi.org/10.1038/nmat4856"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/3020078.3021740", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1084679761"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/3079856.3080246", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1090373941"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1147/jrd.2017.2716579", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1091667770"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/cvpr.2016.308", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1093497718"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/iscas.2011.5937569", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1093780522"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.23919/vlsit.2017.7998164", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1094219033"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/iscas.2017.8050988", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1094786941"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/mwscas.2017.8052950", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1095071231"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/iedm.2015.7409718", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1095456365"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2018-06", 
    "datePublishedReg": "2018-06-01", 
    "description": "Neural-network training can be slow and energy intensive, owing to the need to transfer the weight data for the network between\u00a0conventional digital memory chips and processor chips. Analogue non-volatile memory can accelerate the neural-network training algorithm known as backpropagation by performing parallelized multiply-accumulate operations in the analogue domain at the location of the weight data. However, the classification accuracies of such in situ training using non-volatile-memory hardware have\u00a0generally been\u00a0less than those of software-based training, owing to insufficient dynamic range and excessive weight-update asymmetry. Here we demonstrate mixed hardware-software neural-network implementations that involve up to 204,900 synapses and that combine long-term storage in phase-change memory, near-linear updates of volatile capacitors and weight-data transfer with 'polarity inversion' to cancel out inherent device-to-device variations. We achieve generalization accuracies (on previously unseen data) equivalent to those of software-based training on various commonly used machine-learning test datasets (MNIST, MNIST-backrand, CIFAR-10 and CIFAR-100). The computational energy efficiency of 28,065 billion operations per second per watt and throughput per area of 3.6 trillion operations per second per square millimetre that we calculate for our implementation exceed those of today's graphical processing units by two orders of magnitude. This work provides a path towards hardware accelerators that are both fast and energy efficient, particularly on fully connected neural-network layers.", 
    "genre": "research_article", 
    "id": "sg:pub.10.1038/s41586-018-0180-5", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": false, 
    "isPartOf": [
      {
        "id": "sg:journal.1018957", 
        "issn": [
          "0090-0028", 
          "1476-4687"
        ], 
        "name": "Nature", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "7708", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "558"
      }
    ], 
    "name": "Equivalent-accuracy accelerated neural-network training using analogue memory", 
    "pagination": "60-67", 
    "productId": [
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "6ea63db3f72e7741464f40faf9f24ce05bfe83f1616cfdd304e09e47f9ace784"
        ]
      }, 
      {
        "name": "pubmed_id", 
        "type": "PropertyValue", 
        "value": [
          "29875487"
        ]
      }, 
      {
        "name": "nlm_unique_id", 
        "type": "PropertyValue", 
        "value": [
          "0410462"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1038/s41586-018-0180-5"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1104243621"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1038/s41586-018-0180-5", 
      "https://app.dimensions.ai/details/publication/pub.1104243621"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2019-04-10T13:28", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8659_00000572.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "https://www.nature.com/articles/s41586-018-0180-5"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1038/s41586-018-0180-5'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1038/s41586-018-0180-5'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1038/s41586-018-0180-5'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1038/s41586-018-0180-5'


 

This table displays all metadata directly associated to this object as RDF triples.

269 TRIPLES      21 PREDICATES      63 URIs      21 LITERALS      9 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1038/s41586-018-0180-5 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author N40064b9d6d394e708eeb574f38debd16
4 schema:citation sg:pub.10.1038/323533a0
5 sg:pub.10.1038/nature14441
6 sg:pub.10.1038/nature14539
7 sg:pub.10.1038/nmat4856
8 https://doi.org/10.1002/adma.201604310
9 https://doi.org/10.1016/j.procs.2014.11.094
10 https://doi.org/10.1063/1.4934818
11 https://doi.org/10.1073/pnas.1604850113
12 https://doi.org/10.1080/23746149.2016.1259585
13 https://doi.org/10.1088/0957-4484/23/7/075201
14 https://doi.org/10.1088/0957-4484/26/45/455204
15 https://doi.org/10.1109/4.309904
16 https://doi.org/10.1109/5.726791
17 https://doi.org/10.1109/cvpr.2016.308
18 https://doi.org/10.1109/iedm.2015.7409718
19 https://doi.org/10.1109/iscas.2011.5937569
20 https://doi.org/10.1109/iscas.2017.8050988
21 https://doi.org/10.1109/jssc.1989.572629
22 https://doi.org/10.1109/led.2015.2418342
23 https://doi.org/10.1109/mwscas.2017.8052950
24 https://doi.org/10.1109/ted.2006.888752
25 https://doi.org/10.1109/ted.2014.2331707
26 https://doi.org/10.1109/ted.2015.2439635
27 https://doi.org/10.1116/1.4889999
28 https://doi.org/10.1126/science.1254642
29 https://doi.org/10.1145/1553374.1553380
30 https://doi.org/10.1145/1862891.1862892
31 https://doi.org/10.1145/2897937.2898010
32 https://doi.org/10.1145/3020078.3021740
33 https://doi.org/10.1145/3079856.3080246
34 https://doi.org/10.1147/jrd.2017.2716579
35 https://doi.org/10.1162/neco.1997.9.8.1735
36 https://doi.org/10.23919/vlsit.2017.7998164
37 https://doi.org/10.3389/fnins.2016.00333
38 schema:datePublished 2018-06
39 schema:datePublishedReg 2018-06-01
40 schema:description Neural-network training can be slow and energy intensive, owing to the need to transfer the weight data for the network between conventional digital memory chips and processor chips. Analogue non-volatile memory can accelerate the neural-network training algorithm known as backpropagation by performing parallelized multiply-accumulate operations in the analogue domain at the location of the weight data. However, the classification accuracies of such in situ training using non-volatile-memory hardware have generally been less than those of software-based training, owing to insufficient dynamic range and excessive weight-update asymmetry. Here we demonstrate mixed hardware-software neural-network implementations that involve up to 204,900 synapses and that combine long-term storage in phase-change memory, near-linear updates of volatile capacitors and weight-data transfer with 'polarity inversion' to cancel out inherent device-to-device variations. We achieve generalization accuracies (on previously unseen data) equivalent to those of software-based training on various commonly used machine-learning test datasets (MNIST, MNIST-backrand, CIFAR-10 and CIFAR-100). The computational energy efficiency of 28,065 billion operations per second per watt and throughput per area of 3.6 trillion operations per second per square millimetre that we calculate for our implementation exceed those of today's graphical processing units by two orders of magnitude. This work provides a path towards hardware accelerators that are both fast and energy efficient, particularly on fully connected neural-network layers.
41 schema:genre research_article
42 schema:inLanguage en
43 schema:isAccessibleForFree false
44 schema:isPartOf N50642964333a4c0986315495b8699dfb
45 Nda8280de96ec441cbeb2970d4e00b487
46 sg:journal.1018957
47 schema:name Equivalent-accuracy accelerated neural-network training using analogue memory
48 schema:pagination 60-67
49 schema:productId N0a0c04c0eccd41ef9912e3b279086750
50 Nac4ef5c4ddb243d6ae15cd161563e482
51 Nd3aa1ab6002e46278495f00bebe04db6
52 Nd766254aa2274b0d92d8e5d8e73b87c6
53 Nf40dc03703764847b49dcf2c5d303efa
54 schema:sameAs https://app.dimensions.ai/details/publication/pub.1104243621
55 https://doi.org/10.1038/s41586-018-0180-5
56 schema:sdDatePublished 2019-04-10T13:28
57 schema:sdLicense https://scigraph.springernature.com/explorer/license/
58 schema:sdPublisher N6fb84a51eda74007a1f0659bd770d4d6
59 schema:url https://www.nature.com/articles/s41586-018-0180-5
60 sgo:license sg:explorer/license/
61 sgo:sdDataset articles
62 rdf:type schema:ScholarlyArticle
63 N0a0c04c0eccd41ef9912e3b279086750 schema:name dimensions_id
64 schema:value pub.1104243621
65 rdf:type schema:PropertyValue
66 N0cbadfe7e7274973806836e974ffa2e5 rdf:first sg:person.015312252442.59
67 rdf:rest Nd1e8e77ab7094695a27d0173af069801
68 N30ed5435c25041f6848bca66565b6a6e rdf:first sg:person.013266502467.47
69 rdf:rest Ncac8838a99814828ac10b28261460b55
70 N40064b9d6d394e708eeb574f38debd16 rdf:first sg:person.012046600075.67
71 rdf:rest Nfe7b4eb951894315adde7dc8439a1e5c
72 N4a8868d92e1645c9a61ed71395d61d8e rdf:first sg:person.0741157665.64
73 rdf:rest N4bfdcad7735046a6888df491d6880c84
74 N4bfdcad7735046a6888df491d6880c84 rdf:first sg:person.015061655767.63
75 rdf:rest N58ab407cf72f416da3acfb0f7716bd49
76 N50642964333a4c0986315495b8699dfb schema:issueNumber 7708
77 rdf:type schema:PublicationIssue
78 N51c5ab3568e54e3ca5a7a61f5d8bec26 rdf:first Nd09e2e6fb2144d5c8496a8695319ad05
79 rdf:rest Neb54055c80e1422a9757305105145a73
80 N58ab407cf72f416da3acfb0f7716bd49 rdf:first sg:person.015552176475.94
81 rdf:rest N5c427f91e7354901822dc96c587da638
82 N5c427f91e7354901822dc96c587da638 rdf:first sg:person.016141502521.00
83 rdf:rest N0cbadfe7e7274973806836e974ffa2e5
84 N60326e6760ca4bfda084ca9125c5ff38 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
85 schema:familyName Killeen
86 schema:givenName Benjamin
87 rdf:type schema:Person
88 N68f261aa2f6e43d2aeab3cca6496b559 rdf:first sg:person.011667126025.84
89 rdf:rest N4a8868d92e1645c9a61ed71395d61d8e
90 N6fb84a51eda74007a1f0659bd770d4d6 schema:name Springer Nature - SN SciGraph project
91 rdf:type schema:Organization
92 Nac4ef5c4ddb243d6ae15cd161563e482 schema:name doi
93 schema:value 10.1038/s41586-018-0180-5
94 rdf:type schema:PropertyValue
95 Ncac8838a99814828ac10b28261460b55 rdf:first N60326e6760ca4bfda084ca9125c5ff38
96 rdf:rest Nd91b33c7925041dfafe34b1b16c61841
97 Nd09e2e6fb2144d5c8496a8695319ad05 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
98 schema:familyName Jaoudi
99 schema:givenName Yassine
100 rdf:type schema:Person
101 Nd1e8e77ab7094695a27d0173af069801 rdf:first sg:person.011673541467.01
102 rdf:rest N30ed5435c25041f6848bca66565b6a6e
103 Nd3aa1ab6002e46278495f00bebe04db6 schema:name pubmed_id
104 schema:value 29875487
105 rdf:type schema:PropertyValue
106 Nd766254aa2274b0d92d8e5d8e73b87c6 schema:name readcube_id
107 schema:value 6ea63db3f72e7741464f40faf9f24ce05bfe83f1616cfdd304e09e47f9ace784
108 rdf:type schema:PropertyValue
109 Nd91b33c7925041dfafe34b1b16c61841 rdf:first sg:person.012376324742.40
110 rdf:rest N51c5ab3568e54e3ca5a7a61f5d8bec26
111 Nda8280de96ec441cbeb2970d4e00b487 schema:volumeNumber 558
112 rdf:type schema:PublicationVolume
113 Neb54055c80e1422a9757305105145a73 rdf:first sg:person.0763236107.54
114 rdf:rest rdf:nil
115 Nf40dc03703764847b49dcf2c5d303efa schema:name nlm_unique_id
116 schema:value 0410462
117 rdf:type schema:PropertyValue
118 Nfe7b4eb951894315adde7dc8439a1e5c rdf:first sg:person.07672576623.11
119 rdf:rest N68f261aa2f6e43d2aeab3cca6496b559
120 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
121 schema:name Information and Computing Sciences
122 rdf:type schema:DefinedTerm
123 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
124 schema:name Artificial Intelligence and Image Processing
125 rdf:type schema:DefinedTerm
126 sg:journal.1018957 schema:issn 0090-0028
127 1476-4687
128 schema:name Nature
129 rdf:type schema:Periodical
130 sg:person.011667126025.84 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
131 schema:familyName Tsai
132 schema:givenName Hsinyu
133 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011667126025.84
134 rdf:type schema:Person
135 sg:person.011673541467.01 schema:affiliation https://www.grid.ac/institutes/grid.5333.6
136 schema:familyName Bodini
137 schema:givenName Martina
138 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011673541467.01
139 rdf:type schema:Person
140 sg:person.012046600075.67 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
141 schema:familyName Ambrogio
142 schema:givenName Stefano
143 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012046600075.67
144 rdf:type schema:Person
145 sg:person.012376324742.40 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
146 schema:familyName Cheng
147 schema:givenName Christina
148 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012376324742.40
149 rdf:type schema:Person
150 sg:person.013266502467.47 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
151 schema:familyName Farinha
152 schema:givenName Nathan C. P.
153 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013266502467.47
154 rdf:type schema:Person
155 sg:person.015061655767.63 schema:affiliation https://www.grid.ac/institutes/grid.5333.6
156 schema:familyName Boybat
157 schema:givenName Irem
158 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015061655767.63
159 rdf:type schema:Person
160 sg:person.015312252442.59 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
161 schema:familyName Giordano
162 schema:givenName Massimo
163 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015312252442.59
164 rdf:type schema:Person
165 sg:person.015552176475.94 schema:affiliation https://www.grid.ac/institutes/grid.5333.6
166 schema:familyName di Nolfo
167 schema:givenName Carmelo
168 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015552176475.94
169 rdf:type schema:Person
170 sg:person.016141502521.00 schema:affiliation https://www.grid.ac/institutes/grid.5333.6
171 schema:familyName Sidler
172 schema:givenName Severin
173 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016141502521.00
174 rdf:type schema:Person
175 sg:person.0741157665.64 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
176 schema:familyName Shelby
177 schema:givenName Robert M.
178 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0741157665.64
179 rdf:type schema:Person
180 sg:person.0763236107.54 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
181 schema:familyName Burr
182 schema:givenName Geoffrey W.
183 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0763236107.54
184 rdf:type schema:Person
185 sg:person.07672576623.11 schema:affiliation https://www.grid.ac/institutes/grid.481551.c
186 schema:familyName Narayanan
187 schema:givenName Pritish
188 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07672576623.11
189 rdf:type schema:Person
190 sg:pub.10.1038/323533a0 schema:sameAs https://app.dimensions.ai/details/publication/pub.1018367015
191 https://doi.org/10.1038/323533a0
192 rdf:type schema:CreativeWork
193 sg:pub.10.1038/nature14441 schema:sameAs https://app.dimensions.ai/details/publication/pub.1037542087
194 https://doi.org/10.1038/nature14441
195 rdf:type schema:CreativeWork
196 sg:pub.10.1038/nature14539 schema:sameAs https://app.dimensions.ai/details/publication/pub.1010020120
197 https://doi.org/10.1038/nature14539
198 rdf:type schema:CreativeWork
199 sg:pub.10.1038/nmat4856 schema:sameAs https://app.dimensions.ai/details/publication/pub.1083850903
200 https://doi.org/10.1038/nmat4856
201 rdf:type schema:CreativeWork
202 https://doi.org/10.1002/adma.201604310 schema:sameAs https://app.dimensions.ai/details/publication/pub.1013051311
203 rdf:type schema:CreativeWork
204 https://doi.org/10.1016/j.procs.2014.11.094 schema:sameAs https://app.dimensions.ai/details/publication/pub.1003487184
205 rdf:type schema:CreativeWork
206 https://doi.org/10.1063/1.4934818 schema:sameAs https://app.dimensions.ai/details/publication/pub.1058097778
207 rdf:type schema:CreativeWork
208 https://doi.org/10.1073/pnas.1604850113 schema:sameAs https://app.dimensions.ai/details/publication/pub.1003596781
209 rdf:type schema:CreativeWork
210 https://doi.org/10.1080/23746149.2016.1259585 schema:sameAs https://app.dimensions.ai/details/publication/pub.1000997398
211 rdf:type schema:CreativeWork
212 https://doi.org/10.1088/0957-4484/23/7/075201 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047993575
213 rdf:type schema:CreativeWork
214 https://doi.org/10.1088/0957-4484/26/45/455204 schema:sameAs https://app.dimensions.ai/details/publication/pub.1034921571
215 rdf:type schema:CreativeWork
216 https://doi.org/10.1109/4.309904 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061165444
217 rdf:type schema:CreativeWork
218 https://doi.org/10.1109/5.726791 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061179979
219 rdf:type schema:CreativeWork
220 https://doi.org/10.1109/cvpr.2016.308 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093497718
221 rdf:type schema:CreativeWork
222 https://doi.org/10.1109/iedm.2015.7409718 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095456365
223 rdf:type schema:CreativeWork
224 https://doi.org/10.1109/iscas.2011.5937569 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093780522
225 rdf:type schema:CreativeWork
226 https://doi.org/10.1109/iscas.2017.8050988 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094786941
227 rdf:type schema:CreativeWork
228 https://doi.org/10.1109/jssc.1989.572629 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061328198
229 rdf:type schema:CreativeWork
230 https://doi.org/10.1109/led.2015.2418342 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061357231
231 rdf:type schema:CreativeWork
232 https://doi.org/10.1109/mwscas.2017.8052950 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095071231
233 rdf:type schema:CreativeWork
234 https://doi.org/10.1109/ted.2006.888752 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061592359
235 rdf:type schema:CreativeWork
236 https://doi.org/10.1109/ted.2014.2331707 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061596263
237 rdf:type schema:CreativeWork
238 https://doi.org/10.1109/ted.2015.2439635 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061596827
239 rdf:type schema:CreativeWork
240 https://doi.org/10.1116/1.4889999 schema:sameAs https://app.dimensions.ai/details/publication/pub.1062177371
241 rdf:type schema:CreativeWork
242 https://doi.org/10.1126/science.1254642 schema:sameAs https://app.dimensions.ai/details/publication/pub.1005021843
243 rdf:type schema:CreativeWork
244 https://doi.org/10.1145/1553374.1553380 schema:sameAs https://app.dimensions.ai/details/publication/pub.1012146698
245 rdf:type schema:CreativeWork
246 https://doi.org/10.1145/1862891.1862892 schema:sameAs https://app.dimensions.ai/details/publication/pub.1042606614
247 rdf:type schema:CreativeWork
248 https://doi.org/10.1145/2897937.2898010 schema:sameAs https://app.dimensions.ai/details/publication/pub.1027391721
249 rdf:type schema:CreativeWork
250 https://doi.org/10.1145/3020078.3021740 schema:sameAs https://app.dimensions.ai/details/publication/pub.1084679761
251 rdf:type schema:CreativeWork
252 https://doi.org/10.1145/3079856.3080246 schema:sameAs https://app.dimensions.ai/details/publication/pub.1090373941
253 rdf:type schema:CreativeWork
254 https://doi.org/10.1147/jrd.2017.2716579 schema:sameAs https://app.dimensions.ai/details/publication/pub.1091667770
255 rdf:type schema:CreativeWork
256 https://doi.org/10.1162/neco.1997.9.8.1735 schema:sameAs https://app.dimensions.ai/details/publication/pub.1038140272
257 rdf:type schema:CreativeWork
258 https://doi.org/10.23919/vlsit.2017.7998164 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094219033
259 rdf:type schema:CreativeWork
260 https://doi.org/10.3389/fnins.2016.00333 schema:sameAs https://app.dimensions.ai/details/publication/pub.1007675430
261 rdf:type schema:CreativeWork
262 https://www.grid.ac/institutes/grid.481551.c schema:alternateName IBM Research - Almaden
263 schema:name IBM Research–Almaden, San Jose, CA, USA
264 rdf:type schema:Organization
265 https://www.grid.ac/institutes/grid.5333.6 schema:alternateName École Polytechnique Fédérale de Lausanne
266 schema:name EPFL, Lausanne, Switzerland
267 IBM Research–Almaden, San Jose, CA, USA
268 IBM Research–Zurich, Rueschlikon, Switzerland
269 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...