Ontology type: schema:ScholarlyArticle
2015-02
AUTHORSVolodymyr Mnih, Koray Kavukcuoglu, David Silver, Andrei A. Rusu, Joel Veness, Marc G. Bellemare, Alex Graves, Martin Riedmiller, Andreas K. Fidjeland, Georg Ostrovski, Stig Petersen, Charles Beattie, Amir Sadik, Ioannis Antonoglou, Helen King, Dharshan Kumaran, Daan Wierstra, Shane Legg, Demis Hassabis
ABSTRACTThe theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks. More... »
PAGES529
http://scigraph.springernature.com/pub.10.1038/nature14236
DOIhttp://dx.doi.org/10.1038/nature14236
DIMENSIONShttps://app.dimensions.ai/details/publication/pub.1030517994
PUBMEDhttps://www.ncbi.nlm.nih.gov/pubmed/25719670
JSON-LD is the canonical representation for SciGraph data.
TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT
[
{
"@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json",
"about": [
{
"id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801",
"inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/",
"name": "Artificial Intelligence and Image Processing",
"type": "DefinedTerm"
},
{
"id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08",
"inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/",
"name": "Information and Computing Sciences",
"type": "DefinedTerm"
},
{
"inDefinedTermSet": "https://www.nlm.nih.gov/mesh/",
"name": "Algorithms",
"type": "DefinedTerm"
},
{
"inDefinedTermSet": "https://www.nlm.nih.gov/mesh/",
"name": "Artificial Intelligence",
"type": "DefinedTerm"
},
{
"inDefinedTermSet": "https://www.nlm.nih.gov/mesh/",
"name": "Humans",
"type": "DefinedTerm"
},
{
"inDefinedTermSet": "https://www.nlm.nih.gov/mesh/",
"name": "Models, Psychological",
"type": "DefinedTerm"
},
{
"inDefinedTermSet": "https://www.nlm.nih.gov/mesh/",
"name": "Neural Networks (Computer)",
"type": "DefinedTerm"
},
{
"inDefinedTermSet": "https://www.nlm.nih.gov/mesh/",
"name": "Reinforcement (Psychology)",
"type": "DefinedTerm"
},
{
"inDefinedTermSet": "https://www.nlm.nih.gov/mesh/",
"name": "Reward",
"type": "DefinedTerm"
},
{
"inDefinedTermSet": "https://www.nlm.nih.gov/mesh/",
"name": "Video Games",
"type": "DefinedTerm"
}
],
"author": [
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Mnih",
"givenName": "Volodymyr",
"id": "sg:person.01324724017.03",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01324724017.03"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Kavukcuoglu",
"givenName": "Koray",
"id": "sg:person.01354477236.72",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01354477236.72"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Silver",
"givenName": "David",
"id": "sg:person.0631073136.81",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0631073136.81"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Rusu",
"givenName": "Andrei A.",
"id": "sg:person.0604425216.82",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0604425216.82"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Veness",
"givenName": "Joel",
"id": "sg:person.0745321536.28",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0745321536.28"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Bellemare",
"givenName": "Marc G.",
"id": "sg:person.0703136271.51",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0703136271.51"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Graves",
"givenName": "Alex",
"id": "sg:person.01061550136.00",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01061550136.00"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Riedmiller",
"givenName": "Martin",
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Fidjeland",
"givenName": "Andreas K.",
"id": "sg:person.013266321653.49",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013266321653.49"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Ostrovski",
"givenName": "Georg",
"id": "sg:person.01244111736.76",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01244111736.76"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Petersen",
"givenName": "Stig",
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Beattie",
"givenName": "Charles",
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Sadik",
"givenName": "Amir",
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Antonoglou",
"givenName": "Ioannis",
"id": "sg:person.0703047436.03",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0703047436.03"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "King",
"givenName": "Helen",
"id": "sg:person.0751162636.50",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0751162636.50"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Kumaran",
"givenName": "Dharshan",
"id": "sg:person.01064217473.46",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01064217473.46"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Wierstra",
"givenName": "Daan",
"id": "sg:person.010032372337.16",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010032372337.16"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Legg",
"givenName": "Shane",
"id": "sg:person.01133524436.96",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01133524436.96"
],
"type": "Person"
},
{
"affiliation": {
"name": [
"Google DeepMind, 5 New Street Square, London EC4A 3TW, UK"
],
"type": "Organization"
},
"familyName": "Hassabis",
"givenName": "Demis",
"id": "sg:person.0741767337.55",
"sameAs": [
"https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0741767337.55"
],
"type": "Person"
}
],
"citation": [
{
"id": "https://doi.org/10.1016/j.tins.2010.01.006",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1000486991"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1126/science.275.5306.1593",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1001523695"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1126/science.1127647",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1004607132"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1145/1390156.1390187",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1007561122"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1113/jphysiol.1963.sp007079",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1009974284"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1037/0033-295x.102.3.419",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1014544738"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1007/bf00344251",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1016635886",
"https://doi.org/10.1007/bf00344251"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1007/s10514-009-9120-4",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1018478429",
"https://doi.org/10.1007/s10514-009-9120-4"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1007/s10514-009-9120-4",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1018478429",
"https://doi.org/10.1007/s10514-009-9120-4"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1007/s10514-009-9120-4",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1018478429",
"https://doi.org/10.1007/s10514-009-9120-4"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1016/s0004-3702(98)00023-x",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1020908447"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1038/415318a",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1022954489",
"https://doi.org/10.1038/415318a"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1038/415318a",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1022954489",
"https://doi.org/10.1038/415318a"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1007/bf00993104",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1026205858",
"https://doi.org/10.1007/bf00993104"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1038/nn.3203",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1030841884",
"https://doi.org/10.1038/nn.3203"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1038/nn.2304",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1044077421",
"https://doi.org/10.1038/nn.2304"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1038/nn.2304",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1044077421",
"https://doi.org/10.1038/nn.2304"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1007/s11023-007-9079-x",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1044356842",
"https://doi.org/10.1007/s11023-007-9079-x"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1145/203330.203343",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1052366953"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1007/11564096_32",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1052945304",
"https://doi.org/10.1007/11564096_32"
],
"type": "CreativeWork"
},
{
"id": "sg:pub.10.1007/11564096_32",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1052945304",
"https://doi.org/10.1007/11564096_32"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1109/5.726791",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1061179979"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1109/9.580874",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1061245225"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1109/tnn.1998.712192",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1061716400"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1561/2200000006",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1068001401"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.2307/415721",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1070578988"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1109/iccv.2009.5459469",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1093416695"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1109/ijcnn.2010.5596468",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1094389860"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1109/cvpr.2005.254",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1095271325"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1109/cvpr.2005.254",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1095271325"
],
"type": "CreativeWork"
},
{
"id": "https://doi.org/10.1613/jair.3912",
"sameAs": [
"https://app.dimensions.ai/details/publication/pub.1105689878"
],
"type": "CreativeWork"
}
],
"datePublished": "2015-02",
"datePublishedReg": "2015-02-01",
"description": "The theory of reinforcement learning provides a normative account, deeply rooted in psychological and neuroscientific perspectives on animal behaviour, of how agents may optimize their control of an environment. To use reinforcement learning successfully in situations approaching real-world complexity, however, agents are confronted with a difficult task: they must derive efficient representations of the environment from high-dimensional sensory inputs, and use these to generalize past experience to new situations. Remarkably, humans and other animals seem to solve this problem through a harmonious combination of reinforcement learning and hierarchical sensory processing systems, the former evidenced by a wealth of neural data revealing notable parallels between the phasic signals emitted by dopaminergic neurons and temporal difference reinforcement learning algorithms. While reinforcement learning agents have achieved some successes in a variety of domains, their applicability has previously been limited to domains in which useful features can be handcrafted, or to domains with fully observed, low-dimensional state spaces. Here we use recent advances in training deep neural networks to develop a novel artificial agent, termed a deep Q-network, that can learn successful policies directly from high-dimensional sensory inputs using end-to-end reinforcement learning. We tested this agent on the challenging domain of classic Atari 2600 games. We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks. ",
"genre": "research_article",
"id": "sg:pub.10.1038/nature14236",
"inLanguage": [
"en"
],
"isAccessibleForFree": false,
"isPartOf": [
{
"id": "sg:journal.1018957",
"issn": [
"0090-0028",
"1476-4687"
],
"name": "Nature",
"type": "Periodical"
},
{
"issueNumber": "7540",
"type": "PublicationIssue"
},
{
"type": "PublicationVolume",
"volumeNumber": "518"
}
],
"name": "Human-level control through deep reinforcement learning",
"pagination": "529",
"productId": [
{
"name": "readcube_id",
"type": "PropertyValue",
"value": [
"9162429e9c48d7c985e0b045d760a93bcd11f292cd7fd316e6c1e0a2c885449b"
]
},
{
"name": "pubmed_id",
"type": "PropertyValue",
"value": [
"25719670"
]
},
{
"name": "nlm_unique_id",
"type": "PropertyValue",
"value": [
"0410462"
]
},
{
"name": "doi",
"type": "PropertyValue",
"value": [
"10.1038/nature14236"
]
},
{
"name": "dimensions_id",
"type": "PropertyValue",
"value": [
"pub.1030517994"
]
}
],
"sameAs": [
"https://doi.org/10.1038/nature14236",
"https://app.dimensions.ai/details/publication/pub.1030517994"
],
"sdDataset": "articles",
"sdDatePublished": "2019-04-11T00:28",
"sdLicense": "https://scigraph.springernature.com/explorer/license/",
"sdPublisher": {
"name": "Springer Nature - SN SciGraph project",
"type": "Organization"
},
"sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8695_00000588.jsonl",
"type": "ScholarlyArticle",
"url": "https://www.nature.com/articles/nature14236"
}
]Download the RDF metadata as: json-ld nt turtle xml License info
JSON-LD is a popular format for linked data which is fully compatible with JSON.
curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1038/nature14236'
N-Triples is a line-based linked data format ideal for batch operations.
curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1038/nature14236'
Turtle is a human-readable linked data format.
curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1038/nature14236'
RDF/XML is a standard XML format for linked data.
curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1038/nature14236'
This table displays all metadata directly associated to this object as RDF triples.
341 TRIPLES
21 PREDICATES
62 URIs
29 LITERALS
17 BLANK NODES