Grounding Neural Robot Language in Action View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2005

AUTHORS

Stefan Wermter , Cornelius Weber , Mark Elshaw , Vittorio Gallese , Friedemann Pulvermüller

ABSTRACT

In this paper we describe two models for neural grounding of robotic language processing in actions. These models are inspired by concepts of the mirror neuron system in order to produce learning by imitation by combining high-level vision, language and motor command inputs. The models learn to perform and recognise three behaviours, ‘go’, ‘pick’ and ‘lift’. The first single-layer model uses an adapted Helmholtz machine wake-sleep algorithm to act like a Kohonen self-organising network that receives all inputs into a single layer. In contrast, the second, hierarchical model has two layers. In the lower level hidden layer the Helmholtz machine wake-sleep algorithm is used to learn the relationship between action and vision, while the upper layer uses the Kohonen self-organising approach to combine the output of the lower hidden layer and the language input.On the hidden layer of the single-layer model, the action words are represented on non-overlapping regions and any neuron in each region accounts for a corresponding sensory-motor binding. In the hierarchical model rather separate sensory- and motor representations on the lower level are bound to corresponding sensory-motor pairings via the top level that organises according to the language input. More... »

PAGES

162-181

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/11521082_10

DOI

http://dx.doi.org/10.1007/11521082_10

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1041057060


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/17", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Psychology and Cognitive Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1701", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Psychology", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1702", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Cognitive Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Hybrid Intelligent Systems, School of Computing and Technology, University of Sunderland, St Peter\u2019s Way, SR6 0DD, Sunderland, UK", 
          "id": "http://www.grid.ac/institutes/grid.7110.7", 
          "name": [
            "Hybrid Intelligent Systems, School of Computing and Technology, University of Sunderland, St Peter\u2019s Way, SR6 0DD, Sunderland, UK"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Wermter", 
        "givenName": "Stefan", 
        "id": "sg:person.01162357206.03", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01162357206.03"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Hybrid Intelligent Systems, School of Computing and Technology, University of Sunderland, St Peter\u2019s Way, SR6 0DD, Sunderland, UK", 
          "id": "http://www.grid.ac/institutes/grid.7110.7", 
          "name": [
            "Hybrid Intelligent Systems, School of Computing and Technology, University of Sunderland, St Peter\u2019s Way, SR6 0DD, Sunderland, UK"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Weber", 
        "givenName": "Cornelius", 
        "id": "sg:person.016643273751.54", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016643273751.54"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Hybrid Intelligent Systems, School of Computing and Technology, University of Sunderland, St Peter\u2019s Way, SR6 0DD, Sunderland, UK", 
          "id": "http://www.grid.ac/institutes/grid.7110.7", 
          "name": [
            "Hybrid Intelligent Systems, School of Computing and Technology, University of Sunderland, St Peter\u2019s Way, SR6 0DD, Sunderland, UK"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Elshaw", 
        "givenName": "Mark", 
        "id": "sg:person.01230472406.05", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01230472406.05"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Institute of Neurophysiology, University of Parma, Via Volturno, 39/E, I-43100, Parma, Italy", 
          "id": "http://www.grid.ac/institutes/grid.10383.39", 
          "name": [
            "Institute of Neurophysiology, University of Parma, Via Volturno, 39/E, I-43100, Parma, Italy"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Gallese", 
        "givenName": "Vittorio", 
        "id": "sg:person.01372640110.48", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01372640110.48"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Cognition and Brain Sciences Unit, Medical Research Council, 15 Chaucer Road, Cambridge, UK", 
          "id": "http://www.grid.ac/institutes/grid.415036.5", 
          "name": [
            "Cognition and Brain Sciences Unit, Medical Research Council, 15 Chaucer Road, Cambridge, UK"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Pulverm\u00fcller", 
        "givenName": "Friedemann", 
        "id": "sg:person.01214737170.14", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01214737170.14"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2005", 
    "datePublishedReg": "2005-01-01", 
    "description": "In this paper we describe two models for neural grounding of robotic language processing in actions. These models are inspired by concepts of the mirror neuron system in order to produce learning by imitation by combining high-level vision, language and motor command inputs. The models learn to perform and recognise three behaviours, \u2018go\u2019, \u2018pick\u2019 and \u2018lift\u2019. The first single-layer model uses an adapted Helmholtz machine wake-sleep algorithm to act like a Kohonen self-organising network that receives all inputs into a single layer. In contrast, the second, hierarchical model has two layers. In the lower level hidden layer the Helmholtz machine wake-sleep algorithm is used to learn the relationship between action and vision, while the upper layer uses the Kohonen self-organising approach to combine the output of the lower hidden layer and the language input.On the hidden layer of the single-layer model, the action words are represented on non-overlapping regions and any neuron in each region accounts for a corresponding sensory-motor binding. In the hierarchical model rather separate sensory- and motor representations on the lower level are bound to corresponding sensory-motor pairings via the top level that organises according to the language input.", 
    "editor": [
      {
        "familyName": "Wermter", 
        "givenName": "Stefan", 
        "type": "Person"
      }, 
      {
        "familyName": "Palm", 
        "givenName": "G\u00fcnther", 
        "type": "Person"
      }, 
      {
        "familyName": "Elshaw", 
        "givenName": "Mark", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/11521082_10", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-540-27440-7", 
        "978-3-540-31896-5"
      ], 
      "name": "Biomimetic Neural Learning for Intelligent Robots", 
      "type": "Book"
    }, 
    "keywords": [
      "wake-sleep algorithm", 
      "language input", 
      "mirror neuron system", 
      "high-level vision", 
      "action words", 
      "motor representations", 
      "neuron system", 
      "language processing", 
      "hierarchical model", 
      "self-organising approach", 
      "self-organising network", 
      "language", 
      "hidden layer", 
      "imitation", 
      "words", 
      "vision", 
      "low levels", 
      "robot language", 
      "sensory", 
      "processing", 
      "action", 
      "non-overlapping regions", 
      "representation", 
      "grounding", 
      "relationship", 
      "behavior", 
      "command input", 
      "pairing", 
      "input", 
      "model", 
      "algorithm", 
      "top level", 
      "levels", 
      "concept", 
      "network", 
      "contrast", 
      "approach", 
      "single-layer model", 
      "system", 
      "upper layer", 
      "neurons", 
      "region", 
      "output", 
      "order", 
      "paper", 
      "layer", 
      "single layer", 
      "binding"
    ], 
    "name": "Grounding Neural Robot Language in Action", 
    "pagination": "162-181", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1041057060"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/11521082_10"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/11521082_10", 
      "https://app.dimensions.ai/details/publication/pub.1041057060"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-11-24T21:16", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20221124/entities/gbq_results/chapter/chapter_344.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/11521082_10"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/11521082_10'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/11521082_10'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/11521082_10'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/11521082_10'


 

This table displays all metadata directly associated to this object as RDF triples.

163 TRIPLES      22 PREDICATES      76 URIs      66 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/11521082_10 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 anzsrc-for:17
4 anzsrc-for:1701
5 anzsrc-for:1702
6 schema:author N2ba6d1d9c3e34622b2b84e7dacdb2f31
7 schema:datePublished 2005
8 schema:datePublishedReg 2005-01-01
9 schema:description In this paper we describe two models for neural grounding of robotic language processing in actions. These models are inspired by concepts of the mirror neuron system in order to produce learning by imitation by combining high-level vision, language and motor command inputs. The models learn to perform and recognise three behaviours, ‘go’, ‘pick’ and ‘lift’. The first single-layer model uses an adapted Helmholtz machine wake-sleep algorithm to act like a Kohonen self-organising network that receives all inputs into a single layer. In contrast, the second, hierarchical model has two layers. In the lower level hidden layer the Helmholtz machine wake-sleep algorithm is used to learn the relationship between action and vision, while the upper layer uses the Kohonen self-organising approach to combine the output of the lower hidden layer and the language input.On the hidden layer of the single-layer model, the action words are represented on non-overlapping regions and any neuron in each region accounts for a corresponding sensory-motor binding. In the hierarchical model rather separate sensory- and motor representations on the lower level are bound to corresponding sensory-motor pairings via the top level that organises according to the language input.
10 schema:editor Ne19d8dd6a4a046a08d4d96a20eed62ed
11 schema:genre chapter
12 schema:isAccessibleForFree false
13 schema:isPartOf N5265a24991a24083827f1b78b0ce253e
14 schema:keywords action
15 action words
16 algorithm
17 approach
18 behavior
19 binding
20 command input
21 concept
22 contrast
23 grounding
24 hidden layer
25 hierarchical model
26 high-level vision
27 imitation
28 input
29 language
30 language input
31 language processing
32 layer
33 levels
34 low levels
35 mirror neuron system
36 model
37 motor representations
38 network
39 neuron system
40 neurons
41 non-overlapping regions
42 order
43 output
44 pairing
45 paper
46 processing
47 region
48 relationship
49 representation
50 robot language
51 self-organising approach
52 self-organising network
53 sensory
54 single layer
55 single-layer model
56 system
57 top level
58 upper layer
59 vision
60 wake-sleep algorithm
61 words
62 schema:name Grounding Neural Robot Language in Action
63 schema:pagination 162-181
64 schema:productId N46fbe67f949c41c4a174364f79717341
65 Nbd5a7f2b54314814ae4fe08a37f8e506
66 schema:publisher N8e801a4b04404923baa797ae4f3d0d35
67 schema:sameAs https://app.dimensions.ai/details/publication/pub.1041057060
68 https://doi.org/10.1007/11521082_10
69 schema:sdDatePublished 2022-11-24T21:16
70 schema:sdLicense https://scigraph.springernature.com/explorer/license/
71 schema:sdPublisher N097857322ea04195a510b61a8a49aca2
72 schema:url https://doi.org/10.1007/11521082_10
73 sgo:license sg:explorer/license/
74 sgo:sdDataset chapters
75 rdf:type schema:Chapter
76 N097857322ea04195a510b61a8a49aca2 schema:name Springer Nature - SN SciGraph project
77 rdf:type schema:Organization
78 N12cc1a4c577a4025896ee9991a400a57 rdf:first N72b697b4bc6d44d99be4b643183a5e65
79 rdf:rest rdf:nil
80 N2ba6d1d9c3e34622b2b84e7dacdb2f31 rdf:first sg:person.01162357206.03
81 rdf:rest Nbe9aa625f83b4e4aae078cc0a11f3c6f
82 N3a3e6300ab924b8cbc48036881d1bbf1 schema:familyName Wermter
83 schema:givenName Stefan
84 rdf:type schema:Person
85 N46fbe67f949c41c4a174364f79717341 schema:name doi
86 schema:value 10.1007/11521082_10
87 rdf:type schema:PropertyValue
88 N5265a24991a24083827f1b78b0ce253e schema:isbn 978-3-540-27440-7
89 978-3-540-31896-5
90 schema:name Biomimetic Neural Learning for Intelligent Robots
91 rdf:type schema:Book
92 N620fbec5104c437cbe5151313654d78d rdf:first N7de4a25d75f24804a280991a97a83a56
93 rdf:rest N12cc1a4c577a4025896ee9991a400a57
94 N72b697b4bc6d44d99be4b643183a5e65 schema:familyName Elshaw
95 schema:givenName Mark
96 rdf:type schema:Person
97 N7de4a25d75f24804a280991a97a83a56 schema:familyName Palm
98 schema:givenName Günther
99 rdf:type schema:Person
100 N8e801a4b04404923baa797ae4f3d0d35 schema:name Springer Nature
101 rdf:type schema:Organisation
102 N936e00fe669d40d199cc6a6c8e0edf71 rdf:first sg:person.01214737170.14
103 rdf:rest rdf:nil
104 N9ce5c89c827f42e1aa37649dbf29f458 rdf:first sg:person.01230472406.05
105 rdf:rest Na13e1c94b6db43fbb6df80f3eb08cd3e
106 Na13e1c94b6db43fbb6df80f3eb08cd3e rdf:first sg:person.01372640110.48
107 rdf:rest N936e00fe669d40d199cc6a6c8e0edf71
108 Nbd5a7f2b54314814ae4fe08a37f8e506 schema:name dimensions_id
109 schema:value pub.1041057060
110 rdf:type schema:PropertyValue
111 Nbe9aa625f83b4e4aae078cc0a11f3c6f rdf:first sg:person.016643273751.54
112 rdf:rest N9ce5c89c827f42e1aa37649dbf29f458
113 Ne19d8dd6a4a046a08d4d96a20eed62ed rdf:first N3a3e6300ab924b8cbc48036881d1bbf1
114 rdf:rest N620fbec5104c437cbe5151313654d78d
115 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
116 schema:name Information and Computing Sciences
117 rdf:type schema:DefinedTerm
118 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
119 schema:name Artificial Intelligence and Image Processing
120 rdf:type schema:DefinedTerm
121 anzsrc-for:17 schema:inDefinedTermSet anzsrc-for:
122 schema:name Psychology and Cognitive Sciences
123 rdf:type schema:DefinedTerm
124 anzsrc-for:1701 schema:inDefinedTermSet anzsrc-for:
125 schema:name Psychology
126 rdf:type schema:DefinedTerm
127 anzsrc-for:1702 schema:inDefinedTermSet anzsrc-for:
128 schema:name Cognitive Sciences
129 rdf:type schema:DefinedTerm
130 sg:person.01162357206.03 schema:affiliation grid-institutes:grid.7110.7
131 schema:familyName Wermter
132 schema:givenName Stefan
133 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01162357206.03
134 rdf:type schema:Person
135 sg:person.01214737170.14 schema:affiliation grid-institutes:grid.415036.5
136 schema:familyName Pulvermüller
137 schema:givenName Friedemann
138 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01214737170.14
139 rdf:type schema:Person
140 sg:person.01230472406.05 schema:affiliation grid-institutes:grid.7110.7
141 schema:familyName Elshaw
142 schema:givenName Mark
143 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01230472406.05
144 rdf:type schema:Person
145 sg:person.01372640110.48 schema:affiliation grid-institutes:grid.10383.39
146 schema:familyName Gallese
147 schema:givenName Vittorio
148 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01372640110.48
149 rdf:type schema:Person
150 sg:person.016643273751.54 schema:affiliation grid-institutes:grid.7110.7
151 schema:familyName Weber
152 schema:givenName Cornelius
153 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016643273751.54
154 rdf:type schema:Person
155 grid-institutes:grid.10383.39 schema:alternateName Institute of Neurophysiology, University of Parma, Via Volturno, 39/E, I-43100, Parma, Italy
156 schema:name Institute of Neurophysiology, University of Parma, Via Volturno, 39/E, I-43100, Parma, Italy
157 rdf:type schema:Organization
158 grid-institutes:grid.415036.5 schema:alternateName Cognition and Brain Sciences Unit, Medical Research Council, 15 Chaucer Road, Cambridge, UK
159 schema:name Cognition and Brain Sciences Unit, Medical Research Council, 15 Chaucer Road, Cambridge, UK
160 rdf:type schema:Organization
161 grid-institutes:grid.7110.7 schema:alternateName Hybrid Intelligent Systems, School of Computing and Technology, University of Sunderland, St Peter’s Way, SR6 0DD, Sunderland, UK
162 schema:name Hybrid Intelligent Systems, School of Computing and Technology, University of Sunderland, St Peter’s Way, SR6 0DD, Sunderland, UK
163 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...