Purposive behavior acquisition for a real robot by vision-based reinforcement learning View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

1996-05

AUTHORS

Minoru Asada, Shoichi Noda, Sukoya Tawaratsumida, Koh Hosoda

ABSTRACT

This paper presents a method of vision-based reinforcement learning by which a robot learns to shoot a ball into a goal. We discuss several issues in applying the reinforcement learning method to a real robot with vision sensor by which the robot can obtain information about the changes in an environment. First, we construct a state space in terms of size, position, and orientation of a ball and a goal in an image, and an action space is designed in terms of the action commands to be sent to the left and right motors of a mobile robot. This causes a “state-action deviation” problem in constructing the state and action spaces that reflect the outputs from physical sensors and actuators, respectively. To deal with this issue, an action set is constructed in a way that one action consists of a series of the same action primitive which is successively executed until the current state changes. Next, to speed up the learning time, a mechanism of Learning from Easy Missions (or LEM) is implemented. LEM reduces the learning time from exponential to almost linear order in the size of the state space. The results of computer simulations and real robot experiments are given. More... »

PAGES

279-303

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/bf00117447

DOI

http://dx.doi.org/10.1007/bf00117447

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1016428696


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Osaka University", 
          "id": "https://www.grid.ac/institutes/grid.136593.b", 
          "name": [
            "Dept. of Mech. Eng. for Computer-Controlled Machinery, Osaka University, 2-1, Yamadaoka, 565, Suita, Osaka, Japan"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Asada", 
        "givenName": "Minoru", 
        "id": "sg:person.010006623261.48", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010006623261.48"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Osaka University", 
          "id": "https://www.grid.ac/institutes/grid.136593.b", 
          "name": [
            "Dept. of Mech. Eng. for Computer-Controlled Machinery, Osaka University, 2-1, Yamadaoka, 565, Suita, Osaka, Japan"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Noda", 
        "givenName": "Shoichi", 
        "id": "sg:person.012211726732.50", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012211726732.50"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Osaka University", 
          "id": "https://www.grid.ac/institutes/grid.136593.b", 
          "name": [
            "Dept. of Mech. Eng. for Computer-Controlled Machinery, Osaka University, 2-1, Yamadaoka, 565, Suita, Osaka, Japan"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Tawaratsumida", 
        "givenName": "Sukoya", 
        "id": "sg:person.013604667732.00", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013604667732.00"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Osaka University", 
          "id": "https://www.grid.ac/institutes/grid.136593.b", 
          "name": [
            "Dept. of Mech. Eng. for Computer-Controlled Machinery, Osaka University, 2-1, Yamadaoka, 565, Suita, Osaka, Japan"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Hosoda", 
        "givenName": "Koh", 
        "id": "sg:person.010522647201.89", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010522647201.89"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "sg:pub.10.1007/bf00992699", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1001020186", 
          "https://doi.org/10.1007/bf00992699"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0004-3702(92)90058-6", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1009445883"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0004-3702(92)90058-6", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1009445883"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-1-4615-3184-5", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1015328718", 
          "https://doi.org/10.1007/978-1-4615-3184-5"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-1-4615-3184-5", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1015328718", 
          "https://doi.org/10.1007/978-1-4615-3184-5"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/b978-1-55860-141-3.50025-0", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1019155303"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-1-4615-3184-5_2", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1035522592", 
          "https://doi.org/10.1007/978-1-4615-3184-5_2"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-1-4615-3184-5_5", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1038118425", 
          "https://doi.org/10.1007/978-1-4615-3184-5_5"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/b978-1-55860-335-6.50030-1", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1043356557"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/robot.1994.351015", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1094077747"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/robot.1994.351013", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1094453125"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "1996-05", 
    "datePublishedReg": "1996-05-01", 
    "description": "This paper presents a method of vision-based reinforcement learning by which a robot learns to shoot a ball into a goal. We discuss several issues in applying the reinforcement learning method to a real robot with vision sensor by which the robot can obtain information about the changes in an environment. First, we construct a state space in terms of size, position, and orientation of a ball and a goal in an image, and an action space is designed in terms of the action commands to be sent to the left and right motors of a mobile robot. This causes a \u201cstate-action deviation\u201d problem in constructing the state and action spaces that reflect the outputs from physical sensors and actuators, respectively. To deal with this issue, an action set is constructed in a way that one action consists of a series of the same action primitive which is successively executed until the current state changes. Next, to speed up the learning time, a mechanism of Learning from Easy Missions (or LEM) is implemented. LEM reduces the learning time from exponential to almost linear order in the size of the state space. The results of computer simulations and real robot experiments are given.", 
    "genre": "research_article", 
    "id": "sg:pub.10.1007/bf00117447", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": true, 
    "isPartOf": [
      {
        "id": "sg:journal.1125588", 
        "issn": [
          "0885-6125", 
          "1573-0565"
        ], 
        "name": "Machine Learning", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "2-3", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "23"
      }
    ], 
    "name": "Purposive behavior acquisition for a real robot by vision-based reinforcement learning", 
    "pagination": "279-303", 
    "productId": [
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/bf00117447"
        ]
      }, 
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "4e46c7498d60c903c7636761a5606600a4e78fbeb2b2663b554340689b96287f"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1016428696"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1007/bf00117447", 
      "https://app.dimensions.ai/details/publication/pub.1016428696"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2019-04-15T08:52", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000374_0000000374/records_119747_00000000.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "http://link.springer.com/10.1007/BF00117447"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/bf00117447'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/bf00117447'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/bf00117447'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/bf00117447'


 

This table displays all metadata directly associated to this object as RDF triples.

113 TRIPLES      21 PREDICATES      36 URIs      19 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/bf00117447 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Nd722262efa7a420a9f5fed8c9938413c
4 schema:citation sg:pub.10.1007/978-1-4615-3184-5
5 sg:pub.10.1007/978-1-4615-3184-5_2
6 sg:pub.10.1007/978-1-4615-3184-5_5
7 sg:pub.10.1007/bf00992699
8 https://doi.org/10.1016/0004-3702(92)90058-6
9 https://doi.org/10.1016/b978-1-55860-141-3.50025-0
10 https://doi.org/10.1016/b978-1-55860-335-6.50030-1
11 https://doi.org/10.1109/robot.1994.351013
12 https://doi.org/10.1109/robot.1994.351015
13 schema:datePublished 1996-05
14 schema:datePublishedReg 1996-05-01
15 schema:description This paper presents a method of vision-based reinforcement learning by which a robot learns to shoot a ball into a goal. We discuss several issues in applying the reinforcement learning method to a real robot with vision sensor by which the robot can obtain information about the changes in an environment. First, we construct a state space in terms of size, position, and orientation of a ball and a goal in an image, and an action space is designed in terms of the action commands to be sent to the left and right motors of a mobile robot. This causes a “state-action deviation” problem in constructing the state and action spaces that reflect the outputs from physical sensors and actuators, respectively. To deal with this issue, an action set is constructed in a way that one action consists of a series of the same action primitive which is successively executed until the current state changes. Next, to speed up the learning time, a mechanism of Learning from Easy Missions (or LEM) is implemented. LEM reduces the learning time from exponential to almost linear order in the size of the state space. The results of computer simulations and real robot experiments are given.
16 schema:genre research_article
17 schema:inLanguage en
18 schema:isAccessibleForFree true
19 schema:isPartOf N0a52841841a7471c82b7538392e7aab4
20 N9e00f68b825647d08e8d589a888b97b8
21 sg:journal.1125588
22 schema:name Purposive behavior acquisition for a real robot by vision-based reinforcement learning
23 schema:pagination 279-303
24 schema:productId N292f6c38923c447d94434964f90cdb63
25 N381fdf4d52904517a90c2a2ab634edae
26 Nbfb738462feb49a3a704129779edd80e
27 schema:sameAs https://app.dimensions.ai/details/publication/pub.1016428696
28 https://doi.org/10.1007/bf00117447
29 schema:sdDatePublished 2019-04-15T08:52
30 schema:sdLicense https://scigraph.springernature.com/explorer/license/
31 schema:sdPublisher N669c7ecb598f421bbed4d51db6d2db0a
32 schema:url http://link.springer.com/10.1007/BF00117447
33 sgo:license sg:explorer/license/
34 sgo:sdDataset articles
35 rdf:type schema:ScholarlyArticle
36 N05ce43efa4ab4e5ba897f26f24ea8001 rdf:first sg:person.012211726732.50
37 rdf:rest Nafa6e85a27de42c8982b212d04b4517b
38 N0a52841841a7471c82b7538392e7aab4 schema:volumeNumber 23
39 rdf:type schema:PublicationVolume
40 N292f6c38923c447d94434964f90cdb63 schema:name doi
41 schema:value 10.1007/bf00117447
42 rdf:type schema:PropertyValue
43 N381fdf4d52904517a90c2a2ab634edae schema:name dimensions_id
44 schema:value pub.1016428696
45 rdf:type schema:PropertyValue
46 N38d13c05cb184991b7165757bbc1e370 rdf:first sg:person.010522647201.89
47 rdf:rest rdf:nil
48 N669c7ecb598f421bbed4d51db6d2db0a schema:name Springer Nature - SN SciGraph project
49 rdf:type schema:Organization
50 N9e00f68b825647d08e8d589a888b97b8 schema:issueNumber 2-3
51 rdf:type schema:PublicationIssue
52 Nafa6e85a27de42c8982b212d04b4517b rdf:first sg:person.013604667732.00
53 rdf:rest N38d13c05cb184991b7165757bbc1e370
54 Nbfb738462feb49a3a704129779edd80e schema:name readcube_id
55 schema:value 4e46c7498d60c903c7636761a5606600a4e78fbeb2b2663b554340689b96287f
56 rdf:type schema:PropertyValue
57 Nd722262efa7a420a9f5fed8c9938413c rdf:first sg:person.010006623261.48
58 rdf:rest N05ce43efa4ab4e5ba897f26f24ea8001
59 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
60 schema:name Information and Computing Sciences
61 rdf:type schema:DefinedTerm
62 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
63 schema:name Artificial Intelligence and Image Processing
64 rdf:type schema:DefinedTerm
65 sg:journal.1125588 schema:issn 0885-6125
66 1573-0565
67 schema:name Machine Learning
68 rdf:type schema:Periodical
69 sg:person.010006623261.48 schema:affiliation https://www.grid.ac/institutes/grid.136593.b
70 schema:familyName Asada
71 schema:givenName Minoru
72 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010006623261.48
73 rdf:type schema:Person
74 sg:person.010522647201.89 schema:affiliation https://www.grid.ac/institutes/grid.136593.b
75 schema:familyName Hosoda
76 schema:givenName Koh
77 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010522647201.89
78 rdf:type schema:Person
79 sg:person.012211726732.50 schema:affiliation https://www.grid.ac/institutes/grid.136593.b
80 schema:familyName Noda
81 schema:givenName Shoichi
82 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012211726732.50
83 rdf:type schema:Person
84 sg:person.013604667732.00 schema:affiliation https://www.grid.ac/institutes/grid.136593.b
85 schema:familyName Tawaratsumida
86 schema:givenName Sukoya
87 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013604667732.00
88 rdf:type schema:Person
89 sg:pub.10.1007/978-1-4615-3184-5 schema:sameAs https://app.dimensions.ai/details/publication/pub.1015328718
90 https://doi.org/10.1007/978-1-4615-3184-5
91 rdf:type schema:CreativeWork
92 sg:pub.10.1007/978-1-4615-3184-5_2 schema:sameAs https://app.dimensions.ai/details/publication/pub.1035522592
93 https://doi.org/10.1007/978-1-4615-3184-5_2
94 rdf:type schema:CreativeWork
95 sg:pub.10.1007/978-1-4615-3184-5_5 schema:sameAs https://app.dimensions.ai/details/publication/pub.1038118425
96 https://doi.org/10.1007/978-1-4615-3184-5_5
97 rdf:type schema:CreativeWork
98 sg:pub.10.1007/bf00992699 schema:sameAs https://app.dimensions.ai/details/publication/pub.1001020186
99 https://doi.org/10.1007/bf00992699
100 rdf:type schema:CreativeWork
101 https://doi.org/10.1016/0004-3702(92)90058-6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1009445883
102 rdf:type schema:CreativeWork
103 https://doi.org/10.1016/b978-1-55860-141-3.50025-0 schema:sameAs https://app.dimensions.ai/details/publication/pub.1019155303
104 rdf:type schema:CreativeWork
105 https://doi.org/10.1016/b978-1-55860-335-6.50030-1 schema:sameAs https://app.dimensions.ai/details/publication/pub.1043356557
106 rdf:type schema:CreativeWork
107 https://doi.org/10.1109/robot.1994.351013 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094453125
108 rdf:type schema:CreativeWork
109 https://doi.org/10.1109/robot.1994.351015 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094077747
110 rdf:type schema:CreativeWork
111 https://www.grid.ac/institutes/grid.136593.b schema:alternateName Osaka University
112 schema:name Dept. of Mech. Eng. for Computer-Controlled Machinery, Osaka University, 2-1, Yamadaoka, 565, Suita, Osaka, Japan
113 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...