Grasping Time and Pose Selection for Robotic Prosthetic Hand Control Using Deep Learning Based Object Detection View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2022-08-27

AUTHORS

Hae-June Park, Bo-Hyeon An, Su-Bin Joo, Oh-Won Kwon, Min Young Kim, Joonho Seo

ABSTRACT

This paper presents an algorithm to control a robotic prosthetic hand by applying deep learning (DL) to select a grasping pose and a grasping time from 2D images and 3D point clouds. This algorithm consists of four steps: 1) Acquisition of 2D images and 3D point clouds of objects; 2) Object recognition in the 2D images; 3) Grasping pose selection; 4) Choice of a grasping time and control of the prosthetic hand. The grasping pose selection is necessary when the algorithm detects many objects in the same frame, and must decide which pose of the prosthetic hand should use. The pose was chosen considering the object that was to the prosthesis. The grasping time was determined by the operating point when approaching the selected target after selecting the grasping pose; this choice uses an empirically-determined distance threshold. The proposed method achieved 89% accuracy of the grasping the intended object. The failures occurred because of slight inaccuracy in object localization, occlusion of target objects, and the inability of DL object detection. Work to solve these shortcomings is ongoing. This algorithm will help to improve the convenience of the user of a prosthetic hand. More... »

PAGES

3410-3417

References to SciGraph publications

  • 2011. Control of Powered Prosthetic Hand Using Multidimensional Ultrasound Signals: A Pilot Study in UNIVERSAL ACCESS IN HUMAN-COMPUTER INTERACTION. APPLICATIONS AND SERVICES
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s12555-021-0449-6

    DOI

    http://dx.doi.org/10.1007/s12555-021-0449-6

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1150553852


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/09", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Engineering", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0906", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Electrical and Electronic Engineering", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0910", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Manufacturing Engineering", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0913", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Mechanical Engineering", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "School of Electronics Engineering, Kyungpook National University, 80 Daehakro, Buk-gu, 41566, Daegu, Korea", 
              "id": "http://www.grid.ac/institutes/grid.258803.4", 
              "name": [
                "Assistive Medical Robot Lab., Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea", 
                "School of Electronics Engineering, Kyungpook National University, 80 Daehakro, Buk-gu, 41566, Daegu, Korea"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Park", 
            "givenName": "Hae-June", 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Dept. of Medical Robotics, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea", 
              "id": "http://www.grid.ac/institutes/grid.410901.d", 
              "name": [
                "Dept. of Medical Robotics, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea"
              ], 
              "type": "Organization"
            }, 
            "familyName": "An", 
            "givenName": "Bo-Hyeon", 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Dept. of Medical Robotics, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea", 
              "id": "http://www.grid.ac/institutes/grid.410901.d", 
              "name": [
                "Dept. of Medical Robotics, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Joo", 
            "givenName": "Su-Bin", 
            "id": "sg:person.01217071472.87", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01217071472.87"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Dept. of Medical Device, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea", 
              "id": "http://www.grid.ac/institutes/grid.410901.d", 
              "name": [
                "Dept. of Medical Device, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Kwon", 
            "givenName": "Oh-Won", 
            "id": "sg:person.0716377314.97", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0716377314.97"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Electronics Engineering and Research Center for Neurosurgical Robotic System, Kyungpook National University, 80 Daehakro, Buk-gu, 41566, Daegu, Korea", 
              "id": "http://www.grid.ac/institutes/grid.258803.4", 
              "name": [
                "School of Electronics Engineering and Research Center for Neurosurgical Robotic System, Kyungpook National University, 80 Daehakro, Buk-gu, 41566, Daegu, Korea"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Kim", 
            "givenName": "Min Young", 
            "id": "sg:person.01325532210.05", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01325532210.05"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Dept. of Medical Robotics, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea", 
              "id": "http://www.grid.ac/institutes/grid.410901.d", 
              "name": [
                "Dept. of Medical Robotics, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Seo", 
            "givenName": "Joonho", 
            "id": "sg:person.014617557663.57", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014617557663.57"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/978-3-642-21657-2_34", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1018360517", 
              "https://doi.org/10.1007/978-3-642-21657-2_34"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2022-08-27", 
        "datePublishedReg": "2022-08-27", 
        "description": "This paper presents an algorithm to control a robotic prosthetic hand by applying deep learning (DL) to select a grasping pose and a grasping time from 2D images and 3D point clouds. This algorithm consists of four steps: 1) Acquisition of 2D images and 3D point clouds of objects; 2) Object recognition in the 2D images; 3) Grasping pose selection; 4) Choice of a grasping time and control of the prosthetic hand. The grasping pose selection is necessary when the algorithm detects many objects in the same frame, and must decide which pose of the prosthetic hand should use. The pose was chosen considering the object that was to the prosthesis. The grasping time was determined by the operating point when approaching the selected target after selecting the grasping pose; this choice uses an empirically-determined distance threshold. The proposed method achieved 89% accuracy of the grasping the intended object. The failures occurred because of slight inaccuracy in object localization, occlusion of target objects, and the inability of DL object detection. Work to solve these shortcomings is ongoing. This algorithm will help to improve the convenience of the user of a prosthetic hand.", 
        "genre": "article", 
        "id": "sg:pub.10.1007/s12555-021-0449-6", 
        "isAccessibleForFree": false, 
        "isPartOf": [
          {
            "id": "sg:journal.1041911", 
            "issn": [
              "1598-6446", 
              "2005-4092"
            ], 
            "name": "International Journal of Control, Automation and Systems", 
            "publisher": "Springer Nature", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "10", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "20"
          }
        ], 
        "keywords": [
          "deep learning", 
          "pose selection", 
          "object detection", 
          "point clouds", 
          "object localization", 
          "prosthetic hand", 
          "object recognition", 
          "target object", 
          "robotic prosthetic hand", 
          "pose", 
          "prosthetic hand control", 
          "algorithm", 
          "distance threshold", 
          "slight inaccuracy", 
          "hand control", 
          "same frame", 
          "objects", 
          "cloud", 
          "images", 
          "learning", 
          "users", 
          "detection", 
          "selection", 
          "recognition", 
          "accuracy", 
          "operating point", 
          "convenience", 
          "shortcomings", 
          "frame", 
          "hand", 
          "time", 
          "acquisition", 
          "inaccuracy", 
          "step", 
          "method", 
          "choice", 
          "control", 
          "localization", 
          "point", 
          "occlusion", 
          "threshold", 
          "target", 
          "failure", 
          "inability", 
          "prosthesis", 
          "paper"
        ], 
        "name": "Grasping Time and Pose Selection for Robotic Prosthetic Hand Control Using Deep Learning Based Object Detection", 
        "pagination": "3410-3417", 
        "productId": [
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1150553852"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s12555-021-0449-6"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s12555-021-0449-6", 
          "https://app.dimensions.ai/details/publication/pub.1150553852"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2022-12-01T06:44", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-springernature-scigraph/baseset/20221201/entities/gbq_results/article/article_940.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://doi.org/10.1007/s12555-021-0449-6"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s12555-021-0449-6'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s12555-021-0449-6'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s12555-021-0449-6'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s12555-021-0449-6'


     

    This table displays all metadata directly associated to this object as RDF triples.

    156 TRIPLES      21 PREDICATES      73 URIs      62 LITERALS      6 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s12555-021-0449-6 schema:about anzsrc-for:09
    2 anzsrc-for:0906
    3 anzsrc-for:0910
    4 anzsrc-for:0913
    5 schema:author Nb74c6e5f48ac4e8bb46597d0bc72d6d2
    6 schema:citation sg:pub.10.1007/978-3-642-21657-2_34
    7 schema:datePublished 2022-08-27
    8 schema:datePublishedReg 2022-08-27
    9 schema:description This paper presents an algorithm to control a robotic prosthetic hand by applying deep learning (DL) to select a grasping pose and a grasping time from 2D images and 3D point clouds. This algorithm consists of four steps: 1) Acquisition of 2D images and 3D point clouds of objects; 2) Object recognition in the 2D images; 3) Grasping pose selection; 4) Choice of a grasping time and control of the prosthetic hand. The grasping pose selection is necessary when the algorithm detects many objects in the same frame, and must decide which pose of the prosthetic hand should use. The pose was chosen considering the object that was to the prosthesis. The grasping time was determined by the operating point when approaching the selected target after selecting the grasping pose; this choice uses an empirically-determined distance threshold. The proposed method achieved 89% accuracy of the grasping the intended object. The failures occurred because of slight inaccuracy in object localization, occlusion of target objects, and the inability of DL object detection. Work to solve these shortcomings is ongoing. This algorithm will help to improve the convenience of the user of a prosthetic hand.
    10 schema:genre article
    11 schema:isAccessibleForFree false
    12 schema:isPartOf N250c4cb06db64787a6364ee1e3542b3b
    13 N960bd00ee6774dbd8d1a1d39de357409
    14 sg:journal.1041911
    15 schema:keywords accuracy
    16 acquisition
    17 algorithm
    18 choice
    19 cloud
    20 control
    21 convenience
    22 deep learning
    23 detection
    24 distance threshold
    25 failure
    26 frame
    27 hand
    28 hand control
    29 images
    30 inability
    31 inaccuracy
    32 learning
    33 localization
    34 method
    35 object detection
    36 object localization
    37 object recognition
    38 objects
    39 occlusion
    40 operating point
    41 paper
    42 point
    43 point clouds
    44 pose
    45 pose selection
    46 prosthesis
    47 prosthetic hand
    48 prosthetic hand control
    49 recognition
    50 robotic prosthetic hand
    51 same frame
    52 selection
    53 shortcomings
    54 slight inaccuracy
    55 step
    56 target
    57 target object
    58 threshold
    59 time
    60 users
    61 schema:name Grasping Time and Pose Selection for Robotic Prosthetic Hand Control Using Deep Learning Based Object Detection
    62 schema:pagination 3410-3417
    63 schema:productId N7dda4f3296264eb68575502868cda07a
    64 N94dc58463cb04944a974e18d026fc467
    65 schema:sameAs https://app.dimensions.ai/details/publication/pub.1150553852
    66 https://doi.org/10.1007/s12555-021-0449-6
    67 schema:sdDatePublished 2022-12-01T06:44
    68 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    69 schema:sdPublisher N7d2b3cc2210c446e9a9a2e75056ccd4b
    70 schema:url https://doi.org/10.1007/s12555-021-0449-6
    71 sgo:license sg:explorer/license/
    72 sgo:sdDataset articles
    73 rdf:type schema:ScholarlyArticle
    74 N042c9db176204a96a0a9a68c4bd80128 schema:affiliation grid-institutes:grid.258803.4
    75 schema:familyName Park
    76 schema:givenName Hae-June
    77 rdf:type schema:Person
    78 N250c4cb06db64787a6364ee1e3542b3b schema:issueNumber 10
    79 rdf:type schema:PublicationIssue
    80 N3d7119731e6849dfbd123685e29df740 rdf:first sg:person.01217071472.87
    81 rdf:rest Nbab6ceb4e317408b8ccf205d3e6e1b9d
    82 N3e243b7eedb344f2930ea773540ebb11 rdf:first sg:person.01325532210.05
    83 rdf:rest Na85056b599174dceaa7c098aecbc1d1f
    84 N52f4e979df4a425993b464f133352da3 rdf:first N8f04da2768d44630a5929e81b94ae8da
    85 rdf:rest N3d7119731e6849dfbd123685e29df740
    86 N7d2b3cc2210c446e9a9a2e75056ccd4b schema:name Springer Nature - SN SciGraph project
    87 rdf:type schema:Organization
    88 N7dda4f3296264eb68575502868cda07a schema:name dimensions_id
    89 schema:value pub.1150553852
    90 rdf:type schema:PropertyValue
    91 N8f04da2768d44630a5929e81b94ae8da schema:affiliation grid-institutes:grid.410901.d
    92 schema:familyName An
    93 schema:givenName Bo-Hyeon
    94 rdf:type schema:Person
    95 N94dc58463cb04944a974e18d026fc467 schema:name doi
    96 schema:value 10.1007/s12555-021-0449-6
    97 rdf:type schema:PropertyValue
    98 N960bd00ee6774dbd8d1a1d39de357409 schema:volumeNumber 20
    99 rdf:type schema:PublicationVolume
    100 Na85056b599174dceaa7c098aecbc1d1f rdf:first sg:person.014617557663.57
    101 rdf:rest rdf:nil
    102 Nb74c6e5f48ac4e8bb46597d0bc72d6d2 rdf:first N042c9db176204a96a0a9a68c4bd80128
    103 rdf:rest N52f4e979df4a425993b464f133352da3
    104 Nbab6ceb4e317408b8ccf205d3e6e1b9d rdf:first sg:person.0716377314.97
    105 rdf:rest N3e243b7eedb344f2930ea773540ebb11
    106 anzsrc-for:09 schema:inDefinedTermSet anzsrc-for:
    107 schema:name Engineering
    108 rdf:type schema:DefinedTerm
    109 anzsrc-for:0906 schema:inDefinedTermSet anzsrc-for:
    110 schema:name Electrical and Electronic Engineering
    111 rdf:type schema:DefinedTerm
    112 anzsrc-for:0910 schema:inDefinedTermSet anzsrc-for:
    113 schema:name Manufacturing Engineering
    114 rdf:type schema:DefinedTerm
    115 anzsrc-for:0913 schema:inDefinedTermSet anzsrc-for:
    116 schema:name Mechanical Engineering
    117 rdf:type schema:DefinedTerm
    118 sg:journal.1041911 schema:issn 1598-6446
    119 2005-4092
    120 schema:name International Journal of Control, Automation and Systems
    121 schema:publisher Springer Nature
    122 rdf:type schema:Periodical
    123 sg:person.01217071472.87 schema:affiliation grid-institutes:grid.410901.d
    124 schema:familyName Joo
    125 schema:givenName Su-Bin
    126 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01217071472.87
    127 rdf:type schema:Person
    128 sg:person.01325532210.05 schema:affiliation grid-institutes:grid.258803.4
    129 schema:familyName Kim
    130 schema:givenName Min Young
    131 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01325532210.05
    132 rdf:type schema:Person
    133 sg:person.014617557663.57 schema:affiliation grid-institutes:grid.410901.d
    134 schema:familyName Seo
    135 schema:givenName Joonho
    136 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014617557663.57
    137 rdf:type schema:Person
    138 sg:person.0716377314.97 schema:affiliation grid-institutes:grid.410901.d
    139 schema:familyName Kwon
    140 schema:givenName Oh-Won
    141 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0716377314.97
    142 rdf:type schema:Person
    143 sg:pub.10.1007/978-3-642-21657-2_34 schema:sameAs https://app.dimensions.ai/details/publication/pub.1018360517
    144 https://doi.org/10.1007/978-3-642-21657-2_34
    145 rdf:type schema:CreativeWork
    146 grid-institutes:grid.258803.4 schema:alternateName School of Electronics Engineering and Research Center for Neurosurgical Robotic System, Kyungpook National University, 80 Daehakro, Buk-gu, 41566, Daegu, Korea
    147 School of Electronics Engineering, Kyungpook National University, 80 Daehakro, Buk-gu, 41566, Daegu, Korea
    148 schema:name Assistive Medical Robot Lab., Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea
    149 School of Electronics Engineering and Research Center for Neurosurgical Robotic System, Kyungpook National University, 80 Daehakro, Buk-gu, 41566, Daegu, Korea
    150 School of Electronics Engineering, Kyungpook National University, 80 Daehakro, Buk-gu, 41566, Daegu, Korea
    151 rdf:type schema:Organization
    152 grid-institutes:grid.410901.d schema:alternateName Dept. of Medical Device, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea
    153 Dept. of Medical Robotics, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea
    154 schema:name Dept. of Medical Device, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea
    155 Dept. of Medical Robotics, Korea Institute of Machinery and Materials, 330 Techno sunhwan-ro, Dalseong-gun, 42994, Daegu, Korea
    156 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...