Object detection by crossing relational reasoning based on graph neural network View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2021-10-30

AUTHORS

XiuTing You, He Liu, Tao Wang, Songhe Feng, Congyan Lang

ABSTRACT

Utilizing relational representations to facilitate object detection has attracted growing research attention in recent years. However, previous studies mainly focus on relationships within the region proposals or within the label embeddings and pay less attention to the relationships between them. To fill this gap, we propose a novel object detection framework that fully explores the relationships across visual feature space and label embedding space to facilitate the proposal classification in object detection. Specifically, we model the region proposals and class labels into a uniform relation graph, where the extracted proposals and labels are regarded as nodes and each pair of them is associated by an assignment edge, and convert the problem of classifying proposals to the problem of selecting reliable edges from the constructed relation graph. Furthermore, a graph convolutional module is developed to perform relational reasoning on the graph, which finally predicts a label for each assignment edge to indicate whether the classification is reliable or not. The updated relational representations for proposals are used for bounding box regression. Embedding our framework into state-of-the-art baselines, we perform extensive comparison experiments on two public benchmarks, i.e., Pascal VOC and COCO2017. And the experimental results demonstrate the flexibility and effectiveness of the proposed framework. More... »

PAGES

1

References to SciGraph publications

  • 2009-09-09. The Pascal Visual Object Classes (VOC) Challenge in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2015-07-09. Multi-Cue Illumination Estimation via a Tree-Structured Group Joint Sparse Representation in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2015-04-11. ImageNet Large Scale Visual Recognition Challenge in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2020-10-29. Region Graph Embedding Network for Zero-Shot Learning in COMPUTER VISION – ECCV 2020
  • 2012. Diagnosing Error in Object Detectors in COMPUTER VISION – ECCV 2012
  • 2020-04-28. Local keypoint-based Faster R-CNN in APPLIED INTELLIGENCE
  • 2014. Microsoft COCO: Common Objects in Context in COMPUTER VISION – ECCV 2014
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8

    DOI

    http://dx.doi.org/10.1007/s00138-021-01257-8

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1142266214


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/17", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Psychology and Cognitive Sciences", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1701", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Psychology", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "You", 
            "givenName": "XiuTing", 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Liu", 
            "givenName": "He", 
            "id": "sg:person.011107204575.24", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011107204575.24"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Wang", 
            "givenName": "Tao", 
            "id": "sg:person.015616541403.04", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015616541403.04"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Feng", 
            "givenName": "Songhe", 
            "id": "sg:person.07666300667.45", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07666300667.45"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Lang", 
            "givenName": "Congyan", 
            "id": "sg:person.010463661267.72", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010463661267.72"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/978-3-642-33712-3_25", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1027572846", 
              "https://doi.org/10.1007/978-3-642-33712-3_25"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-009-0275-4", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1014796149", 
              "https://doi.org/10.1007/s11263-009-0275-4"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s10489-020-01665-9", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1127168015", 
              "https://doi.org/10.1007/s10489-020-01665-9"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-015-0844-7", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1050523260", 
              "https://doi.org/10.1007/s11263-015-0844-7"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-10602-1_48", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1045321436", 
              "https://doi.org/10.1007/978-3-319-10602-1_48"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-030-58548-8_33", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1132138019", 
              "https://doi.org/10.1007/978-3-030-58548-8_33"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-015-0816-y", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1009767488", 
              "https://doi.org/10.1007/s11263-015-0816-y"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2021-10-30", 
        "datePublishedReg": "2021-10-30", 
        "description": "Utilizing relational representations to facilitate object detection has attracted growing research attention in recent years. However, previous studies mainly focus on relationships within the region proposals or within the label embeddings and pay less attention to the relationships between them. To fill this gap, we propose a novel object detection framework that fully explores the relationships across visual feature space and label embedding space to facilitate the proposal classification in object detection. Specifically, we model the region proposals and class labels into a uniform relation graph, where the extracted proposals and labels are regarded as nodes and each pair of them is associated by an assignment edge, and convert the problem of classifying proposals to the problem of selecting reliable edges from the constructed relation graph. Furthermore, a graph convolutional module is developed to perform relational reasoning on the graph, which finally predicts a label for each assignment edge to indicate whether the classification is reliable or not. The updated relational representations for proposals are used for bounding box regression. Embedding our framework into state-of-the-art baselines, we perform extensive comparison experiments on two public benchmarks, i.e., Pascal VOC and COCO2017. And the experimental results demonstrate the flexibility and effectiveness of the proposed framework.", 
        "genre": "article", 
        "id": "sg:pub.10.1007/s00138-021-01257-8", 
        "inLanguage": "en", 
        "isAccessibleForFree": false, 
        "isFundedItemOf": [
          {
            "id": "sg:grant.8943848", 
            "type": "MonetaryGrant"
          }
        ], 
        "isPartOf": [
          {
            "id": "sg:journal.1045266", 
            "issn": [
              "0932-8092", 
              "1432-1769"
            ], 
            "name": "Machine Vision and Applications", 
            "publisher": "Springer Nature", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "1", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "33"
          }
        ], 
        "keywords": [
          "object detection", 
          "region proposals", 
          "novel object detection framework", 
          "relation graph", 
          "relational representation", 
          "object detection framework", 
          "graph neural network", 
          "visual feature space", 
          "Extensive comparison experiments", 
          "relational reasoning", 
          "label embedding", 
          "detection framework", 
          "convolutional module", 
          "class labels", 
          "box regression", 
          "art baselines", 
          "public benchmarks", 
          "PASCAL VOC", 
          "neural network", 
          "feature space", 
          "proposal classification", 
          "embedding space", 
          "reliable edges", 
          "comparison experiments", 
          "graph", 
          "experimental results", 
          "framework", 
          "reasoning", 
          "proposal", 
          "labels", 
          "research attention", 
          "classification", 
          "representation", 
          "recent years", 
          "detection", 
          "embedding", 
          "nodes", 
          "benchmarks", 
          "network", 
          "module", 
          "space", 
          "edge", 
          "flexibility", 
          "less attention", 
          "effectiveness", 
          "attention", 
          "experiments", 
          "state", 
          "gap", 
          "pairs", 
          "results", 
          "regression", 
          "relationship", 
          "previous studies", 
          "years", 
          "baseline", 
          "study", 
          "VOCs", 
          "problem", 
          "label embedding space", 
          "uniform relation graph", 
          "assignment edge", 
          "graph convolutional module", 
          "COCO2017"
        ], 
        "name": "Object detection by crossing relational reasoning based on graph neural network", 
        "pagination": "1", 
        "productId": [
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1142266214"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s00138-021-01257-8"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s00138-021-01257-8", 
          "https://app.dimensions.ai/details/publication/pub.1142266214"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2022-01-01T19:02", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-springernature-scigraph/baseset/20220101/entities/gbq_results/article/article_892.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://doi.org/10.1007/s00138-021-01257-8"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8'


     

    This table displays all metadata directly associated to this object as RDF triples.

    179 TRIPLES      22 PREDICATES      96 URIs      81 LITERALS      6 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s00138-021-01257-8 schema:about anzsrc-for:17
    2 anzsrc-for:1701
    3 schema:author Ncf5a6141074349dab5ff855f1d36cfaa
    4 schema:citation sg:pub.10.1007/978-3-030-58548-8_33
    5 sg:pub.10.1007/978-3-319-10602-1_48
    6 sg:pub.10.1007/978-3-642-33712-3_25
    7 sg:pub.10.1007/s10489-020-01665-9
    8 sg:pub.10.1007/s11263-009-0275-4
    9 sg:pub.10.1007/s11263-015-0816-y
    10 sg:pub.10.1007/s11263-015-0844-7
    11 schema:datePublished 2021-10-30
    12 schema:datePublishedReg 2021-10-30
    13 schema:description Utilizing relational representations to facilitate object detection has attracted growing research attention in recent years. However, previous studies mainly focus on relationships within the region proposals or within the label embeddings and pay less attention to the relationships between them. To fill this gap, we propose a novel object detection framework that fully explores the relationships across visual feature space and label embedding space to facilitate the proposal classification in object detection. Specifically, we model the region proposals and class labels into a uniform relation graph, where the extracted proposals and labels are regarded as nodes and each pair of them is associated by an assignment edge, and convert the problem of classifying proposals to the problem of selecting reliable edges from the constructed relation graph. Furthermore, a graph convolutional module is developed to perform relational reasoning on the graph, which finally predicts a label for each assignment edge to indicate whether the classification is reliable or not. The updated relational representations for proposals are used for bounding box regression. Embedding our framework into state-of-the-art baselines, we perform extensive comparison experiments on two public benchmarks, i.e., Pascal VOC and COCO2017. And the experimental results demonstrate the flexibility and effectiveness of the proposed framework.
    14 schema:genre article
    15 schema:inLanguage en
    16 schema:isAccessibleForFree false
    17 schema:isPartOf N7f82c584c99c40ed8d03cf295b31808e
    18 N8c6915e0277444f5856536497af391f0
    19 sg:journal.1045266
    20 schema:keywords COCO2017
    21 Extensive comparison experiments
    22 PASCAL VOC
    23 VOCs
    24 art baselines
    25 assignment edge
    26 attention
    27 baseline
    28 benchmarks
    29 box regression
    30 class labels
    31 classification
    32 comparison experiments
    33 convolutional module
    34 detection
    35 detection framework
    36 edge
    37 effectiveness
    38 embedding
    39 embedding space
    40 experimental results
    41 experiments
    42 feature space
    43 flexibility
    44 framework
    45 gap
    46 graph
    47 graph convolutional module
    48 graph neural network
    49 label embedding
    50 label embedding space
    51 labels
    52 less attention
    53 module
    54 network
    55 neural network
    56 nodes
    57 novel object detection framework
    58 object detection
    59 object detection framework
    60 pairs
    61 previous studies
    62 problem
    63 proposal
    64 proposal classification
    65 public benchmarks
    66 reasoning
    67 recent years
    68 region proposals
    69 regression
    70 relation graph
    71 relational reasoning
    72 relational representation
    73 relationship
    74 reliable edges
    75 representation
    76 research attention
    77 results
    78 space
    79 state
    80 study
    81 uniform relation graph
    82 visual feature space
    83 years
    84 schema:name Object detection by crossing relational reasoning based on graph neural network
    85 schema:pagination 1
    86 schema:productId Ncf06a33e2f0643e1b5bba7b27d45403b
    87 Nfb0edf2d12b449d58abc32313ac35e8f
    88 schema:sameAs https://app.dimensions.ai/details/publication/pub.1142266214
    89 https://doi.org/10.1007/s00138-021-01257-8
    90 schema:sdDatePublished 2022-01-01T19:02
    91 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    92 schema:sdPublisher N3c55041b72584328bb84e562a849cade
    93 schema:url https://doi.org/10.1007/s00138-021-01257-8
    94 sgo:license sg:explorer/license/
    95 sgo:sdDataset articles
    96 rdf:type schema:ScholarlyArticle
    97 N05604e881afe431885fc78bf7e9b698a rdf:first sg:person.015616541403.04
    98 rdf:rest N873cac1ae806432d9c92ee6a6368c7cd
    99 N3c55041b72584328bb84e562a849cade schema:name Springer Nature - SN SciGraph project
    100 rdf:type schema:Organization
    101 N7f82c584c99c40ed8d03cf295b31808e schema:issueNumber 1
    102 rdf:type schema:PublicationIssue
    103 N873cac1ae806432d9c92ee6a6368c7cd rdf:first sg:person.07666300667.45
    104 rdf:rest Nf9883b5981c04af8b8c5b3adc7cddcc6
    105 N8c6915e0277444f5856536497af391f0 schema:volumeNumber 33
    106 rdf:type schema:PublicationVolume
    107 N8c86d5d6cd754360aa00a26e4202a790 schema:affiliation grid-institutes:grid.181531.f
    108 schema:familyName You
    109 schema:givenName XiuTing
    110 rdf:type schema:Person
    111 Nc3576cf4a5b6445e8cb7d5323e3bef62 rdf:first sg:person.011107204575.24
    112 rdf:rest N05604e881afe431885fc78bf7e9b698a
    113 Ncf06a33e2f0643e1b5bba7b27d45403b schema:name doi
    114 schema:value 10.1007/s00138-021-01257-8
    115 rdf:type schema:PropertyValue
    116 Ncf5a6141074349dab5ff855f1d36cfaa rdf:first N8c86d5d6cd754360aa00a26e4202a790
    117 rdf:rest Nc3576cf4a5b6445e8cb7d5323e3bef62
    118 Nf9883b5981c04af8b8c5b3adc7cddcc6 rdf:first sg:person.010463661267.72
    119 rdf:rest rdf:nil
    120 Nfb0edf2d12b449d58abc32313ac35e8f schema:name dimensions_id
    121 schema:value pub.1142266214
    122 rdf:type schema:PropertyValue
    123 anzsrc-for:17 schema:inDefinedTermSet anzsrc-for:
    124 schema:name Psychology and Cognitive Sciences
    125 rdf:type schema:DefinedTerm
    126 anzsrc-for:1701 schema:inDefinedTermSet anzsrc-for:
    127 schema:name Psychology
    128 rdf:type schema:DefinedTerm
    129 sg:grant.8943848 http://pending.schema.org/fundedItem sg:pub.10.1007/s00138-021-01257-8
    130 rdf:type schema:MonetaryGrant
    131 sg:journal.1045266 schema:issn 0932-8092
    132 1432-1769
    133 schema:name Machine Vision and Applications
    134 schema:publisher Springer Nature
    135 rdf:type schema:Periodical
    136 sg:person.010463661267.72 schema:affiliation grid-institutes:grid.181531.f
    137 schema:familyName Lang
    138 schema:givenName Congyan
    139 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010463661267.72
    140 rdf:type schema:Person
    141 sg:person.011107204575.24 schema:affiliation grid-institutes:grid.181531.f
    142 schema:familyName Liu
    143 schema:givenName He
    144 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011107204575.24
    145 rdf:type schema:Person
    146 sg:person.015616541403.04 schema:affiliation grid-institutes:grid.181531.f
    147 schema:familyName Wang
    148 schema:givenName Tao
    149 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015616541403.04
    150 rdf:type schema:Person
    151 sg:person.07666300667.45 schema:affiliation grid-institutes:grid.181531.f
    152 schema:familyName Feng
    153 schema:givenName Songhe
    154 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07666300667.45
    155 rdf:type schema:Person
    156 sg:pub.10.1007/978-3-030-58548-8_33 schema:sameAs https://app.dimensions.ai/details/publication/pub.1132138019
    157 https://doi.org/10.1007/978-3-030-58548-8_33
    158 rdf:type schema:CreativeWork
    159 sg:pub.10.1007/978-3-319-10602-1_48 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045321436
    160 https://doi.org/10.1007/978-3-319-10602-1_48
    161 rdf:type schema:CreativeWork
    162 sg:pub.10.1007/978-3-642-33712-3_25 schema:sameAs https://app.dimensions.ai/details/publication/pub.1027572846
    163 https://doi.org/10.1007/978-3-642-33712-3_25
    164 rdf:type schema:CreativeWork
    165 sg:pub.10.1007/s10489-020-01665-9 schema:sameAs https://app.dimensions.ai/details/publication/pub.1127168015
    166 https://doi.org/10.1007/s10489-020-01665-9
    167 rdf:type schema:CreativeWork
    168 sg:pub.10.1007/s11263-009-0275-4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1014796149
    169 https://doi.org/10.1007/s11263-009-0275-4
    170 rdf:type schema:CreativeWork
    171 sg:pub.10.1007/s11263-015-0816-y schema:sameAs https://app.dimensions.ai/details/publication/pub.1009767488
    172 https://doi.org/10.1007/s11263-015-0816-y
    173 rdf:type schema:CreativeWork
    174 sg:pub.10.1007/s11263-015-0844-7 schema:sameAs https://app.dimensions.ai/details/publication/pub.1050523260
    175 https://doi.org/10.1007/s11263-015-0844-7
    176 rdf:type schema:CreativeWork
    177 grid-institutes:grid.181531.f schema:alternateName School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China
    178 schema:name School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China
    179 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...