Object detection by crossing relational reasoning based on graph neural network View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2021-10-30

AUTHORS

XiuTing You, He Liu, Tao Wang, Songhe Feng, Congyan Lang

ABSTRACT

Utilizing relational representations to facilitate object detection has attracted growing research attention in recent years. However, previous studies mainly focus on relationships within the region proposals or within the label embeddings and pay less attention to the relationships between them. To fill this gap, we propose a novel object detection framework that fully explores the relationships across visual feature space and label embedding space to facilitate the proposal classification in object detection. Specifically, we model the region proposals and class labels into a uniform relation graph, where the extracted proposals and labels are regarded as nodes and each pair of them is associated by an assignment edge, and convert the problem of classifying proposals to the problem of selecting reliable edges from the constructed relation graph. Furthermore, a graph convolutional module is developed to perform relational reasoning on the graph, which finally predicts a label for each assignment edge to indicate whether the classification is reliable or not. The updated relational representations for proposals are used for bounding box regression. Embedding our framework into state-of-the-art baselines, we perform extensive comparison experiments on two public benchmarks, i.e., Pascal VOC and COCO2017. And the experimental results demonstrate the flexibility and effectiveness of the proposed framework. More... »

PAGES

1

References to SciGraph publications

  • 2009-09-09. The Pascal Visual Object Classes (VOC) Challenge in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2015-07-09. Multi-Cue Illumination Estimation via a Tree-Structured Group Joint Sparse Representation in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2015-04-11. ImageNet Large Scale Visual Recognition Challenge in INTERNATIONAL JOURNAL OF COMPUTER VISION
  • 2020-10-29. Region Graph Embedding Network for Zero-Shot Learning in COMPUTER VISION – ECCV 2020
  • 2012. Diagnosing Error in Object Detectors in COMPUTER VISION – ECCV 2012
  • 2020-04-28. Local keypoint-based Faster R-CNN in APPLIED INTELLIGENCE
  • 2014. Microsoft COCO: Common Objects in Context in COMPUTER VISION – ECCV 2014
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8

    DOI

    http://dx.doi.org/10.1007/s00138-021-01257-8

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1142266214


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/17", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Psychology and Cognitive Sciences", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1701", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Psychology", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "You", 
            "givenName": "XiuTing", 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Liu", 
            "givenName": "He", 
            "id": "sg:person.011107204575.24", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011107204575.24"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Wang", 
            "givenName": "Tao", 
            "id": "sg:person.015616541403.04", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015616541403.04"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Feng", 
            "givenName": "Songhe", 
            "id": "sg:person.07666300667.45", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07666300667.45"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China", 
              "id": "http://www.grid.ac/institutes/grid.181531.f", 
              "name": [
                "School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Lang", 
            "givenName": "Congyan", 
            "id": "sg:person.010463661267.72", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010463661267.72"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/978-3-319-10602-1_48", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1045321436", 
              "https://doi.org/10.1007/978-3-319-10602-1_48"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-030-58548-8_33", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1132138019", 
              "https://doi.org/10.1007/978-3-030-58548-8_33"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-009-0275-4", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1014796149", 
              "https://doi.org/10.1007/s11263-009-0275-4"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-015-0844-7", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1050523260", 
              "https://doi.org/10.1007/s11263-015-0844-7"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s10489-020-01665-9", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1127168015", 
              "https://doi.org/10.1007/s10489-020-01665-9"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-33712-3_25", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1027572846", 
              "https://doi.org/10.1007/978-3-642-33712-3_25"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11263-015-0816-y", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1009767488", 
              "https://doi.org/10.1007/s11263-015-0816-y"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2021-10-30", 
        "datePublishedReg": "2021-10-30", 
        "description": "Utilizing relational representations to facilitate object detection has attracted growing research attention in recent years. However, previous studies mainly focus on relationships within the region proposals or within the label embeddings and pay less attention to the relationships between them. To fill this gap, we propose a novel object detection framework that fully explores the relationships across visual feature space and label embedding space to facilitate the proposal classification in object detection. Specifically, we model the region proposals and class labels into a uniform relation graph, where the extracted proposals and labels are regarded as nodes and each pair of them is associated by an assignment edge, and convert the problem of classifying proposals to the problem of selecting reliable edges from the constructed relation graph. Furthermore, a graph convolutional module is developed to perform relational reasoning on the graph, which finally predicts a label for each assignment edge to indicate whether the classification is reliable or not. The updated relational representations for proposals are used for bounding box regression. Embedding our framework into state-of-the-art baselines, we perform extensive comparison experiments on two public benchmarks, i.e., Pascal VOC and COCO2017. And the experimental results demonstrate the flexibility and effectiveness of the proposed framework.", 
        "genre": "article", 
        "id": "sg:pub.10.1007/s00138-021-01257-8", 
        "inLanguage": "en", 
        "isAccessibleForFree": false, 
        "isFundedItemOf": [
          {
            "id": "sg:grant.8943848", 
            "type": "MonetaryGrant"
          }
        ], 
        "isPartOf": [
          {
            "id": "sg:journal.1045266", 
            "issn": [
              "0932-8092", 
              "1432-1769"
            ], 
            "name": "Machine Vision and Applications", 
            "publisher": "Springer Nature", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "1", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "33"
          }
        ], 
        "keywords": [
          "object detection", 
          "region proposals", 
          "novel object detection framework", 
          "relation graph", 
          "relational representation", 
          "object detection framework", 
          "graph neural network", 
          "visual feature space", 
          "Extensive comparison experiments", 
          "relational reasoning", 
          "label embedding", 
          "detection framework", 
          "convolutional module", 
          "class labels", 
          "box regression", 
          "art baselines", 
          "public benchmarks", 
          "PASCAL VOC", 
          "neural network", 
          "feature space", 
          "proposal classification", 
          "embedding space", 
          "reliable edges", 
          "comparison experiments", 
          "graph", 
          "experimental results", 
          "framework", 
          "reasoning", 
          "proposal", 
          "labels", 
          "research attention", 
          "classification", 
          "representation", 
          "recent years", 
          "detection", 
          "embedding", 
          "nodes", 
          "benchmarks", 
          "network", 
          "module", 
          "space", 
          "edge", 
          "flexibility", 
          "less attention", 
          "effectiveness", 
          "attention", 
          "experiments", 
          "state", 
          "gap", 
          "pairs", 
          "results", 
          "regression", 
          "relationship", 
          "previous studies", 
          "years", 
          "baseline", 
          "study", 
          "VOCs", 
          "problem"
        ], 
        "name": "Object detection by crossing relational reasoning based on graph neural network", 
        "pagination": "1", 
        "productId": [
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1142266214"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s00138-021-01257-8"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s00138-021-01257-8", 
          "https://app.dimensions.ai/details/publication/pub.1142266214"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2022-05-10T10:31", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-springernature-scigraph/baseset/20220509/entities/gbq_results/article/article_882.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://doi.org/10.1007/s00138-021-01257-8"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s00138-021-01257-8'


     

    This table displays all metadata directly associated to this object as RDF triples.

    174 TRIPLES      22 PREDICATES      91 URIs      76 LITERALS      6 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s00138-021-01257-8 schema:about anzsrc-for:17
    2 anzsrc-for:1701
    3 schema:author N7d2c99695b4b4256bb7c66991a8f9632
    4 schema:citation sg:pub.10.1007/978-3-030-58548-8_33
    5 sg:pub.10.1007/978-3-319-10602-1_48
    6 sg:pub.10.1007/978-3-642-33712-3_25
    7 sg:pub.10.1007/s10489-020-01665-9
    8 sg:pub.10.1007/s11263-009-0275-4
    9 sg:pub.10.1007/s11263-015-0816-y
    10 sg:pub.10.1007/s11263-015-0844-7
    11 schema:datePublished 2021-10-30
    12 schema:datePublishedReg 2021-10-30
    13 schema:description Utilizing relational representations to facilitate object detection has attracted growing research attention in recent years. However, previous studies mainly focus on relationships within the region proposals or within the label embeddings and pay less attention to the relationships between them. To fill this gap, we propose a novel object detection framework that fully explores the relationships across visual feature space and label embedding space to facilitate the proposal classification in object detection. Specifically, we model the region proposals and class labels into a uniform relation graph, where the extracted proposals and labels are regarded as nodes and each pair of them is associated by an assignment edge, and convert the problem of classifying proposals to the problem of selecting reliable edges from the constructed relation graph. Furthermore, a graph convolutional module is developed to perform relational reasoning on the graph, which finally predicts a label for each assignment edge to indicate whether the classification is reliable or not. The updated relational representations for proposals are used for bounding box regression. Embedding our framework into state-of-the-art baselines, we perform extensive comparison experiments on two public benchmarks, i.e., Pascal VOC and COCO2017. And the experimental results demonstrate the flexibility and effectiveness of the proposed framework.
    14 schema:genre article
    15 schema:inLanguage en
    16 schema:isAccessibleForFree false
    17 schema:isPartOf Na4e86a4386354a6481d25be4d46f20ab
    18 Nd16ef2610d8f412baec2cc395cef6a14
    19 sg:journal.1045266
    20 schema:keywords Extensive comparison experiments
    21 PASCAL VOC
    22 VOCs
    23 art baselines
    24 attention
    25 baseline
    26 benchmarks
    27 box regression
    28 class labels
    29 classification
    30 comparison experiments
    31 convolutional module
    32 detection
    33 detection framework
    34 edge
    35 effectiveness
    36 embedding
    37 embedding space
    38 experimental results
    39 experiments
    40 feature space
    41 flexibility
    42 framework
    43 gap
    44 graph
    45 graph neural network
    46 label embedding
    47 labels
    48 less attention
    49 module
    50 network
    51 neural network
    52 nodes
    53 novel object detection framework
    54 object detection
    55 object detection framework
    56 pairs
    57 previous studies
    58 problem
    59 proposal
    60 proposal classification
    61 public benchmarks
    62 reasoning
    63 recent years
    64 region proposals
    65 regression
    66 relation graph
    67 relational reasoning
    68 relational representation
    69 relationship
    70 reliable edges
    71 representation
    72 research attention
    73 results
    74 space
    75 state
    76 study
    77 visual feature space
    78 years
    79 schema:name Object detection by crossing relational reasoning based on graph neural network
    80 schema:pagination 1
    81 schema:productId N26398136743c42479e84d47d16e90a69
    82 N84513519be834a66b3e4e8c337a39da3
    83 schema:sameAs https://app.dimensions.ai/details/publication/pub.1142266214
    84 https://doi.org/10.1007/s00138-021-01257-8
    85 schema:sdDatePublished 2022-05-10T10:31
    86 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    87 schema:sdPublisher Nebc2345182fe4edaacb931a53c5f5023
    88 schema:url https://doi.org/10.1007/s00138-021-01257-8
    89 sgo:license sg:explorer/license/
    90 sgo:sdDataset articles
    91 rdf:type schema:ScholarlyArticle
    92 N26398136743c42479e84d47d16e90a69 schema:name dimensions_id
    93 schema:value pub.1142266214
    94 rdf:type schema:PropertyValue
    95 N7d2c99695b4b4256bb7c66991a8f9632 rdf:first Nc55c53992d98474881a033107e04a61d
    96 rdf:rest Ne9e947ca9a9146b3a9c11810bd7b3caa
    97 N84513519be834a66b3e4e8c337a39da3 schema:name doi
    98 schema:value 10.1007/s00138-021-01257-8
    99 rdf:type schema:PropertyValue
    100 N85bf55c098684933bf5e198b15789c44 rdf:first sg:person.015616541403.04
    101 rdf:rest Nbfa0aa0c0e734dbbb72cc64d2034a17b
    102 Na4e86a4386354a6481d25be4d46f20ab schema:volumeNumber 33
    103 rdf:type schema:PublicationVolume
    104 Nbd4ebaaa9c6c4b59863dd21eb3cf469b rdf:first sg:person.010463661267.72
    105 rdf:rest rdf:nil
    106 Nbfa0aa0c0e734dbbb72cc64d2034a17b rdf:first sg:person.07666300667.45
    107 rdf:rest Nbd4ebaaa9c6c4b59863dd21eb3cf469b
    108 Nc55c53992d98474881a033107e04a61d schema:affiliation grid-institutes:grid.181531.f
    109 schema:familyName You
    110 schema:givenName XiuTing
    111 rdf:type schema:Person
    112 Nd16ef2610d8f412baec2cc395cef6a14 schema:issueNumber 1
    113 rdf:type schema:PublicationIssue
    114 Ne9e947ca9a9146b3a9c11810bd7b3caa rdf:first sg:person.011107204575.24
    115 rdf:rest N85bf55c098684933bf5e198b15789c44
    116 Nebc2345182fe4edaacb931a53c5f5023 schema:name Springer Nature - SN SciGraph project
    117 rdf:type schema:Organization
    118 anzsrc-for:17 schema:inDefinedTermSet anzsrc-for:
    119 schema:name Psychology and Cognitive Sciences
    120 rdf:type schema:DefinedTerm
    121 anzsrc-for:1701 schema:inDefinedTermSet anzsrc-for:
    122 schema:name Psychology
    123 rdf:type schema:DefinedTerm
    124 sg:grant.8943848 http://pending.schema.org/fundedItem sg:pub.10.1007/s00138-021-01257-8
    125 rdf:type schema:MonetaryGrant
    126 sg:journal.1045266 schema:issn 0932-8092
    127 1432-1769
    128 schema:name Machine Vision and Applications
    129 schema:publisher Springer Nature
    130 rdf:type schema:Periodical
    131 sg:person.010463661267.72 schema:affiliation grid-institutes:grid.181531.f
    132 schema:familyName Lang
    133 schema:givenName Congyan
    134 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010463661267.72
    135 rdf:type schema:Person
    136 sg:person.011107204575.24 schema:affiliation grid-institutes:grid.181531.f
    137 schema:familyName Liu
    138 schema:givenName He
    139 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011107204575.24
    140 rdf:type schema:Person
    141 sg:person.015616541403.04 schema:affiliation grid-institutes:grid.181531.f
    142 schema:familyName Wang
    143 schema:givenName Tao
    144 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015616541403.04
    145 rdf:type schema:Person
    146 sg:person.07666300667.45 schema:affiliation grid-institutes:grid.181531.f
    147 schema:familyName Feng
    148 schema:givenName Songhe
    149 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07666300667.45
    150 rdf:type schema:Person
    151 sg:pub.10.1007/978-3-030-58548-8_33 schema:sameAs https://app.dimensions.ai/details/publication/pub.1132138019
    152 https://doi.org/10.1007/978-3-030-58548-8_33
    153 rdf:type schema:CreativeWork
    154 sg:pub.10.1007/978-3-319-10602-1_48 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045321436
    155 https://doi.org/10.1007/978-3-319-10602-1_48
    156 rdf:type schema:CreativeWork
    157 sg:pub.10.1007/978-3-642-33712-3_25 schema:sameAs https://app.dimensions.ai/details/publication/pub.1027572846
    158 https://doi.org/10.1007/978-3-642-33712-3_25
    159 rdf:type schema:CreativeWork
    160 sg:pub.10.1007/s10489-020-01665-9 schema:sameAs https://app.dimensions.ai/details/publication/pub.1127168015
    161 https://doi.org/10.1007/s10489-020-01665-9
    162 rdf:type schema:CreativeWork
    163 sg:pub.10.1007/s11263-009-0275-4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1014796149
    164 https://doi.org/10.1007/s11263-009-0275-4
    165 rdf:type schema:CreativeWork
    166 sg:pub.10.1007/s11263-015-0816-y schema:sameAs https://app.dimensions.ai/details/publication/pub.1009767488
    167 https://doi.org/10.1007/s11263-015-0816-y
    168 rdf:type schema:CreativeWork
    169 sg:pub.10.1007/s11263-015-0844-7 schema:sameAs https://app.dimensions.ai/details/publication/pub.1050523260
    170 https://doi.org/10.1007/s11263-015-0844-7
    171 rdf:type schema:CreativeWork
    172 grid-institutes:grid.181531.f schema:alternateName School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China
    173 schema:name School of Computer and Information Technology, Beijing Jiaotong University, 100044, Beijing, China
    174 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...