Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2017-03-21

AUTHORS

Abhinav Valada , Gabriel L. Oliveira , Thomas Brox , Wolfram Burgard

ABSTRACT

Semantic scene understanding of unstructured environments is a highly challenging task for robots operating in the real world. Deep Convolutional Neural Network architectures define the state of the art in various segmentation tasks. So far, researchers have focused on segmentation with RGB data. In this paper, we study the use of multispectral and multimodal images for semantic segmentation and develop fusion architectures that learn from RGB, Near-InfraRed channels, and depth data. We introduce a first-of-its-kind multispectral segmentation benchmark that contains 15, 000 images and 366 pixel-wise ground truth annotations of unstructured forest environments. We identify new data augmentation strategies that enable training of very deep models using relatively small datasets. We show that our UpNet architecture exceeds the state of the art both qualitatively and quantitatively on our benchmark. In addition, we present experimental results for segmentation under challenging real-world conditions. Benchmark and demo are publicly available at http://deepscene.cs.uni-freiburg.de. More... »

PAGES

465-477

Book

TITLE

2016 International Symposium on Experimental Robotics

ISBN

978-3-319-50114-7
978-3-319-50115-4

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41

DOI

http://dx.doi.org/10.1007/978-3-319-50115-4_41

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1084722802


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany", 
          "id": "http://www.grid.ac/institutes/grid.5963.9", 
          "name": [
            "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Valada", 
        "givenName": "Abhinav", 
        "id": "sg:person.07371177473.42", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07371177473.42"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany", 
          "id": "http://www.grid.ac/institutes/grid.5963.9", 
          "name": [
            "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Oliveira", 
        "givenName": "Gabriel L.", 
        "id": "sg:person.011310276047.71", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011310276047.71"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany", 
          "id": "http://www.grid.ac/institutes/grid.5963.9", 
          "name": [
            "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Brox", 
        "givenName": "Thomas", 
        "id": "sg:person.012443225372.65", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012443225372.65"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany", 
          "id": "http://www.grid.ac/institutes/grid.5963.9", 
          "name": [
            "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Burgard", 
        "givenName": "Wolfram", 
        "id": "sg:person.014270043511.25", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014270043511.25"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2017-03-21", 
    "datePublishedReg": "2017-03-21", 
    "description": "Semantic scene understanding of unstructured environments is a highly challenging task for robots operating in the real world. Deep Convolutional Neural Network architectures define the state of the art in various segmentation tasks. So far, researchers have focused on segmentation with RGB data. In this paper, we study the use of multispectral and multimodal images for semantic segmentation and develop fusion architectures that learn from RGB, Near-InfraRed channels, and depth data. We introduce a first-of-its-kind multispectral segmentation benchmark that contains 15,\u00a0000 images and 366 pixel-wise ground truth annotations of unstructured forest environments. We identify new data augmentation strategies that enable training of very deep models using relatively small datasets. We show that our UpNet architecture exceeds the state of the art both qualitatively and quantitatively on our benchmark. In addition, we present experimental results for segmentation under challenging real-world conditions. Benchmark and demo are publicly available at http://deepscene.cs.uni-freiburg.de.", 
    "editor": [
      {
        "familyName": "Kuli\u0107", 
        "givenName": "Dana", 
        "type": "Person"
      }, 
      {
        "familyName": "Nakamura", 
        "givenName": "Yoshihiko", 
        "type": "Person"
      }, 
      {
        "familyName": "Khatib", 
        "givenName": "Oussama", 
        "type": "Person"
      }, 
      {
        "familyName": "Venture", 
        "givenName": "Gentiane", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-50115-4_41", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-319-50114-7", 
        "978-3-319-50115-4"
      ], 
      "name": "2016 International Symposium on Experimental Robotics", 
      "type": "Book"
    }, 
    "keywords": [
      "semantic scene understanding", 
      "scene understanding", 
      "deep convolutional neural network architecture", 
      "pixel-wise ground truth annotations", 
      "convolutional neural network architecture", 
      "new data augmentation strategy", 
      "neural network architecture", 
      "ground truth annotations", 
      "data augmentation strategy", 
      "near infrared (NIR) channel", 
      "semantic segmentation", 
      "deep models", 
      "truth annotations", 
      "multimodal fusion", 
      "fusion architecture", 
      "segmentation task", 
      "segmentation benchmarks", 
      "network architecture", 
      "unstructured environments", 
      "RGB data", 
      "small datasets", 
      "multimodal images", 
      "real-world conditions", 
      "challenging task", 
      "depth data", 
      "real world", 
      "segmentation", 
      "architecture", 
      "augmentation strategies", 
      "experimental results", 
      "task", 
      "benchmarks", 
      "images", 
      "environment", 
      "robot", 
      "RGB", 
      "annotation", 
      "dataset", 
      "demo", 
      "art", 
      "Forested Environments", 
      "forest environment", 
      "fusion", 
      "data", 
      "researchers", 
      "training", 
      "model", 
      "state", 
      "world", 
      "channels", 
      "strategies", 
      "use", 
      "results", 
      "understanding", 
      "addition", 
      "Freiburg", 
      "conditions", 
      "paper"
    ], 
    "name": "Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion", 
    "pagination": "465-477", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1084722802"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-50115-4_41"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-50115-4_41", 
      "https://app.dimensions.ai/details/publication/pub.1084722802"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-12-01T06:46", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20221201/entities/gbq_results/chapter/chapter_101.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-319-50115-4_41"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41'


 

This table displays all metadata directly associated to this object as RDF triples.

153 TRIPLES      22 PREDICATES      82 URIs      75 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-50115-4_41 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Nfdb3cbcc3e1049a5a228ad17c74d244b
4 schema:datePublished 2017-03-21
5 schema:datePublishedReg 2017-03-21
6 schema:description Semantic scene understanding of unstructured environments is a highly challenging task for robots operating in the real world. Deep Convolutional Neural Network architectures define the state of the art in various segmentation tasks. So far, researchers have focused on segmentation with RGB data. In this paper, we study the use of multispectral and multimodal images for semantic segmentation and develop fusion architectures that learn from RGB, Near-InfraRed channels, and depth data. We introduce a first-of-its-kind multispectral segmentation benchmark that contains 15, 000 images and 366 pixel-wise ground truth annotations of unstructured forest environments. We identify new data augmentation strategies that enable training of very deep models using relatively small datasets. We show that our UpNet architecture exceeds the state of the art both qualitatively and quantitatively on our benchmark. In addition, we present experimental results for segmentation under challenging real-world conditions. Benchmark and demo are publicly available at http://deepscene.cs.uni-freiburg.de.
7 schema:editor N3574ec6df2074bb89c7bbcf4824aa519
8 schema:genre chapter
9 schema:isAccessibleForFree false
10 schema:isPartOf N7888b447009a4d40bf6c92e2a1145dde
11 schema:keywords Forested Environments
12 Freiburg
13 RGB
14 RGB data
15 addition
16 annotation
17 architecture
18 art
19 augmentation strategies
20 benchmarks
21 challenging task
22 channels
23 conditions
24 convolutional neural network architecture
25 data
26 data augmentation strategy
27 dataset
28 deep convolutional neural network architecture
29 deep models
30 demo
31 depth data
32 environment
33 experimental results
34 forest environment
35 fusion
36 fusion architecture
37 ground truth annotations
38 images
39 model
40 multimodal fusion
41 multimodal images
42 near infrared (NIR) channel
43 network architecture
44 neural network architecture
45 new data augmentation strategy
46 paper
47 pixel-wise ground truth annotations
48 real world
49 real-world conditions
50 researchers
51 results
52 robot
53 scene understanding
54 segmentation
55 segmentation benchmarks
56 segmentation task
57 semantic scene understanding
58 semantic segmentation
59 small datasets
60 state
61 strategies
62 task
63 training
64 truth annotations
65 understanding
66 unstructured environments
67 use
68 world
69 schema:name Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion
70 schema:pagination 465-477
71 schema:productId N468356e4eb0c42b8ab05d814417b5aff
72 Nbd1ec68c8c9a40948d57e2dc20d9c9e6
73 schema:publisher Nc986611a06dd4f2b9482ec36aae02ca1
74 schema:sameAs https://app.dimensions.ai/details/publication/pub.1084722802
75 https://doi.org/10.1007/978-3-319-50115-4_41
76 schema:sdDatePublished 2022-12-01T06:46
77 schema:sdLicense https://scigraph.springernature.com/explorer/license/
78 schema:sdPublisher N1f2694bbc4634ad9bc5665b02fcdefb1
79 schema:url https://doi.org/10.1007/978-3-319-50115-4_41
80 sgo:license sg:explorer/license/
81 sgo:sdDataset chapters
82 rdf:type schema:Chapter
83 N04d8f49ba9f84a7e828bc320714bcbad rdf:first N671d7b4e67eb4517a6cbeef012461ced
84 rdf:rest Ne05cf1e65678400eac7065be9f4ce396
85 N0ff8d18ca2804425b780cf9407210d66 schema:familyName Venture
86 schema:givenName Gentiane
87 rdf:type schema:Person
88 N1f2694bbc4634ad9bc5665b02fcdefb1 schema:name Springer Nature - SN SciGraph project
89 rdf:type schema:Organization
90 N3574ec6df2074bb89c7bbcf4824aa519 rdf:first Nede7a49a440d416f82ee819cfd9bb966
91 rdf:rest N04d8f49ba9f84a7e828bc320714bcbad
92 N468356e4eb0c42b8ab05d814417b5aff schema:name doi
93 schema:value 10.1007/978-3-319-50115-4_41
94 rdf:type schema:PropertyValue
95 N4bec27a9e5074aefb0a42bb542ec1a4e rdf:first N0ff8d18ca2804425b780cf9407210d66
96 rdf:rest rdf:nil
97 N5aed995336c945269af38cac1f6e3642 rdf:first sg:person.012443225372.65
98 rdf:rest Ncec6bd98abfa4107a7916e7ba4d57347
99 N671d7b4e67eb4517a6cbeef012461ced schema:familyName Nakamura
100 schema:givenName Yoshihiko
101 rdf:type schema:Person
102 N7888b447009a4d40bf6c92e2a1145dde schema:isbn 978-3-319-50114-7
103 978-3-319-50115-4
104 schema:name 2016 International Symposium on Experimental Robotics
105 rdf:type schema:Book
106 Na48aae78a3d5411697ee739b68cfe104 schema:familyName Khatib
107 schema:givenName Oussama
108 rdf:type schema:Person
109 Nbd1ec68c8c9a40948d57e2dc20d9c9e6 schema:name dimensions_id
110 schema:value pub.1084722802
111 rdf:type schema:PropertyValue
112 Nc671d07d97fd4b90b5f743464c7785db rdf:first sg:person.011310276047.71
113 rdf:rest N5aed995336c945269af38cac1f6e3642
114 Nc986611a06dd4f2b9482ec36aae02ca1 schema:name Springer Nature
115 rdf:type schema:Organisation
116 Ncec6bd98abfa4107a7916e7ba4d57347 rdf:first sg:person.014270043511.25
117 rdf:rest rdf:nil
118 Ne05cf1e65678400eac7065be9f4ce396 rdf:first Na48aae78a3d5411697ee739b68cfe104
119 rdf:rest N4bec27a9e5074aefb0a42bb542ec1a4e
120 Nede7a49a440d416f82ee819cfd9bb966 schema:familyName Kulić
121 schema:givenName Dana
122 rdf:type schema:Person
123 Nfdb3cbcc3e1049a5a228ad17c74d244b rdf:first sg:person.07371177473.42
124 rdf:rest Nc671d07d97fd4b90b5f743464c7785db
125 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
126 schema:name Information and Computing Sciences
127 rdf:type schema:DefinedTerm
128 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
129 schema:name Artificial Intelligence and Image Processing
130 rdf:type schema:DefinedTerm
131 sg:person.011310276047.71 schema:affiliation grid-institutes:grid.5963.9
132 schema:familyName Oliveira
133 schema:givenName Gabriel L.
134 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011310276047.71
135 rdf:type schema:Person
136 sg:person.012443225372.65 schema:affiliation grid-institutes:grid.5963.9
137 schema:familyName Brox
138 schema:givenName Thomas
139 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012443225372.65
140 rdf:type schema:Person
141 sg:person.014270043511.25 schema:affiliation grid-institutes:grid.5963.9
142 schema:familyName Burgard
143 schema:givenName Wolfram
144 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014270043511.25
145 rdf:type schema:Person
146 sg:person.07371177473.42 schema:affiliation grid-institutes:grid.5963.9
147 schema:familyName Valada
148 schema:givenName Abhinav
149 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07371177473.42
150 rdf:type schema:Person
151 grid-institutes:grid.5963.9 schema:alternateName Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany
152 schema:name Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany
153 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...