Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2017-03-21

AUTHORS

Abhinav Valada , Gabriel L. Oliveira , Thomas Brox , Wolfram Burgard

ABSTRACT

Semantic scene understanding of unstructured environments is a highly challenging task for robots operating in the real world. Deep Convolutional Neural Network architectures define the state of the art in various segmentation tasks. So far, researchers have focused on segmentation with RGB data. In this paper, we study the use of multispectral and multimodal images for semantic segmentation and develop fusion architectures that learn from RGB, Near-InfraRed channels, and depth data. We introduce a first-of-its-kind multispectral segmentation benchmark that contains 15, 000 images and 366 pixel-wise ground truth annotations of unstructured forest environments. We identify new data augmentation strategies that enable training of very deep models using relatively small datasets. We show that our UpNet architecture exceeds the state of the art both qualitatively and quantitatively on our benchmark. In addition, we present experimental results for segmentation under challenging real-world conditions. Benchmark and demo are publicly available at http://deepscene.cs.uni-freiburg.de. More... »

PAGES

465-477

Book

TITLE

2016 International Symposium on Experimental Robotics

ISBN

978-3-319-50114-7
978-3-319-50115-4

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41

DOI

http://dx.doi.org/10.1007/978-3-319-50115-4_41

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1084722802


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany", 
          "id": "http://www.grid.ac/institutes/grid.5963.9", 
          "name": [
            "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Valada", 
        "givenName": "Abhinav", 
        "id": "sg:person.07371177473.42", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07371177473.42"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany", 
          "id": "http://www.grid.ac/institutes/grid.5963.9", 
          "name": [
            "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Oliveira", 
        "givenName": "Gabriel L.", 
        "id": "sg:person.011310276047.71", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011310276047.71"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany", 
          "id": "http://www.grid.ac/institutes/grid.5963.9", 
          "name": [
            "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Brox", 
        "givenName": "Thomas", 
        "id": "sg:person.012443225372.65", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012443225372.65"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany", 
          "id": "http://www.grid.ac/institutes/grid.5963.9", 
          "name": [
            "Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Burgard", 
        "givenName": "Wolfram", 
        "id": "sg:person.014270043511.25", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014270043511.25"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2017-03-21", 
    "datePublishedReg": "2017-03-21", 
    "description": "Semantic scene understanding of unstructured environments is a highly challenging task for robots operating in the real world. Deep Convolutional Neural Network architectures define the state of the art in various segmentation tasks. So far, researchers have focused on segmentation with RGB data. In this paper, we study the use of multispectral and multimodal images for semantic segmentation and develop fusion architectures that learn from RGB, Near-InfraRed channels, and depth data. We introduce a first-of-its-kind multispectral segmentation benchmark that contains 15,\u00a0000 images and 366 pixel-wise ground truth annotations of unstructured forest environments. We identify new data augmentation strategies that enable training of very deep models using relatively small datasets. We show that our UpNet architecture exceeds the state of the art both qualitatively and quantitatively on our benchmark. In addition, we present experimental results for segmentation under challenging real-world conditions. Benchmark and demo are publicly available at http://deepscene.cs.uni-freiburg.de.", 
    "editor": [
      {
        "familyName": "Kuli\u0107", 
        "givenName": "Dana", 
        "type": "Person"
      }, 
      {
        "familyName": "Nakamura", 
        "givenName": "Yoshihiko", 
        "type": "Person"
      }, 
      {
        "familyName": "Khatib", 
        "givenName": "Oussama", 
        "type": "Person"
      }, 
      {
        "familyName": "Venture", 
        "givenName": "Gentiane", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-50115-4_41", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-319-50114-7", 
        "978-3-319-50115-4"
      ], 
      "name": "2016 International Symposium on Experimental Robotics", 
      "type": "Book"
    }, 
    "keywords": [
      "semantic scene understanding", 
      "scene understanding", 
      "deep convolutional neural network architecture", 
      "pixel-wise ground truth annotations", 
      "convolutional neural network architecture", 
      "new data augmentation strategy", 
      "neural network architecture", 
      "ground truth annotations", 
      "data augmentation strategy", 
      "near infrared (NIR) channel", 
      "semantic segmentation", 
      "deep models", 
      "truth annotations", 
      "multimodal fusion", 
      "fusion architecture", 
      "segmentation task", 
      "segmentation benchmarks", 
      "network architecture", 
      "unstructured environments", 
      "RGB data", 
      "small datasets", 
      "multimodal images", 
      "real-world conditions", 
      "challenging task", 
      "depth data", 
      "real world", 
      "segmentation", 
      "architecture", 
      "augmentation strategies", 
      "experimental results", 
      "task", 
      "benchmarks", 
      "images", 
      "environment", 
      "robot", 
      "RGB", 
      "annotation", 
      "dataset", 
      "demo", 
      "art", 
      "Forested Environments", 
      "forest environment", 
      "fusion", 
      "data", 
      "researchers", 
      "training", 
      "model", 
      "state", 
      "world", 
      "channels", 
      "strategies", 
      "use", 
      "results", 
      "understanding", 
      "addition", 
      "Freiburg", 
      "conditions", 
      "paper"
    ], 
    "name": "Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion", 
    "pagination": "465-477", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1084722802"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-50115-4_41"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-50115-4_41", 
      "https://app.dimensions.ai/details/publication/pub.1084722802"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-10-01T06:56", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20221001/entities/gbq_results/chapter/chapter_287.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-319-50115-4_41"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-50115-4_41'


 

This table displays all metadata directly associated to this object as RDF triples.

153 TRIPLES      22 PREDICATES      82 URIs      75 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-50115-4_41 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Nb13371998905446b8d9c1fa2f7211e90
4 schema:datePublished 2017-03-21
5 schema:datePublishedReg 2017-03-21
6 schema:description Semantic scene understanding of unstructured environments is a highly challenging task for robots operating in the real world. Deep Convolutional Neural Network architectures define the state of the art in various segmentation tasks. So far, researchers have focused on segmentation with RGB data. In this paper, we study the use of multispectral and multimodal images for semantic segmentation and develop fusion architectures that learn from RGB, Near-InfraRed channels, and depth data. We introduce a first-of-its-kind multispectral segmentation benchmark that contains 15, 000 images and 366 pixel-wise ground truth annotations of unstructured forest environments. We identify new data augmentation strategies that enable training of very deep models using relatively small datasets. We show that our UpNet architecture exceeds the state of the art both qualitatively and quantitatively on our benchmark. In addition, we present experimental results for segmentation under challenging real-world conditions. Benchmark and demo are publicly available at http://deepscene.cs.uni-freiburg.de.
7 schema:editor Nd1b5f3b504944813995f6e645e2d4dbc
8 schema:genre chapter
9 schema:isAccessibleForFree false
10 schema:isPartOf N7b0c2a7a22264781935f10d8318282c7
11 schema:keywords Forested Environments
12 Freiburg
13 RGB
14 RGB data
15 addition
16 annotation
17 architecture
18 art
19 augmentation strategies
20 benchmarks
21 challenging task
22 channels
23 conditions
24 convolutional neural network architecture
25 data
26 data augmentation strategy
27 dataset
28 deep convolutional neural network architecture
29 deep models
30 demo
31 depth data
32 environment
33 experimental results
34 forest environment
35 fusion
36 fusion architecture
37 ground truth annotations
38 images
39 model
40 multimodal fusion
41 multimodal images
42 near infrared (NIR) channel
43 network architecture
44 neural network architecture
45 new data augmentation strategy
46 paper
47 pixel-wise ground truth annotations
48 real world
49 real-world conditions
50 researchers
51 results
52 robot
53 scene understanding
54 segmentation
55 segmentation benchmarks
56 segmentation task
57 semantic scene understanding
58 semantic segmentation
59 small datasets
60 state
61 strategies
62 task
63 training
64 truth annotations
65 understanding
66 unstructured environments
67 use
68 world
69 schema:name Deep Multispectral Semantic Scene Understanding of Forested Environments Using Multimodal Fusion
70 schema:pagination 465-477
71 schema:productId N2ad7845baeb848669d6f2a11561f2814
72 N8423b5f7b4904749a28f2844e5e11d94
73 schema:publisher N967df74dcc0343cba8a96cb54d960393
74 schema:sameAs https://app.dimensions.ai/details/publication/pub.1084722802
75 https://doi.org/10.1007/978-3-319-50115-4_41
76 schema:sdDatePublished 2022-10-01T06:56
77 schema:sdLicense https://scigraph.springernature.com/explorer/license/
78 schema:sdPublisher N1f78cb6415ba4fd1b5f32bf8ef967f90
79 schema:url https://doi.org/10.1007/978-3-319-50115-4_41
80 sgo:license sg:explorer/license/
81 sgo:sdDataset chapters
82 rdf:type schema:Chapter
83 N0281656fb2da4d6092fab2618414a2f3 schema:familyName Kulić
84 schema:givenName Dana
85 rdf:type schema:Person
86 N1f78cb6415ba4fd1b5f32bf8ef967f90 schema:name Springer Nature - SN SciGraph project
87 rdf:type schema:Organization
88 N2ad7845baeb848669d6f2a11561f2814 schema:name dimensions_id
89 schema:value pub.1084722802
90 rdf:type schema:PropertyValue
91 N4bd7cec87138430aa5bad58301040601 rdf:first Nc70a57d8460a4cd696b234ed01b8cce0
92 rdf:rest N82c4155a928443f1bd8f45e3f115db00
93 N53bddd1cf1c24248885fb2155c456719 rdf:first sg:person.011310276047.71
94 rdf:rest N6e413b25b1f84523965f59f220d97d0a
95 N6e413b25b1f84523965f59f220d97d0a rdf:first sg:person.012443225372.65
96 rdf:rest Nc65fda8b820f4626a63c9632c163a89a
97 N7b0c2a7a22264781935f10d8318282c7 schema:isbn 978-3-319-50114-7
98 978-3-319-50115-4
99 schema:name 2016 International Symposium on Experimental Robotics
100 rdf:type schema:Book
101 N82c4155a928443f1bd8f45e3f115db00 rdf:first Nb6e9438abacd44a8a4c270e8d7f713ec
102 rdf:rest rdf:nil
103 N8423b5f7b4904749a28f2844e5e11d94 schema:name doi
104 schema:value 10.1007/978-3-319-50115-4_41
105 rdf:type schema:PropertyValue
106 N967df74dcc0343cba8a96cb54d960393 schema:name Springer Nature
107 rdf:type schema:Organisation
108 Nb13371998905446b8d9c1fa2f7211e90 rdf:first sg:person.07371177473.42
109 rdf:rest N53bddd1cf1c24248885fb2155c456719
110 Nb6e9438abacd44a8a4c270e8d7f713ec schema:familyName Venture
111 schema:givenName Gentiane
112 rdf:type schema:Person
113 Nc220b61c21354a2cb2ff6b171219db55 schema:familyName Nakamura
114 schema:givenName Yoshihiko
115 rdf:type schema:Person
116 Nc65fda8b820f4626a63c9632c163a89a rdf:first sg:person.014270043511.25
117 rdf:rest rdf:nil
118 Nc70a57d8460a4cd696b234ed01b8cce0 schema:familyName Khatib
119 schema:givenName Oussama
120 rdf:type schema:Person
121 Nd1b5f3b504944813995f6e645e2d4dbc rdf:first N0281656fb2da4d6092fab2618414a2f3
122 rdf:rest Necc798a6275341259a7204dd122b3f91
123 Necc798a6275341259a7204dd122b3f91 rdf:first Nc220b61c21354a2cb2ff6b171219db55
124 rdf:rest N4bd7cec87138430aa5bad58301040601
125 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
126 schema:name Information and Computing Sciences
127 rdf:type schema:DefinedTerm
128 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
129 schema:name Artificial Intelligence and Image Processing
130 rdf:type schema:DefinedTerm
131 sg:person.011310276047.71 schema:affiliation grid-institutes:grid.5963.9
132 schema:familyName Oliveira
133 schema:givenName Gabriel L.
134 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011310276047.71
135 rdf:type schema:Person
136 sg:person.012443225372.65 schema:affiliation grid-institutes:grid.5963.9
137 schema:familyName Brox
138 schema:givenName Thomas
139 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012443225372.65
140 rdf:type schema:Person
141 sg:person.014270043511.25 schema:affiliation grid-institutes:grid.5963.9
142 schema:familyName Burgard
143 schema:givenName Wolfram
144 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014270043511.25
145 rdf:type schema:Person
146 sg:person.07371177473.42 schema:affiliation grid-institutes:grid.5963.9
147 schema:familyName Valada
148 schema:givenName Abhinav
149 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07371177473.42
150 rdf:type schema:Person
151 grid-institutes:grid.5963.9 schema:alternateName Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany
152 schema:name Department of Computer Science, University of Freiburg, Freiburg Im Breisgau, Germany
153 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...