Visualizations of Deep Neural Networks in Computer Vision: A Survey View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2017-05-10

AUTHORS

Christin Seifert , Aisha Aamir , Aparna Balagopalan , Dhruv Jain , Abhinav Sharma , Sebastian Grottel , Stefan Gumhold

ABSTRACT

In recent years, Deep Neural Networks (DNNs) have been shown to outperform the state-of-the-art in multiple areas, such as visual object recognition, genomics and speech recognition. Due to the distributed encodings of information, DNNs are hard to understand and interpret. To this end, visualizations have been used to understand how deep architecture work in general, what different layers of the network encode, what the limitations of the trained model was and how to interactively collect user feedback. In this chapter, we provide a survey of visualizations of DNNs in the field of computer vision. We define a classification scheme describing visualization goals and methods as well as the application areas. This survey gives an overview of what can be learned from visualizing DNNs and which visualization methods were used to gain which insights. We found that most papers use Pixel Displays to show neuron activations. However, recently more sophisticated visualizations like interactive node-link diagrams were proposed. The presented overview can serve as a guideline when applying visualizations while designing DNNs. More... »

PAGES

123-144

Book

TITLE

Transparent Data Mining for Big and Small Data

ISBN

978-3-319-54023-8
978-3-319-54024-5

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-54024-5_6

DOI

http://dx.doi.org/10.1007/978-3-319-54024-5_6

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1085381604


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Technische Universit\u00e4t Dresden, Dresden, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4488.0", 
          "name": [
            "Technische Universit\u00e4t Dresden, Dresden, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Seifert", 
        "givenName": "Christin", 
        "id": "sg:person.010257616672.34", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010257616672.34"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technische Universit\u00e4t Dresden, Dresden, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4488.0", 
          "name": [
            "Technische Universit\u00e4t Dresden, Dresden, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Aamir", 
        "givenName": "Aisha", 
        "id": "sg:person.012207204401.97", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012207204401.97"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technische Universit\u00e4t Dresden, Dresden, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4488.0", 
          "name": [
            "Technische Universit\u00e4t Dresden, Dresden, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Balagopalan", 
        "givenName": "Aparna", 
        "id": "sg:person.014377526001.80", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014377526001.80"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technische Universit\u00e4t Dresden, Dresden, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4488.0", 
          "name": [
            "Technische Universit\u00e4t Dresden, Dresden, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Jain", 
        "givenName": "Dhruv", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technische Universit\u00e4t Dresden, Dresden, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4488.0", 
          "name": [
            "Technische Universit\u00e4t Dresden, Dresden, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Sharma", 
        "givenName": "Abhinav", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technische Universit\u00e4t Dresden, Dresden, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4488.0", 
          "name": [
            "Technische Universit\u00e4t Dresden, Dresden, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Grottel", 
        "givenName": "Sebastian", 
        "id": "sg:person.01146556151.00", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01146556151.00"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technische Universit\u00e4t Dresden, Dresden, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4488.0", 
          "name": [
            "Technische Universit\u00e4t Dresden, Dresden, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Gumhold", 
        "givenName": "Stefan", 
        "id": "sg:person.011202146505.82", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011202146505.82"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2017-05-10", 
    "datePublishedReg": "2017-05-10", 
    "description": "In recent years, Deep Neural Networks (DNNs) have been shown to outperform the state-of-the-art in multiple areas, such as visual object recognition, genomics and speech recognition. Due to the distributed encodings of information, DNNs are hard to understand and interpret. To this end, visualizations have been used to understand how deep architecture work in general, what different layers of the network encode, what the limitations of the trained model was and how to interactively collect user feedback. In this chapter, we provide a survey of visualizations of DNNs in the field of computer vision. We define a classification scheme describing visualization goals and methods as well as the application areas. This survey gives an overview of what can be learned from visualizing DNNs and which visualization methods were used to gain which insights. We found that most papers use Pixel Displays to show neuron activations. However, recently more sophisticated visualizations like interactive node-link diagrams were proposed. The presented overview can serve as a guideline when applying visualizations while designing DNNs.", 
    "editor": [
      {
        "familyName": "Cerquitelli", 
        "givenName": "Tania", 
        "type": "Person"
      }, 
      {
        "familyName": "Quercia", 
        "givenName": "Daniele", 
        "type": "Person"
      }, 
      {
        "familyName": "Pasquale", 
        "givenName": "Frank", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-54024-5_6", 
    "inLanguage": "en", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-319-54023-8", 
        "978-3-319-54024-5"
      ], 
      "name": "Transparent Data Mining for Big and Small Data", 
      "type": "Book"
    }, 
    "keywords": [
      "deep neural networks", 
      "computer vision", 
      "neural network", 
      "survey of visualization", 
      "node-link diagrams", 
      "visualization goals", 
      "sophisticated visualization", 
      "visual object recognition", 
      "user feedback", 
      "speech recognition", 
      "object recognition", 
      "architecture work", 
      "application areas", 
      "visualization method", 
      "network", 
      "visualization", 
      "different layers", 
      "classification scheme", 
      "pixel display", 
      "vision", 
      "recognition", 
      "neuron activation", 
      "recent years", 
      "encoding", 
      "scheme", 
      "presented overview", 
      "information", 
      "most papers", 
      "multiple areas", 
      "display", 
      "overview", 
      "art", 
      "feedback", 
      "method", 
      "goal", 
      "work", 
      "limitations", 
      "model", 
      "area", 
      "encoding of information", 
      "diagram", 
      "field", 
      "end", 
      "survey", 
      "state", 
      "chapter", 
      "layer", 
      "genomics", 
      "insights", 
      "guidelines", 
      "years", 
      "paper", 
      "activation"
    ], 
    "name": "Visualizations of Deep Neural Networks in Computer Vision: A Survey", 
    "pagination": "123-144", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1085381604"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-54024-5_6"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-54024-5_6", 
      "https://app.dimensions.ai/details/publication/pub.1085381604"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-06-01T22:34", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220601/entities/gbq_results/chapter/chapter_423.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-319-54024-5_6"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-54024-5_6'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-54024-5_6'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-54024-5_6'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-54024-5_6'


 

This table displays all metadata directly associated to this object as RDF triples.

163 TRIPLES      23 PREDICATES      77 URIs      70 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-54024-5_6 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Nc5179d38250a4efbba027e958162ff8a
4 schema:datePublished 2017-05-10
5 schema:datePublishedReg 2017-05-10
6 schema:description In recent years, Deep Neural Networks (DNNs) have been shown to outperform the state-of-the-art in multiple areas, such as visual object recognition, genomics and speech recognition. Due to the distributed encodings of information, DNNs are hard to understand and interpret. To this end, visualizations have been used to understand how deep architecture work in general, what different layers of the network encode, what the limitations of the trained model was and how to interactively collect user feedback. In this chapter, we provide a survey of visualizations of DNNs in the field of computer vision. We define a classification scheme describing visualization goals and methods as well as the application areas. This survey gives an overview of what can be learned from visualizing DNNs and which visualization methods were used to gain which insights. We found that most papers use Pixel Displays to show neuron activations. However, recently more sophisticated visualizations like interactive node-link diagrams were proposed. The presented overview can serve as a guideline when applying visualizations while designing DNNs.
7 schema:editor Nab2a84e0fecd437497208f669582e6bf
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree false
11 schema:isPartOf N65f8d9e7ad344e0985863956ce2a46a1
12 schema:keywords activation
13 application areas
14 architecture work
15 area
16 art
17 chapter
18 classification scheme
19 computer vision
20 deep neural networks
21 diagram
22 different layers
23 display
24 encoding
25 encoding of information
26 end
27 feedback
28 field
29 genomics
30 goal
31 guidelines
32 information
33 insights
34 layer
35 limitations
36 method
37 model
38 most papers
39 multiple areas
40 network
41 neural network
42 neuron activation
43 node-link diagrams
44 object recognition
45 overview
46 paper
47 pixel display
48 presented overview
49 recent years
50 recognition
51 scheme
52 sophisticated visualization
53 speech recognition
54 state
55 survey
56 survey of visualization
57 user feedback
58 vision
59 visual object recognition
60 visualization
61 visualization goals
62 visualization method
63 work
64 years
65 schema:name Visualizations of Deep Neural Networks in Computer Vision: A Survey
66 schema:pagination 123-144
67 schema:productId N089b8ecd8b924d81af4928b9be934380
68 N404842d5dd6b456aaa0a138978702db8
69 schema:publisher N5b65e9c9d5d94a4f8103e450e1a3340c
70 schema:sameAs https://app.dimensions.ai/details/publication/pub.1085381604
71 https://doi.org/10.1007/978-3-319-54024-5_6
72 schema:sdDatePublished 2022-06-01T22:34
73 schema:sdLicense https://scigraph.springernature.com/explorer/license/
74 schema:sdPublisher Nf7e4f929183f4e11a29f78f94b92142b
75 schema:url https://doi.org/10.1007/978-3-319-54024-5_6
76 sgo:license sg:explorer/license/
77 sgo:sdDataset chapters
78 rdf:type schema:Chapter
79 N089b8ecd8b924d81af4928b9be934380 schema:name dimensions_id
80 schema:value pub.1085381604
81 rdf:type schema:PropertyValue
82 N0a03a129e0ba4bf4bc31254cbf82410f rdf:first sg:person.014377526001.80
83 rdf:rest N0ef523ed989947ada907d8cdfcfdec4a
84 N0ef523ed989947ada907d8cdfcfdec4a rdf:first N369f5d023a1148089a8910bbec1aa39f
85 rdf:rest N95ca13874145468191d0bb537b0e2a44
86 N1160388198684f87bcf2d78c4b92d3b2 schema:familyName Cerquitelli
87 schema:givenName Tania
88 rdf:type schema:Person
89 N3375bd40502e4a36a6cffb9180d5e38b rdf:first sg:person.011202146505.82
90 rdf:rest rdf:nil
91 N369f5d023a1148089a8910bbec1aa39f schema:affiliation grid-institutes:grid.4488.0
92 schema:familyName Jain
93 schema:givenName Dhruv
94 rdf:type schema:Person
95 N404842d5dd6b456aaa0a138978702db8 schema:name doi
96 schema:value 10.1007/978-3-319-54024-5_6
97 rdf:type schema:PropertyValue
98 N5b65e9c9d5d94a4f8103e450e1a3340c schema:name Springer Nature
99 rdf:type schema:Organisation
100 N65f8d9e7ad344e0985863956ce2a46a1 schema:isbn 978-3-319-54023-8
101 978-3-319-54024-5
102 schema:name Transparent Data Mining for Big and Small Data
103 rdf:type schema:Book
104 N6add1b6901cd4b789416246f07de86c4 rdf:first N8db2409f0f2f4b78bc7d80f95d37c54a
105 rdf:rest N7a33887e711c48c587b3d20d74fc98b4
106 N780ce70e5b2d456eb670dc2ba741816b rdf:first sg:person.01146556151.00
107 rdf:rest N3375bd40502e4a36a6cffb9180d5e38b
108 N78f88bf1d8a8489da4a9a1dbb5d24f76 schema:affiliation grid-institutes:grid.4488.0
109 schema:familyName Sharma
110 schema:givenName Abhinav
111 rdf:type schema:Person
112 N7a33887e711c48c587b3d20d74fc98b4 rdf:first Nc8376037d5574b4a928913dbe2838370
113 rdf:rest rdf:nil
114 N8db2409f0f2f4b78bc7d80f95d37c54a schema:familyName Quercia
115 schema:givenName Daniele
116 rdf:type schema:Person
117 N95ca13874145468191d0bb537b0e2a44 rdf:first N78f88bf1d8a8489da4a9a1dbb5d24f76
118 rdf:rest N780ce70e5b2d456eb670dc2ba741816b
119 Nab2a84e0fecd437497208f669582e6bf rdf:first N1160388198684f87bcf2d78c4b92d3b2
120 rdf:rest N6add1b6901cd4b789416246f07de86c4
121 Nbdb3dd03a38b47c6853cd9f2d738d315 rdf:first sg:person.012207204401.97
122 rdf:rest N0a03a129e0ba4bf4bc31254cbf82410f
123 Nc5179d38250a4efbba027e958162ff8a rdf:first sg:person.010257616672.34
124 rdf:rest Nbdb3dd03a38b47c6853cd9f2d738d315
125 Nc8376037d5574b4a928913dbe2838370 schema:familyName Pasquale
126 schema:givenName Frank
127 rdf:type schema:Person
128 Nf7e4f929183f4e11a29f78f94b92142b schema:name Springer Nature - SN SciGraph project
129 rdf:type schema:Organization
130 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
131 schema:name Information and Computing Sciences
132 rdf:type schema:DefinedTerm
133 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
134 schema:name Artificial Intelligence and Image Processing
135 rdf:type schema:DefinedTerm
136 sg:person.010257616672.34 schema:affiliation grid-institutes:grid.4488.0
137 schema:familyName Seifert
138 schema:givenName Christin
139 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010257616672.34
140 rdf:type schema:Person
141 sg:person.011202146505.82 schema:affiliation grid-institutes:grid.4488.0
142 schema:familyName Gumhold
143 schema:givenName Stefan
144 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011202146505.82
145 rdf:type schema:Person
146 sg:person.01146556151.00 schema:affiliation grid-institutes:grid.4488.0
147 schema:familyName Grottel
148 schema:givenName Sebastian
149 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01146556151.00
150 rdf:type schema:Person
151 sg:person.012207204401.97 schema:affiliation grid-institutes:grid.4488.0
152 schema:familyName Aamir
153 schema:givenName Aisha
154 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012207204401.97
155 rdf:type schema:Person
156 sg:person.014377526001.80 schema:affiliation grid-institutes:grid.4488.0
157 schema:familyName Balagopalan
158 schema:givenName Aparna
159 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014377526001.80
160 rdf:type schema:Person
161 grid-institutes:grid.4488.0 schema:alternateName Technische Universität Dresden, Dresden, Germany
162 schema:name Technische Universität Dresden, Dresden, Germany
163 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...