Neural Codes for Image Retrieval View Full Text


Ontology type: schema:Chapter      Open Access: True


Chapter Info

DATE

2014

AUTHORS

Artem Babenko , Anton Slesarev , Alexandr Chigorin , Victor Lempitsky

ABSTRACT

It has been shown that the activations invoked by an image within the top layers of a large convolutional neural network provide a high-level descriptor of the visual content of the image. In this paper, we investigate the use of such descriptors (neural codes) within the image retrieval application. In the experiments with several standard retrieval benchmarks, we establish that neural codes perform competitively even when the convolutional neural network has been trained for an unrelated classification task (e.g. Image-Net). We also evaluate the improvement in the retrieval performance of neural codes, when the network is retrained on a dataset of images that are similar to images encountered at test time.We further evaluate the performance of the compressed neural codes and show that a simple PCA compression provides very good short codes that give state-of-the-art accuracy on a number of datasets. In general, neural codes turn out to be much more resilient to such compression in comparison other state-of-the-art descriptors. Finally, we show that discriminative dimensionality reduction trained on a dataset of pairs of matched photographs improves the performance of PCA-compressed neural codes even further. Overall, our quantitative experiments demonstrate the promise of neural codes as visual descriptors for image retrieval. More... »

PAGES

584-599

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38

DOI

http://dx.doi.org/10.1007/978-3-319-10590-1_38

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1052242190


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Moscow Institute of Physics and Technology, Russia", 
          "id": "http://www.grid.ac/institutes/grid.18763.3b", 
          "name": [
            "Yandex, Russia", 
            "Moscow Institute of Physics and Technology, Russia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Babenko", 
        "givenName": "Artem", 
        "id": "sg:person.010177223377.36", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010177223377.36"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Yandex, Russia", 
          "id": "http://www.grid.ac/institutes/grid.484753.9", 
          "name": [
            "Yandex, Russia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Slesarev", 
        "givenName": "Anton", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Yandex, Russia", 
          "id": "http://www.grid.ac/institutes/grid.484753.9", 
          "name": [
            "Yandex, Russia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Chigorin", 
        "givenName": "Alexandr", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Skolkovo Institute of Science and Technology (Skoltech), Russia", 
          "id": "http://www.grid.ac/institutes/grid.454320.4", 
          "name": [
            "Skolkovo Institute of Science and Technology (Skoltech), Russia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Lempitsky", 
        "givenName": "Victor", 
        "id": "sg:person.0600611416.12", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0600611416.12"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2014", 
    "datePublishedReg": "2014-01-01", 
    "description": "It has been shown that the activations invoked by an image within the top layers of a large convolutional neural network provide a high-level descriptor of the visual content of the image. In this paper, we investigate the use of such descriptors (neural codes) within the image retrieval application. In the experiments with several standard retrieval benchmarks, we establish that neural codes perform competitively even when the convolutional neural network has been trained for an unrelated classification task (e.g. Image-Net). We also evaluate the improvement in the retrieval performance of neural codes, when the network is retrained on a dataset of images that are similar to images encountered at test time.We further evaluate the performance of the compressed neural codes and show that a simple PCA compression provides very good short codes that give state-of-the-art accuracy on a number of datasets. In general, neural codes turn out to be much more resilient to such compression in comparison other state-of-the-art descriptors. Finally, we show that discriminative dimensionality reduction trained on a dataset of pairs of matched photographs improves the performance of PCA-compressed neural codes even further. Overall, our quantitative experiments demonstrate the promise of neural codes as visual descriptors for image retrieval.", 
    "editor": [
      {
        "familyName": "Fleet", 
        "givenName": "David", 
        "type": "Person"
      }, 
      {
        "familyName": "Pajdla", 
        "givenName": "Tomas", 
        "type": "Person"
      }, 
      {
        "familyName": "Schiele", 
        "givenName": "Bernt", 
        "type": "Person"
      }, 
      {
        "familyName": "Tuytelaars", 
        "givenName": "Tinne", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-10590-1_38", 
    "inLanguage": "en", 
    "isAccessibleForFree": true, 
    "isPartOf": {
      "isbn": [
        "978-3-319-10589-5", 
        "978-3-319-10590-1"
      ], 
      "name": "Computer Vision \u2013 ECCV 2014", 
      "type": "Book"
    }, 
    "keywords": [
      "convolutional neural network", 
      "image retrieval", 
      "neural network", 
      "large convolutional neural network", 
      "high-level descriptors", 
      "image retrieval applications", 
      "dataset of images", 
      "dataset of pairs", 
      "number of datasets", 
      "performance of PCA", 
      "retrieval applications", 
      "retrieval benchmarks", 
      "visual descriptors", 
      "visual content", 
      "art accuracy", 
      "retrieval performance", 
      "classification task", 
      "neural code", 
      "art descriptors", 
      "PCA compression", 
      "dimensionality reduction", 
      "short codes", 
      "quantitative experiments", 
      "datasets", 
      "such compression", 
      "network", 
      "such descriptors", 
      "descriptors", 
      "images", 
      "code", 
      "retrieval", 
      "test time", 
      "performance", 
      "compression", 
      "benchmarks", 
      "task", 
      "accuracy", 
      "PCA", 
      "applications", 
      "experiments", 
      "state", 
      "improvement", 
      "number", 
      "time", 
      "use", 
      "top layer", 
      "photographs", 
      "pairs", 
      "promise", 
      "layer", 
      "content", 
      "comparison", 
      "reduction", 
      "activation", 
      "paper", 
      "standard retrieval benchmarks", 
      "unrelated classification task", 
      "simple PCA compression", 
      "good short codes"
    ], 
    "name": "Neural Codes for Image Retrieval", 
    "pagination": "584-599", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1052242190"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-10590-1_38"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-10590-1_38", 
      "https://app.dimensions.ai/details/publication/pub.1052242190"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2021-11-01T19:03", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20211101/entities/gbq_results/chapter/chapter_93.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-319-10590-1_38"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38'


 

This table displays all metadata directly associated to this object as RDF triples.

160 TRIPLES      23 PREDICATES      85 URIs      78 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-10590-1_38 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Neac778040f05486daf69be5bb389aa03
4 schema:datePublished 2014
5 schema:datePublishedReg 2014-01-01
6 schema:description It has been shown that the activations invoked by an image within the top layers of a large convolutional neural network provide a high-level descriptor of the visual content of the image. In this paper, we investigate the use of such descriptors (neural codes) within the image retrieval application. In the experiments with several standard retrieval benchmarks, we establish that neural codes perform competitively even when the convolutional neural network has been trained for an unrelated classification task (e.g. Image-Net). We also evaluate the improvement in the retrieval performance of neural codes, when the network is retrained on a dataset of images that are similar to images encountered at test time.We further evaluate the performance of the compressed neural codes and show that a simple PCA compression provides very good short codes that give state-of-the-art accuracy on a number of datasets. In general, neural codes turn out to be much more resilient to such compression in comparison other state-of-the-art descriptors. Finally, we show that discriminative dimensionality reduction trained on a dataset of pairs of matched photographs improves the performance of PCA-compressed neural codes even further. Overall, our quantitative experiments demonstrate the promise of neural codes as visual descriptors for image retrieval.
7 schema:editor N7aff1c9ae3f24ae58760fc8eff4d8cef
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree true
11 schema:isPartOf N70c61678e6224ae68c7f00e6489195d2
12 schema:keywords PCA
13 PCA compression
14 accuracy
15 activation
16 applications
17 art accuracy
18 art descriptors
19 benchmarks
20 classification task
21 code
22 comparison
23 compression
24 content
25 convolutional neural network
26 dataset of images
27 dataset of pairs
28 datasets
29 descriptors
30 dimensionality reduction
31 experiments
32 good short codes
33 high-level descriptors
34 image retrieval
35 image retrieval applications
36 images
37 improvement
38 large convolutional neural network
39 layer
40 network
41 neural code
42 neural network
43 number
44 number of datasets
45 pairs
46 paper
47 performance
48 performance of PCA
49 photographs
50 promise
51 quantitative experiments
52 reduction
53 retrieval
54 retrieval applications
55 retrieval benchmarks
56 retrieval performance
57 short codes
58 simple PCA compression
59 standard retrieval benchmarks
60 state
61 such compression
62 such descriptors
63 task
64 test time
65 time
66 top layer
67 unrelated classification task
68 use
69 visual content
70 visual descriptors
71 schema:name Neural Codes for Image Retrieval
72 schema:pagination 584-599
73 schema:productId Nca336465a416438897b97719bc2e0995
74 Ne28fb038be394ac4999470147c7ac71c
75 schema:publisher Naa061f9d09454f6ab4dfdf9a79136e65
76 schema:sameAs https://app.dimensions.ai/details/publication/pub.1052242190
77 https://doi.org/10.1007/978-3-319-10590-1_38
78 schema:sdDatePublished 2021-11-01T19:03
79 schema:sdLicense https://scigraph.springernature.com/explorer/license/
80 schema:sdPublisher N97abcca78ee64ba1b5bba2a88f06763d
81 schema:url https://doi.org/10.1007/978-3-319-10590-1_38
82 sgo:license sg:explorer/license/
83 sgo:sdDataset chapters
84 rdf:type schema:Chapter
85 N0bd83ae8762a4149bdfac043896d4711 schema:familyName Fleet
86 schema:givenName David
87 rdf:type schema:Person
88 N1e9ac47b22e54596a2a3f7e26cf545bc schema:familyName Pajdla
89 schema:givenName Tomas
90 rdf:type schema:Person
91 N275e03f8e16045139d696816ac4e9020 rdf:first N1e9ac47b22e54596a2a3f7e26cf545bc
92 rdf:rest N5ffebed3b6f2460896808397d88a1c83
93 N523caa4ab40e4e97a4e3c7fb72d36929 schema:affiliation grid-institutes:grid.484753.9
94 schema:familyName Slesarev
95 schema:givenName Anton
96 rdf:type schema:Person
97 N5e63a81c7b614462a51a90d1422b6371 rdf:first Na4f3e4b43cc24f40a0d115da1e61171e
98 rdf:rest rdf:nil
99 N5ffebed3b6f2460896808397d88a1c83 rdf:first N9f0cb8f960d54e02ac93cbc7040429d5
100 rdf:rest N5e63a81c7b614462a51a90d1422b6371
101 N68c1971617254da7bc5da797e35e996d rdf:first sg:person.0600611416.12
102 rdf:rest rdf:nil
103 N70c61678e6224ae68c7f00e6489195d2 schema:isbn 978-3-319-10589-5
104 978-3-319-10590-1
105 schema:name Computer Vision – ECCV 2014
106 rdf:type schema:Book
107 N7aff1c9ae3f24ae58760fc8eff4d8cef rdf:first N0bd83ae8762a4149bdfac043896d4711
108 rdf:rest N275e03f8e16045139d696816ac4e9020
109 N942787146e06406a923a50cd80d0f204 schema:affiliation grid-institutes:grid.484753.9
110 schema:familyName Chigorin
111 schema:givenName Alexandr
112 rdf:type schema:Person
113 N97abcca78ee64ba1b5bba2a88f06763d schema:name Springer Nature - SN SciGraph project
114 rdf:type schema:Organization
115 N9f0cb8f960d54e02ac93cbc7040429d5 schema:familyName Schiele
116 schema:givenName Bernt
117 rdf:type schema:Person
118 Na4f3e4b43cc24f40a0d115da1e61171e schema:familyName Tuytelaars
119 schema:givenName Tinne
120 rdf:type schema:Person
121 Naa061f9d09454f6ab4dfdf9a79136e65 schema:name Springer Nature
122 rdf:type schema:Organisation
123 Nb0ed72e58c6446d4b85c64c46b131260 rdf:first N942787146e06406a923a50cd80d0f204
124 rdf:rest N68c1971617254da7bc5da797e35e996d
125 Nc31b8cc8979947d3bce538bc66073bd6 rdf:first N523caa4ab40e4e97a4e3c7fb72d36929
126 rdf:rest Nb0ed72e58c6446d4b85c64c46b131260
127 Nca336465a416438897b97719bc2e0995 schema:name doi
128 schema:value 10.1007/978-3-319-10590-1_38
129 rdf:type schema:PropertyValue
130 Ne28fb038be394ac4999470147c7ac71c schema:name dimensions_id
131 schema:value pub.1052242190
132 rdf:type schema:PropertyValue
133 Neac778040f05486daf69be5bb389aa03 rdf:first sg:person.010177223377.36
134 rdf:rest Nc31b8cc8979947d3bce538bc66073bd6
135 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
136 schema:name Information and Computing Sciences
137 rdf:type schema:DefinedTerm
138 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
139 schema:name Artificial Intelligence and Image Processing
140 rdf:type schema:DefinedTerm
141 sg:person.010177223377.36 schema:affiliation grid-institutes:grid.18763.3b
142 schema:familyName Babenko
143 schema:givenName Artem
144 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010177223377.36
145 rdf:type schema:Person
146 sg:person.0600611416.12 schema:affiliation grid-institutes:grid.454320.4
147 schema:familyName Lempitsky
148 schema:givenName Victor
149 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0600611416.12
150 rdf:type schema:Person
151 grid-institutes:grid.18763.3b schema:alternateName Moscow Institute of Physics and Technology, Russia
152 schema:name Moscow Institute of Physics and Technology, Russia
153 Yandex, Russia
154 rdf:type schema:Organization
155 grid-institutes:grid.454320.4 schema:alternateName Skolkovo Institute of Science and Technology (Skoltech), Russia
156 schema:name Skolkovo Institute of Science and Technology (Skoltech), Russia
157 rdf:type schema:Organization
158 grid-institutes:grid.484753.9 schema:alternateName Yandex, Russia
159 schema:name Yandex, Russia
160 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...