Neural Codes for Image Retrieval View Full Text


Ontology type: schema:Chapter      Open Access: True


Chapter Info

DATE

2014

AUTHORS

Artem Babenko , Anton Slesarev , Alexandr Chigorin , Victor Lempitsky

ABSTRACT

It has been shown that the activations invoked by an image within the top layers of a large convolutional neural network provide a high-level descriptor of the visual content of the image. In this paper, we investigate the use of such descriptors (neural codes) within the image retrieval application. In the experiments with several standard retrieval benchmarks, we establish that neural codes perform competitively even when the convolutional neural network has been trained for an unrelated classification task (e.g. Image-Net). We also evaluate the improvement in the retrieval performance of neural codes, when the network is retrained on a dataset of images that are similar to images encountered at test time.We further evaluate the performance of the compressed neural codes and show that a simple PCA compression provides very good short codes that give state-of-the-art accuracy on a number of datasets. In general, neural codes turn out to be much more resilient to such compression in comparison other state-of-the-art descriptors. Finally, we show that discriminative dimensionality reduction trained on a dataset of pairs of matched photographs improves the performance of PCA-compressed neural codes even further. Overall, our quantitative experiments demonstrate the promise of neural codes as visual descriptors for image retrieval. More... »

PAGES

584-599

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38

DOI

http://dx.doi.org/10.1007/978-3-319-10590-1_38

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1052242190


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Moscow Institute of Physics and Technology, Russia", 
          "id": "http://www.grid.ac/institutes/grid.18763.3b", 
          "name": [
            "Yandex, Russia", 
            "Moscow Institute of Physics and Technology, Russia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Babenko", 
        "givenName": "Artem", 
        "id": "sg:person.010177223377.36", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010177223377.36"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Yandex, Russia", 
          "id": "http://www.grid.ac/institutes/grid.484753.9", 
          "name": [
            "Yandex, Russia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Slesarev", 
        "givenName": "Anton", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Yandex, Russia", 
          "id": "http://www.grid.ac/institutes/grid.484753.9", 
          "name": [
            "Yandex, Russia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Chigorin", 
        "givenName": "Alexandr", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Skolkovo Institute of Science and Technology (Skoltech), Russia", 
          "id": "http://www.grid.ac/institutes/grid.454320.4", 
          "name": [
            "Skolkovo Institute of Science and Technology (Skoltech), Russia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Lempitsky", 
        "givenName": "Victor", 
        "id": "sg:person.0600611416.12", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0600611416.12"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2014", 
    "datePublishedReg": "2014-01-01", 
    "description": "It has been shown that the activations invoked by an image within the top layers of a large convolutional neural network provide a high-level descriptor of the visual content of the image. In this paper, we investigate the use of such descriptors (neural codes) within the image retrieval application. In the experiments with several standard retrieval benchmarks, we establish that neural codes perform competitively even when the convolutional neural network has been trained for an unrelated classification task (e.g. Image-Net). We also evaluate the improvement in the retrieval performance of neural codes, when the network is retrained on a dataset of images that are similar to images encountered at test time.We further evaluate the performance of the compressed neural codes and show that a simple PCA compression provides very good short codes that give state-of-the-art accuracy on a number of datasets. In general, neural codes turn out to be much more resilient to such compression in comparison other state-of-the-art descriptors. Finally, we show that discriminative dimensionality reduction trained on a dataset of pairs of matched photographs improves the performance of PCA-compressed neural codes even further. Overall, our quantitative experiments demonstrate the promise of neural codes as visual descriptors for image retrieval.", 
    "editor": [
      {
        "familyName": "Fleet", 
        "givenName": "David", 
        "type": "Person"
      }, 
      {
        "familyName": "Pajdla", 
        "givenName": "Tomas", 
        "type": "Person"
      }, 
      {
        "familyName": "Schiele", 
        "givenName": "Bernt", 
        "type": "Person"
      }, 
      {
        "familyName": "Tuytelaars", 
        "givenName": "Tinne", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-10590-1_38", 
    "inLanguage": "en", 
    "isAccessibleForFree": true, 
    "isPartOf": {
      "isbn": [
        "978-3-319-10589-5", 
        "978-3-319-10590-1"
      ], 
      "name": "Computer Vision \u2013 ECCV 2014", 
      "type": "Book"
    }, 
    "keywords": [
      "convolutional neural network", 
      "image retrieval", 
      "neural network", 
      "large convolutional neural network", 
      "high-level descriptors", 
      "image retrieval applications", 
      "dataset of images", 
      "dataset of pairs", 
      "number of datasets", 
      "performance of PCA", 
      "retrieval applications", 
      "retrieval benchmarks", 
      "visual descriptors", 
      "visual content", 
      "art accuracy", 
      "retrieval performance", 
      "classification task", 
      "neural code", 
      "art descriptors", 
      "PCA compression", 
      "dimensionality reduction", 
      "short codes", 
      "quantitative experiments", 
      "datasets", 
      "such compression", 
      "network", 
      "such descriptors", 
      "descriptors", 
      "images", 
      "code", 
      "retrieval", 
      "test time", 
      "performance", 
      "compression", 
      "benchmarks", 
      "task", 
      "accuracy", 
      "PCA", 
      "applications", 
      "experiments", 
      "state", 
      "improvement", 
      "number", 
      "time", 
      "use", 
      "top layer", 
      "photographs", 
      "pairs", 
      "promise", 
      "layer", 
      "content", 
      "comparison", 
      "reduction", 
      "activation", 
      "paper", 
      "standard retrieval benchmarks", 
      "unrelated classification task", 
      "simple PCA compression", 
      "good short codes"
    ], 
    "name": "Neural Codes for Image Retrieval", 
    "pagination": "584-599", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1052242190"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-10590-1_38"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-10590-1_38", 
      "https://app.dimensions.ai/details/publication/pub.1052242190"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2021-12-01T19:59", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20211201/entities/gbq_results/chapter/chapter_204.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-319-10590-1_38"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-10590-1_38'


 

This table displays all metadata directly associated to this object as RDF triples.

160 TRIPLES      23 PREDICATES      85 URIs      78 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-10590-1_38 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Nb8e678319b484a81827a14370bfc8191
4 schema:datePublished 2014
5 schema:datePublishedReg 2014-01-01
6 schema:description It has been shown that the activations invoked by an image within the top layers of a large convolutional neural network provide a high-level descriptor of the visual content of the image. In this paper, we investigate the use of such descriptors (neural codes) within the image retrieval application. In the experiments with several standard retrieval benchmarks, we establish that neural codes perform competitively even when the convolutional neural network has been trained for an unrelated classification task (e.g. Image-Net). We also evaluate the improvement in the retrieval performance of neural codes, when the network is retrained on a dataset of images that are similar to images encountered at test time.We further evaluate the performance of the compressed neural codes and show that a simple PCA compression provides very good short codes that give state-of-the-art accuracy on a number of datasets. In general, neural codes turn out to be much more resilient to such compression in comparison other state-of-the-art descriptors. Finally, we show that discriminative dimensionality reduction trained on a dataset of pairs of matched photographs improves the performance of PCA-compressed neural codes even further. Overall, our quantitative experiments demonstrate the promise of neural codes as visual descriptors for image retrieval.
7 schema:editor N366e15b1554347848ca42e0bf2d9e718
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree true
11 schema:isPartOf Nf7337f31c0d341b98a9a77e66bf3fcfd
12 schema:keywords PCA
13 PCA compression
14 accuracy
15 activation
16 applications
17 art accuracy
18 art descriptors
19 benchmarks
20 classification task
21 code
22 comparison
23 compression
24 content
25 convolutional neural network
26 dataset of images
27 dataset of pairs
28 datasets
29 descriptors
30 dimensionality reduction
31 experiments
32 good short codes
33 high-level descriptors
34 image retrieval
35 image retrieval applications
36 images
37 improvement
38 large convolutional neural network
39 layer
40 network
41 neural code
42 neural network
43 number
44 number of datasets
45 pairs
46 paper
47 performance
48 performance of PCA
49 photographs
50 promise
51 quantitative experiments
52 reduction
53 retrieval
54 retrieval applications
55 retrieval benchmarks
56 retrieval performance
57 short codes
58 simple PCA compression
59 standard retrieval benchmarks
60 state
61 such compression
62 such descriptors
63 task
64 test time
65 time
66 top layer
67 unrelated classification task
68 use
69 visual content
70 visual descriptors
71 schema:name Neural Codes for Image Retrieval
72 schema:pagination 584-599
73 schema:productId N68c87687869c428c903dd099c8f322ac
74 Neb0b26def11f4fa6afa0cbf15681d278
75 schema:publisher N8132bad063354af788dd2e2f665d20da
76 schema:sameAs https://app.dimensions.ai/details/publication/pub.1052242190
77 https://doi.org/10.1007/978-3-319-10590-1_38
78 schema:sdDatePublished 2021-12-01T19:59
79 schema:sdLicense https://scigraph.springernature.com/explorer/license/
80 schema:sdPublisher N80759a0c14e340fa86475f5b403def42
81 schema:url https://doi.org/10.1007/978-3-319-10590-1_38
82 sgo:license sg:explorer/license/
83 sgo:sdDataset chapters
84 rdf:type schema:Chapter
85 N0c37e646f7d041a891d9810283e101ba rdf:first N9846d439a9924764b688ced74b074b80
86 rdf:rest N0ec233ab500245ea85a149d797fe138a
87 N0ec233ab500245ea85a149d797fe138a rdf:first sg:person.0600611416.12
88 rdf:rest rdf:nil
89 N35ce923a148c4ea19ec3950f1d1a4472 rdf:first Nc3e97181767345359bb75f63004d5bc0
90 rdf:rest N6a565d77458341b097db964e4ae66cb8
91 N366e15b1554347848ca42e0bf2d9e718 rdf:first N73189896441e4ca39f6f0411f1ed31c8
92 rdf:rest N35ce923a148c4ea19ec3950f1d1a4472
93 N68c87687869c428c903dd099c8f322ac schema:name dimensions_id
94 schema:value pub.1052242190
95 rdf:type schema:PropertyValue
96 N6a565d77458341b097db964e4ae66cb8 rdf:first Ne5e8b064bae541daa2942d1794e580cb
97 rdf:rest Nbb115b98e79a4bcda11d0b1d1d8518b1
98 N73189896441e4ca39f6f0411f1ed31c8 schema:familyName Fleet
99 schema:givenName David
100 rdf:type schema:Person
101 N74b9a3d929714a34aba76997d52503c0 schema:familyName Tuytelaars
102 schema:givenName Tinne
103 rdf:type schema:Person
104 N80395cac399743179b173b18e35feadf schema:affiliation grid-institutes:grid.484753.9
105 schema:familyName Slesarev
106 schema:givenName Anton
107 rdf:type schema:Person
108 N80759a0c14e340fa86475f5b403def42 schema:name Springer Nature - SN SciGraph project
109 rdf:type schema:Organization
110 N8132bad063354af788dd2e2f665d20da schema:name Springer Nature
111 rdf:type schema:Organisation
112 N9846d439a9924764b688ced74b074b80 schema:affiliation grid-institutes:grid.484753.9
113 schema:familyName Chigorin
114 schema:givenName Alexandr
115 rdf:type schema:Person
116 Nb8e678319b484a81827a14370bfc8191 rdf:first sg:person.010177223377.36
117 rdf:rest Nc91ce27a642e4aa3aac7bdb6c381b30a
118 Nbb115b98e79a4bcda11d0b1d1d8518b1 rdf:first N74b9a3d929714a34aba76997d52503c0
119 rdf:rest rdf:nil
120 Nc3e97181767345359bb75f63004d5bc0 schema:familyName Pajdla
121 schema:givenName Tomas
122 rdf:type schema:Person
123 Nc91ce27a642e4aa3aac7bdb6c381b30a rdf:first N80395cac399743179b173b18e35feadf
124 rdf:rest N0c37e646f7d041a891d9810283e101ba
125 Ne5e8b064bae541daa2942d1794e580cb schema:familyName Schiele
126 schema:givenName Bernt
127 rdf:type schema:Person
128 Neb0b26def11f4fa6afa0cbf15681d278 schema:name doi
129 schema:value 10.1007/978-3-319-10590-1_38
130 rdf:type schema:PropertyValue
131 Nf7337f31c0d341b98a9a77e66bf3fcfd schema:isbn 978-3-319-10589-5
132 978-3-319-10590-1
133 schema:name Computer Vision – ECCV 2014
134 rdf:type schema:Book
135 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
136 schema:name Information and Computing Sciences
137 rdf:type schema:DefinedTerm
138 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
139 schema:name Artificial Intelligence and Image Processing
140 rdf:type schema:DefinedTerm
141 sg:person.010177223377.36 schema:affiliation grid-institutes:grid.18763.3b
142 schema:familyName Babenko
143 schema:givenName Artem
144 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010177223377.36
145 rdf:type schema:Person
146 sg:person.0600611416.12 schema:affiliation grid-institutes:grid.454320.4
147 schema:familyName Lempitsky
148 schema:givenName Victor
149 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0600611416.12
150 rdf:type schema:Person
151 grid-institutes:grid.18763.3b schema:alternateName Moscow Institute of Physics and Technology, Russia
152 schema:name Moscow Institute of Physics and Technology, Russia
153 Yandex, Russia
154 rdf:type schema:Organization
155 grid-institutes:grid.454320.4 schema:alternateName Skolkovo Institute of Science and Technology (Skoltech), Russia
156 schema:name Skolkovo Institute of Science and Technology (Skoltech), Russia
157 rdf:type schema:Organization
158 grid-institutes:grid.484753.9 schema:alternateName Yandex, Russia
159 schema:name Yandex, Russia
160 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...