FHE-Compatible Batch Normalization for Privacy Preserving Deep Learning View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2018-09-07

AUTHORS

Alberto Ibarrondo , Melek Önen

ABSTRACT

Deep Learning has recently become very popular thanks to major advances in cloud computing technology. However, pushing Deep Learning computations to the cloud poses a risk to the privacy of the data involved. Recent solutions propose to encrypt data with Fully Homomorphic Encryption (FHE) enabling the execution of operations over encrypted data. Given the serious performance constraints of this technology, recent privacy preserving deep learning solutions aim at first customizing the underlying neural network operations and further apply encryption. While the main neural network layer investigated so far is the activation layer, in this paper we study the Batch Normalization (BN) layer: a modern layer that, by addressing internal covariance shift, has been proved very effective in increasing the accuracy of Deep Neural Networks. In order to be compatible with the use of FHE, we propose to reformulate batch normalization which results in a moderate decrease on the number of operations. Furthermore, we devise a re-parametrization method that allows the absorption of batch normalization by previous layers. We show that whenever these two methods are integrated during the inference phase and executed over FHE-encrypted data, there is a significant performance gain with no loss on accuracy. We also note that this gain is valid both in the encrypted and unencrypted domains. More... »

PAGES

389-404

Book

TITLE

Data Privacy Management, Cryptocurrencies and Blockchain Technology

ISBN

978-3-030-00304-3
978-3-030-00305-0

Author Affiliations

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27

DOI

http://dx.doi.org/10.1007/978-3-030-00305-0_27

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1106836533


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "EURECOM, Sophia-Antipolis, France", 
          "id": "http://www.grid.ac/institutes/grid.28848.3e", 
          "name": [
            "EURECOM, Sophia-Antipolis, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Ibarrondo", 
        "givenName": "Alberto", 
        "id": "sg:person.07371741250.65", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07371741250.65"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "EURECOM, Sophia-Antipolis, France", 
          "id": "http://www.grid.ac/institutes/grid.28848.3e", 
          "name": [
            "EURECOM, Sophia-Antipolis, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "\u00d6nen", 
        "givenName": "Melek", 
        "id": "sg:person.010016772466.14", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010016772466.14"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2018-09-07", 
    "datePublishedReg": "2018-09-07", 
    "description": "Deep Learning has recently become very popular thanks to major advances in cloud computing technology. However, pushing Deep Learning computations to the cloud poses a risk to the privacy of the data involved. Recent solutions propose to encrypt data with Fully Homomorphic Encryption (FHE) enabling the execution of operations over encrypted data. Given the serious performance constraints of this technology, recent privacy preserving deep learning solutions aim at first customizing the underlying neural network operations and further apply encryption. While the main neural network layer investigated so far is the activation layer, in this paper we study the Batch Normalization (BN) layer: a modern layer that, by addressing internal covariance shift, has been proved very effective in increasing the accuracy of Deep Neural Networks. In order to be compatible with the use of FHE, we propose to reformulate batch normalization which results in a moderate decrease on the number of operations. Furthermore, we devise a re-parametrization method that allows the absorption of batch normalization by previous layers. We show that whenever these two methods are integrated during the inference phase and executed over FHE-encrypted data, there is a significant performance gain with no loss on accuracy. We also note that this gain is valid both in the encrypted and unencrypted domains.", 
    "editor": [
      {
        "familyName": "Garcia-Alfaro", 
        "givenName": "Joaquin", 
        "type": "Person"
      }, 
      {
        "familyName": "Herrera-Joancomart\u00ed", 
        "givenName": "Jordi", 
        "type": "Person"
      }, 
      {
        "familyName": "Livraga", 
        "givenName": "Giovanni", 
        "type": "Person"
      }, 
      {
        "familyName": "Rios", 
        "givenName": "Ruben", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-030-00305-0_27", 
    "inLanguage": "en", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-030-00304-3", 
        "978-3-030-00305-0"
      ], 
      "name": "Data Privacy Management, Cryptocurrencies and Blockchain Technology", 
      "type": "Book"
    }, 
    "keywords": [
      "Fully Homomorphic Encryption", 
      "batch normalization", 
      "deep learning", 
      "deep learning computation", 
      "deep learning solution", 
      "neural network layers", 
      "deep neural networks", 
      "neural network operations", 
      "batch normalization layer", 
      "significant performance gains", 
      "execution of operations", 
      "encrypt data", 
      "homomorphic encryption", 
      "network layer", 
      "normalization layer", 
      "inference phase", 
      "recent privacy", 
      "learning solutions", 
      "neural network", 
      "network operation", 
      "performance gains", 
      "performance constraints", 
      "previous layer", 
      "popular thanks", 
      "privacy", 
      "covariance shift", 
      "number of operations", 
      "encryption", 
      "recent solutions", 
      "activation layer", 
      "learning", 
      "technology", 
      "accuracy", 
      "execution", 
      "modern layer", 
      "operation", 
      "cloud", 
      "network", 
      "computation", 
      "constraints", 
      "solution", 
      "data", 
      "method", 
      "thanks", 
      "normalization", 
      "domain", 
      "gain", 
      "order", 
      "advances", 
      "layer", 
      "number", 
      "use", 
      "major advances", 
      "phase", 
      "loss", 
      "shift", 
      "risk", 
      "decrease", 
      "moderate decrease", 
      "paper", 
      "absorption", 
      "Learning computations", 
      "serious performance constraints", 
      "main neural network layer", 
      "internal covariance shift", 
      "use of FHE", 
      "re-parametrization method", 
      "unencrypted domains", 
      "FHE-Compatible Batch Normalization"
    ], 
    "name": "FHE-Compatible Batch Normalization for Privacy Preserving Deep Learning", 
    "pagination": "389-404", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1106836533"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-030-00305-0_27"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-030-00305-0_27", 
      "https://app.dimensions.ai/details/publication/pub.1106836533"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-01-01T19:06", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220101/entities/gbq_results/chapter/chapter_108.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-030-00305-0_27"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27'


 

This table displays all metadata directly associated to this object as RDF triples.

151 TRIPLES      23 PREDICATES      94 URIs      87 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-030-00305-0_27 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author N29a386cbf7014ccaa23f0b21a37c0491
4 schema:datePublished 2018-09-07
5 schema:datePublishedReg 2018-09-07
6 schema:description Deep Learning has recently become very popular thanks to major advances in cloud computing technology. However, pushing Deep Learning computations to the cloud poses a risk to the privacy of the data involved. Recent solutions propose to encrypt data with Fully Homomorphic Encryption (FHE) enabling the execution of operations over encrypted data. Given the serious performance constraints of this technology, recent privacy preserving deep learning solutions aim at first customizing the underlying neural network operations and further apply encryption. While the main neural network layer investigated so far is the activation layer, in this paper we study the Batch Normalization (BN) layer: a modern layer that, by addressing internal covariance shift, has been proved very effective in increasing the accuracy of Deep Neural Networks. In order to be compatible with the use of FHE, we propose to reformulate batch normalization which results in a moderate decrease on the number of operations. Furthermore, we devise a re-parametrization method that allows the absorption of batch normalization by previous layers. We show that whenever these two methods are integrated during the inference phase and executed over FHE-encrypted data, there is a significant performance gain with no loss on accuracy. We also note that this gain is valid both in the encrypted and unencrypted domains.
7 schema:editor Nab16f44202a94ae0af455796d6ca1271
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree false
11 schema:isPartOf Nb4bc0b0dd5e549ce941116019db5444a
12 schema:keywords FHE-Compatible Batch Normalization
13 Fully Homomorphic Encryption
14 Learning computations
15 absorption
16 accuracy
17 activation layer
18 advances
19 batch normalization
20 batch normalization layer
21 cloud
22 computation
23 constraints
24 covariance shift
25 data
26 decrease
27 deep learning
28 deep learning computation
29 deep learning solution
30 deep neural networks
31 domain
32 encrypt data
33 encryption
34 execution
35 execution of operations
36 gain
37 homomorphic encryption
38 inference phase
39 internal covariance shift
40 layer
41 learning
42 learning solutions
43 loss
44 main neural network layer
45 major advances
46 method
47 moderate decrease
48 modern layer
49 network
50 network layer
51 network operation
52 neural network
53 neural network layers
54 neural network operations
55 normalization
56 normalization layer
57 number
58 number of operations
59 operation
60 order
61 paper
62 performance constraints
63 performance gains
64 phase
65 popular thanks
66 previous layer
67 privacy
68 re-parametrization method
69 recent privacy
70 recent solutions
71 risk
72 serious performance constraints
73 shift
74 significant performance gains
75 solution
76 technology
77 thanks
78 unencrypted domains
79 use
80 use of FHE
81 schema:name FHE-Compatible Batch Normalization for Privacy Preserving Deep Learning
82 schema:pagination 389-404
83 schema:productId Ndd5337c8d3164e25992798e13efdb2f5
84 Ndf091a54db0c4f60814fe2114c13c4b2
85 schema:publisher N425fa65da1e64c23920282cdb456b380
86 schema:sameAs https://app.dimensions.ai/details/publication/pub.1106836533
87 https://doi.org/10.1007/978-3-030-00305-0_27
88 schema:sdDatePublished 2022-01-01T19:06
89 schema:sdLicense https://scigraph.springernature.com/explorer/license/
90 schema:sdPublisher Nf5cf6e0fc5bb4235b0584321be2e4db4
91 schema:url https://doi.org/10.1007/978-3-030-00305-0_27
92 sgo:license sg:explorer/license/
93 sgo:sdDataset chapters
94 rdf:type schema:Chapter
95 N29a386cbf7014ccaa23f0b21a37c0491 rdf:first sg:person.07371741250.65
96 rdf:rest Ne6acc5116d31482e9bc6d9d0bebd7c2a
97 N3e005759d3e049db8dcf33a31bc40cc3 schema:familyName Rios
98 schema:givenName Ruben
99 rdf:type schema:Person
100 N425fa65da1e64c23920282cdb456b380 schema:name Springer Nature
101 rdf:type schema:Organisation
102 N6ddfb4b358244ad59095e5761eee0570 rdf:first N3e005759d3e049db8dcf33a31bc40cc3
103 rdf:rest rdf:nil
104 N9b09d3b5aff5426f9dd40b8e2ec314f0 schema:familyName Garcia-Alfaro
105 schema:givenName Joaquin
106 rdf:type schema:Person
107 N9bcc1285d39147b59427990155c88cce rdf:first Nf015950ffe3547e2929e72151c546cbc
108 rdf:rest N6ddfb4b358244ad59095e5761eee0570
109 Na21d2cc52a40458294d610447b281223 rdf:first Ned53bb0c088e4480b45d2db6a85e6274
110 rdf:rest N9bcc1285d39147b59427990155c88cce
111 Nab16f44202a94ae0af455796d6ca1271 rdf:first N9b09d3b5aff5426f9dd40b8e2ec314f0
112 rdf:rest Na21d2cc52a40458294d610447b281223
113 Nb4bc0b0dd5e549ce941116019db5444a schema:isbn 978-3-030-00304-3
114 978-3-030-00305-0
115 schema:name Data Privacy Management, Cryptocurrencies and Blockchain Technology
116 rdf:type schema:Book
117 Ndd5337c8d3164e25992798e13efdb2f5 schema:name doi
118 schema:value 10.1007/978-3-030-00305-0_27
119 rdf:type schema:PropertyValue
120 Ndf091a54db0c4f60814fe2114c13c4b2 schema:name dimensions_id
121 schema:value pub.1106836533
122 rdf:type schema:PropertyValue
123 Ne6acc5116d31482e9bc6d9d0bebd7c2a rdf:first sg:person.010016772466.14
124 rdf:rest rdf:nil
125 Ned53bb0c088e4480b45d2db6a85e6274 schema:familyName Herrera-Joancomartí
126 schema:givenName Jordi
127 rdf:type schema:Person
128 Nf015950ffe3547e2929e72151c546cbc schema:familyName Livraga
129 schema:givenName Giovanni
130 rdf:type schema:Person
131 Nf5cf6e0fc5bb4235b0584321be2e4db4 schema:name Springer Nature - SN SciGraph project
132 rdf:type schema:Organization
133 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
134 schema:name Information and Computing Sciences
135 rdf:type schema:DefinedTerm
136 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
137 schema:name Artificial Intelligence and Image Processing
138 rdf:type schema:DefinedTerm
139 sg:person.010016772466.14 schema:affiliation grid-institutes:grid.28848.3e
140 schema:familyName Önen
141 schema:givenName Melek
142 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010016772466.14
143 rdf:type schema:Person
144 sg:person.07371741250.65 schema:affiliation grid-institutes:grid.28848.3e
145 schema:familyName Ibarrondo
146 schema:givenName Alberto
147 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07371741250.65
148 rdf:type schema:Person
149 grid-institutes:grid.28848.3e schema:alternateName EURECOM, Sophia-Antipolis, France
150 schema:name EURECOM, Sophia-Antipolis, France
151 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...