FHE-Compatible Batch Normalization for Privacy Preserving Deep Learning View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2018-09-07

AUTHORS

Alberto Ibarrondo , Melek Önen

ABSTRACT

Deep Learning has recently become very popular thanks to major advances in cloud computing technology. However, pushing Deep Learning computations to the cloud poses a risk to the privacy of the data involved. Recent solutions propose to encrypt data with Fully Homomorphic Encryption (FHE) enabling the execution of operations over encrypted data. Given the serious performance constraints of this technology, recent privacy preserving deep learning solutions aim at first customizing the underlying neural network operations and further apply encryption. While the main neural network layer investigated so far is the activation layer, in this paper we study the Batch Normalization (BN) layer: a modern layer that, by addressing internal covariance shift, has been proved very effective in increasing the accuracy of Deep Neural Networks. In order to be compatible with the use of FHE, we propose to reformulate batch normalization which results in a moderate decrease on the number of operations. Furthermore, we devise a re-parametrization method that allows the absorption of batch normalization by previous layers. We show that whenever these two methods are integrated during the inference phase and executed over FHE-encrypted data, there is a significant performance gain with no loss on accuracy. We also note that this gain is valid both in the encrypted and unencrypted domains. More... »

PAGES

389-404

Book

TITLE

Data Privacy Management, Cryptocurrencies and Blockchain Technology

ISBN

978-3-030-00304-3
978-3-030-00305-0

Author Affiliations

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27

DOI

http://dx.doi.org/10.1007/978-3-030-00305-0_27

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1106836533


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "EURECOM, Sophia-Antipolis, France", 
          "id": "http://www.grid.ac/institutes/grid.28848.3e", 
          "name": [
            "EURECOM, Sophia-Antipolis, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Ibarrondo", 
        "givenName": "Alberto", 
        "id": "sg:person.07371741250.65", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07371741250.65"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "EURECOM, Sophia-Antipolis, France", 
          "id": "http://www.grid.ac/institutes/grid.28848.3e", 
          "name": [
            "EURECOM, Sophia-Antipolis, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "\u00d6nen", 
        "givenName": "Melek", 
        "id": "sg:person.010016772466.14", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010016772466.14"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2018-09-07", 
    "datePublishedReg": "2018-09-07", 
    "description": "Deep Learning has recently become very popular thanks to major advances in cloud computing technology. However, pushing Deep Learning computations to the cloud poses a risk to the privacy of the data involved. Recent solutions propose to encrypt data with Fully Homomorphic Encryption (FHE) enabling the execution of operations over encrypted data. Given the serious performance constraints of this technology, recent privacy preserving deep learning solutions aim at first customizing the underlying neural network operations and further apply encryption. While the main neural network layer investigated so far is the activation layer, in this paper we study the Batch Normalization (BN) layer: a modern layer that, by addressing internal covariance shift, has been proved very effective in increasing the accuracy of Deep Neural Networks. In order to be compatible with the use of FHE, we propose to reformulate batch normalization which results in a moderate decrease on the number of operations. Furthermore, we devise a re-parametrization method that allows the absorption of batch normalization by previous layers. We show that whenever these two methods are integrated during the inference phase and executed over FHE-encrypted data, there is a significant performance gain with no loss on accuracy. We also note that this gain is valid both in the encrypted and unencrypted domains.", 
    "editor": [
      {
        "familyName": "Garcia-Alfaro", 
        "givenName": "Joaquin", 
        "type": "Person"
      }, 
      {
        "familyName": "Herrera-Joancomart\u00ed", 
        "givenName": "Jordi", 
        "type": "Person"
      }, 
      {
        "familyName": "Livraga", 
        "givenName": "Giovanni", 
        "type": "Person"
      }, 
      {
        "familyName": "Rios", 
        "givenName": "Ruben", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-030-00305-0_27", 
    "inLanguage": "en", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-030-00304-3", 
        "978-3-030-00305-0"
      ], 
      "name": "Data Privacy Management, Cryptocurrencies and Blockchain Technology", 
      "type": "Book"
    }, 
    "keywords": [
      "Fully Homomorphic Encryption", 
      "batch normalization", 
      "deep learning", 
      "deep learning computation", 
      "deep learning solution", 
      "neural network layers", 
      "deep neural networks", 
      "neural network operations", 
      "batch normalization layer", 
      "significant performance gains", 
      "execution of operations", 
      "encrypt data", 
      "homomorphic encryption", 
      "network layer", 
      "normalization layer", 
      "inference phase", 
      "recent privacy", 
      "learning solutions", 
      "neural network", 
      "network operation", 
      "performance gains", 
      "performance constraints", 
      "previous layer", 
      "popular thanks", 
      "privacy", 
      "covariance shift", 
      "number of operations", 
      "encryption", 
      "recent solutions", 
      "activation layer", 
      "learning", 
      "technology", 
      "accuracy", 
      "execution", 
      "modern layer", 
      "operation", 
      "cloud", 
      "network", 
      "computation", 
      "constraints", 
      "solution", 
      "data", 
      "method", 
      "thanks", 
      "normalization", 
      "domain", 
      "gain", 
      "order", 
      "advances", 
      "layer", 
      "number", 
      "use", 
      "major advances", 
      "phase", 
      "loss", 
      "shift", 
      "risk", 
      "decrease", 
      "moderate decrease", 
      "paper", 
      "absorption", 
      "Learning computations", 
      "serious performance constraints", 
      "main neural network layer", 
      "internal covariance shift", 
      "use of FHE", 
      "re-parametrization method", 
      "unencrypted domains", 
      "FHE-Compatible Batch Normalization"
    ], 
    "name": "FHE-Compatible Batch Normalization for Privacy Preserving Deep Learning", 
    "pagination": "389-404", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1106836533"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-030-00305-0_27"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-030-00305-0_27", 
      "https://app.dimensions.ai/details/publication/pub.1106836533"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2021-12-01T19:59", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20211201/entities/gbq_results/chapter/chapter_203.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-030-00305-0_27"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-00305-0_27'


 

This table displays all metadata directly associated to this object as RDF triples.

151 TRIPLES      23 PREDICATES      94 URIs      87 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-030-00305-0_27 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Nad21377c680f4177bdb7906a3355a013
4 schema:datePublished 2018-09-07
5 schema:datePublishedReg 2018-09-07
6 schema:description Deep Learning has recently become very popular thanks to major advances in cloud computing technology. However, pushing Deep Learning computations to the cloud poses a risk to the privacy of the data involved. Recent solutions propose to encrypt data with Fully Homomorphic Encryption (FHE) enabling the execution of operations over encrypted data. Given the serious performance constraints of this technology, recent privacy preserving deep learning solutions aim at first customizing the underlying neural network operations and further apply encryption. While the main neural network layer investigated so far is the activation layer, in this paper we study the Batch Normalization (BN) layer: a modern layer that, by addressing internal covariance shift, has been proved very effective in increasing the accuracy of Deep Neural Networks. In order to be compatible with the use of FHE, we propose to reformulate batch normalization which results in a moderate decrease on the number of operations. Furthermore, we devise a re-parametrization method that allows the absorption of batch normalization by previous layers. We show that whenever these two methods are integrated during the inference phase and executed over FHE-encrypted data, there is a significant performance gain with no loss on accuracy. We also note that this gain is valid both in the encrypted and unencrypted domains.
7 schema:editor N6064ba0316344770893859c37857305a
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree false
11 schema:isPartOf N2b426a75b71d4d23bdc83a84ca17fd03
12 schema:keywords FHE-Compatible Batch Normalization
13 Fully Homomorphic Encryption
14 Learning computations
15 absorption
16 accuracy
17 activation layer
18 advances
19 batch normalization
20 batch normalization layer
21 cloud
22 computation
23 constraints
24 covariance shift
25 data
26 decrease
27 deep learning
28 deep learning computation
29 deep learning solution
30 deep neural networks
31 domain
32 encrypt data
33 encryption
34 execution
35 execution of operations
36 gain
37 homomorphic encryption
38 inference phase
39 internal covariance shift
40 layer
41 learning
42 learning solutions
43 loss
44 main neural network layer
45 major advances
46 method
47 moderate decrease
48 modern layer
49 network
50 network layer
51 network operation
52 neural network
53 neural network layers
54 neural network operations
55 normalization
56 normalization layer
57 number
58 number of operations
59 operation
60 order
61 paper
62 performance constraints
63 performance gains
64 phase
65 popular thanks
66 previous layer
67 privacy
68 re-parametrization method
69 recent privacy
70 recent solutions
71 risk
72 serious performance constraints
73 shift
74 significant performance gains
75 solution
76 technology
77 thanks
78 unencrypted domains
79 use
80 use of FHE
81 schema:name FHE-Compatible Batch Normalization for Privacy Preserving Deep Learning
82 schema:pagination 389-404
83 schema:productId N2cc29ba04b414324868337b9f4db85bb
84 Nde60c181959c4373abaab0f819095658
85 schema:publisher N92f6991f230844fca9d119673aef89b0
86 schema:sameAs https://app.dimensions.ai/details/publication/pub.1106836533
87 https://doi.org/10.1007/978-3-030-00305-0_27
88 schema:sdDatePublished 2021-12-01T19:59
89 schema:sdLicense https://scigraph.springernature.com/explorer/license/
90 schema:sdPublisher N5dfea069d2374aa8b79cd364db22a420
91 schema:url https://doi.org/10.1007/978-3-030-00305-0_27
92 sgo:license sg:explorer/license/
93 sgo:sdDataset chapters
94 rdf:type schema:Chapter
95 N29c70cc298e14b9e97b12d8f6caf3c83 schema:familyName Garcia-Alfaro
96 schema:givenName Joaquin
97 rdf:type schema:Person
98 N2b426a75b71d4d23bdc83a84ca17fd03 schema:isbn 978-3-030-00304-3
99 978-3-030-00305-0
100 schema:name Data Privacy Management, Cryptocurrencies and Blockchain Technology
101 rdf:type schema:Book
102 N2cc29ba04b414324868337b9f4db85bb schema:name dimensions_id
103 schema:value pub.1106836533
104 rdf:type schema:PropertyValue
105 N4c2c75ec4f9f4c83ab50b02d44cf54c4 schema:familyName Herrera-Joancomartí
106 schema:givenName Jordi
107 rdf:type schema:Person
108 N5dfea069d2374aa8b79cd364db22a420 schema:name Springer Nature - SN SciGraph project
109 rdf:type schema:Organization
110 N6064ba0316344770893859c37857305a rdf:first N29c70cc298e14b9e97b12d8f6caf3c83
111 rdf:rest N7fd73e23b9724df083153b6abd27ebb6
112 N7105ed601b0a4363b6ad138df24c49e8 schema:familyName Livraga
113 schema:givenName Giovanni
114 rdf:type schema:Person
115 N7fd73e23b9724df083153b6abd27ebb6 rdf:first N4c2c75ec4f9f4c83ab50b02d44cf54c4
116 rdf:rest Ndd5350e8eb0f428a8f2ae22d7709bbb8
117 N92f6991f230844fca9d119673aef89b0 schema:name Springer Nature
118 rdf:type schema:Organisation
119 Na76968c04bac4ab7aa7bdfdd961df3ec rdf:first Nd81cd9d8d6bf43529372430ba0abe88a
120 rdf:rest rdf:nil
121 Nad21377c680f4177bdb7906a3355a013 rdf:first sg:person.07371741250.65
122 rdf:rest Nc2670a95600c497fadc3f2a1826aef2f
123 Nc2670a95600c497fadc3f2a1826aef2f rdf:first sg:person.010016772466.14
124 rdf:rest rdf:nil
125 Nd81cd9d8d6bf43529372430ba0abe88a schema:familyName Rios
126 schema:givenName Ruben
127 rdf:type schema:Person
128 Ndd5350e8eb0f428a8f2ae22d7709bbb8 rdf:first N7105ed601b0a4363b6ad138df24c49e8
129 rdf:rest Na76968c04bac4ab7aa7bdfdd961df3ec
130 Nde60c181959c4373abaab0f819095658 schema:name doi
131 schema:value 10.1007/978-3-030-00305-0_27
132 rdf:type schema:PropertyValue
133 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
134 schema:name Information and Computing Sciences
135 rdf:type schema:DefinedTerm
136 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
137 schema:name Artificial Intelligence and Image Processing
138 rdf:type schema:DefinedTerm
139 sg:person.010016772466.14 schema:affiliation grid-institutes:grid.28848.3e
140 schema:familyName Önen
141 schema:givenName Melek
142 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010016772466.14
143 rdf:type schema:Person
144 sg:person.07371741250.65 schema:affiliation grid-institutes:grid.28848.3e
145 schema:familyName Ibarrondo
146 schema:givenName Alberto
147 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07371741250.65
148 rdf:type schema:Person
149 grid-institutes:grid.28848.3e schema:alternateName EURECOM, Sophia-Antipolis, France
150 schema:name EURECOM, Sophia-Antipolis, France
151 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...