Towards Reverse-Engineering Black-Box Neural Networks View Full Text


Ontology type: schema:Chapter      Open Access: True


Chapter Info

DATE

2019-09-10

AUTHORS

Seong Joon Oh , Bernt Schiele , Mario Fritz

ABSTRACT

Much progress in interpretable AI is built around scenarios where the user, one who interprets the model, has a full ownership of the model to be diagnosed. The user either owns the training data and computing resources to train an interpretable model herself or owns a full access to an already trained model to be interpreted post-hoc. In this chapter, we consider a less investigated scenario of diagnosing black-box neural networks, where the user can only send queries and read off outputs. Black-box access is a common deployment mode for many public and commercial models, since internal details, such as architecture, optimisation procedure, and training data, can be proprietary and aggravate their vulnerability to attacks like adversarial examples. We propose a method for exposing internals of black-box models and show that the method is surprisingly effective at inferring a diverse set of internal information. We further show how the exposed internals can be exploited to strengthen adversarial examples against the model. Our work starts an important discussion on the security implications of diagnosing deployed models with limited accessibility. The code is available at goo.gl/MbYfsv. More... »

PAGES

121-144

Book

TITLE

Explainable AI: Interpreting, Explaining and Visualizing Deep Learning

ISBN

978-3-030-28953-9
978-3-030-28954-6

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-030-28954-6_7

DOI

http://dx.doi.org/10.1007/978-3-030-28954-6_7

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1120935870


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Max-Planck Institute, Saarbr\u00fccken, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4372.2", 
          "name": [
            "Max-Planck Institute, Saarbr\u00fccken, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Oh", 
        "givenName": "Seong Joon", 
        "id": "sg:person.013506322275.39", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013506322275.39"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Max-Planck Institute, Saarbr\u00fccken, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4372.2", 
          "name": [
            "Max-Planck Institute, Saarbr\u00fccken, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Schiele", 
        "givenName": "Bernt", 
        "id": "sg:person.01174260421.90", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01174260421.90"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Max-Planck Institute, Saarbr\u00fccken, Germany", 
          "id": "http://www.grid.ac/institutes/grid.4372.2", 
          "name": [
            "Max-Planck Institute, Saarbr\u00fccken, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Fritz", 
        "givenName": "Mario", 
        "id": "sg:person.013361072755.17", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013361072755.17"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2019-09-10", 
    "datePublishedReg": "2019-09-10", 
    "description": "Much progress in interpretable AI is built around scenarios where the user, one who interprets the model, has a full ownership of the model to be diagnosed. The user either owns the training data and computing resources to train an interpretable model herself or owns a full access to an already trained model to be interpreted post-hoc. In this chapter, we consider a less investigated scenario of diagnosing black-box neural networks, where the user can only send queries and read off outputs. Black-box access is a common deployment mode for many public and commercial models, since internal details, such as architecture, optimisation procedure, and training data, can be proprietary and aggravate their vulnerability to attacks like adversarial examples. We propose a method for exposing internals of black-box models and show that the method is surprisingly effective at inferring a diverse set of internal information. We further show how the exposed internals can be exploited to strengthen adversarial examples against the model. Our work starts an important discussion on the security implications of diagnosing deployed models with limited accessibility. The code is available at goo.gl/MbYfsv.", 
    "editor": [
      {
        "familyName": "Samek", 
        "givenName": "Wojciech", 
        "type": "Person"
      }, 
      {
        "familyName": "Montavon", 
        "givenName": "Gr\u00e9goire", 
        "type": "Person"
      }, 
      {
        "familyName": "Vedaldi", 
        "givenName": "Andrea", 
        "type": "Person"
      }, 
      {
        "familyName": "Hansen", 
        "givenName": "Lars Kai", 
        "type": "Person"
      }, 
      {
        "familyName": "M\u00fcller", 
        "givenName": "Klaus-Robert", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-030-28954-6_7", 
    "isAccessibleForFree": true, 
    "isPartOf": {
      "isbn": [
        "978-3-030-28953-9", 
        "978-3-030-28954-6"
      ], 
      "name": "Explainable AI: Interpreting, Explaining and Visualizing Deep Learning", 
      "type": "Book"
    }, 
    "keywords": [
      "adversarial examples", 
      "training data", 
      "neural network", 
      "black-box neural networks", 
      "black-box access", 
      "box neural network", 
      "black-box models", 
      "interpretable AI", 
      "deployment mode", 
      "interpretable models", 
      "security implications", 
      "users", 
      "internal information", 
      "full access", 
      "investigated scenario", 
      "internal details", 
      "network", 
      "diverse set", 
      "commercial models", 
      "scenarios", 
      "queries", 
      "architecture", 
      "access", 
      "AI", 
      "optimization procedure", 
      "full ownership", 
      "attacks", 
      "code", 
      "model", 
      "internals", 
      "information", 
      "example", 
      "set", 
      "resources", 
      "limited accessibility", 
      "data", 
      "method", 
      "vulnerability", 
      "accessibility", 
      "output", 
      "work", 
      "important discussion", 
      "detail", 
      "chapter", 
      "progress", 
      "ownership", 
      "discussion", 
      "procedure", 
      "mode", 
      "Goos", 
      "implications", 
      "blacks"
    ], 
    "name": "Towards Reverse-Engineering Black-Box Neural Networks", 
    "pagination": "121-144", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1120935870"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-030-28954-6_7"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-030-28954-6_7", 
      "https://app.dimensions.ai/details/publication/pub.1120935870"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-11-24T21:11", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20221124/entities/gbq_results/chapter/chapter_125.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-030-28954-6_7"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-28954-6_7'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-28954-6_7'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-28954-6_7'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-28954-6_7'


 

This table displays all metadata directly associated to this object as RDF triples.

145 TRIPLES      22 PREDICATES      75 URIs      68 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-030-28954-6_7 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author N67ffc9f66bf744e6a4a06ebfe10169d9
4 schema:datePublished 2019-09-10
5 schema:datePublishedReg 2019-09-10
6 schema:description Much progress in interpretable AI is built around scenarios where the user, one who interprets the model, has a full ownership of the model to be diagnosed. The user either owns the training data and computing resources to train an interpretable model herself or owns a full access to an already trained model to be interpreted post-hoc. In this chapter, we consider a less investigated scenario of diagnosing black-box neural networks, where the user can only send queries and read off outputs. Black-box access is a common deployment mode for many public and commercial models, since internal details, such as architecture, optimisation procedure, and training data, can be proprietary and aggravate their vulnerability to attacks like adversarial examples. We propose a method for exposing internals of black-box models and show that the method is surprisingly effective at inferring a diverse set of internal information. We further show how the exposed internals can be exploited to strengthen adversarial examples against the model. Our work starts an important discussion on the security implications of diagnosing deployed models with limited accessibility. The code is available at goo.gl/MbYfsv.
7 schema:editor Ne89143400a0f49d5b43c1e30626f3caa
8 schema:genre chapter
9 schema:isAccessibleForFree true
10 schema:isPartOf N9882966954554ad3862b20ae412913e5
11 schema:keywords AI
12 Goos
13 access
14 accessibility
15 adversarial examples
16 architecture
17 attacks
18 black-box access
19 black-box models
20 black-box neural networks
21 blacks
22 box neural network
23 chapter
24 code
25 commercial models
26 data
27 deployment mode
28 detail
29 discussion
30 diverse set
31 example
32 full access
33 full ownership
34 implications
35 important discussion
36 information
37 internal details
38 internal information
39 internals
40 interpretable AI
41 interpretable models
42 investigated scenario
43 limited accessibility
44 method
45 mode
46 model
47 network
48 neural network
49 optimization procedure
50 output
51 ownership
52 procedure
53 progress
54 queries
55 resources
56 scenarios
57 security implications
58 set
59 training data
60 users
61 vulnerability
62 work
63 schema:name Towards Reverse-Engineering Black-Box Neural Networks
64 schema:pagination 121-144
65 schema:productId N83002ee9017b4387935ab51b3b0bb8c2
66 Nd599ade0fe804a2689845f9812791a7b
67 schema:publisher Nf74e38513deb4b2a9b5a60552fe04c00
68 schema:sameAs https://app.dimensions.ai/details/publication/pub.1120935870
69 https://doi.org/10.1007/978-3-030-28954-6_7
70 schema:sdDatePublished 2022-11-24T21:11
71 schema:sdLicense https://scigraph.springernature.com/explorer/license/
72 schema:sdPublisher Nef6bf145fea343a9afbcf1a2be9eaccb
73 schema:url https://doi.org/10.1007/978-3-030-28954-6_7
74 sgo:license sg:explorer/license/
75 sgo:sdDataset chapters
76 rdf:type schema:Chapter
77 N1a851967082c4a4cb88a71dbb30bc80b schema:familyName Samek
78 schema:givenName Wojciech
79 rdf:type schema:Person
80 N2d3d5815074547a5af7a2acab173d2a6 rdf:first N316fad693bd34bc9a6b149967a7bd329
81 rdf:rest Nf4527f72a4ec4e989c5853dffff383c0
82 N316fad693bd34bc9a6b149967a7bd329 schema:familyName Vedaldi
83 schema:givenName Andrea
84 rdf:type schema:Person
85 N67ffc9f66bf744e6a4a06ebfe10169d9 rdf:first sg:person.013506322275.39
86 rdf:rest N9c130e4f8fcd4735a00e2b1a6c4719a9
87 N83002ee9017b4387935ab51b3b0bb8c2 schema:name doi
88 schema:value 10.1007/978-3-030-28954-6_7
89 rdf:type schema:PropertyValue
90 N870c8805fe68468988b6365ab6e3a40d schema:familyName Müller
91 schema:givenName Klaus-Robert
92 rdf:type schema:Person
93 N9882966954554ad3862b20ae412913e5 schema:isbn 978-3-030-28953-9
94 978-3-030-28954-6
95 schema:name Explainable AI: Interpreting, Explaining and Visualizing Deep Learning
96 rdf:type schema:Book
97 N9c130e4f8fcd4735a00e2b1a6c4719a9 rdf:first sg:person.01174260421.90
98 rdf:rest Nd67d9967b0174b77a7aaf2c7afc9ade2
99 Nb51c8ad996b649e6a5cb832785e4b184 rdf:first N870c8805fe68468988b6365ab6e3a40d
100 rdf:rest rdf:nil
101 Nc788d18d7c574ed2843c7a5f6bfae38b schema:familyName Hansen
102 schema:givenName Lars Kai
103 rdf:type schema:Person
104 Nd599ade0fe804a2689845f9812791a7b schema:name dimensions_id
105 schema:value pub.1120935870
106 rdf:type schema:PropertyValue
107 Nd67d9967b0174b77a7aaf2c7afc9ade2 rdf:first sg:person.013361072755.17
108 rdf:rest rdf:nil
109 Ne89143400a0f49d5b43c1e30626f3caa rdf:first N1a851967082c4a4cb88a71dbb30bc80b
110 rdf:rest Neb1f6816d9c9435487fee8c618a6a1a9
111 Neb1f6816d9c9435487fee8c618a6a1a9 rdf:first Neb6c7688051147c5a22c94359677e4de
112 rdf:rest N2d3d5815074547a5af7a2acab173d2a6
113 Neb6c7688051147c5a22c94359677e4de schema:familyName Montavon
114 schema:givenName Grégoire
115 rdf:type schema:Person
116 Nef6bf145fea343a9afbcf1a2be9eaccb schema:name Springer Nature - SN SciGraph project
117 rdf:type schema:Organization
118 Nf4527f72a4ec4e989c5853dffff383c0 rdf:first Nc788d18d7c574ed2843c7a5f6bfae38b
119 rdf:rest Nb51c8ad996b649e6a5cb832785e4b184
120 Nf74e38513deb4b2a9b5a60552fe04c00 schema:name Springer Nature
121 rdf:type schema:Organisation
122 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
123 schema:name Information and Computing Sciences
124 rdf:type schema:DefinedTerm
125 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
126 schema:name Artificial Intelligence and Image Processing
127 rdf:type schema:DefinedTerm
128 sg:person.01174260421.90 schema:affiliation grid-institutes:grid.4372.2
129 schema:familyName Schiele
130 schema:givenName Bernt
131 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01174260421.90
132 rdf:type schema:Person
133 sg:person.013361072755.17 schema:affiliation grid-institutes:grid.4372.2
134 schema:familyName Fritz
135 schema:givenName Mario
136 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013361072755.17
137 rdf:type schema:Person
138 sg:person.013506322275.39 schema:affiliation grid-institutes:grid.4372.2
139 schema:familyName Oh
140 schema:givenName Seong Joon
141 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013506322275.39
142 rdf:type schema:Person
143 grid-institutes:grid.4372.2 schema:alternateName Max-Planck Institute, Saarbrücken, Germany
144 schema:name Max-Planck Institute, Saarbrücken, Germany
145 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...