Towards Automated Testing and Robustification by Semantic Adversarial Data Generation View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2020-11-03

AUTHORS

Rakshith Shetty , Mario Fritz , Bernt Schiele

ABSTRACT

Widespread application of computer vision systems in real world tasks is currently hindered by their unexpected behavior on unseen examples. This occurs due to limitations of empirical testing on finite test sets and lack of systematic methods to identify the breaking points of a trained model. In this work we propose semantic adversarial editing, a method to synthesize plausible but difficult data points on which our target model breaks down. We achieve this with a differentiable object synthesizer which can change an object’s appearance while retaining its pose. Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector. Experiments show that our approach effectively synthesizes difficult test data, dropping the performance of YoloV3 detector by more than 20 mAP points by changing the appearance of a single object and discovering failure modes of the model. The generated semantic adversarial data can also be used to robustify the detector through data augmentation, consistently improving its performance in both standard and out-of-dataset-distribution test sets, across three different datasets. More... »

PAGES

489-506

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-030-58536-5_29

DOI

http://dx.doi.org/10.1007/978-3-030-58536-5_29

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1132266640


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbr\u00fccken, Germany", 
          "id": "http://www.grid.ac/institutes/grid.419528.3", 
          "name": [
            "Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbr\u00fccken, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Shetty", 
        "givenName": "Rakshith", 
        "id": "sg:person.012536402327.77", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012536402327.77"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "CISPA Helmholtz Center for Information Security, Saarbr\u00fccken, Germany", 
          "id": "http://www.grid.ac/institutes/grid.507511.7", 
          "name": [
            "CISPA Helmholtz Center for Information Security, Saarbr\u00fccken, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Fritz", 
        "givenName": "Mario", 
        "id": "sg:person.013361072755.17", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013361072755.17"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbr\u00fccken, Germany", 
          "id": "http://www.grid.ac/institutes/grid.419528.3", 
          "name": [
            "Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbr\u00fccken, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Schiele", 
        "givenName": "Bernt", 
        "id": "sg:person.01174260421.90", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01174260421.90"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2020-11-03", 
    "datePublishedReg": "2020-11-03", 
    "description": "Widespread application of computer vision systems in real world tasks is currently hindered by their unexpected behavior on unseen examples. This occurs due to limitations of empirical testing on finite test sets and lack of systematic methods to identify the breaking points of a trained model. In this work we propose semantic adversarial editing, a method to synthesize plausible but difficult data points on which our target model breaks down. We achieve this with a differentiable object synthesizer which can change an object\u2019s appearance while retaining its pose. Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector. Experiments show that our approach effectively synthesizes difficult test data, dropping the performance of YoloV3 detector by more than 20 mAP points by changing the appearance of a single object and discovering failure modes of the model. The generated semantic adversarial data can also be used to robustify the detector through data augmentation, consistently improving its performance in both standard and out-of-dataset-distribution test sets, across three different datasets.", 
    "editor": [
      {
        "familyName": "Vedaldi", 
        "givenName": "Andrea", 
        "type": "Person"
      }, 
      {
        "familyName": "Bischof", 
        "givenName": "Horst", 
        "type": "Person"
      }, 
      {
        "familyName": "Brox", 
        "givenName": "Thomas", 
        "type": "Person"
      }, 
      {
        "familyName": "Frahm", 
        "givenName": "Jan-Michael", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-030-58536-5_29", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-030-58535-8", 
        "978-3-030-58536-5"
      ], 
      "name": "Computer Vision \u2013 ECCV 2020", 
      "type": "Book"
    }, 
    "keywords": [
      "object appearance", 
      "computer vision system", 
      "real-world tasks", 
      "test set", 
      "adversarial data", 
      "Automated Testing", 
      "YOLOv3 detector", 
      "finite test sets", 
      "vision system", 
      "data augmentation", 
      "unseen examples", 
      "object detector", 
      "world tasks", 
      "adversarial optimization", 
      "data generation", 
      "different datasets", 
      "map points", 
      "target model", 
      "single object", 
      "data points", 
      "test data", 
      "unexpected behavior", 
      "objects", 
      "pose", 
      "systematic method", 
      "set", 
      "difficult version", 
      "dataset", 
      "performance", 
      "task", 
      "synthesizer", 
      "widespread application", 
      "model", 
      "optimization", 
      "empirical testing", 
      "data", 
      "applications", 
      "editing", 
      "method", 
      "detector", 
      "robustification", 
      "system", 
      "point", 
      "version", 
      "example", 
      "work", 
      "augmentation", 
      "limitations", 
      "testing", 
      "experiments", 
      "generation", 
      "appearance", 
      "lack", 
      "behavior", 
      "failure modes", 
      "mode", 
      "approach"
    ], 
    "name": "Towards Automated Testing and Robustification by Semantic Adversarial Data Generation", 
    "pagination": "489-506", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1132266640"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-030-58536-5_29"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-030-58536-5_29", 
      "https://app.dimensions.ai/details/publication/pub.1132266640"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-11-24T21:18", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20221124/entities/gbq_results/chapter/chapter_424.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-030-58536-5_29"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-58536-5_29'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-58536-5_29'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-58536-5_29'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-58536-5_29'


 

This table displays all metadata directly associated to this object as RDF triples.

148 TRIPLES      22 PREDICATES      81 URIs      74 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-030-58536-5_29 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Nc4882a8e39d942a39ab81bdea0067777
4 schema:datePublished 2020-11-03
5 schema:datePublishedReg 2020-11-03
6 schema:description Widespread application of computer vision systems in real world tasks is currently hindered by their unexpected behavior on unseen examples. This occurs due to limitations of empirical testing on finite test sets and lack of systematic methods to identify the breaking points of a trained model. In this work we propose semantic adversarial editing, a method to synthesize plausible but difficult data points on which our target model breaks down. We achieve this with a differentiable object synthesizer which can change an object’s appearance while retaining its pose. Constrained adversarial optimization of object appearance through this synthesizer produces rare/difficult versions of an object which fool the target object detector. Experiments show that our approach effectively synthesizes difficult test data, dropping the performance of YoloV3 detector by more than 20 mAP points by changing the appearance of a single object and discovering failure modes of the model. The generated semantic adversarial data can also be used to robustify the detector through data augmentation, consistently improving its performance in both standard and out-of-dataset-distribution test sets, across three different datasets.
7 schema:editor N2aa0fc8cf05f4e3fb92afb7b19e72507
8 schema:genre chapter
9 schema:isAccessibleForFree false
10 schema:isPartOf Ncb87d32792754543ab8e0e4da3b0df4b
11 schema:keywords Automated Testing
12 YOLOv3 detector
13 adversarial data
14 adversarial optimization
15 appearance
16 applications
17 approach
18 augmentation
19 behavior
20 computer vision system
21 data
22 data augmentation
23 data generation
24 data points
25 dataset
26 detector
27 different datasets
28 difficult version
29 editing
30 empirical testing
31 example
32 experiments
33 failure modes
34 finite test sets
35 generation
36 lack
37 limitations
38 map points
39 method
40 mode
41 model
42 object appearance
43 object detector
44 objects
45 optimization
46 performance
47 point
48 pose
49 real-world tasks
50 robustification
51 set
52 single object
53 synthesizer
54 system
55 systematic method
56 target model
57 task
58 test data
59 test set
60 testing
61 unexpected behavior
62 unseen examples
63 version
64 vision system
65 widespread application
66 work
67 world tasks
68 schema:name Towards Automated Testing and Robustification by Semantic Adversarial Data Generation
69 schema:pagination 489-506
70 schema:productId N4fbc032fb5b243fcb724df9e4d55f681
71 Nb2c0e66e0e4d45cd8641470aa4929bcc
72 schema:publisher N3faf43c6e12149cbb730961395f096a6
73 schema:sameAs https://app.dimensions.ai/details/publication/pub.1132266640
74 https://doi.org/10.1007/978-3-030-58536-5_29
75 schema:sdDatePublished 2022-11-24T21:18
76 schema:sdLicense https://scigraph.springernature.com/explorer/license/
77 schema:sdPublisher N0947bae32bd94bd39e1ca5bc8c2c6bd8
78 schema:url https://doi.org/10.1007/978-3-030-58536-5_29
79 sgo:license sg:explorer/license/
80 sgo:sdDataset chapters
81 rdf:type schema:Chapter
82 N0947bae32bd94bd39e1ca5bc8c2c6bd8 schema:name Springer Nature - SN SciGraph project
83 rdf:type schema:Organization
84 N11aa29c745784da3b58210f1bf122a71 schema:familyName Brox
85 schema:givenName Thomas
86 rdf:type schema:Person
87 N14e1ce152d6a432fb10d72961d436161 schema:familyName Frahm
88 schema:givenName Jan-Michael
89 rdf:type schema:Person
90 N1c1b684b7162406db221d2d9aaa37e62 schema:familyName Vedaldi
91 schema:givenName Andrea
92 rdf:type schema:Person
93 N2aa0fc8cf05f4e3fb92afb7b19e72507 rdf:first N1c1b684b7162406db221d2d9aaa37e62
94 rdf:rest N9aaf8df4656b4e54a9a622c00fb5f625
95 N36483ff4c1a943979625ba82db01eb8d schema:familyName Bischof
96 schema:givenName Horst
97 rdf:type schema:Person
98 N3faf43c6e12149cbb730961395f096a6 schema:name Springer Nature
99 rdf:type schema:Organisation
100 N4fbc032fb5b243fcb724df9e4d55f681 schema:name dimensions_id
101 schema:value pub.1132266640
102 rdf:type schema:PropertyValue
103 N9aaf8df4656b4e54a9a622c00fb5f625 rdf:first N36483ff4c1a943979625ba82db01eb8d
104 rdf:rest Ncf7e7e43da844c63b3ba128cdd0dc75c
105 Na27d6a3ffb95466b9df556f7ead99d6e rdf:first sg:person.013361072755.17
106 rdf:rest Ne716fc0082a541adad28f2f7fc10a0cc
107 Nb2c0e66e0e4d45cd8641470aa4929bcc schema:name doi
108 schema:value 10.1007/978-3-030-58536-5_29
109 rdf:type schema:PropertyValue
110 Nc4882a8e39d942a39ab81bdea0067777 rdf:first sg:person.012536402327.77
111 rdf:rest Na27d6a3ffb95466b9df556f7ead99d6e
112 Ncb87d32792754543ab8e0e4da3b0df4b schema:isbn 978-3-030-58535-8
113 978-3-030-58536-5
114 schema:name Computer Vision – ECCV 2020
115 rdf:type schema:Book
116 Ncf7e7e43da844c63b3ba128cdd0dc75c rdf:first N11aa29c745784da3b58210f1bf122a71
117 rdf:rest Nd92a950d0ecf44bfb8d89ae12c3ff3c4
118 Nd92a950d0ecf44bfb8d89ae12c3ff3c4 rdf:first N14e1ce152d6a432fb10d72961d436161
119 rdf:rest rdf:nil
120 Ne716fc0082a541adad28f2f7fc10a0cc rdf:first sg:person.01174260421.90
121 rdf:rest rdf:nil
122 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
123 schema:name Information and Computing Sciences
124 rdf:type schema:DefinedTerm
125 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
126 schema:name Artificial Intelligence and Image Processing
127 rdf:type schema:DefinedTerm
128 sg:person.01174260421.90 schema:affiliation grid-institutes:grid.419528.3
129 schema:familyName Schiele
130 schema:givenName Bernt
131 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01174260421.90
132 rdf:type schema:Person
133 sg:person.012536402327.77 schema:affiliation grid-institutes:grid.419528.3
134 schema:familyName Shetty
135 schema:givenName Rakshith
136 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012536402327.77
137 rdf:type schema:Person
138 sg:person.013361072755.17 schema:affiliation grid-institutes:grid.507511.7
139 schema:familyName Fritz
140 schema:givenName Mario
141 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013361072755.17
142 rdf:type schema:Person
143 grid-institutes:grid.419528.3 schema:alternateName Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany
144 schema:name Max Planck Institute for Informatics, Saarland Informatics Campus, Saarbrücken, Germany
145 rdf:type schema:Organization
146 grid-institutes:grid.507511.7 schema:alternateName CISPA Helmholtz Center for Information Security, Saarbrücken, Germany
147 schema:name CISPA Helmholtz Center for Information Security, Saarbrücken, Germany
148 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...