A Deep Learning Approach to Clustering Visual Arts View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

2022-08-16

AUTHORS

Giovanna Castellano, Gennaro Vessio

ABSTRACT

Clustering artworks is difficult for several reasons. On the one hand, recognizing meaningful patterns based on domain knowledge and visual perception is extremely hard. On the other hand, applying traditional clustering and feature reduction techniques to the highly dimensional pixel space can be ineffective. To address these issues, in this paper we propose DELIUS: a DEep learning approach to cLustering vIsUal artS. The method uses a pre-trained convolutional network to extract features and then feeds these features into a deep embedded clustering model, where the task of mapping the input data to a latent space is jointly optimized with the task of finding a set of cluster centroids in this latent space. Quantitative and qualitative experimental results show the effectiveness of the proposed method. DELIUS can be useful for several tasks related to art analysis, in particular visual link retrieval and historical knowledge discovery in painting datasets. More... »

PAGES

2590-2605

References to SciGraph publications

  • 2012. Artistic Image Classification: An Analysis on the PRINTART Database in COMPUTER VISION – ECCV 2012
  • 2015-03-19. In Search of Art in COMPUTER VISION - ECCV 2014 WORKSHOPS
  • 2019-12-21. ContextNet: representation and exploration for painting classification and retrieval in context in INTERNATIONAL JOURNAL OF MULTIMEDIA INFORMATION RETRIEVAL
  • 2014-06-14. Painting-91: a large scale database for computational painting categorization in MACHINE VISION AND APPLICATIONS
  • 2019-01-29. How to Read Paintings: Semantic Art Understanding with Multi-modal Retrieval in COMPUTER VISION – ECCV 2018 WORKSHOPS
  • 2019-01-29. Weakly Supervised Object Detection in Artworks in COMPUTER VISION – ECCV 2018 WORKSHOPS
  • 2014-08-19. Toward automated discovery of artistic influence in MULTIMEDIA TOOLS AND APPLICATIONS
  • 2020-10-18. Visual link retrieval and knowledge discovery in painting datasets in MULTIMEDIA TOOLS AND APPLICATIONS
  • 2021-04-02. Deep learning approaches to pattern extraction and recognition in paintings and drawings: an overview in NEURAL COMPUTING AND APPLICATIONS
  • 2016-09-18. Detecting People in Artwork with CNNs in COMPUTER VISION – ECCV 2016 WORKSHOPS
  • 2018-12-21. Deep clustering of protein folding simulations in BMC BIOINFORMATICS
  • 2015-05-27. Deep learning in NATURE
  • 2017-10-26. Deep Clustering with Convolutional Autoencoders in NEURAL INFORMATION PROCESSING
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s11263-022-01664-y

    DOI

    http://dx.doi.org/10.1007/s11263-022-01664-y

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1150272116


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Artificial Intelligence and Image Processing", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "Department of Computer Science, University of Bari \u201cAldo Moro\u201d, Bari, Italy", 
              "id": "http://www.grid.ac/institutes/grid.7644.1", 
              "name": [
                "Department of Computer Science, University of Bari \u201cAldo Moro\u201d, Bari, Italy"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Castellano", 
            "givenName": "Giovanna", 
            "id": "sg:person.012417230600.18", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012417230600.18"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Department of Computer Science, University of Bari \u201cAldo Moro\u201d, Bari, Italy", 
              "id": "http://www.grid.ac/institutes/grid.7644.1", 
              "name": [
                "Department of Computer Science, University of Bari \u201cAldo Moro\u201d, Bari, Italy"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Vessio", 
            "givenName": "Gennaro", 
            "id": "sg:person.012646322267.36", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012646322267.36"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/978-3-319-70096-0_39", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1092381123", 
              "https://doi.org/10.1007/978-3-319-70096-0_39"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-46604-0_57", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1030882116", 
              "https://doi.org/10.1007/978-3-319-46604-0_57"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-030-11012-3_53", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1111784315", 
              "https://doi.org/10.1007/978-3-030-11012-3_53"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11042-014-2193-x", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1031839710", 
              "https://doi.org/10.1007/s11042-014-2193-x"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s00521-021-05893-z", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1136887836", 
              "https://doi.org/10.1007/s00521-021-05893-z"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s00138-014-0621-6", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1026943030", 
              "https://doi.org/10.1007/s00138-014-0621-6"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-16178-5_4", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1035800693", 
              "https://doi.org/10.1007/978-3-319-16178-5_4"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1038/nature14539", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1010020120", 
              "https://doi.org/10.1038/nature14539"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1186/s12859-018-2507-5", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1110818855", 
              "https://doi.org/10.1186/s12859-018-2507-5"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-33765-9_11", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1027441221", 
              "https://doi.org/10.1007/978-3-642-33765-9_11"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-030-11012-3_52", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1111784314", 
              "https://doi.org/10.1007/978-3-030-11012-3_52"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s11042-020-09995-z", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1131865630", 
              "https://doi.org/10.1007/s11042-020-09995-z"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s13735-019-00189-4", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1123604260", 
              "https://doi.org/10.1007/s13735-019-00189-4"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2022-08-16", 
        "datePublishedReg": "2022-08-16", 
        "description": "Clustering artworks is difficult for several reasons. On the one hand, recognizing meaningful patterns based on domain knowledge and visual perception is extremely hard. On the other hand, applying traditional clustering and feature reduction techniques to the highly dimensional pixel space can be ineffective. To address these issues, in this paper we propose DELIUS: a DEep learning approach to cLustering vIsUal artS. The method uses a pre-trained convolutional network to extract features and then feeds these features into a deep embedded clustering model, where the task of mapping the input data to a latent space is jointly optimized with the task of finding a set of cluster centroids in this latent space. Quantitative and qualitative experimental results show the effectiveness of the proposed method. DELIUS can be useful for several tasks related to art analysis, in particular visual link retrieval and historical knowledge discovery in painting datasets.", 
        "genre": "article", 
        "id": "sg:pub.10.1007/s11263-022-01664-y", 
        "isAccessibleForFree": true, 
        "isPartOf": [
          {
            "id": "sg:journal.1032807", 
            "issn": [
              "0920-5691", 
              "1573-1405"
            ], 
            "name": "International Journal of Computer Vision", 
            "publisher": "Springer Nature", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "11", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "130"
          }
        ], 
        "keywords": [
          "deep learning approach", 
          "learning approach", 
          "latent space", 
          "pre-trained convolutional networks", 
          "feature reduction techniques", 
          "qualitative experimental results", 
          "knowledge discovery", 
          "convolutional network", 
          "domain knowledge", 
          "traditional clustering", 
          "pixel space", 
          "cluster centroids", 
          "clustering model", 
          "meaningful patterns", 
          "input data", 
          "task", 
          "experimental results", 
          "art analysis", 
          "reduction techniques", 
          "visual perception", 
          "retrieval", 
          "dataset", 
          "network", 
          "clustering", 
          "space", 
          "features", 
          "centroid", 
          "visuals", 
          "set", 
          "visual arts", 
          "art", 
          "method", 
          "effectiveness", 
          "issues", 
          "technique", 
          "hand", 
          "Delius", 
          "artworks", 
          "knowledge", 
          "discovery", 
          "model", 
          "data", 
          "results", 
          "reasons", 
          "patterns", 
          "perception", 
          "analysis", 
          "approach", 
          "paper"
        ], 
        "name": "A Deep Learning Approach to Clustering Visual Arts", 
        "pagination": "2590-2605", 
        "productId": [
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1150272116"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s11263-022-01664-y"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s11263-022-01664-y", 
          "https://app.dimensions.ai/details/publication/pub.1150272116"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2022-12-01T06:44", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-springernature-scigraph/baseset/20221201/entities/gbq_results/article/article_959.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://doi.org/10.1007/s11263-022-01664-y"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s11263-022-01664-y'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s11263-022-01664-y'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s11263-022-01664-y'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s11263-022-01664-y'


     

    This table displays all metadata directly associated to this object as RDF triples.

    165 TRIPLES      21 PREDICATES      86 URIs      65 LITERALS      6 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s11263-022-01664-y schema:about anzsrc-for:08
    2 anzsrc-for:0801
    3 schema:author N5ce2cd1b1b264bfa897ada0e69c672b5
    4 schema:citation sg:pub.10.1007/978-3-030-11012-3_52
    5 sg:pub.10.1007/978-3-030-11012-3_53
    6 sg:pub.10.1007/978-3-319-16178-5_4
    7 sg:pub.10.1007/978-3-319-46604-0_57
    8 sg:pub.10.1007/978-3-319-70096-0_39
    9 sg:pub.10.1007/978-3-642-33765-9_11
    10 sg:pub.10.1007/s00138-014-0621-6
    11 sg:pub.10.1007/s00521-021-05893-z
    12 sg:pub.10.1007/s11042-014-2193-x
    13 sg:pub.10.1007/s11042-020-09995-z
    14 sg:pub.10.1007/s13735-019-00189-4
    15 sg:pub.10.1038/nature14539
    16 sg:pub.10.1186/s12859-018-2507-5
    17 schema:datePublished 2022-08-16
    18 schema:datePublishedReg 2022-08-16
    19 schema:description Clustering artworks is difficult for several reasons. On the one hand, recognizing meaningful patterns based on domain knowledge and visual perception is extremely hard. On the other hand, applying traditional clustering and feature reduction techniques to the highly dimensional pixel space can be ineffective. To address these issues, in this paper we propose DELIUS: a DEep learning approach to cLustering vIsUal artS. The method uses a pre-trained convolutional network to extract features and then feeds these features into a deep embedded clustering model, where the task of mapping the input data to a latent space is jointly optimized with the task of finding a set of cluster centroids in this latent space. Quantitative and qualitative experimental results show the effectiveness of the proposed method. DELIUS can be useful for several tasks related to art analysis, in particular visual link retrieval and historical knowledge discovery in painting datasets.
    20 schema:genre article
    21 schema:isAccessibleForFree true
    22 schema:isPartOf N33cac0cc0954460f8ea045b0770bea76
    23 N9e0b8a48024c4a8fbd008b5b50e66a2d
    24 sg:journal.1032807
    25 schema:keywords Delius
    26 analysis
    27 approach
    28 art
    29 art analysis
    30 artworks
    31 centroid
    32 cluster centroids
    33 clustering
    34 clustering model
    35 convolutional network
    36 data
    37 dataset
    38 deep learning approach
    39 discovery
    40 domain knowledge
    41 effectiveness
    42 experimental results
    43 feature reduction techniques
    44 features
    45 hand
    46 input data
    47 issues
    48 knowledge
    49 knowledge discovery
    50 latent space
    51 learning approach
    52 meaningful patterns
    53 method
    54 model
    55 network
    56 paper
    57 patterns
    58 perception
    59 pixel space
    60 pre-trained convolutional networks
    61 qualitative experimental results
    62 reasons
    63 reduction techniques
    64 results
    65 retrieval
    66 set
    67 space
    68 task
    69 technique
    70 traditional clustering
    71 visual arts
    72 visual perception
    73 visuals
    74 schema:name A Deep Learning Approach to Clustering Visual Arts
    75 schema:pagination 2590-2605
    76 schema:productId Nbc66b7296643404e9de9ada293432ebd
    77 Nfbe2466391b94b219a9c1e75c1d80777
    78 schema:sameAs https://app.dimensions.ai/details/publication/pub.1150272116
    79 https://doi.org/10.1007/s11263-022-01664-y
    80 schema:sdDatePublished 2022-12-01T06:44
    81 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    82 schema:sdPublisher Ne462c65b61574620901408c7ab27a96f
    83 schema:url https://doi.org/10.1007/s11263-022-01664-y
    84 sgo:license sg:explorer/license/
    85 sgo:sdDataset articles
    86 rdf:type schema:ScholarlyArticle
    87 N33cac0cc0954460f8ea045b0770bea76 schema:issueNumber 11
    88 rdf:type schema:PublicationIssue
    89 N5ce2cd1b1b264bfa897ada0e69c672b5 rdf:first sg:person.012417230600.18
    90 rdf:rest Ne01ff702ac894059b9d1b87a7b3a1fe4
    91 N9e0b8a48024c4a8fbd008b5b50e66a2d schema:volumeNumber 130
    92 rdf:type schema:PublicationVolume
    93 Nbc66b7296643404e9de9ada293432ebd schema:name dimensions_id
    94 schema:value pub.1150272116
    95 rdf:type schema:PropertyValue
    96 Ne01ff702ac894059b9d1b87a7b3a1fe4 rdf:first sg:person.012646322267.36
    97 rdf:rest rdf:nil
    98 Ne462c65b61574620901408c7ab27a96f schema:name Springer Nature - SN SciGraph project
    99 rdf:type schema:Organization
    100 Nfbe2466391b94b219a9c1e75c1d80777 schema:name doi
    101 schema:value 10.1007/s11263-022-01664-y
    102 rdf:type schema:PropertyValue
    103 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    104 schema:name Information and Computing Sciences
    105 rdf:type schema:DefinedTerm
    106 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
    107 schema:name Artificial Intelligence and Image Processing
    108 rdf:type schema:DefinedTerm
    109 sg:journal.1032807 schema:issn 0920-5691
    110 1573-1405
    111 schema:name International Journal of Computer Vision
    112 schema:publisher Springer Nature
    113 rdf:type schema:Periodical
    114 sg:person.012417230600.18 schema:affiliation grid-institutes:grid.7644.1
    115 schema:familyName Castellano
    116 schema:givenName Giovanna
    117 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012417230600.18
    118 rdf:type schema:Person
    119 sg:person.012646322267.36 schema:affiliation grid-institutes:grid.7644.1
    120 schema:familyName Vessio
    121 schema:givenName Gennaro
    122 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012646322267.36
    123 rdf:type schema:Person
    124 sg:pub.10.1007/978-3-030-11012-3_52 schema:sameAs https://app.dimensions.ai/details/publication/pub.1111784314
    125 https://doi.org/10.1007/978-3-030-11012-3_52
    126 rdf:type schema:CreativeWork
    127 sg:pub.10.1007/978-3-030-11012-3_53 schema:sameAs https://app.dimensions.ai/details/publication/pub.1111784315
    128 https://doi.org/10.1007/978-3-030-11012-3_53
    129 rdf:type schema:CreativeWork
    130 sg:pub.10.1007/978-3-319-16178-5_4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1035800693
    131 https://doi.org/10.1007/978-3-319-16178-5_4
    132 rdf:type schema:CreativeWork
    133 sg:pub.10.1007/978-3-319-46604-0_57 schema:sameAs https://app.dimensions.ai/details/publication/pub.1030882116
    134 https://doi.org/10.1007/978-3-319-46604-0_57
    135 rdf:type schema:CreativeWork
    136 sg:pub.10.1007/978-3-319-70096-0_39 schema:sameAs https://app.dimensions.ai/details/publication/pub.1092381123
    137 https://doi.org/10.1007/978-3-319-70096-0_39
    138 rdf:type schema:CreativeWork
    139 sg:pub.10.1007/978-3-642-33765-9_11 schema:sameAs https://app.dimensions.ai/details/publication/pub.1027441221
    140 https://doi.org/10.1007/978-3-642-33765-9_11
    141 rdf:type schema:CreativeWork
    142 sg:pub.10.1007/s00138-014-0621-6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1026943030
    143 https://doi.org/10.1007/s00138-014-0621-6
    144 rdf:type schema:CreativeWork
    145 sg:pub.10.1007/s00521-021-05893-z schema:sameAs https://app.dimensions.ai/details/publication/pub.1136887836
    146 https://doi.org/10.1007/s00521-021-05893-z
    147 rdf:type schema:CreativeWork
    148 sg:pub.10.1007/s11042-014-2193-x schema:sameAs https://app.dimensions.ai/details/publication/pub.1031839710
    149 https://doi.org/10.1007/s11042-014-2193-x
    150 rdf:type schema:CreativeWork
    151 sg:pub.10.1007/s11042-020-09995-z schema:sameAs https://app.dimensions.ai/details/publication/pub.1131865630
    152 https://doi.org/10.1007/s11042-020-09995-z
    153 rdf:type schema:CreativeWork
    154 sg:pub.10.1007/s13735-019-00189-4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1123604260
    155 https://doi.org/10.1007/s13735-019-00189-4
    156 rdf:type schema:CreativeWork
    157 sg:pub.10.1038/nature14539 schema:sameAs https://app.dimensions.ai/details/publication/pub.1010020120
    158 https://doi.org/10.1038/nature14539
    159 rdf:type schema:CreativeWork
    160 sg:pub.10.1186/s12859-018-2507-5 schema:sameAs https://app.dimensions.ai/details/publication/pub.1110818855
    161 https://doi.org/10.1186/s12859-018-2507-5
    162 rdf:type schema:CreativeWork
    163 grid-institutes:grid.7644.1 schema:alternateName Department of Computer Science, University of Bari “Aldo Moro”, Bari, Italy
    164 schema:name Department of Computer Science, University of Bari “Aldo Moro”, Bari, Italy
    165 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...