VDDA: automatic visualization-driven data aggregation in relational databases View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2015-08-06

AUTHORS

Uwe Jugel, Zbigniew Jerzak, Gregor Hackenbroich, Volker Markl

ABSTRACT

Contemporary RDBMS-based systems for visualization of high-volume numerical data have difficulty to cope with the hard latency requirements and high ingestion rates of interactive visualizations. Existing solutions for lowering the volume of large data sets disregard the spatial properties of visualizations, resulting in visualization errors. In this work, we introduce VDDA, a visualization-driven data aggregation that models visual aggregation at the pixel level as data aggregation at the query level. Based on the M4 aggregation for producing pixel-perfect line charts from highly reduced data subsets, we define a complete set of data reduction operators that simulate the overplotting behavior of the most frequently used chart types. Relying only on the relational algebra and the common data aggregation functions, our approach is generic and applicable to any visualization system that consumes data stored in relational databases. We demonstrate our visualization-driven data aggregation using real-world data sets from high-tech manufacturing, stock markets, and sports analytics, reducing data volumes by up to two orders of magnitude, while preserving pixel-perfect visualizations, as producible from the raw data. More... »

PAGES

53-77

References to SciGraph publications

  • 2011. Approximate Query on Historical Stream Data in DATABASE AND EXPERT SYSTEMS APPLICATIONS
  • 2010. Recursive Query Facilities in Relational Databases: A Survey in DATABASE THEORY AND APPLICATION, BIO-SCIENCE AND BIO-TECHNOLOGY
  • 2000. A Simple Dimensionality Reduction Technique for Fast Similarity Search in Large Time Series Databases in KNOWLEDGE DISCOVERY AND DATA MINING. CURRENT ISSUES AND NEW APPLICATIONS
  • 2003. Pushing the Limit in Visual Data Exploration: Techniques and Applications in KI 2003: ADVANCES IN ARTIFICIAL INTELLIGENCE
  • 2011. Visualization of Time-Oriented Data in NONE
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s00778-015-0396-z

    DOI

    http://dx.doi.org/10.1007/s00778-015-0396-z

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1003864112


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Artificial Intelligence and Image Processing", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0806", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information Systems", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "SAP SE, Walldorf/Dresden, Germany", 
              "id": "http://www.grid.ac/institutes/grid.19008.30", 
              "name": [
                "SAP SE, Walldorf/Dresden, Germany"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Jugel", 
            "givenName": "Uwe", 
            "id": "sg:person.011044345723.73", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011044345723.73"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "SAP SE, Walldorf/Dresden, Germany", 
              "id": "http://www.grid.ac/institutes/grid.19008.30", 
              "name": [
                "SAP SE, Walldorf/Dresden, Germany"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Jerzak", 
            "givenName": "Zbigniew", 
            "id": "sg:person.013054252115.87", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013054252115.87"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "SAP SE, Walldorf/Dresden, Germany", 
              "id": "http://www.grid.ac/institutes/grid.19008.30", 
              "name": [
                "SAP SE, Walldorf/Dresden, Germany"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Hackenbroich", 
            "givenName": "Gregor", 
            "id": "sg:person.012241057077.14", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012241057077.14"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Technische Universit\u00e4t Berlin, Berlin, Germany", 
              "id": "http://www.grid.ac/institutes/grid.6734.6", 
              "name": [
                "Technische Universit\u00e4t Berlin, Berlin, Germany"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Markl", 
            "givenName": "Volker", 
            "id": "sg:person.016444434731.47", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016444434731.47"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/978-3-642-23091-2_12", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1044882672", 
              "https://doi.org/10.1007/978-3-642-23091-2_12"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/3-540-45571-x_14", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1036055360", 
              "https://doi.org/10.1007/3-540-45571-x_14"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-540-39451-8_4", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1011573470", 
              "https://doi.org/10.1007/978-3-540-39451-8_4"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-17622-7_10", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1029867160", 
              "https://doi.org/10.1007/978-3-642-17622-7_10"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-0-85729-079-3", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1047104370", 
              "https://doi.org/10.1007/978-0-85729-079-3"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2015-08-06", 
        "datePublishedReg": "2015-08-06", 
        "description": "Contemporary RDBMS-based systems for visualization of high-volume numerical data have difficulty to cope with the hard latency requirements and high ingestion rates of interactive visualizations. Existing solutions for lowering the volume of large data sets disregard the spatial properties of visualizations, resulting in visualization errors. In this work, we introduce VDDA, a visualization-driven data aggregation that models visual aggregation at the pixel level as data aggregation at the query level. Based on the M4 aggregation for producing pixel-perfect line charts from highly reduced data subsets, we define a complete set of data reduction operators that simulate the overplotting behavior of the most frequently used chart types. Relying only on the relational algebra and the common data aggregation functions, our approach is generic and applicable to any visualization system that consumes data stored in relational databases. We demonstrate our visualization-driven data aggregation using real-world data sets from high-tech manufacturing, stock markets, and sports analytics, reducing data volumes by up to two orders of magnitude, while preserving pixel-perfect visualizations, as producible from the raw data.", 
        "genre": "article", 
        "id": "sg:pub.10.1007/s00778-015-0396-z", 
        "isAccessibleForFree": false, 
        "isPartOf": [
          {
            "id": "sg:journal.1044889", 
            "issn": [
              "1066-8888", 
              "0949-877X"
            ], 
            "name": "The VLDB Journal", 
            "publisher": "Springer Nature", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "1", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "25"
          }
        ], 
        "keywords": [
          "data aggregation", 
          "relational database", 
          "real-world data sets", 
          "hard latency requirements", 
          "data aggregation functions", 
          "data reduction operators", 
          "data sets", 
          "large data sets", 
          "query level", 
          "interactive visualization", 
          "latency requirements", 
          "sports analytics", 
          "data volume", 
          "relational algebra", 
          "visualization system", 
          "visual aggregation", 
          "visualization error", 
          "pixel level", 
          "raw data", 
          "reduction operators", 
          "chart types", 
          "aggregation functions", 
          "data subsets", 
          "line charts", 
          "visualization", 
          "set", 
          "high-tech manufacturing", 
          "spatial properties", 
          "analytics", 
          "database", 
          "numerical data", 
          "system", 
          "complete set", 
          "requirements", 
          "data", 
          "operators", 
          "error", 
          "aggregation", 
          "manufacturing", 
          "orders of magnitude", 
          "solution", 
          "work", 
          "stock market", 
          "order", 
          "subset", 
          "difficulties", 
          "algebra", 
          "market", 
          "charts", 
          "volume", 
          "types", 
          "higher ingestion rates", 
          "function", 
          "behavior", 
          "levels", 
          "rate", 
          "properties", 
          "magnitude", 
          "ingestion rates", 
          "approach"
        ], 
        "name": "VDDA: automatic visualization-driven data aggregation in relational databases", 
        "pagination": "53-77", 
        "productId": [
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1003864112"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s00778-015-0396-z"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s00778-015-0396-z", 
          "https://app.dimensions.ai/details/publication/pub.1003864112"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2022-12-01T06:33", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-springernature-scigraph/baseset/20221201/entities/gbq_results/article/article_663.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://doi.org/10.1007/s00778-015-0396-z"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s00778-015-0396-z'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s00778-015-0396-z'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s00778-015-0396-z'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s00778-015-0396-z'


     

    This table displays all metadata directly associated to this object as RDF triples.

    165 TRIPLES      21 PREDICATES      90 URIs      76 LITERALS      6 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s00778-015-0396-z schema:about anzsrc-for:08
    2 anzsrc-for:0801
    3 anzsrc-for:0806
    4 schema:author N2e82d9c87e9342a1a2761e6ff5d07b1f
    5 schema:citation sg:pub.10.1007/3-540-45571-x_14
    6 sg:pub.10.1007/978-0-85729-079-3
    7 sg:pub.10.1007/978-3-540-39451-8_4
    8 sg:pub.10.1007/978-3-642-17622-7_10
    9 sg:pub.10.1007/978-3-642-23091-2_12
    10 schema:datePublished 2015-08-06
    11 schema:datePublishedReg 2015-08-06
    12 schema:description Contemporary RDBMS-based systems for visualization of high-volume numerical data have difficulty to cope with the hard latency requirements and high ingestion rates of interactive visualizations. Existing solutions for lowering the volume of large data sets disregard the spatial properties of visualizations, resulting in visualization errors. In this work, we introduce VDDA, a visualization-driven data aggregation that models visual aggregation at the pixel level as data aggregation at the query level. Based on the M4 aggregation for producing pixel-perfect line charts from highly reduced data subsets, we define a complete set of data reduction operators that simulate the overplotting behavior of the most frequently used chart types. Relying only on the relational algebra and the common data aggregation functions, our approach is generic and applicable to any visualization system that consumes data stored in relational databases. We demonstrate our visualization-driven data aggregation using real-world data sets from high-tech manufacturing, stock markets, and sports analytics, reducing data volumes by up to two orders of magnitude, while preserving pixel-perfect visualizations, as producible from the raw data.
    13 schema:genre article
    14 schema:isAccessibleForFree false
    15 schema:isPartOf N0e49476700b94b119e0dcf6b80916e61
    16 Na55ec5e7a3d04ec1b0032c4b7cdcb658
    17 sg:journal.1044889
    18 schema:keywords aggregation
    19 aggregation functions
    20 algebra
    21 analytics
    22 approach
    23 behavior
    24 chart types
    25 charts
    26 complete set
    27 data
    28 data aggregation
    29 data aggregation functions
    30 data reduction operators
    31 data sets
    32 data subsets
    33 data volume
    34 database
    35 difficulties
    36 error
    37 function
    38 hard latency requirements
    39 high-tech manufacturing
    40 higher ingestion rates
    41 ingestion rates
    42 interactive visualization
    43 large data sets
    44 latency requirements
    45 levels
    46 line charts
    47 magnitude
    48 manufacturing
    49 market
    50 numerical data
    51 operators
    52 order
    53 orders of magnitude
    54 pixel level
    55 properties
    56 query level
    57 rate
    58 raw data
    59 real-world data sets
    60 reduction operators
    61 relational algebra
    62 relational database
    63 requirements
    64 set
    65 solution
    66 spatial properties
    67 sports analytics
    68 stock market
    69 subset
    70 system
    71 types
    72 visual aggregation
    73 visualization
    74 visualization error
    75 visualization system
    76 volume
    77 work
    78 schema:name VDDA: automatic visualization-driven data aggregation in relational databases
    79 schema:pagination 53-77
    80 schema:productId N40bb494caec5484e932e65be43a2e952
    81 N44d4736cc9504778adad4ec7cfae5e56
    82 schema:sameAs https://app.dimensions.ai/details/publication/pub.1003864112
    83 https://doi.org/10.1007/s00778-015-0396-z
    84 schema:sdDatePublished 2022-12-01T06:33
    85 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    86 schema:sdPublisher N8ca581e6c4af40658be84dba55bd69d0
    87 schema:url https://doi.org/10.1007/s00778-015-0396-z
    88 sgo:license sg:explorer/license/
    89 sgo:sdDataset articles
    90 rdf:type schema:ScholarlyArticle
    91 N0e49476700b94b119e0dcf6b80916e61 schema:volumeNumber 25
    92 rdf:type schema:PublicationVolume
    93 N1c984c241dbc4fc9b8b6f3438ea7c35a rdf:first sg:person.016444434731.47
    94 rdf:rest rdf:nil
    95 N2e82d9c87e9342a1a2761e6ff5d07b1f rdf:first sg:person.011044345723.73
    96 rdf:rest N40698118c99741aa88af44a9e4498d86
    97 N40698118c99741aa88af44a9e4498d86 rdf:first sg:person.013054252115.87
    98 rdf:rest Nad9a2f53056543278e3f2cd979f7048a
    99 N40bb494caec5484e932e65be43a2e952 schema:name doi
    100 schema:value 10.1007/s00778-015-0396-z
    101 rdf:type schema:PropertyValue
    102 N44d4736cc9504778adad4ec7cfae5e56 schema:name dimensions_id
    103 schema:value pub.1003864112
    104 rdf:type schema:PropertyValue
    105 N8ca581e6c4af40658be84dba55bd69d0 schema:name Springer Nature - SN SciGraph project
    106 rdf:type schema:Organization
    107 Na55ec5e7a3d04ec1b0032c4b7cdcb658 schema:issueNumber 1
    108 rdf:type schema:PublicationIssue
    109 Nad9a2f53056543278e3f2cd979f7048a rdf:first sg:person.012241057077.14
    110 rdf:rest N1c984c241dbc4fc9b8b6f3438ea7c35a
    111 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    112 schema:name Information and Computing Sciences
    113 rdf:type schema:DefinedTerm
    114 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
    115 schema:name Artificial Intelligence and Image Processing
    116 rdf:type schema:DefinedTerm
    117 anzsrc-for:0806 schema:inDefinedTermSet anzsrc-for:
    118 schema:name Information Systems
    119 rdf:type schema:DefinedTerm
    120 sg:journal.1044889 schema:issn 0949-877X
    121 1066-8888
    122 schema:name The VLDB Journal
    123 schema:publisher Springer Nature
    124 rdf:type schema:Periodical
    125 sg:person.011044345723.73 schema:affiliation grid-institutes:grid.19008.30
    126 schema:familyName Jugel
    127 schema:givenName Uwe
    128 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011044345723.73
    129 rdf:type schema:Person
    130 sg:person.012241057077.14 schema:affiliation grid-institutes:grid.19008.30
    131 schema:familyName Hackenbroich
    132 schema:givenName Gregor
    133 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012241057077.14
    134 rdf:type schema:Person
    135 sg:person.013054252115.87 schema:affiliation grid-institutes:grid.19008.30
    136 schema:familyName Jerzak
    137 schema:givenName Zbigniew
    138 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013054252115.87
    139 rdf:type schema:Person
    140 sg:person.016444434731.47 schema:affiliation grid-institutes:grid.6734.6
    141 schema:familyName Markl
    142 schema:givenName Volker
    143 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016444434731.47
    144 rdf:type schema:Person
    145 sg:pub.10.1007/3-540-45571-x_14 schema:sameAs https://app.dimensions.ai/details/publication/pub.1036055360
    146 https://doi.org/10.1007/3-540-45571-x_14
    147 rdf:type schema:CreativeWork
    148 sg:pub.10.1007/978-0-85729-079-3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047104370
    149 https://doi.org/10.1007/978-0-85729-079-3
    150 rdf:type schema:CreativeWork
    151 sg:pub.10.1007/978-3-540-39451-8_4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1011573470
    152 https://doi.org/10.1007/978-3-540-39451-8_4
    153 rdf:type schema:CreativeWork
    154 sg:pub.10.1007/978-3-642-17622-7_10 schema:sameAs https://app.dimensions.ai/details/publication/pub.1029867160
    155 https://doi.org/10.1007/978-3-642-17622-7_10
    156 rdf:type schema:CreativeWork
    157 sg:pub.10.1007/978-3-642-23091-2_12 schema:sameAs https://app.dimensions.ai/details/publication/pub.1044882672
    158 https://doi.org/10.1007/978-3-642-23091-2_12
    159 rdf:type schema:CreativeWork
    160 grid-institutes:grid.19008.30 schema:alternateName SAP SE, Walldorf/Dresden, Germany
    161 schema:name SAP SE, Walldorf/Dresden, Germany
    162 rdf:type schema:Organization
    163 grid-institutes:grid.6734.6 schema:alternateName Technische Universität Berlin, Berlin, Germany
    164 schema:name Technische Universität Berlin, Berlin, Germany
    165 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...