TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark View Full Text


Ontology type: schema:Chapter      Open Access: True


Chapter Info

DATE

2014

AUTHORS

Peter Boncz , Thomas Neumann , Orri Erling

ABSTRACT

The TPC-D benchmark was developed almost 20 years ago, and even though its current existence as TPC-H could be considered superseded by TPC-DS, one can still learn from it. We focus on the technical level, summarizing the challenges posed by the TPC-H workload as we now understand them, which we call “choke points”. We identify 28 different such choke points, grouped into six categories: Aggregation Performance, Join Performance, Data Access Locality, Expression Calculation, Correlated Subqueries and Parallel Execution. On the meta-level, we make the point that the rich set of choke-points found in TPC-H sets an example on how to design future DBMS benchmarks. More... »

PAGES

61-76

References to SciGraph publications

  • 2009. The Art of Building a Good Benchmark in PERFORMANCE EVALUATION AND BENCHMARKING
  • Book

    TITLE

    Performance Characterization and Benchmarking

    ISBN

    978-3-319-04935-9
    978-3-319-04936-6

    Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5

    DOI

    http://dx.doi.org/10.1007/978-3-319-04936-6_5

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1033535672


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0806", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information Systems", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "Centrum Wiskunde and Informatica", 
              "id": "https://www.grid.ac/institutes/grid.6054.7", 
              "name": [
                "CWI, Amsterdam, The Netherlands"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Boncz", 
            "givenName": "Peter", 
            "id": "sg:person.015341641231.33", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015341641231.33"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Technical University Munich", 
              "id": "https://www.grid.ac/institutes/grid.6936.a", 
              "name": [
                "Technical University Munich, Germany"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Neumann", 
            "givenName": "Thomas", 
            "id": "sg:person.015477771362.08", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015477771362.08"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "OpenLink Software (United Kingdom)", 
              "id": "https://www.grid.ac/institutes/grid.426164.3", 
              "name": [
                "Openlink Software, United Kingdom"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Erling", 
            "givenName": "Orri", 
            "id": "sg:person.011715705740.17", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011715705740.17"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "https://doi.org/10.1145/152610.152611", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1007812817"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1457150.1457160", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1032631476"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1376616.1376672", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1037499341"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1559845.1559911", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1040723285"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-10424-4_3", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1048186905", 
              "https://doi.org/10.1007/978-3-642-10424-4_3"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.2013.6544837", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093438827"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.2001.914873", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093803682"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.2013.6544839", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093904188"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.2007.367848", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094882172"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.1996.492194", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095265855"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2014", 
        "datePublishedReg": "2014-01-01", 
        "description": "The TPC-D benchmark was developed almost 20 years ago, and even though its current existence as TPC-H could be considered superseded by TPC-DS, one can still learn from it. We focus on the technical level, summarizing the challenges posed by the TPC-H workload as we now understand them, which we call \u201cchoke points\u201d. We identify 28 different such choke points, grouped into six categories: Aggregation Performance, Join Performance, Data Access Locality, Expression Calculation, Correlated Subqueries and Parallel Execution. On the meta-level, we make the point that the rich set of choke-points found in TPC-H sets an example on how to design future DBMS benchmarks.", 
        "editor": [
          {
            "familyName": "Nambiar", 
            "givenName": "Raghunath", 
            "type": "Person"
          }, 
          {
            "familyName": "Poess", 
            "givenName": "Meikel", 
            "type": "Person"
          }
        ], 
        "genre": "chapter", 
        "id": "sg:pub.10.1007/978-3-319-04936-6_5", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": true, 
        "isPartOf": {
          "isbn": [
            "978-3-319-04935-9", 
            "978-3-319-04936-6"
          ], 
          "name": "Performance Characterization and Benchmarking", 
          "type": "Book"
        }, 
        "name": "TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark", 
        "pagination": "61-76", 
        "productId": [
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/978-3-319-04936-6_5"
            ]
          }, 
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "9fc1322c77497f3d43b333be80ff6bb02e6c9d050d5bdd04318f8eacfe19818a"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1033535672"
            ]
          }
        ], 
        "publisher": {
          "location": "Cham", 
          "name": "Springer International Publishing", 
          "type": "Organisation"
        }, 
        "sameAs": [
          "https://doi.org/10.1007/978-3-319-04936-6_5", 
          "https://app.dimensions.ai/details/publication/pub.1033535672"
        ], 
        "sdDataset": "chapters", 
        "sdDatePublished": "2019-04-15T19:10", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8684_00000264.jsonl", 
        "type": "Chapter", 
        "url": "http://link.springer.com/10.1007/978-3-319-04936-6_5"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5'


     

    This table displays all metadata directly associated to this object as RDF triples.

    121 TRIPLES      23 PREDICATES      37 URIs      20 LITERALS      8 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/978-3-319-04936-6_5 schema:about anzsrc-for:08
    2 anzsrc-for:0806
    3 schema:author Neb00e45cfcc84be886febd4c6a1462e0
    4 schema:citation sg:pub.10.1007/978-3-642-10424-4_3
    5 https://doi.org/10.1109/icde.1996.492194
    6 https://doi.org/10.1109/icde.2001.914873
    7 https://doi.org/10.1109/icde.2007.367848
    8 https://doi.org/10.1109/icde.2013.6544837
    9 https://doi.org/10.1109/icde.2013.6544839
    10 https://doi.org/10.1145/1376616.1376672
    11 https://doi.org/10.1145/1457150.1457160
    12 https://doi.org/10.1145/152610.152611
    13 https://doi.org/10.1145/1559845.1559911
    14 schema:datePublished 2014
    15 schema:datePublishedReg 2014-01-01
    16 schema:description The TPC-D benchmark was developed almost 20 years ago, and even though its current existence as TPC-H could be considered superseded by TPC-DS, one can still learn from it. We focus on the technical level, summarizing the challenges posed by the TPC-H workload as we now understand them, which we call “choke points”. We identify 28 different such choke points, grouped into six categories: Aggregation Performance, Join Performance, Data Access Locality, Expression Calculation, Correlated Subqueries and Parallel Execution. On the meta-level, we make the point that the rich set of choke-points found in TPC-H sets an example on how to design future DBMS benchmarks.
    17 schema:editor N5df6e64d8e154dfba7ed06b5c20c9cf0
    18 schema:genre chapter
    19 schema:inLanguage en
    20 schema:isAccessibleForFree true
    21 schema:isPartOf N024e20f77d654c1792d5d4f03731c3a5
    22 schema:name TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark
    23 schema:pagination 61-76
    24 schema:productId N1d9a87782ab941a6b9e03ad7183cb592
    25 N60e3b00020234715b72d213fd16715ad
    26 Nbe7ea1626e864a8fa59c315f041be654
    27 schema:publisher N38c8d36e6a72401aa02a157c3102e667
    28 schema:sameAs https://app.dimensions.ai/details/publication/pub.1033535672
    29 https://doi.org/10.1007/978-3-319-04936-6_5
    30 schema:sdDatePublished 2019-04-15T19:10
    31 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    32 schema:sdPublisher N244860bf63954a568ce9fb0cee7ff14e
    33 schema:url http://link.springer.com/10.1007/978-3-319-04936-6_5
    34 sgo:license sg:explorer/license/
    35 sgo:sdDataset chapters
    36 rdf:type schema:Chapter
    37 N024e20f77d654c1792d5d4f03731c3a5 schema:isbn 978-3-319-04935-9
    38 978-3-319-04936-6
    39 schema:name Performance Characterization and Benchmarking
    40 rdf:type schema:Book
    41 N1cde3c77508847a19fb15ed6f8b2c881 rdf:first sg:person.011715705740.17
    42 rdf:rest rdf:nil
    43 N1d9a87782ab941a6b9e03ad7183cb592 schema:name dimensions_id
    44 schema:value pub.1033535672
    45 rdf:type schema:PropertyValue
    46 N244860bf63954a568ce9fb0cee7ff14e schema:name Springer Nature - SN SciGraph project
    47 rdf:type schema:Organization
    48 N38c8d36e6a72401aa02a157c3102e667 schema:location Cham
    49 schema:name Springer International Publishing
    50 rdf:type schema:Organisation
    51 N53764e608ee34edda0d32f449c3b762d schema:familyName Poess
    52 schema:givenName Meikel
    53 rdf:type schema:Person
    54 N5df6e64d8e154dfba7ed06b5c20c9cf0 rdf:first N936c89e4acbd44de97cb90aa2ccca827
    55 rdf:rest Nda06664f0b2e40a6aae9d01be8ad7d6f
    56 N60e3b00020234715b72d213fd16715ad schema:name doi
    57 schema:value 10.1007/978-3-319-04936-6_5
    58 rdf:type schema:PropertyValue
    59 N936c89e4acbd44de97cb90aa2ccca827 schema:familyName Nambiar
    60 schema:givenName Raghunath
    61 rdf:type schema:Person
    62 Nb7c467d06ffe4565bef0c81b66eafff6 rdf:first sg:person.015477771362.08
    63 rdf:rest N1cde3c77508847a19fb15ed6f8b2c881
    64 Nbe7ea1626e864a8fa59c315f041be654 schema:name readcube_id
    65 schema:value 9fc1322c77497f3d43b333be80ff6bb02e6c9d050d5bdd04318f8eacfe19818a
    66 rdf:type schema:PropertyValue
    67 Nda06664f0b2e40a6aae9d01be8ad7d6f rdf:first N53764e608ee34edda0d32f449c3b762d
    68 rdf:rest rdf:nil
    69 Neb00e45cfcc84be886febd4c6a1462e0 rdf:first sg:person.015341641231.33
    70 rdf:rest Nb7c467d06ffe4565bef0c81b66eafff6
    71 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    72 schema:name Information and Computing Sciences
    73 rdf:type schema:DefinedTerm
    74 anzsrc-for:0806 schema:inDefinedTermSet anzsrc-for:
    75 schema:name Information Systems
    76 rdf:type schema:DefinedTerm
    77 sg:person.011715705740.17 schema:affiliation https://www.grid.ac/institutes/grid.426164.3
    78 schema:familyName Erling
    79 schema:givenName Orri
    80 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011715705740.17
    81 rdf:type schema:Person
    82 sg:person.015341641231.33 schema:affiliation https://www.grid.ac/institutes/grid.6054.7
    83 schema:familyName Boncz
    84 schema:givenName Peter
    85 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015341641231.33
    86 rdf:type schema:Person
    87 sg:person.015477771362.08 schema:affiliation https://www.grid.ac/institutes/grid.6936.a
    88 schema:familyName Neumann
    89 schema:givenName Thomas
    90 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015477771362.08
    91 rdf:type schema:Person
    92 sg:pub.10.1007/978-3-642-10424-4_3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1048186905
    93 https://doi.org/10.1007/978-3-642-10424-4_3
    94 rdf:type schema:CreativeWork
    95 https://doi.org/10.1109/icde.1996.492194 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095265855
    96 rdf:type schema:CreativeWork
    97 https://doi.org/10.1109/icde.2001.914873 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093803682
    98 rdf:type schema:CreativeWork
    99 https://doi.org/10.1109/icde.2007.367848 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094882172
    100 rdf:type schema:CreativeWork
    101 https://doi.org/10.1109/icde.2013.6544837 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093438827
    102 rdf:type schema:CreativeWork
    103 https://doi.org/10.1109/icde.2013.6544839 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093904188
    104 rdf:type schema:CreativeWork
    105 https://doi.org/10.1145/1376616.1376672 schema:sameAs https://app.dimensions.ai/details/publication/pub.1037499341
    106 rdf:type schema:CreativeWork
    107 https://doi.org/10.1145/1457150.1457160 schema:sameAs https://app.dimensions.ai/details/publication/pub.1032631476
    108 rdf:type schema:CreativeWork
    109 https://doi.org/10.1145/152610.152611 schema:sameAs https://app.dimensions.ai/details/publication/pub.1007812817
    110 rdf:type schema:CreativeWork
    111 https://doi.org/10.1145/1559845.1559911 schema:sameAs https://app.dimensions.ai/details/publication/pub.1040723285
    112 rdf:type schema:CreativeWork
    113 https://www.grid.ac/institutes/grid.426164.3 schema:alternateName OpenLink Software (United Kingdom)
    114 schema:name Openlink Software, United Kingdom
    115 rdf:type schema:Organization
    116 https://www.grid.ac/institutes/grid.6054.7 schema:alternateName Centrum Wiskunde and Informatica
    117 schema:name CWI, Amsterdam, The Netherlands
    118 rdf:type schema:Organization
    119 https://www.grid.ac/institutes/grid.6936.a schema:alternateName Technical University Munich
    120 schema:name Technical University Munich, Germany
    121 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...