TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark View Full Text


Ontology type: schema:Chapter      Open Access: True


Chapter Info

DATE

2014

AUTHORS

Peter Boncz , Thomas Neumann , Orri Erling

ABSTRACT

The TPC-D benchmark was developed almost 20 years ago, and even though its current existence as TPC-H could be considered superseded by TPC-DS, one can still learn from it. We focus on the technical level, summarizing the challenges posed by the TPC-H workload as we now understand them, which we call “choke points”. We identify 28 different such choke points, grouped into six categories: Aggregation Performance, Join Performance, Data Access Locality, Expression Calculation, Correlated Subqueries and Parallel Execution. On the meta-level, we make the point that the rich set of choke-points found in TPC-H sets an example on how to design future DBMS benchmarks. More... »

PAGES

61-76

References to SciGraph publications

  • 2009. The Art of Building a Good Benchmark in PERFORMANCE EVALUATION AND BENCHMARKING
  • Book

    TITLE

    Performance Characterization and Benchmarking

    ISBN

    978-3-319-04935-9
    978-3-319-04936-6

    Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5

    DOI

    http://dx.doi.org/10.1007/978-3-319-04936-6_5

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1033535672


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0806", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information Systems", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "Centrum Wiskunde and Informatica", 
              "id": "https://www.grid.ac/institutes/grid.6054.7", 
              "name": [
                "CWI, Amsterdam, The Netherlands"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Boncz", 
            "givenName": "Peter", 
            "id": "sg:person.015341641231.33", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015341641231.33"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Technical University Munich", 
              "id": "https://www.grid.ac/institutes/grid.6936.a", 
              "name": [
                "Technical University Munich, Germany"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Neumann", 
            "givenName": "Thomas", 
            "id": "sg:person.015477771362.08", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015477771362.08"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "OpenLink Software (United Kingdom)", 
              "id": "https://www.grid.ac/institutes/grid.426164.3", 
              "name": [
                "Openlink Software, United Kingdom"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Erling", 
            "givenName": "Orri", 
            "id": "sg:person.011715705740.17", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011715705740.17"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "https://doi.org/10.1145/152610.152611", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1007812817"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1457150.1457160", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1032631476"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1376616.1376672", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1037499341"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1559845.1559911", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1040723285"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-10424-4_3", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1048186905", 
              "https://doi.org/10.1007/978-3-642-10424-4_3"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.2013.6544837", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093438827"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.2001.914873", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093803682"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.2013.6544839", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1093904188"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.2007.367848", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094882172"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icde.1996.492194", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095265855"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2014", 
        "datePublishedReg": "2014-01-01", 
        "description": "The TPC-D benchmark was developed almost 20 years ago, and even though its current existence as TPC-H could be considered superseded by TPC-DS, one can still learn from it. We focus on the technical level, summarizing the challenges posed by the TPC-H workload as we now understand them, which we call \u201cchoke points\u201d. We identify 28 different such choke points, grouped into six categories: Aggregation Performance, Join Performance, Data Access Locality, Expression Calculation, Correlated Subqueries and Parallel Execution. On the meta-level, we make the point that the rich set of choke-points found in TPC-H sets an example on how to design future DBMS benchmarks.", 
        "editor": [
          {
            "familyName": "Nambiar", 
            "givenName": "Raghunath", 
            "type": "Person"
          }, 
          {
            "familyName": "Poess", 
            "givenName": "Meikel", 
            "type": "Person"
          }
        ], 
        "genre": "chapter", 
        "id": "sg:pub.10.1007/978-3-319-04936-6_5", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": true, 
        "isPartOf": {
          "isbn": [
            "978-3-319-04935-9", 
            "978-3-319-04936-6"
          ], 
          "name": "Performance Characterization and Benchmarking", 
          "type": "Book"
        }, 
        "name": "TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark", 
        "pagination": "61-76", 
        "productId": [
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/978-3-319-04936-6_5"
            ]
          }, 
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "9fc1322c77497f3d43b333be80ff6bb02e6c9d050d5bdd04318f8eacfe19818a"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1033535672"
            ]
          }
        ], 
        "publisher": {
          "location": "Cham", 
          "name": "Springer International Publishing", 
          "type": "Organisation"
        }, 
        "sameAs": [
          "https://doi.org/10.1007/978-3-319-04936-6_5", 
          "https://app.dimensions.ai/details/publication/pub.1033535672"
        ], 
        "sdDataset": "chapters", 
        "sdDatePublished": "2019-04-15T19:10", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8684_00000264.jsonl", 
        "type": "Chapter", 
        "url": "http://link.springer.com/10.1007/978-3-319-04936-6_5"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-04936-6_5'


     

    This table displays all metadata directly associated to this object as RDF triples.

    121 TRIPLES      23 PREDICATES      37 URIs      20 LITERALS      8 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/978-3-319-04936-6_5 schema:about anzsrc-for:08
    2 anzsrc-for:0806
    3 schema:author Na4986dbb745143248b7d09d41b3a9731
    4 schema:citation sg:pub.10.1007/978-3-642-10424-4_3
    5 https://doi.org/10.1109/icde.1996.492194
    6 https://doi.org/10.1109/icde.2001.914873
    7 https://doi.org/10.1109/icde.2007.367848
    8 https://doi.org/10.1109/icde.2013.6544837
    9 https://doi.org/10.1109/icde.2013.6544839
    10 https://doi.org/10.1145/1376616.1376672
    11 https://doi.org/10.1145/1457150.1457160
    12 https://doi.org/10.1145/152610.152611
    13 https://doi.org/10.1145/1559845.1559911
    14 schema:datePublished 2014
    15 schema:datePublishedReg 2014-01-01
    16 schema:description The TPC-D benchmark was developed almost 20 years ago, and even though its current existence as TPC-H could be considered superseded by TPC-DS, one can still learn from it. We focus on the technical level, summarizing the challenges posed by the TPC-H workload as we now understand them, which we call “choke points”. We identify 28 different such choke points, grouped into six categories: Aggregation Performance, Join Performance, Data Access Locality, Expression Calculation, Correlated Subqueries and Parallel Execution. On the meta-level, we make the point that the rich set of choke-points found in TPC-H sets an example on how to design future DBMS benchmarks.
    17 schema:editor N8fede758671e451bb5eb7a9d0c7a8c91
    18 schema:genre chapter
    19 schema:inLanguage en
    20 schema:isAccessibleForFree true
    21 schema:isPartOf Ne63b8a11fdae416e8d239928255df3e3
    22 schema:name TPC-H Analyzed: Hidden Messages and Lessons Learned from an Influential Benchmark
    23 schema:pagination 61-76
    24 schema:productId N6dac2961afc44620940a4e31224cd46a
    25 N85bcae8b7ed446a99cbf05cc96e567b6
    26 Na9d468e180ef4da39ba00c7f25a938e3
    27 schema:publisher Nf6866e5ac9404a5796e640e35df6c19b
    28 schema:sameAs https://app.dimensions.ai/details/publication/pub.1033535672
    29 https://doi.org/10.1007/978-3-319-04936-6_5
    30 schema:sdDatePublished 2019-04-15T19:10
    31 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    32 schema:sdPublisher Nd1bb6f5aad5a433b8141f31a680abc36
    33 schema:url http://link.springer.com/10.1007/978-3-319-04936-6_5
    34 sgo:license sg:explorer/license/
    35 sgo:sdDataset chapters
    36 rdf:type schema:Chapter
    37 N2c83f52ce1bc460c8cac6193ab32a4e2 rdf:first sg:person.015477771362.08
    38 rdf:rest N2fec357de1a740349046bcb9e0303ac3
    39 N2fec357de1a740349046bcb9e0303ac3 rdf:first sg:person.011715705740.17
    40 rdf:rest rdf:nil
    41 N6dac2961afc44620940a4e31224cd46a schema:name dimensions_id
    42 schema:value pub.1033535672
    43 rdf:type schema:PropertyValue
    44 N85bcae8b7ed446a99cbf05cc96e567b6 schema:name doi
    45 schema:value 10.1007/978-3-319-04936-6_5
    46 rdf:type schema:PropertyValue
    47 N8fede758671e451bb5eb7a9d0c7a8c91 rdf:first Nb6c7fe73d9ef4983a76dfe2c46269cdf
    48 rdf:rest Ne6523d6ffca94253955b06f167d82a0d
    49 N9ee0fb55224a472d8a0c9dd7965a0ca9 schema:familyName Poess
    50 schema:givenName Meikel
    51 rdf:type schema:Person
    52 Na4986dbb745143248b7d09d41b3a9731 rdf:first sg:person.015341641231.33
    53 rdf:rest N2c83f52ce1bc460c8cac6193ab32a4e2
    54 Na9d468e180ef4da39ba00c7f25a938e3 schema:name readcube_id
    55 schema:value 9fc1322c77497f3d43b333be80ff6bb02e6c9d050d5bdd04318f8eacfe19818a
    56 rdf:type schema:PropertyValue
    57 Nb6c7fe73d9ef4983a76dfe2c46269cdf schema:familyName Nambiar
    58 schema:givenName Raghunath
    59 rdf:type schema:Person
    60 Nd1bb6f5aad5a433b8141f31a680abc36 schema:name Springer Nature - SN SciGraph project
    61 rdf:type schema:Organization
    62 Ne63b8a11fdae416e8d239928255df3e3 schema:isbn 978-3-319-04935-9
    63 978-3-319-04936-6
    64 schema:name Performance Characterization and Benchmarking
    65 rdf:type schema:Book
    66 Ne6523d6ffca94253955b06f167d82a0d rdf:first N9ee0fb55224a472d8a0c9dd7965a0ca9
    67 rdf:rest rdf:nil
    68 Nf6866e5ac9404a5796e640e35df6c19b schema:location Cham
    69 schema:name Springer International Publishing
    70 rdf:type schema:Organisation
    71 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    72 schema:name Information and Computing Sciences
    73 rdf:type schema:DefinedTerm
    74 anzsrc-for:0806 schema:inDefinedTermSet anzsrc-for:
    75 schema:name Information Systems
    76 rdf:type schema:DefinedTerm
    77 sg:person.011715705740.17 schema:affiliation https://www.grid.ac/institutes/grid.426164.3
    78 schema:familyName Erling
    79 schema:givenName Orri
    80 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011715705740.17
    81 rdf:type schema:Person
    82 sg:person.015341641231.33 schema:affiliation https://www.grid.ac/institutes/grid.6054.7
    83 schema:familyName Boncz
    84 schema:givenName Peter
    85 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015341641231.33
    86 rdf:type schema:Person
    87 sg:person.015477771362.08 schema:affiliation https://www.grid.ac/institutes/grid.6936.a
    88 schema:familyName Neumann
    89 schema:givenName Thomas
    90 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015477771362.08
    91 rdf:type schema:Person
    92 sg:pub.10.1007/978-3-642-10424-4_3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1048186905
    93 https://doi.org/10.1007/978-3-642-10424-4_3
    94 rdf:type schema:CreativeWork
    95 https://doi.org/10.1109/icde.1996.492194 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095265855
    96 rdf:type schema:CreativeWork
    97 https://doi.org/10.1109/icde.2001.914873 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093803682
    98 rdf:type schema:CreativeWork
    99 https://doi.org/10.1109/icde.2007.367848 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094882172
    100 rdf:type schema:CreativeWork
    101 https://doi.org/10.1109/icde.2013.6544837 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093438827
    102 rdf:type schema:CreativeWork
    103 https://doi.org/10.1109/icde.2013.6544839 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093904188
    104 rdf:type schema:CreativeWork
    105 https://doi.org/10.1145/1376616.1376672 schema:sameAs https://app.dimensions.ai/details/publication/pub.1037499341
    106 rdf:type schema:CreativeWork
    107 https://doi.org/10.1145/1457150.1457160 schema:sameAs https://app.dimensions.ai/details/publication/pub.1032631476
    108 rdf:type schema:CreativeWork
    109 https://doi.org/10.1145/152610.152611 schema:sameAs https://app.dimensions.ai/details/publication/pub.1007812817
    110 rdf:type schema:CreativeWork
    111 https://doi.org/10.1145/1559845.1559911 schema:sameAs https://app.dimensions.ai/details/publication/pub.1040723285
    112 rdf:type schema:CreativeWork
    113 https://www.grid.ac/institutes/grid.426164.3 schema:alternateName OpenLink Software (United Kingdom)
    114 schema:name Openlink Software, United Kingdom
    115 rdf:type schema:Organization
    116 https://www.grid.ac/institutes/grid.6054.7 schema:alternateName Centrum Wiskunde and Informatica
    117 schema:name CWI, Amsterdam, The Netherlands
    118 rdf:type schema:Organization
    119 https://www.grid.ac/institutes/grid.6936.a schema:alternateName Technical University Munich
    120 schema:name Technical University Munich, Germany
    121 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...