Machine Learning Using Virtualized GPUs in Cloud Environments View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2017-10-20

AUTHORS

Uday Kurkure , Hari Sivaraman , Lan Vu

ABSTRACT

Using graphic processing units (GPU) to accelerate machine learning applications has become a focus of high performance computing (HPC) in recent years. In cloud environments, many different cloud-based GPU solutions have been introduced to seamlessly and securely use GPU resources without sacrificing their performance benefits. Among them are two main approaches: using direct pass-through technologies available on hypervisors and using virtual GPU technologies introduced by GPU vendors. In this paper, we present a performance study of these two GPU virtualization solutions for machine learning in the cloud. We evaluate the advantages and disadvantages of each solution and introduce new findings of their performance impact on machine learning applications in different real-world use-case scenarios. We also examine the benefits of virtual GPUs for machine learning alone and for machine learning applications running together with other GPU-based applications like 3D-graphics on the same server with multiple GPUs to better leverage computing resources. Based on our experimental results benchmarking machine learning applications developed with TensorFlow, we discuss the scaling from one to multiple GPUs and compare the performance between two virtual GPU solutions. Finally, we show that mixing machine learning and other GPU-based workloads can help to reduce combined execution time as compared to running these workloads sequentially. More... »

PAGES

591-604

References to SciGraph publications

  • 2016-12. A survey of machine learning for big data processing in APPLIED SIGNAL PROCESSING
  • 2003. The Penn Treebank: An Overview in TREEBANKS
  • Book

    TITLE

    High Performance Computing

    ISBN

    978-3-319-67629-6
    978-3-319-67630-2

    Author Affiliations

    Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/978-3-319-67630-2_41

    DOI

    http://dx.doi.org/10.1007/978-3-319-67630-2_41

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1092291356


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Artificial Intelligence and Image Processing", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "DELL (United States)", 
              "id": "https://www.grid.ac/institutes/grid.410541.5", 
              "name": [
                "VMware, 94304, Palo Alto, CA, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Kurkure", 
            "givenName": "Uday", 
            "id": "sg:person.0677441142.46", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0677441142.46"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "DELL (United States)", 
              "id": "https://www.grid.ac/institutes/grid.410541.5", 
              "name": [
                "VMware, 94304, Palo Alto, CA, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Sivaraman", 
            "givenName": "Hari", 
            "id": "sg:person.015657455777.24", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015657455777.24"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "DELL (United States)", 
              "id": "https://www.grid.ac/institutes/grid.410541.5", 
              "name": [
                "VMware, 94304, Palo Alto, CA, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Vu", 
            "givenName": "Lan", 
            "id": "sg:person.07561551534.89", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07561551534.89"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/978-94-010-0201-1_1", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1004962278", 
              "https://doi.org/10.1007/978-94-010-0201-1_1"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-94-010-0201-1_1", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1004962278", 
              "https://doi.org/10.1007/978-94-010-0201-1_1"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/2487575.2487677", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1020845198"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1186/s13634-016-0355-x", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1040032778", 
              "https://doi.org/10.1186/s13634-016-0355-x"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1186/s13634-016-0355-x", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1040032778", 
              "https://doi.org/10.1186/s13634-016-0355-x"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/j.jnca.2016.01.010", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1047430007"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/5.726791", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061179979"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/3079856.3080246", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1090373941"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cloud.2014.90", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094931856"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/hpcs.2017.113", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095021957"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2017-10-20", 
        "datePublishedReg": "2017-10-20", 
        "description": "Using graphic processing units (GPU) to accelerate machine learning applications has become a focus of high performance computing (HPC) in recent years. In cloud environments, many different cloud-based GPU solutions have been introduced to seamlessly and securely use GPU resources without sacrificing their performance benefits. Among them are two main approaches: using direct pass-through technologies available on hypervisors and using virtual GPU technologies introduced by GPU vendors. In this paper, we present a performance study of these two GPU virtualization solutions for machine learning in the cloud. We evaluate the advantages and disadvantages of each solution and introduce new findings of their performance impact on machine learning applications in different real-world use-case scenarios. We also examine the benefits of virtual GPUs for machine learning alone and for machine learning applications running together with other GPU-based applications like 3D-graphics on the same server with multiple GPUs to better leverage computing resources. Based on our experimental results benchmarking machine learning applications developed with TensorFlow, we discuss the scaling from one to multiple GPUs and compare the performance between two virtual GPU solutions. Finally, we show that mixing machine learning and other GPU-based workloads can help to reduce combined execution time as compared to running these workloads sequentially.", 
        "editor": [
          {
            "familyName": "Kunkel", 
            "givenName": "Julian M.", 
            "type": "Person"
          }, 
          {
            "familyName": "Yokota", 
            "givenName": "Rio", 
            "type": "Person"
          }, 
          {
            "familyName": "Taufer", 
            "givenName": "Michela", 
            "type": "Person"
          }, 
          {
            "familyName": "Shalf", 
            "givenName": "John", 
            "type": "Person"
          }
        ], 
        "genre": "chapter", 
        "id": "sg:pub.10.1007/978-3-319-67630-2_41", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": false, 
        "isPartOf": {
          "isbn": [
            "978-3-319-67629-6", 
            "978-3-319-67630-2"
          ], 
          "name": "High Performance Computing", 
          "type": "Book"
        }, 
        "name": "Machine Learning Using Virtualized GPUs in Cloud Environments", 
        "pagination": "591-604", 
        "productId": [
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/978-3-319-67630-2_41"
            ]
          }, 
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "dcc40848d9ae2537ee21aed130dc2efcbce10fe24e4ce0c4a648801536e44453"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1092291356"
            ]
          }
        ], 
        "publisher": {
          "location": "Cham", 
          "name": "Springer International Publishing", 
          "type": "Organisation"
        }, 
        "sameAs": [
          "https://doi.org/10.1007/978-3-319-67630-2_41", 
          "https://app.dimensions.ai/details/publication/pub.1092291356"
        ], 
        "sdDataset": "chapters", 
        "sdDatePublished": "2019-04-16T05:00", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000325_0000000325/records_100788_00000000.jsonl", 
        "type": "Chapter", 
        "url": "https://link.springer.com/10.1007%2F978-3-319-67630-2_41"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-67630-2_41'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-67630-2_41'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-67630-2_41'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-67630-2_41'


     

    This table displays all metadata directly associated to this object as RDF triples.

    120 TRIPLES      23 PREDICATES      34 URIs      19 LITERALS      8 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/978-3-319-67630-2_41 schema:about anzsrc-for:08
    2 anzsrc-for:0801
    3 schema:author N465a1e52efd7455b82e4c144ce73e2a0
    4 schema:citation sg:pub.10.1007/978-94-010-0201-1_1
    5 sg:pub.10.1186/s13634-016-0355-x
    6 https://doi.org/10.1016/j.jnca.2016.01.010
    7 https://doi.org/10.1109/5.726791
    8 https://doi.org/10.1109/cloud.2014.90
    9 https://doi.org/10.1109/hpcs.2017.113
    10 https://doi.org/10.1145/2487575.2487677
    11 https://doi.org/10.1145/3079856.3080246
    12 schema:datePublished 2017-10-20
    13 schema:datePublishedReg 2017-10-20
    14 schema:description Using graphic processing units (GPU) to accelerate machine learning applications has become a focus of high performance computing (HPC) in recent years. In cloud environments, many different cloud-based GPU solutions have been introduced to seamlessly and securely use GPU resources without sacrificing their performance benefits. Among them are two main approaches: using direct pass-through technologies available on hypervisors and using virtual GPU technologies introduced by GPU vendors. In this paper, we present a performance study of these two GPU virtualization solutions for machine learning in the cloud. We evaluate the advantages and disadvantages of each solution and introduce new findings of their performance impact on machine learning applications in different real-world use-case scenarios. We also examine the benefits of virtual GPUs for machine learning alone and for machine learning applications running together with other GPU-based applications like 3D-graphics on the same server with multiple GPUs to better leverage computing resources. Based on our experimental results benchmarking machine learning applications developed with TensorFlow, we discuss the scaling from one to multiple GPUs and compare the performance between two virtual GPU solutions. Finally, we show that mixing machine learning and other GPU-based workloads can help to reduce combined execution time as compared to running these workloads sequentially.
    15 schema:editor N1feeb2f7b8ed41e093837ffeb0da6cd4
    16 schema:genre chapter
    17 schema:inLanguage en
    18 schema:isAccessibleForFree false
    19 schema:isPartOf N0e1aeaaf961b43e7852c1b558a71990d
    20 schema:name Machine Learning Using Virtualized GPUs in Cloud Environments
    21 schema:pagination 591-604
    22 schema:productId Na6ced7b9b4b14780b15dff1191991b7e
    23 Ndde9c8db95f4440c852ad4f2966789cf
    24 Ndf64d39774d949f2bab84723a1ff2d4a
    25 schema:publisher N5815190c332641afa739e515ae636994
    26 schema:sameAs https://app.dimensions.ai/details/publication/pub.1092291356
    27 https://doi.org/10.1007/978-3-319-67630-2_41
    28 schema:sdDatePublished 2019-04-16T05:00
    29 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    30 schema:sdPublisher Nfe34a23c6dc243708cc3058550a9b700
    31 schema:url https://link.springer.com/10.1007%2F978-3-319-67630-2_41
    32 sgo:license sg:explorer/license/
    33 sgo:sdDataset chapters
    34 rdf:type schema:Chapter
    35 N0e1aeaaf961b43e7852c1b558a71990d schema:isbn 978-3-319-67629-6
    36 978-3-319-67630-2
    37 schema:name High Performance Computing
    38 rdf:type schema:Book
    39 N1feeb2f7b8ed41e093837ffeb0da6cd4 rdf:first N84129e785d95459b9ce74b6b3833b3fb
    40 rdf:rest Na17cb12b52664e43b8ffb7737bccf939
    41 N465a1e52efd7455b82e4c144ce73e2a0 rdf:first sg:person.0677441142.46
    42 rdf:rest Nf63017cc341e487f870e18744e375379
    43 N5030ecc78b604216ac47c3e10daae772 rdf:first N54cc4fbfe1ac4973b9d1a907e95de82f
    44 rdf:rest rdf:nil
    45 N54cc4fbfe1ac4973b9d1a907e95de82f schema:familyName Shalf
    46 schema:givenName John
    47 rdf:type schema:Person
    48 N5815190c332641afa739e515ae636994 schema:location Cham
    49 schema:name Springer International Publishing
    50 rdf:type schema:Organisation
    51 N6069081143c1474d9d75a53d4107e812 schema:familyName Taufer
    52 schema:givenName Michela
    53 rdf:type schema:Person
    54 N673a35c8685e406ca5d7b2c6eb07048e rdf:first sg:person.07561551534.89
    55 rdf:rest rdf:nil
    56 N8212df8c542d42189565b31f462a245c schema:familyName Yokota
    57 schema:givenName Rio
    58 rdf:type schema:Person
    59 N84129e785d95459b9ce74b6b3833b3fb schema:familyName Kunkel
    60 schema:givenName Julian M.
    61 rdf:type schema:Person
    62 Na17cb12b52664e43b8ffb7737bccf939 rdf:first N8212df8c542d42189565b31f462a245c
    63 rdf:rest Ndf6420fe115e4fd6a108e96e4df3d935
    64 Na6ced7b9b4b14780b15dff1191991b7e schema:name readcube_id
    65 schema:value dcc40848d9ae2537ee21aed130dc2efcbce10fe24e4ce0c4a648801536e44453
    66 rdf:type schema:PropertyValue
    67 Ndde9c8db95f4440c852ad4f2966789cf schema:name doi
    68 schema:value 10.1007/978-3-319-67630-2_41
    69 rdf:type schema:PropertyValue
    70 Ndf6420fe115e4fd6a108e96e4df3d935 rdf:first N6069081143c1474d9d75a53d4107e812
    71 rdf:rest N5030ecc78b604216ac47c3e10daae772
    72 Ndf64d39774d949f2bab84723a1ff2d4a schema:name dimensions_id
    73 schema:value pub.1092291356
    74 rdf:type schema:PropertyValue
    75 Nf63017cc341e487f870e18744e375379 rdf:first sg:person.015657455777.24
    76 rdf:rest N673a35c8685e406ca5d7b2c6eb07048e
    77 Nfe34a23c6dc243708cc3058550a9b700 schema:name Springer Nature - SN SciGraph project
    78 rdf:type schema:Organization
    79 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    80 schema:name Information and Computing Sciences
    81 rdf:type schema:DefinedTerm
    82 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
    83 schema:name Artificial Intelligence and Image Processing
    84 rdf:type schema:DefinedTerm
    85 sg:person.015657455777.24 schema:affiliation https://www.grid.ac/institutes/grid.410541.5
    86 schema:familyName Sivaraman
    87 schema:givenName Hari
    88 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015657455777.24
    89 rdf:type schema:Person
    90 sg:person.0677441142.46 schema:affiliation https://www.grid.ac/institutes/grid.410541.5
    91 schema:familyName Kurkure
    92 schema:givenName Uday
    93 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0677441142.46
    94 rdf:type schema:Person
    95 sg:person.07561551534.89 schema:affiliation https://www.grid.ac/institutes/grid.410541.5
    96 schema:familyName Vu
    97 schema:givenName Lan
    98 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07561551534.89
    99 rdf:type schema:Person
    100 sg:pub.10.1007/978-94-010-0201-1_1 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004962278
    101 https://doi.org/10.1007/978-94-010-0201-1_1
    102 rdf:type schema:CreativeWork
    103 sg:pub.10.1186/s13634-016-0355-x schema:sameAs https://app.dimensions.ai/details/publication/pub.1040032778
    104 https://doi.org/10.1186/s13634-016-0355-x
    105 rdf:type schema:CreativeWork
    106 https://doi.org/10.1016/j.jnca.2016.01.010 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047430007
    107 rdf:type schema:CreativeWork
    108 https://doi.org/10.1109/5.726791 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061179979
    109 rdf:type schema:CreativeWork
    110 https://doi.org/10.1109/cloud.2014.90 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094931856
    111 rdf:type schema:CreativeWork
    112 https://doi.org/10.1109/hpcs.2017.113 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095021957
    113 rdf:type schema:CreativeWork
    114 https://doi.org/10.1145/2487575.2487677 schema:sameAs https://app.dimensions.ai/details/publication/pub.1020845198
    115 rdf:type schema:CreativeWork
    116 https://doi.org/10.1145/3079856.3080246 schema:sameAs https://app.dimensions.ai/details/publication/pub.1090373941
    117 rdf:type schema:CreativeWork
    118 https://www.grid.ac/institutes/grid.410541.5 schema:alternateName DELL (United States)
    119 schema:name VMware, 94304, Palo Alto, CA, USA
    120 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...