The Implementation of a Hadoop Ecosystem Portal with Virtualization Deployment View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2018-10-17

AUTHORS

Chao-Tung Yang , Chien-Heng Wu , Wen-Yi Chang , Whey-Fone Tsai , Yu-Wei Chan , Endah Kristiani , Yuan-Ping Chiang

ABSTRACT

The requirements of research, analysis, processing and storing of big data are more and more important because big data is increasingly vital for development in the fields of information technology, finance, medicine, etc. Most of the big data environments are built on Hadoop or Spark. However, the constructions of these kinds of big data platform are not easy for ordinary users because of the lacks of professional knowledge and familiarity with the system. To make it easier to use the big data platform for data processing and analysis, we implemented the web user interface combining the big data platform including Hadoop and Spark. Then, we packaged the whole big data platform into the virtual machine image file along with the web user interface so that users can construct the environment and do the job more quickly and efficiently. We provide the convenient web user interface, not only reduce the difficulty of building a big data platform and save time but also provide an excellent performance of the system. And we also made the comparison of performance between the web user interface and the command line using the HiBench benchmark suit. More... »

PAGES

116-127

References to SciGraph publications

  • 2014-04. Big Data: A Survey in MOBILE NETWORKS AND APPLICATIONS
  • Book

    TITLE

    Advances on P2P, Parallel, Grid, Cloud and Internet Computing

    ISBN

    978-3-030-02606-6
    978-3-030-02607-3

    Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/978-3-030-02607-3_11

    DOI

    http://dx.doi.org/10.1007/978-3-030-02607-3_11

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1107669852


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0806", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information Systems", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "Tunghai University", 
              "id": "https://www.grid.ac/institutes/grid.265231.1", 
              "name": [
                "Department of Computer Science, Tunghai University, No.1727, Sec.4, 40704, Taiwan Boulevard, Xitun District, Taichung, Taiwan"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Yang", 
            "givenName": "Chao-Tung", 
            "id": "sg:person.015712700237.70", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015712700237.70"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "National Applied Research Laboratories", 
              "id": "https://www.grid.ac/institutes/grid.36020.37", 
              "name": [
                "High Performance Computing and Applications National Center, High-Performance Computing National Applied Research Laboratories, 30076, Hsinchu, Taiwan"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Wu", 
            "givenName": "Chien-Heng", 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "National Applied Research Laboratories", 
              "id": "https://www.grid.ac/institutes/grid.36020.37", 
              "name": [
                "High Performance Computing and Applications National Center, High-Performance Computing National Applied Research Laboratories, 30076, Hsinchu, Taiwan"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Chang", 
            "givenName": "Wen-Yi", 
            "id": "sg:person.014726265077.89", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014726265077.89"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "National Applied Research Laboratories", 
              "id": "https://www.grid.ac/institutes/grid.36020.37", 
              "name": [
                "High Performance Computing and Applications National Center, High-Performance Computing National Applied Research Laboratories, 30076, Hsinchu, Taiwan"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Tsai", 
            "givenName": "Whey-Fone", 
            "id": "sg:person.010203531135.31", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010203531135.31"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Providence University", 
              "id": "https://www.grid.ac/institutes/grid.412550.7", 
              "name": [
                "College of Computing and Informatics, Providence University, 200, Sec.7, 43301, Taiwan Boulevard, Shalu Dist, Taichung City, Taiwan"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Chan", 
            "givenName": "Yu-Wei", 
            "id": "sg:person.014401756371.51", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014401756371.51"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Tunghai University", 
              "id": "https://www.grid.ac/institutes/grid.265231.1", 
              "name": [
                "Department of Computer Science, Department of Industrial Engineering and Enterprise Information, Tunghai University, No.1727, Sec.4, 40704, Taiwan Boulevard, Xitun District, Taichung, Taiwan"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Kristiani", 
            "givenName": "Endah", 
            "id": "sg:person.010362120214.62", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010362120214.62"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Tunghai University", 
              "id": "https://www.grid.ac/institutes/grid.265231.1", 
              "name": [
                "Department of Computer Science, Tunghai University, No.1727, Sec.4, 40704, Taiwan Boulevard, Xitun District, Taichung, Taiwan"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Chiang", 
            "givenName": "Yuan-Ping", 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/s11036-013-0489-0", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1047473639", 
              "https://doi.org/10.1007/s11036-013-0489-0"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.14778/2367502.2367562", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1067368106"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cts.2013.6567222", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094897584"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2018-10-17", 
        "datePublishedReg": "2018-10-17", 
        "description": "The requirements of research, analysis, processing and storing of big data are more and more important because big data is increasingly vital for development in the fields of information technology, finance, medicine, etc. Most of the big data environments are built on Hadoop or Spark. However, the constructions of these kinds of big data platform are not easy for ordinary users because of the lacks of professional knowledge and familiarity with the system. To make it easier to use the big data platform for data processing and analysis, we implemented the web user interface combining the big data platform including Hadoop and Spark. Then, we packaged the whole big data platform into the virtual machine image file along with the web user interface so that users can construct the environment and do the job more quickly and efficiently. We provide the convenient web user interface, not only reduce the difficulty of building a big data platform and save time but also provide an excellent performance of the system. And we also made the comparison of performance between the web user interface and the command line using the HiBench benchmark suit.", 
        "editor": [
          {
            "familyName": "Xhafa", 
            "givenName": "Fatos", 
            "type": "Person"
          }, 
          {
            "familyName": "Leu", 
            "givenName": "Fang-Yie", 
            "type": "Person"
          }, 
          {
            "familyName": "Ficco", 
            "givenName": "Massimo", 
            "type": "Person"
          }, 
          {
            "familyName": "Yang", 
            "givenName": "Chao-Tung", 
            "type": "Person"
          }
        ], 
        "genre": "chapter", 
        "id": "sg:pub.10.1007/978-3-030-02607-3_11", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": false, 
        "isPartOf": {
          "isbn": [
            "978-3-030-02606-6", 
            "978-3-030-02607-3"
          ], 
          "name": "Advances on P2P, Parallel, Grid, Cloud and Internet Computing", 
          "type": "Book"
        }, 
        "name": "The Implementation of a Hadoop Ecosystem Portal with Virtualization Deployment", 
        "pagination": "116-127", 
        "productId": [
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/978-3-030-02607-3_11"
            ]
          }, 
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "0e5c2acbb3323163917774b6470c56d5de6e52af6f763f05e754f74a639259e5"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1107669852"
            ]
          }
        ], 
        "publisher": {
          "location": "Cham", 
          "name": "Springer International Publishing", 
          "type": "Organisation"
        }, 
        "sameAs": [
          "https://doi.org/10.1007/978-3-030-02607-3_11", 
          "https://app.dimensions.ai/details/publication/pub.1107669852"
        ], 
        "sdDataset": "chapters", 
        "sdDatePublished": "2019-04-16T04:39", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000321_0000000321/records_74935_00000000.jsonl", 
        "type": "Chapter", 
        "url": "https://link.springer.com/10.1007%2F978-3-030-02607-3_11"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-02607-3_11'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-02607-3_11'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-02607-3_11'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-030-02607-3_11'


     

    This table displays all metadata directly associated to this object as RDF triples.

    137 TRIPLES      23 PREDICATES      29 URIs      19 LITERALS      8 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/978-3-030-02607-3_11 schema:about anzsrc-for:08
    2 anzsrc-for:0806
    3 schema:author N6711cf21355e48d0a54f0ec5859a7cab
    4 schema:citation sg:pub.10.1007/s11036-013-0489-0
    5 https://doi.org/10.1109/cts.2013.6567222
    6 https://doi.org/10.14778/2367502.2367562
    7 schema:datePublished 2018-10-17
    8 schema:datePublishedReg 2018-10-17
    9 schema:description The requirements of research, analysis, processing and storing of big data are more and more important because big data is increasingly vital for development in the fields of information technology, finance, medicine, etc. Most of the big data environments are built on Hadoop or Spark. However, the constructions of these kinds of big data platform are not easy for ordinary users because of the lacks of professional knowledge and familiarity with the system. To make it easier to use the big data platform for data processing and analysis, we implemented the web user interface combining the big data platform including Hadoop and Spark. Then, we packaged the whole big data platform into the virtual machine image file along with the web user interface so that users can construct the environment and do the job more quickly and efficiently. We provide the convenient web user interface, not only reduce the difficulty of building a big data platform and save time but also provide an excellent performance of the system. And we also made the comparison of performance between the web user interface and the command line using the HiBench benchmark suit.
    10 schema:editor N42e3b369134b43f5a6d407cbb0c94915
    11 schema:genre chapter
    12 schema:inLanguage en
    13 schema:isAccessibleForFree false
    14 schema:isPartOf N15cb083e2db44fa48fe677c75bf65a5b
    15 schema:name The Implementation of a Hadoop Ecosystem Portal with Virtualization Deployment
    16 schema:pagination 116-127
    17 schema:productId N5c1482acf0f94dea96156bf4935f5e45
    18 N87c59b4a034444efa468cdd1debd0e66
    19 Nfc6aa44311264209964ab39b220d5a31
    20 schema:publisher Nd1a67fddc75c4f5ba7e1cd1d59271776
    21 schema:sameAs https://app.dimensions.ai/details/publication/pub.1107669852
    22 https://doi.org/10.1007/978-3-030-02607-3_11
    23 schema:sdDatePublished 2019-04-16T04:39
    24 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    25 schema:sdPublisher N8181defd24804a47a8bca793969d04b1
    26 schema:url https://link.springer.com/10.1007%2F978-3-030-02607-3_11
    27 sgo:license sg:explorer/license/
    28 sgo:sdDataset chapters
    29 rdf:type schema:Chapter
    30 N028f769508c0481c96d13547f32088d4 schema:affiliation https://www.grid.ac/institutes/grid.36020.37
    31 schema:familyName Wu
    32 schema:givenName Chien-Heng
    33 rdf:type schema:Person
    34 N09c69748a96f4ae2a0315b447113ad9a rdf:first N028f769508c0481c96d13547f32088d4
    35 rdf:rest Nc2d5cb5ce45d47bfb12ce9e3b077dbdd
    36 N132ba2d9133c4eb7ba1a1ac497e1a85e schema:familyName Xhafa
    37 schema:givenName Fatos
    38 rdf:type schema:Person
    39 N1578738f954847d9b6f6a689481877a3 rdf:first Nfd9692bde36c4487ab46ca8c15550a43
    40 rdf:rest rdf:nil
    41 N15cb083e2db44fa48fe677c75bf65a5b schema:isbn 978-3-030-02606-6
    42 978-3-030-02607-3
    43 schema:name Advances on P2P, Parallel, Grid, Cloud and Internet Computing
    44 rdf:type schema:Book
    45 N24002781d7a34b7abb9bb1a1cf9aae7f schema:affiliation https://www.grid.ac/institutes/grid.265231.1
    46 schema:familyName Chiang
    47 schema:givenName Yuan-Ping
    48 rdf:type schema:Person
    49 N2f737846a799469498c4d6a35e3d6ea6 schema:familyName Leu
    50 schema:givenName Fang-Yie
    51 rdf:type schema:Person
    52 N42e3b369134b43f5a6d407cbb0c94915 rdf:first N132ba2d9133c4eb7ba1a1ac497e1a85e
    53 rdf:rest Naf0e0f5dd888442e8e20c800aa70a610
    54 N49b329cc181e4da7b83ff6cf4457ccc3 rdf:first sg:person.010203531135.31
    55 rdf:rest N8f453ca304ea41f79d6b87ba4674a3c5
    56 N5c1482acf0f94dea96156bf4935f5e45 schema:name doi
    57 schema:value 10.1007/978-3-030-02607-3_11
    58 rdf:type schema:PropertyValue
    59 N6711cf21355e48d0a54f0ec5859a7cab rdf:first sg:person.015712700237.70
    60 rdf:rest N09c69748a96f4ae2a0315b447113ad9a
    61 N6c0bc9d2f4b54b8ea116448c06b6294f rdf:first N6db6928b39b14136a1e1d3a45adc110d
    62 rdf:rest N1578738f954847d9b6f6a689481877a3
    63 N6db6928b39b14136a1e1d3a45adc110d schema:familyName Ficco
    64 schema:givenName Massimo
    65 rdf:type schema:Person
    66 N8181defd24804a47a8bca793969d04b1 schema:name Springer Nature - SN SciGraph project
    67 rdf:type schema:Organization
    68 N87c59b4a034444efa468cdd1debd0e66 schema:name readcube_id
    69 schema:value 0e5c2acbb3323163917774b6470c56d5de6e52af6f763f05e754f74a639259e5
    70 rdf:type schema:PropertyValue
    71 N8dd1d86b26c7499bb68372da691a6cb6 rdf:first N24002781d7a34b7abb9bb1a1cf9aae7f
    72 rdf:rest rdf:nil
    73 N8f453ca304ea41f79d6b87ba4674a3c5 rdf:first sg:person.014401756371.51
    74 rdf:rest Nf18bfb4289984922b5152ef7760a942c
    75 Naf0e0f5dd888442e8e20c800aa70a610 rdf:first N2f737846a799469498c4d6a35e3d6ea6
    76 rdf:rest N6c0bc9d2f4b54b8ea116448c06b6294f
    77 Nc2d5cb5ce45d47bfb12ce9e3b077dbdd rdf:first sg:person.014726265077.89
    78 rdf:rest N49b329cc181e4da7b83ff6cf4457ccc3
    79 Nd1a67fddc75c4f5ba7e1cd1d59271776 schema:location Cham
    80 schema:name Springer International Publishing
    81 rdf:type schema:Organisation
    82 Nf18bfb4289984922b5152ef7760a942c rdf:first sg:person.010362120214.62
    83 rdf:rest N8dd1d86b26c7499bb68372da691a6cb6
    84 Nfc6aa44311264209964ab39b220d5a31 schema:name dimensions_id
    85 schema:value pub.1107669852
    86 rdf:type schema:PropertyValue
    87 Nfd9692bde36c4487ab46ca8c15550a43 schema:familyName Yang
    88 schema:givenName Chao-Tung
    89 rdf:type schema:Person
    90 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    91 schema:name Information and Computing Sciences
    92 rdf:type schema:DefinedTerm
    93 anzsrc-for:0806 schema:inDefinedTermSet anzsrc-for:
    94 schema:name Information Systems
    95 rdf:type schema:DefinedTerm
    96 sg:person.010203531135.31 schema:affiliation https://www.grid.ac/institutes/grid.36020.37
    97 schema:familyName Tsai
    98 schema:givenName Whey-Fone
    99 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010203531135.31
    100 rdf:type schema:Person
    101 sg:person.010362120214.62 schema:affiliation https://www.grid.ac/institutes/grid.265231.1
    102 schema:familyName Kristiani
    103 schema:givenName Endah
    104 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010362120214.62
    105 rdf:type schema:Person
    106 sg:person.014401756371.51 schema:affiliation https://www.grid.ac/institutes/grid.412550.7
    107 schema:familyName Chan
    108 schema:givenName Yu-Wei
    109 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014401756371.51
    110 rdf:type schema:Person
    111 sg:person.014726265077.89 schema:affiliation https://www.grid.ac/institutes/grid.36020.37
    112 schema:familyName Chang
    113 schema:givenName Wen-Yi
    114 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014726265077.89
    115 rdf:type schema:Person
    116 sg:person.015712700237.70 schema:affiliation https://www.grid.ac/institutes/grid.265231.1
    117 schema:familyName Yang
    118 schema:givenName Chao-Tung
    119 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015712700237.70
    120 rdf:type schema:Person
    121 sg:pub.10.1007/s11036-013-0489-0 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047473639
    122 https://doi.org/10.1007/s11036-013-0489-0
    123 rdf:type schema:CreativeWork
    124 https://doi.org/10.1109/cts.2013.6567222 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094897584
    125 rdf:type schema:CreativeWork
    126 https://doi.org/10.14778/2367502.2367562 schema:sameAs https://app.dimensions.ai/details/publication/pub.1067368106
    127 rdf:type schema:CreativeWork
    128 https://www.grid.ac/institutes/grid.265231.1 schema:alternateName Tunghai University
    129 schema:name Department of Computer Science, Department of Industrial Engineering and Enterprise Information, Tunghai University, No.1727, Sec.4, 40704, Taiwan Boulevard, Xitun District, Taichung, Taiwan
    130 Department of Computer Science, Tunghai University, No.1727, Sec.4, 40704, Taiwan Boulevard, Xitun District, Taichung, Taiwan
    131 rdf:type schema:Organization
    132 https://www.grid.ac/institutes/grid.36020.37 schema:alternateName National Applied Research Laboratories
    133 schema:name High Performance Computing and Applications National Center, High-Performance Computing National Applied Research Laboratories, 30076, Hsinchu, Taiwan
    134 rdf:type schema:Organization
    135 https://www.grid.ac/institutes/grid.412550.7 schema:alternateName Providence University
    136 schema:name College of Computing and Informatics, Providence University, 200, Sec.7, 43301, Taiwan Boulevard, Shalu Dist, Taichung City, Taiwan
    137 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...