A new data science research program: evaluation, metrology, standards, and community outreach View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

2016-11

AUTHORS

Bonnie J. Dorr, Craig S. Greenberg, Peter Fontana, Mark Przybocki, Marion Le Bras, Cathryn Ploehn, Oleg Aulov, Martial Michel, E. Jim Golden, Wo Chang

ABSTRACT

This article examines foundational issues in data science including current challenges, basic research questions, and expected advances, as the basis for a new data science research program (DSRP) and associated data science evaluation (DSE) series, introduced by the National Institute of Standards and Technology (NIST) in the fall of 2015. The DSRP is designed to facilitate and accelerate research progress in the field of data science and consists of four components: evaluation and metrology, standards, compute infrastructure, and community outreach. A key part of the evaluation and measurement component is the DSE. The DSE series aims to address logistical and evaluation design challenges while providing rigorous measurement methods and an emphasis on generalizability rather than domain- and application-specific approaches. Toward that end, each year the DSE will consist of multiple research tracks and will encourage the application of tasks that span these tracks. The evaluations are intended to facilitate research efforts and collaboration, leverage shared infrastructure, and effectively address crosscutting challenges faced by diverse data science communities. Multiple research tracks will be championed by members of the data science community with the goal of enabling rigorous comparison of approaches through common tasks, datasets, metrics, and shared research challenges. The tracks will permit us to measure several different data science technologies in a wide range of fields and will address computing infrastructure, standards for an interoperability framework, and domain-specific examples. This article also summarizes lessons learned from the data science evaluation series pre-pilot that was held in fall of 2015. More... »

PAGES

177-197

References to SciGraph publications

  • 2012. Data Matching, Concepts and Techniques for Record Linkage, Entity Resolution, and Duplicate Detection in NONE
  • 2014-11. Impact of analytic provenance in genome analysis in BMC GENOMICS
  • 2014. What Is Visualization Really For? in THE PHILOSOPHY OF INFORMATION QUALITY
  • 2000-11-24. Data Provenance: Some Basic Issues in FST TCS 2000: FOUNDATIONS OF SOFTWARE TECHNOLOGY AND THEORETICAL COMPUTER SCIENCE
  • 2014. Privacy through Accountability: A Computer Science Perspective in DISTRIBUTED COMPUTING AND INTERNET TECHNOLOGY
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/s41060-016-0016-z

    DOI

    http://dx.doi.org/10.1007/s41060-016-0016-z

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1010806434


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0806", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information Systems", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Dorr", 
            "givenName": "Bonnie J.", 
            "id": "sg:person.012235671121.11", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012235671121.11"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Greenberg", 
            "givenName": "Craig S.", 
            "id": "sg:person.07541377717.06", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07541377717.06"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Fontana", 
            "givenName": "Peter", 
            "id": "sg:person.07612211461.24", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07612211461.24"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Przybocki", 
            "givenName": "Mark", 
            "id": "sg:person.011506423721.69", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011506423721.69"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Le Bras", 
            "givenName": "Marion", 
            "id": "sg:person.013676745321.84", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013676745321.84"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Ploehn", 
            "givenName": "Cathryn", 
            "id": "sg:person.013233274711.54", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013233274711.54"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Aulov", 
            "givenName": "Oleg", 
            "id": "sg:person.010165437221.98", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010165437221.98"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Michel", 
            "givenName": "Martial", 
            "id": "sg:person.012723713447.59", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012723713447.59"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Golden", 
            "givenName": "E. Jim", 
            "id": "sg:person.014546302221.11", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014546302221.11"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "name": [
                "100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Chang", 
            "givenName": "Wo", 
            "id": "sg:person.016141243221.07", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016141243221.07"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "https://doi.org/10.1016/j.jadohealth.2005.05.008", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1004965366"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-07121-3_5", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1005856317", 
              "https://doi.org/10.1007/978-3-319-07121-3_5"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/3-540-44450-5_6", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1006354016", 
              "https://doi.org/10.1007/3-540-44450-5_6"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/3-540-44450-5_6", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1006354016", 
              "https://doi.org/10.1007/3-540-44450-5_6"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1214/09-ss057", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1008096476"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-04483-5_5", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1009252916", 
              "https://doi.org/10.1007/978-3-319-04483-5_5"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/146565.146567", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1011005778"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://app.dimensions.ai/details/publication/pub.1016571298", 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://app.dimensions.ai/details/publication/pub.1016571298", 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1002/scin.2015.187003022", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1019380626"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1084805.1084812", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1019590669"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/2666680", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1019964287"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-31164-2", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1020812050", 
              "https://doi.org/10.1007/978-3-642-31164-2"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-642-31164-2", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1020812050", 
              "https://doi.org/10.1007/978-3-642-31164-2"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1186/1471-2164-15-s8-s1", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1026431064", 
              "https://doi.org/10.1186/1471-2164-15-s8-s1"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/1541880.1541882", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1030762489"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/2609876.2609888", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1043453265"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/2945.981847", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061146380"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/mcg.2003.1210860", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061391259"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tits.2014.2305334", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061658343"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tvcg.2011.279", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061813667"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/tvcg.2013.126", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1061813994"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.14778/2367502.2367564", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1067368108"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1561/1900000006", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1068001356"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.2200/s00362ed1v01y201105dtm016", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1069288243"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.2307/1403796", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1069474051"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.2307/1403797", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1069474052"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.4018/ijagr.2015040105", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1071883212"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1137/1.9781611972818.4", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1088800536"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1137/1.9781611972818.61", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1088800562"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icdm.2012.161", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094208037"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/dsaa.2015.7344805", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094544877"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/cyber.2012.6392557", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1094895622"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/itsc.2013.6728263", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095410976"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/ipdps.2007.370216", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095542773"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/odyssey.2006.248120", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095801126"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/odyssey.2006.248120", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095801126"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1017/cbo9780511802270", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1098730668"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.3115/1289189.1289273", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1099184024"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1609/aimag.v36i1.2568", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1103067306"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1609/aimag.v36i1.2565", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1103067312"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2016-11", 
        "datePublishedReg": "2016-11-01", 
        "description": "This article examines foundational issues in data science including current challenges, basic research questions, and expected advances, as the basis for a new data science research program (DSRP) and associated data science evaluation (DSE) series, introduced by the National Institute of Standards and Technology (NIST) in the fall of 2015. The DSRP is designed to facilitate and accelerate research progress in the field of data science and consists of four components: evaluation and metrology, standards, compute infrastructure, and community outreach. A key part of the evaluation and measurement component is the DSE. The DSE series aims to address logistical and evaluation design challenges while providing rigorous measurement methods and an emphasis on generalizability rather than domain- and application-specific approaches. Toward that end, each year the DSE will consist of multiple research tracks and will encourage the application of tasks that span these tracks. The evaluations are intended to facilitate research efforts and collaboration, leverage shared infrastructure, and effectively address crosscutting challenges faced by diverse data science communities. Multiple research tracks will be championed by members of the data science community with the goal of enabling rigorous comparison of approaches through common tasks, datasets, metrics, and shared research challenges. The tracks will permit us to measure several different data science technologies in a wide range of fields and will address computing infrastructure, standards for an interoperability framework, and domain-specific examples. This article also summarizes lessons learned from the data science evaluation series pre-pilot that was held in fall of 2015.", 
        "genre": "research_article", 
        "id": "sg:pub.10.1007/s41060-016-0016-z", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": true, 
        "isPartOf": [
          {
            "id": "sg:journal.1156617", 
            "issn": [
              "2364-415X", 
              "2364-4168"
            ], 
            "name": "International Journal of Data Science and Analytics", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "3-4", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "1"
          }
        ], 
        "name": "A new data science research program: evaluation, metrology, standards, and community outreach", 
        "pagination": "177-197", 
        "productId": [
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "337ba77f51e658650403d1ebc16e817e8e180a43ceeef5f4e9fc6b2caace3e5b"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/s41060-016-0016-z"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1010806434"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1007/s41060-016-0016-z", 
          "https://app.dimensions.ai/details/publication/pub.1010806434"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2019-04-11T09:31", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000346_0000000346/records_99806_00000000.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://link.springer.com/10.1007%2Fs41060-016-0016-z"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s41060-016-0016-z'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s41060-016-0016-z'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s41060-016-0016-z'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s41060-016-0016-z'


     

    This table displays all metadata directly associated to this object as RDF triples.

    256 TRIPLES      21 PREDICATES      64 URIs      19 LITERALS      7 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/s41060-016-0016-z schema:about anzsrc-for:08
    2 anzsrc-for:0806
    3 schema:author N0ba2c78e66c0498faf5c61fc3e84ad23
    4 schema:citation sg:pub.10.1007/3-540-44450-5_6
    5 sg:pub.10.1007/978-3-319-04483-5_5
    6 sg:pub.10.1007/978-3-319-07121-3_5
    7 sg:pub.10.1007/978-3-642-31164-2
    8 sg:pub.10.1186/1471-2164-15-s8-s1
    9 https://app.dimensions.ai/details/publication/pub.1016571298
    10 https://doi.org/10.1002/scin.2015.187003022
    11 https://doi.org/10.1016/j.jadohealth.2005.05.008
    12 https://doi.org/10.1017/cbo9780511802270
    13 https://doi.org/10.1109/2945.981847
    14 https://doi.org/10.1109/cyber.2012.6392557
    15 https://doi.org/10.1109/dsaa.2015.7344805
    16 https://doi.org/10.1109/icdm.2012.161
    17 https://doi.org/10.1109/ipdps.2007.370216
    18 https://doi.org/10.1109/itsc.2013.6728263
    19 https://doi.org/10.1109/mcg.2003.1210860
    20 https://doi.org/10.1109/odyssey.2006.248120
    21 https://doi.org/10.1109/tits.2014.2305334
    22 https://doi.org/10.1109/tvcg.2011.279
    23 https://doi.org/10.1109/tvcg.2013.126
    24 https://doi.org/10.1137/1.9781611972818.4
    25 https://doi.org/10.1137/1.9781611972818.61
    26 https://doi.org/10.1145/1084805.1084812
    27 https://doi.org/10.1145/146565.146567
    28 https://doi.org/10.1145/1541880.1541882
    29 https://doi.org/10.1145/2609876.2609888
    30 https://doi.org/10.1145/2666680
    31 https://doi.org/10.1214/09-ss057
    32 https://doi.org/10.14778/2367502.2367564
    33 https://doi.org/10.1561/1900000006
    34 https://doi.org/10.1609/aimag.v36i1.2565
    35 https://doi.org/10.1609/aimag.v36i1.2568
    36 https://doi.org/10.2200/s00362ed1v01y201105dtm016
    37 https://doi.org/10.2307/1403796
    38 https://doi.org/10.2307/1403797
    39 https://doi.org/10.3115/1289189.1289273
    40 https://doi.org/10.4018/ijagr.2015040105
    41 schema:datePublished 2016-11
    42 schema:datePublishedReg 2016-11-01
    43 schema:description This article examines foundational issues in data science including current challenges, basic research questions, and expected advances, as the basis for a new data science research program (DSRP) and associated data science evaluation (DSE) series, introduced by the National Institute of Standards and Technology (NIST) in the fall of 2015. The DSRP is designed to facilitate and accelerate research progress in the field of data science and consists of four components: evaluation and metrology, standards, compute infrastructure, and community outreach. A key part of the evaluation and measurement component is the DSE. The DSE series aims to address logistical and evaluation design challenges while providing rigorous measurement methods and an emphasis on generalizability rather than domain- and application-specific approaches. Toward that end, each year the DSE will consist of multiple research tracks and will encourage the application of tasks that span these tracks. The evaluations are intended to facilitate research efforts and collaboration, leverage shared infrastructure, and effectively address crosscutting challenges faced by diverse data science communities. Multiple research tracks will be championed by members of the data science community with the goal of enabling rigorous comparison of approaches through common tasks, datasets, metrics, and shared research challenges. The tracks will permit us to measure several different data science technologies in a wide range of fields and will address computing infrastructure, standards for an interoperability framework, and domain-specific examples. This article also summarizes lessons learned from the data science evaluation series pre-pilot that was held in fall of 2015.
    44 schema:genre research_article
    45 schema:inLanguage en
    46 schema:isAccessibleForFree true
    47 schema:isPartOf N4b3362c3c4b542a3af719a73310734e5
    48 N5631bbc3a0fd4d378aecc8be8d1e6586
    49 sg:journal.1156617
    50 schema:name A new data science research program: evaluation, metrology, standards, and community outreach
    51 schema:pagination 177-197
    52 schema:productId N383324ab12f84ce5bcc2dc86f3931d54
    53 N5bcfce3770584074854ce09f602ccf5a
    54 Ne73d43562d9c4ae1b987917c554442c7
    55 schema:sameAs https://app.dimensions.ai/details/publication/pub.1010806434
    56 https://doi.org/10.1007/s41060-016-0016-z
    57 schema:sdDatePublished 2019-04-11T09:31
    58 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    59 schema:sdPublisher Nad4a28c0cfbb4223bdfe2c7a62aa3b22
    60 schema:url https://link.springer.com/10.1007%2Fs41060-016-0016-z
    61 sgo:license sg:explorer/license/
    62 sgo:sdDataset articles
    63 rdf:type schema:ScholarlyArticle
    64 N0ba2c78e66c0498faf5c61fc3e84ad23 rdf:first sg:person.012235671121.11
    65 rdf:rest N883909a0b7f640cca8a4ed4e668c208c
    66 N1b35f20e92314b029bcbf9d1a94233a0 rdf:first sg:person.07612211461.24
    67 rdf:rest N979d4e18d691402db09a2665bccb9a26
    68 N1ea7c526b84647e89b439664f2d277e1 schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    69 rdf:type schema:Organization
    70 N33bfe61de6774966b9ec9c2c7e525c53 schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    71 rdf:type schema:Organization
    72 N34b4af3fbd7c42629537b47a566bf766 rdf:first sg:person.010165437221.98
    73 rdf:rest N4698c419d2d44496b27f8013c7626ff8
    74 N383324ab12f84ce5bcc2dc86f3931d54 schema:name readcube_id
    75 schema:value 337ba77f51e658650403d1ebc16e817e8e180a43ceeef5f4e9fc6b2caace3e5b
    76 rdf:type schema:PropertyValue
    77 N4698c419d2d44496b27f8013c7626ff8 rdf:first sg:person.012723713447.59
    78 rdf:rest Na3f450b4c00142c3b1c514dd938d5c70
    79 N4b3362c3c4b542a3af719a73310734e5 schema:volumeNumber 1
    80 rdf:type schema:PublicationVolume
    81 N5631bbc3a0fd4d378aecc8be8d1e6586 schema:issueNumber 3-4
    82 rdf:type schema:PublicationIssue
    83 N57ecfb8970b04babb3037cc06e12714e schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    84 rdf:type schema:Organization
    85 N5bcfce3770584074854ce09f602ccf5a schema:name dimensions_id
    86 schema:value pub.1010806434
    87 rdf:type schema:PropertyValue
    88 N68a9011e36384f21ab871fa07f329a26 rdf:first sg:person.013676745321.84
    89 rdf:rest Nfecb0dcdf8dd491ab113e849fefe2441
    90 N6c1b2d7115484afaa4a1185a79db4ef6 schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    91 rdf:type schema:Organization
    92 N7ae238f7f7b4450c93501f80e83d3f39 rdf:first sg:person.016141243221.07
    93 rdf:rest rdf:nil
    94 N8127f1606f0d4f9bade2cdb0e1dc9a7b schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    95 rdf:type schema:Organization
    96 N883909a0b7f640cca8a4ed4e668c208c rdf:first sg:person.07541377717.06
    97 rdf:rest N1b35f20e92314b029bcbf9d1a94233a0
    98 N979d4e18d691402db09a2665bccb9a26 rdf:first sg:person.011506423721.69
    99 rdf:rest N68a9011e36384f21ab871fa07f329a26
    100 N9db67d29686f462d91e4a8bccdfe00c8 schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    101 rdf:type schema:Organization
    102 Na3f450b4c00142c3b1c514dd938d5c70 rdf:first sg:person.014546302221.11
    103 rdf:rest N7ae238f7f7b4450c93501f80e83d3f39
    104 Na62acf8b25a04d8c99cae369d7ef8c54 schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    105 rdf:type schema:Organization
    106 Nad4a28c0cfbb4223bdfe2c7a62aa3b22 schema:name Springer Nature - SN SciGraph project
    107 rdf:type schema:Organization
    108 Nc8c8914eb293497c8e86d786f254ff5f schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    109 rdf:type schema:Organization
    110 Nd0518ed4d4524e72813ae752d5af45a3 schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    111 rdf:type schema:Organization
    112 Ne73d43562d9c4ae1b987917c554442c7 schema:name doi
    113 schema:value 10.1007/s41060-016-0016-z
    114 rdf:type schema:PropertyValue
    115 Nfbd02fe4d9754d4a8f85f196022e0bcd schema:name 100 Bureau Drive, Mail Stop 8940, 20899, Gaithersburg, MD, USA
    116 rdf:type schema:Organization
    117 Nfecb0dcdf8dd491ab113e849fefe2441 rdf:first sg:person.013233274711.54
    118 rdf:rest N34b4af3fbd7c42629537b47a566bf766
    119 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    120 schema:name Information and Computing Sciences
    121 rdf:type schema:DefinedTerm
    122 anzsrc-for:0806 schema:inDefinedTermSet anzsrc-for:
    123 schema:name Information Systems
    124 rdf:type schema:DefinedTerm
    125 sg:journal.1156617 schema:issn 2364-415X
    126 2364-4168
    127 schema:name International Journal of Data Science and Analytics
    128 rdf:type schema:Periodical
    129 sg:person.010165437221.98 schema:affiliation N33bfe61de6774966b9ec9c2c7e525c53
    130 schema:familyName Aulov
    131 schema:givenName Oleg
    132 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010165437221.98
    133 rdf:type schema:Person
    134 sg:person.011506423721.69 schema:affiliation Na62acf8b25a04d8c99cae369d7ef8c54
    135 schema:familyName Przybocki
    136 schema:givenName Mark
    137 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011506423721.69
    138 rdf:type schema:Person
    139 sg:person.012235671121.11 schema:affiliation N1ea7c526b84647e89b439664f2d277e1
    140 schema:familyName Dorr
    141 schema:givenName Bonnie J.
    142 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012235671121.11
    143 rdf:type schema:Person
    144 sg:person.012723713447.59 schema:affiliation Nd0518ed4d4524e72813ae752d5af45a3
    145 schema:familyName Michel
    146 schema:givenName Martial
    147 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012723713447.59
    148 rdf:type schema:Person
    149 sg:person.013233274711.54 schema:affiliation N6c1b2d7115484afaa4a1185a79db4ef6
    150 schema:familyName Ploehn
    151 schema:givenName Cathryn
    152 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013233274711.54
    153 rdf:type schema:Person
    154 sg:person.013676745321.84 schema:affiliation N9db67d29686f462d91e4a8bccdfe00c8
    155 schema:familyName Le Bras
    156 schema:givenName Marion
    157 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013676745321.84
    158 rdf:type schema:Person
    159 sg:person.014546302221.11 schema:affiliation N57ecfb8970b04babb3037cc06e12714e
    160 schema:familyName Golden
    161 schema:givenName E. Jim
    162 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014546302221.11
    163 rdf:type schema:Person
    164 sg:person.016141243221.07 schema:affiliation Nfbd02fe4d9754d4a8f85f196022e0bcd
    165 schema:familyName Chang
    166 schema:givenName Wo
    167 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016141243221.07
    168 rdf:type schema:Person
    169 sg:person.07541377717.06 schema:affiliation Nc8c8914eb293497c8e86d786f254ff5f
    170 schema:familyName Greenberg
    171 schema:givenName Craig S.
    172 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07541377717.06
    173 rdf:type schema:Person
    174 sg:person.07612211461.24 schema:affiliation N8127f1606f0d4f9bade2cdb0e1dc9a7b
    175 schema:familyName Fontana
    176 schema:givenName Peter
    177 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.07612211461.24
    178 rdf:type schema:Person
    179 sg:pub.10.1007/3-540-44450-5_6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1006354016
    180 https://doi.org/10.1007/3-540-44450-5_6
    181 rdf:type schema:CreativeWork
    182 sg:pub.10.1007/978-3-319-04483-5_5 schema:sameAs https://app.dimensions.ai/details/publication/pub.1009252916
    183 https://doi.org/10.1007/978-3-319-04483-5_5
    184 rdf:type schema:CreativeWork
    185 sg:pub.10.1007/978-3-319-07121-3_5 schema:sameAs https://app.dimensions.ai/details/publication/pub.1005856317
    186 https://doi.org/10.1007/978-3-319-07121-3_5
    187 rdf:type schema:CreativeWork
    188 sg:pub.10.1007/978-3-642-31164-2 schema:sameAs https://app.dimensions.ai/details/publication/pub.1020812050
    189 https://doi.org/10.1007/978-3-642-31164-2
    190 rdf:type schema:CreativeWork
    191 sg:pub.10.1186/1471-2164-15-s8-s1 schema:sameAs https://app.dimensions.ai/details/publication/pub.1026431064
    192 https://doi.org/10.1186/1471-2164-15-s8-s1
    193 rdf:type schema:CreativeWork
    194 https://app.dimensions.ai/details/publication/pub.1016571298 schema:CreativeWork
    195 https://doi.org/10.1002/scin.2015.187003022 schema:sameAs https://app.dimensions.ai/details/publication/pub.1019380626
    196 rdf:type schema:CreativeWork
    197 https://doi.org/10.1016/j.jadohealth.2005.05.008 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004965366
    198 rdf:type schema:CreativeWork
    199 https://doi.org/10.1017/cbo9780511802270 schema:sameAs https://app.dimensions.ai/details/publication/pub.1098730668
    200 rdf:type schema:CreativeWork
    201 https://doi.org/10.1109/2945.981847 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061146380
    202 rdf:type schema:CreativeWork
    203 https://doi.org/10.1109/cyber.2012.6392557 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094895622
    204 rdf:type schema:CreativeWork
    205 https://doi.org/10.1109/dsaa.2015.7344805 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094544877
    206 rdf:type schema:CreativeWork
    207 https://doi.org/10.1109/icdm.2012.161 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094208037
    208 rdf:type schema:CreativeWork
    209 https://doi.org/10.1109/ipdps.2007.370216 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095542773
    210 rdf:type schema:CreativeWork
    211 https://doi.org/10.1109/itsc.2013.6728263 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095410976
    212 rdf:type schema:CreativeWork
    213 https://doi.org/10.1109/mcg.2003.1210860 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061391259
    214 rdf:type schema:CreativeWork
    215 https://doi.org/10.1109/odyssey.2006.248120 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095801126
    216 rdf:type schema:CreativeWork
    217 https://doi.org/10.1109/tits.2014.2305334 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061658343
    218 rdf:type schema:CreativeWork
    219 https://doi.org/10.1109/tvcg.2011.279 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061813667
    220 rdf:type schema:CreativeWork
    221 https://doi.org/10.1109/tvcg.2013.126 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061813994
    222 rdf:type schema:CreativeWork
    223 https://doi.org/10.1137/1.9781611972818.4 schema:sameAs https://app.dimensions.ai/details/publication/pub.1088800536
    224 rdf:type schema:CreativeWork
    225 https://doi.org/10.1137/1.9781611972818.61 schema:sameAs https://app.dimensions.ai/details/publication/pub.1088800562
    226 rdf:type schema:CreativeWork
    227 https://doi.org/10.1145/1084805.1084812 schema:sameAs https://app.dimensions.ai/details/publication/pub.1019590669
    228 rdf:type schema:CreativeWork
    229 https://doi.org/10.1145/146565.146567 schema:sameAs https://app.dimensions.ai/details/publication/pub.1011005778
    230 rdf:type schema:CreativeWork
    231 https://doi.org/10.1145/1541880.1541882 schema:sameAs https://app.dimensions.ai/details/publication/pub.1030762489
    232 rdf:type schema:CreativeWork
    233 https://doi.org/10.1145/2609876.2609888 schema:sameAs https://app.dimensions.ai/details/publication/pub.1043453265
    234 rdf:type schema:CreativeWork
    235 https://doi.org/10.1145/2666680 schema:sameAs https://app.dimensions.ai/details/publication/pub.1019964287
    236 rdf:type schema:CreativeWork
    237 https://doi.org/10.1214/09-ss057 schema:sameAs https://app.dimensions.ai/details/publication/pub.1008096476
    238 rdf:type schema:CreativeWork
    239 https://doi.org/10.14778/2367502.2367564 schema:sameAs https://app.dimensions.ai/details/publication/pub.1067368108
    240 rdf:type schema:CreativeWork
    241 https://doi.org/10.1561/1900000006 schema:sameAs https://app.dimensions.ai/details/publication/pub.1068001356
    242 rdf:type schema:CreativeWork
    243 https://doi.org/10.1609/aimag.v36i1.2565 schema:sameAs https://app.dimensions.ai/details/publication/pub.1103067312
    244 rdf:type schema:CreativeWork
    245 https://doi.org/10.1609/aimag.v36i1.2568 schema:sameAs https://app.dimensions.ai/details/publication/pub.1103067306
    246 rdf:type schema:CreativeWork
    247 https://doi.org/10.2200/s00362ed1v01y201105dtm016 schema:sameAs https://app.dimensions.ai/details/publication/pub.1069288243
    248 rdf:type schema:CreativeWork
    249 https://doi.org/10.2307/1403796 schema:sameAs https://app.dimensions.ai/details/publication/pub.1069474051
    250 rdf:type schema:CreativeWork
    251 https://doi.org/10.2307/1403797 schema:sameAs https://app.dimensions.ai/details/publication/pub.1069474052
    252 rdf:type schema:CreativeWork
    253 https://doi.org/10.3115/1289189.1289273 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099184024
    254 rdf:type schema:CreativeWork
    255 https://doi.org/10.4018/ijagr.2015040105 schema:sameAs https://app.dimensions.ai/details/publication/pub.1071883212
    256 rdf:type schema:CreativeWork
     




    Preview window. Press ESC to close (or click here)


    ...