Recurrent Neural Networks for Multivariate Time Series with Missing Values View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

2018-04-17

AUTHORS

Zhengping Che, Sanjay Purushotham, Kyunghyun Cho, David Sontag, Yan Liu

ABSTRACT

Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It takes two representations of missing patterns, i.e., masking and time interval, and effectively incorporates them into a deep model architecture so that it not only captures the long-term temporal dependencies in time series, but also utilizes the missing patterns to achieve better prediction results. Experiments of time series classification tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets demonstrate that our models achieve state-of-the-art performance and provide useful insights for better understanding and utilization of missing values in time series analysis. More... »

PAGES

6085

References to SciGraph publications

  • 2008-09-09. Wavelet variance analysis for gappy time series in ANNALS OF THE INSTITUTE OF STATISTICAL MATHEMATICS
  • 2016-05-24. MIMIC-III, a freely accessible critical care database in SCIENTIFIC DATA
  • 2009-09-03. Pattern classification with missing data: a review in NEURAL COMPUTING AND APPLICATIONS
  • 1978. A Practical Guide to Splines in NONE
  • 2016-04-12. DeepCare: A Deep Dynamic Memory Model for Predictive Medicine in ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING
  • Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1038/s41598-018-24271-9

    DOI

    http://dx.doi.org/10.1038/s41598-018-24271-9

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1103353461

    PUBMED

    https://www.ncbi.nlm.nih.gov/pubmed/29666385


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Artificial Intelligence and Image Processing", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "University of Southern California, Department of Computer Science, Los Angeles, CA 90089 USA", 
              "id": "http://www.grid.ac/institutes/grid.42505.36", 
              "name": [
                "University of Southern California, Department of Computer Science, Los Angeles, CA 90089 USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Che", 
            "givenName": "Zhengping", 
            "id": "sg:person.0612275423.43", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0612275423.43"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "University of Southern California, Department of Computer Science, Los Angeles, CA 90089 USA", 
              "id": "http://www.grid.ac/institutes/grid.42505.36", 
              "name": [
                "University of Southern California, Department of Computer Science, Los Angeles, CA 90089 USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Purushotham", 
            "givenName": "Sanjay", 
            "id": "sg:person.010016547717.89", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010016547717.89"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "New York University, Department of Computer Science, New York, NY 10012 USA", 
              "id": "http://www.grid.ac/institutes/grid.137628.9", 
              "name": [
                "New York University, Department of Computer Science, New York, NY 10012 USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Cho", 
            "givenName": "Kyunghyun", 
            "id": "sg:person.013505034761.57", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013505034761.57"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, Cambridge, MA 02139 USA", 
              "id": "http://www.grid.ac/institutes/grid.116068.8", 
              "name": [
                "Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, Cambridge, MA 02139 USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Sontag", 
            "givenName": "David", 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "University of Southern California, Department of Computer Science, Los Angeles, CA 90089 USA", 
              "id": "http://www.grid.ac/institutes/grid.42505.36", 
              "name": [
                "University of Southern California, Department of Computer Science, Los Angeles, CA 90089 USA"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Liu", 
            "givenName": "Yan", 
            "id": "sg:person.011411510717.30", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011411510717.30"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "sg:pub.10.1007/s00521-009-0295-6", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1023664205", 
              "https://doi.org/10.1007/s00521-009-0295-6"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s10463-008-0195-z", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1044408815", 
              "https://doi.org/10.1007/s10463-008-0195-z"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-1-4612-6333-3", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1109705102", 
              "https://doi.org/10.1007/978-1-4612-6333-3"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/978-3-319-31750-2_3", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1007623856", 
              "https://doi.org/10.1007/978-3-319-31750-2_3"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1038/sdata.2016.35", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1039633073", 
              "https://doi.org/10.1038/sdata.2016.35"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2018-04-17", 
        "datePublishedReg": "2018-04-17", 
        "description": "Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It takes two representations of missing patterns, i.e., masking and time interval, and effectively incorporates them into a deep model architecture so that it not only captures the long-term temporal dependencies in time series, but also utilizes the missing patterns to achieve better prediction results. Experiments of time series classification tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets demonstrate that our models achieve state-of-the-art performance and provide useful insights for better understanding and utilization of missing values in time series analysis.", 
        "genre": "article", 
        "id": "sg:pub.10.1038/s41598-018-24271-9", 
        "inLanguage": "en", 
        "isAccessibleForFree": true, 
        "isPartOf": [
          {
            "id": "sg:journal.1045337", 
            "issn": [
              "2045-2322"
            ], 
            "name": "Scientific Reports", 
            "publisher": "Springer Nature", 
            "type": "Periodical"
          }, 
          {
            "issueNumber": "1", 
            "type": "PublicationIssue"
          }, 
          {
            "type": "PublicationVolume", 
            "volumeNumber": "8"
          }
        ], 
        "keywords": [
          "Gated Recurrent Unit", 
          "recurrent neural network", 
          "neural network", 
          "art recurrent neural networks", 
          "time series classification tasks", 
          "real-world clinical dataset", 
          "long-term temporal dependencies", 
          "novel deep learning model", 
          "deep model architectures", 
          "multivariate time series data", 
          "deep learning models", 
          "time series prediction", 
          "better prediction results", 
          "classification task", 
          "recurrent unit", 
          "model architecture", 
          "effective imputation", 
          "art performance", 
          "synthetic datasets", 
          "temporal dependencies", 
          "multivariate time series", 
          "target labels", 
          "related tasks", 
          "series prediction", 
          "learning model", 
          "time series data", 
          "missing values", 
          "prediction performance", 
          "clinical datasets", 
          "prediction results", 
          "series data", 
          "dataset", 
          "time series", 
          "network", 
          "task", 
          "limited work", 
          "informative missingness", 
          "practical applications", 
          "architecture", 
          "useful insights", 
          "performance", 
          "time series analysis", 
          "series analysis", 
          "representation", 
          "labels", 
          "geosciences", 
          "model", 
          "applications", 
          "dependency", 
          "imputation", 
          "early attempts", 
          "utilization", 
          "work", 
          "time interval", 
          "prediction", 
          "masking", 
          "data", 
          "state", 
          "experiments", 
          "missingness", 
          "health care", 
          "patterns", 
          "variety", 
          "results", 
          "better understanding", 
          "units", 
          "values", 
          "insights", 
          "attempt", 
          "analysis", 
          "series", 
          "understanding", 
          "biology", 
          "intervals", 
          "care", 
          "paper", 
          "series classification tasks"
        ], 
        "name": "Recurrent Neural Networks for Multivariate Time Series with Missing Values", 
        "pagination": "6085", 
        "productId": [
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1103353461"
            ]
          }, 
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1038/s41598-018-24271-9"
            ]
          }, 
          {
            "name": "pubmed_id", 
            "type": "PropertyValue", 
            "value": [
              "29666385"
            ]
          }
        ], 
        "sameAs": [
          "https://doi.org/10.1038/s41598-018-24271-9", 
          "https://app.dimensions.ai/details/publication/pub.1103353461"
        ], 
        "sdDataset": "articles", 
        "sdDatePublished": "2021-11-01T18:34", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-springernature-scigraph/baseset/20211101/entities/gbq_results/article/article_777.jsonl", 
        "type": "ScholarlyArticle", 
        "url": "https://doi.org/10.1038/s41598-018-24271-9"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1038/s41598-018-24271-9'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1038/s41598-018-24271-9'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1038/s41598-018-24271-9'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1038/s41598-018-24271-9'


     

    This table displays all metadata directly associated to this object as RDF triples.

    191 TRIPLES      22 PREDICATES      108 URIs      95 LITERALS      7 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1038/s41598-018-24271-9 schema:about anzsrc-for:08
    2 anzsrc-for:0801
    3 schema:author N8e13c921e39d44ed82128a94de37f166
    4 schema:citation sg:pub.10.1007/978-1-4612-6333-3
    5 sg:pub.10.1007/978-3-319-31750-2_3
    6 sg:pub.10.1007/s00521-009-0295-6
    7 sg:pub.10.1007/s10463-008-0195-z
    8 sg:pub.10.1038/sdata.2016.35
    9 schema:datePublished 2018-04-17
    10 schema:datePublishedReg 2018-04-17
    11 schema:description Multivariate time series data in practical applications, such as health care, geoscience, and biology, are characterized by a variety of missing values. In time series prediction and other related tasks, it has been noted that missing values and their missing patterns are often correlated with the target labels, a.k.a., informative missingness. There is very limited work on exploiting the missing patterns for effective imputation and improving prediction performance. In this paper, we develop novel deep learning models, namely GRU-D, as one of the early attempts. GRU-D is based on Gated Recurrent Unit (GRU), a state-of-the-art recurrent neural network. It takes two representations of missing patterns, i.e., masking and time interval, and effectively incorporates them into a deep model architecture so that it not only captures the long-term temporal dependencies in time series, but also utilizes the missing patterns to achieve better prediction results. Experiments of time series classification tasks on real-world clinical datasets (MIMIC-III, PhysioNet) and synthetic datasets demonstrate that our models achieve state-of-the-art performance and provide useful insights for better understanding and utilization of missing values in time series analysis.
    12 schema:genre article
    13 schema:inLanguage en
    14 schema:isAccessibleForFree true
    15 schema:isPartOf N37e436d95557423ba9b26893745e51af
    16 Naa144f82c496482fbbb0947ac3de07b2
    17 sg:journal.1045337
    18 schema:keywords Gated Recurrent Unit
    19 analysis
    20 applications
    21 architecture
    22 art performance
    23 art recurrent neural networks
    24 attempt
    25 better prediction results
    26 better understanding
    27 biology
    28 care
    29 classification task
    30 clinical datasets
    31 data
    32 dataset
    33 deep learning models
    34 deep model architectures
    35 dependency
    36 early attempts
    37 effective imputation
    38 experiments
    39 geosciences
    40 health care
    41 imputation
    42 informative missingness
    43 insights
    44 intervals
    45 labels
    46 learning model
    47 limited work
    48 long-term temporal dependencies
    49 masking
    50 missing values
    51 missingness
    52 model
    53 model architecture
    54 multivariate time series
    55 multivariate time series data
    56 network
    57 neural network
    58 novel deep learning model
    59 paper
    60 patterns
    61 performance
    62 practical applications
    63 prediction
    64 prediction performance
    65 prediction results
    66 real-world clinical dataset
    67 recurrent neural network
    68 recurrent unit
    69 related tasks
    70 representation
    71 results
    72 series
    73 series analysis
    74 series classification tasks
    75 series data
    76 series prediction
    77 state
    78 synthetic datasets
    79 target labels
    80 task
    81 temporal dependencies
    82 time interval
    83 time series
    84 time series analysis
    85 time series classification tasks
    86 time series data
    87 time series prediction
    88 understanding
    89 units
    90 useful insights
    91 utilization
    92 values
    93 variety
    94 work
    95 schema:name Recurrent Neural Networks for Multivariate Time Series with Missing Values
    96 schema:pagination 6085
    97 schema:productId N45eed345ec22450f8ffe2ddb6adcb5c0
    98 Ne985d9c0ea6248909090eacb950e2d77
    99 Nf11c6b033b19477198bc6fe97c548152
    100 schema:sameAs https://app.dimensions.ai/details/publication/pub.1103353461
    101 https://doi.org/10.1038/s41598-018-24271-9
    102 schema:sdDatePublished 2021-11-01T18:34
    103 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    104 schema:sdPublisher N64cc3231d84d41ac9b7674b6366573d0
    105 schema:url https://doi.org/10.1038/s41598-018-24271-9
    106 sgo:license sg:explorer/license/
    107 sgo:sdDataset articles
    108 rdf:type schema:ScholarlyArticle
    109 N30a733c51efd41c599eeb73b75ede2bb rdf:first sg:person.011411510717.30
    110 rdf:rest rdf:nil
    111 N37e436d95557423ba9b26893745e51af schema:volumeNumber 8
    112 rdf:type schema:PublicationVolume
    113 N45eed345ec22450f8ffe2ddb6adcb5c0 schema:name doi
    114 schema:value 10.1038/s41598-018-24271-9
    115 rdf:type schema:PropertyValue
    116 N64cc3231d84d41ac9b7674b6366573d0 schema:name Springer Nature - SN SciGraph project
    117 rdf:type schema:Organization
    118 N8e13c921e39d44ed82128a94de37f166 rdf:first sg:person.0612275423.43
    119 rdf:rest Ne8075ef4afb64d9a8f8ba01cc622be87
    120 Naa144f82c496482fbbb0947ac3de07b2 schema:issueNumber 1
    121 rdf:type schema:PublicationIssue
    122 Nbe2cade1ac9945bcaef763a7b14de58f schema:affiliation grid-institutes:grid.116068.8
    123 schema:familyName Sontag
    124 schema:givenName David
    125 rdf:type schema:Person
    126 Ndad3e7f6e5fc40ad9028d5ae31fd74d5 rdf:first sg:person.013505034761.57
    127 rdf:rest Ne89303000dbc4af99281e44bdc19bbc7
    128 Ne8075ef4afb64d9a8f8ba01cc622be87 rdf:first sg:person.010016547717.89
    129 rdf:rest Ndad3e7f6e5fc40ad9028d5ae31fd74d5
    130 Ne89303000dbc4af99281e44bdc19bbc7 rdf:first Nbe2cade1ac9945bcaef763a7b14de58f
    131 rdf:rest N30a733c51efd41c599eeb73b75ede2bb
    132 Ne985d9c0ea6248909090eacb950e2d77 schema:name dimensions_id
    133 schema:value pub.1103353461
    134 rdf:type schema:PropertyValue
    135 Nf11c6b033b19477198bc6fe97c548152 schema:name pubmed_id
    136 schema:value 29666385
    137 rdf:type schema:PropertyValue
    138 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    139 schema:name Information and Computing Sciences
    140 rdf:type schema:DefinedTerm
    141 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
    142 schema:name Artificial Intelligence and Image Processing
    143 rdf:type schema:DefinedTerm
    144 sg:journal.1045337 schema:issn 2045-2322
    145 schema:name Scientific Reports
    146 schema:publisher Springer Nature
    147 rdf:type schema:Periodical
    148 sg:person.010016547717.89 schema:affiliation grid-institutes:grid.42505.36
    149 schema:familyName Purushotham
    150 schema:givenName Sanjay
    151 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010016547717.89
    152 rdf:type schema:Person
    153 sg:person.011411510717.30 schema:affiliation grid-institutes:grid.42505.36
    154 schema:familyName Liu
    155 schema:givenName Yan
    156 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011411510717.30
    157 rdf:type schema:Person
    158 sg:person.013505034761.57 schema:affiliation grid-institutes:grid.137628.9
    159 schema:familyName Cho
    160 schema:givenName Kyunghyun
    161 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013505034761.57
    162 rdf:type schema:Person
    163 sg:person.0612275423.43 schema:affiliation grid-institutes:grid.42505.36
    164 schema:familyName Che
    165 schema:givenName Zhengping
    166 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0612275423.43
    167 rdf:type schema:Person
    168 sg:pub.10.1007/978-1-4612-6333-3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1109705102
    169 https://doi.org/10.1007/978-1-4612-6333-3
    170 rdf:type schema:CreativeWork
    171 sg:pub.10.1007/978-3-319-31750-2_3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1007623856
    172 https://doi.org/10.1007/978-3-319-31750-2_3
    173 rdf:type schema:CreativeWork
    174 sg:pub.10.1007/s00521-009-0295-6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1023664205
    175 https://doi.org/10.1007/s00521-009-0295-6
    176 rdf:type schema:CreativeWork
    177 sg:pub.10.1007/s10463-008-0195-z schema:sameAs https://app.dimensions.ai/details/publication/pub.1044408815
    178 https://doi.org/10.1007/s10463-008-0195-z
    179 rdf:type schema:CreativeWork
    180 sg:pub.10.1038/sdata.2016.35 schema:sameAs https://app.dimensions.ai/details/publication/pub.1039633073
    181 https://doi.org/10.1038/sdata.2016.35
    182 rdf:type schema:CreativeWork
    183 grid-institutes:grid.116068.8 schema:alternateName Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, Cambridge, MA 02139 USA
    184 schema:name Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, Cambridge, MA 02139 USA
    185 rdf:type schema:Organization
    186 grid-institutes:grid.137628.9 schema:alternateName New York University, Department of Computer Science, New York, NY 10012 USA
    187 schema:name New York University, Department of Computer Science, New York, NY 10012 USA
    188 rdf:type schema:Organization
    189 grid-institutes:grid.42505.36 schema:alternateName University of Southern California, Department of Computer Science, Los Angeles, CA 90089 USA
    190 schema:name University of Southern California, Department of Computer Science, Los Angeles, CA 90089 USA
    191 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...