Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2018-05-30

AUTHORS

Suat Gonul , Tuncay Namli , Mert Baskaya , Ali Anil Sinaci , Ahmet Cosar , Ismail Hakki Toroslu

ABSTRACT

Momentary context data is an important source for intelligent decision making towards personalization of mobile phone notifications. We propose a reinforcement learning based personalized notification delivery algorithm, reasoning over momentary context data. Beyond the state of the art, we propose new approaches for faster convergence of the algorithm and jump start of learning performance at the beginning of the learning process. We test our approach in both simulated and real settings trying to optimize the timing of the notifications. Our eventual, practical aim is to make office workers more physically active during the work time. We compare the results obtained for standard and improved algorithms in both testbeds where improved versions yield better results. More... »

PAGES

334-341

References to SciGraph publications

  • 2013-08. Toward a persuasive mobile application to reduce sedentary behavior in PERSONAL AND UBIQUITOUS COMPUTING
  • Book

    TITLE

    Recent Trends and Future Technology in Applied Intelligence

    ISBN

    978-3-319-92057-3
    978-3-319-92058-0

    From Grant

    Identifiers

    URI

    http://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32

    DOI

    http://dx.doi.org/10.1007/978-3-319-92058-0_32

    DIMENSIONS

    https://app.dimensions.ai/details/publication/pub.1104306246


    Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
    Incoming Citations Browse incoming citations for this publication using opencitations.net

    JSON-LD is the canonical representation for SciGraph data.

    TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

    [
      {
        "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
        "about": [
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Artificial Intelligence and Image Processing", 
            "type": "DefinedTerm"
          }, 
          {
            "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
            "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
            "name": "Information and Computing Sciences", 
            "type": "DefinedTerm"
          }
        ], 
        "author": [
          {
            "affiliation": {
              "alternateName": "Software Research and Development Consulting", 
              "id": "https://www.grid.ac/institutes/grid.426455.4", 
              "name": [
                "Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey", 
                "SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Gonul", 
            "givenName": "Suat", 
            "id": "sg:person.01360552653.99", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01360552653.99"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Software Research and Development Consulting", 
              "id": "https://www.grid.ac/institutes/grid.426455.4", 
              "name": [
                "SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Namli", 
            "givenName": "Tuncay", 
            "id": "sg:person.01275626711.65", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01275626711.65"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Software Research and Development Consulting", 
              "id": "https://www.grid.ac/institutes/grid.426455.4", 
              "name": [
                "Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey", 
                "SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Baskaya", 
            "givenName": "Mert", 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Software Research and Development Consulting", 
              "id": "https://www.grid.ac/institutes/grid.426455.4", 
              "name": [
                "SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Sinaci", 
            "givenName": "Ali Anil", 
            "id": "sg:person.01217445075.14", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01217445075.14"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Middle East Technical University", 
              "id": "https://www.grid.ac/institutes/grid.6935.9", 
              "name": [
                "Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Cosar", 
            "givenName": "Ahmet", 
            "id": "sg:person.011166066701.15", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011166066701.15"
            ], 
            "type": "Person"
          }, 
          {
            "affiliation": {
              "alternateName": "Middle East Technical University", 
              "id": "https://www.grid.ac/institutes/grid.6935.9", 
              "name": [
                "Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey"
              ], 
              "type": "Organization"
            }, 
            "familyName": "Toroslu", 
            "givenName": "Ismail Hakki", 
            "id": "sg:person.014316614404.36", 
            "sameAs": [
              "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014316614404.36"
            ], 
            "type": "Person"
          }
        ], 
        "citation": [
          {
            "id": "https://doi.org/10.3109/09638237.2011.638001", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1004225469"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.2196/jmir.4055", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1004623943"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "sg:pub.10.1007/s00779-012-0588-0", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1008315676", 
              "https://doi.org/10.1007/s00779-012-0588-0"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/s2213-8587(13)70067-6", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1017128386"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1016/s2213-8587(13)70067-6", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1017128386"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1370/afm.1952", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1017664429"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.3389/fncom.2012.00082", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1025942996"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1080/13873954.2010.520409", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1036499366"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.2147/ahct.s57844", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1042013642"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1145/3130956", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1091537829"
            ], 
            "type": "CreativeWork"
          }, 
          {
            "id": "https://doi.org/10.1109/icme.2015.7177522", 
            "sameAs": [
              "https://app.dimensions.ai/details/publication/pub.1095489142"
            ], 
            "type": "CreativeWork"
          }
        ], 
        "datePublished": "2018-05-30", 
        "datePublishedReg": "2018-05-30", 
        "description": "Momentary context data is an important source for intelligent decision making towards personalization of mobile phone notifications. We propose a reinforcement learning based personalized notification delivery algorithm, reasoning over momentary context data. Beyond the state of the art, we propose new approaches for faster convergence of the algorithm and jump start of learning performance at the beginning of the learning process. We test our approach in both simulated and real settings trying to optimize the timing of the notifications. Our eventual, practical aim is to make office workers more physically active during the work time. We compare the results obtained for standard and improved algorithms in both testbeds where improved versions yield better results.", 
        "editor": [
          {
            "familyName": "Mouhoub", 
            "givenName": "Malek", 
            "type": "Person"
          }, 
          {
            "familyName": "Sadaoui", 
            "givenName": "Samira", 
            "type": "Person"
          }, 
          {
            "familyName": "Ait Mohamed", 
            "givenName": "Otmane", 
            "type": "Person"
          }, 
          {
            "familyName": "Ali", 
            "givenName": "Moonis", 
            "type": "Person"
          }
        ], 
        "genre": "chapter", 
        "id": "sg:pub.10.1007/978-3-319-92058-0_32", 
        "inLanguage": [
          "en"
        ], 
        "isAccessibleForFree": false, 
        "isFundedItemOf": [
          {
            "id": "sg:grant.5050568", 
            "type": "MonetaryGrant"
          }
        ], 
        "isPartOf": {
          "isbn": [
            "978-3-319-92057-3", 
            "978-3-319-92058-0"
          ], 
          "name": "Recent Trends and Future Technology in Applied Intelligence", 
          "type": "Book"
        }, 
        "name": "Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning", 
        "pagination": "334-341", 
        "productId": [
          {
            "name": "doi", 
            "type": "PropertyValue", 
            "value": [
              "10.1007/978-3-319-92058-0_32"
            ]
          }, 
          {
            "name": "readcube_id", 
            "type": "PropertyValue", 
            "value": [
              "5e7222f9d6943a61cfa210325e921b85eb6da5eb6d96bc33e311370e0734dab3"
            ]
          }, 
          {
            "name": "dimensions_id", 
            "type": "PropertyValue", 
            "value": [
              "pub.1104306246"
            ]
          }
        ], 
        "publisher": {
          "location": "Cham", 
          "name": "Springer International Publishing", 
          "type": "Organisation"
        }, 
        "sameAs": [
          "https://doi.org/10.1007/978-3-319-92058-0_32", 
          "https://app.dimensions.ai/details/publication/pub.1104306246"
        ], 
        "sdDataset": "chapters", 
        "sdDatePublished": "2019-04-16T05:01", 
        "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
        "sdPublisher": {
          "name": "Springer Nature - SN SciGraph project", 
          "type": "Organization"
        }, 
        "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000325_0000000325/records_100801_00000000.jsonl", 
        "type": "Chapter", 
        "url": "https://link.springer.com/10.1007%2F978-3-319-92058-0_32"
      }
    ]
     

    Download the RDF metadata as:  json-ld nt turtle xml License info

    HOW TO GET THIS DATA PROGRAMMATICALLY:

    JSON-LD is a popular format for linked data which is fully compatible with JSON.

    curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32'

    N-Triples is a line-based linked data format ideal for batch operations.

    curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32'

    Turtle is a human-readable linked data format.

    curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32'

    RDF/XML is a standard XML format for linked data.

    curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32'


     

    This table displays all metadata directly associated to this object as RDF triples.

    151 TRIPLES      23 PREDICATES      36 URIs      19 LITERALS      8 BLANK NODES

    Subject Predicate Object
    1 sg:pub.10.1007/978-3-319-92058-0_32 schema:about anzsrc-for:08
    2 anzsrc-for:0801
    3 schema:author Nee99a45eb33e4c288a246a8f3699bab7
    4 schema:citation sg:pub.10.1007/s00779-012-0588-0
    5 https://doi.org/10.1016/s2213-8587(13)70067-6
    6 https://doi.org/10.1080/13873954.2010.520409
    7 https://doi.org/10.1109/icme.2015.7177522
    8 https://doi.org/10.1145/3130956
    9 https://doi.org/10.1370/afm.1952
    10 https://doi.org/10.2147/ahct.s57844
    11 https://doi.org/10.2196/jmir.4055
    12 https://doi.org/10.3109/09638237.2011.638001
    13 https://doi.org/10.3389/fncom.2012.00082
    14 schema:datePublished 2018-05-30
    15 schema:datePublishedReg 2018-05-30
    16 schema:description Momentary context data is an important source for intelligent decision making towards personalization of mobile phone notifications. We propose a reinforcement learning based personalized notification delivery algorithm, reasoning over momentary context data. Beyond the state of the art, we propose new approaches for faster convergence of the algorithm and jump start of learning performance at the beginning of the learning process. We test our approach in both simulated and real settings trying to optimize the timing of the notifications. Our eventual, practical aim is to make office workers more physically active during the work time. We compare the results obtained for standard and improved algorithms in both testbeds where improved versions yield better results.
    17 schema:editor Ncabefdc03ac54363b0bff81f4adc995b
    18 schema:genre chapter
    19 schema:inLanguage en
    20 schema:isAccessibleForFree false
    21 schema:isPartOf Nd8836d408412459c96543b9e3f95a814
    22 schema:name Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning
    23 schema:pagination 334-341
    24 schema:productId N02a9d32996714bc086d8e8b5f9576979
    25 N83075209a50a441ba1bd96a239403e93
    26 Ne83c3cc5a53c46038fc3589871d60fdd
    27 schema:publisher N877c61cdecfb4a329405d5a56414ddcf
    28 schema:sameAs https://app.dimensions.ai/details/publication/pub.1104306246
    29 https://doi.org/10.1007/978-3-319-92058-0_32
    30 schema:sdDatePublished 2019-04-16T05:01
    31 schema:sdLicense https://scigraph.springernature.com/explorer/license/
    32 schema:sdPublisher N4490fe52ea53493e875d318dc5d0ef9e
    33 schema:url https://link.springer.com/10.1007%2F978-3-319-92058-0_32
    34 sgo:license sg:explorer/license/
    35 sgo:sdDataset chapters
    36 rdf:type schema:Chapter
    37 N02a9d32996714bc086d8e8b5f9576979 schema:name readcube_id
    38 schema:value 5e7222f9d6943a61cfa210325e921b85eb6da5eb6d96bc33e311370e0734dab3
    39 rdf:type schema:PropertyValue
    40 N0990a4ffcab142309fb4a1e1a032b9b3 schema:familyName Ali
    41 schema:givenName Moonis
    42 rdf:type schema:Person
    43 N1201d9d49e00422ba5e2a0d3853e7d32 schema:familyName Sadaoui
    44 schema:givenName Samira
    45 rdf:type schema:Person
    46 N1c384dde4916432896320d37517185ef schema:familyName Mouhoub
    47 schema:givenName Malek
    48 rdf:type schema:Person
    49 N27fb452608504a348bbfe7da3797e7e3 rdf:first sg:person.01275626711.65
    50 rdf:rest N9334c44c7d3a484fb17b244cd92936d7
    51 N2df8d2501d4a48389659006f32183544 rdf:first N0990a4ffcab142309fb4a1e1a032b9b3
    52 rdf:rest rdf:nil
    53 N4490fe52ea53493e875d318dc5d0ef9e schema:name Springer Nature - SN SciGraph project
    54 rdf:type schema:Organization
    55 N4caedcbc91c942abb2fa3ffb263fd9ff rdf:first N4e2ddda2176441f986359b5358ae1be0
    56 rdf:rest N2df8d2501d4a48389659006f32183544
    57 N4e2ddda2176441f986359b5358ae1be0 schema:familyName Ait Mohamed
    58 schema:givenName Otmane
    59 rdf:type schema:Person
    60 N6c6eee2f841e4944bb1f40cf20a96173 rdf:first sg:person.011166066701.15
    61 rdf:rest N8f32bb7222274300b6ac7d6e9f0cab3b
    62 N83075209a50a441ba1bd96a239403e93 schema:name dimensions_id
    63 schema:value pub.1104306246
    64 rdf:type schema:PropertyValue
    65 N877c61cdecfb4a329405d5a56414ddcf schema:location Cham
    66 schema:name Springer International Publishing
    67 rdf:type schema:Organisation
    68 N8f32bb7222274300b6ac7d6e9f0cab3b rdf:first sg:person.014316614404.36
    69 rdf:rest rdf:nil
    70 N8fd14cdcb96240c9a0a54c39e2aa35fd rdf:first N1201d9d49e00422ba5e2a0d3853e7d32
    71 rdf:rest N4caedcbc91c942abb2fa3ffb263fd9ff
    72 N9334c44c7d3a484fb17b244cd92936d7 rdf:first Nb31d14ef770f49e4befb4423430ef351
    73 rdf:rest Ncc687c65c71e4fff88c66939396f033d
    74 Nb31d14ef770f49e4befb4423430ef351 schema:affiliation https://www.grid.ac/institutes/grid.426455.4
    75 schema:familyName Baskaya
    76 schema:givenName Mert
    77 rdf:type schema:Person
    78 Ncabefdc03ac54363b0bff81f4adc995b rdf:first N1c384dde4916432896320d37517185ef
    79 rdf:rest N8fd14cdcb96240c9a0a54c39e2aa35fd
    80 Ncc687c65c71e4fff88c66939396f033d rdf:first sg:person.01217445075.14
    81 rdf:rest N6c6eee2f841e4944bb1f40cf20a96173
    82 Nd8836d408412459c96543b9e3f95a814 schema:isbn 978-3-319-92057-3
    83 978-3-319-92058-0
    84 schema:name Recent Trends and Future Technology in Applied Intelligence
    85 rdf:type schema:Book
    86 Ne83c3cc5a53c46038fc3589871d60fdd schema:name doi
    87 schema:value 10.1007/978-3-319-92058-0_32
    88 rdf:type schema:PropertyValue
    89 Nee99a45eb33e4c288a246a8f3699bab7 rdf:first sg:person.01360552653.99
    90 rdf:rest N27fb452608504a348bbfe7da3797e7e3
    91 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
    92 schema:name Information and Computing Sciences
    93 rdf:type schema:DefinedTerm
    94 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
    95 schema:name Artificial Intelligence and Image Processing
    96 rdf:type schema:DefinedTerm
    97 sg:grant.5050568 http://pending.schema.org/fundedItem sg:pub.10.1007/978-3-319-92058-0_32
    98 rdf:type schema:MonetaryGrant
    99 sg:person.011166066701.15 schema:affiliation https://www.grid.ac/institutes/grid.6935.9
    100 schema:familyName Cosar
    101 schema:givenName Ahmet
    102 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011166066701.15
    103 rdf:type schema:Person
    104 sg:person.01217445075.14 schema:affiliation https://www.grid.ac/institutes/grid.426455.4
    105 schema:familyName Sinaci
    106 schema:givenName Ali Anil
    107 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01217445075.14
    108 rdf:type schema:Person
    109 sg:person.01275626711.65 schema:affiliation https://www.grid.ac/institutes/grid.426455.4
    110 schema:familyName Namli
    111 schema:givenName Tuncay
    112 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01275626711.65
    113 rdf:type schema:Person
    114 sg:person.01360552653.99 schema:affiliation https://www.grid.ac/institutes/grid.426455.4
    115 schema:familyName Gonul
    116 schema:givenName Suat
    117 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01360552653.99
    118 rdf:type schema:Person
    119 sg:person.014316614404.36 schema:affiliation https://www.grid.ac/institutes/grid.6935.9
    120 schema:familyName Toroslu
    121 schema:givenName Ismail Hakki
    122 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014316614404.36
    123 rdf:type schema:Person
    124 sg:pub.10.1007/s00779-012-0588-0 schema:sameAs https://app.dimensions.ai/details/publication/pub.1008315676
    125 https://doi.org/10.1007/s00779-012-0588-0
    126 rdf:type schema:CreativeWork
    127 https://doi.org/10.1016/s2213-8587(13)70067-6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1017128386
    128 rdf:type schema:CreativeWork
    129 https://doi.org/10.1080/13873954.2010.520409 schema:sameAs https://app.dimensions.ai/details/publication/pub.1036499366
    130 rdf:type schema:CreativeWork
    131 https://doi.org/10.1109/icme.2015.7177522 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095489142
    132 rdf:type schema:CreativeWork
    133 https://doi.org/10.1145/3130956 schema:sameAs https://app.dimensions.ai/details/publication/pub.1091537829
    134 rdf:type schema:CreativeWork
    135 https://doi.org/10.1370/afm.1952 schema:sameAs https://app.dimensions.ai/details/publication/pub.1017664429
    136 rdf:type schema:CreativeWork
    137 https://doi.org/10.2147/ahct.s57844 schema:sameAs https://app.dimensions.ai/details/publication/pub.1042013642
    138 rdf:type schema:CreativeWork
    139 https://doi.org/10.2196/jmir.4055 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004623943
    140 rdf:type schema:CreativeWork
    141 https://doi.org/10.3109/09638237.2011.638001 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004225469
    142 rdf:type schema:CreativeWork
    143 https://doi.org/10.3389/fncom.2012.00082 schema:sameAs https://app.dimensions.ai/details/publication/pub.1025942996
    144 rdf:type schema:CreativeWork
    145 https://www.grid.ac/institutes/grid.426455.4 schema:alternateName Software Research and Development Consulting
    146 schema:name Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey
    147 SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey
    148 rdf:type schema:Organization
    149 https://www.grid.ac/institutes/grid.6935.9 schema:alternateName Middle East Technical University
    150 schema:name Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey
    151 rdf:type schema:Organization
     




    Preview window. Press ESC to close (or click here)


    ...