Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2018-05-30

AUTHORS

Suat Gonul , Tuncay Namli , Mert Baskaya , Ali Anil Sinaci , Ahmet Cosar , Ismail Hakki Toroslu

ABSTRACT

Momentary context data is an important source for intelligent decision making towards personalization of mobile phone notifications. We propose a reinforcement learning based personalized notification delivery algorithm, reasoning over momentary context data. Beyond the state of the art, we propose new approaches for faster convergence of the algorithm and jump start of learning performance at the beginning of the learning process. We test our approach in both simulated and real settings trying to optimize the timing of the notifications. Our eventual, practical aim is to make office workers more physically active during the work time. We compare the results obtained for standard and improved algorithms in both testbeds where improved versions yield better results. More... »

PAGES

334-341

Book

TITLE

Recent Trends and Future Technology in Applied Intelligence

ISBN

978-3-319-92057-3
978-3-319-92058-0

From Grant

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32

DOI

http://dx.doi.org/10.1007/978-3-319-92058-0_32

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1104306246


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Software Research and Development Consulting", 
          "id": "https://www.grid.ac/institutes/grid.426455.4", 
          "name": [
            "Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey", 
            "SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Gonul", 
        "givenName": "Suat", 
        "id": "sg:person.01360552653.99", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01360552653.99"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Software Research and Development Consulting", 
          "id": "https://www.grid.ac/institutes/grid.426455.4", 
          "name": [
            "SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Namli", 
        "givenName": "Tuncay", 
        "id": "sg:person.01275626711.65", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01275626711.65"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Software Research and Development Consulting", 
          "id": "https://www.grid.ac/institutes/grid.426455.4", 
          "name": [
            "Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey", 
            "SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Baskaya", 
        "givenName": "Mert", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Software Research and Development Consulting", 
          "id": "https://www.grid.ac/institutes/grid.426455.4", 
          "name": [
            "SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Sinaci", 
        "givenName": "Ali Anil", 
        "id": "sg:person.01217445075.14", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01217445075.14"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Middle East Technical University", 
          "id": "https://www.grid.ac/institutes/grid.6935.9", 
          "name": [
            "Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Cosar", 
        "givenName": "Ahmet", 
        "id": "sg:person.011166066701.15", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011166066701.15"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Middle East Technical University", 
          "id": "https://www.grid.ac/institutes/grid.6935.9", 
          "name": [
            "Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Toroslu", 
        "givenName": "Ismail Hakki", 
        "id": "sg:person.014316614404.36", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014316614404.36"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.3109/09638237.2011.638001", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1004225469"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.2196/jmir.4055", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1004623943"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s00779-012-0588-0", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1008315676", 
          "https://doi.org/10.1007/s00779-012-0588-0"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/s2213-8587(13)70067-6", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1017128386"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/s2213-8587(13)70067-6", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1017128386"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1370/afm.1952", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1017664429"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3389/fncom.2012.00082", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1025942996"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1080/13873954.2010.520409", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1036499366"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.2147/ahct.s57844", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1042013642"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/3130956", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1091537829"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/icme.2015.7177522", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1095489142"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2018-05-30", 
    "datePublishedReg": "2018-05-30", 
    "description": "Momentary context data is an important source for intelligent decision making towards personalization of mobile phone notifications. We propose a reinforcement learning based personalized notification delivery algorithm, reasoning over momentary context data. Beyond the state of the art, we propose new approaches for faster convergence of the algorithm and jump start of learning performance at the beginning of the learning process. We test our approach in both simulated and real settings trying to optimize the timing of the notifications. Our eventual, practical aim is to make office workers more physically active during the work time. We compare the results obtained for standard and improved algorithms in both testbeds where improved versions yield better results.", 
    "editor": [
      {
        "familyName": "Mouhoub", 
        "givenName": "Malek", 
        "type": "Person"
      }, 
      {
        "familyName": "Sadaoui", 
        "givenName": "Samira", 
        "type": "Person"
      }, 
      {
        "familyName": "Ait Mohamed", 
        "givenName": "Otmane", 
        "type": "Person"
      }, 
      {
        "familyName": "Ali", 
        "givenName": "Moonis", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-92058-0_32", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": false, 
    "isFundedItemOf": [
      {
        "id": "sg:grant.5050568", 
        "type": "MonetaryGrant"
      }
    ], 
    "isPartOf": {
      "isbn": [
        "978-3-319-92057-3", 
        "978-3-319-92058-0"
      ], 
      "name": "Recent Trends and Future Technology in Applied Intelligence", 
      "type": "Book"
    }, 
    "name": "Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning", 
    "pagination": "334-341", 
    "productId": [
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-92058-0_32"
        ]
      }, 
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "5e7222f9d6943a61cfa210325e921b85eb6da5eb6d96bc33e311370e0734dab3"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1104306246"
        ]
      }
    ], 
    "publisher": {
      "location": "Cham", 
      "name": "Springer International Publishing", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-92058-0_32", 
      "https://app.dimensions.ai/details/publication/pub.1104306246"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2019-04-16T05:01", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000325_0000000325/records_100801_00000000.jsonl", 
    "type": "Chapter", 
    "url": "https://link.springer.com/10.1007%2F978-3-319-92058-0_32"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92058-0_32'


 

This table displays all metadata directly associated to this object as RDF triples.

151 TRIPLES      23 PREDICATES      36 URIs      19 LITERALS      8 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-92058-0_32 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author N36c3260901e048d2a51fb34720bc887f
4 schema:citation sg:pub.10.1007/s00779-012-0588-0
5 https://doi.org/10.1016/s2213-8587(13)70067-6
6 https://doi.org/10.1080/13873954.2010.520409
7 https://doi.org/10.1109/icme.2015.7177522
8 https://doi.org/10.1145/3130956
9 https://doi.org/10.1370/afm.1952
10 https://doi.org/10.2147/ahct.s57844
11 https://doi.org/10.2196/jmir.4055
12 https://doi.org/10.3109/09638237.2011.638001
13 https://doi.org/10.3389/fncom.2012.00082
14 schema:datePublished 2018-05-30
15 schema:datePublishedReg 2018-05-30
16 schema:description Momentary context data is an important source for intelligent decision making towards personalization of mobile phone notifications. We propose a reinforcement learning based personalized notification delivery algorithm, reasoning over momentary context data. Beyond the state of the art, we propose new approaches for faster convergence of the algorithm and jump start of learning performance at the beginning of the learning process. We test our approach in both simulated and real settings trying to optimize the timing of the notifications. Our eventual, practical aim is to make office workers more physically active during the work time. We compare the results obtained for standard and improved algorithms in both testbeds where improved versions yield better results.
17 schema:editor Ne4ef50ec5dde45e5aabf145199cf7901
18 schema:genre chapter
19 schema:inLanguage en
20 schema:isAccessibleForFree false
21 schema:isPartOf N9a7c707e09874ebea568d17559c41732
22 schema:name Optimization of Just-in-Time Adaptive Interventions Using Reinforcement Learning
23 schema:pagination 334-341
24 schema:productId N1b80c1a98e6a40c1896bceb78035e95f
25 Ncc7e669a80bb4288aba961afc9f3e9bb
26 Ndad8e53b99014c819b15a120a5bdbaa1
27 schema:publisher N82e025c3ea8f4b01822c8f6dc22c0962
28 schema:sameAs https://app.dimensions.ai/details/publication/pub.1104306246
29 https://doi.org/10.1007/978-3-319-92058-0_32
30 schema:sdDatePublished 2019-04-16T05:01
31 schema:sdLicense https://scigraph.springernature.com/explorer/license/
32 schema:sdPublisher Ndf1d52bbae2847c4afd98039aad2f9de
33 schema:url https://link.springer.com/10.1007%2F978-3-319-92058-0_32
34 sgo:license sg:explorer/license/
35 sgo:sdDataset chapters
36 rdf:type schema:Chapter
37 N08c41a690a9844e79363f5f926f3a80c rdf:first N23320f73e9094ca58cf429054deec46f
38 rdf:rest rdf:nil
39 N1b80c1a98e6a40c1896bceb78035e95f schema:name doi
40 schema:value 10.1007/978-3-319-92058-0_32
41 rdf:type schema:PropertyValue
42 N22d65e7f43f448b4b0bac6915e766470 schema:familyName Sadaoui
43 schema:givenName Samira
44 rdf:type schema:Person
45 N23320f73e9094ca58cf429054deec46f schema:familyName Ali
46 schema:givenName Moonis
47 rdf:type schema:Person
48 N36c3260901e048d2a51fb34720bc887f rdf:first sg:person.01360552653.99
49 rdf:rest N56f28c39616d460d8c25c4e854261df8
50 N43cf05ee251944cc96aa4c151d063bf0 rdf:first sg:person.011166066701.15
51 rdf:rest Nf739c34f16f84cd3aaa0346cdd7b70db
52 N506cb0a5407249d28027e06a1259d8e0 schema:affiliation https://www.grid.ac/institutes/grid.426455.4
53 schema:familyName Baskaya
54 schema:givenName Mert
55 rdf:type schema:Person
56 N53de510e2b0146d0921223598e8b924a rdf:first N506cb0a5407249d28027e06a1259d8e0
57 rdf:rest Ndd0f0e8bb6e649038fe8026f3a825bfb
58 N56f28c39616d460d8c25c4e854261df8 rdf:first sg:person.01275626711.65
59 rdf:rest N53de510e2b0146d0921223598e8b924a
60 N596801f719784eb69e693ad9b1a8ca48 rdf:first N22d65e7f43f448b4b0bac6915e766470
61 rdf:rest Na77d539ca13d41d09172c5ab2d9974e7
62 N775f99ef3e5144599494d6946088c9bd schema:familyName Ait Mohamed
63 schema:givenName Otmane
64 rdf:type schema:Person
65 N82e025c3ea8f4b01822c8f6dc22c0962 schema:location Cham
66 schema:name Springer International Publishing
67 rdf:type schema:Organisation
68 N9a7c707e09874ebea568d17559c41732 schema:isbn 978-3-319-92057-3
69 978-3-319-92058-0
70 schema:name Recent Trends and Future Technology in Applied Intelligence
71 rdf:type schema:Book
72 Na77d539ca13d41d09172c5ab2d9974e7 rdf:first N775f99ef3e5144599494d6946088c9bd
73 rdf:rest N08c41a690a9844e79363f5f926f3a80c
74 Nb3a0e7b9142d4daa987bed6622176401 schema:familyName Mouhoub
75 schema:givenName Malek
76 rdf:type schema:Person
77 Ncc7e669a80bb4288aba961afc9f3e9bb schema:name readcube_id
78 schema:value 5e7222f9d6943a61cfa210325e921b85eb6da5eb6d96bc33e311370e0734dab3
79 rdf:type schema:PropertyValue
80 Ndad8e53b99014c819b15a120a5bdbaa1 schema:name dimensions_id
81 schema:value pub.1104306246
82 rdf:type schema:PropertyValue
83 Ndd0f0e8bb6e649038fe8026f3a825bfb rdf:first sg:person.01217445075.14
84 rdf:rest N43cf05ee251944cc96aa4c151d063bf0
85 Ndf1d52bbae2847c4afd98039aad2f9de schema:name Springer Nature - SN SciGraph project
86 rdf:type schema:Organization
87 Ne4ef50ec5dde45e5aabf145199cf7901 rdf:first Nb3a0e7b9142d4daa987bed6622176401
88 rdf:rest N596801f719784eb69e693ad9b1a8ca48
89 Nf739c34f16f84cd3aaa0346cdd7b70db rdf:first sg:person.014316614404.36
90 rdf:rest rdf:nil
91 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
92 schema:name Information and Computing Sciences
93 rdf:type schema:DefinedTerm
94 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
95 schema:name Artificial Intelligence and Image Processing
96 rdf:type schema:DefinedTerm
97 sg:grant.5050568 http://pending.schema.org/fundedItem sg:pub.10.1007/978-3-319-92058-0_32
98 rdf:type schema:MonetaryGrant
99 sg:person.011166066701.15 schema:affiliation https://www.grid.ac/institutes/grid.6935.9
100 schema:familyName Cosar
101 schema:givenName Ahmet
102 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011166066701.15
103 rdf:type schema:Person
104 sg:person.01217445075.14 schema:affiliation https://www.grid.ac/institutes/grid.426455.4
105 schema:familyName Sinaci
106 schema:givenName Ali Anil
107 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01217445075.14
108 rdf:type schema:Person
109 sg:person.01275626711.65 schema:affiliation https://www.grid.ac/institutes/grid.426455.4
110 schema:familyName Namli
111 schema:givenName Tuncay
112 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01275626711.65
113 rdf:type schema:Person
114 sg:person.01360552653.99 schema:affiliation https://www.grid.ac/institutes/grid.426455.4
115 schema:familyName Gonul
116 schema:givenName Suat
117 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01360552653.99
118 rdf:type schema:Person
119 sg:person.014316614404.36 schema:affiliation https://www.grid.ac/institutes/grid.6935.9
120 schema:familyName Toroslu
121 schema:givenName Ismail Hakki
122 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014316614404.36
123 rdf:type schema:Person
124 sg:pub.10.1007/s00779-012-0588-0 schema:sameAs https://app.dimensions.ai/details/publication/pub.1008315676
125 https://doi.org/10.1007/s00779-012-0588-0
126 rdf:type schema:CreativeWork
127 https://doi.org/10.1016/s2213-8587(13)70067-6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1017128386
128 rdf:type schema:CreativeWork
129 https://doi.org/10.1080/13873954.2010.520409 schema:sameAs https://app.dimensions.ai/details/publication/pub.1036499366
130 rdf:type schema:CreativeWork
131 https://doi.org/10.1109/icme.2015.7177522 schema:sameAs https://app.dimensions.ai/details/publication/pub.1095489142
132 rdf:type schema:CreativeWork
133 https://doi.org/10.1145/3130956 schema:sameAs https://app.dimensions.ai/details/publication/pub.1091537829
134 rdf:type schema:CreativeWork
135 https://doi.org/10.1370/afm.1952 schema:sameAs https://app.dimensions.ai/details/publication/pub.1017664429
136 rdf:type schema:CreativeWork
137 https://doi.org/10.2147/ahct.s57844 schema:sameAs https://app.dimensions.ai/details/publication/pub.1042013642
138 rdf:type schema:CreativeWork
139 https://doi.org/10.2196/jmir.4055 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004623943
140 rdf:type schema:CreativeWork
141 https://doi.org/10.3109/09638237.2011.638001 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004225469
142 rdf:type schema:CreativeWork
143 https://doi.org/10.3389/fncom.2012.00082 schema:sameAs https://app.dimensions.ai/details/publication/pub.1025942996
144 rdf:type schema:CreativeWork
145 https://www.grid.ac/institutes/grid.426455.4 schema:alternateName Software Research and Development Consulting
146 schema:name Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey
147 SRDC Software Research and Development and Consultancy Ltd., 06800, Ankara, Turkey
148 rdf:type schema:Organization
149 https://www.grid.ac/institutes/grid.6935.9 schema:alternateName Middle East Technical University
150 schema:name Department of Computer Engineering, Middle East Technical University, 06800, Ankara, Turkey
151 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...