An Assessment Framework for DialPort View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2019

AUTHORS

Kyusong Lee , Tiancheng Zhao , Stefan Ultes , Lina Rojas-Barahona , Eli Pincus , David Traum , Maxine Eskenazi

ABSTRACT

Collecting a large amount of real human-computer interaction data in various domains is a cornerstone in the development of better data-driven spoken dialog systems. The DialPort project is creating a portal to collect a constant stream of real user conversational data on a variety of topics. In order to keep real users attracted to DialPort, it is crucial to develop a robust evaluation framework to monitor and maintain high performance. Different from earlier spoken dialog systems, DialPort has a heterogeneous set of spoken dialog systems gathered under one outward-looking agent. In order to access this new structure, we have identified some unique challenges that DialPort will encounter so that it can appeal to real users and have created a novel evaluation scheme that quantitatively assesses their performance in these situations. We look at assessment from the point of view of the system developer as well as that of the end user. More... »

PAGES

79-85

Book

TITLE

Advanced Social Interaction with Agents

ISBN

978-3-319-92107-5
978-3-319-92108-2

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-92108-2_10

DOI

http://dx.doi.org/10.1007/978-3-319-92108-2_10

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1105950434


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0806", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information Systems", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Carnegie Mellon University", 
          "id": "https://www.grid.ac/institutes/grid.147455.6", 
          "name": [
            "Carnegie Mellon University"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Lee", 
        "givenName": "Kyusong", 
        "id": "sg:person.012764435160.34", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012764435160.34"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Carnegie Mellon University", 
          "id": "https://www.grid.ac/institutes/grid.147455.6", 
          "name": [
            "Carnegie Mellon University"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Zhao", 
        "givenName": "Tiancheng", 
        "id": "sg:person.012167054560.39", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012167054560.39"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "University of Cambridge", 
          "id": "https://www.grid.ac/institutes/grid.5335.0", 
          "name": [
            "Cambridge University"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Ultes", 
        "givenName": "Stefan", 
        "id": "sg:person.013716707043.86", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013716707043.86"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "University of Cambridge", 
          "id": "https://www.grid.ac/institutes/grid.5335.0", 
          "name": [
            "Cambridge University"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Rojas-Barahona", 
        "givenName": "Lina", 
        "id": "sg:person.011526365443.03", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011526365443.03"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "University of Southern California", 
          "id": "https://www.grid.ac/institutes/grid.42505.36", 
          "name": [
            "University of Southern California"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Pincus", 
        "givenName": "Eli", 
        "id": "sg:person.011544033014.86", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011544033014.86"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "University of Southern California", 
          "id": "https://www.grid.ac/institutes/grid.42505.36", 
          "name": [
            "University of Southern California"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Traum", 
        "givenName": "David", 
        "id": "sg:person.013767173315.17", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013767173315.17"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Carnegie Mellon University", 
          "id": "https://www.grid.ac/institutes/grid.147455.6", 
          "name": [
            "Carnegie Mellon University"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Eskenazi", 
        "givenName": "Maxine", 
        "id": "sg:person.0654777434.75", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0654777434.75"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.3115/979617.979652", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1003353698"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.csl.2006.06.008", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1015870155"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/slt.2016.7846249", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1094788629"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.18653/v1/w16-6007", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1098653786"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.18653/v1/d16-1230", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1098653802"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/v1/p15-2130", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099110258"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/v1/p15-2130", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099110258"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1620754.1620846", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099151725"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.18653/v1/w17-5521", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1100517460"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2019", 
    "datePublishedReg": "2019-01-01", 
    "description": "Collecting a large amount of real human-computer interaction data in various domains is a cornerstone in the development of better data-driven spoken dialog systems. The DialPort project is creating a portal to collect a constant stream of real user conversational data on a variety of topics. In order to keep real users attracted to DialPort, it is crucial to develop a robust evaluation framework to monitor and maintain high performance. Different from earlier spoken dialog systems, DialPort has a heterogeneous set of spoken dialog systems gathered under one outward-looking agent. In order to access this new structure, we have identified some unique challenges that DialPort will encounter so that it can appeal to real users and have created a novel evaluation scheme that quantitatively assesses their performance in these situations. We look at assessment from the point of view of the system developer as well as that of the end user.", 
    "editor": [
      {
        "familyName": "Eskenazi", 
        "givenName": "Maxine", 
        "type": "Person"
      }, 
      {
        "familyName": "Devillers", 
        "givenName": "Laurence", 
        "type": "Person"
      }, 
      {
        "familyName": "Mariani", 
        "givenName": "Joseph", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-92108-2_10", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-319-92107-5", 
        "978-3-319-92108-2"
      ], 
      "name": "Advanced Social Interaction with Agents", 
      "type": "Book"
    }, 
    "name": "An Assessment Framework for DialPort", 
    "pagination": "79-85", 
    "productId": [
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-92108-2_10"
        ]
      }, 
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "c327661d9f2e76313e389311f9b79e2af561a2190b7cb43731517aaf1d60e1df"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1105950434"
        ]
      }
    ], 
    "publisher": {
      "location": "Cham", 
      "name": "Springer International Publishing", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-92108-2_10", 
      "https://app.dimensions.ai/details/publication/pub.1105950434"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2019-04-15T20:45", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8687_00000605.jsonl", 
    "type": "Chapter", 
    "url": "http://link.springer.com/10.1007/978-3-319-92108-2_10"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92108-2_10'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92108-2_10'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92108-2_10'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-92108-2_10'


 

This table displays all metadata directly associated to this object as RDF triples.

147 TRIPLES      23 PREDICATES      35 URIs      20 LITERALS      8 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-92108-2_10 schema:about anzsrc-for:08
2 anzsrc-for:0806
3 schema:author N5f8684d1ddd344d8834a9b6f6ab4d36e
4 schema:citation https://doi.org/10.1016/j.csl.2006.06.008
5 https://doi.org/10.1109/slt.2016.7846249
6 https://doi.org/10.18653/v1/d16-1230
7 https://doi.org/10.18653/v1/w16-6007
8 https://doi.org/10.18653/v1/w17-5521
9 https://doi.org/10.3115/1620754.1620846
10 https://doi.org/10.3115/979617.979652
11 https://doi.org/10.3115/v1/p15-2130
12 schema:datePublished 2019
13 schema:datePublishedReg 2019-01-01
14 schema:description Collecting a large amount of real human-computer interaction data in various domains is a cornerstone in the development of better data-driven spoken dialog systems. The DialPort project is creating a portal to collect a constant stream of real user conversational data on a variety of topics. In order to keep real users attracted to DialPort, it is crucial to develop a robust evaluation framework to monitor and maintain high performance. Different from earlier spoken dialog systems, DialPort has a heterogeneous set of spoken dialog systems gathered under one outward-looking agent. In order to access this new structure, we have identified some unique challenges that DialPort will encounter so that it can appeal to real users and have created a novel evaluation scheme that quantitatively assesses their performance in these situations. We look at assessment from the point of view of the system developer as well as that of the end user.
15 schema:editor Nea899c6864e64e19836e15fe43cfd9a6
16 schema:genre chapter
17 schema:inLanguage en
18 schema:isAccessibleForFree false
19 schema:isPartOf Ncfd51a2e1acb4084b5bc4bf38e2f0a09
20 schema:name An Assessment Framework for DialPort
21 schema:pagination 79-85
22 schema:productId N346880dcefcc4eac857e0fc3f7ee5ad4
23 N4b511e3dfd2e4c7cbca5085d9a51ddd9
24 N629152e880ed4ba0b4f97975cc1b0464
25 schema:publisher N8fd4f77e5b2443908399eb9dc8ce4655
26 schema:sameAs https://app.dimensions.ai/details/publication/pub.1105950434
27 https://doi.org/10.1007/978-3-319-92108-2_10
28 schema:sdDatePublished 2019-04-15T20:45
29 schema:sdLicense https://scigraph.springernature.com/explorer/license/
30 schema:sdPublisher Nb0c320e9f922450da668dc8d9b736c67
31 schema:url http://link.springer.com/10.1007/978-3-319-92108-2_10
32 sgo:license sg:explorer/license/
33 sgo:sdDataset chapters
34 rdf:type schema:Chapter
35 N08ab121276bc438a90101697ca8f8825 rdf:first Ndfc299c3a2be4a3798d7ce5ba4084a2e
36 rdf:rest N56a19f4fb82d4c5bae010c0ff62895a0
37 N09d61c837af847189263c68f1ab3f555 rdf:first sg:person.0654777434.75
38 rdf:rest rdf:nil
39 N18284c8b2f7f46d19387c586081d5f66 schema:familyName Eskenazi
40 schema:givenName Maxine
41 rdf:type schema:Person
42 N2949069b8d524039ad4646e722ff7444 schema:familyName Mariani
43 schema:givenName Joseph
44 rdf:type schema:Person
45 N346880dcefcc4eac857e0fc3f7ee5ad4 schema:name doi
46 schema:value 10.1007/978-3-319-92108-2_10
47 rdf:type schema:PropertyValue
48 N471f33ff061749ed8eba2572ea634027 rdf:first sg:person.011544033014.86
49 rdf:rest Na5bb4a3888c74a819877bc0264bf9471
50 N4b511e3dfd2e4c7cbca5085d9a51ddd9 schema:name readcube_id
51 schema:value c327661d9f2e76313e389311f9b79e2af561a2190b7cb43731517aaf1d60e1df
52 rdf:type schema:PropertyValue
53 N56a19f4fb82d4c5bae010c0ff62895a0 rdf:first N2949069b8d524039ad4646e722ff7444
54 rdf:rest rdf:nil
55 N5f8684d1ddd344d8834a9b6f6ab4d36e rdf:first sg:person.012764435160.34
56 rdf:rest Nae37b1aec8954e27becee7d9ee921cf9
57 N629152e880ed4ba0b4f97975cc1b0464 schema:name dimensions_id
58 schema:value pub.1105950434
59 rdf:type schema:PropertyValue
60 N8fd4f77e5b2443908399eb9dc8ce4655 schema:location Cham
61 schema:name Springer International Publishing
62 rdf:type schema:Organisation
63 Na5bb4a3888c74a819877bc0264bf9471 rdf:first sg:person.013767173315.17
64 rdf:rest N09d61c837af847189263c68f1ab3f555
65 Nae37b1aec8954e27becee7d9ee921cf9 rdf:first sg:person.012167054560.39
66 rdf:rest Neaa5bb2603094804a9f8968b88c91cb6
67 Nb0c320e9f922450da668dc8d9b736c67 schema:name Springer Nature - SN SciGraph project
68 rdf:type schema:Organization
69 Ncfd51a2e1acb4084b5bc4bf38e2f0a09 schema:isbn 978-3-319-92107-5
70 978-3-319-92108-2
71 schema:name Advanced Social Interaction with Agents
72 rdf:type schema:Book
73 Nd8d397afb28c42e2a037ee421431452d rdf:first sg:person.011526365443.03
74 rdf:rest N471f33ff061749ed8eba2572ea634027
75 Ndfc299c3a2be4a3798d7ce5ba4084a2e schema:familyName Devillers
76 schema:givenName Laurence
77 rdf:type schema:Person
78 Nea899c6864e64e19836e15fe43cfd9a6 rdf:first N18284c8b2f7f46d19387c586081d5f66
79 rdf:rest N08ab121276bc438a90101697ca8f8825
80 Neaa5bb2603094804a9f8968b88c91cb6 rdf:first sg:person.013716707043.86
81 rdf:rest Nd8d397afb28c42e2a037ee421431452d
82 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
83 schema:name Information and Computing Sciences
84 rdf:type schema:DefinedTerm
85 anzsrc-for:0806 schema:inDefinedTermSet anzsrc-for:
86 schema:name Information Systems
87 rdf:type schema:DefinedTerm
88 sg:person.011526365443.03 schema:affiliation https://www.grid.ac/institutes/grid.5335.0
89 schema:familyName Rojas-Barahona
90 schema:givenName Lina
91 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011526365443.03
92 rdf:type schema:Person
93 sg:person.011544033014.86 schema:affiliation https://www.grid.ac/institutes/grid.42505.36
94 schema:familyName Pincus
95 schema:givenName Eli
96 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011544033014.86
97 rdf:type schema:Person
98 sg:person.012167054560.39 schema:affiliation https://www.grid.ac/institutes/grid.147455.6
99 schema:familyName Zhao
100 schema:givenName Tiancheng
101 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012167054560.39
102 rdf:type schema:Person
103 sg:person.012764435160.34 schema:affiliation https://www.grid.ac/institutes/grid.147455.6
104 schema:familyName Lee
105 schema:givenName Kyusong
106 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012764435160.34
107 rdf:type schema:Person
108 sg:person.013716707043.86 schema:affiliation https://www.grid.ac/institutes/grid.5335.0
109 schema:familyName Ultes
110 schema:givenName Stefan
111 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013716707043.86
112 rdf:type schema:Person
113 sg:person.013767173315.17 schema:affiliation https://www.grid.ac/institutes/grid.42505.36
114 schema:familyName Traum
115 schema:givenName David
116 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013767173315.17
117 rdf:type schema:Person
118 sg:person.0654777434.75 schema:affiliation https://www.grid.ac/institutes/grid.147455.6
119 schema:familyName Eskenazi
120 schema:givenName Maxine
121 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0654777434.75
122 rdf:type schema:Person
123 https://doi.org/10.1016/j.csl.2006.06.008 schema:sameAs https://app.dimensions.ai/details/publication/pub.1015870155
124 rdf:type schema:CreativeWork
125 https://doi.org/10.1109/slt.2016.7846249 schema:sameAs https://app.dimensions.ai/details/publication/pub.1094788629
126 rdf:type schema:CreativeWork
127 https://doi.org/10.18653/v1/d16-1230 schema:sameAs https://app.dimensions.ai/details/publication/pub.1098653802
128 rdf:type schema:CreativeWork
129 https://doi.org/10.18653/v1/w16-6007 schema:sameAs https://app.dimensions.ai/details/publication/pub.1098653786
130 rdf:type schema:CreativeWork
131 https://doi.org/10.18653/v1/w17-5521 schema:sameAs https://app.dimensions.ai/details/publication/pub.1100517460
132 rdf:type schema:CreativeWork
133 https://doi.org/10.3115/1620754.1620846 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099151725
134 rdf:type schema:CreativeWork
135 https://doi.org/10.3115/979617.979652 schema:sameAs https://app.dimensions.ai/details/publication/pub.1003353698
136 rdf:type schema:CreativeWork
137 https://doi.org/10.3115/v1/p15-2130 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099110258
138 rdf:type schema:CreativeWork
139 https://www.grid.ac/institutes/grid.147455.6 schema:alternateName Carnegie Mellon University
140 schema:name Carnegie Mellon University
141 rdf:type schema:Organization
142 https://www.grid.ac/institutes/grid.42505.36 schema:alternateName University of Southern California
143 schema:name University of Southern California
144 rdf:type schema:Organization
145 https://www.grid.ac/institutes/grid.5335.0 schema:alternateName University of Cambridge
146 schema:name Cambridge University
147 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...