Reliable Evaluation of Multimodal Dialogue Systems View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2009

AUTHORS

Florian Metze , Ina Wechsung , Stefan Schaffer , Julia Seebode , Sebastian Möller

ABSTRACT

Usability evaluation is an indispensable issue during the development of new interfaces and interaction paradigms [1]. Although a wide range of reliable usability evaluation methods exists for graphical user interfaces, mature methods are rarely available for speech-based interfaces [2]. When it comes to multimodal interfaces, no standardized approach has so far been established. In previous studies [3], it was shown that usability questionnaires initially developed for unimodal systems may lead to unreliable results when applied to multimodal systems. In the current study, we therefore used several data sources (direct and indirect measurements) to evaluate two unimodal versions and one multimodal version of an information system. We investigated, to which extent the different data showed concordance for the three system versions. The aim was to examine, if, and under which conditions, common and widely used methods originally developed for graphical user interfaces are also appropriate for speech-based and multimodal intelligent interfaces. More... »

PAGES

75-83

Book

TITLE

Human-Computer Interaction. Novel Interaction Methods and Techniques

ISBN

978-3-642-02576-1
978-3-642-02577-8

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-642-02577-8_9

DOI

http://dx.doi.org/10.1007/978-3-642-02577-8_9

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1041381268


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0806", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information Systems", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Carnegie Mellon University", 
          "id": "https://www.grid.ac/institutes/grid.147455.6", 
          "name": [
            "School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Metze", 
        "givenName": "Florian", 
        "id": "sg:person.013657424677.17", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013657424677.17"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technical University of Berlin", 
          "id": "https://www.grid.ac/institutes/grid.6734.6", 
          "name": [
            "Deutsche Telekom Laboratories, TU Berlin, Ernst-Reuter-Platz 7, Berlin, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Wechsung", 
        "givenName": "Ina", 
        "id": "sg:person.013166463700.39", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013166463700.39"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technical University of Berlin", 
          "id": "https://www.grid.ac/institutes/grid.6734.6", 
          "name": [
            "Deutsche Telekom Laboratories, TU Berlin, Ernst-Reuter-Platz 7, Berlin, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Schaffer", 
        "givenName": "Stefan", 
        "id": "sg:person.012752344567.30", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012752344567.30"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technical University of Berlin", 
          "id": "https://www.grid.ac/institutes/grid.6734.6", 
          "name": [
            "Deutsche Telekom Laboratories, TU Berlin, Ernst-Reuter-Platz 7, Berlin, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Seebode", 
        "givenName": "Julia", 
        "id": "sg:person.013025676653.43", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013025676653.43"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Technical University of Berlin", 
          "id": "https://www.grid.ac/institutes/grid.6734.6", 
          "name": [
            "Deutsche Telekom Laboratories, TU Berlin, Ernst-Reuter-Platz 7, Berlin, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "M\u00f6ller", 
        "givenName": "Sebastian", 
        "id": "sg:person.011030454635.16", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011030454635.16"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.1145/266180.266328", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1000034459"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/175276.175282", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1003663536"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-322-80058-9_19", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1019950090", 
          "https://doi.org/10.1007/978-3-322-80058-9_19"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.sigpro.2006.02.040", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1020405882"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/800250.807503", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1024545232"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1240624.1240722", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1025837396"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/345513.345232", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1026603960"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1054972.1055028", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1031912423"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-540-69369-7_32", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1032476767", 
          "https://doi.org/10.1007/978-3-540-69369-7_32"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/332040.332455", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1045473583"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/b978-0-12-374708-2.00014-0", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1051486020"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/mis.2001.1183338", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1061405498"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2009", 
    "datePublishedReg": "2009-01-01", 
    "description": "Usability evaluation is an indispensable issue during the development of new interfaces and interaction paradigms [1]. Although a wide range of reliable usability evaluation methods exists for graphical user interfaces, mature methods are rarely available for speech-based interfaces [2]. When it comes to multimodal interfaces, no standardized approach has so far been established. In previous studies [3], it was shown that usability questionnaires initially developed for unimodal systems may lead to unreliable results when applied to multimodal systems. In the current study, we therefore used several data sources (direct and indirect measurements) to evaluate two unimodal versions and one multimodal version of an information system. We investigated, to which extent the different data showed concordance for the three system versions. The aim was to examine, if, and under which conditions, common and widely used methods originally developed for graphical user interfaces are also appropriate for speech-based and multimodal intelligent interfaces.", 
    "editor": [
      {
        "familyName": "Jacko", 
        "givenName": "Julie A.", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-642-02577-8_9", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-642-02576-1", 
        "978-3-642-02577-8"
      ], 
      "name": "Human-Computer Interaction. Novel Interaction Methods and Techniques", 
      "type": "Book"
    }, 
    "name": "Reliable Evaluation of Multimodal Dialogue Systems", 
    "pagination": "75-83", 
    "productId": [
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-642-02577-8_9"
        ]
      }, 
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "c520d535aefed7f4e78236c001a7b5f94e404a92f17bfad05552c6e8864eb1d6"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1041381268"
        ]
      }
    ], 
    "publisher": {
      "location": "Berlin, Heidelberg", 
      "name": "Springer Berlin Heidelberg", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-642-02577-8_9", 
      "https://app.dimensions.ai/details/publication/pub.1041381268"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2019-04-15T13:05", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8663_00000559.jsonl", 
    "type": "Chapter", 
    "url": "http://link.springer.com/10.1007/978-3-642-02577-8_9"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-02577-8_9'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-02577-8_9'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-02577-8_9'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-642-02577-8_9'


 

This table displays all metadata directly associated to this object as RDF triples.

134 TRIPLES      23 PREDICATES      39 URIs      20 LITERALS      8 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-642-02577-8_9 schema:about anzsrc-for:08
2 anzsrc-for:0806
3 schema:author Nb8bf93d567454f84ba191b62958dd506
4 schema:citation sg:pub.10.1007/978-3-322-80058-9_19
5 sg:pub.10.1007/978-3-540-69369-7_32
6 https://doi.org/10.1016/b978-0-12-374708-2.00014-0
7 https://doi.org/10.1016/j.sigpro.2006.02.040
8 https://doi.org/10.1109/mis.2001.1183338
9 https://doi.org/10.1145/1054972.1055028
10 https://doi.org/10.1145/1240624.1240722
11 https://doi.org/10.1145/175276.175282
12 https://doi.org/10.1145/266180.266328
13 https://doi.org/10.1145/332040.332455
14 https://doi.org/10.1145/345513.345232
15 https://doi.org/10.1145/800250.807503
16 schema:datePublished 2009
17 schema:datePublishedReg 2009-01-01
18 schema:description Usability evaluation is an indispensable issue during the development of new interfaces and interaction paradigms [1]. Although a wide range of reliable usability evaluation methods exists for graphical user interfaces, mature methods are rarely available for speech-based interfaces [2]. When it comes to multimodal interfaces, no standardized approach has so far been established. In previous studies [3], it was shown that usability questionnaires initially developed for unimodal systems may lead to unreliable results when applied to multimodal systems. In the current study, we therefore used several data sources (direct and indirect measurements) to evaluate two unimodal versions and one multimodal version of an information system. We investigated, to which extent the different data showed concordance for the three system versions. The aim was to examine, if, and under which conditions, common and widely used methods originally developed for graphical user interfaces are also appropriate for speech-based and multimodal intelligent interfaces.
19 schema:editor N3226033302a145d2a9137d128a526992
20 schema:genre chapter
21 schema:inLanguage en
22 schema:isAccessibleForFree false
23 schema:isPartOf N950216bf6def425d8faaa0eddd1f4e94
24 schema:name Reliable Evaluation of Multimodal Dialogue Systems
25 schema:pagination 75-83
26 schema:productId N3b0d63601366404dbee3c5cb4a789e1b
27 N4558a8a329744313939b14a7e08f252b
28 Ncc57b2287b594ee0bc5146d68c26e057
29 schema:publisher N4c35e1d00c7c404ca586208bf7bb9ce3
30 schema:sameAs https://app.dimensions.ai/details/publication/pub.1041381268
31 https://doi.org/10.1007/978-3-642-02577-8_9
32 schema:sdDatePublished 2019-04-15T13:05
33 schema:sdLicense https://scigraph.springernature.com/explorer/license/
34 schema:sdPublisher N52de5f0a911243b4b6c29d4a64f203e7
35 schema:url http://link.springer.com/10.1007/978-3-642-02577-8_9
36 sgo:license sg:explorer/license/
37 sgo:sdDataset chapters
38 rdf:type schema:Chapter
39 N0985b4ef896d443386a71b971b19718f rdf:first sg:person.013025676653.43
40 rdf:rest N4ee80837591b426fb98d534ac75638df
41 N103b38f2810e4bacbb10cef68ac4a5c7 rdf:first sg:person.012752344567.30
42 rdf:rest N0985b4ef896d443386a71b971b19718f
43 N3226033302a145d2a9137d128a526992 rdf:first N65f590ec152c46989b1f9c046e7b5004
44 rdf:rest rdf:nil
45 N3b0d63601366404dbee3c5cb4a789e1b schema:name dimensions_id
46 schema:value pub.1041381268
47 rdf:type schema:PropertyValue
48 N4558a8a329744313939b14a7e08f252b schema:name readcube_id
49 schema:value c520d535aefed7f4e78236c001a7b5f94e404a92f17bfad05552c6e8864eb1d6
50 rdf:type schema:PropertyValue
51 N4c35e1d00c7c404ca586208bf7bb9ce3 schema:location Berlin, Heidelberg
52 schema:name Springer Berlin Heidelberg
53 rdf:type schema:Organisation
54 N4ee80837591b426fb98d534ac75638df rdf:first sg:person.011030454635.16
55 rdf:rest rdf:nil
56 N52de5f0a911243b4b6c29d4a64f203e7 schema:name Springer Nature - SN SciGraph project
57 rdf:type schema:Organization
58 N5bd6186425404406bd6186211a55399f rdf:first sg:person.013166463700.39
59 rdf:rest N103b38f2810e4bacbb10cef68ac4a5c7
60 N65f590ec152c46989b1f9c046e7b5004 schema:familyName Jacko
61 schema:givenName Julie A.
62 rdf:type schema:Person
63 N950216bf6def425d8faaa0eddd1f4e94 schema:isbn 978-3-642-02576-1
64 978-3-642-02577-8
65 schema:name Human-Computer Interaction. Novel Interaction Methods and Techniques
66 rdf:type schema:Book
67 Nb8bf93d567454f84ba191b62958dd506 rdf:first sg:person.013657424677.17
68 rdf:rest N5bd6186425404406bd6186211a55399f
69 Ncc57b2287b594ee0bc5146d68c26e057 schema:name doi
70 schema:value 10.1007/978-3-642-02577-8_9
71 rdf:type schema:PropertyValue
72 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
73 schema:name Information and Computing Sciences
74 rdf:type schema:DefinedTerm
75 anzsrc-for:0806 schema:inDefinedTermSet anzsrc-for:
76 schema:name Information Systems
77 rdf:type schema:DefinedTerm
78 sg:person.011030454635.16 schema:affiliation https://www.grid.ac/institutes/grid.6734.6
79 schema:familyName Möller
80 schema:givenName Sebastian
81 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011030454635.16
82 rdf:type schema:Person
83 sg:person.012752344567.30 schema:affiliation https://www.grid.ac/institutes/grid.6734.6
84 schema:familyName Schaffer
85 schema:givenName Stefan
86 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012752344567.30
87 rdf:type schema:Person
88 sg:person.013025676653.43 schema:affiliation https://www.grid.ac/institutes/grid.6734.6
89 schema:familyName Seebode
90 schema:givenName Julia
91 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013025676653.43
92 rdf:type schema:Person
93 sg:person.013166463700.39 schema:affiliation https://www.grid.ac/institutes/grid.6734.6
94 schema:familyName Wechsung
95 schema:givenName Ina
96 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013166463700.39
97 rdf:type schema:Person
98 sg:person.013657424677.17 schema:affiliation https://www.grid.ac/institutes/grid.147455.6
99 schema:familyName Metze
100 schema:givenName Florian
101 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013657424677.17
102 rdf:type schema:Person
103 sg:pub.10.1007/978-3-322-80058-9_19 schema:sameAs https://app.dimensions.ai/details/publication/pub.1019950090
104 https://doi.org/10.1007/978-3-322-80058-9_19
105 rdf:type schema:CreativeWork
106 sg:pub.10.1007/978-3-540-69369-7_32 schema:sameAs https://app.dimensions.ai/details/publication/pub.1032476767
107 https://doi.org/10.1007/978-3-540-69369-7_32
108 rdf:type schema:CreativeWork
109 https://doi.org/10.1016/b978-0-12-374708-2.00014-0 schema:sameAs https://app.dimensions.ai/details/publication/pub.1051486020
110 rdf:type schema:CreativeWork
111 https://doi.org/10.1016/j.sigpro.2006.02.040 schema:sameAs https://app.dimensions.ai/details/publication/pub.1020405882
112 rdf:type schema:CreativeWork
113 https://doi.org/10.1109/mis.2001.1183338 schema:sameAs https://app.dimensions.ai/details/publication/pub.1061405498
114 rdf:type schema:CreativeWork
115 https://doi.org/10.1145/1054972.1055028 schema:sameAs https://app.dimensions.ai/details/publication/pub.1031912423
116 rdf:type schema:CreativeWork
117 https://doi.org/10.1145/1240624.1240722 schema:sameAs https://app.dimensions.ai/details/publication/pub.1025837396
118 rdf:type schema:CreativeWork
119 https://doi.org/10.1145/175276.175282 schema:sameAs https://app.dimensions.ai/details/publication/pub.1003663536
120 rdf:type schema:CreativeWork
121 https://doi.org/10.1145/266180.266328 schema:sameAs https://app.dimensions.ai/details/publication/pub.1000034459
122 rdf:type schema:CreativeWork
123 https://doi.org/10.1145/332040.332455 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045473583
124 rdf:type schema:CreativeWork
125 https://doi.org/10.1145/345513.345232 schema:sameAs https://app.dimensions.ai/details/publication/pub.1026603960
126 rdf:type schema:CreativeWork
127 https://doi.org/10.1145/800250.807503 schema:sameAs https://app.dimensions.ai/details/publication/pub.1024545232
128 rdf:type schema:CreativeWork
129 https://www.grid.ac/institutes/grid.147455.6 schema:alternateName Carnegie Mellon University
130 schema:name School of Computer Science, Carnegie Mellon University, Pittsburgh, PA 15213
131 rdf:type schema:Organization
132 https://www.grid.ac/institutes/grid.6734.6 schema:alternateName Technical University of Berlin
133 schema:name Deutsche Telekom Laboratories, TU Berlin, Ernst-Reuter-Platz 7, Berlin, Germany
134 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...