Here or There View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2008

AUTHORS

Ben Carterette , Paul N. Bennett , David Maxwell Chickering , Susan T. Dumais

ABSTRACT

Information retrieval systems have traditionally been evaluated over absolute judgments of relevance: each document is judged for relevance on its own, independent of other documents that may be on topic. We hypothesize that preference judgments of the form “document A is more relevant than document B” are easier for assessors to make than absolute judgments, and provide evidence for our hypothesis through a study with assessors. We then investigate methods to evaluate search engines using preference judgments. Furthermore, we show that by using inferences and clever selection of pairs to judge, we need not compare all pairs of documents in order to apply evaluation methods. More... »

PAGES

16-27

Book

TITLE

Advances in Information Retrieval

ISBN

978-3-540-78645-0
978-3-540-78646-7

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-540-78646-7_5

DOI

http://dx.doi.org/10.1007/978-3-540-78646-7_5

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1039462420


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1117", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Public Health and Health Services", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/11", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Medical and Health Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "University of Massachusetts Amherst", 
          "id": "https://www.grid.ac/institutes/grid.266683.f", 
          "name": [
            "University of Massachusetts Amherst,"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Carterette", 
        "givenName": "Ben", 
        "id": "sg:person.015143634641.13", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015143634641.13"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "Microsoft Research,"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Bennett", 
        "givenName": "Paul N.", 
        "id": "sg:person.015207115744.95", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015207115744.95"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "Microsoft Live Labs,"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Chickering", 
        "givenName": "David Maxwell", 
        "id": "sg:person.011240332636.47", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011240332636.47"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "Microsoft Research,"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Dumais", 
        "givenName": "Susan T.", 
        "id": "sg:person.014627200551.35", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014627200551.35"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.1145/775047.775067", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1002194678"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1148170.1148289", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1006394966"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1076034.1076063", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1011160774"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1008992.1009000", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1014166097"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/345508.345545", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1015128878"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1148170.1148219", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1016626314"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/290941.291017", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1022973258"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0306-4573(91)90046-o", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1024588363"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/0306-4573(91)90046-o", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1024588363"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1102351.1102363", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1025364122"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1002/(sici)1097-4571(199012)41:8<590::aid-asi5>3.0.co;2-t", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1029308691"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1002/0471743984.vse5991", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1053090966"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2008", 
    "datePublishedReg": "2008-01-01", 
    "description": "Information retrieval systems have traditionally been evaluated over absolute judgments of relevance: each document is judged for relevance on its own, independent of other documents that may be on topic. We hypothesize that preference judgments of the form \u201cdocument A is more relevant than document B\u201d are easier for assessors to make than absolute judgments, and provide evidence for our hypothesis through a study with assessors. We then investigate methods to evaluate search engines using preference judgments. Furthermore, we show that by using inferences and clever selection of pairs to judge, we need not compare all pairs of documents in order to apply evaluation methods.", 
    "editor": [
      {
        "familyName": "Macdonald", 
        "givenName": "Craig", 
        "type": "Person"
      }, 
      {
        "familyName": "Ounis", 
        "givenName": "Iadh", 
        "type": "Person"
      }, 
      {
        "familyName": "Plachouras", 
        "givenName": "Vassilis", 
        "type": "Person"
      }, 
      {
        "familyName": "Ruthven", 
        "givenName": "Ian", 
        "type": "Person"
      }, 
      {
        "familyName": "White", 
        "givenName": "Ryen W.", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-540-78646-7_5", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-540-78645-0", 
        "978-3-540-78646-7"
      ], 
      "name": "Advances in Information Retrieval", 
      "type": "Book"
    }, 
    "name": "Here or There", 
    "pagination": "16-27", 
    "productId": [
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-540-78646-7_5"
        ]
      }, 
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "bbfb31f8c3e883c181b6254cd89c211fd9664d282c7b9fdd46ea74920e12e037"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1039462420"
        ]
      }
    ], 
    "publisher": {
      "location": "Berlin, Heidelberg", 
      "name": "Springer Berlin Heidelberg", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-540-78646-7_5", 
      "https://app.dimensions.ai/details/publication/pub.1039462420"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2019-04-15T23:29", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8695_00000554.jsonl", 
    "type": "Chapter", 
    "url": "http://link.springer.com/10.1007/978-3-540-78646-7_5"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-540-78646-7_5'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-540-78646-7_5'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-540-78646-7_5'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-540-78646-7_5'


 

This table displays all metadata directly associated to this object as RDF triples.

145 TRIPLES      23 PREDICATES      38 URIs      20 LITERALS      8 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-540-78646-7_5 schema:about anzsrc-for:11
2 anzsrc-for:1117
3 schema:author N67ef23ed0a60471aa530478b613a8512
4 schema:citation https://doi.org/10.1002/(sici)1097-4571(199012)41:8<590::aid-asi5>3.0.co;2-t
5 https://doi.org/10.1002/0471743984.vse5991
6 https://doi.org/10.1016/0306-4573(91)90046-o
7 https://doi.org/10.1145/1008992.1009000
8 https://doi.org/10.1145/1076034.1076063
9 https://doi.org/10.1145/1102351.1102363
10 https://doi.org/10.1145/1148170.1148219
11 https://doi.org/10.1145/1148170.1148289
12 https://doi.org/10.1145/290941.291017
13 https://doi.org/10.1145/345508.345545
14 https://doi.org/10.1145/775047.775067
15 schema:datePublished 2008
16 schema:datePublishedReg 2008-01-01
17 schema:description Information retrieval systems have traditionally been evaluated over absolute judgments of relevance: each document is judged for relevance on its own, independent of other documents that may be on topic. We hypothesize that preference judgments of the form “document A is more relevant than document B” are easier for assessors to make than absolute judgments, and provide evidence for our hypothesis through a study with assessors. We then investigate methods to evaluate search engines using preference judgments. Furthermore, we show that by using inferences and clever selection of pairs to judge, we need not compare all pairs of documents in order to apply evaluation methods.
18 schema:editor N3e1d5444f147405dacdcbdc60e44ffd6
19 schema:genre chapter
20 schema:inLanguage en
21 schema:isAccessibleForFree false
22 schema:isPartOf Neaa925f4414044769f65de86ec91dd31
23 schema:name Here or There
24 schema:pagination 16-27
25 schema:productId N43314e8339f44d62a3062be8a581d8fc
26 N69fc477ab723440cba2d14e51de1a4bc
27 N8a8e4c6bb80f4ee6ba8b9f7a3db33c6a
28 schema:publisher N3bc81c51a9474af99fc9651413ebbf64
29 schema:sameAs https://app.dimensions.ai/details/publication/pub.1039462420
30 https://doi.org/10.1007/978-3-540-78646-7_5
31 schema:sdDatePublished 2019-04-15T23:29
32 schema:sdLicense https://scigraph.springernature.com/explorer/license/
33 schema:sdPublisher N6e938e498f974cafaada6d830e089c36
34 schema:url http://link.springer.com/10.1007/978-3-540-78646-7_5
35 sgo:license sg:explorer/license/
36 sgo:sdDataset chapters
37 rdf:type schema:Chapter
38 N1a1fedd8ef974d84a4ea1115846ad268 schema:name Microsoft Research,
39 rdf:type schema:Organization
40 N1e51398e9f29494e8051e3682e4c0036 schema:familyName White
41 schema:givenName Ryen W.
42 rdf:type schema:Person
43 N200ba66d8ec64a058dca8e4ce25dfd32 rdf:first sg:person.014627200551.35
44 rdf:rest rdf:nil
45 N384f625198744bc3ada8e55f1a86424b schema:name Microsoft Live Labs,
46 rdf:type schema:Organization
47 N3bc81c51a9474af99fc9651413ebbf64 schema:location Berlin, Heidelberg
48 schema:name Springer Berlin Heidelberg
49 rdf:type schema:Organisation
50 N3e1d5444f147405dacdcbdc60e44ffd6 rdf:first N73847b0ba08442e6a6eefeb2f8c3db57
51 rdf:rest N87f4e6d00c9846179b209bccb94a229b
52 N43314e8339f44d62a3062be8a581d8fc schema:name readcube_id
53 schema:value bbfb31f8c3e883c181b6254cd89c211fd9664d282c7b9fdd46ea74920e12e037
54 rdf:type schema:PropertyValue
55 N4cc92084e477441f916e967ff13a1d0e schema:familyName Ruthven
56 schema:givenName Ian
57 rdf:type schema:Person
58 N5435eba26a464075a440f9a0f6ae317c schema:familyName Ounis
59 schema:givenName Iadh
60 rdf:type schema:Person
61 N675046cb223144d89e0ec4eb936fa9a4 rdf:first N1e51398e9f29494e8051e3682e4c0036
62 rdf:rest rdf:nil
63 N67ef23ed0a60471aa530478b613a8512 rdf:first sg:person.015143634641.13
64 rdf:rest Nfb53e6a910d34c7996486cce43db992b
65 N69fc477ab723440cba2d14e51de1a4bc schema:name dimensions_id
66 schema:value pub.1039462420
67 rdf:type schema:PropertyValue
68 N6e938e498f974cafaada6d830e089c36 schema:name Springer Nature - SN SciGraph project
69 rdf:type schema:Organization
70 N73847b0ba08442e6a6eefeb2f8c3db57 schema:familyName Macdonald
71 schema:givenName Craig
72 rdf:type schema:Person
73 N87f4e6d00c9846179b209bccb94a229b rdf:first N5435eba26a464075a440f9a0f6ae317c
74 rdf:rest Nef2356dfe58e4b03b762405549d6593a
75 N8a8e4c6bb80f4ee6ba8b9f7a3db33c6a schema:name doi
76 schema:value 10.1007/978-3-540-78646-7_5
77 rdf:type schema:PropertyValue
78 Na2eccd2a5439488b83dc17adc2eab9e9 rdf:first N4cc92084e477441f916e967ff13a1d0e
79 rdf:rest N675046cb223144d89e0ec4eb936fa9a4
80 Nc4fea5bfae294a258a72c3b3430a73f6 schema:name Microsoft Research,
81 rdf:type schema:Organization
82 Nd33509e137b54921b57921bb1b368592 rdf:first sg:person.011240332636.47
83 rdf:rest N200ba66d8ec64a058dca8e4ce25dfd32
84 Neaa925f4414044769f65de86ec91dd31 schema:isbn 978-3-540-78645-0
85 978-3-540-78646-7
86 schema:name Advances in Information Retrieval
87 rdf:type schema:Book
88 Nef2356dfe58e4b03b762405549d6593a rdf:first Nf5fda631d6e54f97812318e51424b73b
89 rdf:rest Na2eccd2a5439488b83dc17adc2eab9e9
90 Nf5fda631d6e54f97812318e51424b73b schema:familyName Plachouras
91 schema:givenName Vassilis
92 rdf:type schema:Person
93 Nfb53e6a910d34c7996486cce43db992b rdf:first sg:person.015207115744.95
94 rdf:rest Nd33509e137b54921b57921bb1b368592
95 anzsrc-for:11 schema:inDefinedTermSet anzsrc-for:
96 schema:name Medical and Health Sciences
97 rdf:type schema:DefinedTerm
98 anzsrc-for:1117 schema:inDefinedTermSet anzsrc-for:
99 schema:name Public Health and Health Services
100 rdf:type schema:DefinedTerm
101 sg:person.011240332636.47 schema:affiliation N384f625198744bc3ada8e55f1a86424b
102 schema:familyName Chickering
103 schema:givenName David Maxwell
104 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.011240332636.47
105 rdf:type schema:Person
106 sg:person.014627200551.35 schema:affiliation Nc4fea5bfae294a258a72c3b3430a73f6
107 schema:familyName Dumais
108 schema:givenName Susan T.
109 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014627200551.35
110 rdf:type schema:Person
111 sg:person.015143634641.13 schema:affiliation https://www.grid.ac/institutes/grid.266683.f
112 schema:familyName Carterette
113 schema:givenName Ben
114 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015143634641.13
115 rdf:type schema:Person
116 sg:person.015207115744.95 schema:affiliation N1a1fedd8ef974d84a4ea1115846ad268
117 schema:familyName Bennett
118 schema:givenName Paul N.
119 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.015207115744.95
120 rdf:type schema:Person
121 https://doi.org/10.1002/(sici)1097-4571(199012)41:8<590::aid-asi5>3.0.co;2-t schema:sameAs https://app.dimensions.ai/details/publication/pub.1029308691
122 rdf:type schema:CreativeWork
123 https://doi.org/10.1002/0471743984.vse5991 schema:sameAs https://app.dimensions.ai/details/publication/pub.1053090966
124 rdf:type schema:CreativeWork
125 https://doi.org/10.1016/0306-4573(91)90046-o schema:sameAs https://app.dimensions.ai/details/publication/pub.1024588363
126 rdf:type schema:CreativeWork
127 https://doi.org/10.1145/1008992.1009000 schema:sameAs https://app.dimensions.ai/details/publication/pub.1014166097
128 rdf:type schema:CreativeWork
129 https://doi.org/10.1145/1076034.1076063 schema:sameAs https://app.dimensions.ai/details/publication/pub.1011160774
130 rdf:type schema:CreativeWork
131 https://doi.org/10.1145/1102351.1102363 schema:sameAs https://app.dimensions.ai/details/publication/pub.1025364122
132 rdf:type schema:CreativeWork
133 https://doi.org/10.1145/1148170.1148219 schema:sameAs https://app.dimensions.ai/details/publication/pub.1016626314
134 rdf:type schema:CreativeWork
135 https://doi.org/10.1145/1148170.1148289 schema:sameAs https://app.dimensions.ai/details/publication/pub.1006394966
136 rdf:type schema:CreativeWork
137 https://doi.org/10.1145/290941.291017 schema:sameAs https://app.dimensions.ai/details/publication/pub.1022973258
138 rdf:type schema:CreativeWork
139 https://doi.org/10.1145/345508.345545 schema:sameAs https://app.dimensions.ai/details/publication/pub.1015128878
140 rdf:type schema:CreativeWork
141 https://doi.org/10.1145/775047.775067 schema:sameAs https://app.dimensions.ai/details/publication/pub.1002194678
142 rdf:type schema:CreativeWork
143 https://www.grid.ac/institutes/grid.266683.f schema:alternateName University of Massachusetts Amherst
144 schema:name University of Massachusetts Amherst,
145 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...