Realizing Impact Sourcing by Adaptive Gold Questions: A Socially Responsible Measure for Workers’ Trustworthiness View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2015-06-06

AUTHORS

Kinda El Maarry , Ulrich Güntzer , Wolf-Tilo Balke

ABSTRACT

In recent years, crowd sourcing has emerged as a good solution for digitizing voluminous tasks. What’s more, it offers a social solution promising to extend economic opportunities to low-income countries, alleviating the welfare of poor, honest and yet uneducated labor. On the other hand, crowd sourcing’s virtual nature and anonymity encourages fraudulent workers to misuse the service for quick and easy monetary gain. This in turn compromises the quality of results, and forces task providers to employ strict control measures like gold questions or majority voting, which may gravely misjudge honest workers with lower skills, ultimately discarding them from the labor pool. Thus, the problem of fairly distinguishing between fraudulent and honest workers lacking educational skills becomes vital for supporting the vision of Impact Sourcing and its social responsibility. We develop a technique with socially responsible gold questions as an objective measure of workers’ trustworthiness, rather than a mere discarding mechanism. Our statistical model aligns workers’ skill levels and questions’ difficulty levels, which then allows adapting the gold questions’ difficulty for a fair judgment. Moreover, we illustrate how low-skilled workers’ initial payloads, which are usually discarded along with the worker, can be partially recovered for an increased economic gain, and show how low-skilled workers can be seamlessly integrated into high-performing teams. Our experiments prove that about 75% of misjudged workers can be correctly identified and effectively be integrated into teams with high overall result correctness between 70-95%. More... »

PAGES

17-29

Book

TITLE

Web-Age Information Management

ISBN

978-3-319-21041-4
978-3-319-21042-1

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-21042-1_2

DOI

http://dx.doi.org/10.1007/978-3-319-21042-1_2

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1006104075


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/14", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Economics", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1402", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Applied Economics", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "IFIS, TU Braunschweig, Braunschweig, Germany", 
          "id": "http://www.grid.ac/institutes/grid.6738.a", 
          "name": [
            "IFIS, TU Braunschweig, Braunschweig, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Maarry", 
        "givenName": "Kinda El", 
        "id": "sg:person.010100714373.68", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010100714373.68"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Inst. f. Informatik, Universit\u00e4t T\u00fcbingen, T\u00fcbingen, Germany", 
          "id": "http://www.grid.ac/institutes/grid.10392.39", 
          "name": [
            "Inst. f. Informatik, Universit\u00e4t T\u00fcbingen, T\u00fcbingen, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "G\u00fcntzer", 
        "givenName": "Ulrich", 
        "id": "sg:person.013324511711.75", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013324511711.75"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "IFIS, TU Braunschweig, Braunschweig, Germany", 
          "id": "http://www.grid.ac/institutes/grid.6738.a", 
          "name": [
            "IFIS, TU Braunschweig, Braunschweig, Germany"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Balke", 
        "givenName": "Wolf-Tilo", 
        "id": "sg:person.014313642615.12", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014313642615.12"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2015-06-06", 
    "datePublishedReg": "2015-06-06", 
    "description": "In recent years, crowd sourcing has emerged as a good solution for digitizing voluminous tasks. What\u2019s more, it offers a social solution promising to extend economic opportunities to low-income countries, alleviating the welfare of poor, honest and yet uneducated labor. On the other hand, crowd sourcing\u2019s virtual nature and anonymity encourages fraudulent workers to misuse the service for quick and easy monetary gain. This in turn compromises the quality of results, and forces task providers to employ strict control measures like gold questions or majority voting, which may gravely misjudge honest workers with lower skills, ultimately discarding them from the labor pool. Thus, the problem of fairly distinguishing between fraudulent and honest workers lacking educational skills becomes vital for supporting the vision of Impact Sourcing and its social responsibility. We develop a technique with socially responsible gold questions as an objective measure of workers\u2019 trustworthiness, rather than a mere discarding mechanism. Our statistical model aligns workers\u2019 skill levels and questions\u2019 difficulty levels, which then allows adapting the gold questions\u2019 difficulty for a fair judgment. Moreover, we illustrate how low-skilled workers\u2019 initial payloads, which are usually discarded along with the worker, can be partially recovered for an increased economic gain, and show how low-skilled workers can be seamlessly integrated into high-performing teams. Our experiments prove that about 75% of misjudged workers can be correctly identified and effectively be integrated into teams with high overall result correctness between 70-95%.", 
    "editor": [
      {
        "familyName": "Dong", 
        "givenName": "Xin Luna", 
        "type": "Person"
      }, 
      {
        "familyName": "Yu", 
        "givenName": "Xiaohui", 
        "type": "Person"
      }, 
      {
        "familyName": "Li", 
        "givenName": "Jian", 
        "type": "Person"
      }, 
      {
        "familyName": "Sun", 
        "givenName": "Yizhou", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-21042-1_2", 
    "inLanguage": "en", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-319-21041-4", 
        "978-3-319-21042-1"
      ], 
      "name": "Web-Age Information Management", 
      "type": "Book"
    }, 
    "keywords": [
      "gold questions", 
      "initial payload", 
      "low-skilled workers", 
      "honest workers", 
      "worker skill levels", 
      "low-income countries", 
      "uneducated labor", 
      "skill level", 
      "economic gains", 
      "voluminous tasks", 
      "fraudulent workers", 
      "Impact Sourcing", 
      "economic opportunities", 
      "responsible measures", 
      "labor pool", 
      "task provider", 
      "monetary gains", 
      "low skill", 
      "majority voting", 
      "Crowd Sourcing", 
      "workers", 
      "discarding mechanisms", 
      "sourcing", 
      "welfare", 
      "social solutions", 
      "countries", 
      "measures", 
      "social responsibility", 
      "labor", 
      "statistical model", 
      "questions", 
      "recent years", 
      "voting", 
      "educational skills", 
      "gain", 
      "result correctness", 
      "fair judgment", 
      "impact", 
      "services", 
      "opportunities", 
      "providers", 
      "objective measures", 
      "crowd", 
      "trustworthiness", 
      "model", 
      "responsibility", 
      "levels", 
      "best solution", 
      "skills", 
      "turn", 
      "virtual nature", 
      "judgments", 
      "strict control measures", 
      "control measures", 
      "hand", 
      "quality", 
      "nature", 
      "problem", 
      "years", 
      "difficulties", 
      "results", 
      "anonymity", 
      "quality of results", 
      "pool", 
      "solution", 
      "task", 
      "vision", 
      "mechanism", 
      "team", 
      "technique", 
      "difficulty level", 
      "experiments", 
      "correctness", 
      "payload", 
      "easy monetary gain", 
      "forces task providers", 
      "responsible gold questions", 
      "mere discarding mechanism", 
      "low-skilled workers\u2019 initial payloads", 
      "workers\u2019 initial payloads", 
      "high overall result correctness", 
      "overall result correctness", 
      "Adaptive Gold Questions"
    ], 
    "name": "Realizing Impact Sourcing by Adaptive Gold Questions: A Socially Responsible Measure for Workers\u2019 Trustworthiness", 
    "pagination": "17-29", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1006104075"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-21042-1_2"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-21042-1_2", 
      "https://app.dimensions.ai/details/publication/pub.1006104075"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-01-01T19:15", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220101/entities/gbq_results/chapter/chapter_263.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/978-3-319-21042-1_2"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-21042-1_2'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-21042-1_2'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-21042-1_2'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-21042-1_2'


 

This table displays all metadata directly associated to this object as RDF triples.

175 TRIPLES      23 PREDICATES      108 URIs      101 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-21042-1_2 schema:about anzsrc-for:14
2 anzsrc-for:1402
3 schema:author N6c14ce8df9644b1dac5de177b0925633
4 schema:datePublished 2015-06-06
5 schema:datePublishedReg 2015-06-06
6 schema:description In recent years, crowd sourcing has emerged as a good solution for digitizing voluminous tasks. What’s more, it offers a social solution promising to extend economic opportunities to low-income countries, alleviating the welfare of poor, honest and yet uneducated labor. On the other hand, crowd sourcing’s virtual nature and anonymity encourages fraudulent workers to misuse the service for quick and easy monetary gain. This in turn compromises the quality of results, and forces task providers to employ strict control measures like gold questions or majority voting, which may gravely misjudge honest workers with lower skills, ultimately discarding them from the labor pool. Thus, the problem of fairly distinguishing between fraudulent and honest workers lacking educational skills becomes vital for supporting the vision of Impact Sourcing and its social responsibility. We develop a technique with socially responsible gold questions as an objective measure of workers’ trustworthiness, rather than a mere discarding mechanism. Our statistical model aligns workers’ skill levels and questions’ difficulty levels, which then allows adapting the gold questions’ difficulty for a fair judgment. Moreover, we illustrate how low-skilled workers’ initial payloads, which are usually discarded along with the worker, can be partially recovered for an increased economic gain, and show how low-skilled workers can be seamlessly integrated into high-performing teams. Our experiments prove that about 75% of misjudged workers can be correctly identified and effectively be integrated into teams with high overall result correctness between 70-95%.
7 schema:editor N978571ffd2e646089783dd8b80d8397b
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree false
11 schema:isPartOf N250be41d5f1d45efbdf510b05c361165
12 schema:keywords Adaptive Gold Questions
13 Crowd Sourcing
14 Impact Sourcing
15 anonymity
16 best solution
17 control measures
18 correctness
19 countries
20 crowd
21 difficulties
22 difficulty level
23 discarding mechanisms
24 easy monetary gain
25 economic gains
26 economic opportunities
27 educational skills
28 experiments
29 fair judgment
30 forces task providers
31 fraudulent workers
32 gain
33 gold questions
34 hand
35 high overall result correctness
36 honest workers
37 impact
38 initial payload
39 judgments
40 labor
41 labor pool
42 levels
43 low skill
44 low-income countries
45 low-skilled workers
46 low-skilled workers’ initial payloads
47 majority voting
48 measures
49 mechanism
50 mere discarding mechanism
51 model
52 monetary gains
53 nature
54 objective measures
55 opportunities
56 overall result correctness
57 payload
58 pool
59 problem
60 providers
61 quality
62 quality of results
63 questions
64 recent years
65 responsibility
66 responsible gold questions
67 responsible measures
68 result correctness
69 results
70 services
71 skill level
72 skills
73 social responsibility
74 social solutions
75 solution
76 sourcing
77 statistical model
78 strict control measures
79 task
80 task provider
81 team
82 technique
83 trustworthiness
84 turn
85 uneducated labor
86 virtual nature
87 vision
88 voluminous tasks
89 voting
90 welfare
91 worker skill levels
92 workers
93 workers’ initial payloads
94 years
95 schema:name Realizing Impact Sourcing by Adaptive Gold Questions: A Socially Responsible Measure for Workers’ Trustworthiness
96 schema:pagination 17-29
97 schema:productId Na01cb205c823483a928c6b0500959c50
98 Na74a3d24237f4d1b8be7871fe9d75af1
99 schema:publisher Nf1152c8c3e2e44359caf64eba51b77d3
100 schema:sameAs https://app.dimensions.ai/details/publication/pub.1006104075
101 https://doi.org/10.1007/978-3-319-21042-1_2
102 schema:sdDatePublished 2022-01-01T19:15
103 schema:sdLicense https://scigraph.springernature.com/explorer/license/
104 schema:sdPublisher N771c9cfcdec54ea8ab2149c7082c4926
105 schema:url https://doi.org/10.1007/978-3-319-21042-1_2
106 sgo:license sg:explorer/license/
107 sgo:sdDataset chapters
108 rdf:type schema:Chapter
109 N241032ea59254f09886794a5a52e4d18 schema:familyName Sun
110 schema:givenName Yizhou
111 rdf:type schema:Person
112 N250be41d5f1d45efbdf510b05c361165 schema:isbn 978-3-319-21041-4
113 978-3-319-21042-1
114 schema:name Web-Age Information Management
115 rdf:type schema:Book
116 N6c14ce8df9644b1dac5de177b0925633 rdf:first sg:person.010100714373.68
117 rdf:rest Na7c4fe7d83cc4f32acf22fe3c8ee0ba1
118 N771c9cfcdec54ea8ab2149c7082c4926 schema:name Springer Nature - SN SciGraph project
119 rdf:type schema:Organization
120 N897d31da4e07421d978d4c646dc6275d rdf:first Nbcf558a1671e4a829b5ceca96fddc1ec
121 rdf:rest Nfbc0a0d4da9e43c696b8caf045ab1bde
122 N978571ffd2e646089783dd8b80d8397b rdf:first Ne47c481b1856471c995b4fce2d738125
123 rdf:rest N897d31da4e07421d978d4c646dc6275d
124 Na01cb205c823483a928c6b0500959c50 schema:name dimensions_id
125 schema:value pub.1006104075
126 rdf:type schema:PropertyValue
127 Na105326c9a3a4b10ae4501b845e30393 schema:familyName Li
128 schema:givenName Jian
129 rdf:type schema:Person
130 Na7347bab52f2477887a96cba28cdd811 rdf:first sg:person.014313642615.12
131 rdf:rest rdf:nil
132 Na74a3d24237f4d1b8be7871fe9d75af1 schema:name doi
133 schema:value 10.1007/978-3-319-21042-1_2
134 rdf:type schema:PropertyValue
135 Na7c4fe7d83cc4f32acf22fe3c8ee0ba1 rdf:first sg:person.013324511711.75
136 rdf:rest Na7347bab52f2477887a96cba28cdd811
137 Nbcf558a1671e4a829b5ceca96fddc1ec schema:familyName Yu
138 schema:givenName Xiaohui
139 rdf:type schema:Person
140 Nc1a84ec8d8a24647b98bc96bfc7a0ab3 rdf:first N241032ea59254f09886794a5a52e4d18
141 rdf:rest rdf:nil
142 Ne47c481b1856471c995b4fce2d738125 schema:familyName Dong
143 schema:givenName Xin Luna
144 rdf:type schema:Person
145 Nf1152c8c3e2e44359caf64eba51b77d3 schema:name Springer Nature
146 rdf:type schema:Organisation
147 Nfbc0a0d4da9e43c696b8caf045ab1bde rdf:first Na105326c9a3a4b10ae4501b845e30393
148 rdf:rest Nc1a84ec8d8a24647b98bc96bfc7a0ab3
149 anzsrc-for:14 schema:inDefinedTermSet anzsrc-for:
150 schema:name Economics
151 rdf:type schema:DefinedTerm
152 anzsrc-for:1402 schema:inDefinedTermSet anzsrc-for:
153 schema:name Applied Economics
154 rdf:type schema:DefinedTerm
155 sg:person.010100714373.68 schema:affiliation grid-institutes:grid.6738.a
156 schema:familyName Maarry
157 schema:givenName Kinda El
158 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010100714373.68
159 rdf:type schema:Person
160 sg:person.013324511711.75 schema:affiliation grid-institutes:grid.10392.39
161 schema:familyName Güntzer
162 schema:givenName Ulrich
163 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.013324511711.75
164 rdf:type schema:Person
165 sg:person.014313642615.12 schema:affiliation grid-institutes:grid.6738.a
166 schema:familyName Balke
167 schema:givenName Wolf-Tilo
168 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014313642615.12
169 rdf:type schema:Person
170 grid-institutes:grid.10392.39 schema:alternateName Inst. f. Informatik, Universität Tübingen, Tübingen, Germany
171 schema:name Inst. f. Informatik, Universität Tübingen, Tübingen, Germany
172 rdf:type schema:Organization
173 grid-institutes:grid.6738.a schema:alternateName IFIS, TU Braunschweig, Braunschweig, Germany
174 schema:name IFIS, TU Braunschweig, Braunschweig, Germany
175 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...