Do simple screening statistical tools help to detect reporting bias? View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

2013-09-02

AUTHORS

Romain Pirracchio, Matthieu Resche-Rigon, Sylvie Chevret, Didier Journois

ABSTRACT

BackgroundAs a result of reporting bias, or frauds, false or misunderstood findings may represent the majority of published research claims. This article provides simple methods that might help to appraise the quality of the reporting of randomized, controlled trials (RCT).MethodsThis evaluation roadmap proposed herein relies on four steps: evaluation of the distribution of the reported variables; evaluation of the distribution of the reported p values; data simulation using parametric bootstrap and explicit computation of the p values. Such an approach was illustrated using published data from a retracted RCT comparing a hydroxyethyl starch versus albumin-based priming for cardiopulmonary bypass.ResultsDespite obvious nonnormal distributions, several variables are presented as if they were normally distributed. The set of 16 p values testing for differences in baseline characteristics across randomized groups did not follow a Uniform distribution on [0,1] (p = 0.045). The p values obtained by explicit computations were different from the results reported by the authors for the two following variables: urine output at 5 hours (calculated p value < 10-6, reported p ≥ 0.05); packed red blood cells (PRBC) during surgery (calculated p value = 0.08; reported p < 0.05). Finally, parametric bootstrap found p value > 0.05 in only 5 of the 10,000 simulated datasets concerning urine output 5 hours after surgery. Concerning PRBC transfused during surgery, parametric bootstrap showed that only the corresponding p value had less than a 50% chance to be inferior to 0.05 (3,920/10,000, p value < 0.05).ConclusionsSuch simple evaluation methods might offer some warning signals. However, it should be emphasized that such methods do not allow concluding to the presence of error or fraud but should rather be used to justify asking for an access to the raw data. More... »

PAGES

29

References to SciGraph publications

Identifiers

URI

http://scigraph.springernature.com/pub.10.1186/2110-5820-3-29

DOI

http://dx.doi.org/10.1186/2110-5820-3-29

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1048530347

PUBMED

https://www.ncbi.nlm.nih.gov/pubmed/24004521


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/11", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Medical and Health Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1103", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Clinical Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1117", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Public Health and Health Services", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Department of Informatics and Biostatistics, Unit\u00e9 Inserm UMR S717, H\u00f4pital Saint Louis, Universit\u00e9 Paris 7 Diderot, Paris, France", 
          "id": "http://www.grid.ac/institutes/None", 
          "name": [
            "Department of Anesthesiology and Critical Care Medicine, H\u00f4pital Europ\u00e9en Georges Pompidou, Universit\u00e9 Paris-Descartes Sorbonne Paris Cit\u00e9, Paris, France", 
            "Department of Informatics and Biostatistics, Unit\u00e9 Inserm UMR S717, H\u00f4pital Saint Louis, Universit\u00e9 Paris 7 Diderot, Paris, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Pirracchio", 
        "givenName": "Romain", 
        "id": "sg:person.01047623565.27", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01047623565.27"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Informatics and Biostatistics, Unit\u00e9 Inserm UMR S717, H\u00f4pital Saint Louis, Universit\u00e9 Paris 7 Diderot, Paris, France", 
          "id": "http://www.grid.ac/institutes/None", 
          "name": [
            "Department of Informatics and Biostatistics, Unit\u00e9 Inserm UMR S717, H\u00f4pital Saint Louis, Universit\u00e9 Paris 7 Diderot, Paris, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Resche-Rigon", 
        "givenName": "Matthieu", 
        "id": "sg:person.01036155166.50", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01036155166.50"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Informatics and Biostatistics, Unit\u00e9 Inserm UMR S717, H\u00f4pital Saint Louis, Universit\u00e9 Paris 7 Diderot, Paris, France", 
          "id": "http://www.grid.ac/institutes/None", 
          "name": [
            "Department of Informatics and Biostatistics, Unit\u00e9 Inserm UMR S717, H\u00f4pital Saint Louis, Universit\u00e9 Paris 7 Diderot, Paris, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Chevret", 
        "givenName": "Sylvie", 
        "id": "sg:person.0625147223.89", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0625147223.89"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Department of Anesthesiology and Critical Care Medicine, H\u00f4pital Europ\u00e9en Georges Pompidou, Universit\u00e9 Paris-Descartes Sorbonne Paris Cit\u00e9, Paris, France", 
          "id": "http://www.grid.ac/institutes/grid.414093.b", 
          "name": [
            "Department of Anesthesiology and Critical Care Medicine, H\u00f4pital Europ\u00e9en Georges Pompidou, Universit\u00e9 Paris-Descartes Sorbonne Paris Cit\u00e9, Paris, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Journois", 
        "givenName": "Didier", 
        "id": "sg:person.01171372357.44", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01171372357.44"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "sg:pub.10.1186/cc12611", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1001588772", 
          "https://doi.org/10.1186/cc12611"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2013-09-02", 
    "datePublishedReg": "2013-09-02", 
    "description": "BackgroundAs a result of reporting bias, or frauds, false or misunderstood findings may represent the majority of published research claims. This article provides simple methods that might help to appraise the quality of the reporting of randomized, controlled trials (RCT).MethodsThis evaluation roadmap proposed herein relies on four steps: evaluation of the distribution of the reported variables; evaluation of the distribution of the reported p values; data simulation using parametric bootstrap and explicit computation of the p values. Such an approach was illustrated using published data from a retracted RCT comparing a hydroxyethyl starch versus albumin-based priming for cardiopulmonary bypass.ResultsDespite obvious nonnormal distributions, several variables are presented as if they were normally distributed. The set of 16 p values testing for differences in baseline characteristics across randomized groups did not follow a Uniform distribution on [0,1] (p = 0.045). The p values obtained by explicit computations were different from the results reported by the authors for the two following variables: urine output at 5 hours (calculated p value < 10-6, reported p \u2265 0.05); packed red blood cells (PRBC) during surgery (calculated p value = 0.08; reported p < 0.05). Finally, parametric bootstrap found p value > 0.05 in only 5 of the 10,000 simulated datasets concerning urine output 5 hours after surgery. Concerning PRBC transfused during surgery, parametric bootstrap showed that only the corresponding p value had less than a 50% chance to be inferior to 0.05 (3,920/10,000, p value < 0.05).ConclusionsSuch simple evaluation methods might offer some warning signals. However, it should be emphasized that such methods do not allow concluding to the presence of error or fraud but should rather be used to justify asking for an access to the raw data.", 
    "genre": "article", 
    "id": "sg:pub.10.1186/2110-5820-3-29", 
    "isAccessibleForFree": true, 
    "isPartOf": [
      {
        "id": "sg:journal.1045300", 
        "issn": [
          "2110-5820"
        ], 
        "name": "Annals of Intensive Care", 
        "publisher": "Springer Nature", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "1", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "3"
      }
    ], 
    "keywords": [
      "p-value", 
      "baseline characteristics", 
      "urine output", 
      "red blood cells", 
      "cardiopulmonary bypass", 
      "randomized groups", 
      "hydroxyethyl starch", 
      "surgery", 
      "blood cells", 
      "hours", 
      "RCTs", 
      "PRBC", 
      "bypass", 
      "trials", 
      "evaluation", 
      "albumin", 
      "priming", 
      "cells", 
      "group", 
      "majority", 
      "findings", 
      "reporting", 
      "variables", 
      "data", 
      "differences", 
      "simple evaluation method", 
      "warning signals", 
      "chance", 
      "bias", 
      "values", 
      "presence", 
      "results", 
      "access", 
      "method", 
      "quality", 
      "authors", 
      "characteristics", 
      "distribution", 
      "tool", 
      "nonnormal distributions", 
      "simple method", 
      "evaluation method", 
      "article", 
      "output", 
      "approach", 
      "evaluation roadmap", 
      "raw data", 
      "signals", 
      "claims", 
      "step", 
      "roadmap", 
      "statistical tools", 
      "starch", 
      "such methods", 
      "research claims", 
      "dataset", 
      "bootstrap", 
      "error", 
      "parametric bootstrap", 
      "set", 
      "data simulation", 
      "uniform distribution", 
      "presence of errors", 
      "fraud", 
      "simulations", 
      "computation", 
      "explicit computation"
    ], 
    "name": "Do simple screening statistical tools help to detect reporting bias?", 
    "pagination": "29", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1048530347"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1186/2110-5820-3-29"
        ]
      }, 
      {
        "name": "pubmed_id", 
        "type": "PropertyValue", 
        "value": [
          "24004521"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1186/2110-5820-3-29", 
      "https://app.dimensions.ai/details/publication/pub.1048530347"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2022-08-04T17:01", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220804/entities/gbq_results/article/article_593.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "https://doi.org/10.1186/2110-5820-3-29"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1186/2110-5820-3-29'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1186/2110-5820-3-29'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1186/2110-5820-3-29'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1186/2110-5820-3-29'


 

This table displays all metadata directly associated to this object as RDF triples.

160 TRIPLES      21 PREDICATES      93 URIs      83 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1186/2110-5820-3-29 schema:about anzsrc-for:11
2 anzsrc-for:1103
3 anzsrc-for:1117
4 schema:author Na74a6fb5841443899c8f0671748ad079
5 schema:citation sg:pub.10.1186/cc12611
6 schema:datePublished 2013-09-02
7 schema:datePublishedReg 2013-09-02
8 schema:description BackgroundAs a result of reporting bias, or frauds, false or misunderstood findings may represent the majority of published research claims. This article provides simple methods that might help to appraise the quality of the reporting of randomized, controlled trials (RCT).MethodsThis evaluation roadmap proposed herein relies on four steps: evaluation of the distribution of the reported variables; evaluation of the distribution of the reported p values; data simulation using parametric bootstrap and explicit computation of the p values. Such an approach was illustrated using published data from a retracted RCT comparing a hydroxyethyl starch versus albumin-based priming for cardiopulmonary bypass.ResultsDespite obvious nonnormal distributions, several variables are presented as if they were normally distributed. The set of 16 p values testing for differences in baseline characteristics across randomized groups did not follow a Uniform distribution on [0,1] (p = 0.045). The p values obtained by explicit computations were different from the results reported by the authors for the two following variables: urine output at 5 hours (calculated p value < 10-6, reported p ≥ 0.05); packed red blood cells (PRBC) during surgery (calculated p value = 0.08; reported p < 0.05). Finally, parametric bootstrap found p value > 0.05 in only 5 of the 10,000 simulated datasets concerning urine output 5 hours after surgery. Concerning PRBC transfused during surgery, parametric bootstrap showed that only the corresponding p value had less than a 50% chance to be inferior to 0.05 (3,920/10,000, p value < 0.05).ConclusionsSuch simple evaluation methods might offer some warning signals. However, it should be emphasized that such methods do not allow concluding to the presence of error or fraud but should rather be used to justify asking for an access to the raw data.
9 schema:genre article
10 schema:isAccessibleForFree true
11 schema:isPartOf N0e1b3010481642dfb51a318f2b4c0846
12 N4f3bc9572a974b02870096ef4c08ef22
13 sg:journal.1045300
14 schema:keywords PRBC
15 RCTs
16 access
17 albumin
18 approach
19 article
20 authors
21 baseline characteristics
22 bias
23 blood cells
24 bootstrap
25 bypass
26 cardiopulmonary bypass
27 cells
28 chance
29 characteristics
30 claims
31 computation
32 data
33 data simulation
34 dataset
35 differences
36 distribution
37 error
38 evaluation
39 evaluation method
40 evaluation roadmap
41 explicit computation
42 findings
43 fraud
44 group
45 hours
46 hydroxyethyl starch
47 majority
48 method
49 nonnormal distributions
50 output
51 p-value
52 parametric bootstrap
53 presence
54 presence of errors
55 priming
56 quality
57 randomized groups
58 raw data
59 red blood cells
60 reporting
61 research claims
62 results
63 roadmap
64 set
65 signals
66 simple evaluation method
67 simple method
68 simulations
69 starch
70 statistical tools
71 step
72 such methods
73 surgery
74 tool
75 trials
76 uniform distribution
77 urine output
78 values
79 variables
80 warning signals
81 schema:name Do simple screening statistical tools help to detect reporting bias?
82 schema:pagination 29
83 schema:productId N1f4eaedd20894e1f93780858e0d66c05
84 N28a4b9d340104fb08c23554c73455b8c
85 Nac303905700943ddb7f9f9ae3b60f253
86 schema:sameAs https://app.dimensions.ai/details/publication/pub.1048530347
87 https://doi.org/10.1186/2110-5820-3-29
88 schema:sdDatePublished 2022-08-04T17:01
89 schema:sdLicense https://scigraph.springernature.com/explorer/license/
90 schema:sdPublisher N75c998bcd8b446c18455412275ccf32f
91 schema:url https://doi.org/10.1186/2110-5820-3-29
92 sgo:license sg:explorer/license/
93 sgo:sdDataset articles
94 rdf:type schema:ScholarlyArticle
95 N0e1b3010481642dfb51a318f2b4c0846 schema:volumeNumber 3
96 rdf:type schema:PublicationVolume
97 N1f4eaedd20894e1f93780858e0d66c05 schema:name dimensions_id
98 schema:value pub.1048530347
99 rdf:type schema:PropertyValue
100 N224ced579aca4b54bea5de105a737baf rdf:first sg:person.01036155166.50
101 rdf:rest N3ee4841d82aa4a55b2df71d59509df04
102 N28a4b9d340104fb08c23554c73455b8c schema:name doi
103 schema:value 10.1186/2110-5820-3-29
104 rdf:type schema:PropertyValue
105 N3ee4841d82aa4a55b2df71d59509df04 rdf:first sg:person.0625147223.89
106 rdf:rest N4858d438cc8842dfa3a7ff85862c04b4
107 N4858d438cc8842dfa3a7ff85862c04b4 rdf:first sg:person.01171372357.44
108 rdf:rest rdf:nil
109 N4f3bc9572a974b02870096ef4c08ef22 schema:issueNumber 1
110 rdf:type schema:PublicationIssue
111 N75c998bcd8b446c18455412275ccf32f schema:name Springer Nature - SN SciGraph project
112 rdf:type schema:Organization
113 Na74a6fb5841443899c8f0671748ad079 rdf:first sg:person.01047623565.27
114 rdf:rest N224ced579aca4b54bea5de105a737baf
115 Nac303905700943ddb7f9f9ae3b60f253 schema:name pubmed_id
116 schema:value 24004521
117 rdf:type schema:PropertyValue
118 anzsrc-for:11 schema:inDefinedTermSet anzsrc-for:
119 schema:name Medical and Health Sciences
120 rdf:type schema:DefinedTerm
121 anzsrc-for:1103 schema:inDefinedTermSet anzsrc-for:
122 schema:name Clinical Sciences
123 rdf:type schema:DefinedTerm
124 anzsrc-for:1117 schema:inDefinedTermSet anzsrc-for:
125 schema:name Public Health and Health Services
126 rdf:type schema:DefinedTerm
127 sg:journal.1045300 schema:issn 2110-5820
128 schema:name Annals of Intensive Care
129 schema:publisher Springer Nature
130 rdf:type schema:Periodical
131 sg:person.01036155166.50 schema:affiliation grid-institutes:None
132 schema:familyName Resche-Rigon
133 schema:givenName Matthieu
134 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01036155166.50
135 rdf:type schema:Person
136 sg:person.01047623565.27 schema:affiliation grid-institutes:None
137 schema:familyName Pirracchio
138 schema:givenName Romain
139 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01047623565.27
140 rdf:type schema:Person
141 sg:person.01171372357.44 schema:affiliation grid-institutes:grid.414093.b
142 schema:familyName Journois
143 schema:givenName Didier
144 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01171372357.44
145 rdf:type schema:Person
146 sg:person.0625147223.89 schema:affiliation grid-institutes:None
147 schema:familyName Chevret
148 schema:givenName Sylvie
149 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0625147223.89
150 rdf:type schema:Person
151 sg:pub.10.1186/cc12611 schema:sameAs https://app.dimensions.ai/details/publication/pub.1001588772
152 https://doi.org/10.1186/cc12611
153 rdf:type schema:CreativeWork
154 grid-institutes:None schema:alternateName Department of Informatics and Biostatistics, Unité Inserm UMR S717, Hôpital Saint Louis, Université Paris 7 Diderot, Paris, France
155 schema:name Department of Anesthesiology and Critical Care Medicine, Hôpital Européen Georges Pompidou, Université Paris-Descartes Sorbonne Paris Cité, Paris, France
156 Department of Informatics and Biostatistics, Unité Inserm UMR S717, Hôpital Saint Louis, Université Paris 7 Diderot, Paris, France
157 rdf:type schema:Organization
158 grid-institutes:grid.414093.b schema:alternateName Department of Anesthesiology and Critical Care Medicine, Hôpital Européen Georges Pompidou, Université Paris-Descartes Sorbonne Paris Cité, Paris, France
159 schema:name Department of Anesthesiology and Critical Care Medicine, Hôpital Européen Georges Pompidou, Université Paris-Descartes Sorbonne Paris Cité, Paris, France
160 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...