Systematic analysis of agreement between metrics and peer review in the UK REF View Full Text


Ontology type: schema:ScholarlyArticle     


Article Info

DATE

2019-12

AUTHORS

V. A. Traag, L. Waltman

ABSTRACT

When performing a national research assessment, some countries rely on citation metrics whereas others, such as the UK, primarily use peer review. In the influential Metric Tide report, a low agreement between metrics and peer review in the UK Research Excellence Framework (REF) was found. However, earlier studies observed much higher agreement between metrics and peer review in the REF and argued in favour of using metrics. This shows that there is considerable ambiguity in the discussion on agreement between metrics and peer review. We provide clarity in this discussion by considering four important points: (1) the level of aggregation of the analysis; (2) the use of either a size-dependent or a size-independent perspective; (3) the suitability of different measures of agreement; and (4) the uncertainty in peer review. In the context of the REF, we argue that agreement between metrics and peer review should be assessed at the institutional level rather than at the publication level. Both a size-dependent and a size-independent perspective are relevant in the REF. The interpretation of correlations may be problematic and as an alternative we therefore use measures of agreement that are based on the absolute or relative differences between metrics and peer review. To get an idea of the uncertainty in peer review, we rely on a model to bootstrap peer review outcomes. We conclude that particularly in Physics, Clinical Medicine, and Public Health, metrics agree relatively well with peer review and may offer an alternative to peer review. More... »

PAGES

29

Identifiers

URI

http://scigraph.springernature.com/pub.10.1057/s41599-019-0233-x

DOI

http://dx.doi.org/10.1057/s41599-019-0233-x

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1112706995


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/1605", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Policy and Administration", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/16", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Studies in Human Society", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Leiden University", 
          "id": "https://www.grid.ac/institutes/grid.5132.5", 
          "name": [
            "Centre for Science and Technology Studies (CWTS), Leiden University, Leiden, The Netherlands"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Traag", 
        "givenName": "V. A.", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Leiden University", 
          "id": "https://www.grid.ac/institutes/grid.5132.5", 
          "name": [
            "Centre for Science and Technology Studies (CWTS), Leiden University, Leiden, The Netherlands"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Waltman", 
        "givenName": "L.", 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.1108/eum0000000007207", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1003147541"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/j.1467-8551.2010.00722.x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1003806183"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/j.1467-954x.2012.02053.x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1007603484"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/j.1467-954x.2012.02053.x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1007603484"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s11192-012-0874-7", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1008690909", 
          "https://doi.org/10.1007/s11192-012-0874-7"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.joi.2014.11.010", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1009981400"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1108/00220410310698734", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1011200668"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s11192-015-1567-9", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1012857302", 
          "https://doi.org/10.1007/s11192-015-1567-9"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1080/00220620500211189", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1013629377"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02459050", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1013748721", 
          "https://doi.org/10.1007/bf02459050"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02459050", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1013748721", 
          "https://doi.org/10.1007/bf02459050"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/j.1478-9299.2008.00167.x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1015700222"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/j.1478-9299.2008.00167.x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1015700222"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02020188", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1018020070", 
          "https://doi.org/10.1007/bf02020188"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02020188", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1018020070", 
          "https://doi.org/10.1007/bf02020188"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1136/bmjopen-2015-010916", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1021544275"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1098/rsos.160384", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1022320446"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1371/journal.pone.0132990", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1023638259"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s11192-016-2185-x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1024659616", 
          "https://doi.org/10.1007/s11192-016-2185-x"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s11192-016-2185-x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1024659616", 
          "https://doi.org/10.1007/s11192-016-2185-x"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.respol.2014.08.004", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1025100457"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s11192-014-1512-3", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1027890690", 
          "https://doi.org/10.1007/s11192-014-1512-3"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/j.1467-9515.2011.00824.x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1028453339"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02020076", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1034915524", 
          "https://doi.org/10.1007/bf02020076"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02020076", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1034915524", 
          "https://doi.org/10.1007/bf02020076"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1098/rspl.1896.0076", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1036679935"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1002/asi.22748", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1036746088"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.respol.2011.09.007", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1038742987"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s11192-013-1058-9", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1043016659", 
          "https://doi.org/10.1007/s11192-013-1058-9"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1108/eb026940", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1043531532"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/1467-856x.12006", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1045056270"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/1467-856x.12006", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1045056270"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.joi.2009.11.001", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1046858087"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02020077", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1047385098", 
          "https://doi.org/10.1007/bf02020077"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02020077", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1047385098", 
          "https://doi.org/10.1007/bf02020077"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s11192-007-1832-7", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1047404799", 
          "https://doi.org/10.1007/s11192-007-1832-7"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/j.1467-8551.1994.tb00079.x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1048113776"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1111/j.1468-0335.2009.00837.x", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1049485646"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1093/reseval/rvv038", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1060004526"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1177/016555159502100109", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1063752058"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1177/016555159502100109", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1063752058"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.2307/2982732", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1070160606"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.2307/2982732", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1070160606"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1371/journal.pone.0173152", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1084295409"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.5860/crln.68.5.7804", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1087307504"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.2139/ssrn.1154053", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1102252156"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1093/scipol/scy041", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1103614798"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-030-00066-0_17", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1106624975", 
          "https://doi.org/10.1007/978-3-030-00066-0_17"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-319-63627-6", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1109105930", 
          "https://doi.org/10.1007/978-3-319-63627-6"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-319-63627-6", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1109105930", 
          "https://doi.org/10.1007/978-3-319-63627-6"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2019-12", 
    "datePublishedReg": "2019-12-01", 
    "description": "When performing a national research assessment, some countries rely on citation metrics whereas others, such as the UK, primarily use peer review. In the influential Metric Tide report, a low agreement between metrics and peer review in the UK Research Excellence Framework (REF) was found. However, earlier studies observed much higher agreement between metrics and peer review in the REF and argued in favour of using metrics. This shows that there is considerable ambiguity in the discussion on agreement between metrics and peer review. We provide clarity in this discussion by considering four important points: (1) the level of aggregation of the analysis; (2) the use of either a size-dependent or a size-independent perspective; (3) the suitability of different measures of agreement; and (4) the uncertainty in peer review. In the context of the REF, we argue that agreement between metrics and peer review should be assessed at the institutional level rather than at the publication level. Both a size-dependent and a size-independent perspective are relevant in the REF. The interpretation of correlations may be problematic and as an alternative we therefore use measures of agreement that are based on the absolute or relative differences between metrics and peer review. To get an idea of the uncertainty in peer review, we rely on a model to bootstrap peer review outcomes. We conclude that particularly in Physics, Clinical Medicine, and Public Health, metrics agree relatively well with peer review and may offer an alternative to peer review.", 
    "genre": "research_article", 
    "id": "sg:pub.10.1057/s41599-019-0233-x", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": false, 
    "isPartOf": [
      {
        "id": "sg:journal.1136613", 
        "issn": [
          "2055-1045"
        ], 
        "name": "Palgrave Communications", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "1", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "5"
      }
    ], 
    "name": "Systematic analysis of agreement between metrics and peer review in the UK REF", 
    "pagination": "29", 
    "productId": [
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "de8ce2badffd571184d0e3c7e56f551ff807b071a2b580531b7e8ad8a7f60dee"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1057/s41599-019-0233-x"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1112706995"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1057/s41599-019-0233-x", 
      "https://app.dimensions.ai/details/publication/pub.1112706995"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2019-04-11T11:44", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000358_0000000358/records_127453_00000011.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "https://www.nature.com/articles/s41599-019-0233-x"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1057/s41599-019-0233-x'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1057/s41599-019-0233-x'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1057/s41599-019-0233-x'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1057/s41599-019-0233-x'


 

This table displays all metadata directly associated to this object as RDF triples.

194 TRIPLES      21 PREDICATES      66 URIs      19 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1057/s41599-019-0233-x schema:about anzsrc-for:16
2 anzsrc-for:1605
3 schema:author Ne3269696d5e44c3898b3f5f92179db2c
4 schema:citation sg:pub.10.1007/978-3-030-00066-0_17
5 sg:pub.10.1007/978-3-319-63627-6
6 sg:pub.10.1007/bf02020076
7 sg:pub.10.1007/bf02020077
8 sg:pub.10.1007/bf02020188
9 sg:pub.10.1007/bf02459050
10 sg:pub.10.1007/s11192-007-1832-7
11 sg:pub.10.1007/s11192-012-0874-7
12 sg:pub.10.1007/s11192-013-1058-9
13 sg:pub.10.1007/s11192-014-1512-3
14 sg:pub.10.1007/s11192-015-1567-9
15 sg:pub.10.1007/s11192-016-2185-x
16 https://doi.org/10.1002/asi.22748
17 https://doi.org/10.1016/j.joi.2009.11.001
18 https://doi.org/10.1016/j.joi.2014.11.010
19 https://doi.org/10.1016/j.respol.2011.09.007
20 https://doi.org/10.1016/j.respol.2014.08.004
21 https://doi.org/10.1080/00220620500211189
22 https://doi.org/10.1093/reseval/rvv038
23 https://doi.org/10.1093/scipol/scy041
24 https://doi.org/10.1098/rsos.160384
25 https://doi.org/10.1098/rspl.1896.0076
26 https://doi.org/10.1108/00220410310698734
27 https://doi.org/10.1108/eb026940
28 https://doi.org/10.1108/eum0000000007207
29 https://doi.org/10.1111/1467-856x.12006
30 https://doi.org/10.1111/j.1467-8551.1994.tb00079.x
31 https://doi.org/10.1111/j.1467-8551.2010.00722.x
32 https://doi.org/10.1111/j.1467-9515.2011.00824.x
33 https://doi.org/10.1111/j.1467-954x.2012.02053.x
34 https://doi.org/10.1111/j.1468-0335.2009.00837.x
35 https://doi.org/10.1111/j.1478-9299.2008.00167.x
36 https://doi.org/10.1136/bmjopen-2015-010916
37 https://doi.org/10.1177/016555159502100109
38 https://doi.org/10.1371/journal.pone.0132990
39 https://doi.org/10.1371/journal.pone.0173152
40 https://doi.org/10.2139/ssrn.1154053
41 https://doi.org/10.2307/2982732
42 https://doi.org/10.5860/crln.68.5.7804
43 schema:datePublished 2019-12
44 schema:datePublishedReg 2019-12-01
45 schema:description When performing a national research assessment, some countries rely on citation metrics whereas others, such as the UK, primarily use peer review. In the influential Metric Tide report, a low agreement between metrics and peer review in the UK Research Excellence Framework (REF) was found. However, earlier studies observed much higher agreement between metrics and peer review in the REF and argued in favour of using metrics. This shows that there is considerable ambiguity in the discussion on agreement between metrics and peer review. We provide clarity in this discussion by considering four important points: (1) the level of aggregation of the analysis; (2) the use of either a size-dependent or a size-independent perspective; (3) the suitability of different measures of agreement; and (4) the uncertainty in peer review. In the context of the REF, we argue that agreement between metrics and peer review should be assessed at the institutional level rather than at the publication level. Both a size-dependent and a size-independent perspective are relevant in the REF. The interpretation of correlations may be problematic and as an alternative we therefore use measures of agreement that are based on the absolute or relative differences between metrics and peer review. To get an idea of the uncertainty in peer review, we rely on a model to bootstrap peer review outcomes. We conclude that particularly in Physics, Clinical Medicine, and Public Health, metrics agree relatively well with peer review and may offer an alternative to peer review.
46 schema:genre research_article
47 schema:inLanguage en
48 schema:isAccessibleForFree false
49 schema:isPartOf N3053a28e1fac45f587ac1a8f91d4c0d4
50 Nf28005315b794f158d463e790ddfc1e3
51 sg:journal.1136613
52 schema:name Systematic analysis of agreement between metrics and peer review in the UK REF
53 schema:pagination 29
54 schema:productId N074ae68332a34843bb8e9be26f8cf2d8
55 Na3321ebe916940398d362afc1dec31b4
56 Nea93cd9051f247899c1600fdaf704ae1
57 schema:sameAs https://app.dimensions.ai/details/publication/pub.1112706995
58 https://doi.org/10.1057/s41599-019-0233-x
59 schema:sdDatePublished 2019-04-11T11:44
60 schema:sdLicense https://scigraph.springernature.com/explorer/license/
61 schema:sdPublisher N2df62b911a1b4189b1a6fab89f02a8a8
62 schema:url https://www.nature.com/articles/s41599-019-0233-x
63 sgo:license sg:explorer/license/
64 sgo:sdDataset articles
65 rdf:type schema:ScholarlyArticle
66 N074ae68332a34843bb8e9be26f8cf2d8 schema:name doi
67 schema:value 10.1057/s41599-019-0233-x
68 rdf:type schema:PropertyValue
69 N2523fe10fd1343ca94faad675a8c9988 rdf:first N7210db6fb7b240d88e0cd66746abad46
70 rdf:rest rdf:nil
71 N2df62b911a1b4189b1a6fab89f02a8a8 schema:name Springer Nature - SN SciGraph project
72 rdf:type schema:Organization
73 N3053a28e1fac45f587ac1a8f91d4c0d4 schema:issueNumber 1
74 rdf:type schema:PublicationIssue
75 N7210db6fb7b240d88e0cd66746abad46 schema:affiliation https://www.grid.ac/institutes/grid.5132.5
76 schema:familyName Waltman
77 schema:givenName L.
78 rdf:type schema:Person
79 Na3321ebe916940398d362afc1dec31b4 schema:name readcube_id
80 schema:value de8ce2badffd571184d0e3c7e56f551ff807b071a2b580531b7e8ad8a7f60dee
81 rdf:type schema:PropertyValue
82 Nb7da8cfcf4214b34b38377ee0b5814c5 schema:affiliation https://www.grid.ac/institutes/grid.5132.5
83 schema:familyName Traag
84 schema:givenName V. A.
85 rdf:type schema:Person
86 Ne3269696d5e44c3898b3f5f92179db2c rdf:first Nb7da8cfcf4214b34b38377ee0b5814c5
87 rdf:rest N2523fe10fd1343ca94faad675a8c9988
88 Nea93cd9051f247899c1600fdaf704ae1 schema:name dimensions_id
89 schema:value pub.1112706995
90 rdf:type schema:PropertyValue
91 Nf28005315b794f158d463e790ddfc1e3 schema:volumeNumber 5
92 rdf:type schema:PublicationVolume
93 anzsrc-for:16 schema:inDefinedTermSet anzsrc-for:
94 schema:name Studies in Human Society
95 rdf:type schema:DefinedTerm
96 anzsrc-for:1605 schema:inDefinedTermSet anzsrc-for:
97 schema:name Policy and Administration
98 rdf:type schema:DefinedTerm
99 sg:journal.1136613 schema:issn 2055-1045
100 schema:name Palgrave Communications
101 rdf:type schema:Periodical
102 sg:pub.10.1007/978-3-030-00066-0_17 schema:sameAs https://app.dimensions.ai/details/publication/pub.1106624975
103 https://doi.org/10.1007/978-3-030-00066-0_17
104 rdf:type schema:CreativeWork
105 sg:pub.10.1007/978-3-319-63627-6 schema:sameAs https://app.dimensions.ai/details/publication/pub.1109105930
106 https://doi.org/10.1007/978-3-319-63627-6
107 rdf:type schema:CreativeWork
108 sg:pub.10.1007/bf02020076 schema:sameAs https://app.dimensions.ai/details/publication/pub.1034915524
109 https://doi.org/10.1007/bf02020076
110 rdf:type schema:CreativeWork
111 sg:pub.10.1007/bf02020077 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047385098
112 https://doi.org/10.1007/bf02020077
113 rdf:type schema:CreativeWork
114 sg:pub.10.1007/bf02020188 schema:sameAs https://app.dimensions.ai/details/publication/pub.1018020070
115 https://doi.org/10.1007/bf02020188
116 rdf:type schema:CreativeWork
117 sg:pub.10.1007/bf02459050 schema:sameAs https://app.dimensions.ai/details/publication/pub.1013748721
118 https://doi.org/10.1007/bf02459050
119 rdf:type schema:CreativeWork
120 sg:pub.10.1007/s11192-007-1832-7 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047404799
121 https://doi.org/10.1007/s11192-007-1832-7
122 rdf:type schema:CreativeWork
123 sg:pub.10.1007/s11192-012-0874-7 schema:sameAs https://app.dimensions.ai/details/publication/pub.1008690909
124 https://doi.org/10.1007/s11192-012-0874-7
125 rdf:type schema:CreativeWork
126 sg:pub.10.1007/s11192-013-1058-9 schema:sameAs https://app.dimensions.ai/details/publication/pub.1043016659
127 https://doi.org/10.1007/s11192-013-1058-9
128 rdf:type schema:CreativeWork
129 sg:pub.10.1007/s11192-014-1512-3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1027890690
130 https://doi.org/10.1007/s11192-014-1512-3
131 rdf:type schema:CreativeWork
132 sg:pub.10.1007/s11192-015-1567-9 schema:sameAs https://app.dimensions.ai/details/publication/pub.1012857302
133 https://doi.org/10.1007/s11192-015-1567-9
134 rdf:type schema:CreativeWork
135 sg:pub.10.1007/s11192-016-2185-x schema:sameAs https://app.dimensions.ai/details/publication/pub.1024659616
136 https://doi.org/10.1007/s11192-016-2185-x
137 rdf:type schema:CreativeWork
138 https://doi.org/10.1002/asi.22748 schema:sameAs https://app.dimensions.ai/details/publication/pub.1036746088
139 rdf:type schema:CreativeWork
140 https://doi.org/10.1016/j.joi.2009.11.001 schema:sameAs https://app.dimensions.ai/details/publication/pub.1046858087
141 rdf:type schema:CreativeWork
142 https://doi.org/10.1016/j.joi.2014.11.010 schema:sameAs https://app.dimensions.ai/details/publication/pub.1009981400
143 rdf:type schema:CreativeWork
144 https://doi.org/10.1016/j.respol.2011.09.007 schema:sameAs https://app.dimensions.ai/details/publication/pub.1038742987
145 rdf:type schema:CreativeWork
146 https://doi.org/10.1016/j.respol.2014.08.004 schema:sameAs https://app.dimensions.ai/details/publication/pub.1025100457
147 rdf:type schema:CreativeWork
148 https://doi.org/10.1080/00220620500211189 schema:sameAs https://app.dimensions.ai/details/publication/pub.1013629377
149 rdf:type schema:CreativeWork
150 https://doi.org/10.1093/reseval/rvv038 schema:sameAs https://app.dimensions.ai/details/publication/pub.1060004526
151 rdf:type schema:CreativeWork
152 https://doi.org/10.1093/scipol/scy041 schema:sameAs https://app.dimensions.ai/details/publication/pub.1103614798
153 rdf:type schema:CreativeWork
154 https://doi.org/10.1098/rsos.160384 schema:sameAs https://app.dimensions.ai/details/publication/pub.1022320446
155 rdf:type schema:CreativeWork
156 https://doi.org/10.1098/rspl.1896.0076 schema:sameAs https://app.dimensions.ai/details/publication/pub.1036679935
157 rdf:type schema:CreativeWork
158 https://doi.org/10.1108/00220410310698734 schema:sameAs https://app.dimensions.ai/details/publication/pub.1011200668
159 rdf:type schema:CreativeWork
160 https://doi.org/10.1108/eb026940 schema:sameAs https://app.dimensions.ai/details/publication/pub.1043531532
161 rdf:type schema:CreativeWork
162 https://doi.org/10.1108/eum0000000007207 schema:sameAs https://app.dimensions.ai/details/publication/pub.1003147541
163 rdf:type schema:CreativeWork
164 https://doi.org/10.1111/1467-856x.12006 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045056270
165 rdf:type schema:CreativeWork
166 https://doi.org/10.1111/j.1467-8551.1994.tb00079.x schema:sameAs https://app.dimensions.ai/details/publication/pub.1048113776
167 rdf:type schema:CreativeWork
168 https://doi.org/10.1111/j.1467-8551.2010.00722.x schema:sameAs https://app.dimensions.ai/details/publication/pub.1003806183
169 rdf:type schema:CreativeWork
170 https://doi.org/10.1111/j.1467-9515.2011.00824.x schema:sameAs https://app.dimensions.ai/details/publication/pub.1028453339
171 rdf:type schema:CreativeWork
172 https://doi.org/10.1111/j.1467-954x.2012.02053.x schema:sameAs https://app.dimensions.ai/details/publication/pub.1007603484
173 rdf:type schema:CreativeWork
174 https://doi.org/10.1111/j.1468-0335.2009.00837.x schema:sameAs https://app.dimensions.ai/details/publication/pub.1049485646
175 rdf:type schema:CreativeWork
176 https://doi.org/10.1111/j.1478-9299.2008.00167.x schema:sameAs https://app.dimensions.ai/details/publication/pub.1015700222
177 rdf:type schema:CreativeWork
178 https://doi.org/10.1136/bmjopen-2015-010916 schema:sameAs https://app.dimensions.ai/details/publication/pub.1021544275
179 rdf:type schema:CreativeWork
180 https://doi.org/10.1177/016555159502100109 schema:sameAs https://app.dimensions.ai/details/publication/pub.1063752058
181 rdf:type schema:CreativeWork
182 https://doi.org/10.1371/journal.pone.0132990 schema:sameAs https://app.dimensions.ai/details/publication/pub.1023638259
183 rdf:type schema:CreativeWork
184 https://doi.org/10.1371/journal.pone.0173152 schema:sameAs https://app.dimensions.ai/details/publication/pub.1084295409
185 rdf:type schema:CreativeWork
186 https://doi.org/10.2139/ssrn.1154053 schema:sameAs https://app.dimensions.ai/details/publication/pub.1102252156
187 rdf:type schema:CreativeWork
188 https://doi.org/10.2307/2982732 schema:sameAs https://app.dimensions.ai/details/publication/pub.1070160606
189 rdf:type schema:CreativeWork
190 https://doi.org/10.5860/crln.68.5.7804 schema:sameAs https://app.dimensions.ai/details/publication/pub.1087307504
191 rdf:type schema:CreativeWork
192 https://www.grid.ac/institutes/grid.5132.5 schema:alternateName Leiden University
193 schema:name Centre for Science and Technology Studies (CWTS), Leiden University, Leiden, The Netherlands
194 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...