A Benchmark Evaluation of the French MeSH Indexers View Full Text


Ontology type: schema:Chapter     


Chapter Info

DATE

2005

AUTHORS

Aurélie Névéol , Vincent Mary , Arnaud Gaudinat , Célia Boyer , Alexandrina Rogozan , Stéfan J. Darmoni

ABSTRACT

The increasing demand on both practitioners and librarians to encode medical documents with controlled vocabularies calls for automatic tools and methods to help them perform this task efficiently. This paper presents the Benchmark evaluation of the French MeSH indexing systems carried out under the umbrella of the VUMeF consortium. The CISMeF, NOMINDEX and HONMeSHMapper systems are introduced, and evaluated on a set of 82 resources randomly taken from the CISMeF catalogue. The automatic MeSH indexing produced by each system was compared to the manual gold standard provided by the CISMeF medical librarian team. The automatic systems achieve at best a precision close to 50% at rank 1 (HONMeSHMapper, CISMeF) and HONMeSHMapper achieves the best overall F-measure. A qualitative evaluation of the indexing provided indicates that all systems tend to misevaluate the specificity of the terms to retrieve. More... »

PAGES

251-255

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/11527770_37

DOI

http://dx.doi.org/10.1007/11527770_37

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1004075491


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "CISMeF & CGIS, Rouen, France", 
          "id": "http://www.grid.ac/institutes/None", 
          "name": [
            "PSI Laboratory, Rouen, France", 
            "CISMeF & CGIS, Rouen, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "N\u00e9v\u00e9ol", 
        "givenName": "Aur\u00e9lie", 
        "id": "sg:person.0745527342.69", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0745527342.69"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Rennes Medical School, France", 
          "id": "http://www.grid.ac/institutes/None", 
          "name": [
            "Rennes Medical School, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Mary", 
        "givenName": "Vincent", 
        "id": "sg:person.0615021542.23", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0615021542.23"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "HON Foundation, Geneva, Switzerland", 
          "id": "http://www.grid.ac/institutes/grid.435186.d", 
          "name": [
            "HON Foundation, Geneva, Switzerland"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Gaudinat", 
        "givenName": "Arnaud", 
        "id": "sg:person.01114113373.23", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01114113373.23"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "HON Foundation, Geneva, Switzerland", 
          "id": "http://www.grid.ac/institutes/grid.435186.d", 
          "name": [
            "HON Foundation, Geneva, Switzerland"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Boyer", 
        "givenName": "C\u00e9lia", 
        "id": "sg:person.01062464064.11", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01062464064.11"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "PSI Laboratory, Rouen, France", 
          "id": "http://www.grid.ac/institutes/None", 
          "name": [
            "PSI Laboratory, Rouen, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Rogozan", 
        "givenName": "Alexandrina", 
        "id": "sg:person.012500037620.27", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012500037620.27"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "CISMeF & CGIS, Rouen, France", 
          "id": "http://www.grid.ac/institutes/None", 
          "name": [
            "PSI Laboratory, Rouen, France", 
            "CISMeF & CGIS, Rouen, France"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Darmoni", 
        "givenName": "St\u00e9fan J.", 
        "id": "sg:person.010360752677.38", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010360752677.38"
        ], 
        "type": "Person"
      }
    ], 
    "datePublished": "2005", 
    "datePublishedReg": "2005-01-01", 
    "description": "The increasing demand on both practitioners and librarians to encode medical documents with controlled vocabularies calls for automatic tools and methods to help them perform this task efficiently. This paper presents the Benchmark evaluation of the French MeSH indexing systems carried out under the umbrella of the VUMeF consortium. The CISMeF, NOMINDEX and HONMeSHMapper systems are introduced, and evaluated on a set of 82 resources randomly taken from the CISMeF catalogue. The automatic MeSH indexing produced by each system was compared to the manual gold standard provided by the CISMeF medical librarian team. The automatic systems achieve at best a precision close to 50% at rank 1 (HONMeSHMapper, CISMeF) and HONMeSHMapper achieves the best overall F-measure. A qualitative evaluation of the indexing provided indicates that all systems tend to misevaluate the specificity of the terms to retrieve.", 
    "editor": [
      {
        "familyName": "Miksch", 
        "givenName": "Silvia", 
        "type": "Person"
      }, 
      {
        "familyName": "Hunter", 
        "givenName": "Jim", 
        "type": "Person"
      }, 
      {
        "familyName": "Keravnou", 
        "givenName": "Elpida T.", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/11527770_37", 
    "inLanguage": "en", 
    "isAccessibleForFree": false, 
    "isPartOf": {
      "isbn": [
        "978-3-540-27831-3", 
        "978-3-540-31884-2"
      ], 
      "name": "Artificial Intelligence in Medicine", 
      "type": "Book"
    }, 
    "keywords": [
      "benchmark evaluation", 
      "manual gold standard", 
      "CISMeF catalogue", 
      "MeSH indexing", 
      "automatic tool", 
      "indexing system", 
      "medical documents", 
      "automatic system", 
      "indexing", 
      "qualitative evaluation", 
      "NOMINDEX", 
      "CISMeF", 
      "system", 
      "indexers", 
      "rank 1", 
      "task", 
      "documents", 
      "librarians", 
      "librarian team", 
      "vocabulary", 
      "set", 
      "resources", 
      "tool", 
      "evaluation", 
      "catalogue", 
      "precision", 
      "demand", 
      "team", 
      "standards", 
      "umbrella", 
      "method", 
      "consortium", 
      "terms", 
      "practitioners", 
      "measures", 
      "gold standard", 
      "specificity", 
      "paper"
    ], 
    "name": "A Benchmark Evaluation of the French MeSH Indexers", 
    "pagination": "251-255", 
    "productId": [
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1004075491"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/11527770_37"
        ]
      }
    ], 
    "publisher": {
      "name": "Springer Nature", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/11527770_37", 
      "https://app.dimensions.ai/details/publication/pub.1004075491"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2022-05-20T07:48", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-springernature-scigraph/baseset/20220519/entities/gbq_results/chapter/chapter_459.jsonl", 
    "type": "Chapter", 
    "url": "https://doi.org/10.1007/11527770_37"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/11527770_37'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/11527770_37'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/11527770_37'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/11527770_37'


 

This table displays all metadata directly associated to this object as RDF triples.

150 TRIPLES      23 PREDICATES      64 URIs      57 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/11527770_37 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author N663d8767bc6e49d9a5540a28a02ebbc0
4 schema:datePublished 2005
5 schema:datePublishedReg 2005-01-01
6 schema:description The increasing demand on both practitioners and librarians to encode medical documents with controlled vocabularies calls for automatic tools and methods to help them perform this task efficiently. This paper presents the Benchmark evaluation of the French MeSH indexing systems carried out under the umbrella of the VUMeF consortium. The CISMeF, NOMINDEX and HONMeSHMapper systems are introduced, and evaluated on a set of 82 resources randomly taken from the CISMeF catalogue. The automatic MeSH indexing produced by each system was compared to the manual gold standard provided by the CISMeF medical librarian team. The automatic systems achieve at best a precision close to 50% at rank 1 (HONMeSHMapper, CISMeF) and HONMeSHMapper achieves the best overall F-measure. A qualitative evaluation of the indexing provided indicates that all systems tend to misevaluate the specificity of the terms to retrieve.
7 schema:editor Nf464b047e83447a9923da78399bd6027
8 schema:genre chapter
9 schema:inLanguage en
10 schema:isAccessibleForFree false
11 schema:isPartOf Na85c4206e5094212b16e076b50738774
12 schema:keywords CISMeF
13 CISMeF catalogue
14 MeSH indexing
15 NOMINDEX
16 automatic system
17 automatic tool
18 benchmark evaluation
19 catalogue
20 consortium
21 demand
22 documents
23 evaluation
24 gold standard
25 indexers
26 indexing
27 indexing system
28 librarian team
29 librarians
30 manual gold standard
31 measures
32 medical documents
33 method
34 paper
35 practitioners
36 precision
37 qualitative evaluation
38 rank 1
39 resources
40 set
41 specificity
42 standards
43 system
44 task
45 team
46 terms
47 tool
48 umbrella
49 vocabulary
50 schema:name A Benchmark Evaluation of the French MeSH Indexers
51 schema:pagination 251-255
52 schema:productId N4750eaa2b70943189f5bec7b4ac85f09
53 N51a3ca6b53ba4650b6307a955ebc9a35
54 schema:publisher N1f3cb42d75624c3d9df82d6ff131592d
55 schema:sameAs https://app.dimensions.ai/details/publication/pub.1004075491
56 https://doi.org/10.1007/11527770_37
57 schema:sdDatePublished 2022-05-20T07:48
58 schema:sdLicense https://scigraph.springernature.com/explorer/license/
59 schema:sdPublisher N9e81349b940c48a0b37d73c6d1561959
60 schema:url https://doi.org/10.1007/11527770_37
61 sgo:license sg:explorer/license/
62 sgo:sdDataset chapters
63 rdf:type schema:Chapter
64 N0b547b6833f4427b9f573f83d1524c39 rdf:first N564036a9d23e431ba7d5f7dee65097b6
65 rdf:rest N705f416edf9d44648ec26b51da3cdfb4
66 N1f3cb42d75624c3d9df82d6ff131592d schema:name Springer Nature
67 rdf:type schema:Organisation
68 N293af2ecb3b84573a4ab9887520395ee rdf:first sg:person.012500037620.27
69 rdf:rest N2ff44414e9414267afc26e261cd5f07c
70 N2ff44414e9414267afc26e261cd5f07c rdf:first sg:person.010360752677.38
71 rdf:rest rdf:nil
72 N3b244822f28445729ab8520352b3f82c rdf:first sg:person.01114113373.23
73 rdf:rest N5a9ef4e0e2f54f7b8cfd8c084f1aee1b
74 N4750eaa2b70943189f5bec7b4ac85f09 schema:name dimensions_id
75 schema:value pub.1004075491
76 rdf:type schema:PropertyValue
77 N51a3ca6b53ba4650b6307a955ebc9a35 schema:name doi
78 schema:value 10.1007/11527770_37
79 rdf:type schema:PropertyValue
80 N53736867fafd402a81297fc586544946 rdf:first sg:person.0615021542.23
81 rdf:rest N3b244822f28445729ab8520352b3f82c
82 N564036a9d23e431ba7d5f7dee65097b6 schema:familyName Hunter
83 schema:givenName Jim
84 rdf:type schema:Person
85 N5a9ef4e0e2f54f7b8cfd8c084f1aee1b rdf:first sg:person.01062464064.11
86 rdf:rest N293af2ecb3b84573a4ab9887520395ee
87 N663d8767bc6e49d9a5540a28a02ebbc0 rdf:first sg:person.0745527342.69
88 rdf:rest N53736867fafd402a81297fc586544946
89 N705f416edf9d44648ec26b51da3cdfb4 rdf:first N95b36c66ba824ffba0d767fbc3b32f89
90 rdf:rest rdf:nil
91 N95b36c66ba824ffba0d767fbc3b32f89 schema:familyName Keravnou
92 schema:givenName Elpida T.
93 rdf:type schema:Person
94 N9e81349b940c48a0b37d73c6d1561959 schema:name Springer Nature - SN SciGraph project
95 rdf:type schema:Organization
96 Na85c4206e5094212b16e076b50738774 schema:isbn 978-3-540-27831-3
97 978-3-540-31884-2
98 schema:name Artificial Intelligence in Medicine
99 rdf:type schema:Book
100 Naab33ecc1c934f6eba980c66048bcef7 schema:familyName Miksch
101 schema:givenName Silvia
102 rdf:type schema:Person
103 Nf464b047e83447a9923da78399bd6027 rdf:first Naab33ecc1c934f6eba980c66048bcef7
104 rdf:rest N0b547b6833f4427b9f573f83d1524c39
105 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
106 schema:name Information and Computing Sciences
107 rdf:type schema:DefinedTerm
108 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
109 schema:name Artificial Intelligence and Image Processing
110 rdf:type schema:DefinedTerm
111 sg:person.010360752677.38 schema:affiliation grid-institutes:None
112 schema:familyName Darmoni
113 schema:givenName Stéfan J.
114 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.010360752677.38
115 rdf:type schema:Person
116 sg:person.01062464064.11 schema:affiliation grid-institutes:grid.435186.d
117 schema:familyName Boyer
118 schema:givenName Célia
119 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01062464064.11
120 rdf:type schema:Person
121 sg:person.01114113373.23 schema:affiliation grid-institutes:grid.435186.d
122 schema:familyName Gaudinat
123 schema:givenName Arnaud
124 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01114113373.23
125 rdf:type schema:Person
126 sg:person.012500037620.27 schema:affiliation grid-institutes:None
127 schema:familyName Rogozan
128 schema:givenName Alexandrina
129 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012500037620.27
130 rdf:type schema:Person
131 sg:person.0615021542.23 schema:affiliation grid-institutes:None
132 schema:familyName Mary
133 schema:givenName Vincent
134 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0615021542.23
135 rdf:type schema:Person
136 sg:person.0745527342.69 schema:affiliation grid-institutes:None
137 schema:familyName Névéol
138 schema:givenName Aurélie
139 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.0745527342.69
140 rdf:type schema:Person
141 grid-institutes:None schema:alternateName CISMeF & CGIS, Rouen, France
142 PSI Laboratory, Rouen, France
143 Rennes Medical School, France
144 schema:name CISMeF & CGIS, Rouen, France
145 PSI Laboratory, Rouen, France
146 Rennes Medical School, France
147 rdf:type schema:Organization
148 grid-institutes:grid.435186.d schema:alternateName HON Foundation, Geneva, Switzerland
149 schema:name HON Foundation, Geneva, Switzerland
150 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...