Co-training an Improved Recurrent Neural Network with Probability Statistic Models for Named Entity Recognition View Full Text


Ontology type: schema:Chapter      Open Access: True


Chapter Info

DATE

2017

AUTHORS

Yueqing Sun , Lin Li , Zhongwei Xie , Qing Xie , Xin Li , Guandong Xu

ABSTRACT

Named Entity Recognition (NER) is a subtask of information extraction in Natural Language Processing (NLP) field and thus being wildly studied. Currently Recurrent Neural Network (RNN) has become a popular way to do NER task, but it needs a lot of train data. The lack of labeled train data is one of the hard problems and traditional co-training strategy is a way to alleviate it. In this paper, we consider this situation and focus on doing NER with co-training using RNN and two probability statistic models i.e. Hidden Markov Model (HMM) and Conditional Random Field (CRF). We proposed a modified RNN model by redefining its activation function. Compared to traditional sigmoid function, our new function avoids saturation to some degree and makes its output scope very close to [0, 1], thus improving recognition accuracy. Our experiments are conducted ATIS benchmark. First, supervised learning using those models are compared when using different train data size. The experimental results show that it is not necessary to use whole data, even small part of train data can also get good performance. Then, we compare the results of our modified RNN with original RNN. 0.5% improvement is obtained. Last, we compare the co-training results. HMM and CRF get higher improvement than RNN after co-training. Moreover, using our modified RNN in co-training, their performances are improved further. More... »

PAGES

545-555

References to SciGraph publications

Book

TITLE

Database Systems for Advanced Applications

ISBN

978-3-319-55698-7
978-3-319-55699-4

From Grant

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33

DOI

http://dx.doi.org/10.1007/978-3-319-55699-4_33

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1084712345


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "name": [
            "School of Computer Science and Technlogy Wuhan University of Technology Wuhan China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Sun", 
        "givenName": "Yueqing", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "School of Computer Science and Technlogy Wuhan University of Technology Wuhan China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Li", 
        "givenName": "Lin", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "School of Computer Science and Technlogy Wuhan University of Technology Wuhan China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Xie", 
        "givenName": "Zhongwei", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "School of Computer Science and Technlogy Wuhan University of Technology Wuhan China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Xie", 
        "givenName": "Qing", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "iFLYTEK Big Data Research Institute Hefei China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Li", 
        "givenName": "Xin", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "School of Software University of Technology Sydney Ultimo Australia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Xu", 
        "givenName": "Guandong", 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "sg:pub.10.1007/s10586-015-0491-3", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1001774699", 
          "https://doi.org/10.1007/s10586-015-0491-3"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1155/2016/3289801", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1006713381"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1038/nature14539", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1010020120", 
          "https://doi.org/10.1038/nature14539"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.jbi.2011.10.004", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1010289706"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-642-45185-0_50", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1024691216", 
          "https://doi.org/10.1007/978-3-642-45185-0_50"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s10586-015-0426-z", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1029747547", 
          "https://doi.org/10.1007/s10586-015-0426-z"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.jbi.2004.08.012", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1040725201"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/279943.279962", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1045398430"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.neucom.2016.08.037", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1051672177"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1504/ijdmb.2016.079799", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1067446413"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/bibm.2015.7359761", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1093225548"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2017", 
    "datePublishedReg": "2017-01-01", 
    "description": "Named Entity Recognition (NER) is a subtask of information extraction in Natural Language Processing (NLP) field and thus being wildly studied. Currently Recurrent Neural Network (RNN) has become a popular way to do NER task, but it needs a lot of train data. The lack of labeled train data is one of the hard problems and traditional co-training strategy is a way to alleviate it. In this paper, we consider this situation and focus on doing NER with co-training using RNN and two probability statistic models i.e. Hidden Markov Model (HMM) and Conditional Random Field (CRF). We proposed a modified RNN model by redefining its activation function. Compared to traditional sigmoid function, our new function avoids saturation to some degree and makes its output scope very close to [0, 1], thus improving recognition accuracy. Our experiments are conducted ATIS benchmark. First, supervised learning using those models are compared when using different train data size. The experimental results show that it is not necessary to use whole data, even small part of train data can also get good performance. Then, we compare the results of our modified RNN with original RNN. 0.5% improvement is obtained. Last, we compare the co-training results. HMM and CRF get higher improvement than RNN after co-training. Moreover, using our modified RNN in co-training, their performances are improved further.", 
    "editor": [
      {
        "familyName": "Candan", 
        "givenName": "Sel\u00e7uk", 
        "type": "Person"
      }, 
      {
        "familyName": "Chen", 
        "givenName": "Lei", 
        "type": "Person"
      }, 
      {
        "familyName": "Pedersen", 
        "givenName": "Torben Bach", 
        "type": "Person"
      }, 
      {
        "familyName": "Chang", 
        "givenName": "Lijun", 
        "type": "Person"
      }, 
      {
        "familyName": "Hua", 
        "givenName": "Wen", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-55699-4_33", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": true, 
    "isFundedItemOf": [
      {
        "id": "sg:grant.7208192", 
        "type": "MonetaryGrant"
      }
    ], 
    "isPartOf": {
      "isbn": [
        "978-3-319-55698-7", 
        "978-3-319-55699-4"
      ], 
      "name": "Database Systems for Advanced Applications", 
      "type": "Book"
    }, 
    "name": "Co-training an Improved Recurrent Neural Network with Probability Statistic Models for Named Entity Recognition", 
    "pagination": "545-555", 
    "productId": [
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-55699-4_33"
        ]
      }, 
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "9c657d523aafa1b35524c83e2bd66bf698a183d1ed75b2e16b895cf2b21540f2"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1084712345"
        ]
      }
    ], 
    "publisher": {
      "location": "Cham", 
      "name": "Springer International Publishing", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-55699-4_33", 
      "https://app.dimensions.ai/details/publication/pub.1084712345"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2019-04-16T00:02", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8697_00000331.jsonl", 
    "type": "Chapter", 
    "url": "http://link.springer.com/10.1007/978-3-319-55699-4_33"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33'


 

This table displays all metadata directly associated to this object as RDF triples.

162 TRIPLES      23 PREDICATES      38 URIs      20 LITERALS      8 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-55699-4_33 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author N12aa973bbbc14420a263b39566346a3b
4 schema:citation sg:pub.10.1007/978-3-642-45185-0_50
5 sg:pub.10.1007/s10586-015-0426-z
6 sg:pub.10.1007/s10586-015-0491-3
7 sg:pub.10.1038/nature14539
8 https://doi.org/10.1016/j.jbi.2004.08.012
9 https://doi.org/10.1016/j.jbi.2011.10.004
10 https://doi.org/10.1016/j.neucom.2016.08.037
11 https://doi.org/10.1109/bibm.2015.7359761
12 https://doi.org/10.1145/279943.279962
13 https://doi.org/10.1155/2016/3289801
14 https://doi.org/10.1504/ijdmb.2016.079799
15 schema:datePublished 2017
16 schema:datePublishedReg 2017-01-01
17 schema:description Named Entity Recognition (NER) is a subtask of information extraction in Natural Language Processing (NLP) field and thus being wildly studied. Currently Recurrent Neural Network (RNN) has become a popular way to do NER task, but it needs a lot of train data. The lack of labeled train data is one of the hard problems and traditional co-training strategy is a way to alleviate it. In this paper, we consider this situation and focus on doing NER with co-training using RNN and two probability statistic models i.e. Hidden Markov Model (HMM) and Conditional Random Field (CRF). We proposed a modified RNN model by redefining its activation function. Compared to traditional sigmoid function, our new function avoids saturation to some degree and makes its output scope very close to [0, 1], thus improving recognition accuracy. Our experiments are conducted ATIS benchmark. First, supervised learning using those models are compared when using different train data size. The experimental results show that it is not necessary to use whole data, even small part of train data can also get good performance. Then, we compare the results of our modified RNN with original RNN. 0.5% improvement is obtained. Last, we compare the co-training results. HMM and CRF get higher improvement than RNN after co-training. Moreover, using our modified RNN in co-training, their performances are improved further.
18 schema:editor N63fb9112e62841dfae78c329a8666469
19 schema:genre chapter
20 schema:inLanguage en
21 schema:isAccessibleForFree true
22 schema:isPartOf N5359e263f5f14de595e0b90f21ba538f
23 schema:name Co-training an Improved Recurrent Neural Network with Probability Statistic Models for Named Entity Recognition
24 schema:pagination 545-555
25 schema:productId N089b894ba50949e0b07ac5c3df156715
26 N1a0a026f2157477ca2460968635e18b7
27 N31efb524f8264c45b892711a390d415e
28 schema:publisher N6dab4578815b426db778c1e97fc93633
29 schema:sameAs https://app.dimensions.ai/details/publication/pub.1084712345
30 https://doi.org/10.1007/978-3-319-55699-4_33
31 schema:sdDatePublished 2019-04-16T00:02
32 schema:sdLicense https://scigraph.springernature.com/explorer/license/
33 schema:sdPublisher Nb076c0b0ea204dc1bc191812fa42401d
34 schema:url http://link.springer.com/10.1007/978-3-319-55699-4_33
35 sgo:license sg:explorer/license/
36 sgo:sdDataset chapters
37 rdf:type schema:Chapter
38 N089b894ba50949e0b07ac5c3df156715 schema:name doi
39 schema:value 10.1007/978-3-319-55699-4_33
40 rdf:type schema:PropertyValue
41 N12aa973bbbc14420a263b39566346a3b rdf:first N2a6222f0a4a149d9bd5aa1d40d36c695
42 rdf:rest N94e397422f8443f4a8ab7807fac010b9
43 N15a67460d1124162b2dc46be47f1656a schema:name iFLYTEK Big Data Research Institute Hefei China
44 rdf:type schema:Organization
45 N1a0a026f2157477ca2460968635e18b7 schema:name readcube_id
46 schema:value 9c657d523aafa1b35524c83e2bd66bf698a183d1ed75b2e16b895cf2b21540f2
47 rdf:type schema:PropertyValue
48 N2a6222f0a4a149d9bd5aa1d40d36c695 schema:affiliation Naf2d6a4a97b4401a8a7df9d6074404ef
49 schema:familyName Sun
50 schema:givenName Yueqing
51 rdf:type schema:Person
52 N31efb524f8264c45b892711a390d415e schema:name dimensions_id
53 schema:value pub.1084712345
54 rdf:type schema:PropertyValue
55 N35179cb8e1fc4456bec60a9e4e71ea02 schema:affiliation N4f61683fe65c45ac9390d7490dfd723a
56 schema:familyName Xu
57 schema:givenName Guandong
58 rdf:type schema:Person
59 N363fee9b6a534c3ab10fe55a173f70bc rdf:first N45d107ffe70440d7ab83ed544f1239ce
60 rdf:rest N987c493a63844d1ea7c9ffea2dacca66
61 N45d107ffe70440d7ab83ed544f1239ce schema:affiliation Nc3eec0fd5f8f43ee95087423013819a1
62 schema:familyName Xie
63 schema:givenName Zhongwei
64 rdf:type schema:Person
65 N4f61683fe65c45ac9390d7490dfd723a schema:name School of Software University of Technology Sydney Ultimo Australia
66 rdf:type schema:Organization
67 N5359e263f5f14de595e0b90f21ba538f schema:isbn 978-3-319-55698-7
68 978-3-319-55699-4
69 schema:name Database Systems for Advanced Applications
70 rdf:type schema:Book
71 N63fb9112e62841dfae78c329a8666469 rdf:first Ncf1a6df748ac4affad71604b31ab40e0
72 rdf:rest N728797ec2720419a85b7a2c52f428c1b
73 N6dab4578815b426db778c1e97fc93633 schema:location Cham
74 schema:name Springer International Publishing
75 rdf:type schema:Organisation
76 N6e033f56e3374747b0011fe5e13fb212 rdf:first Nb109e54e18b841aa8aa55f626fb12b98
77 rdf:rest Nca2133c4b10d4e31b19adb27df424330
78 N728797ec2720419a85b7a2c52f428c1b rdf:first Na361b478ef8145afbb05146437eb4529
79 rdf:rest N7cc4c5c6c5fc48dfb752ddfc3c346c6d
80 N7cc4c5c6c5fc48dfb752ddfc3c346c6d rdf:first Nbc8c549f33ad49619c419a4b6c004053
81 rdf:rest N6e033f56e3374747b0011fe5e13fb212
82 N876fd0aa94bf4979a42f41135121067a schema:affiliation Nf8bbc51094554f81ba3224550a7759a2
83 schema:familyName Li
84 schema:givenName Lin
85 rdf:type schema:Person
86 N8c048459c0714249ab82c7690ff524a9 rdf:first Nc14a47eb1f474ccbb1ced450db110d39
87 rdf:rest Ne85117d06a4e4799a08a8b3d05fe4db0
88 N94e397422f8443f4a8ab7807fac010b9 rdf:first N876fd0aa94bf4979a42f41135121067a
89 rdf:rest N363fee9b6a534c3ab10fe55a173f70bc
90 N95c00f3ccfae4c85903b240aa671303e schema:familyName Hua
91 schema:givenName Wen
92 rdf:type schema:Person
93 N987c493a63844d1ea7c9ffea2dacca66 rdf:first Nd5b6738844ef44f7b7c54034eb421553
94 rdf:rest N8c048459c0714249ab82c7690ff524a9
95 Na361b478ef8145afbb05146437eb4529 schema:familyName Chen
96 schema:givenName Lei
97 rdf:type schema:Person
98 Naf2d6a4a97b4401a8a7df9d6074404ef schema:name School of Computer Science and Technlogy Wuhan University of Technology Wuhan China
99 rdf:type schema:Organization
100 Nb076c0b0ea204dc1bc191812fa42401d schema:name Springer Nature - SN SciGraph project
101 rdf:type schema:Organization
102 Nb109e54e18b841aa8aa55f626fb12b98 schema:familyName Chang
103 schema:givenName Lijun
104 rdf:type schema:Person
105 Nbc8c549f33ad49619c419a4b6c004053 schema:familyName Pedersen
106 schema:givenName Torben Bach
107 rdf:type schema:Person
108 Nc14a47eb1f474ccbb1ced450db110d39 schema:affiliation N15a67460d1124162b2dc46be47f1656a
109 schema:familyName Li
110 schema:givenName Xin
111 rdf:type schema:Person
112 Nc3eec0fd5f8f43ee95087423013819a1 schema:name School of Computer Science and Technlogy Wuhan University of Technology Wuhan China
113 rdf:type schema:Organization
114 Nca2133c4b10d4e31b19adb27df424330 rdf:first N95c00f3ccfae4c85903b240aa671303e
115 rdf:rest rdf:nil
116 Ncf1a6df748ac4affad71604b31ab40e0 schema:familyName Candan
117 schema:givenName Selçuk
118 rdf:type schema:Person
119 Nd5b6738844ef44f7b7c54034eb421553 schema:affiliation Ndb6480ca9891471eac6cde0a7433943a
120 schema:familyName Xie
121 schema:givenName Qing
122 rdf:type schema:Person
123 Ndb6480ca9891471eac6cde0a7433943a schema:name School of Computer Science and Technlogy Wuhan University of Technology Wuhan China
124 rdf:type schema:Organization
125 Ne85117d06a4e4799a08a8b3d05fe4db0 rdf:first N35179cb8e1fc4456bec60a9e4e71ea02
126 rdf:rest rdf:nil
127 Nf8bbc51094554f81ba3224550a7759a2 schema:name School of Computer Science and Technlogy Wuhan University of Technology Wuhan China
128 rdf:type schema:Organization
129 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
130 schema:name Information and Computing Sciences
131 rdf:type schema:DefinedTerm
132 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
133 schema:name Artificial Intelligence and Image Processing
134 rdf:type schema:DefinedTerm
135 sg:grant.7208192 http://pending.schema.org/fundedItem sg:pub.10.1007/978-3-319-55699-4_33
136 rdf:type schema:MonetaryGrant
137 sg:pub.10.1007/978-3-642-45185-0_50 schema:sameAs https://app.dimensions.ai/details/publication/pub.1024691216
138 https://doi.org/10.1007/978-3-642-45185-0_50
139 rdf:type schema:CreativeWork
140 sg:pub.10.1007/s10586-015-0426-z schema:sameAs https://app.dimensions.ai/details/publication/pub.1029747547
141 https://doi.org/10.1007/s10586-015-0426-z
142 rdf:type schema:CreativeWork
143 sg:pub.10.1007/s10586-015-0491-3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1001774699
144 https://doi.org/10.1007/s10586-015-0491-3
145 rdf:type schema:CreativeWork
146 sg:pub.10.1038/nature14539 schema:sameAs https://app.dimensions.ai/details/publication/pub.1010020120
147 https://doi.org/10.1038/nature14539
148 rdf:type schema:CreativeWork
149 https://doi.org/10.1016/j.jbi.2004.08.012 schema:sameAs https://app.dimensions.ai/details/publication/pub.1040725201
150 rdf:type schema:CreativeWork
151 https://doi.org/10.1016/j.jbi.2011.10.004 schema:sameAs https://app.dimensions.ai/details/publication/pub.1010289706
152 rdf:type schema:CreativeWork
153 https://doi.org/10.1016/j.neucom.2016.08.037 schema:sameAs https://app.dimensions.ai/details/publication/pub.1051672177
154 rdf:type schema:CreativeWork
155 https://doi.org/10.1109/bibm.2015.7359761 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093225548
156 rdf:type schema:CreativeWork
157 https://doi.org/10.1145/279943.279962 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045398430
158 rdf:type schema:CreativeWork
159 https://doi.org/10.1155/2016/3289801 schema:sameAs https://app.dimensions.ai/details/publication/pub.1006713381
160 rdf:type schema:CreativeWork
161 https://doi.org/10.1504/ijdmb.2016.079799 schema:sameAs https://app.dimensions.ai/details/publication/pub.1067446413
162 rdf:type schema:CreativeWork
 




Preview window. Press ESC to close (or click here)


...