Co-training an Improved Recurrent Neural Network with Probability Statistic Models for Named Entity Recognition View Full Text


Ontology type: schema:Chapter      Open Access: True


Chapter Info

DATE

2017

AUTHORS

Yueqing Sun , Lin Li , Zhongwei Xie , Qing Xie , Xin Li , Guandong Xu

ABSTRACT

Named Entity Recognition (NER) is a subtask of information extraction in Natural Language Processing (NLP) field and thus being wildly studied. Currently Recurrent Neural Network (RNN) has become a popular way to do NER task, but it needs a lot of train data. The lack of labeled train data is one of the hard problems and traditional co-training strategy is a way to alleviate it. In this paper, we consider this situation and focus on doing NER with co-training using RNN and two probability statistic models i.e. Hidden Markov Model (HMM) and Conditional Random Field (CRF). We proposed a modified RNN model by redefining its activation function. Compared to traditional sigmoid function, our new function avoids saturation to some degree and makes its output scope very close to [0, 1], thus improving recognition accuracy. Our experiments are conducted ATIS benchmark. First, supervised learning using those models are compared when using different train data size. The experimental results show that it is not necessary to use whole data, even small part of train data can also get good performance. Then, we compare the results of our modified RNN with original RNN. 0.5% improvement is obtained. Last, we compare the co-training results. HMM and CRF get higher improvement than RNN after co-training. Moreover, using our modified RNN in co-training, their performances are improved further. More... »

PAGES

545-555

References to SciGraph publications

Book

TITLE

Database Systems for Advanced Applications

ISBN

978-3-319-55698-7
978-3-319-55699-4

From Grant

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33

DOI

http://dx.doi.org/10.1007/978-3-319-55699-4_33

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1084712345


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "name": [
            "School of Computer Science and Technlogy Wuhan University of Technology Wuhan China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Sun", 
        "givenName": "Yueqing", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "School of Computer Science and Technlogy Wuhan University of Technology Wuhan China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Li", 
        "givenName": "Lin", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "School of Computer Science and Technlogy Wuhan University of Technology Wuhan China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Xie", 
        "givenName": "Zhongwei", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "School of Computer Science and Technlogy Wuhan University of Technology Wuhan China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Xie", 
        "givenName": "Qing", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "iFLYTEK Big Data Research Institute Hefei China"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Li", 
        "givenName": "Xin", 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "name": [
            "School of Software University of Technology Sydney Ultimo Australia"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Xu", 
        "givenName": "Guandong", 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "sg:pub.10.1007/s10586-015-0491-3", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1001774699", 
          "https://doi.org/10.1007/s10586-015-0491-3"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1155/2016/3289801", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1006713381"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1038/nature14539", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1010020120", 
          "https://doi.org/10.1038/nature14539"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.jbi.2011.10.004", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1010289706"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-642-45185-0_50", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1024691216", 
          "https://doi.org/10.1007/978-3-642-45185-0_50"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/s10586-015-0426-z", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1029747547", 
          "https://doi.org/10.1007/s10586-015-0426-z"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.jbi.2004.08.012", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1040725201"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/279943.279962", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1045398430"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1016/j.neucom.2016.08.037", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1051672177"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1504/ijdmb.2016.079799", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1067446413"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/bibm.2015.7359761", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1093225548"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2017", 
    "datePublishedReg": "2017-01-01", 
    "description": "Named Entity Recognition (NER) is a subtask of information extraction in Natural Language Processing (NLP) field and thus being wildly studied. Currently Recurrent Neural Network (RNN) has become a popular way to do NER task, but it needs a lot of train data. The lack of labeled train data is one of the hard problems and traditional co-training strategy is a way to alleviate it. In this paper, we consider this situation and focus on doing NER with co-training using RNN and two probability statistic models i.e. Hidden Markov Model (HMM) and Conditional Random Field (CRF). We proposed a modified RNN model by redefining its activation function. Compared to traditional sigmoid function, our new function avoids saturation to some degree and makes its output scope very close to [0, 1], thus improving recognition accuracy. Our experiments are conducted ATIS benchmark. First, supervised learning using those models are compared when using different train data size. The experimental results show that it is not necessary to use whole data, even small part of train data can also get good performance. Then, we compare the results of our modified RNN with original RNN. 0.5% improvement is obtained. Last, we compare the co-training results. HMM and CRF get higher improvement than RNN after co-training. Moreover, using our modified RNN in co-training, their performances are improved further.", 
    "editor": [
      {
        "familyName": "Candan", 
        "givenName": "Sel\u00e7uk", 
        "type": "Person"
      }, 
      {
        "familyName": "Chen", 
        "givenName": "Lei", 
        "type": "Person"
      }, 
      {
        "familyName": "Pedersen", 
        "givenName": "Torben Bach", 
        "type": "Person"
      }, 
      {
        "familyName": "Chang", 
        "givenName": "Lijun", 
        "type": "Person"
      }, 
      {
        "familyName": "Hua", 
        "givenName": "Wen", 
        "type": "Person"
      }
    ], 
    "genre": "chapter", 
    "id": "sg:pub.10.1007/978-3-319-55699-4_33", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": true, 
    "isFundedItemOf": [
      {
        "id": "sg:grant.7208192", 
        "type": "MonetaryGrant"
      }
    ], 
    "isPartOf": {
      "isbn": [
        "978-3-319-55698-7", 
        "978-3-319-55699-4"
      ], 
      "name": "Database Systems for Advanced Applications", 
      "type": "Book"
    }, 
    "name": "Co-training an Improved Recurrent Neural Network with Probability Statistic Models for Named Entity Recognition", 
    "pagination": "545-555", 
    "productId": [
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/978-3-319-55699-4_33"
        ]
      }, 
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "9c657d523aafa1b35524c83e2bd66bf698a183d1ed75b2e16b895cf2b21540f2"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1084712345"
        ]
      }
    ], 
    "publisher": {
      "location": "Cham", 
      "name": "Springer International Publishing", 
      "type": "Organisation"
    }, 
    "sameAs": [
      "https://doi.org/10.1007/978-3-319-55699-4_33", 
      "https://app.dimensions.ai/details/publication/pub.1084712345"
    ], 
    "sdDataset": "chapters", 
    "sdDatePublished": "2019-04-16T00:02", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8697_00000331.jsonl", 
    "type": "Chapter", 
    "url": "http://link.springer.com/10.1007/978-3-319-55699-4_33"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/978-3-319-55699-4_33'


 

This table displays all metadata directly associated to this object as RDF triples.

162 TRIPLES      23 PREDICATES      38 URIs      20 LITERALS      8 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/978-3-319-55699-4_33 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author Na877d659250641738a61cb7a90575122
4 schema:citation sg:pub.10.1007/978-3-642-45185-0_50
5 sg:pub.10.1007/s10586-015-0426-z
6 sg:pub.10.1007/s10586-015-0491-3
7 sg:pub.10.1038/nature14539
8 https://doi.org/10.1016/j.jbi.2004.08.012
9 https://doi.org/10.1016/j.jbi.2011.10.004
10 https://doi.org/10.1016/j.neucom.2016.08.037
11 https://doi.org/10.1109/bibm.2015.7359761
12 https://doi.org/10.1145/279943.279962
13 https://doi.org/10.1155/2016/3289801
14 https://doi.org/10.1504/ijdmb.2016.079799
15 schema:datePublished 2017
16 schema:datePublishedReg 2017-01-01
17 schema:description Named Entity Recognition (NER) is a subtask of information extraction in Natural Language Processing (NLP) field and thus being wildly studied. Currently Recurrent Neural Network (RNN) has become a popular way to do NER task, but it needs a lot of train data. The lack of labeled train data is one of the hard problems and traditional co-training strategy is a way to alleviate it. In this paper, we consider this situation and focus on doing NER with co-training using RNN and two probability statistic models i.e. Hidden Markov Model (HMM) and Conditional Random Field (CRF). We proposed a modified RNN model by redefining its activation function. Compared to traditional sigmoid function, our new function avoids saturation to some degree and makes its output scope very close to [0, 1], thus improving recognition accuracy. Our experiments are conducted ATIS benchmark. First, supervised learning using those models are compared when using different train data size. The experimental results show that it is not necessary to use whole data, even small part of train data can also get good performance. Then, we compare the results of our modified RNN with original RNN. 0.5% improvement is obtained. Last, we compare the co-training results. HMM and CRF get higher improvement than RNN after co-training. Moreover, using our modified RNN in co-training, their performances are improved further.
18 schema:editor N3d71ae4f911243f9919cfac3c7b09248
19 schema:genre chapter
20 schema:inLanguage en
21 schema:isAccessibleForFree true
22 schema:isPartOf N39f0be45ab8c495fa353136e3dc90bc8
23 schema:name Co-training an Improved Recurrent Neural Network with Probability Statistic Models for Named Entity Recognition
24 schema:pagination 545-555
25 schema:productId N1f6c4834154346d4a9b8b0dd5f318fe1
26 Ndede4ac1d611413e9f3bb23c21d79973
27 Nedcb4dcca0734a84973563eebd8a51a7
28 schema:publisher Nbdca66629d79489c8c08756975ef675e
29 schema:sameAs https://app.dimensions.ai/details/publication/pub.1084712345
30 https://doi.org/10.1007/978-3-319-55699-4_33
31 schema:sdDatePublished 2019-04-16T00:02
32 schema:sdLicense https://scigraph.springernature.com/explorer/license/
33 schema:sdPublisher N9e8e8e90b2ed495d9915aa0fcd08d174
34 schema:url http://link.springer.com/10.1007/978-3-319-55699-4_33
35 sgo:license sg:explorer/license/
36 sgo:sdDataset chapters
37 rdf:type schema:Chapter
38 N1c46beedec4d4798abe8fd6c17ed7c28 schema:affiliation N3163307661484b51a06c8618442cfc56
39 schema:familyName Sun
40 schema:givenName Yueqing
41 rdf:type schema:Person
42 N1e32b693840246cfb5d7ee7a8c886c8e rdf:first N8023ce9c6a0a453093abc05d4300f946
43 rdf:rest rdf:nil
44 N1f6c4834154346d4a9b8b0dd5f318fe1 schema:name doi
45 schema:value 10.1007/978-3-319-55699-4_33
46 rdf:type schema:PropertyValue
47 N219b39fb55ac47bda108a5a9617cb0ef rdf:first Nc25f975dea0e49949911500b23fab4ee
48 rdf:rest rdf:nil
49 N23be580182de4e079d7ac2761a943785 schema:name School of Computer Science and Technlogy Wuhan University of Technology Wuhan China
50 rdf:type schema:Organization
51 N252d6ba75e724310a1414698b2e06b44 rdf:first N77de029bd4394f29b826ca13f9c16549
52 rdf:rest Neafaf9f49cdb4468a1b2b13b5c000507
53 N2a7692dab81f4940813d2d1497e52686 schema:affiliation N23be580182de4e079d7ac2761a943785
54 schema:familyName Li
55 schema:givenName Lin
56 rdf:type schema:Person
57 N3163307661484b51a06c8618442cfc56 schema:name School of Computer Science and Technlogy Wuhan University of Technology Wuhan China
58 rdf:type schema:Organization
59 N36e3e9dc8f024e7d8ef9f061b8275aaa schema:name iFLYTEK Big Data Research Institute Hefei China
60 rdf:type schema:Organization
61 N39f0be45ab8c495fa353136e3dc90bc8 schema:isbn 978-3-319-55698-7
62 978-3-319-55699-4
63 schema:name Database Systems for Advanced Applications
64 rdf:type schema:Book
65 N3d71ae4f911243f9919cfac3c7b09248 rdf:first N89fcb6e0ecb3495180c98e85540f2454
66 rdf:rest N5509239cc09849138702755bd058e226
67 N52317e352e3d4c998f09627f2b183b18 rdf:first Nc8cbb5e8b15640b38dcfa9e2c6d708a1
68 rdf:rest N219b39fb55ac47bda108a5a9617cb0ef
69 N5509239cc09849138702755bd058e226 rdf:first Nf83ad92ff80843a4822d621b052acc89
70 rdf:rest N96f961fedcf741d08f88f6e771c205de
71 N6122bc5633b44c51a6846d39ba20830d rdf:first Nab6a43adf4864f19a8ebb68983222f6c
72 rdf:rest N1e32b693840246cfb5d7ee7a8c886c8e
73 N72b1f4aa8c294fd6877bea4dfb906ba8 schema:familyName Pedersen
74 schema:givenName Torben Bach
75 rdf:type schema:Person
76 N77de029bd4394f29b826ca13f9c16549 schema:affiliation Nac66dad5e84c4f76afbd7f9a56bad83b
77 schema:familyName Xie
78 schema:givenName Zhongwei
79 rdf:type schema:Person
80 N7ab0f4e76776440799840e0d3979a460 schema:affiliation Ndf3c74a357ba445ab033d40d6bc4e1b4
81 schema:familyName Xie
82 schema:givenName Qing
83 rdf:type schema:Person
84 N8023ce9c6a0a453093abc05d4300f946 schema:familyName Hua
85 schema:givenName Wen
86 rdf:type schema:Person
87 N8027f8908b7949f7b062f21508d26b8b schema:name School of Software University of Technology Sydney Ultimo Australia
88 rdf:type schema:Organization
89 N89fcb6e0ecb3495180c98e85540f2454 schema:familyName Candan
90 schema:givenName Selçuk
91 rdf:type schema:Person
92 N96f961fedcf741d08f88f6e771c205de rdf:first N72b1f4aa8c294fd6877bea4dfb906ba8
93 rdf:rest N6122bc5633b44c51a6846d39ba20830d
94 N9e8e8e90b2ed495d9915aa0fcd08d174 schema:name Springer Nature - SN SciGraph project
95 rdf:type schema:Organization
96 Na877d659250641738a61cb7a90575122 rdf:first N1c46beedec4d4798abe8fd6c17ed7c28
97 rdf:rest Ne160742068e34258b1a1651b63bab323
98 Nab6a43adf4864f19a8ebb68983222f6c schema:familyName Chang
99 schema:givenName Lijun
100 rdf:type schema:Person
101 Nac66dad5e84c4f76afbd7f9a56bad83b schema:name School of Computer Science and Technlogy Wuhan University of Technology Wuhan China
102 rdf:type schema:Organization
103 Nbdca66629d79489c8c08756975ef675e schema:location Cham
104 schema:name Springer International Publishing
105 rdf:type schema:Organisation
106 Nc25f975dea0e49949911500b23fab4ee schema:affiliation N8027f8908b7949f7b062f21508d26b8b
107 schema:familyName Xu
108 schema:givenName Guandong
109 rdf:type schema:Person
110 Nc8cbb5e8b15640b38dcfa9e2c6d708a1 schema:affiliation N36e3e9dc8f024e7d8ef9f061b8275aaa
111 schema:familyName Li
112 schema:givenName Xin
113 rdf:type schema:Person
114 Ndede4ac1d611413e9f3bb23c21d79973 schema:name readcube_id
115 schema:value 9c657d523aafa1b35524c83e2bd66bf698a183d1ed75b2e16b895cf2b21540f2
116 rdf:type schema:PropertyValue
117 Ndf3c74a357ba445ab033d40d6bc4e1b4 schema:name School of Computer Science and Technlogy Wuhan University of Technology Wuhan China
118 rdf:type schema:Organization
119 Ne160742068e34258b1a1651b63bab323 rdf:first N2a7692dab81f4940813d2d1497e52686
120 rdf:rest N252d6ba75e724310a1414698b2e06b44
121 Neafaf9f49cdb4468a1b2b13b5c000507 rdf:first N7ab0f4e76776440799840e0d3979a460
122 rdf:rest N52317e352e3d4c998f09627f2b183b18
123 Nedcb4dcca0734a84973563eebd8a51a7 schema:name dimensions_id
124 schema:value pub.1084712345
125 rdf:type schema:PropertyValue
126 Nf83ad92ff80843a4822d621b052acc89 schema:familyName Chen
127 schema:givenName Lei
128 rdf:type schema:Person
129 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
130 schema:name Information and Computing Sciences
131 rdf:type schema:DefinedTerm
132 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
133 schema:name Artificial Intelligence and Image Processing
134 rdf:type schema:DefinedTerm
135 sg:grant.7208192 http://pending.schema.org/fundedItem sg:pub.10.1007/978-3-319-55699-4_33
136 rdf:type schema:MonetaryGrant
137 sg:pub.10.1007/978-3-642-45185-0_50 schema:sameAs https://app.dimensions.ai/details/publication/pub.1024691216
138 https://doi.org/10.1007/978-3-642-45185-0_50
139 rdf:type schema:CreativeWork
140 sg:pub.10.1007/s10586-015-0426-z schema:sameAs https://app.dimensions.ai/details/publication/pub.1029747547
141 https://doi.org/10.1007/s10586-015-0426-z
142 rdf:type schema:CreativeWork
143 sg:pub.10.1007/s10586-015-0491-3 schema:sameAs https://app.dimensions.ai/details/publication/pub.1001774699
144 https://doi.org/10.1007/s10586-015-0491-3
145 rdf:type schema:CreativeWork
146 sg:pub.10.1038/nature14539 schema:sameAs https://app.dimensions.ai/details/publication/pub.1010020120
147 https://doi.org/10.1038/nature14539
148 rdf:type schema:CreativeWork
149 https://doi.org/10.1016/j.jbi.2004.08.012 schema:sameAs https://app.dimensions.ai/details/publication/pub.1040725201
150 rdf:type schema:CreativeWork
151 https://doi.org/10.1016/j.jbi.2011.10.004 schema:sameAs https://app.dimensions.ai/details/publication/pub.1010289706
152 rdf:type schema:CreativeWork
153 https://doi.org/10.1016/j.neucom.2016.08.037 schema:sameAs https://app.dimensions.ai/details/publication/pub.1051672177
154 rdf:type schema:CreativeWork
155 https://doi.org/10.1109/bibm.2015.7359761 schema:sameAs https://app.dimensions.ai/details/publication/pub.1093225548
156 rdf:type schema:CreativeWork
157 https://doi.org/10.1145/279943.279962 schema:sameAs https://app.dimensions.ai/details/publication/pub.1045398430
158 rdf:type schema:CreativeWork
159 https://doi.org/10.1155/2016/3289801 schema:sameAs https://app.dimensions.ai/details/publication/pub.1006713381
160 rdf:type schema:CreativeWork
161 https://doi.org/10.1504/ijdmb.2016.079799 schema:sameAs https://app.dimensions.ai/details/publication/pub.1067446413
162 rdf:type schema:CreativeWork
 




Preview window. Press ESC to close (or click here)


...