Training parsers by inverse reinforcement learning View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

2009-12

AUTHORS

Gergely Neu, Csaba Szepesvári

ABSTRACT

One major idea in structured prediction is to assume that the predictor computes its output by finding the maximum of a score function. The training of such a predictor can then be cast as the problem of finding weights of the score function so that the output of the predictor on the inputs matches the corresponding structured labels on the training set. A similar problem is studied in inverse reinforcement learning (IRL) where one is given an environment and a set of trajectories and the problem is to find a reward function such that an agent acting optimally with respect to the reward function would follow trajectories that match those in the training set. In this paper we show how IRL algorithms can be applied to structured prediction, in particular to parser training. We present a number of recent incremental IRL algorithms in a unified framework and map them to parser training algorithms. This allows us to recover some existing parser training algorithms, as well as to obtain a new one. The resulting algorithms are compared in terms of their sensitivity to the choice of various parameters and generalization ability on the Penn Treebank WSJ corpus. More... »

PAGES

303

References to SciGraph publications

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/s10994-009-5110-1

DOI

http://dx.doi.org/10.1007/s10994-009-5110-1

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1035491423


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0801", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Artificial Intelligence and Image Processing", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/08", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Information and Computing Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "MTA Institute for Computer Science and Control", 
          "id": "https://www.grid.ac/institutes/grid.4836.9", 
          "name": [
            "Department of Computing Science, Budapest University of Technology and Economics, M\u0171egyetem rakpart 3-9, 1111, Budapest, Hungary", 
            "Computer and Automation Research Institute of the Hungarian Academy of Sciences, Kende utca 13-17, 1111, Budapest, Hungary"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Neu", 
        "givenName": "Gergely", 
        "id": "sg:person.014330214576.44", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014330214576.44"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "University of Alberta", 
          "id": "https://www.grid.ac/institutes/grid.17089.37", 
          "name": [
            "Computer and Automation Research Institute of the Hungarian Academy of Sciences, Kende utca 13-17, 1111, Budapest, Hungary", 
            "Department of Computing Science, University of Alberta, T6G 2E8, Edmonton, Alberta, Canada"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Szepesv\u00e1ri", 
        "givenName": "Csaba", 
        "id": "sg:person.016202177221.23", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016202177221.23"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "https://doi.org/10.1145/1150402.1150466", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1000706999"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1273496.1273535", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1006158294"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1102351.1102464", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1022920538"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1006/jmbi.1998.2436", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1030245324"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1143844.1143936", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1033108091"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1023/a:1007662407062", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1033414988", 
          "https://doi.org/10.1023/a:1007662407062"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-540-74958-5_64", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1034739972", 
          "https://doi.org/10.1007/978-3-540-74958-5_64"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/978-3-540-74958-5_64", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1034739972", 
          "https://doi.org/10.1007/978-3-540-74958-5_64"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1015330.1015430", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1044181534"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/1273496.1273598", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1047753055"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1073445.1073461", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1049307492"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1103/physrev.106.620", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1060418970"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1103/physrev.106.620", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1060418970"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1109/icassp.1984.1172637", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1086174328"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1137/1.9781611970777", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1098556247"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1017/cbo9780511546921", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1098679346"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1118693.1118694", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099201575"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1118693.1118694", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099201575"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1218955.1218970", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099221196"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1218955.1218970", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099221196"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1219840.1219862", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099221866"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1219840.1219862", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099221866"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1220175.1220285", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099222039"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.3115/1220175.1220285", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1099222039"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.7551/mitpress/7443.001.0001", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1111386243"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "2009-12", 
    "datePublishedReg": "2009-12-01", 
    "description": "One major idea in structured prediction is to assume that the predictor computes its output by finding the maximum of a score function. The training of such a predictor can then be cast as the problem of finding weights of the score function so that the output of the predictor on the inputs matches the corresponding structured labels on the training set. A similar problem is studied in inverse reinforcement learning (IRL) where one is given an environment and a set of trajectories and the problem is to find a reward function such that an agent acting optimally with respect to the reward function would follow trajectories that match those in the training set. In this paper we show how IRL algorithms can be applied to structured prediction, in particular to parser training. We present a number of recent incremental IRL algorithms in a unified framework and map them to parser training algorithms. This allows us to recover some existing parser training algorithms, as well as to obtain a new one. The resulting algorithms are compared in terms of their sensitivity to the choice of various parameters and generalization ability on the Penn Treebank WSJ corpus.", 
    "genre": "research_article", 
    "id": "sg:pub.10.1007/s10994-009-5110-1", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": true, 
    "isPartOf": [
      {
        "id": "sg:journal.1125588", 
        "issn": [
          "0885-6125", 
          "1573-0565"
        ], 
        "name": "Machine Learning", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "2-3", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "77"
      }
    ], 
    "name": "Training parsers by inverse reinforcement learning", 
    "pagination": "303", 
    "productId": [
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "a9110cc73eeabb5755f964eef1a0c16e3e734d6cbbc3e678a0ee34a89b30f709"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/s10994-009-5110-1"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1035491423"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1007/s10994-009-5110-1", 
      "https://app.dimensions.ai/details/publication/pub.1035491423"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2019-04-11T09:12", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000338_0000000338/records_47985_00000001.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "http://link.springer.com/10.1007%2Fs10994-009-5110-1"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/s10994-009-5110-1'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/s10994-009-5110-1'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/s10994-009-5110-1'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/s10994-009-5110-1'


 

This table displays all metadata directly associated to this object as RDF triples.

132 TRIPLES      21 PREDICATES      46 URIs      19 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/s10994-009-5110-1 schema:about anzsrc-for:08
2 anzsrc-for:0801
3 schema:author N2b759cd3cbc24100be84dc6511dc1704
4 schema:citation sg:pub.10.1007/978-3-540-74958-5_64
5 sg:pub.10.1023/a:1007662407062
6 https://doi.org/10.1006/jmbi.1998.2436
7 https://doi.org/10.1017/cbo9780511546921
8 https://doi.org/10.1103/physrev.106.620
9 https://doi.org/10.1109/icassp.1984.1172637
10 https://doi.org/10.1137/1.9781611970777
11 https://doi.org/10.1145/1015330.1015430
12 https://doi.org/10.1145/1102351.1102464
13 https://doi.org/10.1145/1143844.1143936
14 https://doi.org/10.1145/1150402.1150466
15 https://doi.org/10.1145/1273496.1273535
16 https://doi.org/10.1145/1273496.1273598
17 https://doi.org/10.3115/1073445.1073461
18 https://doi.org/10.3115/1118693.1118694
19 https://doi.org/10.3115/1218955.1218970
20 https://doi.org/10.3115/1219840.1219862
21 https://doi.org/10.3115/1220175.1220285
22 https://doi.org/10.7551/mitpress/7443.001.0001
23 schema:datePublished 2009-12
24 schema:datePublishedReg 2009-12-01
25 schema:description One major idea in structured prediction is to assume that the predictor computes its output by finding the maximum of a score function. The training of such a predictor can then be cast as the problem of finding weights of the score function so that the output of the predictor on the inputs matches the corresponding structured labels on the training set. A similar problem is studied in inverse reinforcement learning (IRL) where one is given an environment and a set of trajectories and the problem is to find a reward function such that an agent acting optimally with respect to the reward function would follow trajectories that match those in the training set. In this paper we show how IRL algorithms can be applied to structured prediction, in particular to parser training. We present a number of recent incremental IRL algorithms in a unified framework and map them to parser training algorithms. This allows us to recover some existing parser training algorithms, as well as to obtain a new one. The resulting algorithms are compared in terms of their sensitivity to the choice of various parameters and generalization ability on the Penn Treebank WSJ corpus.
26 schema:genre research_article
27 schema:inLanguage en
28 schema:isAccessibleForFree true
29 schema:isPartOf N6470a6d06ab048fc97214019cbd0e914
30 N6d5b6dc621d5444082fae6af38397772
31 sg:journal.1125588
32 schema:name Training parsers by inverse reinforcement learning
33 schema:pagination 303
34 schema:productId N1aad2f167e3f407a83782afa88890073
35 Na71d29cb72a440f0a87478a22b609f9d
36 Nef239753d1c348ee816ecd170c256d59
37 schema:sameAs https://app.dimensions.ai/details/publication/pub.1035491423
38 https://doi.org/10.1007/s10994-009-5110-1
39 schema:sdDatePublished 2019-04-11T09:12
40 schema:sdLicense https://scigraph.springernature.com/explorer/license/
41 schema:sdPublisher Ncb2c911a29cf4d1cad5b82a459277617
42 schema:url http://link.springer.com/10.1007%2Fs10994-009-5110-1
43 sgo:license sg:explorer/license/
44 sgo:sdDataset articles
45 rdf:type schema:ScholarlyArticle
46 N1aad2f167e3f407a83782afa88890073 schema:name doi
47 schema:value 10.1007/s10994-009-5110-1
48 rdf:type schema:PropertyValue
49 N2b759cd3cbc24100be84dc6511dc1704 rdf:first sg:person.014330214576.44
50 rdf:rest Nc229038cbb394153a06dc41b6ff74450
51 N6470a6d06ab048fc97214019cbd0e914 schema:volumeNumber 77
52 rdf:type schema:PublicationVolume
53 N6d5b6dc621d5444082fae6af38397772 schema:issueNumber 2-3
54 rdf:type schema:PublicationIssue
55 Na71d29cb72a440f0a87478a22b609f9d schema:name readcube_id
56 schema:value a9110cc73eeabb5755f964eef1a0c16e3e734d6cbbc3e678a0ee34a89b30f709
57 rdf:type schema:PropertyValue
58 Nc229038cbb394153a06dc41b6ff74450 rdf:first sg:person.016202177221.23
59 rdf:rest rdf:nil
60 Ncb2c911a29cf4d1cad5b82a459277617 schema:name Springer Nature - SN SciGraph project
61 rdf:type schema:Organization
62 Nef239753d1c348ee816ecd170c256d59 schema:name dimensions_id
63 schema:value pub.1035491423
64 rdf:type schema:PropertyValue
65 anzsrc-for:08 schema:inDefinedTermSet anzsrc-for:
66 schema:name Information and Computing Sciences
67 rdf:type schema:DefinedTerm
68 anzsrc-for:0801 schema:inDefinedTermSet anzsrc-for:
69 schema:name Artificial Intelligence and Image Processing
70 rdf:type schema:DefinedTerm
71 sg:journal.1125588 schema:issn 0885-6125
72 1573-0565
73 schema:name Machine Learning
74 rdf:type schema:Periodical
75 sg:person.014330214576.44 schema:affiliation https://www.grid.ac/institutes/grid.4836.9
76 schema:familyName Neu
77 schema:givenName Gergely
78 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.014330214576.44
79 rdf:type schema:Person
80 sg:person.016202177221.23 schema:affiliation https://www.grid.ac/institutes/grid.17089.37
81 schema:familyName Szepesvári
82 schema:givenName Csaba
83 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.016202177221.23
84 rdf:type schema:Person
85 sg:pub.10.1007/978-3-540-74958-5_64 schema:sameAs https://app.dimensions.ai/details/publication/pub.1034739972
86 https://doi.org/10.1007/978-3-540-74958-5_64
87 rdf:type schema:CreativeWork
88 sg:pub.10.1023/a:1007662407062 schema:sameAs https://app.dimensions.ai/details/publication/pub.1033414988
89 https://doi.org/10.1023/a:1007662407062
90 rdf:type schema:CreativeWork
91 https://doi.org/10.1006/jmbi.1998.2436 schema:sameAs https://app.dimensions.ai/details/publication/pub.1030245324
92 rdf:type schema:CreativeWork
93 https://doi.org/10.1017/cbo9780511546921 schema:sameAs https://app.dimensions.ai/details/publication/pub.1098679346
94 rdf:type schema:CreativeWork
95 https://doi.org/10.1103/physrev.106.620 schema:sameAs https://app.dimensions.ai/details/publication/pub.1060418970
96 rdf:type schema:CreativeWork
97 https://doi.org/10.1109/icassp.1984.1172637 schema:sameAs https://app.dimensions.ai/details/publication/pub.1086174328
98 rdf:type schema:CreativeWork
99 https://doi.org/10.1137/1.9781611970777 schema:sameAs https://app.dimensions.ai/details/publication/pub.1098556247
100 rdf:type schema:CreativeWork
101 https://doi.org/10.1145/1015330.1015430 schema:sameAs https://app.dimensions.ai/details/publication/pub.1044181534
102 rdf:type schema:CreativeWork
103 https://doi.org/10.1145/1102351.1102464 schema:sameAs https://app.dimensions.ai/details/publication/pub.1022920538
104 rdf:type schema:CreativeWork
105 https://doi.org/10.1145/1143844.1143936 schema:sameAs https://app.dimensions.ai/details/publication/pub.1033108091
106 rdf:type schema:CreativeWork
107 https://doi.org/10.1145/1150402.1150466 schema:sameAs https://app.dimensions.ai/details/publication/pub.1000706999
108 rdf:type schema:CreativeWork
109 https://doi.org/10.1145/1273496.1273535 schema:sameAs https://app.dimensions.ai/details/publication/pub.1006158294
110 rdf:type schema:CreativeWork
111 https://doi.org/10.1145/1273496.1273598 schema:sameAs https://app.dimensions.ai/details/publication/pub.1047753055
112 rdf:type schema:CreativeWork
113 https://doi.org/10.3115/1073445.1073461 schema:sameAs https://app.dimensions.ai/details/publication/pub.1049307492
114 rdf:type schema:CreativeWork
115 https://doi.org/10.3115/1118693.1118694 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099201575
116 rdf:type schema:CreativeWork
117 https://doi.org/10.3115/1218955.1218970 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099221196
118 rdf:type schema:CreativeWork
119 https://doi.org/10.3115/1219840.1219862 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099221866
120 rdf:type schema:CreativeWork
121 https://doi.org/10.3115/1220175.1220285 schema:sameAs https://app.dimensions.ai/details/publication/pub.1099222039
122 rdf:type schema:CreativeWork
123 https://doi.org/10.7551/mitpress/7443.001.0001 schema:sameAs https://app.dimensions.ai/details/publication/pub.1111386243
124 rdf:type schema:CreativeWork
125 https://www.grid.ac/institutes/grid.17089.37 schema:alternateName University of Alberta
126 schema:name Computer and Automation Research Institute of the Hungarian Academy of Sciences, Kende utca 13-17, 1111, Budapest, Hungary
127 Department of Computing Science, University of Alberta, T6G 2E8, Edmonton, Alberta, Canada
128 rdf:type schema:Organization
129 https://www.grid.ac/institutes/grid.4836.9 schema:alternateName MTA Institute for Computer Science and Control
130 schema:name Computer and Automation Research Institute of the Hungarian Academy of Sciences, Kende utca 13-17, 1111, Budapest, Hungary
131 Department of Computing Science, Budapest University of Technology and Economics, Műegyetem rakpart 3-9, 1111, Budapest, Hungary
132 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...