On the limited memory BFGS method for large scale optimization View Full Text


Ontology type: schema:ScholarlyArticle      Open Access: True


Article Info

DATE

1989-08

AUTHORS

Dong C. Liu, Jorge Nocedal

ABSTRACT

We study the numerical performance of a limited memory quasi-Newton method for large scale optimization, which we call the L-BFGS method. We compare its performance with that of the method developed by Buckley and LeNir (1985), which combines cycles of BFGS steps and conjugate direction steps. Our numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence. We show that the L-BFGS method can be greatly accelerated by means of a simple scaling. We then compare the L-BFGS method with the partitioned quasi-Newton method of Griewank and Toint (1982a). The results show that, for some problems, the partitioned quasi-Newton method is clearly superior to the L-BFGS method. However we find that for other problems the L-BFGS method is very competitive due to its low iteration cost. We also study the convergence properties of the L-BFGS method, and prove global convergence on uniformly convex problems. More... »

PAGES

503-528

References to SciGraph publications

Journal

TITLE

Mathematical Programming

ISSUE

1-3

VOLUME

45

Author Affiliations

Identifiers

URI

http://scigraph.springernature.com/pub.10.1007/bf01589116

DOI

http://dx.doi.org/10.1007/bf01589116

DIMENSIONS

https://app.dimensions.ai/details/publication/pub.1022481421


Indexing Status Check whether this publication has been indexed by Scopus and Web Of Science using the SN Indexing Status Tool
Incoming Citations Browse incoming citations for this publication using opencitations.net

JSON-LD is the canonical representation for SciGraph data.

TIP: You can open this SciGraph record using an external JSON-LD service: JSON-LD Playground Google SDTT

[
  {
    "@context": "https://springernature.github.io/scigraph/jsonld/sgcontext.json", 
    "about": [
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/0103", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Numerical and Computational Mathematics", 
        "type": "DefinedTerm"
      }, 
      {
        "id": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/01", 
        "inDefinedTermSet": "http://purl.org/au-research/vocabulary/anzsrc-for/2008/", 
        "name": "Mathematical Sciences", 
        "type": "DefinedTerm"
      }
    ], 
    "author": [
      {
        "affiliation": {
          "alternateName": "Northwestern University", 
          "id": "https://www.grid.ac/institutes/grid.16753.36", 
          "name": [
            "Department of Electrical Engineering and Computer Science, Northwestern University, 60208, Evanston, IL, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Liu", 
        "givenName": "Dong C.", 
        "id": "sg:person.012661412223.39", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012661412223.39"
        ], 
        "type": "Person"
      }, 
      {
        "affiliation": {
          "alternateName": "Northwestern University", 
          "id": "https://www.grid.ac/institutes/grid.16753.36", 
          "name": [
            "Department of Electrical Engineering and Computer Science, Northwestern University, 60208, Evanston, IL, USA"
          ], 
          "type": "Organization"
        }, 
        "familyName": "Nocedal", 
        "givenName": "Jorge", 
        "id": "sg:person.01157306714.71", 
        "sameAs": [
          "https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01157306714.71"
        ], 
        "type": "Person"
      }
    ], 
    "citation": [
      {
        "id": "sg:pub.10.1007/bf01593790", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1006434394", 
          "https://doi.org/10.1007/bf01593790"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02591943", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1011068366", 
          "https://doi.org/10.1007/bf02591943"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf02591943", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1011068366", 
          "https://doi.org/10.1007/bf02591943"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/214392.214395", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1013767115"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1090/s0025-5718-1980-0572855-7", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1016159641"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf01583777", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1017260290", 
          "https://doi.org/10.1007/bf01583777"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf01407874", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1020894237", 
          "https://doi.org/10.1007/bf01407874"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bfb0099526", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1022656138", 
          "https://doi.org/10.1007/bfb0099526"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/355921.355933", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1024229164"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf01609018", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1027194903", 
          "https://doi.org/10.1007/bf01609018"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf01609018", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1027194903", 
          "https://doi.org/10.1007/bf01609018"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf01399316", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1039339277", 
          "https://doi.org/10.1007/bf01399316"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf01399316", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1039339277", 
          "https://doi.org/10.1007/bf01399316"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1145/355934.355936", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1043056536"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1090/s0025-5718-1978-0483452-7", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1048465494"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "sg:pub.10.1007/bf01588962", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1048725219", 
          "https://doi.org/10.1007/bf01588962"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1093/comjnl/12.2.171", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1052839649"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1137/0715085", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1062852557"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1137/0716059", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1062852620"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1137/0720042", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1062852930"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1137/0726042", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1062853446"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1137/0906042", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1062855754"
        ], 
        "type": "CreativeWork"
      }, 
      {
        "id": "https://doi.org/10.1287/moor.3.3.244", 
        "sameAs": [
          "https://app.dimensions.ai/details/publication/pub.1064724429"
        ], 
        "type": "CreativeWork"
      }
    ], 
    "datePublished": "1989-08", 
    "datePublishedReg": "1989-08-01", 
    "description": "We study the numerical performance of a limited memory quasi-Newton method for large scale optimization, which we call the L-BFGS method. We compare its performance with that of the method developed by Buckley and LeNir (1985), which combines cycles of BFGS steps and conjugate direction steps. Our numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence. We show that the L-BFGS method can be greatly accelerated by means of a simple scaling. We then compare the L-BFGS method with the partitioned quasi-Newton method of Griewank and Toint (1982a). The results show that, for some problems, the partitioned quasi-Newton method is clearly superior to the L-BFGS method. However we find that for other problems the L-BFGS method is very competitive due to its low iteration cost. We also study the convergence properties of the L-BFGS method, and prove global convergence on uniformly convex problems.", 
    "genre": "research_article", 
    "id": "sg:pub.10.1007/bf01589116", 
    "inLanguage": [
      "en"
    ], 
    "isAccessibleForFree": true, 
    "isPartOf": [
      {
        "id": "sg:journal.1047630", 
        "issn": [
          "0025-5610", 
          "1436-4646"
        ], 
        "name": "Mathematical Programming", 
        "type": "Periodical"
      }, 
      {
        "issueNumber": "1-3", 
        "type": "PublicationIssue"
      }, 
      {
        "type": "PublicationVolume", 
        "volumeNumber": "45"
      }
    ], 
    "name": "On the limited memory BFGS method for large scale optimization", 
    "pagination": "503-528", 
    "productId": [
      {
        "name": "readcube_id", 
        "type": "PropertyValue", 
        "value": [
          "6b1b274556c3a6861d2845c8284ac0d884ba1d354e22483f879036b5e6445113"
        ]
      }, 
      {
        "name": "doi", 
        "type": "PropertyValue", 
        "value": [
          "10.1007/bf01589116"
        ]
      }, 
      {
        "name": "dimensions_id", 
        "type": "PropertyValue", 
        "value": [
          "pub.1022481421"
        ]
      }
    ], 
    "sameAs": [
      "https://doi.org/10.1007/bf01589116", 
      "https://app.dimensions.ai/details/publication/pub.1022481421"
    ], 
    "sdDataset": "articles", 
    "sdDatePublished": "2019-04-10T17:35", 
    "sdLicense": "https://scigraph.springernature.com/explorer/license/", 
    "sdPublisher": {
      "name": "Springer Nature - SN SciGraph project", 
      "type": "Organization"
    }, 
    "sdSource": "s3://com-uberresearch-data-dimensions-target-20181106-alternative/cleanup/v134/2549eaecd7973599484d7c17b260dba0a4ecb94b/merge/v9/a6c9fde33151104705d4d7ff012ea9563521a3ce/jats-lookup/v90/0000000001_0000000264/records_8672_00000532.jsonl", 
    "type": "ScholarlyArticle", 
    "url": "http://link.springer.com/10.1007%2FBF01589116"
  }
]
 

Download the RDF metadata as:  json-ld nt turtle xml License info

HOW TO GET THIS DATA PROGRAMMATICALLY:

JSON-LD is a popular format for linked data which is fully compatible with JSON.

curl -H 'Accept: application/ld+json' 'https://scigraph.springernature.com/pub.10.1007/bf01589116'

N-Triples is a line-based linked data format ideal for batch operations.

curl -H 'Accept: application/n-triples' 'https://scigraph.springernature.com/pub.10.1007/bf01589116'

Turtle is a human-readable linked data format.

curl -H 'Accept: text/turtle' 'https://scigraph.springernature.com/pub.10.1007/bf01589116'

RDF/XML is a standard XML format for linked data.

curl -H 'Accept: application/rdf+xml' 'https://scigraph.springernature.com/pub.10.1007/bf01589116'


 

This table displays all metadata directly associated to this object as RDF triples.

136 TRIPLES      21 PREDICATES      47 URIs      19 LITERALS      7 BLANK NODES

Subject Predicate Object
1 sg:pub.10.1007/bf01589116 schema:about anzsrc-for:01
2 anzsrc-for:0103
3 schema:author Ndec30e17ad1245c4b9b1ee8a3c9a4c02
4 schema:citation sg:pub.10.1007/bf01399316
5 sg:pub.10.1007/bf01407874
6 sg:pub.10.1007/bf01583777
7 sg:pub.10.1007/bf01588962
8 sg:pub.10.1007/bf01593790
9 sg:pub.10.1007/bf01609018
10 sg:pub.10.1007/bf02591943
11 sg:pub.10.1007/bfb0099526
12 https://doi.org/10.1090/s0025-5718-1978-0483452-7
13 https://doi.org/10.1090/s0025-5718-1980-0572855-7
14 https://doi.org/10.1093/comjnl/12.2.171
15 https://doi.org/10.1137/0715085
16 https://doi.org/10.1137/0716059
17 https://doi.org/10.1137/0720042
18 https://doi.org/10.1137/0726042
19 https://doi.org/10.1137/0906042
20 https://doi.org/10.1145/214392.214395
21 https://doi.org/10.1145/355921.355933
22 https://doi.org/10.1145/355934.355936
23 https://doi.org/10.1287/moor.3.3.244
24 schema:datePublished 1989-08
25 schema:datePublishedReg 1989-08-01
26 schema:description We study the numerical performance of a limited memory quasi-Newton method for large scale optimization, which we call the L-BFGS method. We compare its performance with that of the method developed by Buckley and LeNir (1985), which combines cycles of BFGS steps and conjugate direction steps. Our numerical tests indicate that the L-BFGS method is faster than the method of Buckley and LeNir, and is better able to use additional storage to accelerate convergence. We show that the L-BFGS method can be greatly accelerated by means of a simple scaling. We then compare the L-BFGS method with the partitioned quasi-Newton method of Griewank and Toint (1982a). The results show that, for some problems, the partitioned quasi-Newton method is clearly superior to the L-BFGS method. However we find that for other problems the L-BFGS method is very competitive due to its low iteration cost. We also study the convergence properties of the L-BFGS method, and prove global convergence on uniformly convex problems.
27 schema:genre research_article
28 schema:inLanguage en
29 schema:isAccessibleForFree true
30 schema:isPartOf Naae1a02577d2411cb088fc70fb2e6d1d
31 Ne991315e0a844b74aa6463fd6ee49b72
32 sg:journal.1047630
33 schema:name On the limited memory BFGS method for large scale optimization
34 schema:pagination 503-528
35 schema:productId N2d0542a2d07f4188a4464124391a4337
36 N681f7315a29d43bbba418f6657216771
37 N8d75d28a91c24757a67c5ce7f7604565
38 schema:sameAs https://app.dimensions.ai/details/publication/pub.1022481421
39 https://doi.org/10.1007/bf01589116
40 schema:sdDatePublished 2019-04-10T17:35
41 schema:sdLicense https://scigraph.springernature.com/explorer/license/
42 schema:sdPublisher N28fd3f372bed4720a301233a369d2c5f
43 schema:url http://link.springer.com/10.1007%2FBF01589116
44 sgo:license sg:explorer/license/
45 sgo:sdDataset articles
46 rdf:type schema:ScholarlyArticle
47 N28fd3f372bed4720a301233a369d2c5f schema:name Springer Nature - SN SciGraph project
48 rdf:type schema:Organization
49 N2d0542a2d07f4188a4464124391a4337 schema:name dimensions_id
50 schema:value pub.1022481421
51 rdf:type schema:PropertyValue
52 N681f7315a29d43bbba418f6657216771 schema:name doi
53 schema:value 10.1007/bf01589116
54 rdf:type schema:PropertyValue
55 N6dcd5c469a8040579b6b13bf1f41c4b2 rdf:first sg:person.01157306714.71
56 rdf:rest rdf:nil
57 N8d75d28a91c24757a67c5ce7f7604565 schema:name readcube_id
58 schema:value 6b1b274556c3a6861d2845c8284ac0d884ba1d354e22483f879036b5e6445113
59 rdf:type schema:PropertyValue
60 Naae1a02577d2411cb088fc70fb2e6d1d schema:volumeNumber 45
61 rdf:type schema:PublicationVolume
62 Ndec30e17ad1245c4b9b1ee8a3c9a4c02 rdf:first sg:person.012661412223.39
63 rdf:rest N6dcd5c469a8040579b6b13bf1f41c4b2
64 Ne991315e0a844b74aa6463fd6ee49b72 schema:issueNumber 1-3
65 rdf:type schema:PublicationIssue
66 anzsrc-for:01 schema:inDefinedTermSet anzsrc-for:
67 schema:name Mathematical Sciences
68 rdf:type schema:DefinedTerm
69 anzsrc-for:0103 schema:inDefinedTermSet anzsrc-for:
70 schema:name Numerical and Computational Mathematics
71 rdf:type schema:DefinedTerm
72 sg:journal.1047630 schema:issn 0025-5610
73 1436-4646
74 schema:name Mathematical Programming
75 rdf:type schema:Periodical
76 sg:person.01157306714.71 schema:affiliation https://www.grid.ac/institutes/grid.16753.36
77 schema:familyName Nocedal
78 schema:givenName Jorge
79 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.01157306714.71
80 rdf:type schema:Person
81 sg:person.012661412223.39 schema:affiliation https://www.grid.ac/institutes/grid.16753.36
82 schema:familyName Liu
83 schema:givenName Dong C.
84 schema:sameAs https://app.dimensions.ai/discover/publication?and_facet_researcher=ur.012661412223.39
85 rdf:type schema:Person
86 sg:pub.10.1007/bf01399316 schema:sameAs https://app.dimensions.ai/details/publication/pub.1039339277
87 https://doi.org/10.1007/bf01399316
88 rdf:type schema:CreativeWork
89 sg:pub.10.1007/bf01407874 schema:sameAs https://app.dimensions.ai/details/publication/pub.1020894237
90 https://doi.org/10.1007/bf01407874
91 rdf:type schema:CreativeWork
92 sg:pub.10.1007/bf01583777 schema:sameAs https://app.dimensions.ai/details/publication/pub.1017260290
93 https://doi.org/10.1007/bf01583777
94 rdf:type schema:CreativeWork
95 sg:pub.10.1007/bf01588962 schema:sameAs https://app.dimensions.ai/details/publication/pub.1048725219
96 https://doi.org/10.1007/bf01588962
97 rdf:type schema:CreativeWork
98 sg:pub.10.1007/bf01593790 schema:sameAs https://app.dimensions.ai/details/publication/pub.1006434394
99 https://doi.org/10.1007/bf01593790
100 rdf:type schema:CreativeWork
101 sg:pub.10.1007/bf01609018 schema:sameAs https://app.dimensions.ai/details/publication/pub.1027194903
102 https://doi.org/10.1007/bf01609018
103 rdf:type schema:CreativeWork
104 sg:pub.10.1007/bf02591943 schema:sameAs https://app.dimensions.ai/details/publication/pub.1011068366
105 https://doi.org/10.1007/bf02591943
106 rdf:type schema:CreativeWork
107 sg:pub.10.1007/bfb0099526 schema:sameAs https://app.dimensions.ai/details/publication/pub.1022656138
108 https://doi.org/10.1007/bfb0099526
109 rdf:type schema:CreativeWork
110 https://doi.org/10.1090/s0025-5718-1978-0483452-7 schema:sameAs https://app.dimensions.ai/details/publication/pub.1048465494
111 rdf:type schema:CreativeWork
112 https://doi.org/10.1090/s0025-5718-1980-0572855-7 schema:sameAs https://app.dimensions.ai/details/publication/pub.1016159641
113 rdf:type schema:CreativeWork
114 https://doi.org/10.1093/comjnl/12.2.171 schema:sameAs https://app.dimensions.ai/details/publication/pub.1052839649
115 rdf:type schema:CreativeWork
116 https://doi.org/10.1137/0715085 schema:sameAs https://app.dimensions.ai/details/publication/pub.1062852557
117 rdf:type schema:CreativeWork
118 https://doi.org/10.1137/0716059 schema:sameAs https://app.dimensions.ai/details/publication/pub.1062852620
119 rdf:type schema:CreativeWork
120 https://doi.org/10.1137/0720042 schema:sameAs https://app.dimensions.ai/details/publication/pub.1062852930
121 rdf:type schema:CreativeWork
122 https://doi.org/10.1137/0726042 schema:sameAs https://app.dimensions.ai/details/publication/pub.1062853446
123 rdf:type schema:CreativeWork
124 https://doi.org/10.1137/0906042 schema:sameAs https://app.dimensions.ai/details/publication/pub.1062855754
125 rdf:type schema:CreativeWork
126 https://doi.org/10.1145/214392.214395 schema:sameAs https://app.dimensions.ai/details/publication/pub.1013767115
127 rdf:type schema:CreativeWork
128 https://doi.org/10.1145/355921.355933 schema:sameAs https://app.dimensions.ai/details/publication/pub.1024229164
129 rdf:type schema:CreativeWork
130 https://doi.org/10.1145/355934.355936 schema:sameAs https://app.dimensions.ai/details/publication/pub.1043056536
131 rdf:type schema:CreativeWork
132 https://doi.org/10.1287/moor.3.3.244 schema:sameAs https://app.dimensions.ai/details/publication/pub.1064724429
133 rdf:type schema:CreativeWork
134 https://www.grid.ac/institutes/grid.16753.36 schema:alternateName Northwestern University
135 schema:name Department of Electrical Engineering and Computer Science, Northwestern University, 60208, Evanston, IL, USA
136 rdf:type schema:Organization
 




Preview window. Press ESC to close (or click here)


...