Hi John, Thanks so much for the informative reply! I'm currently trying to optimize ~10,000 parameters simultaneously - for some reason, when I compare the memory usage for L-BFGS-B and BFGS, the L-BFGS-B only uses about 1/7 of the memory, with all default input parameters, I'm a bit surprised that it isn't a lot less, but BFGS is definitely converging a lot slower.
My other question is that, L-BFGS-B is returning 'non-finite' errors with respect to the gradient function I'm supplying, because again, all the parameters i'm optimizing need to be non-negative (so i'm optimizing the log of the parameters), but the gradient at some point divides by each parameter, so when some of the parameters go to 0, the gradient becomes infinite. Do you (or anyone else) have any suggestions for how to prevent this? Is the only way to force the parameters to be larger than some number close to 0 (i.e. 1e-10), or modify the gradient function to set the entry of small parameters to 0? Thanks! Brian. On Fri, Feb 25, 2011 at 10:51 AM, Prof. John C Nash <nas...@uottawa.ca>wrote: > There are considerable differences between the algorithms. And BFGS is an > unfortunate > nomenclature, since there are so many variants that are VERY different. It > was called > "variable metric" in my book from which the code was derived, and that code > was from Roger > Fletcher's Fortran VM code based on his 1970 paper. L-BFGS-B is a later and > more > complicated algorithm with some pretty nice properties. The code is much > larger. > > Re: "less memory" -- this will depend on the number of parameters, but to > my knowledge > there are no good benchmark studies of memory and performance. Perhaps > someone wants to > propose one for Google Summer of Code (see > http://rwiki.sciviews.org/doku.php?id=developers:projects:gsoc2011 > ). > > The optimx package can call Rvmmin which has box constraints (also Rcgmin > that is intended > for very low memory). Also several other methods with box constraints, > including L-BFGS-B. > Worth a try if you are seeking a method for multiple "production" runs. > Unfortunately, we > seem to have some CRAN check errors on Solaris and some old releases -- > platforms I do not > have -- so it may be a few days or more until we sort out the issues, which > seem to be > related to alignment of the underlying packages for which optimx is a > wrapper. > > Use of transformation can be very effective. But again, I don't think there > are good > studies on whether use of box constraints or transformations is "better" > and when. Another > project, which I have made some tentative beginings to carry out. > Collaborations welcome. > > Best, > > JN > > > On 02/25/2011 06:00 AM, r-help-requ...@r-project.org wrote: > > Message: 86 > > Date: Fri, 25 Feb 2011 00:11:59 -0500 > > From: Brian Tsai <btsa...@gmail.com> > > To: r-help@r-project.org > > Subject: [R] BFGS versus L-BFGS-B > > Message-ID: > > <aanlktimszvkjbuhv-bbr1easpx9ootjxqcujgujr5...@mail.gmail.com> > > Content-Type: text/plain > > > > Hi all, > > > > I'm trying to figure out the effective differences between BFGS and > L-BFGS-B > > are, besides the obvious that L-BFGS-B should be using a lot less memory, > > and the user can provide box constraints. > > > > 1) Why would you ever want to use BFGS, if L-BFGS-B does the same thing > but > > use less memory? > > > > 2) If i'm optimizing with respect to a variable x that must be > non-negative, > > a common approach is to do a change of variables x = exp(y), and optimize > > unconstrained with respect to y. Is optimization using box constraints > on > > x, likely to produce as good a result as unconstrained optimization on y? > > > > - Brian. > > > > [[alternative HTML version deleted]] > > > > > > > > ______________________________________________ > R-help@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/r-help > PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html > and provide commented, minimal, self-contained, reproducible code. > [[alternative HTML version deleted]] ______________________________________________ R-help@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide http://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
