I'm trying FullBatchLBFGS with wolfe line search on a fitting task of a small dataset.

1. Levenbert-Marquardt is giving me fairly accuracy. Can I expect LBFGS provide similar accuracy as LM? Say, for a synthesized dataset, y=1/x
2. I'm getting lots of "Line search failed; curvature pair update skipped". How could I avoid this?
3. I noticed that the result is stochastic. Where does randomness come from since I'm using a full batch training (no shuffling).

Thanks!