If you examine the operation of NonlinearModelFit
you find that the model is evaluated with the same parameters again and again. I have a very expensive model so this wastes much time.
This is the example from Help in version 12.1.
data = {{6.47, 3.65}, {7.43,
3.45}, {3.9, 2.94}, {4.8, 1.29}, {2.48, 0.35}, {6.32,
3.16}, {2.59, 1.19}, {9.13, 2.}, {3.81, 3.04}, {3.33, 2.68}};
ClearAll[model];
model[a_?NumberQ, b_?NumberQ, c_?NumberQ] := Module[{y, x},
Sow[{a, b, c}];
First[y /.
NDSolve[{y''[x] + a y[x] == 0, y[0] == b, y'[0] == c},
y, {x, 0, 10}]]]
t1 = Timing[
r1 = Reap[
nlm1 = NonlinearModelFit[data, model[a, b, c][x], {a, b, c}, x,
Method > "Gradient"]];]
(* {4.375, Null} *)
I have added a Sow
and Reap
so we can find the number of evaluations. Looking at the number of terms that were reaped.
L1 = Length[r1[[2, 1]]]
(* 6280 *)
If we look at the first 100 values they all seem to be the same. However, they might be almost similar so I eliminate identical terms using Union
Union[r1[[2, 1, 1 ;; 100]]] // InputForm
(* {{1., 1., 1.}, {1., 1., 1.0000000149011612},
{1., 1.0000000149011612, 1.}, {1.0000000149011612, 1., 1.}} *)
This might need a closer look because we are looking at small differences, however, it does seem that only 4 evaluations were different. If there is repeat evaluation then memoization would seem to be a good idea. I implement this as follows:
ClearAll[model];
model[a_?NumberQ, b_?NumberQ, c_?NumberQ] :=
model[a, b, c] = Module[{y, x},
Sow[{a, b, c}];
First[y /.
NDSolve[{y''[x] + a y[x] == 0, y[0] == b, y'[0] == c},
y, {x, 0, 10}]]]
t2 = Timing[
r2 = Reap[
nlm2 = NonlinearModelFit[data, model[a, b, c][x], {a, b, c}, x,
Method > "Gradient"]];]
(* {0.1875, Null} *)
The number of function evaluations is now
L2 = Length[r2[[2, 1]]]
(* 227 *)
Thus a much smaller number of function evaluations and the calculation runs much faster. The ratio of time and function evaluations is
{t1[[1]]/t2[[1]], L1/L2 // N}
(* {23.3333, 27.6652} *)
This is a speed up of times 20. So it seems that NonlinearModelFit
is doing useless function evaluations and that memoization makes a huge difference.
My questions are:

Am I missing something here?

Why does
NonlinearModelFit
do useless function evaluations?
Thanks