Mathematica rule substitution memory

While I am not able to explain why there is a difference between the two methods, I would like to suggest using

t = Block[{x=0, y=0}, p];

Block will set a value to x and y only temporarily, and avoids the use of Clear.

Some more things that can be useful for you:

1. When you start working, use $HistoryLength = 0 to avoid remembering past results and wasting memory (note that Mathematica sometimes remembers full results even when the input was ended in a semicolon!)

2. Use Share[] occasionally. It seems that with your polynomial this will help a lot more than in other cases. After reading in the polynomial, it frees 60 MB for me immediately.

The basic issue is that when using:

expr /. x->1

Mathematica will need to examine the entire expression tree to determine what parts can be replaced. After replacing all of these parts, it then evaluates the result. Contrast this with using a Block approach:

Block[{x=1}, expr]

In this case, Mathematica sets x=1 and then evaluates expr. When x is encountered during the evaluation of expr, it evaluates to 1. In contrast to using ReplaceAll, Mathematica does not look for parts that match x. So, it is not surprising that the Block approach can be much faster.

Here are three more observations:

Evaluation

With the Block approach, x only gets replaced with 1 when the evaluator encounters x. Here is an example where in the Block approach the evaluator never sees x:

Hold[x] /. x->1
Block[{x=1}, Hold[x]]

Hold[1]

Hold[x]

Rules with patterns

If the LHS of the rule is a pattern and not a literal, then the naive Block approach won't work.

{x[1], x[2.2]} /. x[_Integer] -> 1
Block[{x[_Integer]=1}, {x[1], x[2.2]}]

{1, x[2.2]}

Block::lvset: Local variable specification {x[_Integer]=1} contains x[_Integer]=1, which is an assignment to x[_Integer]; only assignments to symbols are allowed.

Block[{x[_Integer] = 1}, {x[1], x[2.2]}]

The Block approach can be made to work by doing something like:

Block[{x}, x[_Integer]=1; {x[1], x[2.2]}]

{1, x[2.2]}

but one should know what they are doing, as differences in how DownValues and ReplaceAll/Replace work, as well as possible evaluation issues complicate things.

Replace

The Rule approach can be sped up by only doing replacements at the desired level. For example:

poly = Expand[(x+y+z)^100];
r1 = poly /. {x->1, y->2, z->3}; //RepeatedTiming
r2 = Replace[poly, {x->1, y->2, z->3}, {-1}]; //RepeatedTiming
r3 = Block[{x=1, y=2, z=3}, poly]; //RepeatedTiming

{0.024, Null}

{0.015, Null}

{0.0099, Null}

By using Replace with level {-1}, only the leaves are replaced, while using ReplaceAll, all parts of the expression are replaced, if possible.

I have solved this problem, in a very simple way, as I will demonstrate here:

 In[1]:= f = 3 x + 7 y
 Out[1]= 3 x + 7 y
 In[2]:= x = 0;
 In[3]:= t = f
 Out[3]= 7 y
 In[4]:= Clear[x];
 In[5]:= t
 Out[5]= 7 y
 In[6]:= f
 Out[6]= 3 x + 7 y

Now I only wonder why Mathematica's substitute command does not use this method for substitutions. I'm sure there are situations where more complicated substitution rules may fail with this approach, but in that case some heuristics should determine if this much faster and less RAM intense method would be applicable and, if so, apply it.