double integral getting different results
Both are incorrect. The integral is zero.
To understand why, you can see that the integrand is antisymmetric in $x$ and $y$; specifically, if $$f(x,y) = \frac{y-x}{(x+y)^3},$$ then $$f(y,x) = -f(x,y).$$ So on a square region $[b, 1]^2$, the integral is always zero. Taking the limit as $b \to 0^+$ does not change this fact.
Here is how the integral should be evaluated in Mathematica:
Integrate[(y - x)/(y + x)^3, {x, b, 1}, {y, b, 1}, Assumptions -> 0 < b < 1]
The answer given is 0
. If you instead entered
Integrate[(y - x)/(y + x)^3, {x, 0, 1}, {y, 0, 1}]
You will get -1/2
, which is incorrect, but I should stress here that it is wrong not because Mathematica made a computational error, but because this expression is not the same as what you are actually trying to evaluate! That is to say, $$\int_{x=0}^1 \int_{y=0}^1 \frac{y-x}{(x+y)^3} \, dy \, dx \ne \lim_{b \to 0^+} \int_{x=b}^1 \int_{y=b}^1 \frac{y-x}{(x+y)^3} \, dy \, dx.$$ To give you an sense of why this is the case, try evaluating $$\int_{y=0}^1 \int_{x=0}^1 \frac{y-x}{(x+y)^3} \, dx \, dy.$$ If you do this in Mathematica, the result is 1/2
. The integrand does not satisfy Fubini's theorem.