Prove inequalities $\frac 34 \le I(a,b) \le 1$

Putting everything over a common denominator shows that

$$I(a,b) = \frac{a^2+b^2+a^2b+ab^2+a+b}{a^2+b^2+a^2b+ab^2+a+b+2ab} = \frac{(1+a)(1+b)(a+b)-2ab}{(1+a)(1+b)(a+b)} \leq 1$$

since $a,b\geq 0$. Simplifying this expression, we get

$$I(a,b) = 1 - \frac{2ab}{(1+a)(1+b)(a+b)}$$

Now take a look at the second term and consider it rewritten in the following way:

$$\frac{2ab}{(1+a)(1+b)(a+b)} = \frac{\frac{ab}{4}}{\frac{a^2+b^2+a^2b+ab^2+a+b+ab+ab}{8}}$$

then by AM-GM inequality

$$\frac{a^2+b^2+a^2b+ab^2+a+b+ab+ab}{8} \geq (a^8b^8)^{\frac{1}{8}} = ab$$

$$\implies \frac{2ab}{(1+a)(1+b)(a+b)} \leq \frac{\frac{ab}{4}}{ab} = \frac{1}{4}$$

which means that

$$I(a,b) \geq 1 -\frac{1}{4} = \frac{3}{4}$$


I think, it means that $a+b>0$, otherwise your inequality is wrong for $a=b=0$.

Let $a+b=2x$.

Thus, by C-S and AM-GM we obtain: $$I(a,b)=\frac{1}{a+b}\left(\frac{a^2}{1+a}+\frac{b^2}{1+b}\right)+\frac{1}{1+a+b+ab}\geq$$ $$\geq\frac{1}{2x}\cdot\frac{(a+b)^2}{1+a+1+b}+\frac{1}{1+2x+\left(\frac{a+b}{2}\right)^2}=$$ $$=\frac{x}{1+x}+\frac{1}{(x+1)^2}=\frac{3}{4}+\frac{1}{4}-\frac{1}{1+x}+\frac{1}{(1+x)^2}=$$ $$=\frac{3}{4}+\left(\frac{1}{2}-\frac{1}{1+x}\right)^2\geq\frac{3}{4}.$$

Also, $I(a,b)\leq1$ it's $$a^2(1+b)+b^2(1+a)+a+b\leq(1+a)(1+b)(a+b)$$ or $$ab\geq0$$ and we are done!