When is it possible to have $f(x+y)=f(x)+f(y)+g(xy)$?

If $f$ is not differentiable, we still have $f((x+y)+z)=f(x+(y+z))$, from which follows

$$g(xy)+g(xz+yz)=g(xy+xz)+g(yz)\tag{1}$$

By change of variables

$$x=\sqrt{\frac{ab}{c}}, \quad y=\sqrt{\frac{ca}{b}}, \quad z=\sqrt{\frac{bc}{a}}\tag{2}$$

this becomes

$$g(a)+g(b+c)=g(a+b)+g(c)\tag{3}$$

whenever $abc>0$.

By change of variables $(a,b,c)\to(a+b,-b,b+c)$ we can also move $-b$ across so it holds for all $abc\neq0$. Then

$$g(a+b)+g(b-b)=g(a)+g(b)\tag{4}$$

so $h(x)=g(x)-g(0)$ satisfies $$h(a+b)=h(a)+h(b)\tag{5}$$ This has the well-known solutions $h(x)=kx$, and some very discontinuous ones.


If $f$ is differentiable then taking the derivative with respect to $x$ we get

$$f'(x+y)=f'(x)+yg'(xy).$$ Now, taking the derivative with respect to $y$ one has $$f''(x+y)=g'(xy)+xyg''(xy).$$ If $y=-x$ then $$g'(-x^2)-x^2g''(-x^2)=f''(0).$$ If we denote $t=-x^2$ we have $$tg''(t)+g'(t)-f''(0)=0.$$ Solving the ODE we get $$g(t)=f''(0)t+a\ln t+b,$$ where $a,b$ are arbitrary constants. If we want that $g$ is defined for all $t$ it must be $a=0.$ So $$g(t)=f''(0)t+b.$$ Now, from $$f''(x+y)=g'(xy)+xyg''(xy)$$ we get that $$f''(x+y)=f''(0).$$ So the second derivative of $f$ must be constant. That is, $f$ is a second degree polynomial. It only remain which polynomials of degree two can be a solution. It is a straightforward computation to check that any $f(x)=ax^2+bx+c$ is a solution with $g(x)=2ax-c.$