How do I simplify expressions with square roots?

There is a literature on denesting radicals. Not really my strength. In cases where you know (or suspect) the specific radicals that should appear in the result you can use a Groebner basis computation to recast your algebraic value via a minimal polynomial. Then factor over the extension defined by those radicals, solve, and pick the solution that is numerically correct (the others being algebraic conjugates). I illustrate with that first example.

rootpoly = 
 GroebnerBasis[{x*xrecip - 1, xrecip - (1 + y - z), y^2 - 2, z^2 - 3},
    x, {xrecip, y, z}][[1]]

(* -1 + 4*x + 4*x^2 - 16*x^3 + 8*x^4 *)

fax = 
 Select[FactorList[rootpoly, Extension -> {Sqrt[2], Sqrt[3]}][[All, 
    1]], ! FreeQ[#, x] &]

(* {2 + Sqrt[2] + Sqrt[6] - 4*x, -2 - Sqrt[2] + Sqrt[6] + 4*x, 
   2 - Sqrt[2] + Sqrt[6] - 4*x, -2 + Sqrt[2] + Sqrt[6] + 4*x} *)

candidates = Flatten[Map[x /. Solve[# == 0, x] &, fax]];
First[Select[candidates, (N[#] == 1/(1 + Sqrt[2] - Sqrt[3])) &]]

(* (1/4)*(2 + Sqrt[2] + Sqrt[6]) *)

If you are more familiar with manipulating algebraic numbers than you are with Groebner bases, here is a better way to get that defining polynomial.

RootReduce[1/(1 + Sqrt[2] - Sqrt[3])][[1]][x]

(* Out[35]= -1 + 4 x + 4 x^2 - 16 x^3 + 8 x^4 *)

--- edit ---

I will show this in a way that is more automated, in terms of deciding what to use in the extension for factoring. The idea is to allow roots of all factors of all integers that appear in the nested radical.

val = Sqrt[4 + Sqrt[15]];
rootpoly = RootReduce[val][[1]][x]

(* 1 - 8 x^2 + x^4 *)

ints = 
 Flatten[Map[FactorInteger, Cases[val, _Integer, -1]][[All, All, 1]]]

(* {2, 3, 5} *)

fax = 
 Select[FactorList[rootpoly, Extension -> Sqrt[ints]][[All, 1]], ! 
    FreeQ[#, x] &]

(* {Sqrt[6] + Sqrt[10] - 2 x, Sqrt[6] - Sqrt[10] + 2 x, 
 Sqrt[6] - Sqrt[10] - 2 x, Sqrt[6] + Sqrt[10] + 2 x} *)

candidates = Flatten[Map[x /. Solve[# == 0, x] &, fax]];
First[Select[candidates, (N[#] == val) &]]

(* 1/2 (Sqrt[6] + Sqrt[10] *)

--- end edit ---


Mathematica uses a ComplexityFunction to measure how 'simple' an expression is. According to the documentation:

ComplexityFunction counts the subexpressions and digits of integers

What we need to do is create a ComplexityFunction that considers the expressions you desire to be very cheap. Depth works for your middle expression

f[x_] := Depth[x];
FullSimplify[{1/(1 + Sqrt[2] - Sqrt[3]), Sqrt[5 - 2*Sqrt[6]], 
  Sqrt[4 + Sqrt[15]]}, ComplexityFunction -> f]

gives

enter image description here

Sadly, the expression you want has the same depth as the expression you started with

Depth[Sqrt[4 + Sqrt[15]]]

4

 Depth[1/2 (Sqrt[6] + Sqrt[10])]

 4

I'm afraid that I couldn't find anything that would Simplify that final expression.


Input#1:

a + b == (Sqrt[x] + Sqrt[y])^2 == x + y + 2 Sqrt[x*y];
Solve[{x + y == a, 4 x*y == b^2}, {x, y}]
(*
{{x -> 1/2 (a - Sqrt[a^2 - b^2]), y -> 1/2 (a + Sqrt[a^2 - b^2])}, 
{x -> 1/2 (a + Sqrt[a^2 - b^2]), 
y -> 1/2 (a - Sqrt[a^2 - b^2])}}*)

Define function:

f[a_, b_] := 
 Simplify[Sqrt[1/2 (a - Sqrt[a^2 - b^2])] + 
 Sqrt[1/2 (a + Sqrt[a^2 - b^2])]]

Input#2:

f[5, 2 Sqrt[6]]
f[4, Sqrt[15]]

Output#2:

Sqrt[2] + Sqrt[3]
Sqrt[3/2] + Sqrt[5/2]

The Simplify of 1/(1 + Sqrt[2] - Sqrt[3])

Expand[(1 + Sqrt[2] - Sqrt[3]) (1 + Sqrt[2] + Sqrt[3])]
(*2 Sqrt[2]*)
FullSimplify[(Sqrt[2] + 1 + Sqrt[3])/(2 Sqrt[2])]
(*1/4 (2 + Sqrt[2] + Sqrt[6])*)

Picture: The simplify of 1/(b + Sqrt[a] - Sqrt[c])