How find this maximum $S_{\Delta ABC}$

The maximal area is $36 + 22\sqrt 3$, attained by a triangle with $$ AB = \sqrt{\frac{8128+3840\sqrt3}{73}}, \quad BC = \sqrt{\frac{5323+3024\sqrt3}{73}}. $$

Let $a$ and $c$ be the sides $BC$, $AB$ of the triangle. By the Law of Cosines, side $AC$ has length $\sqrt{a^2-ac+c^2}$. Now any four points $A,B,C,P$ in the plane determine six distances that satisfy an algebraic relation, the vanishing of the Cayley-Menger determinant $$ \det\left| \begin{array}{c&c&c&c&c} 0 & 1 & 1 & 1 & 1 \\ 1 & 0 & AB^2 & AC^2 & AP^2 \\ 1 & AB^2 & 0 & BC^2 & BP^2 \\ 1 & AC^2 & BC^2 & 0 & CP^2 \\ 1 & AP^2 & BP^2 & CP^2 & 0 \end{array} \right| \, . $$ Here we're given $PA = 10$, $PB = 6$, $PC = 7$. So, with $AB^2=c^2$, $AC^2=a^2-ac+c^2$, and $BC^2=a^2$ we calculate (after removing a common factor of $2$): $$ c^2 a^4 + (-c^3 + 64 c) a^3 + (c^4 - 262 c^2 + 4096) a^2 + (13 c^3 - 832 c) a + 169 c^2 = 0. $$ We want to maximize the area of the triangle, which is $\frac12 ac \sin 60^\circ = (\sqrt3/4) ac$; so we seek the maximal value of $M := ac$. Setting $c=M/a$ and multiplying by $a^2$, we find $$ (M^2+64M+4096) a^4 - (M^3+262M^2+832M) a^2 + (M^4+13M^3+169M^2) = 0, $$ a quadratic equation in $a^2$ with negative linear coefficient. Thus there is a positive solution iff the discriminant is nonnegative. We find that this discriminant is $-3 M^2 (M^2-176M+832) (M^2+104M+832)$, and thus that for $M>0$ there is a positive solution iff $M^2-176M+832 \leq 0$. The largest such $M$ is the larger root of this quadratic polynomial, which is $88 + 48 \sqrt{3}$ (and agrees numerically with the value $f_\max = 171.138\ldots$ computed by chenbai). This gives the value $(\sqrt3/4)M = 36 + 22\sqrt 3$ announced in the first sentence.

It remains to check that this is consistent with the given lengths $PA$, $PB$, $PC$: besides the vanishing of the Cayley-Menger determinant, we must verify that each of $PBC$, $PCA$, $PAB$ satisfies the triangle inequality. We compute $a^2 = (5323+3024\sqrt 3)/73$ and $c^2 = (8128+3840\sqrt 3)/73$, which makes $AC^2 = a^2-ac+c^2 = (7027+3360\sqrt 3)/73$; thus numerically our maximal-area triangle $ABC$ is to have sides $12.03$, $13.76$, and $14.23$ opposite vertices $A,B,C$, which is less than $6+7$, $7+10$, and $10+6$ respectively. Thus these distances are relaized by an actual configuration $ABCP$, so we're done.

[Added later: the same technique works for any given values $r,s,t$ of $PA$, $PB$, $PC$, finding that the maximal $M$ is $$ rt + \frac{s^2}{2} + s \sqrt{r^2+rt+t^2-\frac34 s^2} $$ and the maximal area is again $\sqrt3/4$ times that.]

let $B$ is on the $O$ of polar system, $P(6,\alpha)$

so $C$ is on circle $p^2+6^2-2*6*cos(\beta- \alpha)=7^2$ and $\beta=0$

$A$ is on circle $p^2+6^2-2*6*cos(\beta- \alpha)=10^2$ and $\beta=\dfrac{\pi}{3}$

$a=BC=6cos\alpha+\sqrt{(6cos\alpha)^2+13}, c=BA=6cos(\dfrac{\pi}{3}-\alpha)+\sqrt{(6cos(\dfrac{\pi}{3}-\alpha))^2+64}$

$ac=f(\alpha),\alpha=x, f'(x)=(\sqrt{36 cos^2(x)+13}+6 cos(x)) \left(6 cos(x+\dfrac{\pi}{6})+\dfrac{(36 sin(x+\dfrac{\pi}{6}) cos(x+\dfrac{\pi}{6})}{\sqrt{36 sin^2(x+\dfrac{\pi}{6})+64)}}\right)$$+(\sqrt{36 sin^2(x+\dfrac{\pi}{6})+64)+6 sin(x+\dfrac{\pi}{6}}) \left(-6 sin(x)-\dfrac{(36 sin(x) cos(x))}{\sqrt{36 cos^2(x)+13)}}\right)=0$

it is only numeric method can solve the solution is $x=.422064 \implies f_{max}=171.138$

BTW, this triangle also have max area without fixed $\angle B$ but it is more difficult as there is two varies.