V.I. Arnold says Russian students can't solve this problem, but American students can -- why?
There is no such right triangle. The maximum possible altitude is half the hypotenuse (inscribe the triangle into a circle to see this), which here is $5$ inches. You would only get $30$ square inches if you tried to compute the area without checking whether the triangle actually exists.
There are many ways to prove that such triangle does not exist. I am using a different approach.
Suppose that the said right angled triangle can be formed. Then, we are interested in where should the foot of the said altitude (CD) be? [That is, how far is D (on AB) from A (or from B)?]
We assume that D is $\alpha$ and $\beta$ units from A and B respectively.
Clearly, we have $\alpha + \beta = 10$ …… (1)
Also, by a fact on right angled triangles, we have $\alpha \beta= 6^2$ ……… (2)
EDIT : That fact is "Power of a point".
To find $\alpha$ and $\beta$ is equivalent to solving the quadratic equation $x^2 – 10x + 36 = 0$.
Since the discriminant $(= [-10]^2 - 4 \times 36)$ is negative , we can conclude that such roots ($\alpha$ and $\beta$) are not real.
The red line represents all possible third vertices for triangles with base 10 and height 6;
The blue curve represents all possible third vertices for right triangles with hypotenuse 10.
The two sets have null intersection.
(in fact, the maximum possible third angle 6 units away is $\arccos(\frac{11}{61})\approx$ 79.6°)
(and yes, technically we should include the corresponding points below the segment as well)