Factoring polynomials with a 2nd degree coefficient greater than $1$

That AC-method reduces to factoring a polynomial that is $\,\rm\color{#c00}{monic}\,$ (lead coeff $\color{#c00}{=1})$ as follows

$$\begin{eqnarray} \rm\: a\:f(x)\:\! \,=\,\:\! a\:(a\:x^2 + b\:x + c) &\,=\,&\!\!\rm\: \color{#c00}{X^2} + b\:X + \!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!{\overbrace{ac,}^{\rm\qquad\ \ \ \ \ {\bf\large\ \ AC-method}}}\!\!\!\!\!\!\!\!\!\!\!\!\!\!\!\! \ X = a\:x \\ \end{eqnarray}$$

In your case

$$ {\begin{eqnarray} f \, &\,=\,& \ \ \, 6 x^2+\ 11\ x\,\ -\ \ 35\\ \Rightarrow\,\ 6f\, &\,=\,&\!\,\ (6x)^2\! +11(6x)-210\\ &\,=\,& \ \ \ \color{#c00}{X^2}+\, 11\ X\,\ -\ 210,\,\ \ X\, =\, 6x\\ &\,=\,& \ \ (X+21)\ (X-\,10)\\ &\,=\,& \ (6x+21)\,(6x-10)\\ \Rightarrow\ \ f\:=\: \color{#0a0}{6^{-1}}\,(6f)\, &\,=\,& \ (2x+\,\ 7)\ (3x\,-5)\\ \end{eqnarray}}$$

In the final step we cancel $\,\color{#0a0}6\,$ by cancelling $\,3\,$ from the first factor, and $\,2\,$ from the second.

If we denote our factoring algorithm by $\,\cal F,\,$ then the above transformation is simply

$$\cal F f\, = a^{-1}\cal F\, a\,f\quad\,$$

Thus we've transformed by $ $ conjugation $\,\ \cal F = a^{-1} \cal F\, a\ \,$ the problem of factoring non-monic polynomials into the simpler problem of factoring monic polynomials. The same idea also works for higher degree polynomials, see this answer, which also gives links to closely-related ring-theoretic topics.


Your method

Suppose we had something simpler, with a quadratic coefficient ("leading coefficient") of $1$, like $k^2+2k-35$. Then we might expect to be able to factor it with factors whose leading coefficients are also $1$. Then the factorization would look like $\left(k+\square_1\right)\left(k+\square_2\right)$. Then when you multiply things out, you'll have $k^2+(\square_1+\square_2)k+\square_1\square_2$, so that we need the sum of the two numbers to be $2$ and the product to be $-35$. As you noted, this doesn't work the same way when the leading coefficient is not $1$.


Adapting the method to this problem

You want to factor $6k^2+11k-35$.

Well, suppose you factor it with nice integer factors as $\left(\triangle_1k+\square_1\right)\left(\triangle_2k+\square_2\right)$. Then when you multiply things out, you'll have $\left(\triangle_1*\triangle_2\right)k^2+(\square_1*\triangle_2+\triangle_1*\square_2)k+\square_1*\square_2$. This means the triangle numbers have a product of $6$, the square numbers have a product of $-35$, and $\square_1*\triangle_2+\triangle_1*\square_2=11$.

We can assume the two triangle numbers are both positive (just take the negative of both factors otherwise) and that $\triangle_1>\triangle_2$ (just swap the two factors otherwise).

This leaves us with the following table of possibilities: $$\begin{matrix}\triangle_{1} & \triangle_{2} & \square_{1} & \square_{2} & \square_{1}*\triangle_{2}+\triangle_{1}*\square_{2} & = & 11?\\ 6 & 1 & 35 & -1 & 35*1+6*(-1) & 29 & \text{no}\\ 6 & 1 & -35 & 1 & & -29 & \text{no}\\ 6 & 1 & 7 & -5 & 7*1+6*(-5) & -23 & \text{no}\\ 6 & 1 & -7 & 5 & & 23 & \text{no}\\ 6 & 1 & 5 & -7 & 5*1+6*(-7) & -37 & \text{no}\\ 6 & 1 & -5 & 7 & & 37 & \text{no}\\ 6 & 1 & 1 & -35 & 1*1+6*(-35) & -\text{huge} & \text{no}\\ 6 & 1 & -1 & 35 & & \text{huge} & \text{no}\\ 3 & 2 & 35 & -1 & 35*2+3*(-1) & 67 & \text{no}\\ 3 & 2 & -35 & 1 & & -67 & \text{no}\\ 3 & 2 & 7 & -5 & 7*2+3*(-5) & -1 & \text{no}\\ 3 & 2 & -7 & 5 & & 1 & \text{no}\\ 3 & 2 & 5 & -7 & 5*2+3*(-7) & -11 & \text{no}\\ 3 & 2 & -5 & 7 & & 11 & \text{YES!}\\ 3 & 2 & 1 & -35 & 1*2+3*(-35) & -\text{big} & \text{no}\\ 3 & 2 & -1 & 35 & & \text{big} & \text{no} \end{matrix}$$

So the factorization is $6k^2+11k-35=\boxed{(3k-5)(2k+7)}$.


Another method

Now, even ignoring the negatives, the table still had $8$ real cases in it, which is a bit of a pain. You can also factor this with the factor theorem. Basically, if there's a factor like $\left(k-\dfrac{5}{3}\right)$, then since $k=\dfrac{5}{3}$ would make that factor zero, it better make the whole polynomial zero as well. So if you know another way to find zeros/roots of the polynomial, you can use that to solve the problem.

As Battani implied, the quadratic formula is one method we can use. The zeros of the quadratic are therefore $$\dfrac{-11\pm\sqrt{11^{2}-4(6)(-35)}}{2(6)}=\dfrac{-11\pm\sqrt{121+840}}{2(6)}$$ $$=\dfrac{-11\pm\sqrt{900+61}}{12}=\dfrac{-11\pm\sqrt{30^{2}+(31-30)(31+30)}}{12}$$ $$ =\dfrac{-11\pm\sqrt{30^{2}+31^{2}-30^{2}}}{12}=\dfrac{-11\pm31}{12}=\dfrac{20}{12}\text{ and }\dfrac{-42}{12}=\dfrac{5}{3}\text{ and }\dfrac{-7}{2}$$

How does this help us factor things? Except for a constant, these must tell us the factors since we found the two zeros of the quadratic: $6k^2+11k-35=\square\left(k-\dfrac{5}{3}\right)\left(k-\left(\dfrac{-7}{2}\right)\right)$. We know it as to start with $6k^2$ after we multiply out, so we have $6k^2+11k-35=6\left(k-\dfrac{5}{3}\right)\left(k-\left(\dfrac{-7}{2}\right)\right)$. If we want nice integers instead of fractions, we can then take $3$ from the $6$ for the first factor and the $2$ from the $6$ for the second factor, to get $\boxed{\left(3k-5\right)\left(2k+7\right)}$.


We can factor your example as follows ,firslty,find roots of equation of $$6k^{ 2 }+11k-35=0\\ k=\frac { -11\pm 31 }{ 12 } =\frac { 5 }{ 3 } ,\frac { -7 }{ 2 } $$ so we can write it

$$2{ k }^{ 2 }\left( 6\left( k-\frac { 5 }{ 3 } \right) \left( k+\frac { 7 }{ 2 } \right) \right) =2{ k }^{ 2 }\left( 3k-5 \right) \left( 2k+7 \right) $$