Given $ax^2+bxy+cy^2+dx+ey+f$, what is the best way to get {a,b,c,d,e,f}?

CoefficientArrays come to mind, but it needs some tweaking. The below solutions work for polynomials of order $2$.

Demonstration

{m0, m1, m2} = CoefficientArrays[poly[[1]], {x, y}] // Normal

{-4, {3, -3}, {{5, 2}, {0, -5}}}

We want upper-right triangular elements of m2:

Flatten@Table[m2[[i, i ;;]], {i, 1, Length@m2}]

{5, 2, -5}

Make it a function:

RU[m_] := Flatten@Table[m[[i, i ;;]], {i, 1, Length@m}]

and Join it with m1 and m0:

RU[m2]~Join~m1~Join~{m0}

{5, 2, -5, 3, -3, -4}

Function

Clear@RU

coeff[pol_] := 
 Block[{ca = CoefficientArrays[pol, {x, y}] // Normal // Reverse, RU},
   RU[m_] := Flatten@Table[m[[i, i ;;]], {i, 1, Length@m}]; 
  RU[First@ca]~Join~Flatten@Rest@ca]

(coeff /@ poly) == rand

True

The function coeff gives the coefficients in the exact same order as the OP requested.

Three variables

However, a re-arrangement of the terms will make the computations easier:

SeedRandom[0]
rand3 = RandomInteger[{-5, 5}, {3, 10}]
poly3 = rand3.{x^2, y^2, z^2, x y, y z, x z, x, y, z, 1};

{{5, 2, -5, 3, -3, -4, 0, 3, -5, 1}, {2, 5, -3, -4, -5, 1, -4, -3, 5, 5}, {3, 5, 1, 5, 0, 0, 3, -1, 0, 4}}

So here squares come first, then mixed terms, first order terms, and the free term.

{m0, m1, m2} = CoefficientArrays[poly3[[1]], {x, y, z}] // Normal

{1, {0, 3, -5}, {{5, 3, -4}, {0, 2, -3}, {0, 0, -5}}}

Then

Flatten[Table[Diagonal[m2, k], {k, 0, 2}]]~Join~m1~Join~{m0}

{5, 2, -5, 3, -3, -4, 0, 3, -5, 1}

and

% == rand3[[1]]

True

The same line of reasoning should apply to any number of variables, hence

N variables

coeffN = Module[{ca = CoefficientArrays[#, Variables@#] // Normal},
   Flatten[Table[Diagonal[ca[[3]], k], {k, 0, 2}]]~Join~ca[[2]]~Join~{ca[[1]]}
    ] &;

E.g.

(coeffN /@ poly3) == rand3

True


You can also use CoefficientList as follows:

Flatten[Table[Diagonal[#, -i], {i , 0, Length[#[[1]]] - 1}]] & /@ 
  PadLeft[Reverse /@ CoefficientList[poly, {x, y}]]

{{5, 2, -5, 3, -3, -4},
{0, 3, -5, 1, 2, 5},
{-3, -4, -5, 1, -4, -3},
{5, 5, 3, 5, 1, 5},
{0, 0, 3, -1, 0, 4}}

% == rand

True

For arbitrary polynomials in two variables, we can define a function

ClearAll[dlexCL]
dlexCL = Module[{cl = Reverse @ CoefficientList[#, Variables[#]]}, Flatten @ 
 Table[Diagonal[PadLeft[cl, {#, #}&@Length@cl[[1]]], -i], {i, 0, Length@cl[[1]] - 1}]]&;

dlexCL /@ poly == rand

True

dlexCL[1 + x^2 + 3 x^3 y + 2 y^4, {x, y}]

{0, 3, 0, 0, 2, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1}


I do not understand your code, I am just answering the title of your post:

Given ax^2+bxy+cy^2+dx+ey+f, what is the best way to get {a,b,c,d,e,f}?

expr = a x^2 + b x y + c y^2 + d x + e y + f;
r = Flatten[CoefficientList[expr, {x, y}]]
DeleteCases[r, 0]

Mathematica graphics