$X=ABA^T$, X, A are matrix and need to solve matrix B by enforcing it as a diagonal matrix.

Here is an interesting property of the diagonalization operators $$\eqalign{ {\rm diag}(Z\,{\rm Diag}(y)\,W) = (W^T\circ Z)\,y \cr }$$ where $\circ$ is the elementwise/Hadamard product, ${\rm Diag}(v)$ is a function which creates a diagonal matrix from its vector argument, and ${\rm diag}(M)$ creates a column vector from the diagonal of its matrix argument.

Applying this to your equation allows you to solve for $B={\rm Diag}(b)$ directly $$\eqalign{ {\rm diag}(A\,{\rm Diag}(b)\,A^T) &= {\rm diag}(X) \cr (A\circ A)\,b &= {\rm diag}(X) \cr b &= (A\circ A)^+\,{\rm diag}(X) \cr B &= {\rm Diag}\Big((A\circ A)^+\,{\rm diag}(X)\Big) \cr }$$ where $M^+$ is the Moore-Penrose inverse of $M$.

Update
The diagonalization of $X$ throws away some information, so here is another relationship that uses vectorization, and therefore preserves all of that information. $$\eqalign{ {\rm vec}(Z\,{\rm Diag}(y)\,W) = (W^T\otimes u)\circ(v\otimes Z)\,y \cr }$$ the vectors $(u,v)$ are vectors all of whose components equal unity; the dimensions are such that the products $Z^Tu$ and $Wv$ are defined; and $\otimes$ is the Kronecker product.

Reference for this operation can be found in: J Brewer, Kronecker products and matrix calculus in system theory, IEEE Transactions on circuits and systems, 1978. Link to pdf of paper from uta.edu

There the operation was defined in terms of the Khatri-Rao product, which results in $(W^T\otimes u)\circ(v\otimes Z)$.

Once again you can use this to solve for $B$ $$\eqalign{ {\rm vec}(A\,{\rm Diag}(b)\,A^T) &= {\rm vec}(X) \cr (A\otimes u)\circ(u\otimes A)\,b &= {\rm vec}(X) \cr b &= \big((A\otimes u)\circ(u\otimes A)\big)^+\,{\rm vec}(X) \cr B &= {\rm Diag}\Big(\big((A\otimes u)\circ(u\otimes A)\big)^+\,{\rm vec}(X)\Big) \cr }$$ note that $v=u$ for this problem and the dimensions of $u$ are such that $A^Tu$ is defined.