What is the number of ways of arranging two colours such that no more than $k$ pieces of same colour are together?
Since this task comes from programming contest, please update the post by giving the limits on $x$ and $y$.
Let's define a dynamic programming function $F[i][j][0/1]$. Where $F[i][j][0]$ is the number of ways to arrange $i$ A's and $j$ B's, the number $0$ means that the last letter in the arrangement is A. Similarly, $F[i][j][1]$ means the same while the $1$ shows us that the last letter is B.
The base cases would be $F[1][0][0] = 1, F[0][0][0 \text{ or } 1] = 1, F[0][1][1] = 1$.
For the transitions consider adding $d$ letters to the end, make sure that $d < k$ $$F[i][j][0] = F[i-1][j][1] + F[i-2][j][1] + ... + F[i-(k-1)][j][1]\\F[i][j][1] = F[i][j-1][0] + F[i][j-2][0] + ... + F[i][j-(k-1)][0]$$
The result is $F[x][y][0] + F[x][y][1]$. The computational complexity of the algorithm would be $O(x \cdot y \cdot K)$