matrix with chosen elements distributed in a random position

You need RandomChoice,

RandomChoice[{0, b}, {4, 3}]
(* {{0, 0, 0}, {0, b, 0}, {b, b, 0}, {0, 0, b}} *)

For the case with two elements 0 and b you can also use:

b RandomInteger[1, {4, 4}]

{{0, 0, b, 0}, {0, 0, b, b}, {b, b, 0, 0}, {0, b, 0, 0}}