What security considerations are there when developing a random password generator?
Creating a secure password generator is fairly easy, provided you have two components:
A cryptographically secure RNG. This means the output of the RNG is a bit stream that must not be predictable under any conditions. In most cases however, you can just use a PRNG designed specifically for key generation like /dev/urandom, /dev/random, or CryptGenRandom. Regular PRNG like Mersenne Twister should not be used for key generation. You should not attempt to write your own CSPRNG unless you have a deep background in CSPRNG mathematics. This is the part where most "regular people developing password generators" often don't really pay enough attention to.
A one-to-one encoding algorithm. This algorithm encodes the bit stream produced by the RNG into something acceptable by the system that needs to use the password and, for memorizable password, the preference of the user. A one-to-one encoding is a function that does not have any output collisions for its inputs, so a the function does not lose any entropy produced by the RNG during the encoding. This is the part that is actually extremely easy, but most "regular people developing password generators" often overcomplicate unnecessarily, and end up weakening their password generation method.
Example of one-to-one encoding algorithm: Diceware, base64, hex encoding, any perfect hash function. Example of transformation algorithm not suitable for password generation: most regular hash algorithm.
Once you have these two components, all you need to do is ask the CSPRNG for a bit stream of length n, where n is the required strength of your password, and encode that bitstream using your selected encoding algorithm.
Pitfall: You should parameterize the encoding algorithm so it can never produce a password that would be rejected by the target system's password policy. If you simply reject a password and generate it again using different random value, you are reducing the entropy. In theory, this does mean that you will need to know the exact password policy detail of each system you need to generate for to correctly calculate your password strength. In practice, you can often simply add a few more bit strength to compensate for the entropy loss due to such overcomplicated password policy.
Pitfall: For memorizable passwords, the user must be trained not to reject any password that they end up finding hard to memorize as this also reduces the entropy. This means that memorizable password encoder should be designed as much as possible so that it produces passwords/paraphrases that's most likely to be accepted by the user in the first try. There are few encoding algorithms that attempts this, for example EFF Diceware wordlist or Grammatical Diceware.
The principle consideration for a password generator is that it doesn't produce predictable output. The point of random passwords is that the best attack against them is brute force, but if there's a bias, an attacker can take advantage to reduce their search space.
Along similar lines, often people want additional restrictions, like "pronounceable". This can be a security benefit (you can remember passwords instead of storing them) as well as a downside (introducing biases, see above). If you're interested in this, it's important to be very careful about not introducing extra bias into the algorithm, as that's a more complex problem than simply generating random characters.