Why are generated access tokens for APIs much longer than passwords?

Quite simply, the token is not designed to be memorized so it can be as long as they want. A password is limited in length to what a person can practically reliably recall after a short memorization period. This limits it to 7-10 characters for most people. The token is designed to be copy/pasted, just once even (since it is application specific) therefore there is absolutely no penalty to being long, and the great length allows it to be both secure and unique (i.e. the token can have enough embedded information to tie back to a specific account).

I suppose a more pointed question would be, why aren't passwords longer ;-)

There are several reasons for that.

One reason is that memorable passwords are too short for the expected security level. Several methods are used to mitigate this problem, but it would be problematic to apply them to access tokens:

• Some services require a second authentication factor. Access tokens are meant to be used without user intervention, so most other forms of authentication are out (they would either be inapplicable, e.g. biometrics, or they would only result in a second access token).
• Online services protect against brute-force attacks by imposing a delay between authentication attempts. This is problematic because it can lock the legitimate user out and because it's an extra implementation burden.
• Password storage uses slow hashes. This costs a lot of CPU time. If a long enough access token needs to be kept secret and stored in hash form, no special precaution is necessary. Likewise if a key is derived from an access token.

Another reason is that the length of access tokens is typically decided by the author of a protocol or library, not by the author of the service(s) that uses them. The cost of making an access token longer is usually marginal. So the length is set at something that gives the highest security level that anybody requires. You're comparing the access token length with the minimum password length, but a more apt comparison would be with the maximum password length.

Some services use access tokens independently of accounts (i.e. as sessio cookies rather than as authentication parameter that is combined with an identification parameter). There can also be many access tokens for a single account with different privileges. So the access token needs to have more entropy to keep the probability of collisions infinitesimal.

If you need to store relatively short passwords, there is a bunch of precautions you have to take:

1. Salting with a unique long enough salt to prevent rainbow-table attacks.
2. Key-stretching so the calculation needs some time, to hinder brute-forcing.

On the other side, if the token is long enough you can store just a plain SHA-256 without any drawbacks on security. These are the advantages:

1. There is no need for salting. In contrast to salted passwords, the token-hashes can be searched for in a database. So if you got a token you can hash it again and check with an SQL query if the token is valid.
2. The calculation is fast and light on the servers cpu. While we needed a lot of cpu power to hash passwords because of key-stretching, hashing of the token is lightning fast.