Would allowing shorter passwords sometimes be more secure?
One related question that you missed in your list is this one:
How critical is it to keep your password length secret?
The accepted answer there (disclaimer: mine) shows that if you have a password scheme which allows all 95 printable ascii characters, then the key space ramps insanely quickly every time you increase the length of the password by 1. You can check all the passwords up to length N in about 1% of the time that it'll take you to check only passwords of length N+1. By rejecting any password shorter than some cutoff length, you give up far less than 1% of your key space.
So, I strongly second @Iszi in saying
The benefit gained by forcing increased length far outweighs the number of possible passwords lost.
Next point: let's get out of the idea that 8-characters is long for a password. It is not. You say "~13 quadrillion passwords" as if that's a big number. It is not. According to this article (which is a great read btw) his password cracking rig could make 350 billion guesses per second, so every single one of your ~13 quadrillion passwords can cracked one-by-one in ~10 hours. And that's on 2013 hardware, GPUs have come up a lot in power since then.
My opinion is that websites can squabble about who has the better password requirements, but they are all far too weak. Our ability to crack passwords is growing WAY faster than our ability to remember longer ones. This is because security is clashing with usability. Try telling anybody who's not a tech nerd that they need to memorize a 32-character password that doesn't contain any English words, and a different one for each account they have! You'll be laughed at and then ignored. Websites that try to enforce anything better than pathetic password policies have to deal with mountains of angry customers.
The solution is to do away with passwords all together and move towards strong 2-Factor type authentication, where offline cracking isn't feasible. Unfortunately companies have only been seriously thinking about alternatives to passwords for less than a decade and the offerings are far from polished (they are plagued with convenience and usability problems which are preventing mass adoption), so in the meantime we get to continue having these useless debates comparing one mostly useless password scheme against another. End opinion.
The answer is in your question.
Assuming the use of only alphanumeric characters, requiring 8+ characters removes about 3.5 trillion password possibilities (most of them would just be random gibberish). This leaves ~13 quadrillion passwords that are 8-9 characters.
Establishing a minimum length, or even an exact length, for passwords forces the user to choose a password that's in a search space several orders of magnitude larger than the number of weaker passwords that such requirements invalidate.
To better illustrate this, let's simplify and actually write out whole numbers here. Assuming all-lowercase alphabetical-only passwords, there's:
8031810176 possible passwords of length 7 or less.
200795254400 possible passwords with length of exactly 8.
Increase the number of possible characters, and the number of passwords lost becomes even more insignificant in comparison to the complexity that's enforced.
The benefit gained by forcing increased length far outweighs the number of possible passwords lost. And the passwords that are eliminated pose far too high a risk to be allowed when such a simple and effective countermeasure is available.
Not uniformly applying a password policy introduces unnecessary security risks and definitely does not improve security.
Allowing weak passwords to exist just improves the likelihood that the attacker will crack a hash using a list of common passwords. This problem is made worse as the number of users increases. If 1/100 accounts have a password that doesn't conform to the password policy and 100,000 accounts exists, 1000 accounts are going to have weak passwords.
Also, It's actually more work to only selectively enforce a policy for only a subset of users than it would be to enforce the policy for all users because the application would need specific logic to not enforce the policy under certain circumstances. It's easier to require a strong password policy and uniformly apply it to all accounts.