Is information disclosure in a form of a server header a real vulnerability?

OWASP defines Information Leakage as a vulnerability, so the debate is really on whether or not the specific version information should be classified as "Information Leakage".

As @Oasiscircle mentions, this information can be used as a starting point for attackers who know of specific vulnerabilities associated with specific versions. We know attackers use botnets to scan servers looking for version fingerprints, and they use this information to maintain databases of application versions and server addresses. Shodan provides a widely used database of this info. When a new exploit is introduced that targets a specific version, or if an attacker develops a 0day exploit against a version, these attackers do a simple select of their databases and strike quickly, hoping to catch some of these machines before they're patched.

Does that make knowing a specific version number a vulnerability? Not directly, but it puts you in the cross-hairs of a field of snipers, each with a hair trigger and the motivation to strike first. I'll leave it to you to determine for yourself if you consider that a vulnerability.

I'm generally dubious about the benefits - as @MarkBuffalo says, lots of attackers will try every attack they know on every server they find, whether it's likely to work or not (even to the extent of trying Windows specific attacks against systems reporting that they're running RHEL).

However, it's not information you need to give out, so I am generally in favour of removing it, where this can be accomplished with minimal effort. This sometimes means reducing the output to just the name of the server (e.g. Apache, IIS) rather than completely removing the Server header, as entirely removing the header may well require introducing new software elements into the server stack.

For example, it takes one line in apache.conf to reduce an httpd Server header to "Apache", but the addition of a proxy to reduce it further. In that case, it seems safer to stick with the minimal line, rather than add an additional element, which may introduce its own flaws.

Similarly, it's easy to see when a site runs Wordpress, but I'd still recommend removing the readme file - it's just not needed, and acts a bit like the bowl of brown M&Ms rider that some rock bands infamously used: you can check it easily, and it shows whether other steps are likely to have been taken.

Let's say that today a vulnerability got announced for all Microsoft IIS 7.5 servers with a small range of ASP Net versions. I'm a malicious attacker, and I want to figure out as many servers that I can exploit as possible before sysadmins patch up the vulnerability. Wouldn't it be a much easier problem to figure out if I could ask each individual server if they are vulnerable rather than having to try the vulnerability on every single server? Plus I could log all this information on a per-server basis and know exactly who I want to attack the instant a vulnerability is found for any type of server.

Hiding your version information will not stop an attack from happening, but it will make you less of a target if attackers are looking for a specific version of hardware or software. Security by obscurity doesn't mean you're safe, but it does slow people down sometimes, and that's exactly what's needed for day zero vulnerabilities.