Apache 2.4.6 on Ubuntu Server: Client denied by server configuration (PHP FPM) [While loading PHP file]

I have exactly the same issue. I ran a couple of virtual hosts on my local machine for developing.

First, I changed /etc/apache2/conf-available/php5-fpm.conf. I replaced every

Order Deny,Allow
Deny from all

to

Require all granted

The configuration has to be enabled by a2enconf php5-fpm. I did the same with my virtual hosts configurations and made the replacements.

I think this is not advised for security reasons, but as long as I use my server for local purposes only I can live with it.

I ran into this exact issue upon a new install of Apache 2.4. After a few hours of googling and testing I finally found out that I also had to allow access to the directory that contains the (non-existent) target of the Alias directive. That is, this worked for me:

# File: /etc/apache2/conf-available/php5-fpm.conf
<IfModule mod_fastcgi.c>
    AddHandler php5-fcgi .php
    Action php5-fcgi /php5-fcgi
    Alias /php5-fcgi /usr/lib/cgi-bin/php5-fcgi
    FastCgiExternalServer /usr/lib/cgi-bin/php5-fcgi -socket /var/run/php5-fpm.sock -pass-header Authorization

    # NOTE: using '/usr/lib/cgi-bin/php5-cgi' here does not work,
    #   it doesn't exist in the filesystem!
    <Directory /usr/lib/cgi-bin>
        Require all granted
    </Directory>
</Ifmodule>

I ran into a similar problem today (but with mod_wsgi). It might be an Apache 2.2-to-2.4 problem. A comprehensive list of changes can be found here.

For me, it helped to add an additional <Directory>-entry for every path the error-log was complaining about and filling the section with Require all granted.

So in your case you could try

<Directory /usr/lib/cgi-bin/php5-fcgi>
    Require all granted
    Options FollowSymLinks
</Directory>

and I had to move my configuration file from folder conf.d to folder sites-enabled.

All in all, that did the trick for me, but I don't guarantee it works in your case as well.