ErrorDocument 404 /404.php is not working in .htaccess file in PHP

Where is your 404.php actually located in relation to your .htaccess file? Can you simply run it as a direct URL? Is the file readable by the server? Or is it in a nested subdirectory? You can also try the full URL as well:

ErrorDocument 404 http://mygreat.server/404.php 

Full details in the official Apache documentation here.

I'll consolidate my comments to this answer:

When setting ...

ErrorDocument 404 /404.php

the /404.php path may not be the absolute path to your htdocs folder root but instead the root of your filesystem. This may be, based on your configuration, e.g. /home/htdocs/ or ~ and so on.

So what one need to do is find out the absolute path and set it accordingly.