The correct way to delete all files older than 2 days in PHP

You should add an is_file() check, because PHP normally lists . and .., as well as sub-directories that could reside in the the directory you're checking.

Also, as this answer suggests, you should replace the pre-calculated seconds with a more expressive notation.

<?php
  $files = glob(cacheme_directory()."*");
  $now   = time();

  foreach ($files as $file) {
    if (is_file($file)) {
      if ($now - filemtime($file) >= 60 * 60 * 24 * 2) { // 2 days
        unlink($file);
      }
    }
  }
?>

Alternatively you could also use the DirectoryIterator, as shown in this answer. In this simple case it doesn't really offer any advantages, but it would be OOP way.

Another simplier and more modern way, using FilesystemIterator.

I'm using 'logs' directory as an example.

$fileSystemIterator = new FilesystemIterator('logs');
$now = time();
foreach ($fileSystemIterator as $file) {
    if ($now - $file->getCTime() >= 60 * 60 * 24 * 2) // 2 days 
        unlink('logs/'.$file->getFilename());
}

Main advantage is: DirectoryIterator returns virtual directories "." and ".." in a loop. But FilesystemIterator ignores them.

The easiest way is by using DirectoryIterator:

<?php
if (file_exists($folderName)) {
    foreach (new DirectoryIterator($folderName) as $fileInfo) {
        if ($fileInfo->isDot()) {
        continue;
        }
        if ($fileInfo->isFile() && time() - $fileInfo->getCTime() >= 2*24*60*60) {
            unlink($fileInfo->getRealPath());
        }
    }
}
?>