Ignore folders/files when Directory.GetFiles() is denied access

You will have to do the recursion manually; don't use AllDirectories - look one folder at a time, then try getting the files from sub-dirs. Untested, but something like below (note uses a delegate rather than building an array):

using System;
using System.IO;
static class Program
{
    static void Main()
    {
        string path = ""; // TODO
        ApplyAllFiles(path, ProcessFile);
    }
    static void ProcessFile(string path) {/* ... */}
    static void ApplyAllFiles(string folder, Action<string> fileAction)
    {
        foreach (string file in Directory.GetFiles(folder))
        {
            fileAction(file);
        }
        foreach (string subDir in Directory.GetDirectories(folder))
        {
            try
            {
                ApplyAllFiles(subDir, fileAction);
            }
            catch
            {
                // swallow, log, whatever
            }
        }
    }
}

Since .NET Standard 2.1, you can now just do:

var filePaths = Directory.EnumerateFiles(@"C:\my\files", "*.xml", new EnumerationOptions
{
    IgnoreInaccessible = true,
    RecurseSubdirectories = true
});

According to the MSDN docs about IgnoreInaccessible:

Gets or sets a value that indicates whether to skip files or directories when access is denied (for example, UnauthorizedAccessException or SecurityException). The default is true.

Default value is actually true, but I've kept it here just to show the property.

The same overload is available for DirectoryInfo as well.