How can I get an OpenFileDialog in a custom control's property grid?

You can use built-in UITypeEditor. It is called FileNameEditor

[EditorAttribute(typeof(System.Windows.Forms.Design.FileNameEditor), typeof(System.Drawing.Design.UITypeEditor))]

public string SomeFilePath
{
 get;
 set;
}

You can do this by adding a UITypeEditor.

Here is an example of a UITypeEditor that gives you the OpenFileDialog for chossing a filename.

Here's another example comes with customizing File Dialog :

CustomFileEditor.cs

using System.Windows.Forms;
using System.Windows.Forms.Design;

namespace YourNameSpace
{
    class CustomFileBrowser : FileNameEditor
    {
        protected override void InitializeDialog(OpenFileDialog openFileDialog)
        {
            base.InitializeDialog(openFileDialog);
            openFileDialog.Title = "Select Project File : ";
            openFileDialog.Filter = "Project File (*.proj)|*.proj"; ;
        }
    }

}

Usage :

            [Category("Settings"), DisplayName("Project File:")]
            [EditorAttribute(typeof(CustomFileBrowser), typeof(System.Drawing.Design.UITypeEditor))]
            public string Project_File { get; set; }