How to "grep" for a filename instead of the contents of a file?

You need to use find instead of grep in this case.

You can also use find in combination with grep or egrep:

$ find | grep "f[[:alnum:]]\.frm"

Example

find <path> -name *FileName*

From manual:

find -name pattern

Base of file name (the path with the leading directories removed) matches shell pattern pattern. Because the leading directories are removed, the file names considered for a match with -name will never include a slash, so "-name a/b" will never match anything (you probably need to use -path instead). The metacharacters ("*", "?", and "[]") match a "." at the start of the base name (this is a change in find‐ utils-4.2.2; see section STANDARDS CONFORMANCE below). To ignore a directory and the files under it, use -prune; see an example in the description of -path. Braces are not recognised as being special, despite the fact that some shells including Bash imbue braces with a special meaning in shell patterns. The filename matching is performed with the use of the fnmatch(3) library function. Don't forget to enclose the pattern in quotes in order to protect it from expansion by the shell.

As Pablo said, you need to use find instead of grep, but there's no need to pipe find to grep. find has that functionality built in:

find . -regex 'f[[:alnum:]]\.frm'

find is a very powerful program for searching for files by name and supports searching by file type, depth limiting, combining different search terms with boolean operations, and executing arbitrary commands on found files. See the find man page for more information.