How can I get the assembly last modified date?

How about this?

System.Reflection.Assembly assembly = System.Reflection.Assembly.GetExecutingAssembly();
System.IO.FileInfo fileInfo = new System.IO.FileInfo(assembly.Location);
DateTime lastModified = fileInfo.LastWriteTime;

I'll second pYrania's answer:

System.Reflection.Assembly assembly = System.Reflection.Assembly.GetExecutingAssembly();
System.IO.FileInfo fileInfo = new System.IO.FileInfo(assembly.Location);
DateTime lastModified = fileInfo.LastWriteTime;

But add this:

You mention you don't want to access the file system since it's in your master page and you don't want to make that extra file system hit for every page. So don't, just access it once in the Application load event and then store it as an application-level variable.

If you default the Revision and Build Numbers in AssemblyInfo:

[assembly: AssemblyVersion("1.0.*")]

You can get the an approximate build date with:

Version version = typeof(MyType).Assembly.GetName().Version;
DateTime date = new DateTime(2000, 1, 1)
    .AddDays(version.Build)
    .AddSeconds(version.Revision * 2);