Is it possible to find the difference between two arbitrary text files using Git?

Yes, git diff works on two unversioned paths, both files and directories (recursive diff). I don't think you can do this from the GUI; you'll need to use the command line.

If you want GUI, I suggest installing KDiff3 instead. It can also be used by Git as a merge tool than if you set diff.tool = kdiff3, diff.guitool = kdiff3 and difftool.kdiff3.path = path-to-kdiff3.exe

You'll need to use the --no-index option unless one or both of the files are outside of a Git repository:

git diff --no-index path/to/file.txt path/to/other/file.txt

You can also use git difftool with the same arguments to invoke your GUI tool.