brew install gcc too time consuming

You do need gcc installed to get gfortran, and you do need a fortran compiler for scipy. Homebrew will install a "bottled" (i.e., precompiled) version of the gcc package, which is very fast, if you have the Xcode Command Line Tools installed. These are separate from XCode proper. You can install them with xcode-select --install.

There is no particular need to install a particular version of gcc (and I think those may not be bottled, so they will be equally slow).

In general, interrupting Homebrew with Ctrl+C is safe and Homebrew will automatically recover.

You may be interested in the homebrew-python tap.

Try this to force the bottle (pre-compiled) installation

brew install gcc --force-bottle