Can I delete these build folders when I am not running Android Studio?

You can do one thing as I do. Just keep app folder and delete remaining folders. When you need to use that project again. Create new project and replace that app folder with your and done.

will my project rebuild again on next startup

Yes. For situations where backups are expensive in terms of space, time, or money, follow the rules for what goes into version control (e.g., git), and back up only those things.

If you are using a unix based OS you might be able to run following command using terminal:

 find . -name build -exec rm -rf {} \;

which causes to delete all build folders inside a specific folder.