How to ignore specific type of files to download in wget?
Worked example to download all files excluding archives:
wget -r -k -l 7 -E -nc \
-R "*.gz, *.tar, *.tgz, *.zip, *.pdf, *.tif, *.bz, *.bz2, *.rar, *.7z" \
-erobots=off \
--user-agent="Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" \
http://misis.ru/
Use the
--reject jpg,png --accept html
options to exclude/include files with certain extensions, see http://www.gnu.org/software/wget/manual/wget.html#Recursive-Accept_002fReject-Options.
Put patterns with wildcard characters in quotes, otherwise your shell will expand them, see http://www.gnu.org/software/wget/manual/wget.html#Types-of-Files
# -r : recursive
# -nH : Disable generation of host-prefixed directories
# -nd : all files will get saved to the current directory
# -np : Do not ever ascend to the parent directory when retrieving recursively.
# -R : don't download files with this files pattern
# -A : get only *.html files (for this case)
For instance:
wget -r -nH -nd -np -A "*.html" -R "*.gz, *.tar" http://www1.ncdc.noaa.gov/pub/data/noaa/1990/