How to reschedule 403 HTTP status codes to be crawled later in scrapy?
You can find the default statuses to retry here.
Adding 403 to
RETRY_HTTP_CODESin thesettings.pyfile should handle that request and retry.The ones inside the
RETRY_HTTP_CODES, we already checked the default ones.- The
RETRY_TIMEShandles how many times to try an error page, by default it is set to2, and you can override it on thesettings.pyfile.