scrapy schell code example

Example 1: scrapy shell

scrapy shell <url>

Example 2: scrapy spider shell from spider

from scrapy.shell import inspect_response

def parse_details(self, response, item=None):
    if item:
        # populate more `item` fields
        return item
    else:
        inspect_response(response, self)