Scrapy : storing the data
You can view a list of available commands by typing scrapy crawl -h
from within your project directory.
scrapy crawl spidername -o items.json -t json
-o
specifies the output filename for dumped items (items.json)-t
specifies the format for dumping items (json)
scrapy crawl spidername --set FEED_URI=output.csv --set FEED_FORMAT=csv
--set
is used to set/override a settingFEED_URI
is used to set the storage backend for the item dumping. In this instance it is set to "output.csv" which is using the local filesystem ie a simple output file.(for current example - output.csv)FEED_FORMAT
is used to set the serialization format for the (output) feed ie (for current example csv)
References (Scrapy documentation):
- Available tool commands (for the command line)
- Feed exports