Scrapy Commands
Fri Nov 06 2020 23:28:22 GMT+0000 (Coordinated Universal Time)
Saved by
@ianh
scrapy startproject <project_name>
scrapy genspider <spider_name> <URL>
scrapy crawl <spider_name>
scrapy shell <'URL to scrape'>
view(response) #Open webpage
response.xpath('<xpath to test>') #test scrapy code below
all_titles = response.xpath('//td[@class="titleColumn"]')
first_title = all_titles[0]
first_title.xpath('.//a/text()')
scrapy crawl <spider_name> -o <'file_name.csv'>
content_copyCOPY
Comments