scrapy startproject <project_name>
scrapy genspider <spider_name> <URL>
scrapy crawl <spider_name>
scrapy shell <'URL to scrape'>
view(response) #Open webpage
response.xpath('<xpath to test>') #test scrapy code below
all_titles = response.xpath('//td[@class="titleColumn"]')
first_title = all_titles[0]
first_title.xpath('.//a/text()')
scrapy crawl <spider_name> -o <'file_name.csv'>
Preview:
downloadDownload PNG
downloadDownload JPEG
downloadDownload SVG
Tip: You can change the style, width & colours of the snippet with the inspect tool before clicking Download!
Click to optimize width for Twitter