Scrapy – Call function when spider closed

Spread the love

Hello Wednesday,

Today at work I had the chance to play with Scrapy. It is quite fast and really easy to use.
I will write posts about this crawling framework later, but for the time being I will take a quick note about how to call a function when spider closed.

Trigger Function after finish crawling items

After finish crawling all the items and wanna do something else. Simply use the closed function.


# -*- coding: utf-8 -*-
import scrapy

class TcvStockSpider(scrapy.Spider):
	def closed(self, reason):
           #do something

 

Conclusion

In their document, there is another way to achieve this but I myself think this is simple and easy to achieve way.