Today at work I had the chance to play with Scrapy. It is quite fast and really easy to use.
I will write posts about this crawling framework later, but for the time being I will take a quick note about how to call a function when spider closed.
Trigger Function after finish crawling items
After finish crawling all the items and wanna do something else. Simply use the
# -*- coding: utf-8 -*-
def closed(self, reason):
In their document, there is another way to achieve this but I myself think this is simple and easy to achieve way.