Resonsible for downloading item(s) to be scraped urls and enqueue(s) them in ItemUrlQueue
Receives item urls from UrlDownloaderWorker and enqueues them for feeding them to CrawlerWorker
Fetches item web pages and scrapes or extract data from them and enqueues the data in DatabaseQueue
Receives scraped item data from CrawlerWorker(s) and enques them for feeding them to DatabaseWorker.
Receives scraped data from DatabaseQueue and stores it in a persistent database.