I will sell web crawler, scraper, data extractor

About:

FEATURES:

• extract data from Amazon, eBay, or any simmilar website
• deploy infinite ∞ number of web robots in the cloud
• huge capacity - each robot is deployed on different physical machine
• great extract power - robots work simultaneously (asynchronous mode)
• each robot can run several scraping tasks
• prevent bouncing by web proxy
• robot controlled via web panel: start, stop, pause
• change robot settings on the fly (no need to stop robot)
• realtime monitoring - use browser to track extracted data in web panel
• scraped data can be filtered and modified before inserting it into database
• extracted data is stored in mongoDB database
• HTTP client error reporting in realtime
• errors are stored in database for better debugging
• robot task scheduler (periodically execute robot tasks)

TECHNOLOGY:

• NodeJS + ExpressJS
• HTTP clients: chrome headless with puppeteer, phantomJS, casperJS, nightmareJS, node request ...etc
• socket.io for realtime messaging between robot and web panel
• Angular 6+ (frontend)
• MongoDB (database)

Reviews

:

:

:

:

: