BullionPricer.com

BullionPricer is a site showing prices of bullion(precious metal products) across a variety of online dealers. The site is currently a minimum viable product that will be extended further.

For the back-end I've designed a cluster of scrapers that go out every hour to collect product pricing. The leader assigns jobs to the nodes whenever it detects new products. The scraper nodes collect the data and return data in a series of chunks. In the future this cluster can be expanded to handle leader failure via RAFT or LCR.

The site has been deployed on Digital Ocean given the expensive outbound traffic costs on AWS.

You may find the project live at bullionpricer.com.