How to scrape Product Hunt

Product Hunt is a relatively new (founded in 2013) but well-known social media platform for sharing and discovering new products. In fact, more than 100 million products from 50,000 companies have been discovered on Product Hunt as of 2016!

The site curates a number of new products and startups people haven’t heard of and allows them to vote, comment, and discuss them. Hence, Product Hunt can be an ideal place to discover new products or promote your own products. 

This platform is increasingly being used by start-up founders as a launchpad. On the other hand, Investment opportunities abound there for investors.

Consequently, scraping data from Product Hunt can be beneficial both for founders and investors.  There are many usecases including scraping ProductHunt daily for inspiration or potential leads as well as competitor research on specific listings.

In this tutorial, we will show you how to save time on scraping start-up data from Product Hunt in a few clicks using Hexomatic.  

Not a Hexomatic user yet? Click here to register for a free account.

How to scrape the top products of the day from the Product Hunt homepage.

Step 1: Create a new scraping recipe

Go to your dashboard and create a new scraping recipe. 

Step 2: Add web page URL

Next, add the Product Hunt home page URL and click Preview. 

We recommend you choose “Full-stack” as your browser mode for getting better results. 

Step 3: Select elements to scrape

Now, you can select the elements to scrape. You can easily scrape the product listing on the page first, then scrape each product page separately. 

So, first, we will scrape product titles with their descriptions. 

To get the scraped data, you need to run the recipe in a workflow, then export the results to CSV or Google Sheets.

Step 4: Scrape separate product pages

Now, you need to go to www.producthunt.com and capture the first product page from the previously scraped products. 

Then, create a new scraping recipe, adding the capture URL like in the first scraping recipe. 

After, you can select the desired elements to scrape. 

Here, we select to scrape the upvote count, product logo, and reviews. 

To get the scraped data, you need to run the recipe in a workflow. 

Step 5: Create a new Workflow

Create a blank workflow and choose the data input automation as your starting point.

Step 6: Add Webpage URLs

Add the URLs of other product pages from the scraped product list to scrape their data automatically without creating separate scraping recipes for each. 

Step 7: Add your scraping recipe

Add the previously created scraping recipe of a product page, selecting data input as the source. 

Step 8: Run the workflow

Run the workflow to get the results. 

Step 9: View and save the results

Once the workflow has finished running, you can view and save the results.

---

Automate & scale time-consuming tasks like never before