How to scrape products from eCommerce websites

Scraping data from eCommerce websites can be used for different purposes, including:

⭐ Downloading entire product catalogs from competitors to analyze products, prices, or reviews. 

⭐ Getting product data from suppliers to enrich the existing product listings (for example, if you are a reseller or have a brand swag shop).

⭐ Affiliates can use product scraping to get product data at scale.

Hexomatic allows you to scrape products from eCommerce websites at scale in a few minutes. 

No need to copy and paste all the products’ information, including thumbnails, descriptions, customer reviews, and more into a single spreadsheet for record-keeping and analysis. 

In this tutorial, we will show you different ways how to scrape products from e-Commerce websites.

How to scrape products from e-Commerce websites using product sitemap and scraping recipe

In this section, we will demonstrate how to scrape products from e-commerce websites using sitemap, then scrape data from the URLs that you got from the sitemap extraction.

Please, note that for using this option, pages need to have the same HTML structure.

Before getting started, go to the targeted website and get its sitemap. You can find it by entering your website URL, then adding /sitemap.xml. (for example, https://appsumo.com/sitemap.xml). 

Step 1: Create a new workflow

From your dashboard, create a new workflow by choosing the “blank” option. Then, select Data automation as a starting point.

Step 2: Add the sitemap of your website

Next, add the sitemap of the website using Manual paste/list of inputs option. 

Step 3: Add the sitemap extractor automation

Now, you should add the sitemap extractor automation, selecting data input as the source. You can also set to extract all the URLs or limit URLs. Then, set the limit of the results to be displayed. 

After filling in all the required fields, click Continue. 

Step 4: Run or schedule the workflow

You can run the workflow to see the results. 

Step 5: View and Save the results

After viewing the results, you can scrape data from each of the URLs using the Hexomatic scraping recipe builder. 

How to use 2-step scraping to scrape products from an e-Commerce website

In this section, we will demonstrate how to easily scrape products from e-Commerce website by using Hexomatic.

Step 1: Create a new scraping recipe

To get started, create a blank scraping recipe from your dashboard.

Step 2: Add the website URL

Add the product page URL you want to scrape. Then, scrape the elements you wish. 

 In this case, we will scrape the name and the price of the product.

Step 3: Create a new workflow

From your dashboard, create a new workflow by choosing the “blank” option. Then, select Data automation as a starting point.

Step 4: Add the product page URLs

Now, you can add specific product URLs to automatically scrape data from them.  

Add the URLs of specific products from the scraped product list page. 

Step 6: Add your scraping recipe

Add the previously-created scraping recipe, selecting data input as the source.  
Then, click Continue.

Step 7: Run the workflow

Now you can run your workflow to automatically scrape data from the product pages.

 

---

Automate & scale time-consuming tasks like never before