How to use public scraping recipe templates

One of our latest updates was the creation of our public scraping library allowing you to automatically scrape data from a number of common websites like eBay, Product Hunt, AppSumo, Quora, Indeed, etc.

We have created these recipes to make your life much easier. Just choose the recipe, run it in a workflow and get the scraped data automatically. No need to choose elements and scrape the data manually.

This tutorial will demonstrate 3 different ways you can use our public scraping recipes: 

Method #1 – Creating a workflow using the scraping recipe as the input to get the scraped data.

Method #2 – Creating a workflow using a data input list of search query URLs or listing URLs as the input and then adding the scraping recipe. 

Method #3 – Creating a workflow using the scraping recipe as the input and another scraping recipe to get the details (two-step scraping).

Method #1 Create a new workflow from a recipe

This method highlights how to create a workflow from the saved public recipe to get the scraped data and export it to CSV or Google Sheets.

You can run this method to scrape Product Hunt homepage listing, Subreddits, AppSumo trending deals, etc.

Step 1: Create a new workflow

From your dashboard, create a new workflow using the blank version.

Step 2: Add your recipe

From the Public scraping recipes list, select the desired recipe. In this case, we will add the Product Hunt homepage data recipe to get a list of startup URLs, startup names, and descriptions from the home page of Product Hunt.

Then, click Continue.

Step 3: Run your workflow

Now you can run your workflow. 

Step 4: View and save the results

Once the workflow has finished running, you can view the results and export them to CSV or Google Sheets. 

Method #2 Create a workflow using a data input with a list of URLs

This section will demonstrate how you can add a list of URLs using the data input option to scrape different categories automatically. 

You can apply this method to scrape different category data from Yellow pages, eBay, Indeed listing, AppSumo product page, Quora, Craigslist, Shopee.

Step 1: Create a new workflow with data input

Get back to your dashboard and create a new workflow by choosing the “blank” version. Then select the Data input automation as your starting point.

Step 2: Add a list of URLs

Next, add a list of different category URLs using the Manual paste/List of inputs option.
In this case, we are using Yellow pages public scraping recipe where there is restaurant data of Los Angeles. Here, we will add URLs of other category pages (for example, dentists, hotels, and travel agencies)  to get these data automatically.

Step 3: Add the previously saved recipe

Next, add the desired public scraping recipe (In this example, it’s the Yellow pages recipe), selecting data input as the source. 

Then click Continue.

Step 4: Run your workflow

Now, you can run your workflow to get the scraped data. 

Step 5: View and save the results

Once the workflow has finished running, you can view the results and export them to CSV or Google Sheets.

Method #3 – Create a workflow to run two-step scraping

This section demonstrates how to use our public scraping recipes for the two-step scraping. So, we will use one scraping recipe as the input and another scraping recipe to get the details.

This method can be applied for the following public scraping recipes: AppSumo trending deals + AppSumo product page, Product Hunt homepage + Product Hunt product page, Indeed listing page + Indeed single job posting. 

Step 1: Create a new workflow with scraping recipe

Go to your dashboard and create a new workflow by choosing the “Blank” version.

Step 2: Add the product listing recipe

In this case, we will first add the AppSumo trending deals page from the list of public scraping recipes. 

Step 3: Add the product page recipe

Next, we will add AppSumo separate product detailed page, selecting “Link” as the source.

Then, click Continue. 

Step 4: Run your workflow

Now, you can run your workflow to get the results. 

Step 5: View and Save the results

Once the workflow has finished running, you can view the results and export them to CSV or Google Sheets. 

As a result, we have got the AppSumo trending deals along with detailed information for each deal (reviews, overview, description, terms and conditions, founder name, website URL, etc).

---

Automate & scale time-consuming tasks like never before

.