Quickstart

Registration

After clicking, you will be redirected to the registration form at https://app.scrapesuite.com/auth/sign-up. You need to enter your Email, Name, Password, Password Confirmation, and check the box for “I Accept the Terms”. Setting an agreement to “I Accept the Terms” during registration or login is standard practice to ensure that the user has reviewed and agreed to the terms of use or privacy policy.

If you already have an account and mistakenly landed on this page, use Sign in button to navigate to the login page.

After registration, you need to confirm your email. Upon completing the registration, you will be redirected to the Login page, where you can log into the application. The first page is the list of projects at https://app.scrapesuite.com/projects, where you can start your work.

Login

If you already have an account, you can always log in by visiting the https://scrapesuite.com/ page, where, in the upper right corner, you need to click the Login button.

After clicking, you will be redirected to the login form on the https://app.scrapesuite.com/auth/sign-in page.

Next, you need to enter the email of the registered account and the password, then click Sign in. As an alternative, you can use the Google account login option or register through a Google account.

If you have forgotten your password and need to recover it, you can use the Forgot password option. This will redirect you to the password recovery form, where you need to enter the previously registered email and click Submit.

Create project

After registration, you can finally start configuring the parser. The first step is to create a project on the https://app.scrapesuite.com/projects page.

To create a new project, click the + button in the upper right corner under your account name.

A form for creating a new project will open.

In the open form, configure the following items:

Project Name

Please be sure that your project name follows the submission guidelines – the name has to contain only latin lowercases, numbers, and underscore.

Default Localization
Load Setting

We recommend giving your project a meaningful name that reflects the purpose of your project. Keep in mind that after creating the project, you cannot change its name. We’ll name our project “Demo”. Next, choose the desired localization from the drop-down list for Default Localization.

In this field, it is crucial to choose the abbreviation of the country for which the resource you want to scrape is targeted. Some data, such as numbers, time, and prices, are localization-dependent. This setting will be used by default when loading new pages and creating controllers and jobs. We recommend using “en-US” if the resource of interest is targeted at the American market.

Load Setting block:
These settings will be used by default when loading new pages, creating controllers, and creating jobs.

Proxy country: Specifies the country for the requests if it matters from which country the requests will come for scraping the resource. We will leave the default value as Worldwide.
Enable web browser rendering: This needs to be enabled if the website has content or blocks that require JavaScript execution to fetch data. This is useful if certain elements do not display correctly without JavaScript execution. You can enable this setting at any time in the parser configuration.

Therefore, we recommend ensuring that you cannot gather data without enabling this setting in the first place.

After filling in the details, click Create.

We have created our first project!!

Result of creating the project:

After creating the project, you can modify the project settings at any time by clicking the Settings button, which will open the settings form you filled out earlier.

(Please note that you cannot change the project name.)

To delete the project, you can use the Delete Project (bin) button

After clicking, we want to ensure that you are ready to delete your project, as deleting the project may be irreversible. Therefore, we ask you to confirm the deletion by clicking Yes in the prompt window.

Be sure of your decision, as after deleting your project, it cannot be recovered.

To access your project, click the project name. Now you can start working.

You will be greeted by our Dashboard, where only the most essential points for your project are gathered.
You can learn more about each of them in the Documentation section.

Add HTML

We go to the HTML tab to add URLs based on which we will configure our parser settings for the project. Click the tab named “HTMLs”.

You will see a list of HTMLs. Since this is the first time, and you haven’t added your HTMLs yet, don’t worry that it’s empty. Ours is empty right now too.

In this tab, you need to create an HTML. Click the + sign under your name:

A form will appear that you need to fill out.

Enter the HTML name, then in the URL field, enter the specific page based on which you will build your parser. The Localization section will, by default, use what you specified in the project. You can change it if you want, but we’ll leave it as is for now.

Please be sure that your project name follows the submission guidelines.

In our test example, we will use the URL https://us.ecoflow.com/products/delta-2-portable-power-station?variant=40569176326217.

We will add it to our HTML and click Create.

Now, by clicking on the name of your entered HTML, you can go to the parser settings.

Create the First Parser

Settings Page:

At this point, we haven’t started the configuration yet, so you can see only default data when starting to work.

At this stage, we recommend thinking about the information you would like to extract for products from this category or website.

Our List: title, URL, price, description, option, specs, breadcrumb

Title
First of all, you need to create a Property. This is a customizable attribute that needs to be created for each element. We will name it “title” (based on the data we want to retrieve). You can create Properties for all elements at once and then add selectors for the data you want to retrieve. For the first time, we recommend that you follow along with us step by step.

For this Property, we will choose Create text selector from the Property dropdown menu.

After creating the Text selector, you need to choose the element that will be in this binder, and click it.

It’s worth checking what you get as a result of your selection. You can see that the selected element in Selected Elements matches the Result Elements, and more importantly, we are getting these elements, as indicated by the Parsed icon.

In this case, there is no need to reconfigure our text selector; this is the result we wanted to achieve.

If you are satisfied with the result of the element you selected, you can use the Deploy button.

After clicking it, these settings will be applied to your parser’s operation.
You can also always go back a step in your configuration using the parser Timeline located at the bottom of the page. It shows all the actions you perform in the element tree, including deletions or additions.
Timeline (example)

You may notice that the icons match the icons in your settings.
(example)

Let’s continue!

URLs
Next in our list, we want to obtain URLs. To get URLs, we need to create a new Property (as mentioned earlier). Let’s do that. And immediately for URLs, choose Create attribute selector.

Next, we need to select our URLs.

In some cases, you may need to hide certain elements on the page for the convenience of selecting the element you need, and that’s what we’ll do in our case.
To do this, you have the option in the Main container called “Hidden Elements”.

Click

and choose what specific elements you want to visually hide from the page. This setting will not affect the HTML representation of the page in any way.
If you accidentally included the element you need in this list, just click

and the page element will be visible again in the display.

In our case, we want to obtain the elements of the product tree.
Here you can see where the URL is located in the HTML markup and where it is on the actual page.

We also used the Post-processing setting to obtain the full product URL. To get the full URL, you need to choose Get absolute URL in the Post-processing settings.

You can also compare your result with our result, which we obtain.

Price
To obtain the Price data, we should initially pay attention to the fact that this item is built from several parameters: regular price, discounted price, and the discount amount.

In this case, it would be better for us to use a container. First, create a Property (main -> Create Property) and name it “price.” Also, name the container “price” (Property -> Create Container). As a result, we should have such a hierarchy.

Colors may differ, don’t worry, as it doesn’t affect anything.

Next, we need to select elements for the container. To choose the price, we need to understand that not all products may have a discount. Therefore, for the container, we need to choose a more general setting to ensure that the elements are always in their places.

To start, we need to select the price element and click the Additional select toggle:

It will help us capture the price group in the HTML markup of the page. If we look at the HTML markup, we will see that the group is called “IncrementPriceGroup.” So, let’s choose this element in the markup:

<div class="price__pricing-group">

Next, we want to retrieve each price from our selected container. Let’s create a Property in our container named “Price” with the title “Sale”. In this Property, create a text selector (Create text selector), and here we will add the sale price. Choose this price.

Note that we obtained the result “USD $649.00” with the currency symbol and conditional abbreviation. If you want to get only “649” then you need to use Post-processing with the “Get number from string” option and click Save.

We will apply these changes on our end and get a result like this.

Let’s add the regular price of this product. In our Price container, create a Property named “regular,” and within this Property, create a text selector (Create text selector). Here, we’ll add the regular price of the product. Repeat the steps as with the sale price. On the page display, the price you want to obtain should be highlighted, even if it’s blurred.

Great!

For your convenience, let’s also get the currency of the product so that you can be sure you are comparing identical values in the future. Let’s do this through the element where we see the currency symbol.

In the Price container, create a Property named “currency”, and create a text selector (Create text selector). Next, select the element “USD $649.00” and apply Post-processing with the Custom post-processing value, where we can use return value?value.split(' ')[0]:null. After this, you should be able to obtain “USD” and click save. Our result:

Note that your Timeline may not match ours, as you may have had additional actions; that’s okay.

Description

Next, we will gather the “description” element. This element will already be in the general “main” container, so we’ll create it there. In the “main” container, create a Property named “description” and choose “Create text selector.” On the page and in the HTML markup, you can see that the description has a convenient class:

<div class="product-desc">

For convenient selection, you can enable

so that you can choose all items at once.

In the end, you will receive all the information from the description.

Let’s try to get the description result, but broken down into sections. In the main container, create a Property named “description_parts” and enable the Is Array option

Now, in the Property, select Create text selector. Choose the desired sections; it’s advisable to select 2 or 3 sections, as the rest will automatically appear in your Result Elements. These sections will be highlighted with dashed lines on the page. As a result, you should get an array of data.

Let’s collect our next set of data.

Options

We see that the data referred to as Options on the website have its own Title and Price. In such a case, we recommend creating a new container and naming it in a way that makes sense to you. In our example, we will name it “Option”.

First, we create a Property named “Option”, where we should choose the Is array setting since we will have an array of data. It may also be helpful to add the Is optional setting (but this remains at your discretion), as not all products may have Options.

Next, we create a container with the same name, “Option”. Here, we need to choose what we want to see in this container.

Use this Additional select toggle:

Select the element from the Option item, the first item Delta 2, our smart parser will suggest you the full class you need to choose. Next, for Delta 2 + Delta 2 Smart Extra Battery, repeat and press Press to compile selecting.

This way you should get the result as shown in the screenshot below.

Now we need to add a Property for adding the selector itself. Let’s name this Property “options_title”. Our next step is to add a Create text selector, where we will choose the title we need. As a result, you should get such a tree and JSON result:

Next, for our Options, we want to add options_price. In the Option container, create a Property named “option_price,” and then, as before, create a text selector, Create text selector. Use Get number from string in Post-processing, as in the previous steps, to obtain only the price.

As a result of selecting the price and processing, we should obtain two numbers without currency symbols.

Specs
Now we can move on to parsing a section such as Specs. On our page, this section is displayed as follows:

Here, we need to, as in the previous steps, create a Property, and we’ll name it “specs.” Don’t forget, as before, in the Property settings, we need to enable the Is array option since we’ll have an array of data. You can also enable Is optional; not all products may have this data. In our parser, we’ll leave it as is.

Add to our Property a container named “specs” and select several blocks with information. In this way, as before, specify to our parser where exactly we will be collecting data.
As a result, you will have something like this:

You may notice that there are blocks selected with a solid line and those selected with a dashed line. The solid line is our choice, and the dashed line is the choice of our smart parser, as we specified before. If you want to exclude an element, you need to select the Exclude status for it in the container settings.

If you want to remove an element that the parser added automatically, then you need to add it through the Result element

and choose the Exclude status from the drop-down menu.

Next, create a property called “specs_title” without changing the settings and add a text selector using Create text selector. After selecting several elements, you should get something like this:

The complete result will look something like this:

Now let’s also add to the specs container a Property named “specs_value” and add a text selector using Create text selector.
The result should be something like this:

Breadcrumb
The last element we want to add is the breadcrumb. Let’s add it.
The first thing we’ll do is create a Property named “breadcrumb.” Since we need to obtain both the title and URL here, we will select the Is array option for our Property. Immediately after, add a container named “breadcrumb” and choose the elements we need using the Select tool.

The result of the selection using the Additional select will be as follows:

Next, we need to create a Property in the breadcrumb container with the name “breadcrumb_title” and add a text selector. Use Create text selector.
And get such a result:

Don’t worry if our Timeline doesn’t match yours; it’s normal. You may have taken different steps, or we might have used the Timeline as intended, revisiting steps as needed. Also, remember that the colors of our settings tree may differ.

Add to the breadcrumb container a Property named “breadcrumb_url”. Create an attribute selector using Create attribute selector in the Property.
Select the elements you need and press Press to compile selecting button to confirm your choice.

Also, in the attribute selector, set up Post-processing and choose Get absolute URL, so we will obtain the full path of these links. You can compare it with our result.

With that, our parser is ready!

Don’t forget to click the Deploy button at this stage, which you can find at the top of your parser settings.

If you haven’t exited or pressed it, you can always click the button in the project Dashboard.

The Usage actual version status indicates that you have pressed the Deploy button. Next, you can configure the Controller, which you can find in the product Dashboard.

If you don’t know how to access the project dashboard, click ScrapeSuite and navigate to your project.

Configure controller

After configuring the parser and deploying the settings, we can go to the Controllers section and further customize the controller or use the base controller.

Here, you can modify/add a controller if you decide that the settings of the base controller do not suit your job. However, there is no need to change settings for the entire project. We will use the base controller.

Add a new controller by clicking the Add button.

Configure job

Next, go to the Jobs tab, and you need to create your first job.

You can find the Jobs tab on the project dashboard page.

Click Add button to add the settings.

Fill in the fields.

In our case, we will name our Job “Demo” and leave the base_controller. In our example, we will not activate Notification settings, but you can use these settings, and they will be related to receiving notifications at the completion of the job.

After clicking Create, your Job will be created.

If you need to reconfigure your Job, you need to revisit this window and click the gear button next to your Job.

Also, if you want to delete your Job, you can click the bin button and press Yes.

If everything is fine, and you are ready to proceed to get data using your configured parser, click the Job configuration button. The configuration button is represented by the wrench icon:

Configuration:

Here, we create a specific list of URLs that we need to process using your configured parser.

There are several ways to configure the job:
1. If you already have the URLs you need.
2. If you do NOT have the URLs you need.

We will use the method where we do NOT have pre-existing URLs that we could add, but let’s talk about both methods.

1. If you already have the required URLs:

If you have a list of already prepared URLs, you can paste it directly into the URLs field. If you already have product names, you can do the following.

If you already have product names, you can do the following:

and paste this list into the Plain text area. In the Field name, add name, and click

The result should be something like this, but with your list.

Now, when you return to the Jobs configuration, you need to paste https://us.ecoflow.com/products/$(name) into the URLs field. This way, you’ll obtain your URLs.

Also, if you have page numbering, you can add “page” as the name using the Add field and append it after “name” in the Range.

And on the Jobs configuration page, add /$(page) at the end. You will get https://us.ecoflow.com/products/$[name]/$[page] in the result.

Another way is if you have a CSV file, you can add such a file using the button you see labeled “Import CSV.”

2. If you DO NOT have the URLs you need:

If you don’t have such a list, as we do, we recommend you use the following steps:

1. Click the Import from sitemap button

2. Fill in the following fields:

Website – us.ecoflow.com
Pattern – */products/*
Limits – 10

In this field, you specify the number of pages you want to retrieve from this site according to your Pattern. You can specify more pages. We want to test our parser on 10 pages before a full launch.

3. Click Import:

As a result, we will get 10 products from EcoFlow.

Don’t worry if you don’t know the exact quantity of these products on the site; you can simply try adding 100 items, and our configuration will return the actual number available on the given resource. Currently, we see 217 products after setting the limit to 300.

Next, click Save, which will automatically take you to the list of all available Jobs. Let’s go to our Job and click Run Job to get the result of our work. Now, let’s wait for all our pages to be processed. We can see that all our URLs have been processed and are ready for viewing.

Let’s click View result and see what results we have.

If you encounter any issues or see something like this:

Our next step will also be to click View result, and see what results we have.

As a result, we need to check, configure, and add the settings for the parser to one of the products in HTMLs. It’s normal if you couldn’t configure it on the first try.
Press Add HTML to project, and add the URLs.

For example, the product “ecoflow_river_2_portable” and add the name, which is “ecoflow_river_2_portable.” Press

After completion, it will take you to the configuration page of this URL. For example, it might look like this:

Let’s start with edits!

First of all, we collected data on Price. On the page, we can see that there is only a regular price here. In that case, we can indicate that Properties like price, sale, currency should be set as is_optional. Let’s add this setting. Also, it’s worth noting that in this case, the Regular and Sale prices will be the same.

You will see the following:

The data didn’t parse, but it wasn’t present on this page, so we indicate that it may not be present.
Our next correction will address the “description_parts” section.

In this case, we can see that the markup for this element may have changed.

Let’s add this element again.
For the text selector in the “description_parts” Property, select Add element.

And properties like option and specs need to be checked first to see if such data is present on the page. In our case, there is no such data on this URL, and to eliminate the error, we will activate the Is optional setting for the option and specs properties.

Sometimes it might help to remove the element and create it again, as one of the ways to set it up correctly.