Storage API

The Storage API in ScrapeSuite enables the retrieval of results from the storage. This section introduces five primary endpoints:

Explorer

This endpoint is designed for navigation, allowing you to obtain a list of folders in a directory or results for a specific path. The resulting output can be with “File” or “Directory” type.

GET https://app.scrapesuite.com/public/storage/explorer?path={path}&token={token}

Description of the fields for sending a request:

Field	Type	Description
`path`	string	file or directory path required parameter
`token`	string	authorization token required parameter

Description of the fields in the results array for the “file” type:

Field	Type	Description
`result`	object	object with the result
`type`	string	result type: "file"
`name`	string	name of the ScrapeSuite parser result file
`create_time`	datetime	timestamp of the file creation
`last_update_time`	datetime	timestamp of the last file update
`request_url`	string	the original URL provided for parsing
`response_url`	string	the URL used for the API request
`result_status`	enum	internal server status code
`status_code`	integer	response HTML status code
`processing_time`	float	time taken to complete the task
`cost`	float	cost of the task in credits
`error_message`	string	general informational message
`has_html`	boolean	indicates whether associated HTML content is present
`has_result`	boolean	indicates whether associated parsing result is present
`result`	object	object containing results
`status`	enum	server response code
`elapsed`	time	time of request execution

Response example:

{
  "result":{
    "type":"file",
    "name":"https_scrapesuite_com_2350893657",
    "create_time":"2024-02-06T17:57:33.301Z",
    "last_update_time":"2024-02-06T17:57:33.301Z",
    "request_url":"https://scrapesuite.com/",
    "response_url":"https://scrapesuite.com/",
    "result_status":"ok",
    "status_code":200,
    "processing_time":8.6598564,
    "cost":10.0,
    "error_message":null,
    "has_html":true,
    "has_result":true,
    "result":{
      "Main":"Welcome to ScrapeSuite - Your Ultimate Website Parsing Solution",
      "xpath":"/html[1]",
      "stream_position":15
    }
  },
  "status":"ok",
  "elapsed":"00:00:00.0082719"
}

Description of the fields in the results array for the “directory” type:

Field	Type	Description
`result`	object	object with the result
`type`	string	result type: "directory"
`name`	string	name of the ScrapeSuite parser result file
`create_time`	datetime	timestamp of the file creation
`items`	array	array of files and subdirectories in the directory
`status`	enum	server response code
`elapsed`	time	time of request execution

Response example:

{
  "result":{
    "type":"directory",
    "name":"ScrapeSuite_API",
    "create_time":"2024-01-23T10:46:14.67Z",
    "items":[
      {
        "type":"directory",
        "name":"controllers",
        "create_time":"2024-01-23T10:46:14.67Z"
      },
      {
        "type":"directory",
        "name":"jobs",
        "create_time":"2024-01-23T10:46:14.67Z"
      }
    ]
  },
  "status":"ok",
  "elapsed":"00:00:00.0138808"
}

Bulk

This endpoint is useful for retrieving a list of all parsing results from a folder. You can get 100 results in a single request. You can use offset_token to get more results.

GET https://app.scrapesuite.com/public/storage/bulk?path={path}&token={token}&offset_token={offset_token}

Description of the fields for sending a request:

Field	Type	Description
`path`	string	directory path required parameter
`token`	string	authorization token required parameter
`offset_token`	string	token for subsequent requests optional parameter

Description of the fields in the results array:

Field	Type	Description
`result`	object	object with the result
`type`	string	result type: "file"
`name`	string	name of the ScrapeSuite parser result file
`create_time`	datetime	timestamp of the file creation
`last_update_time`	datetime	timestamp of the last file update
`request_url`	string	the original URL provided for parsing
`response_url`	string	the URL used for the API request
`result_status`	enum	internal server status code
`status_code`	integer	response HTML status code
`processing_time`	float	time taken to complete the task
`cost`	float	cost of the task in credits
`error_message`	string	general informational message
`has_html`	boolean	indicates whether associated HTML content is present
`has_result`	boolean	indicates whether associated parsing result is present
`result`	object	object containing results
`status`	enum	server response code
`elapsed`	time	time of request execution

Response example:

{
  "result":[
    {
      "type":"file",
      "name":"https_scrapesuite_com_2350893657",
      "create_time":"2024-02-06T19:10:22.633Z",
      "last_update_time":"2024-02-06T19:10:22.633Z",
      "request_url":"https://scrapesuite.com/",
      "response_url":"https://scrapesuite.com/",
      "result_status":"ok",
      "status_code":200,
      "processing_time":3.3728007,
      "cost":10.0,
      "error_message":null,
      "has_html":true,
      "has_result":true,
      "result":{
        "Main":"Welcome to ScrapeSuite - Your Ultimate Website Parsing Solution",
        "xpath":"/html[1]",
        "stream_position":15
      }
    },
    {
      "type":"file",
      "name":"https_scrapesuite_com_docs_708564445",
      "create_time":"2024-02-06T19:10:23.643Z",
      "last_update_time":"2024-02-06T19:10:23.643Z",
      "request_url":"https://scrapesuite.com/docs/",
      "response_url":"https://scrapesuite.com/docs/",
      "result_status":"not_valid_result",
      "status_code":200,
      "processing_time":4.3824,
      "cost":10.0,
      "error_message":null,
      "has_html":true,
      "has_result":false,
      "result":null
    }
  ],
  "next_token":null,
  "status":"ok",
  "elapsed":"00:00:00.0167370"
}

Results

This endpoint is convenient for obtaining multiple results, knowing the directory and file names.

POST https://app.scrapesuite.com/public/storage/results?path={path}&token={token}

Description of the fields for configuring a URL:

Field	Type	Description
`path`	string	directory path required parameter
`token`	string	authorization token required parameter

Description of the fields for configuring a JSON body of the POST request:

Field Type Description

files

array

file names

required parameter

example: ["https_scrapesuite_com_docs_708564445","https_scrapesuite_com_2350893657"]

Description of the fields in the results array:

Field	Type	Description
`result`	object	object with the result
`type`	string	result type: "file"
`name`	string	name of the ScrapeSuite parser result file
`create_time`	datetime	timestamp of the file creation
`last_update_time`	datetime	timestamp of the last file update
`request_url`	string	the original URL provided for parsing
`response_url`	string	the URL used for the API request
`result_status`	enum	internal server status code
`status_code`	integer	response HTML status code
`processing_time`	float	time taken to complete the task
`cost`	float	cost of the task in credits
`error_message`	string	general informational message
`has_html`	boolean	indicates whether associated HTML content is present
`has_result`	boolean	indicates whether associated parsing result is present
`result`	object	object containing results
`status`	enum	server response code
`elapsed`	time	time of request execution

Response example:

{
  "result":[
    {
      "type":"file",
      "name":"https_scrapesuite_com_2350893657",
      "create_time":"2024-02-06T19:10:22.633Z",
      "last_update_time":"2024-02-06T19:10:22.633Z",
      "request_url":"https://scrapesuite.com/",
      "response_url":"https://scrapesuite.com/",
      "result_status":"ok",
      "status_code":200,
      "processing_time":3.3728007,
      "cost":10.0,
      "error_message":null,
      "has_html":true,
      "has_result":true,
      "result":{
        "Main":"Welcome to ScrapeSuite - Your Ultimate Website Parsing Solution",
        "xpath":"/html[1]",
        "stream_position":15
      }
    },
    {
      "type":"file",
      "name":"https_scrapesuite_com_docs_708564445",
      "create_time":"2024-02-06T19:10:23.643Z",
      "last_update_time":"2024-02-06T19:10:23.643Z",
      "request_url":"https://scrapesuite.com/docs/",
      "response_url":"https://scrapesuite.com/docs/",
      "result_status":"not_valid_result",
      "status_code":200,
      "processing_time":4.3824,
      "cost":10.0,
      "error_message":null,
      "has_html":true,
      "has_result":false,
      "result":null
    }
  ],
  "status":"ok",
  "elapsed":"00:00:00.0173737"
}

View HTML

This endpoint is convenient for viewing or displaying HTML content.

GET https://app.scrapesuite.com/public/storage/view_html?path={path}&token={token}

Description of the fields for sending a request:

Field	Type	Description
`path`	string	file path required parameter
`token`	string	authorization token required parameter

Response example:

HTML

This endpoint is useful for storing HTML content.

GET https://app.scrapesuite.com/public/storage/html?path={path}&token={token}

Description of the fields for sending a request:

Field	Type	Description
`path`	string	file path required parameter
`token`	string	authorization token required parameter

Description of the fields in the results array:

Field	Type	Description
`result`	object	object with the result
`html_content`	string	HTML of the specified webpage
`status`	enum	server response code
`elapsed`	time	time of request execution

Response example:

{
  "result":{
    "datetime":"2024-02-05T15:49:29.963+00:00",
    "html_content":"..."
  },
  "status":"ok",
  "elapsed":"00:00:00.0068108"
}

Storage API

Explorer

Bulk

Results

View HTML

HTML

PRIVACY & TOS

CONTACT US