【Smart Mode】【Flowchart Mode】How to set up IP Rotation | Web Scraping Tool | ScrapeStorm

In the process of scraping the webpage data, we may encounter the situation that the IP is blocked by the website. At this time, we can enable “IP Rotation” to switch the IP.

This function is set in the run settings, as shown below:

ScrapeStorm currently supports two IP Type：Bright Data Zone and Custom IP.

If you choose Bright Data Zone, you can refer to this turorial:

How to Set up Bright Data Zone

If you choose Custome IP, you can click on “Set Up”.

IP will automatically switch according to the conditions you set. There are two switching conditions:

(1) Interval: You can set the proxy to switch every 1 minute, 3 minutes or 5 minutes.

Note: Switching by time does not mean that the IP is switched when the time is up, because the operation of the webpage (such as opening a webpage or clicking a button, etc.) requires a loading process. After the previous page operation and before the next operation, the webpage needs load and display the content. If the set time interval has been reached, the proxy IP will not be swith at this time, because switching the proxy IP will cause the current page content to change, so even if the set switching time has arrived, you still need to wait until the next page operation before switching the proxy IP.

(2) Text appears on the page: You can enter text as the switching condition.

As show below, you can set the switching condition to “Type the character you see in this image“, the proxy will be switched once the corresponding text appears on the page.

If there are multiple conditions for switching text, please use || to split, for example, A||B||C, which means that the proxy is switched when A or B or C appears.

Note：When setting the switching condition to this option, the IP will be switched only when the text is completely matched. If the text changes, it will not work. Sometimes the anti-spider hint in the page is not a text, but a picture. In this case, this option is not appropriate.

There are no advantages or disadvantages between the two options. You need to debug according to the characteristics of the website to achieve a higher cost-effective ratio.

 

Common problem:

1. Why can’t I buy a proxy IP?

IP Rotation is an advanced feature and is only available to users of the Professional Plan and above.

2. How long can I use the proxy IP I bought?

The proxy IP is used according to your setup conditions. The setup conditions are different and the speed consumed is different.

3. Can proxy IP be used for monthly subscription?

Temporarily not.

4. Is the proxy IP working for the current task or for all tasks?

Each task is independent, and the proxy IP set by the current task only works for the current task.

5. When switching proxy, will the software automatically detect the verification code and prompt

One of the purposes of the IP Rotation is to not require a CAPTCHA operation.
For the case where a CAPTCHA appears during scraping, we recommend using the text of the CAPTCHA page as the switching condition.
If the CAPTCHA appears again after the proxy switch, the software will automatically switch the proxy until the CAPTCHA no longer appears.

Therefore, when using it for the first time, we recommend that you observe the test results and give us feedback in time.

If the CAPTCHA page is a picture and there are no text conditions available, you will need to use time as the switch condition.
Between the two proxy switches, if a CAPTCHA appears, the software will automatically pause the task and prompt.
If the user has not operated when the next switch time arrives, the software will switch the proxy and continue to run the scraping task.