Python y VPNs (1)

Today many people already knows VPNs, whether for personal or professional use. One of the advantages that they offer are for things such as scrapping, because many requests from the same IP might be sus and will get us flagged, thwarting our projects...

Info

This post is quite old, the mentioned API was deprecated logn ago, but I’ll keep the post here.

Today many people already use VPN, wether for personal or professional use.

One of the advantages that VPN’s offer is in the use of scraping. This is because a lot of requests made from the same IP will probably raise a flag from the provider, thwarting the project we are working on.

Back in the day I got myself a nordVPN account, this company has servers all around the world. However there’s a problem, if we want to connecto the Spanish servers, the app chooses automatically by location and connectivity rating, so we cannot just choose servers freely.

Luckily for us, the NordVPN client allows us to choose arbritarily (knowing the server’s name), so let’s create a list of NordVPN servers and have a pool of random VPNs.

Easy, right?

Retrieving the server list

Good news, the own API of NordVPN gives us a JSON with all the servers, this will make our life much easier.

If we CURL https://nordvpn.com/api/server, we get the following:

...
{
  "id": 976484,
  "ip_address": "122.155.174.96",
  "search_keywords": [
    "sha512"
  ],
  "categories": [
    {
      "name": "Standard VPN servers"
    }
  ],
  "name": "Thailand #19",
  "domain": "th19.nordvpn.com",
  "price": 0,
  "flag": "TH",
  "country": "Thailand",
  "location": {
    "lat": 13.753979,
    "long": 100.501444
  },
  "load": 88,
  "features": {
    "ikev2": true,
    "openvpn_udp": true,
    "openvpn_tcp": true,
    "socks": false,
    "proxy": false,
    "pptp": false,
    "l2tp": false,
    "openvpn_xor_udp": false,
    "openvpn_xor_tcp": false,
    "proxy_cybersec": false,
    "proxy_ssl": true,
    "proxy_ssl_cybersec": true,
    "ikev2_v6": false
  }
}
...

The keys that are of interest for us are categories.name and domain.

Categories.name

This key gives us the type of VPN, having:

  • Standard VPN servers
  • Dedicated IP
  • Double VPN
  • Obfuscated servers
  • P2P

We’re only interested in the Standard VPN servers because the others are already used for certain user cases.

Domain

Remember we said before we can connect to them arbitrarily?

In this case, the subdomain gives us the server name, so if we have th19.nordvpn.com, we have that th19 is the server name, so we can connect with:

$ nordvpn c th19

Dealing with the JSON

Firstly, let’s download the JSON to test in local, once we finish the test, we can replace the local with the webpage:

$ curl https://nordvpn.com/api/server > dump

Now, let’s check the number of servers:

import json

with open('dump', 'r') as c:
    servers = json.load(c)

print(len(servers))

This gives us 5469 servers. As we mentioned before, we are only interested in the Standard, so let’s filter:

import json

with open("dump", "r") as c:
    servers = json.load(c)

servers_def = []

for server in servers:
    try:
        if server["categories"][0]["name"] == "Standard VPN servers":
            servers_def.append(server)
    except:
        pass

print(len(servers_def))

With this, we get 4840 servers ready for our use.

Now let’s extract the names of the filtered servers:

import json

with open("dump", "r") as c:
    servers = json.load(c)

def vpn_extract():
    servers_def = []
    for server in servers:
        try:
            if server["categories"][0]["name"] == "Standard VPN servers":
                servers_def.append(server["domain"].split(".")[0])
        except:
            pass
    return servers_def

With this function, we are extracting a list with the names of the servers that we want, we’re almost done, now let’s just implement it in the script.

It’s simple, we create a function that will invoke the program’s command with the randomly chosen server by our script, it will connect and then it will do whatever we want with it.

It’s a simple script, but this easy IP rotation will save us a lot of headaches, we can also change the UA to make the requests more legit.

import json
from time import sleep
import subprocess
import random
import requests

def vpn_extract():
    servers_def = []
    web = requests.get("https://nordvpn.com/api/server")
    servers = json.loads(web.text)

    for server in servers:
        try:
            if server["categories"][0]["name"] == "Standard VPN servers":
                servers_def.append(server["domain"].split(".")[0])
        except:
            pass

    return servers_def

def vpn_connect():
    result = subprocess.run(
        ["nordvpn", "c", random.choice(vpn_extract())],
        capture_output=True
    )

while 1:
    vpn_connect()
    sleep(5)  # Hacemos lo que tengamos que hacer.

In next episodes we will continue with this base script doing VPN rotation and applying this to real uses, for example scraping a web so we’re not easily banned.