Python y VPNs (1)
Today many people already knows VPNs, whether for personal or professional use. One of the advantages that they offer are for things such as scrapping, because many requests from the same IP might be sus and will get us flagged, thwarting our projects...
This post is quite old, the mentioned API was deprecated logn ago, but I’ll keep the post here.
Today many people already use VPN, wether for personal or professional use.
One of the advantages that VPN’s offer is in the use of scraping. This is because a lot of requests made from the same IP will probably raise a flag from the provider, thwarting the project we are working on.
Back in the day I got myself a nordVPN account, this company has servers all around the world. However there’s a problem, if we want to connecto the Spanish servers, the app chooses automatically by location and connectivity rating, so we cannot just choose servers freely.
Luckily for us, the NordVPN client allows us to choose arbritarily (knowing the server’s name), so let’s create a list of NordVPN servers and have a pool of random VPNs.
Easy, right?
Retrieving the server list
Good news, the own API of NordVPN gives us a JSON with all the servers, this will make our life much easier.
If we CURL https://nordvpn.com/api/server, we get the following:
...
{
"id": 976484,
"ip_address": "122.155.174.96",
"search_keywords": [
"sha512"
],
"categories": [
{
"name": "Standard VPN servers"
}
],
"name": "Thailand #19",
"domain": "th19.nordvpn.com",
"price": 0,
"flag": "TH",
"country": "Thailand",
"location": {
"lat": 13.753979,
"long": 100.501444
},
"load": 88,
"features": {
"ikev2": true,
"openvpn_udp": true,
"openvpn_tcp": true,
"socks": false,
"proxy": false,
"pptp": false,
"l2tp": false,
"openvpn_xor_udp": false,
"openvpn_xor_tcp": false,
"proxy_cybersec": false,
"proxy_ssl": true,
"proxy_ssl_cybersec": true,
"ikev2_v6": false
}
}
...
The keys that are of interest for us are categories.name and domain.
Categories.name
This key gives us the type of VPN, having:
- Standard VPN servers
- Dedicated IP
- Double VPN
- Obfuscated servers
- P2P
We’re only interested in the Standard VPN servers because the others are already used for certain user cases.
Domain
Remember we said before we can connect to them arbitrarily?
In this case, the subdomain gives us the server name, so if we have th19.nordvpn.com, we have that th19 is the server name, so we can connect with:
$ nordvpn c th19
Dealing with the JSON
Firstly, let’s download the JSON to test in local, once we finish the test, we can replace the local with the webpage:
$ curl https://nordvpn.com/api/server > dump
Now, let’s check the number of servers:
import json
with open('dump', 'r') as c:
servers = json.load(c)
print(len(servers))
This gives us 5469 servers. As we mentioned before, we are only interested in the Standard, so let’s filter:
import json
with open("dump", "r") as c:
servers = json.load(c)
servers_def = []
for server in servers:
try:
if server["categories"][0]["name"] == "Standard VPN servers":
servers_def.append(server)
except:
pass
print(len(servers_def))
With this, we get 4840 servers ready for our use.
Now let’s extract the names of the filtered servers:
import json
with open("dump", "r") as c:
servers = json.load(c)
def vpn_extract():
servers_def = []
for server in servers:
try:
if server["categories"][0]["name"] == "Standard VPN servers":
servers_def.append(server["domain"].split(".")[0])
except:
pass
return servers_def
With this function, we are extracting a list with the names of the servers that we want, we’re almost done, now let’s just implement it in the script.
It’s simple, we create a function that will invoke the program’s command with the randomly chosen server by our script, it will connect and then it will do whatever we want with it.
It’s a simple script, but this easy IP rotation will save us a lot of headaches, we can also change the UA to make the requests more legit.
import json
from time import sleep
import subprocess
import random
import requests
def vpn_extract():
servers_def = []
web = requests.get("https://nordvpn.com/api/server")
servers = json.loads(web.text)
for server in servers:
try:
if server["categories"][0]["name"] == "Standard VPN servers":
servers_def.append(server["domain"].split(".")[0])
except:
pass
return servers_def
def vpn_connect():
result = subprocess.run(
["nordvpn", "c", random.choice(vpn_extract())],
capture_output=True
)
while 1:
vpn_connect()
sleep(5) # Hacemos lo que tengamos que hacer.
In next episodes we will continue with this base script doing VPN rotation and applying this to real uses, for example scraping a web so we’re not easily banned.