Internet users know a “whitelist” as a list of IP addresses considered to be acceptable senders. Requests from addresses on a server’s whitelist are not filtered out. Whitelists are maintained by many websites, private companies, and domain name registries such as who.is.
Proxy servers also maintain their own lists of acceptable senders – users who register their IP addresses with the server. A proxy’s process is like whitelisting – identifying an IP address as an acceptable sender of data requests. You are, in effect, whitelisting when you authenticate your IP address to a proxy server.
In proxying, IP address authentication is one of two methods for authenticating to a proxy server. The other is username:password authentication. Many proxy services support configuration of the proxy host with Basic Authentication as well as IP address authentication. With the latter, you do not need to use a username and password. Although both methods increase your security, there are advantages as well as disadvantages to each authentication. Let’s talk about both methods.
Advantages of Username:Password Authentication
Username:password authentication requires the user to log in with a username and password, ensuring that only authorized users have access to sensitive information.
Username:password authentication is the most frequently used HTTP proxy authentication method. The credentials are created with the Basic Access authentication method and are passed in with an HTTP request in the
Proxy-Authorization header. Most HTTP client libraries support this method.
You have the advantage of needing only one username and password to connect all the proxies you have. You also may have the option of choosing a different password for each one of your proxies, to increase proxy security. Also, you can change your password(s) frequently, making it more difficult for hackers to gain access.
A username and password allows you to access your proxies from any location, especially useful if you frequently travel or access your proxies from more than one location.
Disadvantages of Username:Password Authentication
This method of authentication only provides basic access authentication – a lower level of security than IP authentication. That’s because it is not encrypted – it is only encoded, so it’s possible that someone in the middle could intercept and decode it.
And, of course, you have to remember your password. If you choose to keep a different password for each of your proxies to increase the security factor, you’ll have more passwords to remember. Easy-to-remember passwords are not recommended, since they’re also easier for hackers to figure out. To be strong, your password should be at least 8 characters long and include a mix of upper and lower case letters, numbers and symbols.
Another disadvantage is that the
Proxy-Authentication header is not always passed to the proxies correctly for HTTPS requests. Even many programming language clients don’t do that. Python Requests is one of the few that can do it correctly.
Advantages of IP Authentication/Whitelisting
IP Authentication/whitelisting is the easiest and most secure authentication method. When you add your IP address to your proxy service via dashboard or API, you are “whitelisting” that address with the proxy. This method allows you access to your proxies without the use of a password or username. Instead of sending a
Proxy-Authorization header, you use the IP address assigned to you by your Internet provider. Whitelisting is a great timesaver, especially if you have a lot of proxies.
Configuring for your browser or scraping client is simple. Just add your IP address for authentication to the proxy service. In many proxy services, you can do that either on a dashboard or through the service’s API.
For HTTPS requests with web browsers or Selenium, IP authentication is the most reliable and easiest method. Since there is no
Proxy-Authentication header, it is guaranteed to work without issues.
Disadvantages of IP Authentication/Whitelisting
Since access to your proxy is dependent upon the IP address where it was authenticated, you can only access your proxies from that IP address. This makes whitelisting less desirable if you frequently travel or access your proxies from different locations.
Some Internet service providers use a dynamic IP system that allows them to change your IP address without warning for security reasons. Whenever this occurs, you would need to authenticate a new IP address for your proxy.
Repeated failed login attempts can cause IP address blocks, which are usually temporary. You’ll need to wait until the ban expires even if an IP update was performed during a ban.
Unlike username and password authentication, a proxy system does not allow the same IP to be authenticated for multiple accounts.
Basically, the decision on which proxy authentication method to use depends upon your needs and any differences in cost for each method, as well as the level of security required. Having a good proxy service is also a vital part of the decision.