Using Proxy Server with Python Requests

To use a proxy server with Python requests, you can pass the proxy address to the Requests library method with the "proxies" parameter. The "proxies" parameter accepts a dictionary that can contain proxy servers for the HTTP, HTTPS, and FTP protocols: proxies={"http": http_proxy, "https": https_proxy, "ftp" : ftp_proxy}. You can also specify proxy servers using the HTTP_PROXY, HTTPS_PROXY, and FTP_PROXY environment variables (uppercase and lowercase names are supported). If both environment variables and the "proxies" parameter are specified, the Requests library will use the "proxies" value. In this Python Requests Proxy example, we are passing an HTTPS proxy to the request.get() method and requesting an "https" URL. Click Execute to run Python Proxy Example online and see the result.
Using Proxy Server with Python Requests Execute 
import requests

proxies = {'https': '10.10.10.10:3128'}

r = requests.get('https://reqbin.com/echo', proxies=proxies)

print(f'Status Code: {r.status_code}')
Updated: Viewed: 17448 times

What is a Proxy Server?

A Proxy server is an intermediate link between a user (browser, mobile application) and a website that transfers traffic between them. Proxies diverge end-user clients from the websites they browse and provide multiple levels of functionality and security. Proxy servers work as web filters and firewalls between the client and the Internet. When using proxy servers, all requests to the Internet first go to the proxy server, which evaluates the request, applies a set of rules to it, and then, if necessary, redirects the request to the Internet.

What is the Python Request Library?

Requests Library is a most popular Library that makes it uncomplicated to send HTTP requests using POST, GET and DELETE methods. The Requests library is based on the urllib3 library and hides the complexity of making HTTP requests behind a simple API. The Requests Library supports SSL connections, international domain names, and session cookies. The Requests Library is not contained in the Python distribution, but almost everyone uses Requests Library because the Python code for working with HTTP becomes straightforward, simple, and understandable.

How to install the Python Requests Library?

You can install the Requests Library using the pip package installer for Python:

Install Python Requests Library
pip install requests

After installing the Requests Library, you can use it in your code by importing the Requests Library with the following Python code: 

import requests

Passing Proxy Address to Python via Environment Variables

Another way to pass a proxy address to Python is to set the http_proxy and https_proxy environment variables. If you set environment variables, you don't need to define any proxies in your code. Just be aware that proxy settings passed through code now override settings set via environment variables.

Python Proxy Settings via Environment Variables (Linux)
export http_proxy="[host][:port]"
export https_proxy="[host][:port]"

Python Proxy Settings via Environment Variables (Windows)
set http_proxy="[host][:port]"
set https_proxy="[host][:port]"

What is an Environment Variable?

The environment variable is a key-value pair that is set at the system level and can affect all running processes on the computer. An environment variable is part of the operating system in which a process runs. In 1979, Unix introduced environment variables. Today, all major operating systems, including Linux, macOS, and Windows, support environment variables.

How to set a proxy for Requests Session object?

Sometimes you need to use a proxy for a Session object in Python. You can do this by creating a Session object and setting the Session.proxies object with a dictionary containing the list of proxies.

Python Proxy Requests Session Example
import requests

s = requests.Session()

s.proxies = {
  'http': '10.10.10.10:3128',
  'https': '10.10.10.10:3128',
}

r = s.get('https://reqbin.com/echo')

print(f'Status Code: {r.status_code}')

How to specify proxy authentication in Python?

To specify proxy server authentication, use the following syntax for your proxy server:

Python Proxy Authentication Example
import requests

proxies = {'https': 'user:[email protected]'}

r = requests.get('https://reqbin.com/echo', proxies=proxies)

print(f'Status Code: {r.status_code}')

How to use SOCKS proxy server with Python Requests?

In addition to the basic HTTP proxies, the Request Library also supports proxies that use the SOCKS protocol. To use the SOCKS proxy, you first need to install an additional library with pip: 

-m pip install requests[socks]

Once these dependencies are installed, a SOCKS proxy is no more complicated than an HTTP proxy. The following is an example of SOCKS proxy requests in Python:

Python SOCKS Example
import requests

proxies = {
  'http': 'socks5://user:pass@host:port',
  'https': 'socks5://user:pass@host:port'
}

r = requests.get('https://reqbin.com/echo', proxies=proxies)

print(f'Status Code: {r.status_code}')

What is a SOCKS Proxy?

Socket Secure (SOCKS) is a network communication protocol used to redirect network traffic from clients to real servers. The SOCKS proxy exchanges network packets between clients and web servers like the HTTP proxy. With SOCKS, you can transfer data transparently over HTTP, HTTPS, POP3, SMTP, FTP, and SFT streams.

See also

Python Requests Proxy Related API examples and articles