Selenium Proxy Configuration in Python: Practical Guide

TL;DR

• A Selenium proxy is configured before WebDriver starts; changing options after Chrome launches does not reroute the existing browser.
• Selenium 4 can pass an IP-allowlisted HTTP proxy through ChromeOptions and the W3C Proxy capability.
• Username-and-password proxy authentication is not a portable WebDriver capability. Prefer IP allowlisting or a local forwarder that authenticates upstream.
• Rotate proxies between browser sessions so cookies, IP address, and browser state stay coherent.
• Verify both the exit IP and expected page content; a loaded tab alone does not prove that the proxy worked.

What a Selenium Proxy Changes

A Selenium proxy routes browser traffic through an intermediary before it reaches the destination. Selenium controls Chrome or Firefox through WebDriver, while the browser opens the proxied connections. This distinction matters: a Requests proxy changes one HTTP client call, but a Selenium proxy also affects subresources, redirects, scripts, and browser navigation.

Use a proxy for authorized localization tests, regional QA, ad verification, or permitted public-data collection. Nstdata supplies several proxy types, so the session model should follow whether your test needs a stable identity or a fresh route. The rotating proxy guide explains that choice separately from browser automation.

Selenium 4 expects browser-specific options rather than the older Desired Capabilities pattern. The official Selenium browser options documentation shows a Python Proxy object assigned to options.proxy. The Python API documents manual fields such as http_proxy, ssl_proxy, and socks_proxy in selenium.webdriver.common.proxy.

Prerequisites

A working Selenium proxy setup needs Python, Selenium, a compatible browser, and an authorized test URL. Selenium Manager normally locates or obtains a matching driver when webdriver.Chrome() starts, but locked-down CI runners may need the browser and driver installed in advance.

python -m venv .venv
source .venv/bin/activate
python -m pip install selenium

Keep credentials outside source code:

export PROXY_HOST="gateway.example"
export PROXY_PORT="8000"
export PROXY_USER="channel-id"
export PROXY_PASSWORD="replace-with-secret"

Do not put a production password in a Git URL, screenshot, exception, or CI log. A .env file is only safer if it is excluded from version control and protected like any other secret.

Detailed Tutorial

Method 1: Configure an IP-allowlisted HTTP proxy

An IP-allowlisted proxy is the simplest Selenium configuration because the browser does not need to answer an authentication challenge. Add the proxy to the browser options before creating the driver.

Step 1: Build the proxy capability

import os
from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType

proxy_address = f"{os.environ['PROXY_HOST']}:{os.environ['PROXY_PORT']}"
options = webdriver.ChromeOptions()
options.add_argument("--headless=new")
options.add_argument("--window-size=1280,900")
options.proxy = Proxy({
    "proxyType": ProxyType.MANUAL,
    "httpProxy": proxy_address,
    "sslProxy": proxy_address,
})

driver = webdriver.Chrome(options=options)
try:
    driver.set_page_load_timeout(30)
    driver.get("https://example.com/")
    assert "Example Domain" in driver.title
    print(driver.title)
finally:
    driver.quit()

httpProxy governs HTTP destinations and sslProxy governs HTTPS destinations. Supplying both avoids the common mistake of testing HTTP successfully and then sending HTTPS traffic outside the intended path. Do not set accept_insecure_certs merely to silence proxy errors; a trusted proxy should preserve or correctly terminate TLS according to your approved network design.

Step 2: Verify the route

Use an IP-check endpoint that you are allowed to call, then validate its returned schema. A page-title assertion checks the destination, while an exit-IP assertion proves routing.

import json

driver.get("https://api.ipify.org?format=json")
payload = json.loads(driver.find_element("tag name", "body").text)
assert payload.get("ip"), payload
print({"proxy_exit_ip": payload["ip"]})

Run the check once per new session rather than before every page. For browser behavior beyond navigation, compare Puppeteer and Selenium before standardizing the automation stack.

Method 2: Handle authenticated proxies safely

Authenticated Selenium proxies work most reliably when authentication is removed from the browser-facing hop. WebDriver's standard proxy capability describes an endpoint, but it does not define a universal username/password field. Chrome command-line proxy configuration should not be treated as a safe credential store.

Use one of three patterns. Provider IP allowlisting authorizes the CI runner and keeps Method 1 unchanged. An approved local forward proxy can bind to 127.0.0.1, read secrets from its environment, and authenticate to the upstream service. A managed remote-browser grid can store the credentials and expose only its documented capability to the test.

For a local forwarder, Selenium remains credential-free:

from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType

options = webdriver.ChromeOptions()
options.add_argument("--headless=new")
options.proxy = Proxy({
    "proxyType": ProxyType.MANUAL,
    "httpProxy": "127.0.0.1:8899",
    "sslProxy": "127.0.0.1:8899",
})
driver = webdriver.Chrome(options=options)
driver.get("https://example.com/")
print(driver.title)
driver.quit()

Configure the forwarder separately with PROXY_USER and PROXY_PASSWORD, restrict local access, and redact logs. Avoid copy-pasted Chrome extensions that embed credentials in generated JavaScript; they enlarge the secret surface and depend on extension behavior outside Selenium's stable API.

How to Rotate Proxies Without Breaking Sessions

Selenium proxy rotation should align one proxy identity with one browser session. Closing the driver, choosing the next proxy, and starting a new driver prevents stored browser state from being paired with an unrelated IP.

from itertools import cycle
from selenium import webdriver

PROXIES = ["127.0.0.1:9001", "127.0.0.1:9002"]

def open_with_proxy(proxy_address: str):
    options = webdriver.ChromeOptions()
    options.add_argument("--headless=new")
    options.add_argument(f"--proxy-server=http://{proxy_address}")
    return webdriver.Chrome(options=options)

for proxy_address, url in zip(cycle(PROXIES), [
    "https://example.com/", "https://example.org/"
]):
    driver = open_with_proxy(proxy_address)
    try:
        driver.set_page_load_timeout(30)
        driver.get(url)
        print(proxy_address, driver.title)
    finally:
        driver.quit()

The URL list is deliberately bounded. Track each proxy's cooldown, consecutive transport failures, and last accepted result. A retry should not switch IPs for application errors such as 404; rotation helps with dead gateways and network failures, not every unexpected page.

Nstdata Residential Prime Proxies are a practical fit when authorized browser tests need geo-targeting plus rotating or sticky sessions. The product addresses the gap between a single fixed gateway and workflows needing repeatable regional identities. Current product materials describe HTTP, HTTPS, and SOCKS5 support with rotating and sticky session control, while exact gateways and credentials remain dashboard-specific. Choose a sticky session for a multi-page journey and rotation for independent jobs; neither guarantees access to a site that rejects automation.

• Session control: Match one session identifier to one Selenium run so navigation, cookies, and exit identity stay consistent.
• Protocol fit: Use HTTP/HTTPS for Chrome proxy configuration; confirm SOCKS behavior separately when DNS routing matters.
• Operational validation: Generate proxy credentials in a Channel, store them as secrets, and verify the observed exit before processing results.

The residential versus datacenter proxy comparison helps when cost, speed, and network origin change the decision. For Chrome-specific risks, read the undetected ChromeDriver proxy guide; a modified driver never creates permission to bypass site controls.

Troubleshooting Selenium Proxy Failures

Selenium proxy failures become easier to diagnose when you separate browser startup, proxy transport, authentication, TLS, and page acceptance.

Add explicit waits for application elements, but do not use longer sleeps to mask a broken proxy. Selenium's waits documentation explains why readiness conditions are preferable to arbitrary delays. Log the selected proxy ID, elapsed navigation time, exception class, and semantic result—never the password.

Conclusion

A reliable Selenium proxy configuration is created before WebDriver starts, verified through an observable exit, and kept stable for the browser's logical session. Begin with IP allowlisting or a local authenticated forwarder, rotate only between sessions, and fail closed when the page is a challenge or unexpected response. If the workflow expands beyond Selenium, compare the same session rules with Nstdata's proxy documentation and your authorization policy.

FAQ

Q: How do I set a proxy in Selenium Python?

Assign a manual Proxy object to ChromeOptions.proxy before calling webdriver.Chrome(options=options). Set both httpProxy and sslProxy when the browser visits HTTP and HTTPS destinations.

Q: Can Selenium authenticate with a username and password proxy?

Selenium has no portable WebDriver capability for proxy username and password. Use provider IP allowlisting, an approved loopback forwarder, or documented remote-grid secret handling instead of embedding credentials in arguments.

Q: Can I change the Selenium proxy without restarting Chrome?

You should start a new WebDriver session to change the Selenium proxy reliably. Restarting also prevents cookies and local storage from being paired with an unrelated IP identity.

Q: Why does HTTP work while HTTPS fails?

HTTPS often fails because sslProxy is missing or the proxy cannot establish a CONNECT tunnel. Check both proxy fields, authentication, firewall policy, and certificates before changing waits.

Q: Should I use a free Selenium proxy list?

Do not use an untrusted free proxy for sensitive or production traffic. Operators can observe unencrypted traffic, modify responses, or disappear without notice; the free-proxy safety guide explains the risk.

Q: Is using Selenium with proxies legal?

Using Selenium with proxies is legal in many legitimate testing contexts, but permission, terms, data rights, and local law still govern the activity. Use public or authorized pages, minimize collection, respect rate limits, and stop at access-control challenges.