Mobile Proxy with Selenium and Playwright

Use a real browser only when the data does not exist without one. If the markup arrives in the initial response, an HTTP client is an order of magnitude cheaper — that path is covered in the Python guide. What follows is for the cases where JavaScript has to run, and where the proxy has to survive contact with a browser.

Playwright: Proxy at Launch, Proxy per Context

Playwright takes proxy credentials as first-class arguments and handles the authentication challenge itself. There is no extension to load and no header to inject:

from playwright.sync_api import sync_playwright

with sync_playwright() as p:
    browser = p.chromium.launch(proxy={
        "server": "http://proxy.pxm2.io:8000",
        "username": "user",
        "password": "pass",
    })
    ctx = browser.new_context(
        locale="en-GB",
        timezone_id="Europe/London",
        viewport={"width": 390, "height": 844},
        is_mobile=True,
    )
    page = ctx.new_page()
    page.goto("https://example.com/listings", wait_until="domcontentloaded")
    page.wait_for_selector(".result-card")
    html = page.content()
    browser.close()

Set the proxy per context rather than per browser when you are running several identities in one process — new_context accepts the same proxy dictionary, so each context gets its own exit IP, its own cookie jar and its own storage, with no cross-contamination between them. Tabs inside one context share a network stack, so if you need separate identities you need separate contexts, not separate tabs.

Notice the context options in that snippet doing a second job. A mobile viewport with is_mobile set, a locale and a timezone are not decoration — they are the browser half of a claim whose network half is the mobile exit IP. Getting one right and the other wrong is worse than getting neither right, because the mismatch is itself a signal.

Selenium and the Authenticated-Proxy Trap

Selenium is the older option and it has one sharp edge worth knowing before you commit: Chrome’s --proxy-server switch does not accept a username and password. Passing them in the URL is silently ignored, and the browser then sits on a native authentication dialog that WebDriver cannot dismiss. The symptom is a script that hangs on the first navigation with no error at all.

from selenium import webdriver

opts = webdriver.ChromeOptions()
opts.add_argument("--proxy-server=http://proxy.pxm2.io:8000")

# NOTE: user:pass in the URL above is IGNORED by Chrome.
# Pick one of:
#   1. IP authorisation  - allow-list your server IP, use no credentials
#   2. selenium-wire-lw  - injects the Proxy-Authorization header
#   3. a small MV3 extension that answers onAuthRequired

driver = webdriver.Chrome(options=opts)
driver.get("https://example.com/listings")

IP authorisation is the least fragile of the three whenever your scraper runs from a fixed host. No credentials travel, nothing has to be injected, and there is no extra dependency to keep alive.

If you are on Selenium mainly out of habit and you need authenticated proxies, Playwright removes the whole problem — and that is a more durable fix than any of the three workarounds.

Debugging 407s and Proxy Auth Failures

A 407 Proxy Authentication Required means the proxy received your request and refused it because it could not authenticate you. In browser automation it almost always means the credentials never arrived, rather than that they were wrong. Work through it in this order and you will usually find the cause in the first two steps:

1. Get the unauthenticated shape working first

   Allow-list your IP temporarily, or use a credential-free endpoint, and confirm the browser reaches the target at all. This separates a proxy problem from a browser-configuration problem, which are debugged very differently.

2. Run headed before headless

   A native authentication dialog is invisible in headless mode and obvious in a real window. If the headed run shows a credentials prompt, you have found it — that is Chrome telling you the switch never carried your username and password.

3. Verify the exit IP through the same browser

   Navigate to an IP-echo service and read the address. If it is your own, the proxy setting is being ignored entirely — a common outcome of a malformed endpoint string, since most stacks fail open rather than raising.

4. Then, and only then, add authentication

   With a known-good route to the target, an auth failure now has exactly one plausible cause instead of four.

One more thing worth checking before you assume the credentials are at fault: whether the host you are running from is on the provider’s allow-list at all. A server that moved, or a container that came up with a new address, produces exactly the same 407 as a mistyped password.

Headless Leakage: Making the Browser Match the IP

A clean mobile IP paired with a default headless browser is still an obvious bot. The proxy addresses one layer of a score built from several, and these are the layers it does not touch:

• Automation flags — navigator.webdriver, missing plugin and codec surfaces, and a WebGL renderer that reads as SwiftShader all mark an automated browser. Run headful under a virtual display, or use a stealth plugin, and verify against a fingerprint-testing page rather than assuming.
• Geo consistency — A UK exit IP paired with a browser reporting America/New_York and en-US is a mismatch checked routinely. Set locale and timezone_id in the context to match the country you are exiting from.
• Device coherence — If you set a phone viewport and is_mobile, the exit IP should be a mobile one, and the User-Agent should agree with both. Half a mobile identity is more conspicuous than none.
• Behaviour — Perfectly even intervals, no mouse movement and instantaneous form fills are all measured. No proxy hides an access pattern; only changing the pattern does.

Get a Dedicated Browser Proxy

Live PXM2 locations — pick the country your target should see the request coming from, and get a dedicated 4G/5G IP with unlimited bandwidth and rotations:

常见问题解答

相关移动代理指南

Browser automation is the expensive path. The rest of the cluster covers the cheaper ones and how to choose a proxy tier in the first place.

Web scraping guides

核心移动代理指南