Python urllib Tunnel Proxy: Standard Library Four Scenarios

Python standard library urllib implements four tunnel proxy scenarios with custom HTTPSConnection for HTTPS Proxy-Tunnel.

16Yun Engineering TeamMay 13, 20261 min read

urllib:无需三方依赖的方案

urllib 是 Python 标准库,无需安装任何第三方包即可使用代理。通过重载 HTTPSConnection.set_tunnel 方法,可以在 CONNECT 阶段注入 Proxy-Tunnel 头,实现 HTTPS Proxy-Tunnel。

场景 A:强制切换 IP

import urllib.request, os, ssl, time

host = os.getenv("PROXY_HOST", "t.16yun.cn")
port = os.getenv("PROXY_PORT", "31111")
user = os.getenv("PROXY_USERNAME", "user")
pwd = os.getenv("PROXY_PASSWORD", "password")

proxy_url = f"http://{user}:{pwd}@{host}:{port}"
ctx = ssl.create_default_context()

for i in range(1, 4):
    handler = urllib.request.ProxyHandler({"http": proxy_url, "https": proxy_url})
    https_handler = urllib.request.HTTPSHandler(context=ctx)
    opener = urllib.request.build_opener(handler, https_handler)

    req = urllib.request.Request("https://httpbin.org/ip",
        headers={"Connection": "close", "Proxy-Connection": "close"})

    with opener.open(req, timeout=15) as resp:
        print(f"请求 {i}: {resp.read().decode()}")
    time.sleep(0.5)

场景 B:Keep-Alive

handler = urllib.request.ProxyHandler({"http": proxy_url, "https": proxy_url})
opener = urllib.request.build_opener(handler, urllib.request.HTTPSHandler(context=ctx))

for i in range(1, 4):
    req = urllib.request.Request("https://httpbin.org/ip",
        headers={"Connection": "keep-alive", "Proxy-Connection": "keep-alive"})
    with opener.open(req, timeout=15) as resp:
        print(f"请求 {i}: {resp.read().decode()}")
    time.sleep(0.2)

场景 C-HTTPS:Proxy-Tunnel(自定义 HTTPSConnection)

urllib 需要重载 set_tunnel 方法才能为 HTTPS 目标注入 CONNECT 头:

import http.client, base64, json

class TunnelHTTPSConnection(http.client.HTTPSConnection):
    def set_tunnel(self, host, port=None, headers=None):
        if headers is None:
            headers = {}
        headers["Proxy-Tunnel"] = os.getenv("PROXY_TUNNEL", "urllib-demo")
        super().set_tunnel(host, port, headers)

proxy_url = f"http://{user}:{pwd}@{host}:{port}"
conn = TunnelHTTPSConnection(host, int(port), context=ctx)
auth = base64.b64encode(f"{user}:{pwd}".encode()).decode()
conn.set_tunnel("httpbin.org", 443, headers={"Proxy-Authorization": f"Basic {auth}"})
conn.request("GET", "/ip")
resp = conn.getresponse()
print(resp.read().decode())

四种场景对比

场景urllib 实现关键 API
A每次新建 openerProxyHandler + Connection: close
B复用 opener同一 build_opener 实例
C-HTTP请求头 Proxy-TunnelRequest headers
C-HTTPS自定义 set_tunnelHTTPSConnection.set_tunnel

对于更简洁的 HTTPS Proxy-Tunnel 方案,建议使用 httpx.Proxy(headers=...) 或自定义 requests.HTTPAdapter.proxy_headers()

Need an enterprise proxy plan?

We can tailor architecture to your target domains, concurrency, and reliability goals.