Python mitmproxy Tunnel Proxy: Forward to Crawler Proxy via Upstream Mode

Architecture

Client → mitmproxy (localhost:38080) → 16Yun Crawler Proxy (t.16yun.cn:31111) → Target

Clients don't need proxy credentials — all proxy logic is centralized in the mitmproxy layer.

Start mitmproxy

mitmdump \
  --listen-host 127.0.0.1 \
  --listen-port 38080 \
  --mode upstream:http://t.16yun.cn:31111 \
  --upstream-auth username:password \
  -s addon.py

Addon Script

# addon.py
import os
from mitmproxy import http

SEED = os.getenv("PROXY_TUNNEL", "mitmproxy-demo-1")

class UpstreamProxy:
    def request(self, flow: http.HTTPFlow) -> None:
        flow.request.headers.setdefault("Proxy-Tunnel", SEED)

addons = [UpstreamProxy()]

Client Usage

Clients point to local mitmproxy only:

import requests
session = requests.Session()
session.proxies = {
    "http": "http://127.0.0.1:38080",
    "https": "http://127.0.0.1:38080",
}
resp = session.get("https://httpbin.org/ip", timeout=30)
print(resp.json()['origin'])

Scenario Reference

Troubleshooting