Node.js Playwright Tunnel Proxy: Browser Automation with Crawler Proxy

Playwright + Crawler Proxy

Playwright is a browser automation framework by Microsoft supporting Chromium, Firefox, and WebKit. Its built-in proxy parameter integrates directly with 16Yun's Crawler Proxy.

Setup

export PROXY_HOST=t.16yun.cn
export PROXY_PORT=31111
export PROXY_USERNAME=your-username
export PROXY_PASSWORD=your-password

Basic Usage

import { chromium } from 'playwright';

const host = process.env.PROXY_HOST || 't.16yun.cn';
const port = process.env.PROXY_PORT || '31111';
const user = process.env.PROXY_USERNAME || 'user';
const pass = process.env.PROXY_PASSWORD || 'password';

async function main() {
  const browser = await chromium.launch({
    headless: true,
    proxy: { server: `http://${host}:${port}`, username: user, password: pass },
  });

  const page = await browser.newPage({ ignoreHTTPSErrors: true });

  page.on('request', (req) => console.log('>>', req.method(), req.url()));
  page.on('response', (res) => console.log('<<', res.status(), res.url()));

  await page.goto('https://httpbin.org/ip', { waitUntil: 'load', timeout: 60000 });

  const text = await page.innerText('body');
  console.log(text);

  await browser.close();
}
main().catch((err) => { console.error(err); process.exit(1); });

Playwright's proxy parameter handles 407 authentication challenges automatically.

Request/Response Monitoring

page.on('request', (req) => console.log(`[REQ] ${req.method()} ${req.url()}`));
page.on('response', (res) => console.log(`[RES] ${res.status()} ${res.url()}`));
page.on('requestfailed', (req) => console.log(`[FAIL] ${req.url()} - ${req.failure().errorText}`));

Page Content Extraction

const text = await page.innerText('body');
const html = await page.content();
await page.screenshot({ path: 'page.png', fullPage: true });

Scenarios in Playwright

Playwright excels at browser rendering and screenshots. For pure HTTP-level IP control, pair Playwright with axios or httpx.

Troubleshooting