@ extra/ 代理路由器 
安装
yarn add @extra/proxy-router
# - or -
npm install @extra/proxy-router
剧作家
如果这是你第一次 剧作家外出 插件这里的所有您需要的 :
yarn add playwright playwright-extra @extra/proxy-router
# - or -
npm install playwright playwright-extra @extra/proxy-router
傀儡
如果这是你第一次 木偶外传 插件这里的所有您需要的 :
yarn add puppeteer puppeteer-extra @extra/proxy-router
# - or -
npm install puppeteer puppeteer-extra @extra/proxy-router
兼容性
| 💫 |  铬 |  颜色 |  火花 |  维基文库 |
|---|---|---|---|---|
| 剧作家外出 | ✅ | ✅ | ✅ | ✅ |
| 木偶外传 | ✅ | ✅ | 🕒 | - 说吧 |
| 无头 | 头 | 发射 | 连接 |
|---|---|---|---|
| ✅ | ✅ | ✅ | ✅ (当地) |
特征
插件使浏览器中使用代理更方便:
- 处理代理认证
- 可使用多个代理
- 使用主机/域的灵活代理路由
- 浏览器启动后动态更改代理
- 每个代理或主机收集流量统计
- 使用本地浏览器特性, 不丢失性能
使用量
用木偶吗? 要使用以下剧作家示例,只需更改您的 进口
简单
所有浏览器连接的单一代理
// playwright-extra is a drop-in replacement for playwright,
// it augments the installed playwright with plugin functionality
// Note: Instead of chromium you can use firefox and webkit as well.
const { chromium } = require('playwright-extra')
// Configure and add the proxy router plugin with a default proxy
const ProxyRouter = require('@extra/proxy-router')
chromium.use(
ProxyRouter({
proxies: { DEFAULT: 'http://user:pass@proxyhost:port' },
})
)
// That's it, the default proxy will be used and proxy authentication handled automatically
chromium.launch({ headless: false }).then(async (browser) => {
const page = await browser.newPage()
await page.goto('https://canhazip.com', { waitUntil: 'domcontentloaded' })
const ip = await page.evaluate('document.body.innerText')
console.log('Outbound IP:', ip)
await browser.close()
})
动态路由
灵活使用多个代理和路由连接
// playwright-extra is a drop-in replacement for playwright,
// it augments the installed playwright with plugin functionality
// Note: Instead of chromium you can use firefox and webkit as well.
const { chromium } = require('playwright-extra')
// Configure the proxy router plugin
const ProxyRouter = require('@extra/proxy-router')
const proxyRouter = ProxyRouter({
// define the available proxies (replace this with your proxies)
proxies: {
// the default browser proxy, can be `null` as well for direct connections
DEFAULT: 'http://user:pass@proxyhost:port',
// optionally define more proxies you can use in `routeByHost`
// you can use whatever names you'd like for them
DATACENTER: 'http://user:pass@proxyhost2:port',
RESIDENTIAL_US: 'http://user:pass@proxyhost3:port',
},
// optional function for flexible proxy routing
// if this is not specified the `DEFAULT` proxy will be used for all connections
routeByHost: async ({ host }) => {
if (['pagead2.googlesyndication.com', 'fonts.gstatic.com'].includes(host)) {
return 'ABORT' // block connection to certain hosts
}
if (host.includes('google')) {
return 'DIRECT' // use a direct connection for all google domains
}
if (host.endsWith('.tile.openstreetmap.org')) {
return 'DATACENTER' // route heavy images through datacenter proxy
}
if (host === 'canhazip.com') {
return 'RESIDENTIAL_US' // special proxy for this domain
}
// everything else will use `DEFAULT` proxy
},
})
// Add the plugin
chromium.use(proxyRouter)
// Launch a browser and run some IP checks
chromium.launch({ headless: true }).then(async (browser) => {
const page = await browser.newPage()
await page.goto('https://showmyip.com/', { waitUntil: 'domcontentloaded' })
const ip1 = await page.evaluate("document.querySelector('#ipv4').innerText")
console.log('Outbound IP #1:', ip1)
// => 77.191.128.0 (the DEFAULT proxy)
await page.goto('https://canhazip.com', { waitUntil: 'domcontentloaded' })
const ip2 = await page.evaluate('document.body.innerText')
console.log('Outbound IP #2:', ip2)
// => 104.179.129.27 (the RESIDENTIAL_US proxy)
console.log(proxyRouter.stats.connectionLog) // list of connections (host => proxy name)
// { id: 0, proxy: 'DIRECT', host: 'accounts.google.com' },
// { id: 1, proxy: 'DEFAULT', host: 'www.showmyip.com' },
// { id: 2, proxy: 'ABORT', host: 'pagead2.googlesyndication.com' },
// { id: 3, proxy: 'DEFAULT', host: 'unpkg.com' },
// ...
console.log(proxyRouter.stats.byProxy) // bytes used by proxy
// {
// DATACENTER: 441734,
// DEFAULT: 125823,
// DIRECT: 100457,
// RESIDENTIAL_US: 4764,
// ABORT: 0
// }
console.log(proxyRouter.stats.byHost) // bytes used by host
// {
// 'a.tile.openstreetmap.org': 150685,
// 'c.tile.openstreetmap.org': 147054,
// 'b.tile.openstreetmap.org': 143995,
// 'unpkg.com': 57621,
// 'www.googletagmanager.com': 49572,
// 'www.showmyip.com': 40408,
// ...
await browser.close()
})
进口
使用 Puppeteer 软件
该代码与上方剧作家例子基本相同. :-)
只需更改导入和包名 :
- const { chromium } = require('playwright-extra')
+ const puppeteer = require('puppeteer-extra')
// ...
- chromium.use(proxyRouter)
+ puppeteer.use(proxyRouter)
// ...
- chromium.launch()
+ puppeteer.launch()
// ...
字典和无害环境管理
该插件以Typescript和有型号的船来写.
剧作家 :
// You can use any browser: chromium, firefox, webkit
import { firefox } from 'playwright-extra'
import ProxyRouter from '@extra/proxy-router'
// ...
firefox.use(proxyRouter)
傀儡:
import puppeteer from 'puppeteer-extra'
import ProxyRouter from '@extra/proxy-router'
// ...
puppeteer.use(proxyRouter)
除错日志
如果您想看到调试输出 只运行您的脚本这样:
# macOS/Linux (Bash)
DEBUG=*proxy-router* node myscript.js
# Windows (Powershell)
$env:DEBUG='*proxy-router*';node myscript.js
如何运作
代理路由器会推出本地代理服务器并指示浏览器使用. 该本地代理服务器将依可选用户自定义的路由功能而与配置的上游代理服务器和中继连接相接,同时处理上游代理认证和其他几件事.
API 密码
选项
export interface ProxyRouterOpts {
/**
* A dictionary of proxies to be made available to the browser and router.
*
* An optional entry named `DEFAULT` will be used for all requests, unless overriden by `routeByHost`.
* If the `DEFAULT` entry is omitted no proxy will be used by default.
*
* The value of an entry can be a string (format: `http://user:pass@proxyhost:port`) or `null` (direct connection).
* Proxy authentication is handled automatically by the router.
*
* @example
* proxies: {
* DEFAULT: "http://user:pass@proxyhost:port", // use this proxy by default
* RESIDENTIAL_US: "http://user:pass@proxyhost2:port" // use this for specific hosts with `routeByHost`
* }
*/
proxies?: {
/**
* The default proxy for the browser (format: `http://user:pass@proxyhost:port`),
* if omitted or `null` no proxy will be used by default
*/
DEFAULT?: string | null
/**
* Any other custom proxy names which can be used for routing later
* (e.g. `'DATACENTER_US': 'http://user:pass@proxyhost:port'`)
*/
[key: string]: string | null
}
/**
* An optional function to allow proxy routing based on the target host of the request.
*
* A return value of nothing, `null` or `DEFAULT` will result in the DEFAULT proxy being used as configured.
* A return value of `DIRECT` will result in no proxy being used.
* A return value of `ABORT` will cancel/block this request.
*
* Any other string as return value is assumed to be a reference to the configured `proxies` dict.
*
* @note The browser will most often establish only a single proxy connection per host.
*
* @example
* routeByHost: async ({ host }) => {
* if (host.includes('google')) { return "DIRECT" }
* return 'RESIDENTIAL_US'
* }
*
*/
routeByHost?: RouteByHostFn
/** Collect traffic and connection stats, default: true */
collectStats?: boolean
/** Don't print any proxy connection errors to stderr, default: false */
muteProxyErrors?: boolean
/** Suppress proxy errors for specific hosts */
muteProxyErrorsForHost?: string[]
/** Options for the local proxy-chain server */
proxyServerOpts?: ProxyServerOpts
/**
* Optionally exempt hosts from going through a proxy, even our internal routing proxy.
*
* Examples:
* `.com` or `chromium.org` or `.domain.com`
*
* @see
* https://chromium.googlesource.com/chromium/src/+/HEAD/net/docs/proxy.md#proxy-bypass-rules
* https://www-archive.mozilla.org/quality/networking/docs/aboutno_proxy_for.html
*/
proxyBypassList?: string[]
}
替代品
代理. pace 文件 参考文献
- 仅在头部模式下支持铬
- 虽有名相
FindProxyForURL) 只能通过主机进行路由
- 虽有名相
- Firefox 支持 PAC 文件, 并且通过预览路径包含路径
- 仅在浏览器启动时装入一次, 无法进行动态代理
- 不处理认证
用于木偶机的各种“ 每页代理” 插件
- 优点: 路由代理按页面而非主机
- 他们依靠一个巨大的黑客:使用Node.js发送请求而不是浏览器
- 将更改 TLS 指纹, 容易出错
- CDP 请求截取仅为铬
- 增加延迟和资源间接费用
许可证
版权: 2018-2023, 贝斯特恩德根据麻省理工学院的许可证释放.