os-scrapy-httpproxy-middleware 0.0.2
Last updated:
0 purchases
Description:
osscrapyhttpproxymiddleware 0.0.2
os-scrapy-httpproxy-middleware
This project provide middleware to enhance Scrapy built-in HttpProxyMiddleware. You can set proxy without sheme as the following format:
[user:password@]proxy:port
Install
pip install os-scrapy-httpproxy-middleware
You can run example spider directly in the project root path
scrapy crawl example
Usage
Settings
enable middleware, it is better disable Scrapy built-in HttpProxyMiddleware
DOWNLOADER_MIDDLEWARES = {
"scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware": None,
"os_scrapy_httpproxy_middleware.middlewares.HttpProxyMiddleware": 750,
}
Unit Tests
sh scripts/test.sh
License
MIT licensed.
License:
For personal and professional use. You cannot resell or redistribute these repositories in their original state.
Customer Reviews
There are no reviews.
