Description:

scrapyproxyport 1.1.1

Proxy Port integration for Scrapy.
About Proxy Port
Prerequisites
To use this package you will need a free API key. Get your API key here.
Detailed instructions here.
Installation
$ pip install scrapyproxyport

Getting Started
You need to assign an API key.
This can be done either through an environment variable
$ export PROXY_PORT_API_KEY=<API_KEY>

or inside settings.py.
# inside <your_project>/settings.py

PROXY_PORT_API_KEY = '<API_KEY>'
DOWNLOADER_MIDDLEWARES = {
# Add middleware with order number right before CookiesMiddleware
'scrapyproxyport.middlewares.ProxyMiddleware': 898,
'scrapy.downloadermiddlewares.cookies.CookiesMiddleware': 899,
}
DOWNLOAD_TIMEOUT = 10
RETRY_TIMES = 20
DUPEFILTER_CLASS = 'scrapy.dupefilters.BaseDupeFilter'

Example Scrapy project
scrapy-proxyport-example

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter