Description:

osscrapy 0.0.8

os-scrapy




Ozzy's Scrapy scaffold.
Requirements

Python 3.6+ (pypy3.6+)
Scrapy 2.0+

Install
pip install os-scrapy

Usage
Command line
The command is same as scrapy
os-scrapy -h

CRAWLER_CLASS
os-scrapy support config CRAWLER_CLASS replace the default scrapy.crawler.Crawler
It can be configured in settings.py file or command line
os-scrapy crawl -c <your_crawler_class> <spider_name>

Enhanced startproject
Thanks to os-scrapy-cookiecutter, -p option can be used to create project as well as python package
os-scrapy startproject -p <project_name> [project_dir]

Set TWISTED_REACTOR on command line
-r option can be used to set TWISTED_REACTOR
os-scrapy crawl -r asyncio <spider_name>

Unit Tests
tox

License
MIT licensed.

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter