proxypy 2.0

Description:

proxypy 2.0

# proxy_pyproxy_py is a program which collects proxies, saves them in a databaseand makes periodically checks.It has a server for getting proxies with nice API(see below).## Where is the documentation?It's [here](http://proxy-py.readthedocs.io)## How to build?1 Clone this repository`git clone https://github.com/DevAlone/proxy_py.git`2 Install requirements```bashcd proxy_pypip3 install -r requirements.txt```3 Create settings file`cp config_examples/settings.py proxy_py/settings.py`4 Install postgresql and change database configuration in settings.py file5 (Optional) Configure alembic6 Run your application`python3 main.py`7 Enjoy!## I'm too lazy. Can I just use it?`TODO: update, old version!`Yes, you can download virtualbox image[here](https://drive.google.com/file/d/1oPf6xwOADRH95oZW0vkPr1Uu_iLDe9jc/view?usp=sharing).After downloading check that port forwarding is still working,you need forwarding of 55555 host port to 55555 guest.## How to get proxies?proxy_py has a server, based on aiohttp, which is listening 127.0.0.1:55555(you can change it in the settings file) and provides proxies.To get proxies you should send the following json requeston address `http://127.0.0.1:55555/api/v1/`(or other domain if behind reverse proxy):```json{ "model": "proxy", "method": "get", "order_by": "response_time, uptime"}```Note: order_by makes the result sortedby one or more fields(separated by comma).You can skip it. The required fields are `model` and `method`.It's gonna return you the json response like this:```json{ "count": 1, "data": [{ "address": "http://127.0.0.1:8080", "auth_data": "", "bad_proxy": false, "domain": "127.0.0.1", "last_check_time": 1509466165, "number_of_bad_checks": 0, "port": 8080, "protocol": "http", "response_time": 461691, "uptime": 1509460949 } ], "has_more": false, "status": "ok", "status_code": 200}```Note: All fields except *protocol*, *domain*, *port*, *auth_data*,*checking_period* and *address* CAN be nullOr error if something went wrong:```json{ "error_message": "You should specify \"model\"", "status": "error", "status_code": 400}```Note: status_code is also duplicated in HTTP status codeExample using curl:`curl -X POST http://127.0.0.1:55555/api/v1/ -H "Content-Type: application/json" --data '{"model": "proxy", "method": "get"}'`Example using httpie:`http POST http://127.0.0.1:55555/api/v1/ model=proxy method=get`Example using python's `requests` library:```pythonimport requestsimport jsondef get_proxies(): result = [] json_data = { "model": "proxy", "method": "get", } response = requests.post('http://127.0.0.1:55555/api/v1/', json=json_data) if response.status_code == 200: response = json.loads(response.text) for proxy in response['data']: result.append(proxy['address']) else: # check error here pass return result```Example using aiohttp library:```pythonimport aiohttpasync def get_proxies(): result = [] json_data = { "model": "proxy", "method": "get", } async with aiohttp.ClientSession() as session: async with session.post('http://127.0.0.1:55555/api/v1/', json=json_data) as response: if response.status == 200: response = json.loads(await response.text()) for proxy in response['data']: result.append(proxy['address']) else: # check error here pass return result```## How to interact with API?Read more about API [here](https://github.com/DevAlone/proxy_py/tree/master/docs/API.md)## How to contribute?`TODO: write guide about it`## How to test it?If you made the changes to code and want to check that you didn't breakanything, go [here](https://github.com/DevAlone/proxy_py/tree/master/docs/tests.md)## How to deploy on production using supervisor, nginx and postgresql in 8 steps?1 Install supervisor, nginx and postgresql`root@server:~aptinstallsupervisornginxpostgresql‘2Createvirtualenvironmentandinstallrequirementsonit3Copysettings.pyexample:‘proxypy@server: /proxypy cp config_examples/settings.py proxy_py/`4 create unprivileged user in postgresql databaseand change database authentication data in settings.py```bashproxy_py@server:~/proxy_pyYou can't use 'macro parameter character #' in math modeYou can't use 'macro parameter character #' in math mode cp /home/proxy_py/proxy_py/config_examples/proxy_py.supervisor.conf /etc/supervisor/conf.d/proxy_py.confroot@server:~vim/etc/supervisor/conf.d/proxypy.conf‘‘‘6Copynginxconfigexample,enableitandchangeifyouneed‘‘‘bashroot@server:  cp /home/proxy_py/proxy_py/config_examples/proxy_py.nginx.conf /etc/nginx/sites-available/proxy_pyroot@server:~ln−s/etc/nginx/sites−available/proxypy/etc/nginx/sites−enabled/root@server:  vim /etc/nginx/sites-available/proxy_py```7 Restart supervisor and Nginx```bashroot@server:~supervisorctlrereadroot@server:  supervisorctl updateroot@server:~/etc/init.d/nginxconfigtestroot@server:  /etc/init.d/nginx restart```8 Enjoy using it on your server!## What is it depend on?See `requirements.txt`