GitLocker: The Coding Marketplace

Description:

pyprojectops 0.8.1

Welcome to pyproject_ops Documentation

What is this project?
There are various folder structures for Python projects, each with its own merits. Drawing from my extensive Python career experience, I have developed a personal best practice. To streamline the creation of the codebase skeleton according to this practice, I have introduced a tool called cookiecutter-pyproject.
By adopting this folder structure setup, pyproject_ops is an automation tool, capable of handling common tasks throughout the Python project development lifecycle. These tasks include “creating virtual environments”, “installing dependencies”, “running tests”, “building documentation sites”, and more.
This tool is, in essence, my secret weapon for efficiently managing over 120 Python open source projects, all available on PyPI at https://pypi.org/user/machugwu/. Each of these projects is equipped with essential features such as continuous integration (CI), code coverage testing, matrix testing, and documentation websites.
A little history about this project:

I previously had an automation tool called pygitrepo that was tailored to my former development workflow. pygitrepo primarily relied on setup.py and requirements.txt files. Even today, pyproject_ops continues to support the old convention while also accommodating pyproject.toml and poetry. This versatility enables me to write less code when automating Python development workflows.

How to Use

Use as a CLI
$ pip install pyproject_ops
$ pyops --help
NAME
pyops - python project ops command line interface.

SYNOPSIS
pyops COMMAND | <flags>

DESCRIPTION
python project ops command line interface.

FLAGS
-v, --version=VERSION
Type: bool
Default: False

COMMANDS
COMMAND is one of the following:

build_doc
** 📔 Build documentation website locally

build_doc_only
📔 Build documentation website locally without checking doc dependencies

bump_version
🔼 Bump semantic version.

cov
** 🧪 Run code coverage test

cov_only
🧪 Run code coverage test without checking test dependencies

deploy_latest_doc
🚀 📔 Deploy Documentation Site To S3 as Latest Doc

deploy_versioned_doc
🚀 📔 Deploy Documentation Site To S3 as Versioned Doc

install
** 💾 Install main dependencies and Package itself

install_all
** 💾 💻 🧪 📔 🤖 Install All Dependencies

install_automation
💾 🤖 Install Dependencies for Automation Script

install_dev
💾 💻 Install Development Dependencies

install_doc
💾 📔 Install Document Dependencies

install_test
💾 🧪 Install Test Dependencies

int
** 🧪 Run integration test

int_only
🧪 Run integration test without checking test dependencies

poetry_export
Export requirements-*.txt from poetry.lock file

poetry_lock
** Resolve dependencies using poetry, update poetry.lock file

publish
📦 Publish package to PyPI

test
** 🧪 Run test

test_only
🧪 Run test without checking test dependencies

venv_create
** 🐍 Create Virtual Environment

venv_remove
** 🗑 🐍 Remove Virtual Environment

view_cov
👀 🧪 View coverage test output html file locally in web browser.

view_doc
** 👀 📔 View documentation website locally

view_latest_doc
👀 📔 View the latest documentation website on S3

Use as a Python library
>>> from pyproject_ops.api import PyProjectOps
>>> pyops = PyProjectOps.from_pyproject_toml("pyproject.toml")
>>> pyops.create_virtualenv()
>>> pyops.pip_install_all()
>>> pyops.poetry_lock()
>>> pyops.poetry_install_all()
>>> pyops.run_cov_test()
>>> pyops.view_cov()
>>> pyops.build_doc()
>>> pyops.view_cov()
>>> pyops.python_build()
>>> pyops.twine_upload()

Folder Structure
Below is the folder structured used in pyproject_ops. The first item is the relative path from the project root directory. The second item is the attribute name that you can use to access the path in pyproject_ops. The third item is the description of the path. You can find a concrete example at HERE

.venv: PyProjectOps.dir_venv, The virtualenv directory.
.venv/bin: PyProjectOps.dir_venv_bin, The bin folder in virtualenv.
.venv/bin/pip: PyProjectOps.path_venv_bin_pip, The pip command in virtualenv.
.venv/bin/pytest: PyProjectOps.path_venv_bin_pytest, The pytest command in virtualenv.
.venv/bin/python: PyProjectOps.path_sys_executable, The current Python interpreter path.
.venv/bin/python: PyProjectOps.path_venv_bin_python, The python executable in virtualenv.
.venv/bin/twine: PyProjectOps.path_bin_twine, The twine CLI command path.
build: PyProjectOps.dir_build, The build folder for Python or artifacts build.
build/glue: PyProjectOps.dir_build_glue, The AWS glue artifacts build folder.
build/glue/extra_py_files: PyProjectOps.dir_build_glue_extra_py_files, The AWS glue extra Python files build folder.
build/glue/extra_py_files.zip: PyProjectOps.path_build_glue_extra_py_files_zip, The AWS glue extra Python files zip file path.
build/lambda: PyProjectOps.dir_build_lambda, The AWS Lambda artifacts build folder.
build/lambda/layer.zip: PyProjectOps.path_build_lambda_layer_zip, The AWS Lambda layer zip file path.
build/lambda/python: PyProjectOps.dir_build_lambda_python, The AWS Lambda layer build folder. This folder contains the dependencies.
build/lambda/python/aws: PyProjectOps.path_build_lambda_bin_aws, This is the AWS CLI executable path in Lambda layer.
build/lambda/source.zip: PyProjectOps.path_build_lambda_source_zip, The AWS Lambda source code deployment package zip file path.
config: PyProjectOps.dir_config, The folder that stores the config files.
config/config.json: PyProjectOps.path_config_json, Path to the JSON file that stores the non-sensitive config.
dist: PyProjectOps.dir_dist, The dist folder for Python package distribution (.whl file).
docs: PyProjectOps.dir_sphinx_doc, Sphinx docs folder.
docs/build: PyProjectOps.dir_sphinx_doc_build, The temp Sphinx doc build folder.
docs/build/html: PyProjectOps.dir_sphinx_doc_build_html, The built Sphinx doc build HTML folder.
docs/build/html/index.html: PyProjectOps.path_sphinx_doc_build_index_html, The built Sphinx doc site entry HTML file path.
docs/source: PyProjectOps.dir_sphinx_doc_source, Sphinx docs source code folder.
docs/source/conf.py: PyProjectOps.dir_sphinx_doc_source_conf_py, Sphinx docs conf.py file path.
docs/source/pyproject_ops: PyProjectOps.dir_sphinx_doc_source_python_lib, The generated Python library API reference Sphinx docs folder.
htmlcov: PyProjectOps.dir_htmlcov, The code coverage test results HTML output folder.
htmlcov/index.html: PyProjectOps.path_htmlcov_index_html, The code coverage test results HTML file.
lambda_app: PyProjectOps.dir_lambda_app, The AWS Lambda app handler file and Lambda related code directory.
lambda_app/.chalice/config.json: PyProjectOps.path_chalice_config, The AWS Chalice framework’s config file path.
lambda_app/.chalice/deployed: PyProjectOps.dir_lambda_app_deployed, The generated deployed.json file for AWS Chalice framework’s.
lambda_app/app.py: PyProjectOps.path_lambda_app_py, The app.py file for AWS Chalice framework.
lambda_app/lambda_function.py: PyProjectOps.path_lambda_function_py, The lambda_function.py handler file for AWS Lambda, if you are not using
lambda_app/update_chalice_config.py: PyProjectOps.path_lambda_update_chalice_config_script, Example: ${dir_project_root}/lambda_app/update_chalice_config.py
lambda_app/vendor: PyProjectOps.dir_lambda_app_vendor, The vendor folder for AWS Chalice framework’s packaging.
lambda_app/vendor/pyproject_ops: PyProjectOps.dir_lambda_app_vendor_python_lib, The source python library folder in AWS Chalice framework’s vendor folder.
poetry-lock-hash.json: PyProjectOps.path_poetry_lock_hash_json, The poetry-lock-hash.json file path. It is the cache of the poetry.lock file hash.
poetry.lock: PyProjectOps.path_poetry_lock, The poetry.lock file path.
pyproject.toml: PyProjectOps.path_pyproject_toml, The pyproject.toml file path.
pyproject_ops: PyProjectOps.dir_python_lib, The current Python library directory.
pyproject_ops/_version.py: PyProjectOps.path_version_py, Path to the _version.py file where the package version is defined.
requirements-automation.txt: PyProjectOps.path_requirements_automation, The requirements-automation.txt file path.
requirements-dev.txt: PyProjectOps.path_requirements_dev, The requirements-dev.txt file path.
requirements-doc.txt: PyProjectOps.path_requirements_doc, The requirements-doc.txt file path.
requirements-test.txt: PyProjectOps.path_requirements_test, The requirements-test.txt file path.
requirements.txt: PyProjectOps.path_requirements, The requirements.txt file path.
tests: PyProjectOps.dir_tests, Unit test folder.
tests_int: PyProjectOps.dir_tests_int, Integration test folder.
tests_load: PyProjectOps.dir_tests_load, Load test folder.

Develop and Release Strategy
This project is a “meta” project for other projects, it is very hard to test. I keep using this project in many of my production projects, and continuously improving it. I will merge all the changes manually into this every three months.

Install
pyproject_ops is released on PyPI, so all you need is:
$ pip install pyproject_ops
To upgrade to latest version:
$ pip install --upgrade pyproject_ops