powerflex-logging-utilities 1.3.1

Description:

powerflexloggingutilities 1.3.1

powerflex-logging-utilities


Helpful code for logging in Python by PowerFlex.



Module
Description




forbid_toplevel_logging
Disable logging with the top-level root logging functions such as logging.info.


log_slow_callbacks
Either warn or info log when an async callback runs for too long.


init_loggers
A function for easily setting up logging to a file and to stdout.






Class
Description




JsonFormatter
A JSON log formatter to enable structured logging. It depends on the python-json-logger package.


TraceLogger
A Python Logger subclass that adds a TRACE logging level


AsyncNatsLogLevelListener
A NATS interface for changing the program's log level by sending a NATS request



Installation
You can install from PyPi directly:
pip install powerflex-logging-utilities

Sample usage
Initializing Loggers
Setup all Loggers to output JSON to stdout and to a file:
import logging
import sys

from powerflex_logging_utilities import (
JsonFormatter,
init_loggers,
TraceLogger,
)

LOG_LEVEL = "DEBUG"
FILE_LOG_LEVEL = "TRACE"
LOG_FILE = "./logs/trace.log"

MAX_LOG_FILE_MB = 10
MAX_TOTAL_LOG_FILE_MB = 10000

root_logger = logging.getLogger()

# Log warnings with the py.warnings logger
logging.captureWarnings(True)

# Fix iPython autocomplete
logging.getLogger("parso").propagate=False

init_loggers.init_loggers(
[root_logger],
log_level=LOG_LEVEL,
file_log_level=FILE_LOG_LEVEL,
filename=LOG_FILE,
max_bytes=1000 * 1000 * MAX_LOG_FILE_MB,
backup_count=MAX_TOTAL_LOG_FILE_MB // MAX_LOG_FILE_MB,
stream=sys.stdout,
formatter=JsonFormatter,
info_logger=root_logger,
)

# Either use logging.getLogger or don't initialize a logger until your root logger is configured.
logging.setLoggerClass(TraceLogger)
logger = logging.getLogger(__name__)

This uses Python's logger propagation feature.
We only need to configure the root Logger in order to make sure all other Loggers output in the desired format.
You can pass formatter_kwargs to enable logging with a different JSON serializer.
To use:
logger = logging.getLogger(__name__)
logger.info("hello world")

Explicitly listing loggers
You can also list the loggers you'd like to configure instead of configuring
the root logger.
This could be useful if you configure your package's main logger
logging.getLogger("package"). You can then use Python's logger propagation by calling
logging.getLogger("package.submodule.a.b.c") to get Logger instances for all
other submodules.
import logging

from powerflex_logging_utilities import (
JsonFormatter,
init_loggers,
)

logger = logging.getLogger("your_package_name")

# Log warnings with the py.warnings logger
logging.captureWarnings(True)

init_loggers.init_loggers(
[logger, "asyncio", "py.warnings"],
log_level="DEBUG",
file_log_level="TRACE",
filename="./logs/trace-no-root.log",
formatter=JsonFormatter,
info_logger=logger,
)

NOTICE: if you use this method, any loggers you do not explicitly list will have non-JSON output.
Using several other utilities
import logging
from powerflex_logging_utilities import (
forbid_toplevel_logging,
log_slow_callbacks,
)

logger = logging.getLogger(__name__)

# Log slow async callbacks with two log levels
log_slow_callbacks.log_slow_callbacks(logger)

# Forbid functions such as logging.info since they implicitly use the root logger
forbid_toplevel_logging.forbid_logging_with_logging_toplevel()

Using the JSON formatter
import logging
import sys
from powerflex_logging_utilities import JsonFormatter

log_handler = logging.StreamHandler(stream=sys.stdout)
log_handler.setLevel("DEBUG")
log_handler.setFormatter(JsonFormatter())
logger = logging.getLogger(__name__)
logger.addHandler(log_handler)
logger.setLevel("DEBUG")

logger.info("hello world", extra={
"data": ["log structured data", ":D"],
1: "handles non string key",
})

{
"message": "hello world",
"name": "__main__",
"module": "<ipython-input-10-b016ce80d46f>",
"lineno": 1,
"funcName": "<cell line: 1>",
"filename": "<ipython-input-10-b016ce80d46f>",
"asctime": "2022-05-12 01:04:16,824",
"data": [
"log structured data",
":D"
],
"severity": "INFO",
"1": "handles non string key"
}

Using pipenv

Run make setup-with-pipenv to install all dependencies.
Make sure you have the version of Python specified in .tool-versions or simply change this file to your Python version (must be 3.8+).
Run pipenv shell or run the following make commands with pipenv run make ....
You could also alias pmake to pipenv run make for convenience.

Tests
There is 100% code coverage.
make test-unit

To test in several versions of Python, run:
tox

To download several versions of Python, use pyenv or asdf
To use pyenv, install it here and run the following script:
./install_python_versions_pyenv.sh

To use asdf, install the core parts here and run the following commands:
./install_python_versions_asdf.sh

Testing the code in this README
make test-readme

Checking code quality
The Github Actions will run all of the following checks on the code.
Code formatting
make format-fix

Linting
make lint

Type checking
make type-check-strict

Releasing to PyPi.org

Make sure all code checks have passed with make commitready.
Make sure you commit all code you wish to release with git commit.
Set the version in ./src/powerflex_monitoring/VERSION
Please attempt to follow semantic versioning.
Run make bump-version to commit the change to the VERSION file.
Run make release to upload the package to pypi.org and to push a new git tag

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter