GitLocker: The Coding Marketplace

Description:

orwelltranslators 1.1.5

Orwell Package For Building Translators
About
Orwell is an open-source platform that allows you to integrate different metrics collectors into one large super-system, storing the data from different sources in Prometheus' format.
With that goal in mind, this package tries to facilitate as much as possible the task of creating translator services compatible with Orwell's architecture.
Development
If a given service's output follows the scheme "host,metric,value", its translator could be built with the following example:
from orwell import Metric, Runner

# The translator function should be able to receive multiple lines
# of the desired service output and return a list of Metric objects

def translate (lines: str) -> list[Metric]:
metrics = [ line.split(',') for line in lines.split('\n') ]
return [ Metric(metric, value, { 'host': host }) for host, metric, value in metrics ]

translator = Runner(translate)
translator.run()

Usage
With the example given above we now have three options for running the service:
Command-Line Mode
python main.py cmd <output>
The output will be processed and printed.
Server Mode
python main.py server
A Flask server will start accepting POST requests for /metrics.
The body of the request must be a dictionary with the only key metrics.
The response will be the processed output whenever the status code is 200.
Environment Variables
FLASK_HOST
Default "localhost"
FLASK_PORT
Default "5000""
FLASK_DEBUG
If set the server will start in debug mode.
Production Mode
python main.py prod
This is the most important mode as it allows to read messages from Kafka and write them to a Redis database, according to the Orwell's architecture.
Environment Variables
REDIS_HOST
Default "localhost"
REDIS_PASSWORD
Default "root"
KAFKA_HOST
Default "localhost""
KAFKA_PORT
Default 9093
KAFKA_TOPIC
Default "general"
Pull Mode
python main.py pull
This mode allows gathering data from a custom endpoint.
Environment Variables
PULL_ENDPOINT
PULL_INTERVAL
Default "localhost"
REDIS_PASSWORD
Default "root"