GitLocker: The Coding Marketplace

Description:

prombzex 0.3

Prometheus Bugzilla Exporter
Introduction
The purpose of this project is to export basic data about Bugzilla search results, in a format that Prometheus can consume. Specifically, as text files that can be ingested by node-exporter's textfile collector.
Requirements

You should have a directory somewhere that is being monitored by node-exporter's textfile collector
Python 3 (tested against 3.7)
The simple_rest_client Python module
Bugzilla REST API (available from v5 of Bugzilla)

Installation
You have three main options here:
From Source
pip install /path/to/this/repo
From pip
pip install prombzex
Via Docker
docker pull cmsj/prometheus-bugzilla-exporter
Note that if you use the Docker image, there are two expected volumes to mount:

/config.json - The configuration file for prombzex (see Configuration below)
/outdir - A directory that the textfile outputs will be written to

Configuration
Prombzex is configured via a JSON file, the path/name of which should be supplied as the only command line argument when running prombzex.
The file should look like this:
{
"default": {
"timeout": 60,
"output_dir": "/outdir"
},
"https://bugzilla.someproject.org": {
"output_file": "someproject.prom",
"name": "Some Project",
"api_key": "12345678abcdef",
"queries": [
{
"name": "my_open_bugs",
"help": "Count of bugs I have open right now",
"type": "gauge",
"query": "https://bugzilla.someproject.org/buglist.cgi?bug_status=__open__&email1=MYEMAIL%40SOMEPROJECT.ORG&emailassigned_to1=1&emailtype1=substring&list_id=9810932&query_format=advanced"
},
{
"name": "my_closed_all_time",
"help": "Count of bugs I've ever closed",
"type": "counter",
"query": "https://bugzilla.someproject.org/buglist.cgi?bug_status=__closed__&email1=MYEMAIL%40SOMRPROJECT.ORG&emailassigned_to1=1&emailtype1=substring&list_id=9810932&query_format=advanced"
}
]
}
"https://bugzilla.othergroup.net": {
"output_file": "othergroup.prom",
"name": "Other Group",
"api_key": "abcdef1234568",
"queries": [
{
"name": "some_query_name",
"help": "Some interesting query",
"type": "gauge",
"group_field": "status",
"params": [
{ "key": "status", "value": "__open__" },
{ "key": "f1", "value": "assigned_to" },
{ "key": "o1", "value": "substring" },
{ "key": "v1", "value": "helpfulcorp.com" },
{ "key": "list_id", "value": "12345678" },
{ "key": "query_format", "value": "advanced" },
{ "key": "include_fields", "value": ["id", "status"] }
]
}
]
}
}

Obviously, some explanation is required here!
The structure is that there should be dictionaries for each Bugzilla server you want to talk to and optionally a default dictionary to provide default values that may be overidden in some of the server configs.
The keys available in each server are:
Note that the URL for the server is the key in the top-level dictionary. Hostnames specified in a query field are ignored, only the query parameters are parsed.

name: A friendly name for the Bugzilla server
api_key: The API key you obtain from your Bugzilla preferences
queries: An array of dictionaries, each of which defines a search query to run against the server (see below)
timeout: A number of seconds to wait before giving up on search queries. The underlying default in simple_rest_client is a few seconds and is likely to be much too short for even moderately complex Bugzilla queries. Strongly consider setting a high timeout in the default section of your config.json.
output_dir: The directory where output files should be written - in the case of Docker, to match the Dockerfile, you should set this in default to /outdir.
output_file: The filename to create in output_dir with the Prometheus values. There is no particular need for this to be specified per-server, but you may prefer that layout. Note that for the textfile collector to notice the file, it must end with .prem.

The keys available on queries are:

name: A Prometheus-compatible name to uniquely identify the query
help: A human-compatible description of the data the query represents
type: At the moment, only gauge and counter are supported - these are hints to Prometheus for whether the value can go up and down, or only up (respectively).
group_field: (optional) If present, this field will group the results by a given field in the output (it by a particular column in the Bugzilla results). Useful if you want to do things like separate graph lines for different bug states with group_field set to status (assuming the Status column is included in your result fields)
query: A complete Bugzilla URL, which you should generate using the normal Bugzilla advanced search interface.
params: A representation of all of the query parameters from a complete Bugzilla URL. This is easier to read than a complete Bugzilla URL, but harder to create and doesn't offer any strong benefits unless you will be editing the query a lot.