GitLocker: The Coding Marketplace

Description:

logsanalyzer 0.5.2

Logs-analyzer
Logs-analyzer is a Python based library containing functions that can help you extract usable data from logs.
Status
Master:

Develop:

Quickstart
Support
Python 2: 2.7
Python 3: 3.6, 3.7
Install
using pip : pip install logs-analyzer
Code sample
from logs_analyzer.lib import LogsAnalyzer

nginx_logsanalyzer = LogsAnalyzer('nginx')
nginx_logsanalyzer.add_date_filter(minute='*', hour=22, day=4, month=5)
nginx_logsanalyzer.add_filter('192.10.1.1')
requests = nginx_logsanalyzer.get_requests()

Non-object functions
Function get_service_settings
Get default settings for the said service from the settings file, three type
of logs are supported right now: nginx, apache2 and auth.
Parameters
service_name: service name (e.g. nginx, apache2...).
Return
Returns a dictionary containing the chosen service settings or None if the
service doesn't exists.
Sample
nginx_settings = get_service_settings('nginx')
Function get_date_filter
Get the date's pattern that can be used to filter data from
logs based on the parameters.
Parameters
settings: the target logs settings.
minute: default now, minutes or * to ignore.
hour: default now, hours or * to ignore.
day: default now, day of month.
month: default now, month number.
year: default now, year.
Sample
nginx_settings = get_service_settings('nginx')
date_pattern = get_date_filter(nginx_settings, 13, 13, 16, 1, 1989)
print(date_pattern)

Prints [16/Jan/1989:13:13
Function filter_data
Filter received data/file content and return the results.
Parameters
log_filter: string that will be used to filter data
data: data to be filtered (String) or None if the data will
be loaded from a file.
filepath: filepath from where data will be loaded or None if
the data has been passed as a parameter.
is_casesensitive: if the filter has to be case sensitive
(default True).
is_regex: if the filter string is a regular expression
(default False).
is_reverse: (boolean) invert selection.
Return
Returns filtered data (String).
Sample
nginx_settings = get_service_settings('nginx')
date_filter = get_date_filter(nginx_settings, '*', '*', 27, 4, 2016)
base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
file_name = os.path.join(base_dir, 'logs-samples/nginx1.sample')
data = filter_data('192.168.5', filepath=file_name)
data = filter_data(date_filter, data=data)

Function get_web_requests
Analyze the web logs (Nginx & Apache2 for now) data and return list of requests
formatted as the model (pattern) defined.
Parameters
data: (String) data to analyzed.
pattern: (Regular expression) used to extract requests.
date_pattern: (Regular expression or None) used to extract date elements
to have ISO formatted dates.
date_keys: (List or None) list of extracted date elements placements.
Sample
apache2_settings = get_service_settings('apache2')
base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
file_name = os.path.join(base_dir, 'logs-samples/apache1.sample')
data = filter_data('127.0.1.1', filepath=file_name)
requests = get_web_requests(data, apache2_settings['request_model'],
nginx_settings['date_pattern'], nginx_settings['date_keys'])

Function get_auth_requests
Analyze the Auth logs data and return list of requests
formatted as the model (pattern) defined.
Parameters
data: (String) data to analyzed.
pattern: (Regular expression) used to extract requests.
date_pattern: (Regular expression or None) used to extract date elements
to have ISO formatted dates.
date_keys: (List or None) list of extracted date elements placements.
Sample
auth_settings = get_service_settings('auth')
base_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
date_filter = get_date_filter(auth_settings, '*', 22, 4, 5)
file_name = os.path.join(base_dir, 'logs-samples/auth.sample')
data = filter_data('120.25.229.167', filepath=file_name)
data = filter_data(date_filter, data=data)
requests = get_auth_requests(data, auth_settings['request_model'],
auth_settings['date_pattern'], auth_settings['date_keys'])