GitLocker: The Coding Marketplace

Description:

iosnooptool 0.1.0

iosnoop-tool
parse and visualize iosnoop output
What is iosnoop?
iosnoop is a tracing tool for disk I/O events and created by Brendan D. Gregg. Originally, it is for Solaris/MacOS X using DTrace.

iosnoop - A program to print disk I/O events as they happen, with useful details such as UID, PID, filename (if available), command, etc. Written using DTrace (Solaris 10 3/05, MacOS X 10.5).
http://www.brendangregg.com/DTrace/iosnoop

And then, iosnoop was ported for Linux using ftrace. Refer to his blog for iosnoop detail.

I'm probably dreaming. I just ported my popular iosnoop tool to Linux:
http://www.brendangregg.com/blog/2014-07-16/iosnoop-for-linux.html

Linux ported iosnoop is a simple bash script and can be downloaded from here.

https://github.com/brendangregg/perf-tools/blob/master/iosnoop

The output is like this. iosnoop requires root permission to run. iosnoop summarizes block device I/O (disk I/O) latency (LATms) with 1 line.
$ sudo path/to/iosnoop -ts
Tracing block I/O. Ctrl-C to end.
STARTs ENDs COMM PID TYPE DEV BLOCK BYTES LATms
14545946.332115 14545946.344366 <...> 17185 WS 8,160 4840587442 4096 12.25
14545946.331036 14545946.347082 main 17197 WS 8,160 4840666914 4096 16.05
14545946.332136 14545946.348852 <...> 17185 WS 8,160 4840643666 4096 16.72
14545946.331067 14545946.352752 main 17197 WS 8,160 4844855458 4096 21.69
14545946.332146 14545946.355893 <...> 17185 WS 8,160 4844863762 4096 23.75
14545946.356584 14545946.358920 <idle> 0 WS 8,160 4844859722 4096 2.34
14545946.356238 14545946.361128 <idle> 0 WS 8,160 4844855466 4096 4.89
14545946.356943 14545946.364070 <idle> 0 WS 8,160 4844863770 4096 7.13

Why iosnoop-tool?
iosnoop-tool is a visualizing tool for iosnoop output.
Brendan also has provided a visualization tool named trace2heatmap.pl, and introduced how to make Heatmap from iosnoop output.

Linux iosnoop Latency Heat Maps

I confirmed this tool works and know heatmap seems good enough. However, I am not good at reading/writing perl script, so I created my visualization tool in order to customize for my purpose.
Requirements

Python >= 3.4
Seaborn
Support output of iosnoop for Linux only

need -ts arguments

$ sudo iosnoop -ts

How to install
for developer
Create virtualenv to install iosnoop-tool.
$ mkdir virtualenvs
$ virtualenv -p python3 virtualenvs/venv
$ source virtualenvs/venv/bin/activate
(venv) $

Install iosnoop-tool package from the repository.
(venv) $ git clone https://github.com/t2y/iosnoop-tool.git
(venv) $ cd iosnoop-tool
(venv) $ python setup.py develop

Base CLI options
Confirm iosnoop-cli works as below.
(venv) $ iosnoop-cli --help
usage: iosnoop-cli [-h] [--basedate BASEDATE] --data DATA
[--columns COLUMNS [COLUMNS ...]]
[--io-commands IO_COMMANDS [IO_COMMANDS ...]]
[--io-device IO_DEVICE] [--io-pids IO_PIDS [IO_PIDS ...]]
[--io-types IO_TYPES [IO_TYPES ...]] [--since SINCE]
[--until UNTIL] [-v] [--version]
{csv,plot} ...

positional arguments:
{csv,plot}

optional arguments:
-h, --help show this help message and exit
--basedate BASEDATE set base datetime to convert kernel timestamp to
localtime, format: yyyymmddHHMISS
--data DATA set path to iosnoop output file
--columns COLUMNS [COLUMNS ...]
set columns name in iosnoop output
--io-commands IO_COMMANDS [IO_COMMANDS ...]
set commands in iosnoop output
--io-device IO_DEVICE
set device in iosnoop output
--io-pids IO_PIDS [IO_PIDS ...]
set process ids in iosnoop output
--io-types IO_TYPES [IO_TYPES ...]
set types in iosnoop output
--since SINCE set seconds since relative difference from start
--until UNTIL set seconds until relative difference from start
-v, --verbose set verbose mode
--version show program version

Sub Commands
csv
Create csv/tsv file from output of iosnoop.
(venv) $ iosnoop-cli csv --help
usage: iosnoop-cli csv [-h] [--dialect {excel,excel-tab,unix}]
[--output OUTPUT] [--separator SEPARATOR]

optional arguments:
-h, --help show this help message and exit
--dialect {excel,excel-tab,unix}
set dialect for csv writer, default is excel
--output OUTPUT set path to save csv file of iosnoop
--separator SEPARATOR
set separator (choose from comma, tab), default is comma

plot
Create image file rendered by seaborn from output of iosnoop.
(venv) $ iosnoop-cli plot --help
usage: iosnoop-cli plot [-h] [--backend BACKEND] [--colormap COLORMAP]
[--fig-output FIGOUTPUT] [--fig-size FIGSIZE]
[--hspace HSPACE] [--plot-type {heatmap}] [--square]
[--subplot-conditions SUBPLOT_CONDITIONS [SUBPLOT_CONDITIONS ...]]
[--x-interval X_INTERVAL] [--x-max X_MAX]
[--y-interval Y_INTERVAL] [--y-max Y_MAX]

optional arguments:
-h, --help show this help message and exit
--backend BACKEND set backend for matplotlib, use TkAgg to monitor in
the foreground
--colormap COLORMAP set color map for seaborn heatmap
--fig-output FIGOUTPUT
set path to save graph
--fig-size FIGSIZE set figure size
--hspace HSPACE set hspace for subplot
--plot-type {heatmap}
set plot type ("heatmap" by default)
--square set square mode for heatmap
--subplot-conditions SUBPLOT_CONDITIONS [SUBPLOT_CONDITIONS ...]
set DataFrame conditions to filter
--x-interval X_INTERVAL
set value of interval for x bins
--x-max X_MAX set maximum value for x-axis
--y-interval Y_INTERVAL
set value of interval for y bins
--y-max Y_MAX set maximum value for y-axis

How to use
This is sample heatmap rendered by seaborn.

(venv) $ iosnoop-cli --data tests/fixtures/iosnoop-sample.log plot --y-max 600 --y-interval 10 --fig-output tests/fixtures/iosnoop-sample.png
...
(venv) $ file tests/fixtures/iosnoop-sample.png
tests/fixtures/iosnoop-sample.png: PNG image data, 1024 x 800, 8-bit/color RGBA, non-interlaced

Specifying --basedate options converts kernel timestamp to localtime.
(vent) $ iosnoop-cli --basedate 20180702230100 --data tests/fixtures/iosnoop-sample.log plot --y-max 600 --y-interval 10 --fig-output tests/fixtures/iosnoop-sample-with-basedate.png

Examples
plot

show graphical window instead of saviing image file

(venv) $ iosnoop-cli --data tests/fixtures/iosnoop-sample.log plot --backend TkAgg

adjust x-axis and y-axis as needed, default settings might not appropriate depends on output of iosnoop

(venv) $ iosnoop-cli --data tests/fixtures/iosnoop-sample.log plot --x-max 100 --x-interval 2.0 --y-max 300 --y-interval 5

compare multiple subplots
Use header name to retrieve particular data in pandas.DataFrame.
(venv) $ head tests/fixtures/iosnoop-sample.log
STARTs ENDs COMM PID TYPE DEV BLOCK BYTES LATms
14545946.332115 14545946.344366 <...> 17185 WS 8,160 4840587442 4096 12.25
14545946.331036 14545946.347082 main 17197 WS 8,160 4840666914 4096 16.05
14545946.356943 14545946.364070 <idle> 0 WS 8,160 4844863770 4096 7.13

retrieved by exact match and contains string, --hspace is used for adjusting height space for each subplots

(venv) $ iosnoop-cli --data tests/fixtures/iosnoop-sample.log plot --y-max 300 --y-interval 30 --subplot-conditions "COMM == 'main'" "COMM.str.contains('jbd2|kblockd|flush')" "COMM == '<idle>'" --hspace 0.9 --fig-output tests/fixtures/iosnoop-sample-comm.png

retrieved by bytes between, --hspace is used for adjusting height space for each subplots

(venv) $ iosnoop-cli --data tests/fixtures/iosnoop-sample.log plot --subplot-conditions "BYTES.between(0, 131072)" "BYTES.between(131073, 262144)" "BYTES.between(262145, 524287)" "BYTES == 524288" --hspace 1.2 --fig-output tests/fixtures/iosnoop-sample-bytes.png

csv

write contents parsed from iosnoop output as csv file

(venv) $ iosnoop-cli --data tests/fixtures/iosnoop-sample.log csv --output iosnoop.csv
(venv) $ head iosnoop.csv
STARTs,ENDs,COMM,PID,TYPE,DEV,BLOCK,BYTES,LATms,STARTs_DIFF
14545946.332115,14545946.344366,<...>,17185,WS,"8,160",4840587442,4096,12.25,0
14545946.331036,14545946.347082,main,17197,WS,"8,160",4840666914,4096,16.05,-0.0010790005326271057
14545946.332136,14545946.348852,<...>,17185,WS,"8,160",4840643666,4096,16.72,2.099946141242981e-05