pipeline-csv 1.10

Description:

pipelinecsv 1.10

PipelineCsv library




In Russian
The free, open source PipelineCsv library is designed to work with the results of analysis of in-line flaw detection data in the form of a CSV file.
The library provides a set of high-level operations with CSV file.
Data can be

mirrored
glued together from several CSV files
stretched/compressed along the distance according to a given set of intermediate points
interpreted as an iterable sequence of pipes with geodata

Installation
pip install pipeline-csv

Usage
It is necessary to define the sets of defects and markers used in your project.
To do this, you need to define your class for CSV row by deriving it from the pipeline_csv.csvfile.row.Row class and
override two methods of this class: defekts_dict and lineobj_dict.
from pipeline_csv.csvfile.row import Row

class TypeMarker:
VALVE = 0
CASE_START = 1
CASE_END = 2

class TypeDefekt:
CORROZ = 0
DENT = 1

class MyRow(Row):

@staticmethod
def defekts_dict():
return {
TypeDefekt.CORROZ: "Corrosion",
TypeDefekt.DENT: "Dent",
}

@staticmethod
def lineobj_dict():
return {
TypeMarker.VALVE: "Valve",
TypeMarker.CASE_START: "Casing start",
TypeMarker.CASE_END: "Casing end",
}

For the data mirroring operation, you need to override the markers_reverse method, which returns a dictionary that specifies the rules for replacing when mirroring.
class MyRow(Row):

@staticmethod
def markers_reverse():
return {
TypeMarker.CASE_START: TypeMarker.CASE_END,
TypeMarker.CASE_END: TypeMarker.CASE_START,
}

Further, the MyRow class can be used in operations with data of CSV files.
Creating a CSV file
Construct new csv file from scratch for pipeline 1000 mm diameter.
from pipeline_csv.csvfile import File

csv_file = File(1000)

Define tube at distance 1.0 m length = 11.0 m, thick = 10.5 mm with one seam with orientation 3 hour 00 minutes.
from pipeline_csv import TypeHorWeld
from pipeline_csv.orientation import Orientation

csv_file.data = [
MyRow.as_weld(1000),
MyRow.as_thick(1010, 105),
MyRow.as_seam(1020, TypeHorWeld.HORIZONTAL, Orientation(3, 0), None),
MyRow.as_weld(12000),
]

Add outside defect to tube at distance 5.0 m from left tube weld,
length = 20 mm, width = 10 mm, depth = 30% tube wall thickness,
orientation from 4 hours 00 minutes to 5 hours 00 minutes,
maximum depth point at distance 5.01 m from left tube weld, orientation 4 hours 30 minutes
with comment 'metal loss'.
from pipeline_csv import DefektSide

csv_file.data.append(MyRow.as_defekt(
6000,
TypeDefekt.CORROZ,
DefektSide.OUTSIDE
'20', '10', '30',
Orientation(4, 0), Orientation(5, 0),
6010, Orientation(4, 30),
'metal loss'
))

Save csv to file.
import os

csv_file.to_file('example.csv')
assert os.path.getsize('example.csv') > 0

Data mirroring
Create copy from saved file.
csv_copy = File.from_file('example.csv', 1000)

Check distance of the last object in copy and defect orientation.
assert csv_copy.total_length == 12000
assert len(csv_copy.data) == 5

defect_row = csv_copy.data[3]
assert defect_row.is_defect
assert defect_row.orient_td == '4,00'
assert defect_row.orient_bd == '5,00'

Reverse copy.
csv_copy.reverse()

Relative position of defekt must change and defect orientation must be mirrored.
defect_row = csv_copy.data[2]
assert defect_row.is_defect

assert defect_row.orient_td == '7,00'
assert defect_row.orient_bd == '8,00'

Save reversed copy to file.
csv_file.to_file('reversed.csv')
assert os.path.getsize('reversed.csv') > 0

Docking new data and compression/stretching
Append to initial CSV empty pipe with length 10.0 m and reversed copy from the file.
csv_file.join([10000, 'reversed.csv'])
assert csv_file.total_length == 28000
assert len(csv_file.data) == 11

Compress distances and length of all objects in half.
csv_file.dist_modify(
# table of corrections
# each node define as pair 'existing distance', 'new distance'
[[0, 0],
[28000, 14000],
])
assert csv_file.total_length == 14000

Save file with compress distances.
csv_file.to_file('transformed.csv')
assert os.path.getsize('transformed.csv') > 0

Pipe sequence
Iterate by pipes.
csv_trans = File.from_file('transformed.csv', 1000)
warnings = []
current_dist = 0
for i in csv_trans.get_tubes(warnings):
assert i.dist >= current_dist
current_dist = i.dist
tube = i

assert not warnings

Set geodata for tube
assert tube.latitude == ''
assert tube.longtitude == ''
assert tube.altitude == ''

tube.set_geo(10, 11, 12)

assert tube.latitude == 10
assert tube.longtitude == 11
assert tube.altitude == 12

csv_trans.to_file('geo.csv')
assert os.path.getsize('geo.csv') > 0

Load from saved file and check geodata from last pipe.
csv_geo = File.from_file('geo.csv', 1000)
last_tube = list(csv_geo.get_tubes(warnings))[-1]

assert last_tube.latitude == '10'
assert last_tube.longtitude == '11'
assert last_tube.altitude == '12'

Defect location at the pipe
Pipeline diameter 1000 mm.
csv = File(1000)

Define one pipe at distance 1.0 m, length = 11.0 m with one seam with orientation 3 hour 00 minutes and one defect at distance 5.0 m from left tube weld.
Defect length = 20 mm, width = 10 mm, depth = 30% tube wall thickness, orientation from 4 hours 00 minutes to 5 hours 00 minutes.
Maximum depth point at 10 mm from left border of defect, orientation 4 hours 30 minutes.
csv.data = [
Row.as_weld(1000),
Row.as_seam(1020, TypeHorWeld.HORIZONTAL, Orientation(3, 0), None),
Row.as_defekt(
6000,
TypeDefekt.CORROZ, DefektSide.INSIDE,
'20', '10', '30',
Orientation(4, 0), Orientation(5, 0),
Orientation(4, 30), 6010,
'corrozion'
),
Row.as_weld(12000),
]

pipes = list(csv.get_tubes())
assert len(pipes) == 1
pipe = pipes[0]
assert pipe.diam == 1000

One defect at the pipe.
assert len(pipe.defects) == 1
defect = pipe.defects[0]

Defect is metal loss, not dent.
Defect is not located at the weld/seam.
assert defect.is_metal_loss
assert not defect.is_dent
assert not defect.is_at_weld
assert not defect.is_at_seam

Defect as point orientation is maximum depth point orientation at 4:30.
assert defect.orientation_point.as_minutes == 270

Distance (mm) from maximum depth point to upstream weld.
assert defect.mp_left_weld == 5010

Distance (mm) from maximum depth point to downstream weld.
assert defect.mp_right_weld == 5990

Distance (mm) from maximum depth point to seam.
assert defect.mp_seam == 392

Distance (mm) from maximum depth point to nearest seam/weld.
assert defect.mp_seam_weld == 392

Distance (mm) from left defect border to upstream weld.
assert defect.to_left_weld == 5000

Distance (mm) from right defect border to downstream weld.
assert defect.to_right_weld == 5980

Distance (mm) from defect borders to seam.
assert defect.to_seam == 60

Distance (mm) from defect borders to nearest seam/weld.
assert defect.to_seam_weld == 60

Development
git clone [email protected]:vb64/pipeline.csv.git
cd pipeline.csv
make setup PYTHON_BIN=/path/to/python3
make tests