readdat 2.6.13

Description:

readdat 2.6.13

ReadDat
Read/write various data file formats in python.
1. Installation instructions
Download this repo from gitlab (download button)
Move to the location of file setup.py and install the package
# eventually :
# conda create -n <name_of_environment> python=3.9 --yes
# conda activate <name_of_environment>
pip install -e .

# test the package with
pytest --verbose .

2. Supported files formats
2.1 Waveforms
The waveforms are packed into an obspy.core.stream.Stream object.
This program includes a python layer added on top of obspy.read for file formats
that are not handled by Obspy or to account for some specific header conventions.
File Format
The field format is used to specify the file format,
use format="AUTO" for automatic format detection (based on extension name for now).
See the file formats supported below.
Header conventions
The header conventions are handled using the argument acquistion_system.
Use acquistion_system=None to stick to the standard obspy behavior.
Use acquistion_system="AUTO" to attempt automatic detection of the acquisition system.
warning :
reading a file with a given acquistion_system will modify some field values (like unit conversions).
Please keep this in mind when you export your file to a new one.
Calendar time managment
Obspy is designed for seismologists and work with universal times (UTC+0).
The default behavior of this program is to store calendar times as if they were expressed in UTC+0.
To convert times to UTC+0 from local times in a given timezone,
you can use the argument timezone of readdat.read.
Be careful when converting files from one format to another.
SEG2 concatenation
Obspy cannot export seg2 files, this program let you combine seismic traces
from several files and reexport them to new seg2 files.
Database
Readdat can store header and metadata in a local sqlite database.
You can then access the data using SQL requests.
This procedure avoids converting the experimental data, since the database loads the data directly from the original files.
The database can then be enriched with attributes, quality controls, processing results...
2.1.1 SEG2
(i) Standard SEG2 files
Seg2 files can be read using the obspy standard conventions using acquisition_system=None
from readdat import read
stream = read('filename.sg2', format="SEG2", acquisition_system=None)

Obspy cannot write standard seg2 files by default, this package uses pyseg2 to
export an obspy stream into seg2 file.
Warning : do not overwrite experimental data,
any remaining bug in the writing method might result in corrupted data files,
I decline any responsibility in case of overwritten data files.
from readdat.write import write
stream = write('filename_test.sg2', format="SEG2", acquisition_system=None)

To interpret calendar times in the headers as local french times, use
from readdat import read, print_stream
stream = read('filename.sg2', format="SEG2", timezone="Europe/Paris")
print_stream(stream) # Have a look at the starttime/endtime fields of the trace headers

(ii) SEG2MUSC
Seg2 Files from MUSC are expressed in milliseconds and millimeters
(except for the fields RECEIVER_LOCATION and SOURCE_LOCATION which are expressed in meters).
The argument acquisition_system="MUSC" can be used to convert all time units to seconds
and all distance units to meters.
from readdat.read import read
stream = read('filename_from_MUSC.sg2', format="SEG2", acquisition_system="MUSC")

or equivalently
stream = read('filename_from_MUSC.sg2', format="AUTO", acquisition_system="AUTO")

The stream can be modified as saved as SEGYMUSC or SUMUSC formats
that are regular segy and su with nanoseconds / tenth of millimeters convention
(the reason is that segy and su headers only support integers).
NB : to load/modify/export SEG2 files, I recommand working with acquisition_system=None
so that the attribute units remain exactly as the original (see (i))
from readdat.write import write
write(stream, filename='filename_from_MUSC.segy', format="SEGY", acquisition_system="MUSC", endian="big")

which can be read with Seismic Unix (not installed with this package) as
segyread tape=filename_from_MUSC.segy endian=0 | suxwigb

(iii) SEG2ONDULYS
For seg2 files from Ondulys, use acquisition_system="ONDULYS" in readdat.read.read.


This option will collect the receiver coordinates (text entry from seg2 trace attributes)
and pack them in the receiver coordinates fields trace.stats.receiver_{x,y,z} in meters.
the source coordinates fields trace.stats.source_{x-z} are nan.


The year is corrected to 4 digits (instead of 2) to avoid obspy issues.


See also option --ondulys in read_seg2.py and show_seg2.py
Example in linux terminal (or wsl for windows), you may look at the file content using :
read_seg2.py --ondulys seg2file_ondulys.sg2 | grep 'receiver_x\|starttime'


(iv) SEG2CODA
For seg2 files from CODA, use acquisition_system="CODA" in readdat.read.read.
Changes implied by the mode acquisition_system="CODA"


Acquisition times
the field trace.stats.seg2['NOTE'] is used
to determine the acquisition time of the trace in trace.stats.starttime instead of the field trace.stats.seg2['ACQUISITION_DATE'] and trace.stats.seg2['ACQUISITION_TIME'].


Coordinates
The available source/receiver coordinates are stored in
trace.stats.seg2.receiver_{x,y,z} and trace.stats.seg2.source_{x,y,z}


Temperature / Relative humidity
when available, these values are loaded from the trace NOTES and stored into
trace.stats.seg2.temperature_degc
trace.stats.seg2.relative_humidity_percent


See also the script plot_temperature_humidity_from_seg2coda.py /path/to/files*.sg2
to display temperature humidity curves.


(v) SEG2CDZ
Obspy may fail at loading the files because the time string is not formatted as usual.
Readdat loads the data using pyseg2 instead.
The starttime is inferred from the file creation time.
2.1.2 SEGY/SU
(i) Standard SEGY/SU files
Segy/Su files can be read with the default read function from obspy
from readdat.read import read
stream = read('filename.sgy', format="SEGY", acquisition_system=None)

(ii) SEGYMUSC/SUMUSC
To reload MUSC files converted from seg2musc (millimeters & milliseconds convention, see above) to segymusc or sumusc (nanoseconds & tenth of millimeters),
use simply
from readdat.read import read
stream = read('filename_from_MUSC.segy', format="SEGY", acquisition_system="MUSC")

2.1.3 SEGD
Revision 2.X modified after the original program from C. Satriano (IPGP)
https://github.com/claudiodsf/read_segd.git
Revision 3.0 writen by M.L. 16/11/2022
2.1.4 TUSMUS
File Format from O. Durand, Univ Gustav Eiffel,
program modfied after Tom Druet, CEA
2.1.5 SIG
read_sig : M.L. after GageScope user's guide for version 3.1
2.1.6 DZT
read DZT binary File: T. Devie 09/2022
2.1.7 MAT
(i) MATQUANTUM
mat files from HBM QuantumX
modified after P.Mora 2023/05/09
from readdat.read import read
stream = read('filename_from_QUANTUM.mat', format="MAT", acquisition_system="QUANTUM")

(ii) ...
2.1.999 MSEED
Readdat will use the default read function from obspy.
2.2 Resistivity data
BINIris
Read resistivity data from Iris .bin bindary format,
Writen by Thibaud Devie, oct. 2023
2.3 Miscelaneous
SPS : Shell Processing Support
writen by M.L. after doc from SEG SPS
revisions 0 and 2.1
Tomographic files from P. Cote
Read and display tomographic files as implemented by P. Cote in PRIAM
Travel time data (.dat)
Command line
show_datfiles.py /path/to/file.dat

Programming
from readdat.tomocote.datfiles import DatFile
import matplotlib.pyplot as plt

dat = DatFile('/path/to/file.dat')
dat.show(plt.gca(), which="time")
plt.show()

Tomographic models (.map)
show_mapfiles.py /path/to/file.map

3. Usage in a python program
from readdat import read

stream = read(filename="./readdat/filesamples/seg2file_musc.sg2",
format="SEG2", acquisition_system="MUSC")
# see some useful information about the stream in stream.stats

# to quickly see the content of the stream, use
from readdat import print_stream
print_stream(stream[:3]) # print the first 3 traces in the stream

# to loop over the traces
for trace in stream:
# print the sampling rate and the first 3 samples
print(trace.stats.sampling_rate, trace.data[:3])

4. Callable scripts
Some additionnal scripts are installed with the package
# Print the content of a seg2 file in terminal
read_seg2.py --ondulys readdat/readdat/filesamples/seg2file_ondulys.sg2
read_seg2.py --musc readdat/readdat/filesamples/seg2file_musc.sg2
read_seg2.py --coda readdat/readdat/filesamples/seg2file_coda.sg2

# Quick display of a seg2 file (wiggle)
show_seg2.py --musc readdat/readdat/filesamples/seg2file_musc.sg2

# Convert seg2musc to sumusc, (suxwigb assumes that seismic unix is installed):
seg2musc_to_segymusc.py readdat/readdat/filesamples/seg2file_musc.sg2 toto.segy big && segyread tape=toto.segy endian=0 | suxwigb
seg2musc_to_sumusc.py readdat/readdat/filesamples/seg2file_musc.sg2 toto.su big && suxwigb < toto.su

# Show segd rev 2.X
read_segd2X.py readdat/readdat/filesamples/segdfile.segd

# Show segd rev 3.0
read_segd30.py readdat/readdat/filesamples/segdfile_rev3_0.segd

5. Unit tests
ReadDat is equipped with a unit test sequence, to run the tests
pytest --verbose .

In case of failing tests, please contact me.