Description:

readcsvturbo 0.0.6

Improved method for reading the first/last/specific line from csv into a DataFrame
Ever deal with multiple huge csv files and and the panads read_csv/skiprows method is slowing you down? You are not alone.
read-csv-turbo is an improved method of reading the first and last lines using unix head and tail commands to get the data you want in a dataframe as fast as possible. I may include Windows support in the future if requested.
Reading a large csv once is “fine” but often I find myself looping through many files and this process is painfully slow which is why StackOverflow suggestions didn’t cut it. There may be a newer/smarter way of approaching this but this method should be as fast as you could get.
At the moment the use case of this is quite limited as it just provides a fast way to read the first, last or n row of a csv into a dataframe
github: https://github.com/donjor/read-csv-turbo

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter