GitLocker: The Coding Marketplace

Description:

pandasaws 0.1.6

Pandas AWS - AWS made easy for data scientists
Pandas AWS makes it super easy to use a pandas.DataFrame along with AWS services.
Working with S3
First create an S3 client to be used later and define a bucket
from pandas_aws import get_client

s3 = get_client('s3')
MY_BUCKET= 'pandas-aws-bucket'

Example 1: get a DataFrame from a parquet file stored in S3
from pandas_aws.s3 import get_df

df_from_parquet_file = get_df(s3, MY_BUCKET, 'my_parquet_file_path', format='parquet')

Example 2: get a DataFrame from multiple CSV files (with same schema) stored in S3
from pandas_aws.s3 import get_df_from_keys

df_from_list = get_df_from_keys(s3, MY_BUCKET, prefix='my-folder', suffix='.csv')

Example 3: put a DataFrame into S3 using an xlsx (Excel) file format
from pandas_aws.s3 import put_df

put_df(s3, my_dataframe, MY_BUCKET, 'target_file_path', format='xlsx')

Example 4: put a DataFrame into S3 using multi parts upload
from pandas_aws.s3 import put_df

put_df(s3, my_dataframe, MY_BUCKET, 'target_file_path', format='csv', compression='gzip', parts=8)

Installing pandas-aws
Pip installation
You can use pip to download the package
pip install pandas-aws
Contributing to pandas-aws
Git clone
We use the develop brand as the release branch, thus git clone the repository and git checkout develop in order to get the latest version in development.
git clone git@github.com:FlorentPajot/pandas-aws.git

Preparing your environment
Pandas AWS uses poetry to manage dependencies. Thus, poetry is required:
curl -SSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python
Create a separate Python environment, for example using pyenv along with pyenv-virtualenv and Python 3.7.7:
pyenv install 3.7.7
pyenv virtualenv 3.7.7 pandas-aws
pyenv activate pandas-aws

Check your environment using:
which python
// should show something like .pyenv/shims/python
python -V
// should show python 3.7.7 (or any other version you selected)
pip list
// should show barely nothing except pip and setuptools

In cas your encounter a problem, check Pyenv documentation.
Then install dependencies with poetry after your git clone from the project repository:
poetry install
Guidelines
Todo
Requires
The project needs the following dependencies:

libpq-dev (psycopg2 dependency)