GitLocker: The Coding Marketplace

Description:

csv2ofx 0.30.0

csv2ofx

INTRODUCTION
csv2ofx is a Python library and command line interface program that converts CSV files to OFX and QIF files for importing into GnuCash or similar financial accounting programs. csv2ofx has built in support for importing csv files from mint, yoodlee, and xero.
Requirements
csv2ofx has been tested and is known to work on Python 3.7, 3.8, and 3.9; and PyPy3.7.
INSTALLATION
(You are using a virtualenv, right?)
sudo pip install csv2ofx
Usage
csv2ofx is intended to be used either directly from Python or from the command line.
Library Examples
normal OFX usage
import itertools as it

from meza.io import read_csv, IterStringIO
from csv2ofx import utils
from csv2ofx.ofx import OFX
from csv2ofx.mappings.default import mapping

ofx = OFX(mapping)
records = read_csv('path/to/file.csv', has_header=True)
groups = ofx.gen_groups(records)
trxns = ofx.gen_trxns(groups)
cleaned_trxns = ofx.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([ofx.header(), ofx.gen_body(data), ofx.footer()])

for line in IterStringIO(content):
print(line)

normal QIF usage
import itertools as it

from tabutils.io import read_csv, IterStringIO
from csv2ofx import utils
from csv2ofx.qif import QIF
from csv2ofx.mappings.default import mapping

qif = QIF(mapping)
records = read_csv('path/to/file.csv', has_header=True)
groups = qif.gen_groups(records)
trxns = qif.gen_trxns(groups)
cleaned_trxns = qif.clean_trxns(trxns)
data = utils.gen_data(cleaned_trxns)
content = it.chain([qif.gen_body(data), qif.footer()])

for line in IterStringIO(content):
print(line)

CLI Examples
show help
csv2ofx -h
usage: csv2ofx [options] <source> <dest>

description: csv2ofx converts a csv file to ofx and qif

positional arguments:
source the source csv file (defaults to stdin)
dest the output file (defaults to stdout)

optional arguments:
-h, --help show this help message and exit
-a TYPE, --account TYPE
default account type 'CHECKING' for OFX and 'Bank' for QIF.
-e DATE, --end DATE end date
-l LANGUAGE, --language LANGUAGE
the language
-s DATE, --start DATE
the start date
-y, --dayfirst interpret the first value in ambiguous dates (e.g. 01/05/09) as the day
-m MAPPING, --mapping MAPPING
the account mapping
-x FILE_PATH, --custom FILE_PATH
path to a custom mapping file
-c FIELD_NAME, --collapse FIELD_NAME
field used to combine transactions within a split for double entry statements
-C ROWS, --chunksize ROWS
number of rows to process at a time (default: 2 ** 14)
-r ROWS, --first-row ROWS
number of initial rows to skip (default: 0)
-r ROWS, --last-row ROWS
the final rows to process, negative values count from the end (default: inf)
-O COLS, --first-col COLS
number of initial cols to skip (default: 0)
-L, --list-mappings list the available mappings
-V, --version show version and exit
-q, --qif enables 'QIF' output instead of 'OFX'
-o, --overwrite overwrite destination file if it exists
-D SERVER_DATE, --server-date SERVER_DATE
OFX server date (default: source file mtime)
-E ENCODING, --encoding ENCODING
File encoding (default: utf-8)
-d, --debug display the options and arguments passed to the parser
-v, --verbose verbose output

normal usage
csv2ofx file.csv file.ofx

print output to stdout
csv2ofx ~/Downloads/transactions.csv

read input from stdin
cat file.csv | csv2ofx

qif output
csv2ofx -q file.csv

specify date range from one year ago to yesterday with qif output
csv2ofx -s '-1 year' -e yesterday -q file.csv

use yoodlee settings
csv2ofx -m yoodlee file.csv

CUSTOMIZATION
Code modification
If you would like to import csv files with field names different from the default, you can modify the mapping file or create your own. New mappings must be placed in the csv2ofx/mappings folder (otherwise you must use the ). The mapping object consists of a dictionary whose keys are OFX/QIF attributes and whose values are functions which should return the corresponding value from a record (csv row). The mapping function will take in a record, e.g.,
{'Account': 'savings 2', 'Date': '1/3/15', 'Amount': 5000}

The most basic mapping function just returns a specific field or value, e.g.,
from operator import itemgetter

mapping = {
'bank': 'BetterBank',
'account': itemgetter('Account'),
'date': itemgetter('Date'),
'amount': itemgetter('Amount')}

But more complex parsing is also possible, e.g.,
mapping = {
'account': lambda r: r['Details'].split(':')[0],
'date': lambda r: '%s/%s/%s' % (r['Month'], r['Day'], r['Year']),
'amount': lambda r: r['Amount'] * 2,
'first_row': 1,
'last_row': 10,
'filter': lambda r: r['Amount'] > 10,
}

Required field attributes

attribute
description
default field
example

account
transaction account
Account
BetterBank Checking

date
transaction date
Date
itemgetter('Transaction Date')

amount
transaction amount
Amount
itemgetter('Transaction Amount')

Optional field attributes

attribute
description
default field
default value
example

desc
transaction description
Reference
n/a
shell station

payee
transaction payee
Description
n/a
Shell

notes
transaction notes
Notes
n/a
for gas

check_num
the check or transaction number
Row
n/a
2

id
transaction id
check_num
Num
n/a

bank
the bank name
n/a
account
Bank

account
transaction account type
n/a
checking
savings

account_id
transaction account id
n/a
hash of account
bb_checking

type
transaction type (either debit or credit)
n/a
CREDIT if amount > 0 else DEBIT
debit

balance
account balance
n/a
n/a
$23.00

class
transaction class
n/a
n/a
travel

Optional value attributes

attribute
description
default value
example

has_header
does the csv file have a header row
True

is_split
does the csv file contain split (double entry) transactions
False

currency
the currency ISO code
USD
GBP

delimiter
the csv field delimiter
,
;

date_fmt
custom QIF date output format
%m/%d/%y
%m/%d/%Y

dayfirst
interpret the first value in ambiguous dates (e.g. 01/05/09) as the day (ignored if parse_fmt is present)
False
True

parse_fmt
transaction date parsing format

%m/%d/%Y

first_row
the first row to process (zero based)
0
2

last_row
the last row to process (zero based, negative values count from the end)
inf
-2

first_col
the first column to process (zero based)
0
2

filter
keep transactions for which function returns true

lambda tr: tr['amount'] > 10

Scripts
csv2ofx comes with a built in task manager manage.py.
Setup
pip install -r dev-requirements.txt
Examples
Run python linter and nose tests
manage lint
manage test

Contributing
Please mimic the coding style/conventions used in this repo. If you add new classes or functions, please add the appropriate doc blocks with examples. Also, make sure the python linter and nose tests pass.
Ready to contribute? Here's how:

Fork and clone.

git clone git@github.com:<your_username>/csv2ofx.git
cd csv2ofx

Setup a new virtualenv

mkvirtualenv --no-site-packages csv2ofx
activate csv2ofx
python setup.py develop
pip install -r dev-requirements.txt

Create a branch for local development

git checkout -b name-of-your-bugfix-or-feature

Make your changes, run linter and tests (see above), and submit a pull request through the GitHub website.

Adding Mappings
How to contribute a mapping:

Add the mapping in csv2ofx/mappings/
Add a simple example CSV file in data/test/.
Add the OFX or QIF file that results from the mapping and example CSV file in data/converted/.
Add a csv2ofx call for your mapping to the tests in tests/test.py, in PRE_TESTS. If you added an OFX (not QIF) converted file, pay attention to the -e (end date) and -D (server date) arguments in the test- otherwise tests may pass on your workstation and fail on the build server.
Ensure your test succeeds (see above).

License
csv2ofx is distributed under the MIT License, the same as meza.