Description:

flexds 1.1

Muscle for data science
Data science projects get disorganized quickly. Every test involves a new
script, each script requires a panoply of arguments and produces one or more
data files. Keeping track of all this implied structure is a pain - what is
this script for? What does it depend on? What created this data file? Which
parameters updated this table in the database?
Enter flex - a utility that allows you to express and run all the computational
tasks in a project. Crucially, it captures the specific parameters used for
each task, the data files produced, and any dependencies that task has on other
tasks. All this is captured in files called pipelines (which can even be
connected to one another). Toss in some helpful comments, and you have
executable documentation for your project.
It aims to achieve all this without introducing any overhead. You
won’t have to write more code than you currently are doing, write or maintain
any extra documentation, or use fancy data management solutions. Whatever you
are already doing, flex is compatible with it.

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter