Description:

pyxet 0.1.10

Pyxet is a Python library that provides a pythonic interface for
XetHub. Xethub is simple git-based system capable of
storing TBs of ML data and models in a single repository, with block-level
data deduplication that enables hundreds of versions of similar data to be
stored without requiring much storage.
The pyxet library has 3 components:


A fsspec
interface that allows compatible libraries such as Pandas, Polars and Duckdb
to directly access any version of any file in a Xet repository. See below
for some examples.


A command line interface inspired by AWSCLI that allows files to be
uploaded to and downloaded from Xet repository conveniently and efficiently.


A file system mount mechanism that allows any version of any Xet repository
to be mounted. This works on Mac, Linux, and Windows 11 Pro.


For API documentation and full examples, please see here.

License

For personal and professional use. You cannot resell or redistribute these repositories in their original state.

Share

• Share on Facebook
• Share on Twitter