GitLocker: The Coding Marketplace

Description:

glacierupload 2.1.1

glacier-upload

A helper tool to upload and manage archives in
Amazon S3 Glacier
Vaults. Amazon S3 Glacier is a cloud storage service that is optimized for long
term storage for a relatively cheap price. NOT to be confused with Amazon S3
with Glacier (Instant Retrieval, Flexible Retrieval, and Deep Archive) tier
storage, which uses the S3 API and does not deal with vaults and archives.
Installation
Minimum required Python version is 3.9. To install, run this in your terminal:
$ pip install glacier_upload

Usage
Prerequisites
To upload an archive to Amazon S3 Glacier vault, ensure you have:

Created an AWS account
Created an Amazon S3 Glacier vault from the AWS CLI tool or the Management
Console

Uploading an archive
An upload can be performed by running glacier upload followed by the vault
name and the file name(s) that you want to upload.
glacier upload VAULT_NAME FILE_NAME [FILE_NAME ...]

FILE_NAME can be one or more files or directories.
The script will:

Read the file(s)
Consolidate them into a .tar.xz archive if multiple FILE_NAMEs are
specified or FILE_NAME is one or more directories
Upload the file in one go if the file is less than 100 MB in size, or
Split the file into chunks
Spawn a number of threads that will upload the chunks in parallel. Note that
it will not read the entire file into memory, but only parts of the file as
it processes the chunks.
Return the archive ID when complete. Consider saving this archive ID in a
safe place for retrieval purposes, because Amazon Glacier does not provide a
list of archives in realtime. See the "Requesting an inventory" section below
for details.

There are additional options to customize your upload, such as adding a
description to the archive or configuring the number of threads or the size of
parts. Run glacier upload --help for more information.
If a multipart upload is interrupted in the middle (because of an exception,
interrupted manually, or other reason), the script will show you the upload ID.
That upload ID can be used to resume the upload, using the same command but
adding the --upload-id option, like so:
glacier upload --upload-id UPLOAD_ID VAULT_NAME FILE_NAME [FILE_NAME ...]

Retrieving an archive
Retrieving an archive in glacier requires two steps. First, initiate a
"retrieval job" using:
glacier archive init-retrieval VAULT_NAME ARCHIVE_ID

To see a list of archive IDs in a vault, see "Requesting an inventory" below.
Then, the retrieval job will take some time to complete. Run the next step to
both check whether the job is complete and retrieve the archive if it has been
completed.
glacier archive get VAULT_NAME JOB_ID FILE_NAME

Requesting an inventory
Vaults do not provide realtime access to a list of their contents. To know what
a vault contains, you need to request an inventory of the archive, in a similar
manner to retrieving an archive. To initiate an inventory, run:
glacier inventory init-retrieval VAULT_NAME

Then, the inventory job will take some time to complete. Run the next step to
both check whether the job is complete and retrieve the inventory if it has been
completed.
glacier inventory get VAULT_NAME JOB_ID

Deleting an archive, deleting an upload job, creating/deleting a vault, etc.
All jobs other than uploading an archive and requesting/downloading an inventory
or archive can be done using the AWS CLI. Those functionalities are not
implemented here to avoid duplication of work, and minimize maintenance efforts
of this package.
Contributing
Contributions and/or bug fixes are welcome! Just make sure you've read the below
requirements, then feel free to fork, make a topic branch, make your changes,
and submit a PR.
Development Requirements
Before committing to this repo, install poetry on
your local machine, then run these commands to setup your environment:
poetry install
pre-commit install

All code is formatted with black. Consider
installing an integration for it in your favourite text editor.