GitLocker: The Coding Marketplace

Description:

pipkin 2.1b1

Tool for managing distribution packages for MicroPython and CircuitPython on target devices or in a local directory.
Supports mip- and upip-compatible packages,
and regular pip-compatible packages.
By default it prefers micrpython-lib packages at https://micropython.org/pi/v2/index.json. If the package is not
found there, then it turns to PyPI.
Unlike some of the package managers for MicroPython and CircuitPython, pipkin keeps a minimal set of package
metadata (trimmed .dist-info/METADATA and .dist-info/RECORD) next to the package itself, just like pip.
This enables straigthforward approach for uninstalling, listing and freezing.
When installing with --compile switch, pipkin uses suitable mpy-cross to compile the
py-files on the fly and transfers resulting mpy-files to the target.

Installation
pip install pipkin

Usage
The basic structure of the command line is pipkin <target selection> <command> <command arguments>.
For example:

pipkin --port /dev/ttyACM0 install micropython-logging
pipkin --mount G:\lib install adafruit-circuitpython-ssd1306
pipkin --mount G:\lib install --compile adafruit-circuitpython-ssd1306
pipkin --dir my_project/lib install micropython-logging micropython-oled
pipkin --port COM5 uninstall micropython-logging micropython-oled
pipkin --port COM5 list --outdated

If you have attached a single CircuitPython device (with its filesystem mounted as a disk) or
a single well known MicroPython device (eg. Raspberry Pi Pico), then you can omit the target selection
part:

pipkin install adafruit-circuitpython-ssd1306

pipkin -h
usage: pipkin [-h] [-V] [-v | -q] [-p <port> | -m <path> | -d <path>] {install,uninstall,list,show,freeze,check,download,wheel,cache} ...

Tool for managing MicroPython and CircuitPython packages

general:
-h, --help Show this help message and exit
-V, --version Show program version and exit
-v, --verbose Show more details about the process
-q, --quiet Don't show non-error output

target selection (pick one or let pipkin autodetect the port or mount):
-p <port>, --port <port>
Serial port of the target device
-m <path>, --mount <path>
Mount point (volume, disk, drive) of the target device
-d <path>, --dir <path>
Directory in the local filesystem

commands:
Use "pipkin <command> -h" for usage help of a command

{install,uninstall,list,show,freeze,check,download,wheel,cache}
install Install packages.
uninstall Uninstall packages.
list List installed packages.
show Show information about one or more installed packages.
freeze Output installed packages in requirements format.
check Verify installed packages have compatible dependencies.
download Download packages.
wheel Build Wheel archives for your requirements and dependencies.
cache Inspect and manage pipkin cache.

pipkin install -h
usage: pipkin install [-h] [-r [<file> [<file> ...]]] [-c [<file> [<file> ...]]] [--no-deps] [--pre] [-i <url>] [--extra-index-url [<url> [<url> ...]]]
[--no-index] [--no-mp-org] [-f <url|file|dir>] [-U] [--upgrade-strategy <upgrade_strategy>] [--force-reinstall] [--compile]
[<spec> [<spec> ...]]

Installs upip or pip compatible distribution packages onto a MicroPython/CircuitPython device or into a local directory.

positional arguments:
<spec> Package specification, eg. 'micropython-os' or 'micropython-os>=0.6'

optional arguments:
-h, --help show this help message and exit
-U, --upgrade Upgrade all specified packages to the newest available version. The handling of dependencies depends on the upgrade-strategy used.
--upgrade-strategy <upgrade_strategy>
Determines how dependency upgrading should be handled [default: only-if-needed]. 'eager' - dependencies are upgraded regardless of
whether the currently installed version satisfies the requirements of the upgraded package(s). 'only-if-needed' - are upgraded only when
they do not satisfy the requirements of the upgraded package(s).
--force-reinstall Reinstall all packages even if they are already up-to-date.
--compile Compile and install mpy files.

package selection:
-r [<file> [<file> ...]], --requirement [<file> [<file> ...]]
Install from the given requirements file.
-c [<file> [<file> ...]], --constraint [<file> [<file> ...]]
Constrain versions using the given constraints file.
--no-deps Don't install package dependencies.
--pre Include pre-release and development versions. By default, pipkin only finds stable versions.

index selection:
-i <url>, --index-url <url>
Base URL of the Python Package Index (default https://pypi.org/simple).
--extra-index-url [<url> [<url> ...]]
Extra URLs of package indexes to use in addition to --index-url.
--no-index Ignore package index (only looking at --find-links URLs instead).
--no-mp-org Don't let micropython.org/pi override other indexes.
-f <url|file|dir>, --find-links <url|file|dir>
If a URL or path to an html file, then parse for links to archives such as sdist (.tar.gz) or wheel (.whl) files. If a local path or
file:// URL that's a directory, then look for archives in the directory listing.

Adafruit-Blinka and co
Adafruit-Blinka is a compatibility library which allows
running CircuitPython code with CPython. When publishing CircuitPython libraries at PyPI, Adafruit
and the community have so far targeted only CPython users, because tools for connecting PyPI with bare metal
CircuitPython did not exist (or because at the moment it is not clear how to publish wheels for Pythons
which can’t run pip themselves). Therefore the CircuitPython libraries at PyPI usually have Adafruit-Blinka
dependency, which is not relevant (and would even cause problems) on bare metal CircuitPython devices.
pipkin’s current approach is to have its proxy-index return dummy Adafruit-Blinka distribution, which contains
no modules and has no dependencies. This means when you’re installing a library which depends on Adafruit-Blinka,
you’ll get Blinka’s .dist-info directory with METADATA and RECORD, but nothing else. Let’s call it
an optimized build.
Dummies are returned for all dists, which are currently omitted by
adafruit/circuitpython-build-tools

Current state and goals
Handling packages meant for upip, micropython.org/pi overrides and the problems outlined in the
previous section, all together make pipkin less elegant and slower than one would like. Still, this is just
a start. There are several optimizations possible within current approach. Also, PyPI, pip, wheel
and packaging standards are evolving – in the future it may become easy to publish separate wheels
for MicroPython and/or CircuitPython and pip may become usable for “cross-installing” packages for
other platforms.
Even if clumsy at times, pipkin tries to be the proof-of-concept for demonstrating that even in
the world of MicroPython and CircuitPython, we could continue publishing standard sdists
and wheels on PyPI and re-use the familiar approach for package management. While introducing
new formats and distribution mechanisms have their benefits, we shouldn’t dismiss the standard approach
yet.

Implementation
pipkin delegates most of its work to our old friend pip. This is the reason it is able to offer
so much functionality.
Both upip-compatibility and support for micropython.org-s
index is achieved by using up a temporary local index, which proxies both PyPI (or another specified index)
and micropython.org/pi and restores missing setup.py for upip-compatible packages.
Non-CPython installation target is achieved by creating and maintaining private working environment (venv).
(As creating a venv can be slow in Windows, be prepared for longer wait when using pipkin for the first time.)
In the beginning of the session, pipkin collects package metadata from the target (eg. from the /lib directory
of the device connected over serial) and creates corresponding dummy packages in the working environment.
Then it starts the temporary local index and invokes venv-s pip aginst it. When pip finishes, it detects the
distributions which are removed, added or changed and applies corresponding changes to the target device or
directory.