GitLocker: The Coding Marketplace

Description:

argtyped 0.4.0

argtyped: Command Line Argument Parser, with Types

argtyped is an command line argument parser with that relies on type annotations. It is built on
argparse, the command line argument parser library built into
Python. Compared with argparse, this library gives you:

More concise and intuitive syntax, less boilerplate code.
Type checking and IDE auto-completion for command line arguments.
A drop-in replacement for argparse in most cases.

Since v0.4.0, argtyped also supports parsing arguments defined with an attrs-class. See
Attrs Support for more details.
Installation
Install stable release from PyPI:
pip install argtyped

Or, install the latest commit from GitHub:
pip install git+https://github.com/huzecong/argtyped.git

Usage
With argtyped, you can define command line arguments in a syntax similar to
typing.NamedTuple. The syntax is intuitive and can
be illustrated with an example:
from typing import Optional
from typing_extensions import Literal # or directly import from `typing` in Python 3.8+

from argtyped import Arguments, Switch
from argtyped import Enum, auto

class LoggingLevels(Enum):
Debug = auto()
Info = auto()
Warning = auto()
Error = auto()
Critical = auto()

class MyArguments(Arguments):
model_name: str # required argument of `str` type
hidden_size: int = 512 # `int` argument with default value of 512

activation: Literal["relu", "tanh", "sigmoid"] = "relu" # argument with limited choices
logging_level: LoggingLevels = LoggingLevels.Info # using `Enum` class as choices

use_dropout: Switch = True # switch argument, enable with "--use-dropout" and disable with "--no-use-dropout"
dropout_prob: Optional[float] = 0.5 # optional argument, "--dropout-prob=none" parses into `None`

args = MyArguments()

This is equivalent to the following code with Python built-in argparse:
import argparse
from enum import Enum

class LoggingLevels(Enum):
Debug = "debug"
Info = "info"
Warning = "warning"
Error = "error"
Critical = "critical"

parser = argparse.ArgumentParser()

parser.add_argument("--model-name", type=str, required=True)
parser.add_argument("--hidden-size", type=int, default=512)

parser.add_argument("--activation", choices=["relu", "tanh", "sigmoid"], default="relu")
parser.add_argument("--logging-level", choices=list(LoggingLevels), type=LoggingLevels, default="info")

parser.add_argument("--use-dropout", action="store_true", dest="use_dropout", default=True)
parser.add_argument("--no-use-dropout", action="store_false", dest="use_dropout")
parser.add_argument("--dropout-prob", type=lambda s: None if s.lower() == 'none' else float(s), default=0.5)

args = parser.parse_args()

Save the code into a file named main.py. Suppose the following arguments are provided:
python main.py \
--model-name LSTM \
--activation sigmoid \
--logging-level debug \
--no-use-dropout \
--dropout-prob none

Then the parsed arguments will be equivalent to the following structure returned by argparse:
argparse.Namespace(
model_name="LSTM", hidden_size=512, activation="sigmoid", logging_level="debug",
use_dropout=False, dropout_prob=None)

Arguments can also be pretty-printed:
>>> print(args)
<class '__main__.MyArguments'>
╔═════════════════╤══════════════════════════════════╗
║ Arguments │ Values ║
╠═════════════════╪══════════════════════════════════╣
║ model_name │ 'LSTM' ║
║ hidden_size │ 512 ║
║ activation │ 'sigmoid' ║
║ logging_level │ <MyLoggingLevels.Debug: 'debug'> ║
║ use_dropout │ False ║
║ dropout_prob │ None ║
║ label_smoothing │ 0.1 ║
║ some_true_arg │ True ║
║ some_false_arg │ False ║
╚═════════════════╧══════════════════════════════════╝

It is recommended though to use the args.to_string() method, which gives you control of the table width.
Attrs Support (New)
The way we define the arguments is very similar to defining a dataclass
or an attrs-class, so it seems natural to just write an attrs-class, and add parsing support to it.
To use argtyped with attrs, simply define an attrs-class as usual, and have it subclass AttrsArguments. Here's
the same example above, but implemented with attrs-classes, and with some bells and whistles:
import attr # note: new style `attrs` syntax is also supported
from argtyped import AttrsArguments

def _convert_logging_level(s: str) -> LoggingLevels:
# Custom conversion function that takes the raw string value from the command line.
return LoggingLevels[s.lower()]

@attr.s(auto_attribs=True)
class MyArguments(AttrsArguments):
model_name: str = attr.ib(metadata={"positional": True}) # positional argument
# Or: `model_name: str = argtyped.positional_arg()`.
layer_sizes: List[int] = attr.ib(metadata={"nargs": "+"}) # other metadata are treated as `argparse` options

activation: Literal["relu", "tanh", "sigmoid"] = "relu"
logging_level: LoggingLevels = attr.ib(default=LoggingLevels.Info, converter=_convert_logging_level)

use_dropout: Switch = True
dropout_prob: Optional[float] = 0.5

_activation_fn: Callable[[float], float] = attr.ib(init=False) # `init=False` attributes are not parsed

@dropout_prob.validator # validators still work as you would expect
def _dropout_prob_validator(self, attribute, value):
if not 0.0 <= value <= 1.0:
raise ValueError(f"Invalid probability {value}")

@_activation_fn.default
def _activation_fn(self):
return _ACTIVATION_FNS[self.activation]

A few things to note here:

You can define positional arguments by adding "positional": True as metadata. If this feels unnatural, you could
also use argtyped.positional_arg(), which takes the same arguments as attr.ib.
You can pass additional options to ArgumentParser.add_argument by listing them as metadata as well. Note that
these options take precedence over argtyped's computed arguments, for example, sequence arguments (List[T]) by
default uses nargs="*", but you could override it with metadata.
Attributes with custom converters will not be parsed; its converter will be called with the raw string value from
command line. If the attribute also has a default value, you should make sure that your converter works with both
strings and the default value.
init=False attributes are not treated as arguments, but they can be useful for storing computed values based on
arguments.
The default value logic is the same as normal attrs classes, and thus could be different from non-attrs argtyped
classes. For example, optional arguments are not considered to have an implicit default of None, and no type
validation is performed on default values.

Here are the (current) differences between an attrs-based arguments class (AttrsArguments) versus the normal arguments
class (Arguments):

AttrsArguments supports positional arguments and custom options via metadata.
AttrsArguments handles default values with attrs, so there's no validation of default value types. This also
means that nullable arguments must have an explicit default value of None, otherwise it becomes a required
argument.
AttrsArguments does not support underscore=True.
AttrsArguments does not have to_dict, to_string methods.
AttrsArguments needs to be called with the factory parse_args method to parse, while Arguments parses command
line arguments on construction.

Reference
The argtyped.Arguments Class
The argtyped.Arguments class is main class of the package, from which you should derive your custom class that holds
arguments. Each argument takes the form of a class attribute, with its type annotation and an optional default value.
When an instance of your custom class is initialized, the command line arguments are parsed from sys.argv into values
with your annotated types. You can also provide the list of strings to parse by passing them as the parameter.
The parsed arguments are stored in an object of your custom type. This gives you arguments that can be auto-completed
by the IDE, and type-checked by a static type checker like mypy.
The following example illustrates the keypoints:
class MyArgs(argtyped.Arguments):
# name: type [= default_val]
value: int = 0

args = MyArgs() # equivalent to `parser.parse_args()`
args = MyArgs(["--value", "123"]) # equivalent to `parser.parse_args(["--value", "123"])
assert isinstance(args, MyArgs)

Arguments.to_dict(self)
Convert the set of arguments to a dictionary (OrderedDict).
Arguments.to_string(self, width: Optional[int] = None, max_width: Optional[int] = None)
Represent the arguments as a table.

width: Width of the printed table. Defaults to None, which fits the table to its contents. An exception is raised
when the table cannot be drawn with the given width.
max_width: Maximum width of the printed table. Defaults to None, meaning no limits. Must be None if width is
not None.

argtyped.argument_specs
Return a dictionary mapping argument names to their specifications, represented as the argtyped.ArgumentSpec type.
This is useful for programmatically accessing the list of arguments.
Argument Types
To summarize, whatever works for argparse works here. The following types are supported:

Built-in types such as int, float, str.

Boolean type bool. Accepted values (case-insensitive) for True are: y, yes, true, ok; accepted values
for False are: n, no, false.

Choice types Literal[...]. A choice argument is essentially an str argument with limited
choice of values. The ellipses can be filled with a tuple of strs, or an expression that evaluates to a list of
strs:
from argtyped import Arguments
from typing_extensions import Literal

class MyArgs(Arguments):
foo: Literal["debug", "info", "warning", "error"] # 4 choices

# argv: ["--foo=debug"] => foo="debug"

This is equivalent to the choices keyword in argparse.add_argument.
Note: The choice type was previously named Choices. This is deprecated in favor of the
Literal type introduced in Python 3.8 and back-ported to
3.6 and 3.7 in the typing_extensions library. Choices was removed since version 0.4.0.

Enum types derived from enum.Enum. It is recommended to use argtyped.Enum which uses the instance names as
values:
from argtyped import Enum

class MyEnum(Enum):
Debug = auto() # "debug"
Info = auto() # "info"
Warning = auto() # "warning"

Switch types Switch. Switch arguments are like bool arguments, but they don't take values. Instead, a switch
argument switch requires --switch to enable and --no-switch to disable:
from argtyped import Arguments, Switch

class MyArgs(Arguments):
switch: Switch = True
bool_arg: bool = False

# argv: [] => flag=True, bool_arg=False
# argv: ["--switch", "--bool-arg=false"] => flag=True, bool_arg=False
# argv: ["--no-switch", "--bool-arg=true"] => flag=False, bool_arg=True
# argv: ["--switch=false"] => WRONG
# argv: ["--no-bool-arg"] => WRONG

List types List[T], where T is any supported type except switch types. List arguments allow passing multiple
values on the command line following the argument flag, it is equivalent to setting nargs="*" in argparse.
Although there is no built-in support for other nargs settings such as "+" (one or more) or N (fixed number),
you can add custom validation logic by overriding the __init__ method in your Arguments subclass.

Optional types Optional[T], where T is any supported type except list or switch types. An optional argument
will be filled with None if no value is provided. It could also be explicitly set to None by using none as value
in the command line:
from argtyped import Arguments
from typing import Optional

class MyArgs(Arguments):
opt_arg: Optional[int] # implicitly defaults to `None`

# argv: [] => opt_arg=None
# argv: ["--opt-arg=1"] => opt_arg=1
# argv: ["--opt-arg=none"] => opt_arg=None

Any other type that takes a single str as __init__ parameters. It is also theoretically possible to use a function
that takes an str as input, but it's not recommended as it's not type-safe.

Composing Arguments Classes
You can split your arguments into separate Arguments classes and then compose them together by inheritance. A subclass
will have the union of all arguments in its base classes. If the subclass contains an argument with the same name as an
argument in a base class, then the subclass definition takes precedence. For example:
class BaseArgs(Arguments):
a: int = 1
b: Switch = True

class DerivedArgs(BaseArgs):
b: str

# args = DerivedArgs([]) # bad; `b` is required
args = DerivedArgs(["--b=1234"])

Caveat: For simplicity, we do not completely follow the C3 linearization algorithm that determines the class MRO in Python. Thus, it is a bad idea to have
overridden arguments in cases where there's diamond inheritance.
If you don't understand the above, that's fine. Just note that generally, it's a bad idea to have too complicated
inheritance relationships with overridden arguments.
Argument Naming Styles
By default argtyped uses --kebab-case (with hyphens connecting words), which is the convention for UNIX command line
tools. However, many existing tools use the awkward --snake_case (with underscores connecting words), and sometimes
consistency is preferred over aesthetics. If you want to use underscores, you can do so by setting underscore=True
inside the parentheses where you specify base classes, like this:
class UnderscoreArgs(Arguments, underscore=True):
underscore_arg: int
underscore_switch: Switch = True

args = UnderscoreArgs(["--underscore_arg", "1", "--no_underscore_switch"])

The underscore settings only affect arguments defined in the class scope; (non-overridden) inherited arguments are not
affects. Thus, you can mix-and-match snake_case and kebab-case arguments:
class MyArgs(UnderscoreArgs):
kebab_arg: str

class MyFinalArgs(MyArgs, underscore=True):
new_underscore_arg: float

args = MyArgs(["--underscore_arg", "1", "--kebab-arg", "kebab", "--new_underscore_arg", "1.0"])

Notes

Advanced argparse features such as subparsers, groups, argument lists, and custom actions are not supported.
Using switch arguments may result in name clashes: if a switch argument has name arg, there can be no argument with
the name no_arg.
Optional types:

Optional can be used with Literal:
from argtyped import Arguments
from typing import Literal, Optional

class MyArgs(Arguments):
foo: Optional[Literal["a", "b"]] # valid
bar: Literal["a", "b", "none"] # also works but is less obvious

Optional[str] would parse a value of "none" (case-insensitive) into None.

List types:

List[Optional[T]] is a valid type. For example:
from argtyped import Arguments
from typing import List, Literal, Optional

class MyArgs(Arguments):
foo: List[Optional[Literal["a", "b"]]] = ["a", None, "b"] # valid

# argv: ["--foo", "a", "b", "none", "a", "b"] => foo=["a", "b", None, "a", "b"]

List types cannot be nested inside a list or an optional type. Types such as Optional[List[int]] and
List[List[int]] are not accepted.

Under the Hood
This is what happens under the hood:

When a subclass of argtyped.Arguments is constructed, type annotations and class-level attributes (i.e., the
default values) are collected to form argument declarations.
After verifying the validity of declared arguments, argtyped.ArgumentSpec are created for each argument and stored
within the subclass as the __arguments__ class attribute.
When an instance of the subclass is initialized, if it's the first time, an instance of argparse.ArgumentParser is
created and arguments are registered with the parser. The parser is cached in the subclass as the __parser__
attribute.
The parser's parse_args method is invoked with either sys.argv or strings provided as parameters, returning
parsed arguments.
The parsed arguments are assigned to self (the instance of the Arguments subclass being initialized).

Todo

Support action="append" or action="extend" for List[T] types.

Technically this is not a problem, but there's no elegant way to configure whether this behavior is desired.

Throw (suppressible) warnings on using non-type callables as types.
Support converting an attrs class into Arguments.
Support forward references in type annotations.

MIT License
Copyright (c) 2020 Zecong Hu
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.