GitLocker: The Coding Marketplace

Description:

pytorchicem 0.1.0

PyTorch MPPI Implementation
This repository implements the improved Cross Entropy Method (iCEM)
with approximate dynamics in pytorch, from this paper.
MPPI typically requires actual
trajectory samples, but this paper
showed that it could be done with approximate dynamics (such as with a neural network)
using importance sampling.
Thus it can be used in place of other trajectory optimization methods
such as the Cross Entropy Method (CEM), or random shooting.
Related projects

pytorch CEM - alternative sampling based MPC
pytorch MPPI - alternative sampling based MPC
iCEM - original paper's numpy implementation and experiments code

Installation
pip install pytorch-icem

for running tests, install with
pip install pytorch-icem[test]

for development, clone the repository then install in editable mode
pip install -e .

Usage
See tests/pendulum_approximate_continuous.py for usage with a neural network approximating
the pendulum dynamics. Basic use case is shown below
from pytorch_icem import iCEM

# create controller with chosen parameters
ctrl = icem.iCEM(dynamics, terminal_cost, nx, nu, sigma=sigma,
warmup_iters=10, online_iters=10,
num_samples=N_SAMPLES, num_elites=10, horizon=TIMESTEPS, device=d, )

# assuming you have a gym-like env
obs = env.reset()
for i in range(100):
action = ctrl.command(obs)
obs, reward, done, _, _ = env.step(action.cpu().numpy())

Requirements

pytorch (>= 1.0)
next state <- dynamics(state, action) function (doesn't have to be true dynamics)

state is K x nx, action is K x nu

trajectory cost <- cost(state, action) function for the whole state action trajectory, T is the horizon

cost is K x 1, state is K x T x nx, action is K x T x nu

Features

Parallel/batch pytorch implementation for accelerated sampling