Pyhorn 0.9.0

Description:

pyhorn 0.9.0

pyhorn
pyhorn is a python client for accessing the RESTful API of the
Opencast Matterhorn
video capture system. It provides a client interface, MHClient, that exposes methods
for accessing both raw and ‘objectified’ response data from the various Matterhorn
REST API endpoints.
This software should be considered Alpha, therefore likely to change/break in the
near future.

Install
pip install pyhorn

Overview
Data available from the REST endpoints is accessible through methods of the MHClient
object. In many cases the response data is encapsulated in additional classes to make
accessing various attributes of the data possible using object notation. For instance,
data about Matterhorn workflows can be accessed using the client.workflows()
method. The return value will be a list of endpoints.workflow.Workflow objects.
The operations attribute of those objects will get you a list of
endpoints.workflow.WorkflowOperation objects. And so forth.
Currently there are only a handfull of endpoints wrapped in this way, and only a
few convenience classes and methods defined for each one. The idea is to continue
adding more as I (or you) need them. Pull requests welcome!

MHClient Method List
In the case of methods that accept a set of keyword arguments, the list of expected
kwargs is mapped directly from the Matterhorn endpoint. In other words, if you
want to know what kwargs to use for MHClient.workflows(), check the corresponding
entry in the REST API docs at http://matterhorn.example.edu/docs.html?path=/workflow.

endpoints() - /info/components.json
me() - /info/me.json
workflows(**kwargs) - /workflow/instances.json
workflow(instance_id) - /workflow/instance/{id}.json
episodes(**kwargs) - /episode/episode.json
episode(episode_id) - /episode/episode.json
user_actions(**kwargs) - /usertracking/actions.json
agents() - /capture-admin/agents.json
agent(agent_name) - /capture-admin/agents/{agent_name}.json
hosts() - /services/hosts.json
job(job_id) - /services/job/{job_id}.json
search_episodes(**kwargs) - /search/episode.json
search_episode(episode_id) - /search/episode.json
statistics() - /services/statistics.json

Example Usage
Create the client interface…
>>> from pyhorn import MHClient
>>> client = MHClient('http://matterhorn.example.edu', 'user', 'passwd')
User/pass combo should be for your REST API user, typically the
“matterhorn_system_account” user. It is possible to create and use a client
object without a user/pass combo (omit the constructor args), but requests
will only work for endpoints that do not require auth (e.g., episode search)
The default request timeout is 5 seconds. Pass timeout=n to the MHClient
constructor to use something else.
Get a list of available endpoints…
>>> client.endpoints()
[{u'description': u'Capture Agent Admin REST Endpoint',
u'docs': u'http://matterhorn.example.edu80/capture-admin/docs',
u'path': u'/capture-admin',
u'type': u'org.opencastproject.capture.admin',
u'version': u'1.4.4',
u'wadl': u'http://matterhorn.example.edu80/capture-admin/?_wadl&_type=xml'},
{u'description': u'EpisodeService REST Endpoint',
u'docs': u'http://matterhorn.example.edu80/episode/docs',
u'path': u'/episode',
u'type': u'org.opencastproject.episode',
u'version': u'1.4.4',
u'wadl': u'http://matterhorn.example.edu80/episode/?_wadl&_type=xml'},
...
Get list of current workflow instances…
>>> wfs = client.workflows()
>>> for wf in wfs:
print wf.id + ": " + wf.state
1646: STOPPED
1649: STOPPED
1651: STOPPED
1655: STOPPED
4211: SUCCEEDED
14479: SUCCEEDED
14486: SUCCEEDED
441: STOPPED
445: STOPPED
...
… or just the successful ones…
>>> wfs = client.workflows(state="SUCCEEDED")
… or the operations for a particular instance…
>>> wf = client.workflow(instance_id=1646)
>>> ops = wf.operations
>>> for op in ops:
print op.id + ": " + op.state
apply-acl: SUCCEEDED
tag: SUCCEEDED
tag: SUCCEEDED
inspect: SUCCEEDED
prepare-av: SUCCEEDED
prepare-av: SUCCEEDED
compose: SUCCEEDED
compose: SUCCEEDED
...
Get the list of currently configured capture agents
>>> cas = client.agents()
>>> for ca in cas:
print ca.name + ": " + ca.state
epiphan001: unknown
epiphan002: unknown
ewave001: idle
ewave002: idle
ncast001: idle
ncast002: shutting_down

Endpoint Object Wrappers
pyhorn attempts to make the Matterhorn API responses more convenient to work with
by wrapping the json response data in a set of classes that provide easy access
via object attributes and automatic “dereferencing” of associated data.
The following endpoint data classes are defined:

Workflow
WorkflowOperation
ServiceJob
Episode
Mediapackage
MediaTrack
CaptureAgent
UserAction
Search

These are just the initial set because they represent the data I needed to deal
with in the other projects that prompted the creation of pyhorn. It is trivial
to add additional wrapper classes. Pull requests welcome!
Attribute access
Endpoint data classes inherit from pyhorn.endpoints.base.EndpointObj. The
json response data is stored in a _raw attribute and made accessible via
dot-notation by overriding __getattr__. A simple illustration:
>>> from pyhorn.endpoints.base import EndpointObj
>>> obj = EndpointObj({"foo": "bar", "baz": [1,2,3]}, client)
>>> obj.foo
bar
>>> obj.baz
[1, 2, 3]
>>> obj.abc
Traceback ...
...
AttributeError: response data for <class 'pyhorn.endpoints.base.EndpointObj'> has no key 'abc'
At this point the dot-notation access only works for top-level values. There is a EndpointObj.raw_get method
that accepts a path_key argument if you need to access something deeper in the response
structure.
>>> obj = EndpointObj({"foo": {"bar": {"baz": 1}}})
>>> obj.raw_get("foo.bar.baz")
1
Dereferencing
In a handful of cases accessing certain attributes (@property, actually)
of an endpoint data wrapper object
will return an instance or instances of a different wrapper class. For example,
Workflow.operations will extract the operation data from the raw json and
return a list of WorkflowOperation objects that wrap the individual operation
data structures contained in the original response.
This works also for dereferencing data that requires an additional request to the
Matterhorn API. For instance, Accessing the WorkflowOperation.job property
triggers a request to the /services/job/{job_id}.json, with the response
being wrapped in a ServiceJob object and returned.
The current list of these dereferencing relationships is:

Workflow.operations -> list of WorkflowOperation objects
Workflow.job -> ServiceJob
Workflow.episode -> Episode
Workflow.mediapackage -> Mediapackage
WorkflowOperation.job -> ServiceJob
ServiceJob.parent -> ServiceJob
ServiceJob.children -> list of ServiceJob objects
Episode.mediapackage -> Mediapackage
Mediapackage.tracks -> list of MediaTrack objects
UserAction.episode -> SearchEpisode

Setting Maintenance Mode
As of v0.4.0 you can toggle the maintenance mode on a host.
>>> hosts = client.hosts()
>>> for host in hosts:
host.set_maintenance(True)
Caching
As of v0.7.0 the use of requests_cache has been dropped in favor of an internal
cache that stores the responses from a subset of the single-item endpoint
methods. A future goal is to allow more granular control over the caching policy,
but for now the following endpoint responses are cached:

CaptureEndpoint.agent
EpisodeEndpoint.episode
SearchEndpoint.episode
Workflow.instance

Caching works via a decorator function on the endpoint methods. The JSON response
data from the Matterhorn API is cached in-memory with each entry assigned a
time-to-live (ttl) value to control expiration. Each cached method also has
a configured max_entries value. If/when the number of entries reaches that
limit a cull operation will prune 1/3 of the existing entries.
To disable caching altogether pass cache_enabled=False to the MHClient
constructor.
To clear the cache call client.clear_cache().

Testing
During development pyhorn tests are executed using pytest.
To run the tests from a local git clone:
pip install -r tests/requirements.txt
then
python setup.py test

License
pyhorn is licensed under the Apache 2.0 license

Copyright
2015 President and Fellows of Harvard College

Release History

0.1.0 (2014-10-23)

Birth!

0.1.1 (2014-10-28)

copyright tweak

0.1.2 (2014-10-28)

blerg. forgot to update history with last version bump.

0.2.0 (2014-11-12)

expanded endpoint methods and wrapper objects
better endpoint class test coverage and fixture/mocking approach

0.3.0 (2015-05-07)
UserAction.episode now fetches from SearchEndpoint

new SearchEndpoint for fetching episode data
new client methods: search_episodes(), search_episode()

0.3.1 (2015-08-04)
bug fix in client http exception handling

0.4.1 (2015-08-27)
Additional services endpoint methods

basic service listing and statistics
methods for getting count of queued/running jobs
maintenance on/off

Added request timeouts
Unhandled http exceptions are now re-raised

0.5.0 (2015-10-30)

added is_live() method to UserAction
workflow is now a property Mediapackage objects

0.6.0 (2016-04-14)

add new includeDeleted param for episode search

0.7.0 (2016-04-14)

ripped out the braindead use of requests_cache
new per-endpoint method caching mechanism

0.8.0 (2016-07-29)

allow client creation and usage without user/pass auth
(note: requests will fail for endpoints that require auth)

0.8.1 (2016-09-20)

SearchEpisode endpoint was missing sort param

0.9.0 (2018-08-29)

Python 3 compatibility!