GitLocker: The Coding Marketplace

Description:

redbike 0.13.1

# RedbikeA simple, flexible scheduler for pools of distributed workers.Redbike is a helpful building block with intentionally limited features.It's easy to schedule and distribute work however you like.Redbike is available from the Python Package Index:```bashYou can't use 'macro parameter character #' in math modeYou can't use 'macro parameter character #' in math mode redbike --config=/etc/redbike.conf ...```Redbike needs to know the worker class, the prefix to use for all ofits keys and the location of the time file (see below). Alsoit needs to know how to connect to Redis. By default it looks in `.redbike.conf` but you can specify any file.Connection information is found in a `[redbike-redis]` stanza andmirrors the arguments of py-redis' `StrictRedis` class. The restof the settings are found in a `[redbike]` stanza.Other information in the config file is ignored, so you can usea shared config as long as it has something like this in it:```ini[redbike]prefix: myappworker: somepackage.somemodule:MyWorkerClass('initstring')timefile: /var/log/myapp.redbike.timefile```In Python:```pythonbike = Redbike(MyWorkerClass('initstring'), prefix='myapp', timefile='/var/log/myapp.redbike.timefile')```## Entering Jobs```bashYou can't use 'macro parameter character #' in math modeYou can't use 'macro parameter character #' in math mode redbike dispatch [<WORKER>] [--schedules=<SCHEDULESCSV> [--after=<TIMESTAMP>]]```Dispatch keeps checking the timeline and places jobs into work queueswhen they are due.The dispatcher continually updates the contents of the timefile withthe current timestamp as it runs. When starting dispatch or settingschedules, the time in the timefile is used to determine when and ifthe next run of the job should occur. When no `--after`argument is provided and no timefile is found, the default value isthe current time.A CSV of JOBID,SHEDULE pairs can also be provided. These scheduleswill be set before dispatch begins. ## Consuming Work```bashYou can't use 'macro parameter character #' in math modeYou can't use 'macro parameter character #' in math mode redbike set job1:A CONTINUEredbikesetjob2:BCONTINUE redbike work mymodule:Work("A:A:B")```## BackoffWorkers with `CONTINUE` schedules may sometimes wish to introduce a backoff. The worker's `work()` method mayreturn an interger value of backoff seconds and the jobwill be entered into the timeline to be enqueued for work again after the specified backoff has elapsed.## Outstanding JobsWhen workers consume jobs from the queue, they set a keywith an expiration and unique value to mark the occasion. Whena worker completes normally, it removes that key and reschedules.If that key no longer exists or the value is changed, it is considered timed-out it will not reschedule. This behavior prevents the worker from stepping on schedule changes (typicallymade by `set()` and `unset()`) that may have happened while thejob was being worked.## TimeoutsTimeouts can be controlled by your worker class. Just definea `timeout(self, queue_name)` and return an integerfor timeout in seconds or `None` to use the default timeout.The default default timeout is 10 seconds. You can also overridethis in your `[redbike]` config stanza with a `default-timeout`setting.You should set a timeout that your jobs wont overrun unless itis ok for them to overlap.## Stopping```bashYou can't use 'macro parameter character #' in math modeYou can't use 'macro parameter character #' in math mode redbike statuses [--before=<TIMESTAMP>]JOBID,statusJOBID,status...```Output is in CSV format.Statuses are set throughout the lifecycle of jobs in Redbike.* TML - entered into the timeline* ENQ - entered into a work queue* BAD - failed to schedule due to a bad RRULE* WRK - picked up by a worker* STP - stopped when the worked raise StopWork* DIE - worker raised an unexpected exceptionTo dump the schedules of all the jobs:```bashredbikeschedulesJOBID,scheduleJOBID,schedule...‘‘‘Hereagain,outputisinCSVformat.Finally,tospyonanindividualjob:‘‘‘bash redbike tell <JOBID>```This will output a json representation like this:```json{ "status": "ENQ:1381720578", "next_run": null, "schedule": "CONTINUE"}```## Removing Unwanted JobsOnce a job is no longer relevant and you want to take it out ofrotation:```bash$ redbike unset <JOBID>```The job will complete if it is already running, but will otherwisebe removed and not worked.