GitLocker: The Coding Marketplace

Description:

powerfulpipeswatchbus 1.0.11

Powerful Pipes Watcher - Multi-broker watcher tool with the power of UNIX Pipes

Index

Install
Quick Start

WatchBus
SendBus
As a Python library

Supported brokers

Redis examples

Redis Simple Queue Listener
Redis Pub / Sub
Redis Streams

Authors
License
Contributions
Acknowledgements

In a nutshell Powerful Pipes Watcher is a multi broker watcher tool that connect them with UNIX pipes.
Install
> pip install powerful-pipes-watch-bus

Quick Start
WatchBus
Powerful Pipes Watcher read from a broker queue and re-send received data to the UNIX pipeline.
Example
> watch-bus -c redis:// -q my-queue

Above command will connect to the localhost Redis at default port, at database number 1, and will listen for my-queue queue.
Example with multiple queues
> watch-bus -c redis:// -q my-queue -q second-queue

With this command watch-bus will listen from two queues events.
SendBus
Powerful Pipes SendBus read from stdin and re-send received JSON data to the Broker queue(s).
Example
> cat data.json | send-bus -c redis:// -q my-queue

Above command will connect to the localhost Redis at default port, at database number 1, and will listen for my-queue queue.
Example with multiple queues
> cat data.json | send-bus -c redis:// -q my-queue -q second-queue

With this command sendBus will send data to two broker queues.
As a Python library
You also can use Powerful Pipes Watcher as a Python library in your Python code:
from powerful_pipes_watch_bus import connect_bus

def main():
redis_connection = connect_bus("redis+pubsub://localhost:6379/?db=0&channel=my-channel")

for message in redis_connection.read_json_messages():
redis_connection.send_json_message(message, "my-other-queue")

if __name__ == "__main__":
main()

Supported brokers
Currently, supported brokers are:

Redis Pub/Sub
Redis Streams
Redis simple Queue listener

We'll be happy if you want to contribute adding new brokers! :)

Redis examples
Redis Simple Queue Listener
Bus connections for Redis should look like:
redis://[[user]:[password]@][host][:port][/?db=INTEGER][&queue=STRING]]
Default values:

user: Null
password: Null
Host: 127.0.0.1
port: 6379
db: 1
queue: default

Redis Pub / Sub
Bus connections for Redis should look like:
redis+pubsub://[[user]:[password]@][host][:port][/?[db=INTEGER][&channel=STRING]]

channel is the pub/sub channel to listen for.

Default values:

user: Null
password: Null
Host: 127.0.0.1
port: 6379
db: 1
channel: default

Redis Streams
Bus connections for Redis should look like:
redis+streams://[[user]:[password]@][host][:port]/?stream=STRING[&db=INTEGER][&group=STRING][&consumer=STRING][&timeout=INTEGER][&batch_size=INTEGER][&persistent=BOOLEAN]
This mode can works as a consumer or as a producer, depending on the parameters.
Consumer
If you want to use the Redis Streams as a consumer, you should specify the group and consumer parameters.
Producer
If you want to use the Redis Streams as a producer, you should specify ONLY the stream parameter.
Mandatory parameters

stream: The stream name to listen for.

Default values:

user: Null
password: Null
Host: 127.0.0.1
port: 6379
db: 1
consumer: "auto" (will be generated automatically)
persistent: False. If True, the consumer will be created as persistent.

Authors
Powerful Pipes was made by 42Crunch Research Team:

jc42
cr0hn

License
Powerful Pipes is Open Source and available under the AGPLv3+.
Contributions
Contributions are very welcome. See CONTRIBUTING.md or skim existing tickets to see where you could help out.
Acknowledgements
Project logo thanks to Camera control vector created by upklyak.