GitLocker: The Coding Marketplace

Description:

pyngsi 2.1.10

pyngsi

What is it ?
pyngsi is a Python framework that helps you write a pipeline for your Fiware dataflow.
Writing a NGSI agent that relies on pyngsi avoids all the plumbing so you can focus on writing your own logic to build NGSI entities.
Key Features

NGSI v2 support
Map Python-native data to NGSI entities
Write NGSI entities to Fiware Orion
Handle incoming data through a common interface
Compute statistics
Allow visualization/debugging facilities

Where to get it
The source code is currently hosted on GitHub at :
https://github.com/Orange-OpenSource/pyngsi
Binary installer for the latest released version is available at the Python
package index.
pip install pyngsi

Getting started
Build your first NGSI entity
from pyngsi.ngsi import DataModel

m = DataModel(id="Room1", type="Room")
m.add_url("dataProvider", "https://fiware-orion.readthedocs.io/en/master/user/walkthrough_apiv2/index.html#entity-creation")
m.add("pressure", 720)
m.add("temperature", 23.0)

m.pprint()

The resulting JSON looks like this :
{
"id": "Room1",
"type": "Room",
"dataProvider": {
"value": "https://fiware-orion.readthedocs.io/en/master/user/walkthrough_apiv2/index.html#entity-creation",
"type": "URL"
},
"pressure": {
"value": 720,
"type": "Integer"
},
"temperature": {
"value": 23.0,
"type": "Float"
}
}

Send the NGSI entity to the Orion broker
from pyngsi.sink import SinkOrion

sink = SinkOrion()
sink.write(m.json())

Develop your own NGSI Agent
Let's quickly create a CSV file to store values from our room sensors
echo -e "Room1;23;720\nRoom2;21;711" > room.csv

Let's code a function that converts incoming rows to NGSI entities
def build_entity(row: Row) -> DataModel:
id, temperature, pressure = row.record.split(';')
m = DataModel(id=id, type="Room")
m.add_url("dataProvider", row.provider)
m.add("temperature", float(temperature))
m.add("pressure", int(pressure))
return m

Let's use it in in our new NGSI Agent
from pyngsi.sources.source import Source, Row
from pyngsi.sink import SinkOrion
from pyngsi.agent import NgsiAgent

src = Source.from_file("room.csv")
sink = SinkOrion()
agent = NgsiAgent.create_agent(src, sink, process=build_entity)
agent.run()

This basic example shows how the pyngsi framework is used to build a NGSI Agent.
Here data are stored on the local filesystem.
By changing just one line you could retrieve incoming data from a FTP server or HTTP server.
from pyngsi.sources.source import Source, Row
from pyngsi.sources.server import ServerHttpUpload
from pyngsi.sink import SinkOrion
from pyngsi.agent import NgsiAgent

src = ServerHttpUpload() # line updated !
sink = SinkOrion()
agent = NgsiAgent.create_agent(src, sink, process=build_entity)
agent.run()

The HTTP server is running. Now you can send the file to the endpoint.
curl -F file=@room.csv http://127.0.0.1:8880/upload

JSON and text formats are natively supported.
Many sources and sinks are provided, i.e. SinkStdout to just displays entities, eliminating the need of having an Orion server running.
One could create a custom Source to handle custom data. The MicrosoftExcelSource is given as exemple.
One could extend the framework according to his needs.
Dependencies

loguru
requests
requests-toolbelt
shortuuid
more_itertools
geojson
flask
cherrypy
schedule
openpyxl

License
Apache 2.0
Documentation
The official documentation is hosted at https://pixel-ports.github.io/pyngsi-tutorial.html
Known Issues
SourceMicrosoftExcel may fail to open some odd Excel files due to an openpyxl bug (i.e. sometimes cannot read graphs).
In this case try to remove the offending sheet or prefer working with CSV files.
Background
Work on pyngsi started at Orange in 2019 for the needs of the PIXEL european project.
Funding
pyngsi has been developed as part of the PIXEL project, H2020, funded by the EC under Grant Agreement number 769355.