GitLocker: The Coding Marketplace

Description:

pyqgiswps 1.8.6

Py-QGIS-WPS

New in 1.8: OGC api processes support
Py-QGIS-WPS is an implementation of the Web Processing Service
standard from the Open Geospatial Consortium based on the QGIS Processing API.
Since 1.8 Py-QGIS-WPS supports OGC API REST processes api
This implementation allows you to expose and run on a server:

QGIS Processing algorithms available on Desktop
QGIS Processing models and scripts
QGIS plugins having a Processing provider according to their metadata.txtfile

It is written in Python and was originally a fork of PyWPS.
Requirements and limitations :

Python 3.7 minimum
Windows not officially supported
Redis server

Documentation
Latest documentation is available on docs.3liz.org
Why Py-QGIS-WPS ?
Py-QGIS-WPS differs from PyWPS in the following:

QGIS centric
Handle all request in asynchronous way: all jobs run in a non-blocking way, even
when synchronous operation is requested.
Use multiprocessing Pool to handle task queue instead instantiating a new process each time.
Uniform Logging with the 'logging' module
Implements OGC processes api.
Use Redis for asynchronous status storage.
Support streamed/chunked requests for stored data
Add extensions to WPS: TIMEOUT and EXPIRE
No Windows support

All these changes where not easy to implement without some drastic changes of the original code and we think
that it deviates too much from the PyWPS original intentions.
That is, we have decided to fork the original project and go along with it.
So, we are really grateful to the original authors of PyWPS for the nice piece of software that helped us very much
to start quickly this project.
Why moving to Tornado instead WSGI

We need to support asyncio: asyncio requires a blocking running loop. This cannot be achieved simply in a WSGI architecture.
Tornado is fully integrated with native python asyncio library and provide a great framework for developing a http server.

Extensions to WPS
TIMEOUT extension
Specify the timeout for a process: if the process takes more than TIMEOUT seconds to run, the worker is then killed and an
error status is returned.
Set the TIMEOUT=<seconds> in GET requests.
In POST requests, set the timeout=<seconds> attribut in the <ResponseDocument> tag.
The server may configure maximum timeout value.
EXPIRE extension
Specify the expiration time for stored results: after EXPIRE seconds after the end of the wps process, all results will be
flushed from disks and local cache. Trying to request the results again will return a 404 HTTP error.
Set the EXPIRE=<seconds> in GET requests.
In POST requests, set the expire=<seconds> attribut int the <ResponseDocument> tag.
The server may configure maximum expiration value.
Status API
Now implemented with the processes api:
The status REST api will return the list of the stored status for all running and terminated wps processes.
Example for returning all stored status:
http://localhost:8080/jobs

Example for returning status for one given job from its id:
http://localhost:8080/jobs/<job_id>

Extensions to processes api:
Files
http://localhost:8080/jobs/<job_id>/files

Running QGIS processing
WPS input/output layer mapping
With QGIS desktop, QGIS processing algorithms usually apply on a QGIS source project and computed layers are displayed in the same context as the source project.
Py-qgis-wps works the same way: a qgis project will be used as a source of input layers.
The difference is that, when an algorithm runs, it creates a qgis project file associated to the current task and register computed layers to it.
The created project may be used as OWS source with Qgis Server. Output layers are returned as complex objects
holding a reference to a WMS/WFS uri that can be used directly with Qgis server. The uri template is configurable
using the server/wms_response_uri configuration setting.
Contextualized input parameters
Tasks parameters are contextualized using the MAP query param. If a MAP parameters is given when
doing a DescripProcess requests, allowed values for input layers will be taken from the qgis source project
according the type of the input layers.
QGIS project (.qgs) files and project stored in Postgres databases are both supported.
The best practice is to always provide a MAP parameters and include the possible input layer in a qgs project. This way you
may connect whatever data source supported by qgis and use them as input data in a safe way.
If you need to pass data to your algorithm from client-side, prefer inputs file parameter and small payloads.
Dependencies
See requirements.txt file.
Installation from python package
ADVICE: You should always install in a python virtualenv. If you want to use system packages, setup your environment
with the --system-site-packages option.
See the official documentation for how to setup a python virtualenv: https://virtualenv.pypa.io/en/stable/.
From source
Install in development mode
make build
pip install -e .

From python package server
pip install py-qgis-wps

Running the server
The server from a command line interface:
The server does not run as a daemon by itself, there are several ways to run a command as a daemon.
For example:

Use Supervisor http://supervisord.org/ will give you full control over logs and server status notifications.
Use the daemon command.
Use Docker

Running the server
Usage
usage: wpsserver [-h] [-d] [-c [PATH]]
[--version] [-p PORT] [-b IP] [-u SETUID]

WPS server

optional arguments:
-h, --help show this help message and exit
-d, --debug Set debug mode
-c [PATH], --config [PATH]
Configuration file
--version Return version number and exit
-p PORT, --port PORT http port
-b IP, --bind IP Interface to bind to
-u SETUID, --setuid SETUID
uid to switch to

Requests to OWS services
The OWS requests use the following format: /ows/?<ows_query_params>
Example:
http://myserver:8080/ows/?SERVICE=WPS&VERSION=1.0.0&REQUEST=GetCapabilities

Configuration
From config ini file
By default, the wps server is not using any config file, but one can be used with the --config option.
A config file is a simple ini file, a sample config file is given with the sources.
From environment variables
The server can be configured with environnement variables:
Configuration is done with environment variables:

QGSWPS_SERVER_WORKDIR: set the current dir processes, all processes will be running in that directory.
QGSWPS_SERVER_HOST_PROXY: When the service is behind a reverse proxy, set this to the proxy entrypoint.
QGSWPS_SERVER_PARALLELPROCESSES: Number of parallel process workers
QGSWPS_SERVER_RESPONSE_TIMEOUT: The max response time before killing a process.
QGSWPS_SERVER_RESPONSE_EXPIRATION: The max time (in seconds) the response from a WPS process will be available.
QGSWPS_SERVER_WMS_SERVICE_URL: The base url for WMS service. Default to /wms. Responses from processing will
be retourned as WMS urls. This configuration variable set the base url for accessing results.
QGSWPS_SERVER_RESULTS_MAP_URI

Logging

QGSWPS_LOGLEVEL: the log level, should be INFO in production mode, DEBUG for debug output.

REDIS storage configuration

QGSWPS_REDIS_HOST: The redis host
QGSWPS_REDIS_PORT: The redis port. Default to 6379
QGSWPS_REDIS_DBNUM: The redis database number used. Default to 0

Qgis project Cache configuration

QGSWPS_CACHE_ROOTDIR: Absolute path to the qgis projects root directory, projects referenced with the MAP parameter will be searched at this location

Processing configuration

QGSWPS_PROCESSSING_PROVIDERS_MODULE_PATH: Path to look for processing algorithms provider to publish, algorithms from providers specified here will be runnable as WPS processes.

Exposing algorithms as WPS services
Note that since 1.1 , the __algorithms__.py method for declaring providers is no longer supported.
Processing providers following the same rules as Qgis regular plugin with a special factory entrypoint: WPSClassFactory(iface) in the __init__.py file.
The metadata.txt file
As regular QGIS plugin, a metadata.txt file must be present with a special entry wps=True indicating that
the plugin is available as a WPS service provider.
Registering providers
The iface parameter is a instance of WPSServerInterface which provide a
registerProvider( provider: QgsAlgorithmProvider, expose: bool = True) -> Any method.
Exposed providers as WPS services must be registered using the registerProvider method
Example:
def WPSClassFactory(iface: WPSServerInterface) -> Any:

from TestAlgorithmProvider1 import AlgorithmProvider1
from TestAlgorithmProvider2 import AlgorithmProvider2

iface.registerProvider( AlgorithmProvider1() )
iface.registerProvider( AlgorithmProvider2() )

Controlling what is exposed:
Processing algorithm with the flag FlagHideFromToolbox set will not be exposed as WPS process.
Parameters with the flag FlagHidden set won't be exposed in a DescribeProcess request
References

OGC standards
OGC Api processes
Introduction to WPS
Py-qgis-server at FOSS4G 2019.