GitLocker: The Coding Marketplace

Description:

apiquery 0.2.0

api-query
A simple tool to query REST APIs with very little boilerplate. For example, to query and print the
title and URL of new stories on HackerNews:
GET https://hacker-news.firebaseio.com/v0/newstories.json
[
-> STORY_ID

GET https://hacker-news.firebaseio.com/v0/item/{STORY_ID}.json
{ title? -> TITLE, url? -> URL }

! print(f'Story {STORY_ID}: {TITLE} ({URL})')
]

(examples/hn.query)
Installation
pip install api-query
Python 3.10+ is required.
Usage
api-query
[--max-concurrent 1]
[--log-level info]
[--compile-only]
[--http-rate-limit 1]
[--http-retry-count 1]
[--http-base-delay 1.0]
[--http-max-delay 10.0]
query-to-run.query

query-to-run.query should be of the format described in the next section.
--max-concurrent specifies the maximum number of concurrently running statements.
--compile-only causes the generated Python program to only be printed to stdout, instead of being run.
--http-rate-limit is the maximum number of HTTP requests sent per second (on average).
--http-retry-count is the maximum number of times HTTP requests will be retried. The time between
retries starts with the value of --http-base-delay (in seconds), and follows binary exponential
backoff until it hits --http-max-delay (in seconds).
Query Format
Query files consist of four types of statements:
Assignment
A line of the form:
VAR_NAME = value here

assigns "value here" to VAR_NAME. The value is treated as a Python f-string, so you may write, e.g. VAR_NAME_1 = {VAR_NAME}.jpg.
If no value is provided, e.g.
PASSWORD =

then the environmental variable of the same name is used instead ($PASSWORD in this case).
Python Statement
Python statements can be inserted inline by prefixing them with a !. For example,
! print('hello')

Imports are handled automatically, but can also be specified manually with a Python statement.
Shell Command
Shell commands can be inserted inline by prefixing them with a >. For example,
> ls

Output of these commands is not captured (and gets mixed into STDOUT), but the command itself is
treated as a f-string, and thus can use variables:
> ffmpeg -i {URL} -c copy "{TITLE}.mp4"

HTTP Query
An HTTP query must be of the format:
METHOD http://...
- Header1: value (as needed)
...
- Request Body Here (as needed)
Response Handler Here

For example, a simple GET query can be done with:
GET http://example.com
- User-Agent: value
- Another-HTTP-Header: value
-> EXAMPLE_COM_RESPONSE

The URL, as well as the HTTP header values, are converted to f-strings, so you may write, e.g. - Authorization: Bearer {TOKEN}.
Request Body
The request body must come after all headers, before the response handler, and is optional. There are two options for the request body:
String
You can specify the response body as a string. This will be treated as an f-string.
- "response_body_here {VAR_NAME}"

JSON
You can alternatively specify the request body in the following format, which will be encoded and sent as application/json.
- {
field1: "FIELD 1",
field2: VAR_NAME,
field3.subfield[0].id: USER_ID,
field4: [
{ subfield: 3 }
]
}

which will be converted into
{
"field1": "FIELD 1",
"field2": "VALUE OF VAR_NAME HERE",
"field3": {
"subfield": [
{ "id": "VALUE OF USER_ID HERE" }
]
},
"field4": [
{ "subfield": 3 }
]
}

Response Handler
There are two types of ways to handle HTTP responses, similar to the two ways to specify request body.
String
You can save the response body as a string to a variable with:
-> OUTPUT_VAR

JSON
You can also deconstruct a JSON response body to only check certain fields and extract certain values. For example:
{
status: 200,
response: {
userId: USER_ID,
documents[0].id -> FIRST_DOCUMENT_ID
}
}

This checks that the response is of the format:
{
"status": 200,
"response": {
"userId": "VALUE OF USER_ID HERE",
"documents": [
{ "id": ..., ... },
...
],
...
},
...
}

and saves the value of the first document id to FIRST_DOCUMENT_ID.
It is also possible to loop through an array in the response, and do further work, for example:
{
documents: [
{ id -> DOCUMENT_ID }

GET http://.../{DOCUMENT_ID}
-> DOCUMENT_CONTENTS

! print(DOCUMENT_CONTENTS)
]
}

This will iterate through every item of the documents array in the response, perform a GET request, then print out the response.
For object keys, adding a ? between the end of the key and the start of the arrow will allow
the key to not exist, in which case the variable is set to None instead, e.g.
{
user_id? -> USER_ID
}

will set USER_ID = None if user_id is not a key in the response, instead of throwing an error.
Example
For examples, please see the examples directory.
Generated Code
To only view the generated code without running, you can run:
api-query --compile-only file.query

or use:
import api_query

generated_code = '\n'.join(api_query.compile(api_query.parse(api_query.lex(query_source))))

License
This utility is licensed under the MIT License.