GitLocker: The Coding Marketplace

Description:

discordgateway 0.4.0

Discord gateway
Sans-I/O Python implementation of the Discord gateway.
Sans-I/O means that this implements no I/O (network) and operates purely on the
bytes given using wsproto.
It means that this implementation can be reused for libraries implemented in a
threading fashion or asyncio/trio/curio.
Reference Implementation
For a reference implementation see
wumpy-gateway.
It is designed to handle all sort of network failures and race conditions.
Quickstart
Here's a very minimal implementation using the socket library and threading
for the heartbeating (it does not handle reconnecting or any form of
unexpected disconnections):
import socket
import ssl
import threading
import time
from sys import platform

import certifi
from discord_gateway import DiscordConnection

TOKEN = 'YOUR_VERY.WELL.HIDDEN_TOKEN'
RECV_SIZE = 65536
SERVER_NAME = 'gateway.discord.gg'

def heartbeat(conn, sock):
while True:
sock.send(conn.heartbeat())
time.sleep(conn.heartbeat_interval)

def recv_event(conn, sock):
while True:
for event in conn.events():
return event

for to_send in conn.receive(sock.recv(RECV_SIZE)):
sock.send(to_send)

def main():
# Setup the socket and SSL for the WebSocket Secure connection.
conn = DiscordConnection(SERVER_NAME, encoding='json')
ctx = ssl.create_default_context(cafile=certifi.where())
sock = socket.create_connection(conn.destination)
sock = ctx.wrap_socket(sock, server_hostname=SERVER_NAME)

sock.send(conn.connect()) # Convert to a WebSocket

# Receive the very first HELLO event.
hello = recv_event(conn, sock)

# Send RESUME or IDENTIFY depending on state (will always be False
# when initially connecting, but may be different when reconnecting).
if conn.should_resume:
sock.send(conn.resume(TOKEN))
else:
sock.send(conn.identify(
token=TOKEN,
intents=65535,
properties={
'$os': platform,
'$browser': 'discord-gateway',
'$device': 'discord-gateway'
},
))

heartbeater = threading.Thread(target=heartbeat, args=(conn,sock))
heartbeater.start()

try:
while True:
event = recv_event(conn, sock)
print('Received:', event)
finally:
sock.shutdown(socket.SHUT_WR)
sock.close()

if __name__ == '__main__':
main()

Comprehensive Guide
Connecting
The very first thing required to use the Discord Gateway is to connect using a
TCP socket with TLS enabled.
Start by creating an instance of DiscordConnection, passing in the URI and
encoding to use (most likely 'json'), assign it to a variable called conn.
Now, connect a TCP socket to conn.destination (this is a property that
returns a tuple with the host and port to use - as often used when connecting)
and wrap it with TLS. Using the built-in socket and ssl module with
certifi that looks like this:
SERVER_NAME = 'gateway.discord.gg'

conn = DiscordConnection(SERVER_NAME, encoding='json')
sock = socket.create_connection(conn.destination)

ctx = ssl.create_default_context(cafile=certifi.where())
sock = ctx.wrap_socket(sock, server_hostname=SERVER_NAME)

After the connection is established, generate a HTTP 101 Switching Protocols
request by calling conn.connect() and sending it over the socket.
Bootstrapping and HELLOs
It is now time to communicate over the WebSocket - or almost!
You have yet to receive anything over the socket, not even the response to the
HTTP request made. Create a while-loop that will iterate until a full event
has been received, but it is important to wrap it in a try/except statement
that catches the ConnectionRejected in-case Discord rejects the WebSocket
upgrade.
In each iteration the data from the socket should be passed to
conn.receive(), which then returns a list of bytes that should be sent back
to the socket. The code should roughly look like this:
hello = None
while hello is None:
try:
for to_send in conn.receive(sock.recv(65535)):
sock.send(to_send)
except ConnectionRejected:
print('Discord rejected the connection!')
raise

for hello in conn.events():
# After this has been executed, the hello variable will no longer
# be set to None and the while-loop will exit.
break

When this loop has exited all internal state of discord-gateway should be ready
to IDENTIFY or RESUME depending on conn.should_resume (which is always
falsely during startup - but useful during reconnections).
Heartbeating
The next part is maintaining the connection established, this is done by
launching a concurrent thread/task (depending on how concurrency is handled).
The only purpose of this concurrent heartbeater is to periodically generate
a HEARTBEAT commands and sleep:
def heartbeater(conn: DiscordConnection, sock: socket.socket) -> typing.NoReturn:
# Discord recommends sleeping this random amount of time before the first
# heartbeat, this is to relieve Discord's servers when they are starting
# up again after downtimes.
time.sleep(random.random() * conn.heartbeat_interval)

while True:
sock.send(conn.heartbeat())
time.sleep(conn.heartbeat_interval)

After this, you are now fully connected and can receive events similar to how
you received the HELLO event.
Disconnecting
Expected Disconnections
When disconnecting you need to follow the WebSocket protocol, to start off
generate a closing frame using conn.close() and send it over the socket.
Afterwards, receive data until discord-gateway raises CloseDiscordConnection
and send the data attribute over the socket. Lastly shutdown the write end
of the socket using sock.shutdown(socket.SHUT_WR) and then fully close the
socket. In code that would look like this:
sock.send(conn.close())

try:
while True:
for data in conn.receive(socket.recv(65535)):
sock.send(data)
except CloseDiscordConnection as err:
if err.data is not None:
sock.send(err.data)

# Shutdown the socket now that the WebSocket is closed
sock.shutdown(socket.SHUT_WR)

received = None
while received != b'':
received = sock.recv(65535)

sock.close()

Unexpected Disconnections
Discord-gateway automatically schedules for reconnections in cases such as
un-acknowledged heartbeats and RECONNECT events. That said, discord-gateway
still needs your help to actually reconnect the underlying socket.
Like with expected disconnections discord-gateway raises
CloseDiscordConnection when the closing handshake is being made, just follow
what was done for expected disconnects and then reconnect but do not create a
new DiscordConnection instance. Doing so looses information such as why the
reconnection is happening in the first place. Instead call the reconnect()
method to reset the internal state.
reconnect() returns an integer representing the amount of seconds to wait
before retrying, this will expotentially increase until a successful HELLO
event is received. To take advantage of this, you should structure a retry loop
like this:
while True:
time.sleep(conn.reconnect())

try:
# Connect to the socket as normal, see "Bootstrapping and HELLOs" above
...
except EOFError:
# Note that in some implementations an empty bytes object is returned
# to signal this. It essentially means that the socket closed. Don't
# forget to clean up the socket in all of these except-clauses.
...
continue

except ConnectionRejected as err:
...
continue
else:
# If the bootstrapping succeeded and we received a HELLO event, then
# we can finally exit this loop.
break

Use the should_reconnect() function to determine whether to reconnect, if it
returns False that signals that the connection must not be reconnected.
It is recommended to raise an error or similar in this case.
Race conditions
Because of the concurrent heartbeater there are potential race conditions,
primarily when reconnecting.
The easiest way to fix this is to use a lock that is held when reconnecting,
the heartbeater has to acquire it everytime it tries to send a heartbeat.
Another potential issue is sending a heartbeat while the closing handshake is
in progress - that is, the client has sent one closing frame but not yet
received it from the other end. Discord-gateway provides a simple property
called closed, if this is set just skip sending anything and sleep again.
Following the theme of sockets and threads here is an example of an improved
heartbeater:
lock = ...

def heartbeater(conn, sock):
# Discord recommends sleeping this random amount of time before the first
# heartbeat, this is to relieve Discord's servers when they are starting
# up again after downtimes.
time.sleep(random.random() * conn.heartbeat_interval)

while True:
with lock:
if not conn.closing:
sock.send(conn.heartbeat())
time.sleep(conn.heartbeat_interval)