GitLocker: The Coding Marketplace

Description:

djangovitess 0.0.2

django-vitess
Django-vitess repository is a direct copy from
the Django support
files of Vitess.io. This repo exists as a packaging source for
the pypi package. The people of
Vitess promised a long time ago to extract this driver as a separate project and
publish it on pypi, but I decided to do it myself as I needed this package.
If someone from Vitess.io wants to publish this officially, please let me know
and I will release the project from pypi
Usage
The following guide is written by Alkin Tezuysal and published
at Vitess.io blog. I
modified some paths and package names, otherwise it is repost here verbatim.
Django is a popular framework for Python application developers. It includes
packages which make tasks like authorization and content administration easier.
Django supports a number of databases including MySQL which makes it possible to
run a Django application over Vitess without having to change the application
code. Let’s take a look at how to combine the strengths of these two open source
frameworks.
We built this example using Vitess operator. You can see the details of the
implementation in the blog post Vitess Operator for Kubernetes.
Prerequisites

Python environment locally (3.X)
Kubernetes access (minikube, GKE)
Support for Django ORM
via Vitess

For this example, we’re using GKE with an existing Kubernetes cluster. You can
also do this via minikube locally.
The Django example we are using is the
weather app
. We first launch the vitess operator using the
provided configuration
.
The following section includes these steps:

Creating a Vitess Operator pod
Building Vitess Cluster Components (1x primary tablet, 1x replica tablet, 3x
etcd pods, 1x vtgate, 1x vtctld, 1x vitessbackup)
Create the ‘weatherapp’ database schema and users.

$ kubectl apply -f operator.yaml
customresourcedefinition.apiextensions.k8s.io/etcdlockservers.planetscale.com created
customresourcedefinition.apiextensions.k8s.io/vitessbackups.planetscale.com created
customresourcedefinition.apiextensions.k8s.io/vitessbackupstorages.planetscale.com created
customresourcedefinition.apiextensions.k8s.io/vitesscells.planetscale.com created
customresourcedefinition.apiextensions.k8s.io/vitessclusters.planetscale.com created
customresourcedefinition.apiextensions.k8s.io/vitesskeyspaces.planetscale.com created
customresourcedefinition.apiextensions.k8s.io/vitessshards.planetscale.com created
serviceaccount/vitess-operator created
role.rbac.authorization.k8s.io/vitess-operator created
rolebinding.rbac.authorization.k8s.io/vitess-operator created
priorityclass.scheduling.k8s.io/vitess created
priorityclass.scheduling.k8s.io/vitess-operator-control-plane created
deployment.apps/vitess-operator created

$ kubectl get pods
NAME READY STATUS RESTARTS AGE
vitess-operator-7f9c9d58f6-q5zlf 1/1 Running 0 20s

Initialize this cluster with a sample database called ‘weatherapp’ and
user/password to access it will be embedded in the
configuration file
. We are basically creating a database which is analogous to a keyspace in
Vitess.
$ kubectl apply -f 101_initial_cluster.yaml.django
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
example-90089e05-vitessbackupstorage-subcontroller 1/1 Running 0 94s
example-etcd-faf13de3-1 1/1 Running 0 94s
example-etcd-faf13de3-2 1/1 Running 0 94s
example-etcd-faf13de3-3 1/1 Running 0 94s
example-vttablet-zone1-1542279354-edf1c7bf 2/3 Running 1 94s
example-vttablet-zone1-3763665199-476cbd65 2/3 Running 2 94s
example-weatherapp-x-x-vtbackup-init-75efaeeb 0/1 Completed 0 74s
example-zone1-vtctld-1d4dcad0-67bfd56b8b-4dr9s 1/1 Running 2 94s
example-zone1-vtgate-bc6cde92-59b88bc8d8-6wz86 1/1 Running 2 94s
vitess-operator-7f9c9d58f6-q5zlf 1/1 Running 0 4m30s

As you can see this brings up a fully functional managed Vitess cluster with an
unsharded keyspace consisting of one “Primary (Master)” and one “Replica”.
Step 1 - Set portforwards:
$ cat pf.sh ; ./pf.sh &
#!/bin/sh

kubectl port-forward --address localhost "$(kubectl get service --selector="planetscale.com/component=vtctld" -o name | head -n1)" 15000 15999 &
process_id1=$!
kubectl port-forward --address localhost "$(kubectl get service --selector="planetscale.com/component=vtgate,!planetscale.com/cell" -o name | head -n1)" 15306:3306 &
process_id2=$!
sleep 2
echo "You may point your browser to http://localhost:15000, use the following aliases as shortcuts:"
echo 'alias vtctlclient="vtctlclient -server=localhost:15999 -logtostderr"'
echo 'alias mysql="mysql -h 127.0.0.1 -P 15306 -u user"'
echo "Hit Ctrl-C to stop the port forwards"
wait $process_id1
wait $process_id2

Check Tablets:
$ vtctlclient ListAllTablets
Handling connection for 15999
zone1-1542279354 weatherapp - replica 10.100.1.75:15000 10.100.1.75:3306 [] <null>
zone1-3763665199 weatherapp - master 10.100.3.57:15000 10.100.3.57:3306 [] 2020-10-16T09:06:59Z

Step 2 - Verify database:
$ alias mysql="mysql -h 127.0.0.1 -P 15306 -u djangouser -p"
$ mysql
mysql: [Warning] Using a password on the command line interface can be insecure.
Handling connection for 15306
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.7.9-Vitess MySQL Community Server (GPL)

Copyright (c) 2000, 2020, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> show databases;
+------------+
| Databases |
+------------+
| weatherapp |
+------------+
1 row in set (0.16 sec)

Step 3 - Setup application environment
Now that we have set up the Vitess cluster with a MySQL backend, we can proceed
to building the Django application. We will build a Django project using the
django-admin command.
$ mkdir my_weather_app
$ cd my_weather_app
$ python3 -m venv env
$ . env/bin/activate
(env) askdba:my_weather_app askdba$
$ pip install django django-vitess
$ django-admin startproject weatherapp
$ cd weatherapp/
$ ls -la
total 8
drwxr-xr-x 4 askdba staff 128 Oct 16 12:19 .
drwxr-xr-x 4 askdba staff 128 Oct 16 12:18 ..
-rwxr-xr-x 1 askdba staff 666 Oct 16 12:18 manage.py
drwxr-xr-x 7 askdba staff 224 Oct 16 12:18 weatherapp

Edit configuration file and update following section.
$ vi weatherapp/settings.py
import os
# Database
# https://docs.djangoproject.com/en/3.0/ref/settings/#databases

DATABASES = {
'default': {
'ENGINE': 'vitess',
'OPTIONS': {
'read_default_file': '/usr/local/mysql/my.cnf',
},
}
}

STATIC_ROOT = os.path.join(BASE_DIR, 'static')

...
# SECURITY WARNING: don't run with debug turned on in production!
DEBUG = True

ALLOWED_HOSTS = ['127.0.0.1']

# Application definition
...

The contents of mysql configuration:
$ vi /usr/local/mysql/my.cnf [link to sample my.cnf]
[client]
database = weatherapp
user = djangouser
password = ********
port = 15306
host = 127.0.0.1
default-character-set = utf8mb4

Step 4 - Install MySQL Client Connector
$ pip install mysqlclient
Collecting mysqlclient
Using cached mysqlclient-2.0.1.tar.gz (87 kB)
Using legacy 'setup.py install' for mysqlclient, since package 'wheel' is not installed.
Installing collected packages: mysqlclient
Running setup.py install for mysqlclient ... done
Successfully installed mysqlclient-2.0.1

Step 5 - Build Django Framework over Vitess cluster
At this stage, we’re ready to run the migrations to create initial Django
metadata.
$ python manage.py migrate
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying admin.0003_logentry_add_action_flag_choices... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_alter_user_last_name_max_length... OK
Applying auth.0010_alter_group_name_max_length... OK
Applying auth.0011_update_proxy_permissions... OK
Applying auth.0012_alter_user_first_name_max_length... OK
Applying sessions.0001_initial... OK

Step 6 - Create an admin user
Create an administrative user to access the Django Admin interface.
$ python manage.py createsuperuser
Username (leave blank to use 'askdba'): askdba
Email address: alkin@planetscale.com
Password:
Password (again):
The password is too similar to the email address.
This password is too short. It must contain at least 8 characters.
Bypass password validation and create user anyway? [y/N]: y
Superuser created successfully.
(env) askdba:weatherapp askdba$

Step 7 - Start Django daemon.
$ python manage.py runserver 127.0.0.1:8000
Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
October 16, 2020 - 09:37:02
Django version 3.1.2, using settings 'weatherapp.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Step 8 - Go to Django Admin page
Point your browser to http://127.0.0.1:8000/admin
You may continue to build the application from this point by following
this example
.
Conclusion
Vitess is a very powerful sharding framework that comes with a built-in control
plane that allows backend developers to adapt their applications easily.
Combining powerful application frameworks such as Django allows developers to
create scalable applications out of the box with the power of open source
tooling.