appqos 4.6.0

Description:

appqos 4.6.0

Introduction
App QoS is a software created to demonstrate the use of Intel(R) RDT
technologies (CAT, MBA) to improve QoS for applications via partitioning system
resources.
NOTE: This is just a quick start-up guide. For more information about App QoS please see Advanced_AppQoS_Usage_Guide.txt
Installation
App QoS is a part of intel-cmt-cat repo, available at GitHub.
Clone the repository
$ git clone https://github.com/intel/intel-cmt-cat.git

Cloning into 'intel-cmt-cat'...
...

NOTE: For further information about cloning repository from github please see Cloning a repository from GitHub
Install libpqos and virtualenv
Install libpqos
$ cd ./intel-cmt-cat
$ make && sudo make install

Create virtualenv
$ cd ./appqos/
$ make setup
...
Creating a virtualenv for this project...
...
Successfully created virtual environment!
...

All required dependencies are now installed in virtualenv.
Creating evaluation mTLS certificate for testing
The appqos/ca/ directory contains sample script called 'gen_test_certs.sh'. The certificates generated by App QoS use a simple Certificate Authority setup and are not suitable for a production deployment. Only certificates signed by a trusted Certificate Authority should be used in a production deployment.
$ cd appqos/ca
$ ./gen_test_certs.sh

This script will create:

ca certificate
appqos certificate
client certificate

NOTE: Generated keys are not for production use - you should obtain production certificate and keys from your administrator. More information about requirements for App QoS mTLS certificates can be found in Advanced_AppQoS_Usage_Guide.txt
Configuration
App QoS reads its initial configuration from configuration file. During the runtime it accepts commands via REST API.
NOTE: None of configuration changes made via REST API are saved to configuration file.
Configuration file
You can specify a configuration file using --config option. Otherwise App QoS searches for a configuration file is the following order

appqos.conf file from current working directory.
appqos/appqos.conf file in your home directory which isn't root
/opt/intel/appqos/appqos.conf

Create ./appqos.conf file.
cat ./appqos.conf
{
"rdt_iface": {
"interface": "os"
},

"mba_ctrl": {
"enabled": false
},

"apps": [],

"pools": []
}

Basic configuration file with essential configuration only:

REST API Authentication details,
RDT interface and MBA mode configuration

Execution
"make run" command executes App QoS in virtualenv with all required dependencies already installed and appqos.conf configuration file will be used by default.
NOTE: App QoS requires root privileges.
$ sudo make run

WORKON_HOME=../venv/appqos_localhost pipenv run python3 ./appqos
2021-05-07 16:58:37,472 INFO Interface MSR, MBA BW: unsupported.
2021-05-07 16:58:37,499 INFO Interface OS, MBA BW: supported.
2021-05-07 16:58:37,499 INFO Supported RDT interfaces: ['msr', 'os']
2021-05-07 16:58:37,508 INFO RDT initialized with 'os' interface
2021-05-07 16:58:37,665 INFO Supported capabilities:
2021-05-07 16:58:37,665 INFO ['cat', 'mba']
2021-05-07 16:58:37,700 INFO SST-BF not enabled
2021-05-07 16:58:37,700 INFO Power Profiles/EPP not enabled
2021-05-07 16:58:37,701 INFO Configuring RDT
2021-05-07 16:58:37,872 INFO RDT MBA CTRL disabled
2021-05-07 16:58:37,879 INFO Configuration changed, processing new config...

App QoS is up and running ready for REST API commands.
System capabilities detection
During start-up process App QoS performs system capabilities detection and prints out info on the console.
2020-12-14 19:30:52,497 INFO Interface MSR, MBA BW: unsupported.
2020-12-14 19:30:52,527 INFO Interface OS, MBA BW: supported.
2020-12-14 19:30:52,527 INFO Supported RDT interfaces: ['msr', 'os']

System supports RDT MSR and OS interfaces. MBA BW/MBA CTRL is supported only for OS interface.
2020-12-14 19:30:52,539 INFO RDT initialized with 'os' interface
2020-12-14 19:30:52,701 INFO Supported capabilities:
2020-12-14 19:30:52,701 INFO ['cat', 'mba']

RDT is initialized with OS interface and supports both CAT and MBA.
Preparing mTLS key and certificate for REST API client
Before establishing the connection from client side, we need to make sure that private key and the certificates needed for mTLS connection are accessible.
REST client needs App QoS ca.crt certificate, client certificate signed by App QoS ca.crt and a accompanying private key for connecting with App QoS server. If you are using evaluation mTLS certificates, generated by the gen_test_certs.sh scripts, those files can be found in the appqos/ca directory
$ mkdir ~/appqos_client_workspace
$ cp appqos/ca/client_appqos.key appqos/ca/client_appqos.crt appqos/ca/ca.crt ~/appqos_client_workspace
$ cd ~/appqos_client_workspace
$ ls
ca.crt client_appqos.crt client_appqos.key

NOTE: More information about requirements for App QoS mTLS client certificates can be found in Advanced_AppQoS_Usage_Guide.txt
Capabilities queries via REST API
It is also possible to get supported capabilities information via REST API
NOTE: 5000 is default REST API port. Is also assumed that every curl command will be running in directory where ca.crt, client_appqos.crt and client_appqos.key are locally accessible
List supported and current RDT interface
$ curl https://localhost:5000/caps/rdt_iface -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"interface": "os",
"interface_supported": [
"msr",
"os"
]
}


NOTE: OS is current RDT interface, but MSR interface is also supported.
List MBA CTRL status
$ curl https://localhost:5000/caps/mba_ctrl -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"supported": true,
"enabled": false
}

NOTE: MBA CTRL is supported but not enabled
Verify MBA CTRL status
$ mount | grep resctrl

resctrl on /sys/fs/resctrl type resctrl (rw,relatime)

NOTE: resctrl fs is mounted but no "mba_MBps" option is used, MBA CTRL is not enabled.
List RDT capabilities
$ curl https://localhost:5000/caps -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"capabilities": [
"cat",
"mba"
]
}

NOTE: RDT supports CAT and MBA.
Pool operations via REST API
Default pool
If there is no "Default" pool (with "id" equal to 0) defined in config file,
App QoS will dynamically create one on start-up.
All unassigned cores will be assigned to "Default" pool with default RDT configuration,
MBA will be configured to "no throttling" (100% (default) or ~2^32MBps (MBA CTRL enabled))
and CAT CBM to all cache ways.
List all configured pools
$ curl https://localhost:5000/pools -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

[
{
"id": 0,
"mba": 100,
"cbm": 2047,
"name": "Default",
"cores": [
0,
1,
2,
...
45,
46,
47
]
}
]

NOTE: The only configured pool is a "Default" one, with MBA configured to "no throttling" (100%) "mba": 100.
Modify "Default" pool
Modify "Default" pool to exclude multiple cores.
$ curl https://localhost:5000/pools/0 -X PUT --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"cores": [5,6,7,8,9,17,18,19,20,21,22,23,24,26,27,28,29,30,31,32,33,41,42,43,44,45,46,47]}'

{"message": "POOL 0 updated"}

Check new "Default" pool configuration
$ curl https://localhost:5000/pools/0 -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"id": 0,
"mba": 100,
"cbm": 2047,
"name": "Default",
"cores": [
5,
6,
7,
...
45,
46,
47
]
}


NOTE: New cores configuration "cores": [5, 6, 7, ... , 45, 46, 47].
Create new pool
$ curl https://localhost:5000/pools -X POST --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"name": "HP", "cores": [0,1,2,3,4], "mba": 100, "cbm": 2047}'

{"id": 7, "message": "New POOL 7 added"}

NOTE: New pool with "id=7" was created.
Get new pool details
$ curl https://localhost:5000/pools/7 -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"name": "HP",
"cores": [
0,
1,
2,
3,
4
],
"mba": 100,
"cbm": 2047,
"id": 7
}

NOTE: New pool's MBA is configured to 100% "mba": 100.
Modify new pool's MBA
$ curl https://localhost:5000/pools/7 -X PUT --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"mba": 70}'

{"message": "POOL 7 updated"}

Get new pool details
$ curl https://localhost:5000/pools/7 -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"name": "HP",
"cores": [
0,
1,
2,
3,
4
],
"mba": 70,
"cbm": 2047,
"id": 7
}

NOTE: New pool's MBA is configured to 70% "mba": 70.
Modify new pool's cache ways configuration
Set HP pool to use 7 (isolated) LLC Cache Ways
$ curl https://localhost:5000/pools/7 -X PUT --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"cbm": "0x7F0"}'

{"message": "POOL 7 updated"}

Verify the pool configuration
$ curl https://localhost:5000/pools/7 -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"name": "HP",
"cores": [
0,
1,
2,
3,
4
],
"mba": 70,
"cbm": 2032,
"id": 7
}

NOTE: New pool's CBM value is updated "cbm": 2032 (0x7F0)
Enable MBA CTRL
Enable RDT MBA CTRL
$ curl https://localhost:5000/caps/mba_ctrl -X PUT --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"enabled": true}'

{"message": "Please remove all Pools first!"}

NOTE: MBA CTRL state can only be changed when there are no Pools configured (other than "default" Pool #0)
(e.g.: all non-default, configured Pools were removed).
$ curl https://localhost:5000/pools/7 -X DELETE --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{"message": "POOL 7 deleted"}

curl https://localhost:5000/caps/mba_ctrl -X PUT --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"enabled": true}'

{"message": "MBA CTRL status changed."}

NOTE: After MBA CTRL state change, "Default" pool is reset to default values (all cores, full cache and MEM BW access).
Verify MBA CTRL status
$ mount | grep resctrl

resctrl on /sys/fs/resctrl type resctrl (rw,relatime,mba_MBps)

NOTE: resctrl fs mounted with "mba_MBps" option, MBA CTRL is now enabled.
$ curl https://localhost:5000/caps/mba_ctrl -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"supported": true,
"enabled": true
}

$ curl https://localhost:5000/caps/mba -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"mba_enabled": false,
"mba_bw_enabled": true
}

NOTE: MBA CTRL is now enabled. User can use "mba_bw" to configure pool's MBA allocation.
Verify MBA configuration for "Default" Pool
$ curl https://localhost:5000/pools -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

[
{
"id": 0,
"mba_bw": 4294967295,
"cbm": 2047,
"name": "Default",
"cores": [
0,
1,
2,
...
45,
46,
47
]
}
]

NOTE: MBA for Pool #0 set to maximum value 4294967295MBps (~2^32MBps) "mba_bw": 4294967295.
Create new pool and limit MBA to ~5GBps and then ~10GBps
Modify "Default" pool to exclude multiple cores
$ curl https://localhost:5000/pools/0 -X PUT --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"cores": [5,6,7,8,9,17,18,19,20,21,22,23,24,26,27,28,29,30,31,32,33,41,42,43,44,45,46,47]}'

{"message": "POOL 0 updated"}

Create new pool with 5GBps MBA limit
$ curl https://localhost:5000/pools -X POST --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"name": "HP", "cores": [0,1,2,3,4,10,11,12,13,14,15,16,25,34,35,36,37,38,39,40], "mba_bw": 5000, "cbm": 2047}'

{"id": 7, "message": "New POOL 7 added"}

Verify MBA configuration for new pool, COS#7
$ curl https://localhost:5000/pools/7 -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"name": "HP",
"cores": [
0,
1,
2,
...
38,
39,
40
],
"mba_bw": 5000,
"cbm": 2047,
"id": 7
}

cat /sys/fs/resctrl/COS7/schemata

L3:0=7ff;1=7ff
MB:0=5000;1=5000

NOTE: Both App QoS and Resctrl show new MBA configuration (5000MBps) for Pool #7.
Modify new pool MBA to 10GBps
$ curl https://localhost:5000/pools/7 -X PUT --cert client_appqos.crt --key client_appqos.key --cacert ca.crt -H "Content-Type: application/json" -d '{"mba_bw": 10000}'

{"message": "POOL 7 updated"}

Verify new pool MBA configuration
$ curl https://localhost:5000/pools/7 -X GET --cert client_appqos.crt --key client_appqos.key --cacert ca.crt

{
"name": "HP",
"cores": [
0,
1,
2,
...
38,
39,
40
],
"mba_bw": 10000,
"cbm": 2047,
"id": 7
}

$ cat /sys/fs/resctrl/COS7/schemata

L3:0=7ff;1=7ff
MB:0=10000;1=10000

NOTE: Both App QoS and Resctrl show new MBA configuration (10000MBps) for Pool #7.