humanframework 0.6.2
Human Framework: Test Automation Framework for Humans™
NOTICE: Human Framework is still in the Proof-of-Concept stage
CONTRIBUTE! I am releasing Human Framework under GPLv3 which makes it open source and free.
Contribute by:
Localization of training data (translation)
Adding more intents and actions
Reporting bugs
Suggesting new features
NOTICE: I am working on a Rasa NLU support
Human Framework is a test automation framework designed for testers without software programming background by allowing them to write test steps or instructions in their own language like English.
Human Framework uses Natural Language Understanding (NLU) for intent classification and entity extraction to be able to perform specific actions.
It currently depend on Microsoft LUIS (Language Understanding).
Below is an example of test instructions that Human Framework can understand.
To get an idea of what specific sentences are supported, check the the content of trials and tests folder.
Open chrome browser https://devpost.com
Page title should be "Devpost - The home for hackathons"
Page should contain link "About"
Click link "About"
Page title should be "About us · Devpost"
Close browser
LUIS (and other NLU tools) makes it possible to write test cases on any human language.
Current version of Human Framework only supports English.
Human Framework was inspired by Robot Framework but leans towards
Natural Language Processing (NLP) for writing test cases.
"Human Framework is the Siri or Alexa of automation testing." (minus the voice, but it is possible in the future)
How it works
The architecture of Human Framework can be simplified into 2 steps- Intent Classification and Action Execution:
Human Framework reads the test case files and then uses an Intent Classifier (e.g. LUIS.ai) to identify intents and entities.
For example, the sentence "open chrome browser" will be identified with the intent "web.open_browser" and entity "chrome" as the browser type.
Human Framework then executes the action specific for the classified intent.
For example, the intent "web.open_browser" will use Selenium to automate opening a chrome browser.
Usage / Installation
Create an Azure account
Go to Azure portal and create an account.
On the Azure portal dashboard, click Create a resource, look for Cognitive Services and click Create
Fill up form and click Create button
Create a LUIS.ai account
Download the Human Framework training data from https://raw.githubusercontent.com/roniemartinez/HumanFramework/master/train/en.json and save it with .json extension
Go to LUIS.ai and create an account
Click Import new app
Click Choose app file (JSON format) ... and select the downloaded Human Framework training data
Optionally, type your desired app name
Click Done
Click on the application you created.
On the application dashboard, click Train to train your LUIS.ai app
Connecting Azure account to LUIS.ai and publishing
Click MANAGE > Azure Resources > Add prediction resource
Fill form
Click Publish, select Production and then click the Publish button
Go back to Azure Resources. If you published your application to Staging environment, use Change query parameters to switch between environments.
Copy the endpoint assigned resource in #2
NOTE: After copying the endpoint, enter it to a browser's address bar to make sure that it is accessible., Otherwise, repeat the steps or use the Starter_Key which will only give you 1000 API calls
Installing Human Framework
Human Framework is written in Python 3. Start by downloading/installing Python from python.org.
To install Human Framework, enter the following command into the terminal/cmd:
Directly from Github
pip install -e git+git@github.com:roniemartinez/HumanFramework.git#egg=humanframework
From PyPI
pip install humanframework
Setting up Human Framework
Use the copied endpoint from LUIS.ai and setup Human Framework. Paste the endpoint enclosed in double quotes.
human config --luis-endpoint "<endpoint>"
Installing Drivers
Human Framework depends on Selenium for testing web applications. Download your desired
webdriver and extract them to your local working directory.
Writing tests
Write a text file (test_*.txt) containing your tests and save it to local working directory where you placed the Selenium Drivers.
For example, if we have a file named test_web.txt:
open chrome https://python.org
page title should be "Welcome to Python.org"
close browser
Running tests
To run the tests in test_web.txt, enter the following on the terminal/cmd:
human --test test_web.txt
Test autodiscovery
Files inside the trials folder that starts with the text test_ are automatically executed.
trials
|- test_simple.txt
|- test_simple_2.txt
|- ...
With this structure, you can simply type human to run the test cases.
The future of Human Framework
Integrated Editor - While testing was greatly simplified because Human Framework will just execute sentences,
using the terminal to install Python and several libraries is still more developer-centric.
To remove this technical requirement, another major goal of the Human Framework project is to release a
one-click installer which should include new features like an integrated IDE/Editor.
This will greatly increase tester's productivity.
Language Support - Not all testers use the English language. Supporting most languages will benefit more testers around the world.
NLU Options - While LUIS.ai is very simple to use and accurate compared to other Intent Classifiers, it is not cheap.
To support more testers, a free option like Rasa NLU can be used as an alternative.
Limitations
I only implemented a few intents and actions for the proof-of-concept stage.
To know what are the possible tests that you can do with Human Framework, check the contents of trials and tests folders.
Author
Ronie Martinez
For personal and professional use. You cannot resell or redistribute these repositories in their original state.
There are no reviews.