bertnlp 0.0.7

Description:

bertnlp 0.0.7

BERT NLP toolkit
BERT NLP toolkit (https://pypi.org/project/bertnlp) is a Python package that performs various NLP tasks using Bidirectional Encoder Representations from Transformers (BERT) related models.
Installation
To Install this package using pip:
pip install bertnlp-0.0.x-py3-none-any.whl -f https://download.pytorch.org/whl/torch_stable.html

To fetch the nightly version that is in-development, visit this project homepage (https://github.com/daishuanglu/bertnlp).
Implemented NLP Solutions

BERT tokenizer
BERT word embedding and fuzzy matcher
BERT sentence embedding
Modified BERT sentiment score
Text classifier based on KNN-bert and trainer
Text classifier based on FastText and trainer
Multi-labelled text intent detector based on FastText and trainer

Usage
To use this bert package as a SDK,
from bertnlp.fuzzy_matcher import semanticMatcher
from bertnlp.pipeline import sentiment,embeddings,tokenizer
from bertnlp.text_classifier import knnbert as bert_clf

corpus = ['The cat sits outside',
'A man is playing guitar',
'I love pasta',
'The new movie is awesome',
'The cat plays in the garden',
'A woman watches TV',
'The new movie is so great',
'Do you like pizza?','cat','TV']


feature_list=['cat','dog','television','guitar','movie','pizza','pasta']

matcher=semanticMatcher()
sentimentScorer = sentiment(neu_range=0.2)
senti_pred = sentimentScorer.score(corpus)
for j, sent in enumerate(corpus):
features=matcher.match_sent(sent, feature_list, threshold=0.3)
feature_mentioned= ';'.join(['{:s}, score:{:.4f}'.format(f['label'],f['score'] ) for f in features])
print("[Sentence] {:s}; [Sentiment] {:s}, score:{:.4f}; [Feature Mentioned] {:s}".format(
sent,senti_pred[j]['label'],senti_pred[j]['score'],feature_mentioned)
)

print('\n + Extra pipeline features added 12142020:')
senti_pred=sentimentScorer.predict(corpus)
senti_score=sentimentScorer.predict_proba(corpus)
bert_tok=tokenizer()
emb=embeddings()
# sentence bert embeddings: Input - list of sentences, Output - 2D numpy array
sent_emb= emb.sbert_emb(corpus)
print('embedding shape:', sent_emb.shape)
for j, sent in enumerate(corpus):
print(
"[Sentence] {:s}; [Sentiment proba] :{:s}; [1-5th dimensions of sentence embedding] {:s}".format(
sent, str(senti_score[j,:].tolist()), str(sent_emb[j,:5].tolist()) )
)
# bert word embeddings: Input - list of words, Output - 2D numpy array
tokens=bert_tok.token(sent)
word_emb=emb.bert_emb(tokens)
for i,tok in enumerate(tokens):
print("[Token] {}; [1-5th dimension of Word Embeddings] {}".format(tok, word_emb[i,:5]) )

print('Embedding cosine similarity as text relevancy:')
print(emb.cos_sim(sent_emb[:3],sent_emb[-3:]))

To train a text classifier
from bertnlp.text_classifier import knnbert,trainer
import numpy as np
from bertnlp.utils import get_example_data
from bertnlp.measure import plotConfMat


def drop_class(X,y,classname):
sel_id=[i for i,yy in enumerate(y) if yy!=classname]
return [X[i] for i in sel_id],[y[i] for i in sel_id]

data,senti_cat,subcat,senti_label,featureMent=get_example_data('train','ISO-8859-1')
test_data,test_senti_cat,test_subcat,test_senti_label,test_featureMent=get_example_data('test','ISO-8859-1')
sbert_model_name='roberta-base-nli-stsb-mean-tokens'

cat_data_tr,cat_tr=drop_class(data,senti_cat,"Critique")
cat_data_te, cat_te = drop_class(test_data, test_senti_cat, "Critique")
cat_model = knnbert(sbert_model_name=sbert_model_name)

# using evaluation mode to check training-validation performance stats
trainer(cat_data_tr+cat_data_te,cat_tr+cat_te,cat_model,eval_round=10)
# enable save_model_path to generate a deployable model on the overall dataset.
cat_model =trainer(cat_data_tr+cat_data_te,cat_tr+cat_te,cat_model,save_model_path='./model.pkl')
cat_pred=cat_model.predict(cat_data_te)
print('senti_cat sbert model test accuracy {}'.format((cat_pred==np.array(cat_te)).mean() ))
plotConfMat(cat_te,cat_pred,cat_model.classes_,'cat_sbertClf_confmat.png')

The bert package support multi-labelled text intent detection, which can adapted to multiple NLP tasks such as (1) intent detection, (2) multi-intent detection, and other text classification tasks.
Different from text classification, the multi-labelled text intent detection can (1) check 'None' class and (2) classify a single text into multiple labels as many as it detects.
To train a multi-labelled text detection model,
from bertnlp.text_classifier import fasttextClf
import numpy as np
from bertnlp.utils import get_example_data
from bertnlp.measure import plotPrecisionRecall
from bertnlp.fuzzy_matcher import feat_predict_func

# load the example data for training
data,senti_cat,subcat,senti_label,featureMent=get_example_data('train','ISO-8859-1')
test_data,test_senti_cat,test_subcat,test_senti_label,test_featureMent=get_example_data('test','ISO-8859-1')
# save all the Mentioned features as a list of feature names
featurelist=list(set(sum(test_featureMent,[])+ sum(featureMent,[])))


X_tr=[data[i] for i,f in enumerate(featureMent) if 'None' not in ' '.join(f)]
X_te=test_data

# To reduce data imbalance, drop None classes during training.
featMent_ftmodel=fasttextClf()
tr_featMent=[''.join(f) for f in featureMent if 'None' not in ' '.join(f)]
featMent_ftmodel.fit(X_tr,tr_featMent,lr=1.0,epoch=100,wordNgrams=2,loss='ova')
featMent_ftmodel.model.save_model('./featMent_ftClf.bin')

# mixing a edit-distance based text fuzzy matcher with the multi-intent detector to improve simple cases.
combinedModel={'ftmodel':featMent_ftmodel,'featlist':featurelist}

def predict_func(data,model,conf_th):
return feat_predict_func(data,model,conf_th)[0]

feat_pred,_=feat_predict_func(X_te,combinedModel,0.2)
print(feat_pred)
test_featMent=[''.join(f) for f in test_featureMent]
print(test_featureMent)
# using test features to check performance stats: Precision-Recall are used
best_prec_rec,best_thresh=plotPrecisionRecall(X_te,test_featureMent,combinedModel, predict_func, \
conf_thresh_range=np.arange(0,1,0.1),fig_path= './aspect_ftClf_prec_rec.png')

print('The best Precision- Recall is evaluated at confident threshold {}:'.format(best_thresh))
print('Precision: ',best_prec_rec['precision'],'Recall: ',best_prec_rec['recall'])
for c,rate in best_prec_rec['prec_by_class'].items():
print('class',c,': ',rate)