A Zeebe text classifier worker based on Hugging Face NLP pipeline
Set a virtual python environment for version 3.7.10 and install requirements:
pip install -r requirements.txt Specify a local (after downloading under models folder) or an Hugging Face zero-shot classification model in the .env file
Due to high resource consumption of some models, we decided to make this worker configurable in term of task name and associated model. For example, so it is possible to separate tasks and models with multiple workers for language handling :
- task
text-classification-enand model (default will be downloaded at worker startup from Hugging Face's website) - task
text-classification-frand local modelmodels/camembert-base-xnli - task
text-classification-mland local modelmodels/xlm-roberta-large-xnli
If you have a local/docker-compose Zeebe running locally you can run/debug with:
python index.pypython -m unittestdocker build -t zeebe-text-classification-french-worker -f Dockerfile.fr .You must have a local or a port-forwarded Zeebe gateway for the worker to connect then:
docker run --name zb-txt-class-fr-wkr zeebe-text-classification-french-workerExample BPMN with service task:
<bpmn:serviceTask id="my-text-classification" name="My text classification">
<bpmn:extensionElements>
<zeebe:taskDefinition type="my-env-var-task-name" />
</bpmn:extensionElements>
</bpmn:serviceTask>- the worker is registered for the type of your choice (set as an env var)
- required variables:
sequence- the phrase to classifycandidate_labels- the list of possible classes
- jobs are completed with variables:
labels- the list of classes (highest scores first)scores- the list of scores sorted in descending order