By default, Botpress gives you a list of 12 languages to work with. These languages were selected based on our experience and the experience of developers using Botpress.
In this blog post, I will go over the language extension of the NLU service for Botpress. You will need to have an on-premises or hosted version of the NLU service.
Requirements
You will need the following:
- Internet connection to load the model and word token
- On-premises version of the NLU service
- Linux
Important files
To add a new language to botpress you will need two files:
- Word embedding (.bin) file
- Model (.model) file
Word embedding
You can select from this list of languages (make sure to NOT USE THE WIKI on that list). Download the bin file. Heads up, this file is BIG, so don't be surprised.
This file needs to follow this format: bp.{Language - 2 Char ISO Code}.{Dimensions}.bin
Model
You can find the model here. The model needs to match the same language you have chosen in the word embedding section.
The bigger the vocab size, the longer the NLU will run (find the right balance for you).
This file needs to follow this format: bp.{Language - 2 Char ISO Code}.bpe.model
Transport and Load the File
Those two files need to be in the same folder. Example:
$ git clone https://github.com/botpress/botpress.git$ cd examples/docker-compose$ mkdir -p botpress/language$ mv ~/Download/BIN_FILE botpress/language$ mv ~/Download/MODEL_FILE botpress/language
Word Embedding
The word embedding needs to be gunzipped (german example) and put in the Botpress format:
$ gunzip cc.de.300.bin.gz$ mv cc.de.300.bin bp.de.300.bin
Model
The model needs to be in the botpress format
$ mv de.wiki.bpe.vs5000.model bp.de.bpe.model
Start the docker-compose or your NLU binary
Modify the docker-compose-standalone-nlu.yaml and modify the command: value under the botpress_lang service
botpress_lang: build: context: ./botpress dockerfile: Dockerfile image: botpress/server command: ./bp lang --offline --dim 300 --langDir /botpress/lang --port 3100 expose: - 3100 ports: - 3100:3100 volumes: - ./botpress/language:/botpress/lang
Pull the image and load the service:
$ docker-compose -f docker-compose-standalone-nlu.yaml pull $ docker-compose -f docker-compose-standalone-nlu.yaml up -d
It's a bit annoying, but you need to be patient. The botpress_lang needs to load the model and can take up to five minutes. So you might need to reload your service for a second-time docker-compose -f docker-compose-standalone-nlu.yaml up -d
In Botpress
In Botpress, you need to modify the nlu.json config file to point on your local NLU
And that's it! Congrats on adding a language to the Botpress NLU!
Share this on:
Build your own personalized AI chatbot for free
Get started building a personalized GPT bot with our intuitive drag & drop interface.
Get started – it's free! 🤖No credit card required
Stay up to date with the latest on AI chatbots