Question
Ford Motor Pvt Ltd.
IN
Last activity: 12 May 2021 9:30 EDT
Queries regarding migration of chatbot channel from non-prod environments to prod environment.
- We have created a chatbot which includes iNLP Text Analyzer, Topic Model (Keyword + ML Based) in DEV Environment. We have migrated this application to QA / PROD environment. Now we have trained the ML model with Feedback data in production.
- Now when we are adding some new commands, also training data for these new commands in DEV environment, and planning to migrate it again to QA/PROD. It will overwrite the model we have available in PROD, which has been trained with Feedback data. So we will lose the trained model from PROD.
- So we want to know-
- What are the best practices for training model in production ?
- How to migrate the chatbot rules, so it doesn’t overwrite the trained model from PROD ?
Accepted Solution
Updated: 5 Nov 2020 5:25 EST
Pegasystems Inc.
IN
Hi Vikas, there's a utility on Channel landing page that lets you export and import training data from one system to another - https://community.pega.com/knowledgebase/articles/conversational-channels/84/transferring-training-data-another-email-bot
Building model in production is never encouraged because you need open rulesets and also you need to assess whether you really want the updated model or not for immediate consumption. Recommendation from platform is this -
- Keep collecting feedback in production
- All this feedback is present in Training data tab of channel where you can -
- Correct topic feedback
- Correct entity feedback
- Create new entity, if there is open ruleset
- Add new records
- *Do not* build model in production
- Use 'Export' in production
- Download the generated RAP
- 'Import' in pre-prod or staging env. The channels should be same in both cases.
- Give additional feedback, if needed
- Build model
- Assess the performance vis-a-vis old models
- Now, you can take your built models to production. Either
- Use deployment pipeline to move models to higher env
- (Manual) Or take RAP of individual models in your Text Analyzer and import in production
In short, these are guidelines -
Hi Vikas, there's a utility on Channel landing page that lets you export and import training data from one system to another - https://community.pega.com/knowledgebase/articles/conversational-channels/84/transferring-training-data-another-email-bot
Building model in production is never encouraged because you need open rulesets and also you need to assess whether you really want the updated model or not for immediate consumption. Recommendation from platform is this -
- Keep collecting feedback in production
- All this feedback is present in Training data tab of channel where you can -
- Correct topic feedback
- Correct entity feedback
- Create new entity, if there is open ruleset
- Add new records
- *Do not* build model in production
- Use 'Export' in production
- Download the generated RAP
- 'Import' in pre-prod or staging env. The channels should be same in both cases.
- Give additional feedback, if needed
- Build model
- Assess the performance vis-a-vis old models
- Now, you can take your built models to production. Either
- Use deployment pipeline to move models to higher env
- (Manual) Or take RAP of individual models in your Text Analyzer and import in production
In short, these are guidelines -
- Same channel must be used in all environments
- Collect and provide feedback in production
- Export(of Training data tab) in Production
- Import(of Training data tab) in Staging
- Build model(of Training data tab) in Staging
- Move built models back to production