Maximum no.of ADM Models instances to be created in an ideal CDH implementation
Hi All,
I would like to know what is the maximum number of ADM Models can be created at run time in a CDH application.
I am trying to build ADM Model by adding one additional context variable along with default 5 context variable. This 6th variable may have lot of variable values which would be making those many ADM model instances created in the system.
This number could be more than 1 lac. Will it be ok to create those many number of ADM Models ?
Thanks in advance.
Pegasystems Inc.
NL
ADM scales horizontally so theoretically this could work. However, > 100.000 models is a lot more than we usually see, and the computational performance, sizing and storage requirements need to be thought through carefully. If each of these 100.000 models has say 100 inputs, you'll be carrying around the statistics for 10.000.000 fields. That is a lot.
But perhaps even more important than computational performance: you need to wonder whether defining models at this level of granularity will give the models enough evidence to learn from. If you make them too specific, they won't be able to generalize and learn.
So the question is really: why would you want to do this? What is your use case? And what volumes/response rates would you expect?
Thanks
Otto
Virtusa Consulting Services Pvt.ltd
US
Thanks for your reply Otto,
The use case that we are trying is to build ADM models to predict approval probability of a Procedure Code (in HealthCare system) to be approved by an insurance company based on that procedure code attributes as predictors.
There could be more than 1 lac Industry standard procedure codes available in current system. We won't see all these models created immediately , but over the time it can scale to that extent.
We are building this solution as a POC and need to confirm about the volume of responses to receive.
Thanks,
Raju
Pegasystems Inc.
US
Why do you want to make it a context variable and not just a predictor? From your description it sounds like you just want it to be a potential predictor. If that is the case, just leave it out as a context variable and add it as a predictor.