Pega version 7.1.7
I have a modal dialog which is a form that user will fill the form. In this form I have a button which should open another modal dialog over existing one. Pega cannot open two modal dialogs (they are same type) at the same time. Do you have any idea how can manage this requirement?
Nested Modal Dialog isn't supported in Pega, as there are many OOTB in-built features which would be very messy to handle with nested modal dialog and of course it isn't a best UI practice to do so. Though you can have the combination of Overlay/New Window/Modal Dialog in sync to have the best user experience.
You can write custom section, uncheck "Auto-Generated" option in HTML tab of a section rule with a JS file to invoke nested modal dialog/popup.
You always have the option to go for custom code, we don't recommend it unless there is no other alternative. However, if you still want to go for the custom section, I think the below link should help.