Robot Manager is designed to work best with Robot Activities, so that would be the ideal way to work with it and have all the functions remain intact. As for a refactor to eliminate Interaction Activities, you should be able to perform a direct swap. Basically, create a work queue representing each of your activities and then replace the Interaction Activity with the corresponding Robot Activity. Depending on how nested and convoluted your are, this may be easy or may require some work, however in my experience, the use of activities has generally been consistent with places where you might use a Robot Activity (i.e. distinct functionality that you wanted to re-use).

This might require you to have the case handle the various activities and call them in sequence within the case rather than have one bot (or one activity) to fulfill the entire request end-to-end, but the end result is the same. If your process requires the application be in a specific state between your activities that it is not currently handling, then of course that would make your refactor a little more challenging.