Depending upon how the section is included could alter what happens. I believe this should only be used when a work item has been opened in a new flow on a tab, and the close action will close that tab. You might see what you are describing if the section has been added directly to the portal.
If you still need more help, please include some screen shots of your configuration. Tx.