How to forbid one to open a case

Scenario:

1. After user A submit a case and the case handler need A provide more info, the case will be assigned back to A and A will receive a mail including a link to the case. user A can open the case by the link. it's normal behavior.

2. If user B get the link by other way, then B can also open the case by the link, which is unexpected.

So, how we can apply access control on the case open? (Note: here is talking opening a case, instead of opening an assignment inside the case, on which we have known how to apply access control).

Thanks a lot