You can extend work table with one more column for storing the operator list who rejected the case.
Whenever reject action is taken updated the property with a delimiter(like ',' '|'..etc) separated values of operators.
In report definition for displaying records you can use "OperatorList" does not contains "pyUserIdentifier" as filter condition.
Please let us know if you find any other approach to handle it.
in otherway, Work history table will have details of the Rejection notes and the operator who has done rejection
pyMessageKey , Pyperformer columns will have the details.
when an user clicked on an Rejection Flow action, in pre-processing activity of flow action, get the records from history table and compare the performer value with current user details. if it matches then you can Show message like "You already rejected once"