RUTA script is the best approach to detect a word(s) as entity when the pattern of the text is important and not the surrounding context (like email, phone etc., policy number, zip code)

ML work better where the context of surrounding words determine the entity.

So if the policy number is the only 4 digit text, expected RUTA works better as it can pick any 4 digit text as mark as policy number,

For the context based detection of policy number where we need to pick a word with context like... my policy number  is <policy number> ML will work better, and we can add the training data for the various examples with start and end tags after the word which needs to be detected.

If you decide to go with RUTA ... for 4 digit policy number something like below should work.

PACKAGE uima.ruta.example;

Token{REGEXP("....") -> MARK(EntityType)};   //4 ... for 4 digits