@Deven ,

Create a function with below code, call this function after uploading  the file using AttachFile Control(may be in post activity of Flow action where upload control is present).

If function returns true, file is valid PDF format, else not. 

PRThread  thisThread = (PRThread)ThreadContainer.get();
ClipboardPage pgRequestorPage = thisThread.getPublicAPI().getRequestor().getRequestorPage();        
PRFile objFile = pega_rules_utilities.pzGetUploadedFile(pgRequestorPage);
try
{
  org.apache.pdfbox.pdmodel.PDDocument doc=org.apache.pdfbox.pdmodel.PDDocument.load(new PRInputStream(objFile));
}
catch (Throwable e)
{
  return false;  
}
return true;

Hope this helps.

Thanks

Krishna