Skip to main content

Discussion

 Kensho Tsuchihashi (KenshoTsuchihashi)
PEGA
Senior Project Delivery Leader
Pegasystems Inc.
JP
KenshoTsuchihashi Member since 2010 205 posts
PEGA
Posted: Aug 7, 2021
Last activity: May 4, 2023
Posted: 7 Aug 2021 23:36 EDT
Last activity: 4 May 2023 16:40 EDT

Database encryption

Hi,

 

In this post, I am sharing what I know about database encryption. I hope this article will be of help to you in gaining a better understanding. If you find any information inaccurate, please let me know.

 

1. Type of encryption in high level

In general, there are three types of database encryption at high level. 

1-1. Application level encryption

Application level encryption

This approach encrypts data at application level using encryption API, and then stores it in the database. Hence data is stored as encrypted. Encrypted data is returned to SQL and application needs to decrypt it.

1-2. DBMS level encryption

Database level encryption

This approach encrypts data at database level using DBMS functionality. For example, in Oracle there is a TDE (Transparent Data Encryption) option in Oracle Advanced Security to achieve this. With this approach, encryption / decryption occurs internally within the database. Decrypted data is returned to SQL and you do not need to decrypt it at application side.

1-3. Lower layer level (OS, storage) encryption

Lower layer level (OS, storage) encryption

This approach encrypts data using lower level functionality such as operating system, filesystem, or storage.

 

No Approach Threats protected by this approach Pros & Cons
1 Application level encryption
  • Storage device stolen
  • Data file stolen
  • Direct access to database
 Performance issue may arise. Depending on encryption algorithm, data length or data type is changed and performance tuning may get harder. Also key maintenance could be a hurdle.
2 Database level encryption
  • Storage device stolen
  • Data file stolen
 Encryption and decryption occurs at file I/O, so data on OS memory is not encrypted. Also, this option is costly (Oracle TDE is very expensive).
3 Storage level encryption
  • Storage device stolen
 From OS, data is not encrypted. So when OS is hacked, it is not protected. The scope of protection is small compared to other two approaches.

 

* Most of information you find on Pega Community, such as Blob encryption or Property encryption, is Pega level - approach 1. When we discuss encryption with customer, I think it is important to step back a little bit to figure which threats customer wants to protect themselves from in a big picture, before diving into Pega level encryption in the first place. In my experience, customer was investigating Property encryption and I told them that Pega Cloud comes with AWS RDS's 256-bit AES encryption by default, which is approach 3, and they found the solution meets their security requirements. In this case we were able to stop them from unnecessary work.

* Personally I didn't really recommend Pega level encryption until Pega 7.3 because Custom cipher doesn't support key rotation. I used to suggest to customer approach 2 instead, because developer didn't have to worry about key maintenance and it was only a matter of infrastructure team. Now that we have Platform cipher that supports automatic key rotation, we are in a better position.

 

2. Pega level encryption

If customer still wants to go for Pega level encryption, we have two approaches - one for Blob encryption, and the other one for Property encryption. Both approaches require Cipher setup in advance. Prior to Pega 7.3, Custom cipher was the only option but Platform cipher was newly added from 7.3. The difference between these two is, Platform cipher is based on external KMS (Key Management Service) solution, and Custom cipher is based on Java solution. KMS takes care of key rotation automatically, and in that sense Platform cipher is recommended. For Custom cipher, you are responsible to take care of key rotation manually. In the past, I have personally developed a key rotation PoC but I wouldn't have been confident enough to provide that to customer as mistake leads to production data loss, which is a disaster.

Cipher setup overview

 

2-1. Platform cipher vs Custom cipher

No Cipher Feature
1 Platform cipher
  • New cipher introduced from Pega 7.3. Recommended.
  • No need to code your own site-specific cipher.
  • No need to share knowledge about sensitive data in your application with Pega staff, as their assistance is not required to install a cipher.
  • Customer needs to set up KMS externally and provide a CMK (Customer Master Key).
  • No need to handle key rotation manually because KMS supports it (time-based basis and on-demand basis).
  • Currently Pega supports four KMS (Amazon KMS, Microsoft Azure Key Vault, HashiCorp Vault, and Google Cloud KMS).
  • Need to create a Data-Admin-Security-Keystore and enter CMK or other key information (it is different per KMS).
  • Uses the AES256-CBC with PKCS7 Padding cryptographic algorithm.
2 Custom cipher
  • Classic approach. Consider this only when the Platform cipher doesn't suit your customer’s needs. Not recommended.
  • Need to manually take care of key rotation. If you really need to do this, pay special attention not to lose production data.
  • Need to code your own site-specific cipher.
  • You can choose encryption algorithm, from the ones that are supported by JVM.
  • Pega installer provides tools and scripts to assist you build Custom cipher. Here are the steps:
  1. Confirm supported encryption algorithm, key length, and provider by JVM (runPega.bat)
  2. Develop your own custom cipher class (runPega.bat)
  3. Compile and upload your custom cipher class to the database (compileAndLoad.bat)
  4. Activate Custom cipher from Dev Studio

 

2-2. Blob encryption vs Property encryption

After cipher is set up, you are to decide which Pega level encryption you want to build - Blob encryption, or Property encryption. Let's understand the difference. With Blob encryption approach, encryption occurs when an instance is saved to the database, and decryption occurs when an instance is retrieved and opened. This means, when the information is loaded onto memory (clipboard) or shown in XML, data is already decrypted and shown as clear text. Also, this approach works only for Blob, not for exposed column. If you optimize a Blob field, data gets decrypted and stored as clear text. Blob is encrypted only when it sits in the database, so it is hard for developers to verify the encryption. On the other hand, with Property encryption approach, data is encrypted in and out of the database - clipboard, XML, logs, or search indexes. Even if you optimize the Blob field, data is stored as encrypted. In my opinion, Blob encryption requirement is rare in a real world because most of the security requirements are property basis - customer wants specific properties to be encrypted regardless of its exposed or Blob state.

No Encryption Feature
1 Blob encryption
  • Also known as Class level encryption. Enabled by checking "Encrypt BLOB" at class rule form.
  • Encryption occurs when Pega saves an instance to the database, and decryption occurs when Pega retrieves and opens an instance.
  • Only works for Blob. If you optimize a Blob field, data gets decrypted and stored as clear text.
  • Unlike Property encryption, data is shown as clear text on clipboard, logs, and search indexes. It is encrypted only when it sits in the database.
  • UDF fails against encrypted Blob. The error ”The DirectStreamReader UDF cannot read from encrypted BLOBs” is thrown. Personally I use this method to test if Blob encryption is working.
  • You need to set up Cipher in advance. If you enable Blob encryption without cipher, there is no error thrown at runtime. Some people misbelieve it is encrypted but actually no encryption is happening without cipher.
  • "Encrypt BLOB" checkbox is only editable when the class instances do not exist in the database. If exists, you'll need to delete the records first.
2 Property encryption
  • Whether it is in Blob or exposed, it is always encrypted.
  • Whether it is in database, clipboard, logs, or search indexes, it is always encrypted.
  • If you are using TextEncrypted property, and if you haven't set up cipher, error is thrown when data is stored at runtime.
  • There are two approaches - one for TextEncrypted property (deprecated) and the other one for Text + Access Control Policy (recommended).
  • If you want a particular group of people to be able to view original data thru UI, you can set up an Access When rule along with TextEncrypted property. If you take Text + Access Control Policy approach, you will need to create another Access Control Policy Condition rule, but be noted the left-hand side needs to point to the property in the target class.
  • Access Control Policy can be created for Work-, Data-, Assign-, and Index- class.
  • If developer wants to decrypt it in an activity, use @decryptPropertyValue() for TextEncrypted property, and @decryptPW() for Text + Access Control Policy. 
  • If you set up many encrypt properties, performance issues may arise.

 

2-2-1. Blob encryption

In order to enable Blob encryption, go to the class rule and check "Encrypt BLOB" checkbox. Be noted this checkbox is disabled if you already have an instance derived from this class in the database. You'll need to delete the records and try again.

Encrypt BLOB

How do we know if Blob encryption is really working? It is not possible to tell from clipboard or XML, because once you do Obj-Open the instance, data is already decrypted. Personally I use UDF from DBMS as below - UDF is not allowed against encrypted Blob table, and you will face below error:

”The DirectStreamReader UDF cannot read from encrypted BLOBs”

UDF error

 

2-2-2. Property encryption

2-2-1. TextEncrypted property (deprecated)

1. Create a "TextEncrypted" property.

TextEncrypted property

2. TextEncrypted property always require Access When rule.

TextEncrypted with Access When

Access When

 

2-2-2. Text + Access Control Policy (recommended)

1. Create a regular "Text" property.

Text

2. Create an Access Control Policy rule with "PropertyEncrypt" action. Add the property in the list.

Access Control Policy

 

* "Password" property

When we talk about property encryption, "Password" property may also be brought up for discussion. Some people may be confused, but this Password property is not "encryption" but "hashing". Hashing, is a one-way process; after a password has been hashed, it can never be reverted to its original text, while encryption is a two-way process and can be decrypted with a proper key. The Password type requires no advanced configuration or Java skills to set up. This is only used for password purpose, where only the person can validate it by typing it. The display of a value of a Password property is asterisk, for all users, in all situations. Defaulted hashing algorithm used to be MD5 but beginning with Pega 7.2.2, the stronger salted BCrypt became the default. If desired, you can change it from MD5, SHA-1, SHA-256, SHA-512, and BCrypt using Dynamic System Settings.

 

Please see attached How-To documentation for detailed steps:

- Platform cipher

- Custom cipher

- Blob encryption

- Property encryption

  1. TextEncrypted property
  2. Text + Access Control Policy

 

Thanks,

 

***Edited by Moderator: Pooja Gadige to add Developer Knowledge Share tag, add platform capability tag***
To see attachments, please log in.
Pega Platform 8.4.1
Security
Developer Knowledge Share

  • Reply
  • Kensho Tsuchihashi Tanay Bal Henrik Westrell Meera Narayanan Cloe Walker and 9 More

Related content:

We'd prefer it if you saw us at our best.

Pega Collaboration Center has detected you are using a browser which may prevent you from experiencing the site as intended. To improve your experience, please update your browser.

Close Deprecation Notice