Discussion
Pegasystems Inc.
JP
Last activity: 12 Dec 2023 6:11 EST
How to set up Repository
Hi,
I have documented how to set up Repository, which is a recommended approach over database. Please find attached file for the detailed steps. You can also refer to https://collaborate.pega.com/discussion/how-use-repository-api for how to use Repository API.
- Advantages of Repository
The default container for attachment files in Pega is database. Database is fine, but if the volume of files are gigantic, consider repository such as Amazon S3, JFrog Artifactory, Microsoft Azure, or File system because of the following reasons:
- Writing to database is costly. System converts FileStream into Base64 Strings and stores it into Blob field in the database, which increases the size by 33%. Also retrieving by SQL and decoding it to a file back takes time. Repository largely reduces loads on the database.
- If you are using database, when end user tries to attach a file to a case, system temporarily loads the entire file onto a memory, doubly. Say, the attachment file is 1GB, then 133% * 2 = 2.6667GB is consumed out of app node's heap size. On the other hand, repository makes it possible to upload the file directly to Repository, without loading the file onto Pega app node. In my experiment, only about 30KB is used regardless of its file size.
Attachment through Pega |
Attachmet Direct to Repository |
|
* Also, be noded that Repository is not limited to file attachments usage but any file storage requirements. For example, if you are planning to install Deployment Manager on-premise, you will need an individual Repository for storing product files. Database can't be used for this use case.
- Steps overview
Below is the summary of what is needed for each Repository.
No | Type | Required fields | Authentication Profile | Authentication Note |
---|---|---|---|---|
1 | JFrog Artifactory |
|
|
There is no special type of Authentication Profile for JFrog Artifactory. Create "Basic" one and type username and password that you use to log in to JFrog Artifactory cloud. |
2 | Amazon S3 |
|
|
There is a special type of Authentication Profile for AWS. Create "AWS" one and copy Access key ID and Secret access key from IAM > Users screen in AWS management console. |
3 | Microsoft Azure |
|
|
There is a special type of Authentication Profile for Azure. Create "Azure" one and copy Storage account name and access key. Select "HTTPS" for Endpoint protocol. |
4 | File system |
|
- | You can't specify Authentication Profile. You are supposed to set this up in operating system level in advance. |
1. JFrog Artifactory
1. Sign up for JFrog Artifactory Cloud and issue a unique URL for your instance.
2. Create a local repository and name the Repository Key.
3. Create a Repository rule in Pega and enter Host ID and Repository Key.
4. Create a "Basic" Authentication Profile and enter your credentials to log in to JFrog Artifactory Cloud.
2. Amazon S3
1. Sign up for Amazon Cloud.
2. Create a bucket and unblock all public access to the bucket.
3. Create an IAM user and grant "AmazonS3FullAccess" to the user.
4. Create a Repository rule in Pega and enter bucket name.
5. Create a "AWS" Authentication Profile and enter Access key ID and Secret access key.
3. Microsoft Azure
1. Sign up for Microsoft Azure Cloud.
2. Create an Azure Storage account.
3. Create a Container.
4. Create a Repository rule in Pega and enter Container name.
5. Create a "Microsoft Azure" Authentication Profile and enter Endpoint protocol, Azure Storage account name, and Access key.
4. File system
Unlike other type of Repositories above, Authentication Profile can't be used in the Repository rule with file system. Authentication to remote File server must be taken care of in operating system level before Pega configuration. Depending upon the operating system, there are two different protocols used for network file sharing.
4-1. Windows
If Windows is involved in either of Pega server or file server, SMB (CIFS) protocol is used for file sharing. In this case you have two options to specify remote file server in Repository rule.
- UNC approach - specify remote host by back slash (ex. \\<remotehost>\Data).
- Map a network drive approch - specify network drive (ex. "Z:\").
Personally I would suggest to use network drive over UNC. Setting this up in Windows level should make sure two nodes are reachable every time system is rebooted. If there are any network trouble, you can detect it by event logs and system administrators can start troubleshooting sooner.
Steps:
1. In file server, share a folder and grant read & write access to everyone.
2. In Pega server, map a network drive.
3. Create a Repository rule in Pega.
4-2. Linux
If it's Linux to Linux, NFS protocol is used. You need to install nfs-utils for both servers. nfs-utils have both NFS client and NFS server functionalities. You can map your local directory with remote directory by mount command manually, but I would recommend that you use /etc/fstab because this way, mapping remains even if system restarts. In real world, mount has to be done automatically.
Steps:
1. In file server, create a directory and grant read & write access to everyone.
2. In file server, add an entry to /etc/exports file.
3. In Pega server, create a directory to mount local directory to remote directory.
4. In Pega server, add an entry to /etc/fstab file and reboot system.
5. Create a Repository rule in Pega.
Notes:
The latest NFS version is 4.2 and TCP 2049 port is used. The latest SMB version is 3.1.1 and TCP 445 port is used. You may need to configure your firewall to open these ports.
Thanks,