File Listener setup on AWS S3 bucket is failing to start
Hi,
We have set up file listener to poll a AWS S3 bucket as defined in below article. The connectivity test from listener rule was success but not able to start the listener instance. Is there any other configuration we are missing here?
We ensured read/write/delete policies setup for IAM role attached to EC2. PegaRULES logs says as below.
2020-01-09 15:56:28,173 [ FILE-Thread-65] [ STANDARD] [ ] [ ] ( aws.s3.FileS3Adapter) INFO
File.FileListnerName|initialize - s3 meta data doesn't exist for container: dev-S3bucket and directory: dev-S3bucket/work_FileListnerName/
com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: 555F75D2D0F072F0; S3 Extended Request ID:
u4dArFTaXPggBGbYuBvffbM8aTSu7CukhfkoSH9VWw929EhBfrz8D4uhJanNvioSKPr9jUQ2iqg=)
Hi,
We have set up file listener to poll a AWS S3 bucket as defined in below article. The connectivity test from listener rule was success but not able to start the listener instance. Is there any other configuration we are missing here?
We ensured read/write/delete policies setup for IAM role attached to EC2. PegaRULES logs says as below.
2020-01-09 15:56:28,173 [ FILE-Thread-65] [ STANDARD] [ ] [ ] ( aws.s3.FileS3Adapter) INFO
File.FileListnerName|initialize - s3 meta data doesn't exist for container: dev-S3bucket and directory: dev-S3bucket/work_FileListnerName/
com.amazonaws.services.s3.model.AmazonS3Exception: Not Found (Service: Amazon S3; Status Code: 404; Error Code: 404 Not Found; Request ID: 555F75D2D0F072F0; S3 Extended Request ID:
u4dArFTaXPggBGbYuBvffbM8aTSu7CukhfkoSH9VWw929EhBfrz8D4uhJanNvioSKPr9jUQ2iqg=)
Pegasystems Inc.
US
A sanity check: have you actually confirmed that an independent AWS S3 client (either simple java code or an AWS client plugin for eclipse) can create work directory using the same IAM user? If yes, then we need to investigate further within Pega (btw, what is your Pega version?). Otherwise, it is a generic AWS S3 bucket setup issue.
Avani
US
Kevin - thank you for your response. We are on Pega 8.3.1. Creating folder from AWS CLI works fine but haven't tried from any third party S3 client as we have a security requirement to seek S3 object permissions from attached IAM role of EC2.
Also on another note, lets say we have to store files on S3, I understand that standard way to set it up is to configure Repository rule and add authentication profile with secret id/ access key. But again, as we have enterprise security requirement to not rely on programmatic IAM user, is there a way Pega can assume attached IAM role of EC2 to connect to S3?
Pegasystems Inc.
US
Pega uses the access key/secret to make the call, can you turn on debug log
com.pega.platform.physicaldatalayer.internal.cloud.aws.s3.FileS3Adapter
recreating the error and attach the complete log?
Avani
US
Sure, please see the attached log S3-file-listner startup log.txt . Thanks!