Time out exception in Kafka connection when pushing data to KAFKA
We have notice randomly throw time out exception in KAFKA connection while pushing data to KAFKA. There is no specific pattern or scenarios, however there are lots of time out exceptions in production log .
org.apache.kafka.common.errors.TimeoutException: Topic XXXXXXX not present in metadata after 10000 ms.
Accepted Solution
Updated: 4 May 2023 3:30 EDT
OCBC
SG
@KasunJ following property settings updated and uploaded to kafka client properties.
retries=20
retry.backoff.ms=200
we don't see time out exceptions now.
HCL
NL
@KasunJ check the authentication that your KAFKA server is expecting from Pega and also see the reliability of the KAFKA server from the KAFKA control centre.
Updated: 6 Mar 2023 3:53 EST
OCBC
SG
@JBRaghuraamm Is there way to check from Pega side that the message pass correctly along with Kerberos authentication?
we push around 12K records to KAFKA topic however around 2K records are missing from KAFKA topic daily due to timeout exception.
HCL
NL
@KasunJ I think the time out was internal configuration of KAFKA server, Talk to your KAFKA Admin , what was the availability of KAFKA server and at wha frequency is it setup to rebound/ refresh.
Analyse the failed cases from the logs and see if there is any pattern on the data for the failure ones.
Accepted Solution
Updated: 4 May 2023 3:30 EDT
OCBC
SG
@KasunJ following property settings updated and uploaded to kafka client properties.
retries=20
retry.backoff.ms=200
we don't see time out exceptions now.
