Redis - High availability options
In chat config we are defining master and slave for redis. We are looking at high availability options for chat.
- Let’s say if master is down, will the chat app be running on slave automatically or do we need to manually change the config on chat app?
- Are these master and slave nodes for redis are particularly for performance improvement or they are there for high availability?
- We can't go with redis sentinel option, as redis recommeds using atleast 3 nodes for achieving HA and we only have 2 nodes assigned for redis.
Thanks for your answers!
***Edited by Moderator Marissa to update categories***
Pegasystems Inc.
IN
Hi,
Please check below link. Hope this will help you.
https://collaborate.pega.com/question/redis-automatic-fail-over-pega-chat
Pegasystems Inc.
IN
Hi Anjan,
We use master and slave for better performance. Chat will not be automatically running on slave when master is down. You need to promote slave to master as below.
A Cron Job can be written to ping the Master Redis node continuously. This will not be a drain on Redis as Redis is built for high request-response setups. Whenever this ping fails, this Redis Master should be taken out and one of the slaves should be promoted as the master. Also, there should be a re-pointing of the DNS entry that was pointing to the old Redis master to this new Redis master. All of this will be transparent to the Chat or Co-browse setup.
In Pega Chat Cloud environment, we have code in place which will take care of the DNS remapping. Whenever the Redis goes down, we intentionally crash the server. We suggest that our Node Applications be running with the forever(https://www.npmjs.com/package/forever ) node module. This ensures that if the server crashes the application is restarted. This coupled with our intentional crashing will ensure that the new DNS is resolved, thereby bringing the new Redis master into use.
Thanks,
Srikanth
Accenture
AU
Hi Srikanth,
Referring to the points
Also, there should be a re-pointing of the DNS entry that was pointing to the old Redis master to this new Redis master.
We have code in place which will take care of the DNS remapping.
Our redis config looks like below.
"redis" : {
"master" : {
"host": "master.xx.xx.",
"port": 6379,
"password": "***********"
},
"slave" : {
"host" : "slave.xx.xx",
"port" : 6380,
"password" : "**************"
}
},
Let’s say if master is down in the above case,
- If we have a cron job to promote slave to master then we would be able to achieve HA in redis?
- Do we need to make any changes to the configs of chat app?
- If no changes are needed on chat app side, how would chat app know which redis server is the new master as the defined master in config is down?
Pegasystems Inc.
IN
Hi Anjan,
To elaborate Srikanth's reply : You can have a cron job running which will periodically check the health of the redis master, now suppose the master goes down, the cron job comes to know. At this point of time chat server will crash as redis connection is interrupted and now you need to promote the slave to act as master, but the URL of the slave in your case 'slave.xx.xx' is not changed , so the chat server cannot connect to it. You need to remap "slave.xx.xx" to "master.xx.xx." using some APIs provided by your DNS mapping provider. After the remapping is done the chat server will connect to the new redis master.