Hi Hemen. I just checked all three of these RBL's on one of our servers, and I'm not seeing any delays. I would try switching your DNS at Settings >> General Settings to Google's 8.8.8.8 / 8.8.4.4.
Afterwards, stop/start the SmarterMail service and then see if the response times improve. I hope this helps.