Hi Hemen. I just checked all three of these RBL's on one of our servers, and I'm not seeing any delays. I would try switching your DNS at Settings >> General Settings to Google's 184.108.40.206 / 220.127.116.11.
Afterwards, stop/start the SmarterMail service and then see if the response times improve. I hope this helps.
Technical Support Specialist